首页 > AI前沿 > The Limits of Speculation: Bounding Speculative Decoding in Mixture-of-Experts

The Limits of Speculation: Bounding Speculative Decoding in Mixture-of-Experts

arXiv机器学习 2026-08-26 18:40 6 阅读 查看原文

Speculative decoding in Mixture-of-Experts (MoE) models faces the problem of unstable verification cost caused by input-dependent expert loading.

To study the physics of this process, we formulate speculation-budget selection as an offline Stochastic Shortest Path (SSP) problem over reference sequences and build a diagnostic Oracle that uses counterfactual simulation to account for MoE verification cost.

A detailed analysis of the Oracle's decisions on the Qwen3-Coder and EAGLE-3 pairing, in the space of marginal deltas (Delta Space), shows that rejected candidates form a strict linear boundary.

This result demonstrates that a complex global optimization is locally governed by a necessary condition balancing marginal cost against expected progress ($\frac{Δ\mathbb{E}[Cost]}{Δ\mathbb{E}[a]}$), providing a rigorous mathematical reference point for designing future adaptive online heuristics.