Looped language models deepen computation without adding parameters, yet caching full-context key-value (KV) states across layers and rounds severely bounds serving batch size.
Looped CED Design
We design a looped causal encoder-decoder (CED) that achieves recurrent depth with zero round-wise KV growth, addressing where and how to loop an asymmetric stack in which only sparse encoder layers generate global KV.
Where to Loop
We find that recurrence must be placed in the encoder to iteratively refine global KV representations rather than re-reading static decoder projections, which both drives language-modeling gains and preserves long-context scaling.
Stochastic Recurrence and Fixed-Point KV Sharing
Furthermore, training the encoder loop under stochastic recurrence depth induces a fixed-point equilibrium where every round losslessly shares the last round’s KV cache.
Memory and Effectiveness
Compounding cross-layer and cross-round KV sharing, the resulting Looped CED matches an unlooped CED’s minimal memory footprint and recovers 86–93% of a looped Transformer’s gain with 1/6 of its global KV.
Production-Scale Serving Evaluation
To test serving at production scale, we loop the 552B-parameter DeepSeek-V4.1-Flash for 8 rounds on one 4×GB200 node: with per-layer and per-round KV the loop cannot fit a single 128K-token request, whereas Looped CED holds 36.8× less KV per token, fits 38 such requests (as many as the unlooped model), and serves 3.6× higher throughput on 8K-token prompts.