首页 > AI前沿 > Beyond Truncation? Entailment-Licensed Elision and the Limits of Semantic Preservation in LLM Decoding

Beyond Truncation? Entailment-Licensed Elision and the Limits of Semantic Preservation in LLM Decoding

arXiv自然语言 2026-11-06 12:39 22 阅读 查看原文

Instruction-tuned large language models (LLMs) answer at greater length than a request needs. We ask whether a training-free controller inside the decoding loop can remove that surplus while preserving content, judged against the simplest possible baseline: truncating the model's own answer to the same length. An entropy-gated logits processor removes 25.5–34.9% of tokens on MT-Bench development questions and 19.9–34.2% on a sealed test split within pre-registered BERTScore non-inferiority margins, yet we prove and confirm by audit that its output is always a prefix of the unmodified answer: the saving is truncation.

A survey of 23 training-free mechanism families finds none that selects content better than truncation without oracle labels.

We therefore introduce Entailment-Licensed In-Decoding Elision (ELIDE), which, at each sentence boundary, withholds a span from the reader only when a natural-language inference model judges it entailed by the text already shown, while leaving the model's generation unchanged. ELIDE beats matched-length truncation on three model families, but reading 936 of its elisions on 670 public prompts shows that 59% harm the answer, a failure that no automatic measure detects.

Scoping the licence to its discourse unit reduces the harmful share to about one elision in ten on two fresh tests.