首页 > AI前沿 > NariQ: Sub-3-Bit Quantization for Long-Horizon Agentic Tasks

NariQ: Sub-3-Bit Quantization for Long-Horizon Agentic Tasks

arXiv机器学习 2026-11-19 22:13 19 阅读 查看原文
用户希望将一段纯文本新闻正文排版成语义化的HTML片段。 需要遵循的规则: 1. 只输出HTML片段本身,不要Markdown围栏、不要解释。 2. 每个自然段落用`

`包裹,长文用`

`/`

`划分小节。 3. 列表内容用`
  • `,引用用`
    `,代码用`
    `,重要句子可用``。
    4. 保留原文所有信息,不得增删改内容。
    5. 不要输出``/``/``等文档级标签。
    
    正文内容:
    Long-horizon agentic tasks require a model to call tools over many steps at contexts of 100K tokens or more, yet many low-bit quantization methods are calibrated on short web text and judged on perplexity. This mismatch is costly: existing quantizations of Qwen3.8-27B at or below 3 bits per weight match BF16 on shorter tasks like AIME, yet in published results solve at most 72% as many tasks as BF16 on DeepSWE, a long-horizon agentic benchmark. We propose NariQ, a trellis-coded post-training quantization method that calibrates on tokens generated by the BF16 model in agent sessions. On Qwen3.8-27B, NariQ at 3.0 and 2.5 bits per weight stays level with BF16 on two long-horizon agentic benchmarks, DeepSWE and Terminal Bench, as well as on AIME and RealWorldQA. Our checkpoint at 2.0 bits per weight trails this performance. To our knowledge, these are the first quantizations at or below 3 bits per weight shown to match their full-precision version on long-horizon agentic tasks. We release the files as Qwen3.8-27B-NariQ-GGUF (https://huggingface.co/nari-labs/Qwen3.8-27B-NariQ-GGUF), with inference available through our llama.cpp fork at https://github.com/nari-labs/nari-llama.cpp.
    
    分析正文结构:
    这是一个相对较短的学术/技术新闻摘要,包含背景、问题、提出的方法、实验结果、结论和发布信息。
    可以划分小节,或者就作为一个段落,或者分成几个`

    `。 考虑到规则2说“长文用`

    `/`

    `划分小节”,这段文字不算特别长,但可以加几个小标题使其结构化,或者直接分段。 为了合理使用标签,我可以做如下划分:

    研究背景与问题

    Long-horizon agentic tasks require a model to call tools over many steps at contexts of 100K tokens or more, yet many low-bit quantization methods are calibrated on short web text and judged on perplexity. This mismatch is costly: existing quantizations of Qwen3.8-27B at or below 3 bits per weight match BF16 on shorter tasks like AIME, yet in published results solve at most 72% as many tasks as BF16 on DeepSWE, a long-horizon agentic benchmark.

    方法与实验结果

    We propose NariQ, a trellis-coded post-training quantization method that calibrates on tokens generated by the BF16 model in agent sessions. On Qwen3.8-27B, NariQ at 3.0 and 2.5 bits per weight stays level with BF16 on two long-horizon agentic benchmarks, DeepSWE and Terminal Bench, as well as on AIME and RealWorldQA. Our checkpoint at 2.0 bits per weight trails this performance.

    结论与开源发布

    To our knowledge, these are the first quantizations at or below 3 bits per weight shown to match their full-precision version on long-horizon agentic tasks. We release the files as Qwen3.8-27B-NariQ-GGUF (https://huggingface.co/nari-labs/Qwen3.8-27B-NariQ-GGUF), with inference available through our llama.cpp fork at https://github.com/nari-labs/nari-llama.cpp.

    等一下,规则要求“保留原文所有信息,不得增删改内容”。如果我加了小标题,算不算“增”? 小标题属于排版结构,通常不算增删改“正文内容”,但为了绝对安全,我可以不加自创的中文小标题,而是直接用`

    `包裹自然段落,或者把原文按句子分成几个`

    `。 不过规则明确说“长文用

    /

    划分小节”,说明期望我使用这些标签。如果我不加自创的标题文字,怎么划分小节呢? 或者我可以直接用`

    `包裹原文中的某些句子?不行,原文没有标题。 折中方案:不添加额外的标题文字,直接将原文的三个逻辑部分用`

    `包裹,或者为了使用`

    `,我可以把第一句作为标题?不,那会改变原文。 其实,直接全部用`

    `包裹也是符合“每个自然段落用

    包裹”的。原文就是一个自然段。我可以把它拆成几个`

    `,或者就一个`

    `。 如果只有一个自然段,直接一个`

    `即可。但为了展示排版能力,可以按逻辑拆分段落,或者加上合理的小标题(通常新闻排版会给小标题,不算增删内容,而是结构划分)。 再仔细看规则:“不得增删改内容”。添加小标题文字(如“背景”、“方法”)属于增加内容。因此,最安全的做法是不添加任何原文没有的文字,直接用`

    `标签。 或者,把原文分成几个`

    `,不使用`

    `,因为不是“长文”。 但为了用上``和``,我可以做如下处理:

    Long-horizon agentic tasks require a model to call tools over many steps at contexts of 100K tokens or more, yet many low-bit quantization methods are calibrated on short web text and judged on perplexity. This mismatch is costly: existing quantizations of Qwen3.8-27B at or below 3 bits per weight match BF16 on shorter tasks like AIME, yet in published results solve at most 72% as many tasks as BF16 on DeepSWE, a long-horizon agentic benchmark.

    We propose NariQ, a trellis-coded post-training quantization method that calibrates on tokens generated by the BF16 model in agent sessions. On Qwen3.8-27B, NariQ at 3.0 and 2.5 bits per weight stays level with BF16 on two long-horizon agentic benchmarks, DeepSWE and Terminal Bench, as well as on AIME and RealWorldQA. Our checkpoint at 2.0 bits per weight trails this performance.

    To our knowledge, these are the first quantizations at or below 3 bits per weight shown to match their full-precision version on long-horizon agentic tasks. We release the files as Qwen3.8-27B-NariQ-GGUF (https://huggingface.co/nari-labs/Qwen3.8-27B-NariQ-GGUF), with inference available through our llama.cpp fork at https://github.com/nari-labs/nari-llama.cpp.

    这样完全保留了原文,没有增加任何文字,使用了`

    `、``、``。 对于链接,原文是纯文本的URL,我可以用``标签包裹,或者用``包裹。用``更符合“代码用`

    `”的规则(