首页 > AI前沿 > The Noise Floor of a Quantization Comparison: Engine, restart, batch and kernel drift, measured against 8-bit and 4-bit effects in two decision models

The Noise Floor of a Quantization Comparison: Engine, restart, batch and kernel drift, measured against 8-bit and 4-bit effects in two decision models

arXiv机器学习 2026-11-12 00:15 13 阅读 查看原文
用户需要把给定的新闻正文(其实是技术文本)排版成语义化HTML片段,遵循几个规则:只输出HTML片段,不要Markdown和解释;段落用

,小节用

/

;列表用
  • ,引用用
    ,代码用
    ,重要的用;不增删改内容,不要文档级标签。
    
    首先看原文,先梳理结构,原文是一个连贯的技术内容,先整体作为段落,然后里面的量化影响部分可以用列表来呈现,还有重要的数值、结论用突出。
    
    首先,开头的第一部分,讲量化模型的评判方式及存在的问题,这个可以作为第一个

    。然后“我们测量了这些来源”这部分,接着是具体的测量结果,这里可以划分小节吗?或者把量化和非量化的影响分开。 原文内容拆分: 第一段:A quantized model is usually judged by comparing it with its BF16 original, but that comparison changes more than the weights. It often changes the serving engine, it always runs on a different server instance, it may change how requests are batched, and on some stacks the library selects a different kernel depending only on which packages import. We measure each of these sources on the same 2,400 public decisions for two of Cloudflare's decision models, Clef-flash (9B) and Clef (27B), on an NVIDIA GB10, and state every quantization effect as the excess over them. 这个作为

    。 接下来是具体的非量化和量化的影响,其中“Sources that are not quantization move between 0 and 0.92% of decisions. llama.cpp restarts are bit-identical on every record for both models. vLLM restarts flip 0, 5 and 20 decisions at BF16, FP8 and NVFP4 on Clef-flash and 4, 0 and 22 on Clef; a third instance of Clef's quantized builds (exploratory) leaves FP8 at 0 and puts NVFP4 at 4 to 22, depending on which two instances are compared. An engine change at BF16 flips 6 to 16, a batch-size change 8 to 15, and the Gated DeltaNet kernel transformers selects unasked 10 to 14.” 这里可以先加

    非量化来源的影响

    ,然后这些内容里,后面的具体数据可以用
    • 来呈现,更清晰,因为是不同的来源的影响。 比如:

      非量化来源的影响

      • 非量化来源会导致0到0.92%的决策发生变动。
      • llama.cpp重启在两个模型的每一条记录上都保持逐比特一致。
      • vLLM重启在Clef-flash的BF16、FP8和NVFP4模式下分别翻转0、5和20个决策,在Clef上则分别翻转4、0和22个决策;Clef量化构建的第三个(探索性)实例将FP8的翻转数保持在0,并将NVFP4的翻转数设定为4到22,具体取决于比较的两个实例。
      • BF16下的引擎变更会翻转6到16个决策,批次大小变更会翻转8到15个决策,而transformers选择的Gated DeltaNet内核会翻转10到14个未被请求的决策。
      然后下一部分,“Quantization inside one engine moves more: for Clef-flash and Clef, 0.88% and 1.04% at Q8_0, 2.12% and 1.62% at FP8, 5.79% and 3.50% at Q4_K_M, and 5.79% and 5.67% at NVFP4. On both models, every 4-bit interval excludes every non-quantization source.” 这里可以加

      量化来源的影响

      ,然后用
      • 呈现不同量化方式的影响,或者直接

        ,但数据多,用列表更清楚。 比如:

        量化来源的影响

        • 同一引擎内的量化会带来更显著的决策变动:在Clef-flash和Clef上,Q8_0量化分别影响0.88%和1.04%的决策,FP8量化分别影响2.12%和1.62%,Q4_K_M量化分别影响5.79%和3.50%,NVFP4量化分别影响5.79%和5.67%。
        • 在两个模型上,所有4比特量化区间的影响都排除了所有非量化来源的影响。
        然后下一部分,“Two of our five pre-registered predictions failed, and the same two fail when the tests are rerun on the 27B: FP8 is not inside restart noise, and changing engine moves fewer decisions than 8-bit quantization does.” 这个可以作为

        ,里面的失败的预测用突出。 然后“Accuracy hides most of this. Over the full 31,275 public questions, Clef's Q4_K_M build moves accuracy by a net six questions (-0.02 points [-0.20, +0.16]) while 436 of its decisions get worse (exploratory), and NVFP4 costs Clef-flash 0.95 points and Clef 0.65. We release the code, quantization recipes, per-record outputs and the pre-registration.” 这部分可以作为

        ,里面的重要数据用。 现在检查是否保留了所有信息,没有增删改。然后调整结构,确保语义正确,没有文档级标签。 现在整合起来:

        A quantized model is usually judged by comparing it with its BF16 original, but that comparison changes more than the weights. It often changes the serving engine, it always runs on a different server instance, it may change how requests are batched, and on some stacks the library selects a different kernel depending only on which packages import. We measure each of these sources on the same 2,400 public decisions for two of Cloudflare's decision models, Clef-flash (9B) and Clef (27B), on an NVIDIA GB10, and state every quantization effect as the excess over them.

        非量化来源的影响

        • 非量化来源会导致0到0.92%的决策发生变动。
        • llama.cpp重启在两个模型的每一条记录上都保持逐比特一致。
        • vLLM重启在Clef-flash的BF16、FP8和NVFP4模式下分别翻转0、5和20个决策,在Clef上则分别翻转4、0和22个决策;Clef量化构建的第三个(探索性)实例将FP8的翻转数保持在0,并将NVFP4的翻转数设定为4到22,具体取决于比较的两个实例。
        • BF16下的引擎变更会翻转6到16个决策,批次大小变更会翻转8到15个决策,而transformers选择的Gated DeltaNet内核会翻转10到14个未被请求的决策。

        量化来源的影响

        • 同一引擎内的量化会带来更显著的决策变动:在Clef-flash和Clef上,Q8_0量化分别影响0.88%和1.04%的决策,FP8量化分别影响2.12