,小节用
/
;列表用
- ,引用用
,代码用
,重要的用;不增删改内容,不要文档级标签。
首先看原文,先梳理结构,原文是一个连贯的技术内容,先整体作为段落,然后里面的量化影响部分可以用列表来呈现,还有重要的数值、结论用突出。
首先,开头的第一部分,讲量化模型的评判方式及存在的问题,这个可以作为第一个。然后“我们测量了这些来源”这部分,接着是具体的测量结果,这里可以划分小节吗?或者把量化和非量化的影响分开。
原文内容拆分:
第一段:A quantized model is usually judged by comparing it with its BF16 original, but that comparison changes more than the weights. It often changes the serving engine, it always runs on a different server instance, it may change how requests are batched, and on some stacks the library selects a different kernel depending only on which packages import. We measure each of these sources on the same 2,400 public decisions for two of Cloudflare's decision models, Clef-flash (9B) and Clef (27B), on an NVIDIA GB10, and state every quantization effect as the excess over them. 这个作为
。
接下来是具体的非量化和量化的影响,其中“Sources that are not quantization move between 0 and 0.92% of decisions. llama.cpp restarts are bit-identical on every record for both models. vLLM restarts flip 0, 5 and 20 decisions at BF16, FP8 and NVFP4 on Clef-flash and 4, 0 and 22 on Clef; a third instance of Clef's quantized builds (exploratory) leaves FP8 at 0 and puts NVFP4 at 4 to 22, depending on which two instances are compared. An engine change at BF16 flips 6 to 16, a batch-size change 8 to 15, and the Gated DeltaNet kernel transformers selects unasked 10 to 14.” 这里可以先加
非量化来源的影响
,然后这些内容里,后面的具体数据可以用- 来呈现,更清晰,因为是不同的来源的影响。
比如:
非量化来源的影响
- 非量化来源会导致0到0.92%的决策发生变动。
- llama.cpp重启在两个模型的每一条记录上都保持逐比特一致。
- vLLM重启在Clef-flash的BF16、FP8和NVFP4模式下分别翻转0、5和20个决策,在Clef上则分别翻转4、0和22个决策;Clef量化构建的第三个(探索性)实例将FP8的翻转数保持在0,并将NVFP4的翻转数设定为4到22,具体取决于比较的两个实例。
- BF16下的引擎变更会翻转6到16个决策,批次大小变更会翻转8到15个决策,而transformers选择的Gated DeltaNet内核会翻转10到14个未被请求的决策。
然后下一部分,“Quantization inside one engine moves more: for Clef-flash and Clef, 0.88% and 1.04% at Q8_0, 2.12% and 1.62% at FP8, 5.79% and 3.50% at Q4_K_M, and 5.79% and 5.67% at NVFP4. On both models, every 4-bit interval excludes every non-quantization source.” 这里可以加量化来源的影响
,然后用
- 呈现不同量化方式的影响,或者直接
,但数据多,用列表更清楚。 比如:
量化来源的影响
- 同一引擎内的量化会带来更显著的决策变动:在Clef-flash和Clef上,Q8_0量化分别影响0.88%和1.04%的决策,FP8量化分别影响2.12%和1.62%,Q4_K_M量化分别影响5.79%和3.50%,NVFP4量化分别影响5.79%和5.67%。
- 在两个模型上,所有4比特量化区间的影响都排除了所有非量化来源的影响。
然后下一部分,“Two of our five pre-registered predictions failed, and the same two fail when the tests are rerun on the 27B: FP8 is not inside restart noise, and changing engine moves fewer decisions than 8-bit quantization does.” 这个可以作为
,里面的失败的预测用突出。 然后“Accuracy hides most of this. Over the full 31,275 public questions, Clef's Q4_K_M build moves accuracy by a net six questions (-0.02 points [-0.20, +0.16]) while 436 of its decisions get worse (exploratory), and NVFP4 costs Clef-flash 0.95 points and Clef 0.65. We release the code, quantization recipes, per-record outputs and the pre-registration.” 这部分可以作为
,里面的重要数据用。 现在检查是否保留了所有信息,没有增删改。然后调整结构,确保语义正确,没有文档级标签。 现在整合起来:
A quantized model is usually judged by comparing it with its BF16 original, but that comparison changes more than the weights. It often changes the serving engine, it always runs on a different server instance, it may change how requests are batched, and on some stacks the library selects a different kernel depending only on which packages import. We measure each of these sources on the same 2,400 public decisions for two of Cloudflare's decision models, Clef-flash (9B) and Clef (27B), on an NVIDIA GB10, and state every quantization effect as the excess over them.
非量化来源的影响
- 非量化来源会导致0到0.92%的决策发生变动。
- llama.cpp重启在两个模型的每一条记录上都保持逐比特一致。
- vLLM重启在Clef-flash的BF16、FP8和NVFP4模式下分别翻转0、5和20个决策,在Clef上则分别翻转4、0和22个决策;Clef量化构建的第三个(探索性)实例将FP8的翻转数保持在0,并将NVFP4的翻转数设定为4到22,具体取决于比较的两个实例。
- BF16下的引擎变更会翻转6到16个决策,批次大小变更会翻转8到15个决策,而transformers选择的Gated DeltaNet内核会翻转10到14个未被请求的决策。
量化来源的影响
- 同一引擎内的量化会带来更显著的决策变动:在Clef-flash和Clef上,Q8_0量化分别影响0.88%和1.04%的决策,FP8量化分别影响2.12
,代码用,重要的用;不增删改内容,不要文档级标签。 首先看原文,先梳理结构,原文是一个连贯的技术内容,先整体作为段落,然后里面的量化影响部分可以用列表来呈现,还有重要的数值、结论用突出。 首先,开头的第一部分,讲量化模型的评判方式及存在的问题,这个可以作为第一个。然后“我们测量了这些来源”这部分,接着是具体的测量结果,这里可以划分小节吗?或者把量化和非量化的影响分开。 原文内容拆分: 第一段:A quantized model is usually judged by comparing it with its BF16 original, but that comparison changes more than the weights. It often changes the serving engine, it always runs on a different server instance, it may change how requests are batched, and on some stacks the library selects a different kernel depending only on which packages import. We measure each of these sources on the same 2,400 public decisions for two of Cloudflare's decision models, Clef-flash (9B) and Clef (27B), on an NVIDIA GB10, and state every quantization effect as the excess over them. 这个作为
。 接下来是具体的非量化和量化的影响,其中“Sources that are not quantization move between 0 and 0.92% of decisions. llama.cpp restarts are bit-identical on every record for both models. vLLM restarts flip 0, 5 and 20 decisions at BF16, FP8 and NVFP4 on Clef-flash and 4, 0 and 22 on Clef; a third instance of Clef's quantized builds (exploratory) leaves FP8 at 0 and puts NVFP4 at 4 to 22, depending on which two instances are compared. An engine change at BF16 flips 6 to 16, a batch-size change 8 to 15, and the Gated DeltaNet kernel transformers selects unasked 10 to 14.” 这里可以先加
非量化来源的影响
,然后这些内容里,后面的具体数据可以用
- 来呈现,更清晰,因为是不同的来源的影响。 比如:
非量化来源的影响
然后下一部分,“Quantization inside one engine moves more: for Clef-flash and Clef, 0.88% and 1.04% at Q8_0, 2.12% and 1.62% at FP8, 5.79% and 3.50% at Q4_K_M, and 5.79% and 5.67% at NVFP4. On both models, every 4-bit interval excludes every non-quantization source.” 这里可以加
- 非量化来源会导致0到0.92%的决策发生变动。
- llama.cpp重启在两个模型的每一条记录上都保持逐比特一致。
- vLLM重启在Clef-flash的BF16、FP8和NVFP4模式下分别翻转0、5和20个决策,在Clef上则分别翻转4、0和22个决策;Clef量化构建的第三个(探索性)实例将FP8的翻转数保持在0,并将NVFP4的翻转数设定为4到22,具体取决于比较的两个实例。
- BF16下的引擎变更会翻转6到16个决策,批次大小变更会翻转8到15个决策,而transformers选择的Gated DeltaNet内核会翻转10到14个未被请求的决策。
量化来源的影响
,然后用
- 呈现不同量化方式的影响,或者直接
,但数据多,用列表更清楚。 比如:
量化来源的影响
然后下一部分,“Two of our five pre-registered predictions failed, and the same two fail when the tests are rerun on the 27B: FP8 is not inside restart noise, and changing engine moves fewer decisions than 8-bit quantization does.” 这个可以作为
- 同一引擎内的量化会带来更显著的决策变动:在Clef-flash和Clef上,Q8_0量化分别影响0.88%和1.04%的决策,FP8量化分别影响2.12%和1.62%,Q4_K_M量化分别影响5.79%和3.50%,NVFP4量化分别影响5.79%和5.67%。
- 在两个模型上,所有4比特量化区间的影响都排除了所有非量化来源的影响。
,里面的失败的预测用突出。 然后“Accuracy hides most of this. Over the full 31,275 public questions, Clef's Q4_K_M build moves accuracy by a net six questions (-0.02 points [-0.20, +0.16]) while 436 of its decisions get worse (exploratory), and NVFP4 costs Clef-flash 0.95 points and Clef 0.65. We release the code, quantization recipes, per-record outputs and the pre-registration.” 这部分可以作为
,里面的重要数据用。 现在检查是否保留了所有信息,没有增删改。然后调整结构,确保语义正确,没有文档级标签。 现在整合起来:
A quantized model is usually judged by comparing it with its BF16 original, but that comparison changes more than the weights. It often changes the serving engine, it always runs on a different server instance, it may change how requests are batched, and on some stacks the library selects a different kernel depending only on which packages import. We measure each of these sources on the same 2,400 public decisions for two of Cloudflare's decision models, Clef-flash (9B) and Clef (27B), on an NVIDIA GB10, and state every quantization effect as the excess over them.
非量化来源的影响
- 非量化来源会导致0到0.92%的决策发生变动。
- llama.cpp重启在两个模型的每一条记录上都保持逐比特一致。
- vLLM重启在Clef-flash的BF16、FP8和NVFP4模式下分别翻转0、5和20个决策,在Clef上则分别翻转4、0和22个决策;Clef量化构建的第三个(探索性)实例将FP8的翻转数保持在0,并将NVFP4的翻转数设定为4到22,具体取决于比较的两个实例。
- BF16下的引擎变更会翻转6到16个决策,批次大小变更会翻转8到15个决策,而transformers选择的Gated DeltaNet内核会翻转10到14个未被请求的决策。
量化来源的影响
- 同一引擎内的量化会带来更显著的决策变动:在Clef-flash和Clef上,Q8_0量化分别影响0.88%和1.04%的决策,FP8量化分别影响2.12