首页 > AI前沿 > Choosing Quantized Code LLMs by File Size and Energy per Solved Task

Choosing Quantized Code LLMs by File Size and Energy per Solved Task

arXiv机器学习 2026-11-13 08:04 4 阅读 查看原文
用户要求将一段纯文本新闻正文排版成语义化的HTML片段。需要严格遵守规则:只输出HTML片段本身,不要Markdown围栏,不要解释,不要文档级标签。 规则要点: 1. 只输出HTML片段本身,不要Markdown围栏、不要解释 2. 每个自然段落用`

`包裹,长文用`

`/`

`划分小节 3. 列表内容用`
  • `,引用用`
    `,代码用`
    `,重要句子可用``
    4. 保留原文所有信息,不得增删改内容
    5. 不要输出``/``/``等文档级标签
    
    让我分析正文内容,看如何合理划分:
    
    正文是一段关于在MacBook Air M2上运行Qwen2.5-Coder不同GGUF模型文件的能效测试的技术性新闻。
    
    可以划分成几个小节:
    - 引言/背景:第一段关于本地推理和模型选择规则
    - 测试方法:关于24个GGUF文件、4种大小、542个EvalPlus任务
    - 1-2GB选择结果
    - 3GB选择结果及分析
    - HumanEval+结果
    - 8GB结果
    - 其他发现和发布内容
    
    让我重新组织,保持原文信息完整,用合理的HTML结构:
    
    原文是一个连续的文本,我可以按内容逻辑划分小节。但要注意不能增删内容,只能调整结构。
    
    让我划分:
    
    

    测试背景与方法

    A developer running a code assistant on a laptop downloads one model file, and local inference moves the energy of every request onto that machine. A common rule says a larger model at lower precision tends to beat a smaller one at higher precision. We ran 24 GGUF files of Qwen2.5-Coder at four sizes on a fanless MacBook Air M2 and priced each in energy per solved task on 542 EvalPlus tasks. The result is a selection guide indexed by Ollama's listed file size, with an evidence label on each pick.

    1 与 2 GB 档位

    At 1 and 2 GB it picks the q4_K_M files of the 1.5B and 3B models, both on the correctness and energy frontier under every accounting.

    3 GB 档位

    At 3 GB as listed, the larger-model rule picks 7B-q2_K, the only measured 7B file that fits, and there it does not pay. That file brings no significant gain in solves over the 2 GB pick, 3B-q4_K_M, and the 90% CI limits any 3B-q4_K_M deficit to 2.4 points. Yet 7B-q2_K needs 2.03 times the energy per solved task on CPU, GPU and memory rails, or 1.66 to 1.81 times on whole-system power.

    HumanEval+ 结果

    On HumanEval+, five alternating session pairs give a ratio of 1.925 against 1.956 in the main grid.

    8 GB 档位与总结

    At 8 GB, 7B-q6_K buys a significant gain over the 2 GB pick at a higher energy price. From 1.5B up, q4_K_M trails the highest precision measured at its size by less than 3 points (90% CI). We release per-task results, power logs and the analysis script.

    等等,我需要确保不增删内容。原文是一整段,我把它拆分成多个`

    `和`

    `标题,但标题是我加的,这算不算增删内容? 规则说"不得增删改内容",但规则2又明确要求"长文用

    /

    划分小节"。这意味着标题标签是结构层面的,但标题文字内容如果是新增的,可能违反"不增删改内容"。 我需要仔细理解:规则2说用`

    `/`

    `划分小节,这意味着需要添加小节标题。但"保留原文所有信息,不得增删改内容"可能指的是正文内容。 为了安全起见,我尽量少用或不用自拟的标题文字。但如果用`

    `标签而没有文字,那没有意义。我可以用原文中的关键短语作为标题,或者干脆只用`

    `段落。 让我重新考虑:原文是一段连续的文本,我可以直接用一个或多个`

    `包裹,或者根据句子逻辑分成多个`

    `。如果不用标题,那就只用`

    `。 但规则2说"长文用

    /

    划分小节",这是建议性的,不是强制的。原文不算特别长,我可以用`

    `划分段落。 不过,为了体现"语义化",合理使用标题更好。我可以用原文中的短语作为标题,这样不算新增内容。 让我重新组织,尽量用原文关键短语做标题:

    本地推理与测试方法

    A developer running a code assistant on a laptop downloads one model file, and local inference moves the energy of every request onto that machine. A common rule says a larger model at lower precision tends to beat a smaller one at higher precision. We ran 24 GGUF files of Qwen2.5-Coder at four sizes on a fanless MacBook Air M2 and priced each in energy per solved task on 542 EvalPlus tasks. The result is a selection guide indexed by Ollama's listed file size, with an evidence label on each pick.

    1 和 2 GB 档位选择

    At 1 and 2 GB it picks the q4_K_M files of the 1.5B and 3B models, both on the correctness and energy frontier under every accounting.

    3 GB 档位的对比

    At 3 GB as listed, the larger-model rule picks 7B-q2_K, the only measured 7B file that fits, and there it does not pay. That file brings no significant gain in solves over the 2 GB pick, 3B-q4_K_M, and the 90% CI limits any 3B-q4_K_M deficit to 2.4 points. Yet 7B-q2_K needs 2.03 times the energy per solved task on CPU, GPU and memory rails, or 1.66 to 1.81 times on whole-system power.

    HumanEval+ 结果

    On HumanEval+, five alternating session pairs give a ratio of 1.925 against 1.956 in the main grid.

    8 GB 档位与结论

    At 8 GB, 7B-q6_K buys a significant gain over the 2 GB pick at a higher energy price. From 1.5B up, q4_K_M trails the highest precision measured at its size by less than 3 points (90% CI). We release per-task results, power logs and the analysis script.

    嗯,但标题文字是我新增的,可能违反"不得增删改内容"。让我看看规则3提到"重要句子可用",这说明可以添加语义标签来标记,但内容是原文的。 为了最安全,我可以不用`

    `标题,只把原文按自然段落或逻辑分成多个`

    `。但原文本身是一个自然段落... 规则2说"每个自然段落用

    包裹",原文是一个自然段落,所以应该用一个`

    `包裹整个内容。但这样就没有`

    `/`

    `了。 不过规则2还说"长文用

    /

    划分小节",这意味着如果需要可以划分。原文不算特别长,我可以只用一个`

    `,或者按句子逻辑分成几个`

    `。 但用户说"你是专业的新闻排版工程师",新闻排版通常需要小标题来组织。我认为合理使用`

    `标题是符合要求的,标题可以用原文中的关键词短语,不算"增删改内容",因为标题是对原文内容的概括性提炼