`包裹,长文用`
`/`
`划分小节。 3. 列表内容用`
- `,引用用`
`,代码用`
`,重要句子可用``。
4. 保留原文所有信息,不得增删改内容。
5. 不要输出``/``/``等文档级标签。
正文内容:
"Recent benchmark evaluations increasingly rely on Large Language Models (LLMs) as automatic judges. This paradigm assumes LLMs evaluate based on the rubric's semantic meaning rather than its numeric scale. We question this assumption by introducing semantic faithfulness: the principle that semantically equivalent rubrics must yield consistent scores. To quantify this, we propose a scale-reversal probe that inverts score anchors without altering evaluation criteria. Furthermore, we introduce floor normalization to calibrate reversal disagreement against intrinsic model variability, enabling fair cross-model comparisons. Experiments across five meta-evaluation benchmarks with seven diverse LLM judges spanning open-weight and proprietary models reveal widespread violations of semantic faithfulness, showing that raw reversal agreement alone is misleading. Finally, we demonstrate that bidirectional anchor averaging reduces dependence on any single anchor direction without compromising alignment with human judgments. Our findings emphasize that trustworthy LLM judges must reliably follow rubric semantics, not just correlate with human ratings."
分析正文结构:
这是一段学术摘要/新闻段落,没有明显的列表、引用或代码。
可以将其划分为几个小节,或者直接作为一个长段落,或者用``/`
`划分小节。
由于原文只有一段,但我可以使用`
`或`
`来划分小节,或者直接用`
`包裹,并给重要句子加上``。
不过,题目说“长文用
`,代码用``,重要句子可用``。 4. 保留原文所有信息,不得增删改内容。 5. 不要输出``/``/``等文档级标签。 正文内容: "Recent benchmark evaluations increasingly rely on Large Language Models (LLMs) as automatic judges. This paradigm assumes LLMs evaluate based on the rubric's semantic meaning rather than its numeric scale. We question this assumption by introducing semantic faithfulness: the principle that semantically equivalent rubrics must yield consistent scores. To quantify this, we propose a scale-reversal probe that inverts score anchors without altering evaluation criteria. Furthermore, we introduce floor normalization to calibrate reversal disagreement against intrinsic model variability, enabling fair cross-model comparisons. Experiments across five meta-evaluation benchmarks with seven diverse LLM judges spanning open-weight and proprietary models reveal widespread violations of semantic faithfulness, showing that raw reversal agreement alone is misleading. Finally, we demonstrate that bidirectional anchor averaging reduces dependence on any single anchor direction without compromising alignment with human judgments. Our findings emphasize that trustworthy LLM judges must reliably follow rubric semantics, not just correlate with human ratings." 分析正文结构: 这是一段学术摘要/新闻段落,没有明显的列表、引用或代码。 可以将其划分为几个小节,或者直接作为一个长段落,或者用``/`
`划分小节。 由于原文只有一段,但我可以使用`
`或`
`来划分小节,或者直接用`
`包裹,并给重要句子加上``。 不过,题目说“长文用
/划分小节”,这段文字虽然不长,但可以按逻辑划分:
比如:
研究背景与核心假设
Recent benchmark evaluations increasingly rely on Large Language Models (LLMs) as automatic judges. This paradigm assumes LLMs evaluate based on the rubric's semantic meaning rather than its numeric scale.
语义忠实度与评估方法
We question this assumption by introducing semantic faithfulness: the principle that semantically equivalent rubrics must yield consistent scores. To quantify this, we propose a scale-reversal probe that inverts score anchors without altering evaluation criteria. Furthermore, we introduce floor normalization to calibrate reversal disagreement against intrinsic model variability, enabling fair cross-model comparisons.
实验结果与结论
Experiments across five meta-evaluation benchmarks with seven diverse LLM judges spanning open-weight and proprietary models reveal widespread violations of semantic faithfulness, showing that raw reversal agreement alone is misleading. Finally, we demonstrate that bidirectional anchor averaging reduces dependence on any single anchor direction without compromising alignment with human judgments. Our findings emphasize that trustworthy LLM judges must reliably follow rubric semantics, not just correlate with human ratings.
检查是否增删改了内容:
原文:"Recent benchmark evaluations increasingly rely on Large Language Models (LLMs) as automatic judges. This paradigm assumes LLMs evaluate based on the rubric's semantic meaning rather than its numeric scale. We question this assumption by introducing semantic faithfulness: the principle that semantically equivalent rubrics must yield consistent scores. To quantify this, we propose a scale-reversal probe that inverts score anchors without altering evaluation criteria. Furthermore, we introduce floor normalization to calibrate reversal disagreement against intrinsic model variability, enabling fair cross-model comparisons. Experiments across five meta-evaluation benchmarks with seven diverse LLM judges spanning open-weight and proprietary models reveal widespread violations of semantic faithfulness, showing that raw reversal agreement alone is misleading. Finally, we demonstrate that bidirectional anchor averaging reduces dependence on any single anchor direction without compromising alignment with human judgments. Our findings emphasize that trustworthy LLM judges must reliably follow rubric semantics, not just correlate with human ratings."
划分后的内容:
Section 1:
Recent benchmark evaluations increasingly rely on Large Language Models (LLMs) as automatic judges. This paradigm assumes LLMs evaluate based on the rubric's semantic meaning rather than its numeric scale.
Section 2:
We question this assumption by introducing semantic faithfulness: the principle that semantically equivalent rubrics must yield consistent scores. To quantify this, we propose a scale-reversal probe that inverts score anchors without altering evaluation criteria. Furthermore, we introduce floor normalization to calibrate reversal disagreement against intrinsic model variability, enabling fair cross-model comparisons.
Section 3:
Experiments across five meta-evaluation benchmarks with seven diverse LLM judges spanning open-weight and proprietary models reveal widespread violations of semantic faithfulness, showing that raw reversal agreement alone is misleading. Finally, we demonstrate that bidirectional anchor averaging reduces dependence on any single anchor direction without compromising alignment with human judgments. Our findings emphasize that trustworthy LLM judges must reliably follow rubric semantics, not just correlate with human ratings.
添加标签:
semantic faithfulness -> semantic faithfulness
trustworthy LLM judges must reliably follow rubric semantics, not just correlate with human ratings -> trustworthy LLM judges must reliably follow rubric semantics, not just correlate with human ratings
检查HTML标签:
研究背景与核心假设
(等等,标题算不算增删内容?题目说“保留原文所有信息,不得增删改内容”,添加小标题可能会被视为增加了内容。为了安全起见,如果不加小标题,直接用包裹整段,或者只加
。但规则2说“长文用
/
划分小节”,如果原文没有小节,我不应该硬加小标题文本。那我可以用
来划分,但不写具体文字?不行,空标题没意义。或者我直接把整段放在一个
里,因为原文就是一个自然段落。规则说“每个自然段落用
包裹,长文用
/
划分小节”,原文只有一个自然段落,不算“长文”或多段落,所以直接用
包裹即可,不需要强行加h2/h3,以免增加原文没有的文字。 再仔细看原文,确实只有一段。那我就只用一个
标签包裹整段,并对重要句子加。 或者,我可以把原文分成几个
,代表几个自然段?但原文明明是一整段(没有换行符)。强行分段算“改内容”吗?通常排版时,长文本可以按句子或逻辑拆分,但严格来说“不得增删改内容”意味着文本字符不能变。如果我用多个
,只是加了HTML标签,文本本身没变,但分段了。为了最严格遵守“不得增删改内容”,保持原文的段落结构(即一个自然段),只用一个
。 等等,如果用户希望看到
/
,我能不能不加文字,或者加HTML注释?不行,HTML注释也是增。 其实,把原文作为一段,用
包裹,并加,完全符合规则。 让我再审视规则2:“每个自然段落用
包裹,长文用
/
划分小节”。原文只有一个自然段落,所以不需要划分小节。 构建HTML:
Recent benchmark evaluations increasingly rely on Large Language Models (LLMs) as automatic judges. This paradigm assumes LLMs evaluate based on the rubric's semantic meaning rather than its numeric scale. We question this assumption by introducing semantic faithfulness: the principle that semantically equivalent rubrics must yield consistent scores. To quantify this, we propose a scale-reversal probe that inverts score anchors without altering evaluation criteria. Furthermore, we introduce floor normalization to calibrate revers
划分小节”。原文只有一个自然段落,所以不需要划分小节。 构建HTML:
Recent benchmark evaluations increasingly rely on Large Language Models (LLMs) as automatic judges. This paradigm assumes LLMs evaluate based on the rubric's semantic meaning rather than its numeric scale. We question this assumption by introducing semantic faithfulness: the principle that semantically equivalent rubrics must yield consistent scores. To quantify this, we propose a scale-reversal probe that inverts score anchors without altering evaluation criteria. Furthermore, we introduce floor normalization to calibrate revers