, use
/
for sections 3. Lists use
- , quotes use
, code uses
, important sentences can use
4. Keep all original information, no additions, deletions, or modifications
5. No document-level tags like //
Let me analyze the text structure:
The text is one long paragraph. It covers:
- Introduction: LLMs as math tutors, two questions raised
- The framework MISCONCEPTDIALOGUE and methodology
- Measurement findings: discrepancy between exact-match scoring (42.1%) and LLM-as-judge (81.3%), 39.2 percentage-point spread
- Not a single-judge artifact: four LLM judges and 200 human annotations
- Capability findings: tutors barely use diagnostic information
- Conclusion/recommendation
Since the text is one long paragraph, I should decide whether to split it or keep it. The rule says "每个自然段落用包裹" - wrap each natural paragraph. The original is one paragraph, but for better semantic structure, I could split it into logical sections with
/
headings.
Let me structure it:
1. Introduction - could be
or just
2. The framework MISCONCEPTDIALOGUE -
3. Measurement findings -
maybe, with on key numbers
4. Judge agreement -
5. Capability findings -
maybe
6. Conclusion -
with Let me think about what makes sense semantically: - Opening paragraph about LLMs as math tutors and the two questions - Section about MISCONCEPTDIALOGUE framework - Section about measurement findings - Section about judge validation - Section about capability findings - Conclusion Actually, the text is quite dense. Let me preserve it faithfully while adding semantic structure. Key facts to preserve: - LLMs increasingly used as math tutors - Two questions: can a single LLM detect error, diagnose misconception, remediate through multi-turn dialogue; can we trust the numbers - MISCONCEPTDIALOGUE: fully automated framework - Five open-weight LLM tutors (7B-14B) - LLM-simulated students - 5,000+ dialogues - Deliberately simulated protocol (paired conditions, wrong-diagnosis control unsafe with real learners, scale) - Results do not establish real-student learning - Benchmark-style exact-match scoring: 42.1% measured Remediation Success Rate - Human-calibrated LLM-as-judge: 81.3% - 39.2 percentage-point spread - Larger than combined effects of tutor model, teaching strategy, and supplied diagnostic information - Not a single-judge artifact: four independent LLM judges and 200 human annotations converge within a few points (kappa up to 0.91) - Score barely changes whether tutor is told actual misconception or deliberately swapped one - Under tested diagnosis-presentation format, tutors show little evidence of reliably conditioning on diagnostic information - Recommendation: multi-criterion scoring with multi-judge convergence and human anchoring should become standard practice - Measurement question deserves as much scrutiny as capability question Structure plan:
Opening paragraph about LLMs as math tutors and the two questions
MISCONCEPTDIALOGUE Framework
(or just a paragraph)
Description of the framework...
Measurement Findings
...
Judge Convergence
...
Capability Verdict
...
Conclusion...
Hmm, but I need to be careful - the original text is ONE paragraph. Should I split it? The rule says "长文用
/
划分小节" (use h2/h3 to divide sections for long texts). This text is fairly long, so dividing into sections makes sense. Let me draft:
Large language models are increasingly used as math tutors, raising two questions: can a single LLM detect a student's error, diagnose the underlying misconception, and remediate it through multi-turn dialogue - and can we trust the numbers that report how well it does?
We address both with MISCONCEPTDIALOGUE, a fully automated framework that runs the full pipeline on five open-weight LLM tutors (7B-14B) interacting with LLM-simulated students over 5,000+ dialogues - a deliberately simulated protocol (paired conditions, a wrong-diagnosis control unsafe with real learners, scale) whose results do not establish real-student learning.
Measurement Findings
On measurement, our headline finding is uncomfortable for the field: benchmark-style exact-match scoring common in standard math NLP - a lower-bound stress test - reports 42.1% measured Remediation Success Rate on identical dialogues where a human-calibrated LLM-as-judge reports 81.3%, a 39.2 percentage-point spread larger than the combined effects of tutor model, teaching strategy, and supplied diagnostic information.
This is not a single-judge artifact: four independent LLM judges and 200 human annotations converge to within a few points (kappa up to 0.91).
Capability Verdict
On capability, the verdict is equally striking: the score barely changes whether we tell the tutor the student's actual misconception or a deliberately swapped one - under the tested diagnosis-presentation format, these tutors show little evidence of reliably conditioning on the diagnostic information they are handed.
For LLM tutoring evaluation to mature, multi-criterion scoring with multi-judge convergence and human anchoring should become standard practice - and the measurement question deserves as much scrutiny as the capability question it claims to answer.
Wait, I need to check the original text carefully. Let me re-read: "We address both with MISCONCEPTDIALOGUE, a fully automated framework that runs the full pipeline on five open-weight LLM tutors (7B-14B) interacting with LLM-simulated students over 5,000+ dialogues - a deliberately simulated protocol (paired conditions, a wrong-diagnosis control unsafe with real learners, scale) whose results do not establish real-student learning." OK. Now let me also consider whether to use
anywhere. The text doesn't have explicit quotes, so probably not needed. Should I keep it as fewer paragraphs? The rule says each natural paragraph wrapped in
. The original is one paragraph, but splitting for readability with headings seems appropriate given the "长文" (long text) clause. Actually, let me reconsider. Maybe I should be more conservative and split into fewer sections. Let me think about the natural flow: 1. Hook/intro: LLMs as math tutors, two questions 2. Framework: MISCONCEPTDIALOGUE description 3. Measurement findings: the 42.1% vs 81.3% discrepancy 4. Judge validation: four judges + 200 annotations 5. Capability: tutors don't use diagnostic info 6. Conclusion: recommendations I think my draft above is good. Let me refine the headings: -
MISCONCEPTDIALOGUE 框架
- no, should I translate headings? The rule doesn't say to translate. Better to keep headings in English since the content is English. Actually, headings could be in either. Let me keep them in English to match the text. Let me reconsider whether to add headings at all. The text is about 200 words. It
, code uses, important sentences can use 4. Keep all original information, no additions, deletions, or modifications 5. No document-level tags like // Let me analyze the text structure: The text is one long paragraph. It covers: - Introduction: LLMs as math tutors, two questions raised - The framework MISCONCEPTDIALOGUE and methodology - Measurement findings: discrepancy between exact-match scoring (42.1%) and LLM-as-judge (81.3%), 39.2 percentage-point spread - Not a single-judge artifact: four LLM judges and 200 human annotations - Capability findings: tutors barely use diagnostic information - Conclusion/recommendation Since the text is one long paragraph, I should decide whether to split it or keep it. The rule says "每个自然段落用包裹" - wrap each natural paragraph. The original is one paragraph, but for better semantic structure, I could split it into logical sections with
/
headings. Let me structure it: 1. Introduction - could be
or just
2. The framework MISCONCEPTDIALOGUE -
3. Measurement findings -
maybe, with on key numbers 4. Judge agreement -
5. Capability findings -
maybe 6. Conclusion -
with Let me think about what makes sense semantically: - Opening paragraph about LLMs as math tutors and the two questions - Section about MISCONCEPTDIALOGUE framework - Section about measurement findings - Section about judge validation - Section about capability findings - Conclusion Actually, the text is quite dense. Let me preserve it faithfully while adding semantic structure. Key facts to preserve: - LLMs increasingly used as math tutors - Two questions: can a single LLM detect error, diagnose misconception, remediate through multi-turn dialogue; can we trust the numbers - MISCONCEPTDIALOGUE: fully automated framework - Five open-weight LLM tutors (7B-14B) - LLM-simulated students - 5,000+ dialogues - Deliberately simulated protocol (paired conditions, wrong-diagnosis control unsafe with real learners, scale) - Results do not establish real-student learning - Benchmark-style exact-match scoring: 42.1% measured Remediation Success Rate - Human-calibrated LLM-as-judge: 81.3% - 39.2 percentage-point spread - Larger than combined effects of tutor model, teaching strategy, and supplied diagnostic information - Not a single-judge artifact: four independent LLM judges and 200 human annotations converge within a few points (kappa up to 0.91) - Score barely changes whether tutor is told actual misconception or deliberately swapped one - Under tested diagnosis-presentation format, tutors show little evidence of reliably conditioning on diagnostic information - Recommendation: multi-criterion scoring with multi-judge convergence and human anchoring should become standard practice - Measurement question deserves as much scrutiny as capability question Structure plan:
Opening paragraph about LLMs as math tutors and the two questions
MISCONCEPTDIALOGUE Framework
(or just a paragraph)Description of the framework...
Measurement Findings
...
Judge Convergence
...
Capability Verdict
...
Conclusion...
Hmm, but I need to be careful - the original text is ONE paragraph. Should I split it? The rule says "长文用/
划分小节" (use h2/h3 to divide sections for long texts). This text is fairly long, so dividing into sections makes sense. Let me draft:
Large language models are increasingly used as math tutors, raising two questions: can a single LLM detect a student's error, diagnose the underlying misconception, and remediate it through multi-turn dialogue - and can we trust the numbers that report how well it does?
We address both with MISCONCEPTDIALOGUE, a fully automated framework that runs the full pipeline on five open-weight LLM tutors (7B-14B) interacting with LLM-simulated students over 5,000+ dialogues - a deliberately simulated protocol (paired conditions, a wrong-diagnosis control unsafe with real learners, scale) whose results do not establish real-student learning.
Measurement Findings
On measurement, our headline finding is uncomfortable for the field: benchmark-style exact-match scoring common in standard math NLP - a lower-bound stress test - reports 42.1% measured Remediation Success Rate on identical dialogues where a human-calibrated LLM-as-judge reports 81.3%, a 39.2 percentage-point spread larger than the combined effects of tutor model, teaching strategy, and supplied diagnostic information.
This is not a single-judge artifact: four independent LLM judges and 200 human annotations converge to within a few points (kappa up to 0.91).
Capability Verdict
On capability, the verdict is equally striking: the score barely changes whether we tell the tutor the student's actual misconception or a deliberately swapped one - under the tested diagnosis-presentation format, these tutors show little evidence of reliably conditioning on the diagnostic information they are handed.
For LLM tutoring evaluation to mature, multi-criterion scoring with multi-judge convergence and human anchoring should become standard practice - and the measurement question deserves as much scrutiny as the capability question it claims to answer.
Wait, I need to check the original text carefully. Let me re-read: "We address both with MISCONCEPTDIALOGUE, a fully automated framework that runs the full pipeline on five open-weight LLM tutors (7B-14B) interacting with LLM-simulated students over 5,000+ dialogues - a deliberately simulated protocol (paired conditions, a wrong-diagnosis control unsafe with real learners, scale) whose results do not establish real-student learning." OK. Now let me also consider whether to useanywhere. The text doesn't have explicit quotes, so probably not needed. Should I keep it as fewer paragraphs? The rule says each natural paragraph wrapped in. The original is one paragraph, but splitting for readability with headings seems appropriate given the "长文" (long text) clause. Actually, let me reconsider. Maybe I should be more conservative and split into fewer sections. Let me think about the natural flow: 1. Hook/intro: LLMs as math tutors, two questions 2. Framework: MISCONCEPTDIALOGUE description 3. Measurement findings: the 42.1% vs 81.3% discrepancy 4. Judge validation: four judges + 200 annotations 5. Capability: tutors don't use diagnostic info 6. Conclusion: recommendations I think my draft above is good. Let me refine the headings: -
MISCONCEPTDIALOGUE 框架
- no, should I translate headings? The rule doesn't say to translate. Better to keep headings in English since the content is English. Actually, headings could be in either. Let me keep them in English to match the text. Let me reconsider whether to add headings at all. The text is about 200 words. It