首页 > AI前沿 > Tracking Representations in LLM Reasoning Traces for Iterated Games

Tracking Representations in LLM Reasoning Traces for Iterated Games

arXiv自然语言 2026-11-19 13:00 3 阅读 查看原文
用户需要我将一段纯文本新闻正文排版成语义化的HTML片段。规则很明确: 1. 只输出HTML片段本身,不要Markdown围栏,不要解释 2. 每个自然段落用`

`包裹,长文用`

`/`

`划分小节 3. 列表内容用`
  • `,引用用`
    `,代码用`
    `,重要句子可用``
    4. 保留原文所有信息,不得增删改内容
    5. 不要输出``/``/``等文档级标签
    
    现在来看正文内容。这是一段关于大语言模型(LLM)智能体研究的新闻/学术文本。我来分析一下结构:
    
    原文是一整段,包含以下信息:
    - LLM智能体通过思维链(CoT)推理在各领域取得最先进表现
    - 尽管广泛使用,但外部化轨迹与模型实际计算之间的关系知之甚少
    - 研究探讨了LLM智能体如何在不完整信息的迭代游戏中做决策
    - 使用一系列机制可解释性技术,发现模型在整个CoT过程中形成并迭代修正游戏的内部表示
    - 一旦模型过早坚持错误信念,就很少能恢复
    - 这种失败在字面轨迹和层激活中都有体现
    - 随着多智能体系统越来越普遍,这项工作为更有针对性的策略推理错误诊断开辟了道路
    - 提供交互式可视化工具,网址在 https://interp-self.github.io/gametrace/
    
    这段文字虽然不长,但逻辑上有层次。用户要求用`

    `/`

    `划分小节,但原文并不是特别长的文本,也没有明显的小节结构。不过为了符合规则,我可以考虑是否需要划分。原文是一个连续的论述,从背景到方法到发现到意义。硬要划分小节可能会破坏原文的连贯性,而且规则说"长文用`

    `/`

    `划分小节",这段并不算长文。 我觉得最忠实的做法是将其作为几个自然段落用`

    `包裹,保持原文的连贯性,不强行添加小节标题(因为那会引入原文没有的结构信息,违背"不得增删改内容"的原则)。但原文其实是一整段,我需要合理拆分。 让我重新读一下,原文是一个连续的段落。为了排版美观和可读性,我可以按语义自然拆分为几个`

    `: - 第一句关于LLM智能体的表现:`Large language model (LLM) agents achieve state-of-the-art performance across domains through chain-of-thought (CoT) reasoning.` - 第二句关于研究空白:`Despite their ubiquitous use, little is understood about the relationship between the externalized trace and the model's actual computation.` - 第三句关于方法:`We address this by investigating how LLM agents make decisions in iterative games with incomplete information.` - 第四句关于技术发现:`Using a series of mechanistic interpretability techniques, we find that the model forms and iteratively revises an internal representation of the game throughout the CoT.` - 第五句关于失败模式:`Once the model prematurely commits to an incorrect belief, it rarely recovers.` - 第六句关于表现:`This failure is manifested in both the literal trace and the layer activations.` - 第七句关于意义:`Ultimately, as multi-agent systems are more prevalent, our work opens the door to a more focused diagnosis of their strategic reasoning errors.` - 第八句关于可视化:`For illustrative purposes, we offer an interactive visualizer of our methodology on selected sample traces at https://interp-self.github.io/gametrace/` 这些可以组合成几个段落。我把它们分组,形成逻辑连贯的段落,而不是每句一个段落(那样太碎了)。 我觉得可以这样分组: 段落1(背景与问题):前两句 段落2(方法与核心发现):第三、四句 段落3(失败模式与表现):第五、六句 段落4(意义与可视化):第七、八句 关于``,规则说"重要句子可用``"。我可以给核心发现加粗,比如"the model forms and iteratively revises an internal representation of the game throughout the CoT"以及"Once the model prematurely commits to an incorrect belief, it rarely recovers",这些是研究的核心发现。但加粗要适度,不要过度。我觉得给关键的发现性句子加粗比较合适。 不过,加粗属于一种强调,只要不改变内容,应该是可以的。让我合理地选择: - "the model forms and iteratively revises an internal representation of the game throughout the CoT" 是核心发现 - "Once the model prematurely commits to an incorrect belief, it rarely recovers" 是关键失败模式 我考虑是否加粗。规则是可选的("可用"),为了突出重点,我对这两句的核心部分加粗。但要注意,加粗部分应该是完整的语义单元。 让我重新组织,避免过度拆分导致信息割裂。原文其实是一个连贯的论述,我倾向于保持相对完整的段落,只做必要的拆分。 方案: ```

    Large language model (LLM) agents achieve state-of-the-art performance across domains through chain-of-thought (CoT) reasoning. Despite their ubiquitous use, little is understood about the relationship between the externalized trace and the model's actual computation.

    We address this by investigating how LLM agents make decisions in iterative games with incomplete information. Using a series of mechanistic interpretability techniques, we find that the model forms and iteratively revises an internal representation of the game throughout the CoT. Once the model prematurely commits to an incorrect belief, it rarely recovers. This failure is manifested in both the literal trace and the layer activations.

    Ultimately, as multi-agent systems are more prevalent, our work opens the door to a more focused diagnosis of their strategic reasoning errors. For illustrative purposes, we offer an interactive visualizer of our methodology on selected sample traces at https://interp-self.github.io/gametrace/

    ``` 这样比较合理。不过关于``,我是否应该加更多?原文中"rarely recovers"也是重要结论。但加太多就失去重点了。我觉得当前选择可以,或者也可以把那句失败模式也加粗。让我再想想。 其实"Once the model prematurely commits to an incorrect belief, it rarely recovers"是这篇研究的一个非常重要的发现,值得加粗。我可以这样: ```

    We address this by investigating how LLM agents make decisions in iterative games with incomplete information. Using a series of mechanistic interpretability techniques, we find that the model forms and iteratively revises an internal representation of the game throughout the CoT. Once the model prematurely commits to an incorrect belief, it rarely recovers. This failure is manifested in both the literal trace and the layer activations.

    ``` 这样两个核心发现都突出了。我觉得这是合理的。 关于是否使用`

    `/`

    `:这段文本不够长,没有明显的小节结构,强行加标题会引入原文没有的内容,违反"不得增删改内容"。所以我不用标题标签,只用`

    `。这是正确的判断。 关于网址:用户提到代码用`

    `,但网址不是代码,就是普通文本中的URL,不需要用代码块。保持原样在`

    `中即可。 最后确认:只输出HTML片段,没有Markdown围栏,没有解释,没有文档级标签。好的,我来输出。 等一下,我再检查一遍,确保所有原文信息都保留了,没有增删改。 原文逐句对照: 1. "Large language model (LLM) agents achieve state-of-the-art performance across domains through chain-of-thought (CoT) reasoning." ✓ 2. "Despite their ubiquitous use, little is understood about the relationship between the externalized trace and the model's actual computation." ✓ 3. "We address this by investigating how LLM agents make decisions in iterative games with incomplete information." ✓ 4. "Using a series of mechanistic interpretability techniques, we find that the model forms and iteratively revises an internal representation of the game throughout the CoT." ✓ 5. "Once the model prematurely commits to an incorrect belief, it rarely recovers." ✓ 6. "This failure is manifested in both the literal trace and the layer activations." ✓ 7. "Ultimately, as multi-agent systems are more prevalent, our work opens the door to a more focused diagnosis of their strategic reasoning errors." ✓ 8. "For illustrative purposes