首页 > AI前沿 > A Mechanistic Analysis of In-Context Learning and Out-of-Context Hallucination

A Mechanistic Analysis of In-Context Learning and Out-of-Context Hallucination

arXiv自然语言 2026-11-19 13:00 6 阅读 查看原文
用户需要我将一段纯文本的新闻(实际是学术摘要)正文排版成语义化的HTML片段。我需要仔细遵循给定的规则: 1. 只输出HTML片段本身,不要Markdown围栏,不要解释。 2. 每个自然段落用`

`包裹,长文用`

`/`

`划分小节。 3. 列表用`
  • `,引用用`
    `,代码用`
    `,重要句子可用``。
    4. 保留原文所有信息,不得增删改内容。
    5. 不要输出``/``/``等文档级标签。
    
    现在看原文,它是单个学术段落。我需要把它拆分成逻辑上的小节,可能加上标题,并给重要句子加``。
    
    原文是关于大语言模型的上下文学习(ICL)和上下文外(OOC)幻觉的研究。我可以将其拆分为几个逻辑部分:
    
    1. 引言/问题:ICL和OOC幻觉
    2. 关键观察与方法
    3. 因果分析与发现
    4. 结论
    
    我可以添加`

    `或`

    `标题,但规则说"保留原文所有信息,不得增删改内容"。添加标题算不算"增"?标题本身是对内容的结构化,通常被认为是排版的一部分,但严格来说,如果我添加的标题文字不是原文的,那就算增。不过用户要求"长文用

    /

    划分小节",这意味着我需要用标题来划分,这是用户明确要求的。所以标题是排版结构的一部分,是允许的。 但我需要小心,标题内容应该是原文内容的概括,或者我可以用原文中的短语作为标题。为了不"增删改",我可以用原文中的关键短语作为小节标题,或者用通用的、描述性的标题。 实际上,用户说"长文用

    /

    划分小节",这暗示我需要主动划分。原文是一个段落,但内部有逻辑结构。我可以将其拆分成几个`

    `,并为每个小节加一个`

    `标题。 但标题如果完全是我新创的词,可能算"增"。不过,标题是排版元素,用户明确要求用`

    /

    `划分,所以这是规则允许的。我会用简洁的、基于原文内容的标题。 让我规划一下结构: - `

    Background and Problem

    ` 或中文?原文是英文,标题用英文还是中文?原文是英文,用户要求"保留原文所有信息",标题作为结构元素,用英文更协调,也避免增删中文。 不过用户是中文指令,但正文是英文。我会用英文标题,基于原文内容。 让我拆分原文: 原文:"Large language models can infer and follow specific rules through in-context learning (ICL), yet may fail to remain faithful to provided evidence, giving rise to out-of-context (OOC) hallucination. Existing work has largely investigated the underlying mechanisms of ICL and OOC hallucination along separate lines. However, this separation obscures the mechanistic basis for these seemingly opposing outcomes." 这是第一部分,介绍问题和背景。 "In this work, we study this contrast through the unified view of functional roles in dynamic context. The key observation is that role information can be internally represented without reliably constraining output. We derive a first-order decomposition to characterize the role effect, showing that its substantial influence depends on two necessary conditions, namely sufficient contribution magnitude and coherent signed aggregation." 这是第二部分,方法。 "Causal analysis further shows that role span contributions affect key token choices involving response structure, content selection, and factual values. We further show that role-consistent choices depend on sufficient role span support at these key decisions, while failing choices can be reversed when this support overcomes the deficit." 这是第三部分,发现。 "These findings support a common mechanistic view of ICL and OOC hallucination: each functional role defines the pattern of output decisions over which role span contributions must remain effective, and differences in this organization account for the seemingly opposing outcomes." 这是第四部分,结论。 我可以为每个部分加`

    `标题,比如: - Background and Problem - Method and Key Observation - Causal Analysis and Findings - Conclusion 但这样标题文字是我加的,算不算"增"?用户规则4说"保留原文所有信息,不得增删改内容"。标题是结构标签,用户明确要求用`

    /

    `划分,所以标题是必要的。但标题的文本内容,如果我是新写的,可能算增。不过,在排版中,小节标题是标准做法,用户要求"划分小节",意味着需要标题。我会用非常简洁的、直接反映原文内容的标题,避免引入新信息。 或者,我可以用原文中的关键短语作为标题,比如"Background"、"Method"、"Findings"、"Conclusion",这些是通用的,不增加新信息。 实际上,考虑到这是一个学术摘要,通常不会有小节标题。但用户要求"长文用

    /

    划分小节",所以我需要划分。我可以用`

    `标签,标题用简短的概括。 另一种方式:不添加标题,只拆分成多个`

    `。但用户明确说"长文用

    /

    划分小节",所以最好添加标题。 我决定使用`

    `来划分,标题用基于原文内容的短语,比如: - `

    Background

    ` - `

    Method and Key Observation

    ` - `

    Causal Analysis

    ` - `

    Conclusion

    ` 这样既划分了小节,又没有引入原文没有的信息,只是对原文逻辑结构的概括。 现在,重要句子用``。哪些是重要句子?比如关键观察、两个必要条件、结论等。 我可以这样: - 第一段中"may fail to remain faithful to provided evidence, giving rise to out-of-context (OOC) hallucination"可能重要,但不要过度。可以加在关键结论或核心发现上。 比如: - "The key observation is that role information can be internally represented without reliably constraining output." 这句是关键观察,可以加``。 - "showing that its substantial influence depends on two necessary conditions, namely sufficient contribution magnitude and coherent signed aggregation." 这两个条件很重要,可以加``,或者把"sufficient contribution magnitude and coherent signed aggregation"加``。 - 结论句可以加``。 注意不要过度使用``,只用于真正重要的句子。 现在构建HTML: ```html

    Background

    Large language models can infer and follow specific rules through in-context learning (ICL), yet may fail to remain faithful to provided evidence, giving rise to out-of-context (OOC) hallucination. Existing work has largely investigated the underlying mechanisms of ICL and OOC hallucination along separate lines. However, this separation obscures the mechanistic basis for these seemingly opposing outcomes.

    Method and Key Observation

    In this work, we study this contrast through the unified view of functional roles in dynamic context. The key observation is that role information can be internally represented without reliably constraining output. We derive a first-order decomposition to characterize the role effect, showing that its substantial influence depends on two necessary conditions, namely sufficient contribution magnitude and coherent signed aggregation.

    Causal Analysis

    Causal analysis further shows that role span contributions affect key token choices involving response structure, content selection, and factual values. We further show that role-consistent choices depend on sufficient role span support at these key decisions, while failing choices can be reversed when this support overcomes the deficit.

    Conclusion

    These findings support a common mechanistic view of ICL and OOC hallucination: each functional role defines the pattern of output decisions over which role span contributions must remain effective, and differences in this organization account for the seemingly opposing outcomes.

    ``` 这看起来不错。标题是通用的,没有增加原文信息。重要句子用了``。 检查规则: 1. 只输出HTML片段,无Markdown围栏,无解释。—— 符合。 2