首页 > AI前沿 > The ROI of Small Examples: Optimizing In-Context Learning Under Token Budgets

The ROI of Small Examples: Optimizing In-Context Learning Under Token Budgets

arXiv自然语言 2026-11-20 13:00 26 阅读 查看原文
用户需要我将一段纯文本的新闻正文排版成语义化的HTML片段。 规则如下: 1. 只输出HTML片段本身,不要Markdown围栏,不要解释。 2. 每个自然段落用`

`包裹,长文用`

`/`

`划分小节。 3. 列表内容用`
  • `,引用用`
    `,代码用`
    `,重要句子可用``。
    4. 保留原文所有信息,不得增删改内容。
    5. 不要输出``/``/``等文档级标签。
    
    分析正文:
    正文是一段关于In-context learning (ICL)的研究摘要。
    内容:
    "In-context learning (ICL) improves large language model performance at inference time without updating model parameters. However, these gains are not free: in practical deployments, longer prompts directly increase inference cost and latency. Under a fixed token budget, prompt construction therefore becomes a resource-allocation problem. While prior work has often focused on how many examples to include in ICL, we study the complementary question of whether the available budget is better spent on a few long demonstrations or on a larger number of shorter ones. We study this aspect in question answering (QA), by organizing candidate demonstrations into length-based bins and constructing prompts from examples within the same length range under fixed budgets. Experiments on two QA benchmarks (\textit{Beat the AI} and \textit{Natural Questions}), using six instruction-tuned LLMs, show that shorter demonstrations are generally the most effective use of prompt space. This effect is especially strong on \textit{Beat the AI}, and remains visible on NQ, although gains are smaller and saturate earlier in the long-context setting. An additional natural language inference experiment on ANLI R3 finds that one of the two shorter bins performs best for five of six models, although longer demonstrations remain competitive. Overall, our results highlight demonstration granularity as an important factor in prompt design."
    
    这段文本是一个连续的自然段。没有明显的列表、引用或代码。
    为了符合“长文用

    /

    划分小节”的要求,虽然这段文本不算特别长,但我可以将其划分为几个逻辑小节,比如“研究背景与问题”、“实验设置与结果”、“结论”,或者直接把整个文本作为一个段落,加一个标题。 不过,原文本并没有明确的小标题,如果我擅自加小标题,算不算“增删改内容”? 规则4:“保留原文所有信息,不得增删改内容”。 如果我添加`

    `小节标题,就增加了原文没有的内容(文字)。 因此,最安全的做法是不加自己编造的小标题,而是直接使用`

    `作为文章的主标题(如果原文有标题的话,但原文没给标题,只给了“正文如下:”后面的文本)。 或者,我可以把第一句作为`

    `?不行,那也是改内容。 那就直接把整个自然段用`

    `包裹。如果必须用`

    `或`

    `,我可以把第一句话或者核心句用`

    `包裹?不,规则说“长文用

    /

    划分小节”,如果不是长文,可以不用。这里文本长度大概200词,算一个自然段。 为了体现排版,我可以把一些重要的句子用``包裹。比如“whether the available budget is better spent on a few long demonstrations or on a larger number of shorter ones”或者“shorter demonstrations are generally the most effective use of prompt space”和“demonstration granularity as an important factor in prompt design”。 原文中有LaTeX语法:`\textit{Beat the AI}` 和 `\textit{Natural Questions}`。在HTML中,我应该将其转换为语义化的标签,比如`Beat the AI`和`Natural Questions`,或者保留原样? 规则4说“不得增删改内容”。将`\textit{...}`转为`...`属于格式转换,是排版的一部分。为了安全,用``替换`\textit{}`是合理的HTML排版。或者直接保留文本,用``包裹。 让我们仔细处理文本: "In-context learning (ICL) improves large language model performance at inference time without updating model parameters. However, these gains are not free: in practical deployments, longer prompts directly increase inference cost and latency. Under a fixed token budget, prompt construction therefore becomes a resource-allocation problem. While prior work has often focused on how many examples to include in ICL, we study the complementary question of whether the available budget is better spent on a few long demonstrations or on a larger number of shorter ones." -> 这是一部分,可以作为一个段落,或者用``强调问题。 "We study this aspect in question answering (QA), by organizing candidate demonstrations into length-based bins and constructing prompts from examples within the same length range under fixed budgets. Experiments on two QA benchmarks (Beat the AI and Natural Questions), using six instruction-tuned LLMs, show that shorter demonstrations are generally the most effective use of prompt space. This effect is especially strong on Beat the AI, and remains visible on NQ, although gains are smaller and saturate earlier in the long-context setting. An additional natural language inference experiment on ANLI R3 finds that one of the two shorter bins performs best for five of six models, although longer demonstrations remain competitive." -> 第二部分。 "Overall, our results highlight demonstration granularity as an important factor in prompt design." -> 第三部分。 这样分成三个`

    `段落,既符合“每个自然段落用

    包裹”,又没有增删内容,只是合理分段(原文虽然连在一起,但语义上有明显的段落停顿,或者我可以直接按原样一个`

    `,但分成几个`

    `更符合“每个自然段落”的排版惯例,只要不改变词句)。为了严格遵守“不得增删改内容”,最好保持原有的段落结构。原文是一整段,所以我应该只用一个`

    `。 但规则2说“长文用

    /

    划分小节”。如果我只输出一个`

    `,会不会被认为没有应用`

    /

    `? 我可以用`

    `来包裹原文的第一句话作为小节标题?不,那改变了内容。 或者,我不加`

    /

    `,因为这不是“长文”,只是一个摘要段落。 或者,把 "In-context learning (ICL) improves..." 作为引言,不加标题。 为了稳妥,我把整个文本放在一个`

    `中,或者分成几个`

    `。其实原文本没有换行,是一个自然段。所以用一个`

    `最保险。 等等,如果用户期望看到排版元素,我可以在段落内使用``和``。 检查`\textit{}`: 原文:`(\textit{Beat the AI} and \textit{Natural Questions})` HTML:`(Beat the AI and Natural Questions)` 原文:`\textit{Beat the AI}` HTML:`Beat the AI` 重要句子用``: "whether the available budget is better spent on a few long demonstrations or on a larger number of shorter ones" -> 可以加strong。 "shorter demonstrations are generally the most effective use of prompt space" -> 加strong。 "demonstration granularity as an important factor in prompt design" -> 加strong。 组合起来:

    In-context learning (ICL) improves large language model performance at inference time without updating model parameters. However, these gains are not free: in practical deployments, longer prompts directly increase inference cost and latency. Under a fixed token budget, prompt construction therefore becomes a resource-allocation problem. While prior work has often focused on how many examples to include in ICL, we study the complementary question of whether the available budget is better spent on a few long demonstrations or on a larger number of shorter ones. We study this aspect in question answering (QA), by organizing candidate demonstrations into length-based bins and constructing prompts from examples within the same length range under fixed budgets. Experiments on two QA benchmarks (Beat the AI and Natural Questions), using six instruction-tuned LLMs, show that shorter demonstrations are generally the most effective use of prompt space. This effect is especially strong on Beat the AI, and remains