` 包裹；长文用 `
`/`
` 划分小节（虽然当前文本不长，但为了体现排版，可以考虑在关键转折处或者标题处用一下，不过这篇新闻看起来就是一段完整的报道，没有明显的分段，那我需要人工切分段落）；列表用 `
- `（如果存在）；引用用 `
`（如果存在）；代码用 `
`（如果存在）；重要句子用 ``。
* 内容保全：严格保留原文所有信息，绝对不能增删改任何内容（零容错）。
2. 分析原文内容与结构：
* 原文是一整段关于强化学习（RLLF）的新闻/摘要。
* 为了使用 `` 标签，我需要根据逻辑或句子群将其切分为几个自然段。
* 原文：“In reinforcement learning from limited feedback (RLLF), only a small fraction of an offline dataset can be labeled with rewards, and the central question is which samples should be labeled to learn a high-performing policy from the resulting partially labeled dataset. Prior work formalized this as a reward-selection problem by focusing on the selection stage while treating downstream policy learning as a black box, in a regime where queried rewards are not retained for reward-model training.” —— 这是背景和以往工作的介绍。可以作为第一段。
* 原文：“We instead study the retained-label setting, where queried rewards can be stored and used to fit a reward model before policy learning. We bound the suboptimality of the learned policy in terms of two sources of error: one arising from offline RL on a fixed, offline dataset, and the other from uncertainty in the reward model. Since reward selection cannot change the offline dataset, the limited labeling budget must be used to strategically reduce reward uncertainty.” —— 引入本文的方法和理论边界。可以作为第二段。
* 原文：“Motivated by RLLF's observation that useful rewards tend to keep the agent on high-return trajectories, we introduce Successor-Guided Uncertainty REduction (SURE), a novel method that uses successor features to select rewards that are both uncertain and associated with states from which high-valued states are reachable. We formally derive SURE from a bound-induced design objective and characterize its exact one-step marginal gain.” —— 介绍 SURE 方法。可以作为第三段，这里可以用 `
` 加个小标题或者直接作为段落。考虑到题目说“长文用
/
划分小节”，这篇新闻整体不算特别长，但为了体现排版技能，可以在引言（背景）之后加一个 `
` 或 `
` 来引出方法介绍。或者按常规新闻排版，不需要子标题，直接分段落。
* 原文：“Empirically, SURE reaches near full-feedback performance with few reward labels across a variety of domains and offline-RL backbones, yielding an effective method for feedback-efficient reinforcement learning.” —— 实验结果与结论。可以作为第四段。
让我们重新审视切分，确保原词原句不被破坏（连空格都不改）：
段落1：In reinforcement learning from limited feedback (RLLF), only a small fraction of an offline dataset can be labeled with rewards, and the central question is which samples should be labeled to learn a high-performing policy from the resulting partially labeled dataset. Prior work formalized this as a reward-selection problem by focusing on the selection stage while treating downstream policy learning as a black box, in a regime where queried rewards are not retained for reward-model training.
段落2：We instead study the retained-label setting, where queried rewards can be stored and used to fit a reward model before policy learning. We bound the suboptimality of the learned policy in terms of two sources of error: one arising from offline RL on a fixed, offline dataset, and the other from uncertainty in the reward model. Since reward selection cannot change the offline dataset, the limited labeling budget must be used to strategically reduce reward uncertainty.
段落3：Motivated by RLLF's observation that useful rewards tend to keep the agent on high-return trajectories, we introduce Successor-Guided Uncertainty REduction (SURE), a novel method that uses successor features to select rewards that are both uncertain and associated with states from which high-valued states are reachable. We formally derive SURE from a bound-induced design objective and characterize its exact one-step marginal gain.
段落4：Empirically, SURE reaches near full-feedback performance with few reward labels across a variety of domains and offline-RL backbones, yielding an effective method for feedback-efficient reinforcement learning.
3. 应用HTML标签规则：
* 段落1 -> `
...
`
* 段落2 -> `...
`
* 关于 `
`（如果存在）；代码用 ``（如果存在）；重要句子用 ``。 * 内容保全：严格保留原文所有信息，绝对不能增删改任何内容（零容错）。 2. 分析原文内容与结构： * 原文是一整段关于强化学习（RLLF）的新闻/摘要。 * 为了使用 `` 标签，我需要根据逻辑或句子群将其切分为几个自然段。 * 原文：“In reinforcement learning from limited feedback (RLLF), only a small fraction of an offline dataset can be labeled with rewards, and the central question is which samples should be labeled to learn a high-performing policy from the resulting partially labeled dataset. Prior work formalized this as a reward-selection problem by focusing on the selection stage while treating downstream policy learning as a black box, in a regime where queried rewards are not retained for reward-model training.” —— 这是背景和以往工作的介绍。可以作为第一段。 * 原文：“We instead study the retained-label setting, where queried rewards can be stored and used to fit a reward model before policy learning. We bound the suboptimality of the learned policy in terms of two sources of error: one arising from offline RL on a fixed, offline dataset, and the other from uncertainty in the reward model. Since reward selection cannot change the offline dataset, the limited labeling budget must be used to strategically reduce reward uncertainty.” —— 引入本文的方法和理论边界。可以作为第二段。 * 原文：“Motivated by RLLF's observation that useful rewards tend to keep the agent on high-return trajectories, we introduce Successor-Guided Uncertainty REduction (SURE), a novel method that uses successor features to select rewards that are both uncertain and associated with states from which high-valued states are reachable. We formally derive SURE from a bound-induced design objective and characterize its exact one-step marginal gain.” —— 介绍 SURE 方法。可以作为第三段，这里可以用 `
` 加个小标题或者直接作为段落。考虑到题目说“长文用
/
划分小节”，这篇新闻整体不算特别长，但为了体现排版技能，可以在引言（背景）之后加一个 `
` 或 `
` 来引出方法介绍。或者按常规新闻排版，不需要子标题，直接分段落。 * 原文：“Empirically, SURE reaches near full-feedback performance with few reward labels across a variety of domains and offline-RL backbones, yielding an effective method for feedback-efficient reinforcement learning.” —— 实验结果与结论。可以作为第四段。 让我们重新审视切分，确保原词原句不被破坏（连空格都不改）： 段落1：In reinforcement learning from limited feedback (RLLF), only a small fraction of an offline dataset can be labeled with rewards, and the central question is which samples should be labeled to learn a high-performing policy from the resulting partially labeled dataset. Prior work formalized this as a reward-selection problem by focusing on the selection stage while treating downstream policy learning as a black box, in a regime where queried rewards are not retained for reward-model training. 段落2：We instead study the retained-label setting, where queried rewards can be stored and used to fit a reward model before policy learning. We bound the suboptimality of the learned policy in terms of two sources of error: one arising from offline RL on a fixed, offline dataset, and the other from uncertainty in the reward model. Since reward selection cannot change the offline dataset, the limited labeling budget must be used to strategically reduce reward uncertainty. 段落3：Motivated by RLLF's observation that useful rewards tend to keep the agent on high-return trajectories, we introduce Successor-Guided Uncertainty REduction (SURE), a novel method that uses successor features to select rewards that are both uncertain and associated with states from which high-valued states are reachable. We formally derive SURE from a bound-induced design objective and characterize its exact one-step marginal gain. 段落4：Empirically, SURE reaches near full-feedback performance with few reward labels across a variety of domains and offline-RL backbones, yielding an effective method for feedback-efficient reinforcement learning. 3. 应用HTML标签规则： * 段落1 -> `
...
` * 段落2 -> `...
` * 关于 `
/
` 划分：原文没有标题，我要不要硬加标题？规则说“长文用
/
划分小节”，这篇新闻很短，如果我不加标题，可能无法体现这个规则。如果加上标题就改变了内容（“不得增删改内容”）。因此，不加伪造的标题是最安全的做法，避免违反“不得增删改内容”。但我可以把重点概念用 `` 加粗，比如“Successor-Guided Uncertainty REduction (SURE)”。
* 检查重要句子：
* "we introduce Successor-Guided Uncertainty REduction (SURE)" -> 用 `` 包裹。
* "Empirically, SURE reaches near full-feedback performance..." 这句也是重点，可以加粗。
* 由于原文没有列表、引用、代码，所以不需要 `
`, `
`, `
`。
4. 细化HTML结构：
```html
In reinforcement learning from limited feedback (RLLF), only a small fraction of an offline dataset can be labeled with rewards, and the central question is which samples should be labeled to learn a high-performing policy from the resulting partially labeled dataset. Prior work formalized this as a reward-selection problem by focusing on the selection stage while treating downstream policy learning as a black box, in a regime where queried rewards are not retained for reward-model training.
We instead study the retained-label setting, where queried rewards can be stored and used to fit a reward model before policy learning. We bound the suboptimality of the learned policy in terms of two sources of error: one arising from offline RL on a fixed, offline dataset, and the other from uncertainty in the reward model. Since reward selection cannot change the offline dataset, the limited labeling budget must be used to strategically reduce reward uncertainty.
Motivated by RLLF's observation that useful rewards tend to keep the agent on high-return trajectories, we introduce Successor-Guided Uncertainty REduction (SURE), a novel method that uses successor features to select rewards that are both uncertain and associated with states from which high-valued states are reachable. We formally derive SURE from a bound-induced design objective and characterize its exact one-step marginal gain.
Empirically, SURE reaches near full-feedback performance with few reward labels across a variety of domains and offline-RL backbones, yielding an effective method for feedback-efficient reinforcement learning.
```
这样分了4个段落，每个段落用 `` 包裹，重点词汇和总结句用 `` 强调。由于原文篇幅较短，强行加入 `
`/`` 会违背“不得增删改内容”的硬性规定。因此不加入小标题是更合理的做法。
5. 最后检查约束条件：
* 只输出HTML片段本身？是。
* 不要Markdown围栏？是。
* 不要解释？是。
* 每个自然段落用`
`包裹？是。
* 长文用`
`, ``。 4. 细化HTML结构： ```htmlIn reinforcement learning from limited feedback (RLLF), only a small fraction of an offline dataset can be labeled with rewards, and the central question is which samples should be labeled to learn a high-performing policy from the resulting partially labeled dataset. Prior work formalized this as a reward-selection problem by focusing on the selection stage while treating downstream policy learning as a black box, in a regime where queried rewards are not retained for reward-model training.
We instead study the retained-label setting, where queried rewards can be stored and used to fit a reward model before policy learning. We bound the suboptimality of the learned policy in terms of two sources of error: one arising from offline RL on a fixed, offline dataset, and the other from uncertainty in the reward model. Since reward selection cannot change the offline dataset, the limited labeling budget must be used to strategically reduce reward uncertainty.
Motivated by RLLF's observation that useful rewards tend to keep the agent on high-return trajectories, we introduce Successor-Guided Uncertainty REduction (SURE), a novel method that uses successor features to select rewards that are both uncertain and associated with states from which high-valued states are reachable. We formally derive SURE from a bound-induced design objective and characterize its exact one-step marginal gain.
Empirically, SURE reaches near full-feedback performance with few reward labels across a variety of domains and offline-RL backbones, yielding an effective method for feedback-efficient reinforcement learning.
``` 这样分了4个段落，每个段落用 `` 包裹，重点词汇和总结句用 `` 强调。由于原文篇幅较短，强行加入 `
`/`
` 会违背“不得增删改内容”的硬性规定。因此不加入小标题是更合理的做法。 5. 最后检查约束条件： * 只输出HTML片段本身？是。 * 不要Markdown围栏？是。 * 不要解释？是。 * 每个自然段落用`
`包裹？是。 * 长文用`