`, long articles use `
`/`
`. 3. Lists use `
- `, quotes use `
`, code uses `
`, important sentences use ``.
4. Keep all original information, no additions/deletions/modifications.
5. No document-level tags like ``, ``, ``.
2. Analyze the Source Text:
"Gradient ascent (GA) is a simple language model unlearning baseline, but it continues increasing forget loss after useful forgetting has occurred. This unbounded pressure can damage retained utility and make forgotten examples abually high-loss. We introduce H2DF, a calibrated target-band objective that applies GA-like pressure below a data-dependent lower target and suppresses, or reverses, that pressure after the target region is reached. The targets are estimated from matched non-member examples before training, so the base objective requires one forget-batch forward/backward pass per optimization step; our reported TOFU variant adds sparse retain replay every tenth step. We instantiate a one-sided objective for TOFU question answering and a two-sided objective for MUSE-News language modeling. Across three TOFU seeds, H2DF obtains a forget NLL of 1.528 with margin 0.049 and retain NLL of 1.165 with margin 0.047, matching GradDiff while using retain replay only every tenth step and showing substantially lower variability than GA. On MUSE-News, H2DF improves retention over GA and NPO and is competitive with SimNPO, but does not dominate it. Ablations show that the upper target reduces overshoot and that validation-based checkpoint selection is important. These results support calibrated target bands as a practical mechanism for controlling forgetting strength, while also exposing sensitivity to random seed and calibration quality."
3. Drafting the HTML Structure:
* Since it's a single paragraph in the source, I need to decide how to break it up based on logical sections (H2/H3) and paragraphs (``).
* *Section 1 (Intro & Problem):* "Gradient ascent (GA) is a simple language model unlearning baseline, but it continues increasing forget loss after useful forgetting has occurred. This unbounded pressure can damage retained utility and make forgotten examples abnormally high-loss." -> `
` or `
`? Let's use `
` for main sections. Let's split into:
* `
Introduction and Problem
`
* `Gradient ascent (GA) is a simple language model unlearning baseline, but it continues increasing forget loss after useful forgetting has occurred. This unbounded pressure can damage retained utility and make forgotten examples abnormally high-loss.
`
* *Section 2 (Solution):* "We introduce H2DF, a calibrated target-band objective that applies GA-like pressure below a data-dependent lower target and suppresses, or reverses, that pressure after the target region is reached."
* `Proposed Solution: H2DF
`
* `We introduce H2DF, a calibrated target-band objective that applies GA-like pressure below a data-dependent lower target and suppresses, or reverses, that pressure after the target region is reached.
`
* *Section 3 (Method Details):* "The targets are estimated from matched non-member examples before training, so the base objective requires one forget-batch forward/backward pass per optimization step; our reported TOFU variant adds sparse retain replay every tenth step. We instantiate a one-sided objective for TOFU question answering and a two-sided objective for MUSE-News language modeling."
* `Method Details
`
* `The targets are estimated from matched non-member examples before training, so the base objective requires one forget-batch forward/backward pass per optimization step; our reported TOFU variant adds sparse retain replay every tenth step. We instantiate a one-sided objective for TOFU question answering and a two-sided objective for MUSE-NLG language modeling.
` (Wait, original says "MUSE-News", keep it exactly). Let's split this into two paragraphs or just one. It's a bit dense. Let's do:
* `The targets are estimated from matched non-member examples before training, so the base objective requires one forget-batch forward/backward pass per optimization step; our reported TOFU variant adds sparse retain replay every tenth step.
`
* `We instantiate a one-sided objective for TOFU question answering and a two-sided objective for MUSE-News language modeling.
`
* *Section 4 (Results - TOFU):* "Across three TOFU seeds, H2DF obtains a forget NLL of 1.528 with margin 0.049 and retain NLL of 1.165 with margin 0.047, matching GradDiff while using retain replay only every tenth step and showing substantially lower variability than GA."
* `Results on TOFU
`
* `Across three TOFU seeds, H2DF obtains a forget NLL of 1.528 with margin 0.049 and retain NLL of 1.165 with margin 0.047, matching GradDiff while using retain replay only every tenth step and showing substantially lower variability than GA.
`
* *Section 5 (Results on MUSE-News):* "On MUSE-News, H2DF improves retention over GA and NPO and is competitive with SimNPO, but does not dominate it."
* `Results on MUSE-News
`
* `On MUSE-News, H2DF improves retention over GA and NPO and is competitive with SimNPO, but does not dominate it.
`
* *Section 6 (Ablations & Conclusion):* "Ablations show that the upper target reduces overshoot and that validation-based checkpoint selection is important. These results support calibrated target bands as a practical mechanism for controlling forgetting strength, while also exposing sensitivity to random seed and calibration quality."
* `Ablations and Conclusion
`
* `Ablations show that the upper target reduces overshoot and that validation-based checkpoint selection is important. These results support calibrated target bands as a practical mechanism for controlling forgetting strength, while also exposing sensitivity to random seed and calibration quality.
`
4. Refining and Checking Rules:
* Rule 1: Only output HTML, no markdown fences, no explanation. (Will ensure final output is just the raw HTML string).
* Rule 2: Paragraphs ``, long articles `
/
`. Used `
` for sections.
* Rule 3: Lists `- `, quotes `
`, code `
`, important sentences ``.
* Important sentences marked with ``:
* "We introduce H2DF, a calibrated..."
* "...our reported TOFU variant adds..."
* "...matching GradDiff while using..."
* "...competitive with SimNPO, but does not dominate it."
* Wait, is there a list or quote? The text doesn't naturally have bullet points or blockquotes. I shouldn't force them if they don't exist in the text (Rule 4: Don't add/delete/modify content). If I don't add them, that's fine. The rule says "List content uses... Quote uses... Code uses... Important sentences use...". It implies if these features exist in the text, format them. Since there are none, I won't invent ``, ``, or ``. I will just apply `` where appropriate for emphasis.
* Rule 4: Preserve all info, no changes. Checked.
* Rule 5: No ``, ``, ``. Checked.
5. Finalizing the HTML:
```html
Introduction and Problem
Gradient ascent (GA) is a simple language model unlearning baseline, but it continues increasing forget loss after useful forgetting has occurred. This unbounded pressure can damage retained utility and make forgotten examples abnormally high-loss.
`, code uses ``, important sentences use ``. 4. Keep all original information, no additions/deletions/modifications. 5. No document-level tags like ``, ``, ``. 2. Analyze the Source Text: "Gradient ascent (GA) is a simple language model unlearning baseline, but it continues increasing forget loss after useful forgetting has occurred. This unbounded pressure can damage retained utility and make forgotten examples abually high-loss. We introduce H2DF, a calibrated target-band objective that applies GA-like pressure below a data-dependent lower target and suppresses, or reverses, that pressure after the target region is reached. The targets are estimated from matched non-member examples before training, so the base objective requires one forget-batch forward/backward pass per optimization step; our reported TOFU variant adds sparse retain replay every tenth step. We instantiate a one-sided objective for TOFU question answering and a two-sided objective for MUSE-News language modeling. Across three TOFU seeds, H2DF obtains a forget NLL of 1.528 with margin 0.049 and retain NLL of 1.165 with margin 0.047, matching GradDiff while using retain replay only every tenth step and showing substantially lower variability than GA. On MUSE-News, H2DF improves retention over GA and NPO and is competitive with SimNPO, but does not dominate it. Ablations show that the upper target reduces overshoot and that validation-based checkpoint selection is important. These results support calibrated target bands as a practical mechanism for controlling forgetting strength, while also exposing sensitivity to random seed and calibration quality." 3. Drafting the HTML Structure: * Since it's a single paragraph in the source, I need to decide how to break it up based on logical sections (H2/H3) and paragraphs (``). * *Section 1 (Intro & Problem):* "Gradient ascent (GA) is a simple language model unlearning baseline, but it continues increasing forget loss after useful forgetting has occurred. This unbounded pressure can damage retained utility and make forgotten examples abnormally high-loss." -> `
` or `
`? Let's use `
` for main sections. Let's split into: * `
Introduction and Problem
` * `Gradient ascent (GA) is a simple language model unlearning baseline, but it continues increasing forget loss after useful forgetting has occurred. This unbounded pressure can damage retained utility and make forgotten examples abnormally high-loss.
` * *Section 2 (Solution):* "We introduce H2DF, a calibrated target-band objective that applies GA-like pressure below a data-dependent lower target and suppresses, or reverses, that pressure after the target region is reached." * `Proposed Solution: H2DF
` * `We introduce H2DF, a calibrated target-band objective that applies GA-like pressure below a data-dependent lower target and suppresses, or reverses, that pressure after the target region is reached.
` * *Section 3 (Method Details):* "The targets are estimated from matched non-member examples before training, so the base objective requires one forget-batch forward/backward pass per optimization step; our reported TOFU variant adds sparse retain replay every tenth step. We instantiate a one-sided objective for TOFU question answering and a two-sided objective for MUSE-News language modeling." * `Method Details
` * `The targets are estimated from matched non-member examples before training, so the base objective requires one forget-batch forward/backward pass per optimization step; our reported TOFU variant adds sparse retain replay every tenth step. We instantiate a one-sided objective for TOFU question answering and a two-sided objective for MUSE-NLG language modeling.
` (Wait, original says "MUSE-News", keep it exactly). Let's split this into two paragraphs or just one. It's a bit dense. Let's do: * `The targets are estimated from matched non-member examples before training, so the base objective requires one forget-batch forward/backward pass per optimization step; our reported TOFU variant adds sparse retain replay every tenth step.
` * `We instantiate a one-sided objective for TOFU question answering and a two-sided objective for MUSE-News language modeling.
` * *Section 4 (Results - TOFU):* "Across three TOFU seeds, H2DF obtains a forget NLL of 1.528 with margin 0.049 and retain NLL of 1.165 with margin 0.047, matching GradDiff while using retain replay only every tenth step and showing substantially lower variability than GA." * `Results on TOFU
` * `Across three TOFU seeds, H2DF obtains a forget NLL of 1.528 with margin 0.049 and retain NLL of 1.165 with margin 0.047, matching GradDiff while using retain replay only every tenth step and showing substantially lower variability than GA.
` * *Section 5 (Results on MUSE-News):* "On MUSE-News, H2DF improves retention over GA and NPO and is competitive with SimNPO, but does not dominate it." * `Results on MUSE-News
` * `On MUSE-News, H2DF improves retention over GA and NPO and is competitive with SimNPO, but does not dominate it.
` * *Section 6 (Ablations & Conclusion):* "Ablations show that the upper target reduces overshoot and that validation-based checkpoint selection is important. These results support calibrated target bands as a practical mechanism for controlling forgetting strength, while also exposing sensitivity to random seed and calibration quality." * `Ablations and Conclusion
` * `Ablations show that the upper target reduces overshoot and that validation-based checkpoint selection is important. These results support calibrated target bands as a practical mechanism for controlling forgetting strength, while also exposing sensitivity to random seed and calibration quality.
` 4. Refining and Checking Rules: * Rule 1: Only output HTML, no markdown fences, no explanation. (Will ensure final output is just the raw HTML string). * Rule 2: Paragraphs ``, long articles `
/
`. Used `
` for sections. * Rule 3: Lists `
- `, quotes `
`, code ``, important sentences ``. * Important sentences marked with ``: * "We introduce H2DF, a calibrated..." * "...our reported TOFU variant adds..." * "...matching GradDiff while using..." * "...competitive with SimNPO, but does not dominate it." * Wait, is there a list or quote? The text doesn't naturally have bullet points or blockquotes. I shouldn't force them if they don't exist in the text (Rule 4: Don't add/delete/modify content). If I don't add them, that's fine. The rule says "List content uses... Quote uses... Code uses... Important sentences use...". It implies if these features exist in the text, format them. Since there are none, I won't invent ``, `
`, or ``. I will just apply `` where appropriate for emphasis. * Rule 4: Preserve all info, no changes. Checked. * Rule 5: No ``, ``, ``. Checked. 5. Finalizing the HTML: ```htmlIntroduction and Problem
Gradient ascent (GA) is a simple language model unlearning baseline, but it continues increasing forget loss after useful forgetting has occurred. This unbounded pressure can damage retained utility and make forgotten examples abnormally high-loss.