首页 > AI前沿 > LadderEdit: Edit-Level Residual Compression for Memory-Efficient Lifelong Editing of LLMs

LadderEdit: Edit-Level Residual Compression for Memory-Efficient Lifelong Editing of LLMs

arXiv自然语言 2026-10-08 11:20 6 阅读 查看原文

Lifelong editing of LLMs requires storing thousands of edits after acquisition.

A widely used family of approaches attaches one LoRA adapter per edit, which preserves behavior but grows linearly in storage.

To address this challenge, we propose LadderEdit, a method that compresses each LoRA adapter after it is acquired.

Each edit is first stored at low rank as a cheap sketch.

We then check whether this sketch still satisfies the rewrite, generalization, and locality contract on probe prompts.

Edits that pass keep the sketch; those that fail are promoted to a higher rank along a ladder until the contract is met.

Because every edit retains some representation, coverage is maintained, and only hard edits consume more rank.

Across ZsRE, CounterFact, and WikiBigEdit benchmarks on LLaMA-3-8B, Mistral-7B, and Qwen2.5-7B, LadderEdit tracks exact LoRA storage at 5.2x less memory and remains effective at 50,000 sequential edits.