首页 > AI前沿 > OpticalRAG: Pixel-Space Compression for Token-Efficient Retrieval-Augmented Generation

OpticalRAG: Pixel-Space Compression for Token-Efficient Retrieval-Augmented Generation

arXiv自然语言 2026-11-10 15:24 28 阅读 查看原文

Retrieval-augmented generation (RAG) is bottlenecked by the token budget of large language models: feeding long documents into the context window is expensive, while aggressive textual compression discards fine-grained information that is irreversibly lost in a one-dimensional token sequence. We argue that this bottleneck is not fundamental to information density but to the choice of compression modality. Modern visual encoders can pack the contents of a rendered text page into roughly a hundred visual tokens with little semantic loss, opening a new compression-fidelity tradeoff that purely textual methods cannot reach.

We present OpticalRAG, a framework that renders documents as page images, encodes them with a frozen vision encoder, and distills the retrieved visual tokens into as few as $16$ injected tokens before passing them to the LLM.

Two ingredients make this work without expensive cross-modal training:

  • Encoder-Bridged Retrieval, which projects text-query states into the visual feature space of the frozen encoder, through a projection shared with the compressor, to retrieve visual tokens from a text query, with no contrastive training;
  • Query-Driven Distillation, a lightweight cross-attention compressor that condenses the retrieved visual tokens into a small set of query-conditioned prefix tokens.

Encoder-Bridged Retrieval

This approach projects text-query states into the visual feature space of the frozen encoder, through a projection shared with the compressor, to retrieve visual tokens from a text query, with no contrastive training.

Query-Driven Distillation

A lightweight cross-attention compressor condenses the retrieved visual tokens into a small set of query-conditioned prefix tokens.

Benchmark results

On LongBench-v2, BAMBOO, and LooGLE-v2, OpticalRAG achieves the best compression-aware accuracy (CAP) on all three benchmarks while injecting only $16$ tokens, and improves the closed-book Qwen2.5-7B-Instruct backbone by $5.17$ points on LongBench-v2.

Code is available at https://github.com/brighterr/OpticalRAG.