A Language-Informed Loss for Visual Generation

1. Analyze the Request: * Role: Professional news layout engineer. * Task: Convert plain text news body into a semantic HTML fragment. * Rules: 1. Output *only* the HTML fragment (no Markdown fences, no explanations). 2. Paragraphs wrapped in `

`, long text divided into `

`/`

` sections. 3. Lists use `

• `, quotes use `

  `, code uses `

  `, important sentences use ``.
          4.  Preserve all original information (no addition/deletion/modification).
          5.  No document-level tags (``, ``, ``).
      *   Input Text: A short abstract-like paragraph about a paper called "LILIE".
  
  2.  Analyze the Input Text:
      "End-to-end visual generation with pixel-space loss has recently achieved competitive performance without the need for learned image tokenizers. In this work, we study whether language-informed supervision can provide a richer training signal than pixel loss for visual generation. We introduce LILIE (pronounced "lily"), a Language-Informed and Lossless Image Encoder. LILIE is an invertible image transform trained such that a vision-language model can recover captions from noised features. The encoder uses a normalizing flow transformer trained jointly with a vision-language model via next token prediction. We then use the encoder to extract a simple language-informed loss for text-to-image generation. We train JiT models with LILIE loss, reducing gFD by 14 points (66.84 $\to$ 53.01) against a pixel-only baseline on the GPIC dataset. Our code, encoders, and generative models are available at https://www.arxiv.org/abs/2410.03587." (Wait, the input says https://www.arxiv.org/abs/2410.03587, I must keep exactly what's given, even if it looks like a standard arxiv link, I shouldn't correct it based on external knowledge unless instructed. Actually, the prompt says "保留原文所有信息，不得增删改内容", so I must copy the URL exactly as provided: `https://www.arxiv.org/abs/2410.03587` - wait, the prompt has `https://www.arxiv.org/abs/2410.03587`. Let me double check the exact string: `https://www.arxiv.org/abs/2410.03587`. Yes.)
  
  3.  Drafting the HTML Structure:
      *   The text is a single paragraph in the source, but it contains distinct logical sections/sentences.
      *   Rule 2 says "长文用
  /
  划分小节" (long text divided into sections with h2/h3). Since it's a short abstract, I can structure it into logical paragraphs or sections. Let's break it down into a few `
  ` tags or an `
  ` for the main contribution and `
  ` for the results.
      *   Let's create a section for the problem statement, one for the method introduction, and one for the results. Or just split into 3 `
  ` elements, or 1 `
  ` and an `
  `. The rule says "每个自然段落用
  包裹，长文用
  /
  划分小节". A single paragraph might be considered "长文" if it's dense, so adding an `
  ` for the method section makes sense.
      *   Let's structure it like this:
          *   `
  `: First two sentences (Problem & Goal).
          *   `
  `: Method introduction (LILIE definition).
          *   `
  `: Encoder mechanism.
          *   `
  `: Results.
      *   Wait, is it better to just output multiple `
  ` tags? The prompt says "长文用
  /
  划分小节". A short abstract is technically a long text in the context of the rules if we want to show semantic formatting. Let's make it 3 parts:
          *   `
  End-to-end visual generation with pixel-space loss has recently achieved competitive performance without the need for learned image tokenizers. In this work, we study whether language-informed supervision can provide a richer training signal than pixel loss for visual generation.
  `
          *   `
  Method
  `
          *   `
  We introduce LILIE (pronounced "lily"), a Language-Informed and Lossless Image Encoder. LILIE is an invertible image transform trained such that a vision-language model can recover captions from noised features.
  `
          *   `
  The encoder uses a normalizing flow transformer trained jointly with a vision-language model via next token prediction. We then use the encoder to extract a simple language-informed loss for text-to-image generation.
  `
          *   `
  Results
  `
          *   `
  We train JiT models with LILIE loss, reducing gFD by 14 points (66.84 $\to$ 53.01) against a pixel-only baseline on the GPIC dataset. Our code, encoders, and generative models are available at https://www.arxiv.org/abs/2410.03587.
  `
      *   Let's refine the semantic tags. "重要句子可用" (important sentences can use strong). Which sentences are important?
          *   "We introduce LILIE (pronounced "lily"), a Language-Informed and Lossless Image Encoder." -> Important.
          *   "reducing gFD by 14 points (66.84 $\to$ 53.01)" -> Important metric.
      *   Let's format carefully to ensure no content is added or deleted.
  
  4.  Refining the HTML:
      ```html
      
  End-to-end visual generation with pixel-space loss has recently achieved competitive performance without the need for learned image tokenizers. In this work, we study whether language-informed supervision can provide a richer training signal than pixel loss for visual generation.
      
  Methodology
      
  We introduce LILIE (pronounced "lily"), a Language-Informed and Lossless Image Encoder. LILIE is an invertible image transform trained such that a vision-language model can recover captions from noised features.
      
  The encoder uses a normalizing flow transformer trained jointly with a vision-language model via next token prediction. We then use the encoder to extract a simple language-informed loss for text-to-image generation.
      
  Results
      
  We train JiT models with LILIE loss, reducing gFD by 14 points (66.84 → 53.01) against a pixel-only baseline on the GPIC dataset. Our code, encoders, and generative models are available at https://www.arxiv.org/abs/2410.03587.
      ```
      Wait, the arrow in the original text is `→`. I should preserve it exactly as `→` or `->`? The original text has `$\to$`. In HTML, `→` is fine. Let's stick strictly to the characters provided: `66.84 $\to$ 53.01`. I will write `66.84 $\rightarrow$ 53.01` or just `66.84 $\to$ 53.01`. Let's use `→` directly since it's the same character, but keeping `$\to$` inside `` is also fine. Let's just use `→` for cleaner HTML, or keep `$\to$` to be safe. Actually, the prompt has `$\to$`. I will write `66.84 $\rightarrow$ 53.01`. Wait, the prompt literally has `$\to$`. I will just put `66.84 $\to$ 53.01`.
  
      Let's review the rules again:
      1. Only output HTML fragment, no markdown fences, no explanations.
      2. Paragraphs in `
  `, long text in `
  /
  `.
      3. Lists `
  
             
  `, quotes ` 
              
  
               `, code ` 
               
  `, important sentences ``.
      4. Keep all info, no add/delete/modify.
      5. No `//`.
  
      Is there any list,