首页 > AI前沿 > FERN: Cross-Benchmark Table Retrieval for Text-to-SQL with Frozen Retrievers and Learned Ranking

FERN: Cross-Benchmark Table Retrieval for Text-to-SQL with Frozen Retrievers and Learned Ranking

arXiv自然语言 2026-11-11 13:00 32 阅读 查看原文
To fulfill the user's request, I need to act as a professional news layout engineer and format the provided plain text into semantic HTML fragments. First, I will analyze the rules: 1. Only output the HTML fragment itself, no Markdown fences, no explanations. 2. Every natural paragraph must be wrapped in `

`, and long text should use `

`/`

` to divide sections. 3. Lists use `
  • `, quotes use `
    `, code uses `
    `, and important sentences can use ``.
    4. Keep all original information, do not add, delete, or modify content.
    5. Do not output document-level tags like ``, ``, ``.
    
    Looking at the input text, it's a technical comparison between two systems: "Text-to-SQL systems" and "FERN". 
    I can divide the content into logical sections using headers:
    - `

    ` for the first main topic: Retrieval Requirements and Baseline. - `

    ` for the second main topic: The FERN Approach. - `

    ` for the third main topic: Performance Evaluation. - Under "Performance Evaluation", I can use `

    ` for specific datasets like Spider, BIRD, BEAVER, and Fiben. Now, let's map the text to these elements step-by-step: 1. "Text-to-SQL systems over large pooled catalogs must first retrieve the tables a question needs. We compare against the published table recall of RASL, which uses Claude 3.5 models to extract keywords and select tables, on pooled BIRD, Spider, and Fiben catalogs." -> This will be the introduction paragraph under a general context, or maybe just a standalone `

    ` before the `

    ` sections. Let's make it an intro `

    `. 2. "FERN (Feature-Engineered Ranking with No generative LLM) instead fuses three frozen retrievers into 100 candidates, describes each with up to 120 features, and reorders them with LightGBM rankers fitted once on BIRD's training questions." -> This defines the FERN system. I'll put it under `

    The FERN Approach

    ` and wrap it in `

    `. I can highlight "FERN (Feature-Engineered Ranking with No generative LLM)" with ``. 3. The rest of the text details the performance across different datasets. -> I'll create `

    Performance Evaluation

    ` and add specific `

    ` tags for each dataset. For Spider: "On Spider, where it was not fitted, its candidate pool alone reaches RASL's published recall at fifteen tables (98.0), and the ranker raises the share of questions with every gold table in the top five (complete@5) from 83.9% to 95.6%." "At five tables, FERN's recall is within one point of RASL's on Spider (96.7 vs 97.0)" For BIRD: "and on BIRD with its evidence hints (96.6 vs 97.5), and 8.2 points below on BIRD without them (89.3)." "On BIRD dev with evidence, non-retrieval features raise complete@5 by 7.7 points over retrieval signals alone." "On BIRD dev, median latency is 208 ms per question on one A100, without hosted model calls." For BEAVER: "On 121 real BEAVER questions, without BEAVER fitting, its recall at fifteen (82.1) falls between two BEAVER-trained retrievers (80.8, 84.5), but its complete@15 is below both (49.6 vs 54.5 and 62.0)." For Fiben: "On Fiben, it falls far short at fifteen tables (54.2 vs 69.2, different question sets), although its recall at fifty is numerically close to RASL's question-only retrieval result (80.7 vs 79.4)." Let's assemble the HTML carefully, ensuring no content is lost or altered. I need to break down the long sentences logically but strictly preserve the original wording. Drafting the HTML structure: ```html

    Text-to-SQL systems over large pooled catalogs must first retrieve the tables a question needs. We compare against the published table recall of RASL, which uses Claude 3.5 models to extract keywords and select tables, on pooled BIRD, Spider, and Fiben catalogs.

    The FERN Approach

    FERN (Feature-Engineered Ranking with No generative LLM) instead fuses three frozen retrievers into 100 candidates, describes each with up to 120 features, and reorders them with LightGBM rankers fitted once on BIRD's training questions.

    Performance Evaluation

    Spider Dataset

    On Spider, where it was not fitted, its candidate pool alone reaches RASL's published recall at fifteen tables (98.0), and the ranker raises the share of questions with every gold table in the top five (complete@5) from 83.9% to 95.6%.

    At five tables, FERN's recall is within one point of RASL's on Spider (96.7 vs 97.0)

    BIRD Dataset

    ...and on BIRD with its evidence hints (96.6 vs 97.5), and 8.2 points below on BIRD without them (89.3).

    On BIRD dev with evidence, non-retrieval features raise complete@5 by 7.7 points over retrieval signals alone.

    On BIRD dev, median latency is 208 ms per question on one A100, without hosted model calls.

    BEAVER Dataset

    On 121 real BEAVER questions, without BEAVER fitting, its recall at fifteen (82.1) falls between two BEAVER-trained retrievers (80.8, 84.5), but its complete@15 is below both (49.6 vs 54.5 and 62.0).

    Fiben Dataset

    On Fiben, it falls far short at fifteen tables (54.2 vs 69.2, different question sets), although its recall at fifty is numerically close to RASL's question-only retrieval result (80.7 vs 79.4).

    ``` Wait, looking at the original text: "...and reorders them with LightGBM rankers fitted once on BIRD's training questions. On Spider, where it was not fitted..." The text flows from Spider to BIRD: "...(96.7 vs 97.0) and on BIRD with its evidence hints (96.6 vs 97.5), and 8.2 points below on BIRD without them (89.3)." I should ensure the flow is perfectly maintained without breaking sentences awkwardly if possible, or just segment them logically. The current segmentation looks good and keeps all conjunctions intact where needed. Let's double check if I missed any text. Original: "Text-to-SQL systems over large pooled catalogs must first retrieve the tables a question needs. We compare against the published table recall of RASL, which uses Claude 3.5 models to extract keywords and select tables, on pooled BIRD, Spider, and Fiben catalogs. FERN (Feature-Engineered Ranking with No generative LLM) instead fuses three frozen retrievers into 100 candidates, describes each with up to 120 features, and reorders them with LightGBM rankers fitted once on BIRD's training questions. On Spider, where it was not fitted, its