首页 > AI前沿 > SAGE: Schema-Guided LLMs for Grant Review

SAGE: Schema-Guided LLMs for Grant Review

arXiv自然语言 2026-07-23 01:55 6 阅读 查看原文

Grant reviewers must apply detailed criteria to application forms, budgets, and supporting documents while producing assessments that colleagues can inspect.

We present SAGE, Schema-Guided Aspect-Based Grant Evaluation, a system that translates a grant rubric into structured checks and links its judgements to evidence from the application package.

We evaluate SAGE in two stages on 35 nonprofit grant applications.

A post-factum comparison with 105 reviews from the original competition shows fair ordinal agreement (kappa = 0.29).

The foundation then conducted a criterion-level re-review after inspecting SAGE, producing 202 assessments.

In this assisted round, SAGE reached kappa = 0.58 and outperformed a one-prompt-per-criterion baseline (kappa = 0.33 on the common subset), with higher rank correlation and lower error.

A claim-level audit further identifies confirmed, disputed, and unaddressed parts of the structured draft.

SAGE operationalizes the review methodology by producing a detailed, evidence-linked, and auditable draft for expert correction.