首页 > AI前沿 > MInfoVisQA: A Multilingual Benchmark for Information-Rich Visual Question Answering

MInfoVisQA: A Multilingual Benchmark for Information-Rich Visual Question Answering

arXiv自然语言 2026-11-18 13:00 19 阅读 查看原文
用户需要我扮演一个专业的新闻排版工程师,将一段纯文本的新闻正文(实际上是学术论文摘要)排版成语义化的HTML片段。 规则如下: 1. 只输出HTML片段本身,不要Markdown围栏,不要解释。 2. 每个自然段落用`

`包裹,长文用`

`/`

`划分小节。 3. 列表内容用`
  • `,引用用`
    `,代码用`
    `,重要句子可用``。
    4. 保留原文所有信息,不得增删改内容。
    5. 不要输出``/``/``等文档级标签。
    
    正文内容:
    "Multimodal large language models (MLLMs) excel in vision-language tasks; yet, whether they maintain consistent understanding of information-rich visuals (InfoVis) when the surface language changes remains unclear. To quantify this capability, we introduce MInfoVisQA, a parallel benchmark with 8,960 QA configurations derived from 128 seed instances across 24 languages. We construct seed-matched, language-parallel visuals and QA pairs by disentangling numerical-structural data from text and programmatically synchronizing translated labels. We evaluate 25 MLLMs under two settings: Language-Aligned QA (LQA, matching visual-text languages) and Cross-Language QA (XQA, fixed QA language with varying visual languages). The results show that aggregate accuracy masks substantial robustness gaps: two models with comparable mean LQA accuracy (80.4\%) exhibit performance spreads of 7.8 and 20.3 percentage points across languages. Moreover, all 25 models score lower with non-English visuals under fixed English QA, underscoring the need for cross-language sensitivity metrics."
    
    分析正文:
    这是一段连续的学术摘要,没有明显的列表、引用或代码。可以将其分为几个逻辑段落,或者加上小标题来满足“长文用

    /

    划分小节”的要求(虽然这里不算特别长,但加上小标题更符合排版工程师的专业性,且不违反“不得增删改内容”的规则,因为标题是对段落的概括,不过规则说“不得增删改内容”,加标题算不算增?最好把小标题和正文结合,或者直接将这段分为几个自然段落用`

    `包裹。为了稳妥,避免添加原文没有的文字,我可以将其分为几个自然段,或者使用`

    `作为小节标题但尽量不添加新词,或者干脆只用`

    `。规则2说“长文用

    /

    划分小节”,这段文字不算特别长,可以直接用几个`

    `来分段。 分段策略: 第一段:背景与提出的方法。 "Multimodal large language models (MLLMs) excel in vision-language tasks; yet, whether they maintain consistent understanding of information-rich visuals (InfoVis) when the surface language changes remains unclear." 第二段:基准构建。 "To quantify this capability, we introduce MInfoVisQA, a parallel benchmark with 8,960 QA configurations derived from 128 seed instances across 24 languages. We construct seed-matched, language-parallel visuals and QA pairs by disentangling numerical-structural data from text and programmatically synchronizing translated labels." 第三段:评估设置。 "We evaluate 25 MLLMs under two settings: Language-Aligned QA (LQA, matching visual-text languages) and Cross-Language QA (XQA, fixed QA language with varying visual languages)." 第四段:结果与结论。 "The results show that aggregate accuracy masks substantial robustness gaps: two models with comparable mean LQA accuracy (80.4\%) exhibit performance spreads of 7.8 and 20.3 percentage points across languages. Moreover, all 25 models score lower with non-English visuals under fixed English QA, underscoring the need for cross-language sensitivity metrics." 为了体现“重要句子可用``”,我可以把一些核心结论或关键术语加粗。比如: - MInfoVisQA - Language-Aligned QA (LQA, matching visual-text languages) 和 Cross-Language QA (XQA, fixed QA language with varying visual languages) - aggregate accuracy masks substantial robustness gaps 检查是否有需要转义的字符: `80.4\%` 在原文中是 `80.4\%`,在HTML中应该输出为 `80.4%` 或者保留 `80.4\%`?原文是 `80.4\%`,可能是LaTeX转义,但作为纯文本输入,我最好原样保留 `80.4\%` 或者转成HTML实体 `80.4\%`。其实直接输出 `80.4\%` 也可以,或者 `80.4%`。为了严格“不增删改”,保留 `80.4\%`。在HTML中反斜杠不是特殊字符,直接输出即可。 构建HTML:

    Multimodal large language models (MLLMs) excel in vision-language tasks; yet, whether they maintain consistent understanding of information-rich visuals (InfoVis) when the surface language changes remains unclear.

    To quantify this capability, we introduce MInfoVisQA, a parallel benchmark with 8,960 QA configurations derived from 128 seed instances across 24 languages. We construct seed-matched, language-parallel visuals and QA pairs by disentangling numerical-structural data from text and programmatically synchronizing translated labels.

    We evaluate 25 MLLMs under two settings: Language-Aligned QA (LQA, matching visual-text languages) and Cross-Language QA (XQA, fixed QA language with varying visual languages).

    The results show that aggregate accuracy masks substantial robustness gaps: two models with comparable mean LQA accuracy (80.4\%) exhibit performance spreads of 7.8 and 20.3 percentage points across languages. Moreover, all 25 models score lower with non-English visuals under fixed English QA, underscoring the need for cross-language sensitivity metrics.

    检查规则: 1. 只输出HTML片段本身,不要Markdown围栏、不要解释。 -> 满足 2. 每个自然段落用

    包裹,长文用

    /

    划分小节。 -> 这里没有用h2/h3,因为不算长文,只用了p。如果非要加,可以加,但不加也不违规。为了更符合“语义化”,保持p即可。 3. 列表内容用
    • ,引用用
      ,代码用
      ,重要句子可用。 -> 使用了。
      4. 保留原文所有信息,不得增删改内容。 -> 满足,仅添加了strong标签。
      5. 不要输出//等文档级标签。 -> 满足。
      
      再审视一下原文的 `80.4\%`,在HTML里直接写 `80.4\%` 是安全的。
      
      最终输出应该就是这几行HTML代码。不需要任何多余的换行或说明。
      确保没有 markdown 的 ```html ``` 包裹。直接输出 `

      ...

      ` 等。 检查拼写和标点,与原文完全一致。 原文: Multimodal large language models (MLLMs) excel in vision-language tasks; yet, whether they maintain consistent understanding of information-rich visuals (InfoVis) when the surface language changes remains unclear. To quantify this capability, we introduce MInfoVisQA, a parallel benchmark with 8,960 QA configurations derived from 128 seed instances across 24 languages. We construct seed-matched, language-parallel visuals and QA pairs by disentangling numerical-structural data from text and programmatically synchronizing translated labels. We evaluate 25 MLLMs under two settings: Language-Aligned QA (LQA, matching visual-text languages) and Cross-Language QA (XQA, fixed QA language with varying visual languages). The results show that aggregate accuracy masks substantial robustness gaps: two models with comparable mean LQA accuracy (80.4\%) exhibit performance spreads of 7.8 and 20.