首页 > AI前沿 > A Scalable Multi Task Learning Framework for Modeling Demographic Disagreements in Annotation Tasks

A Scalable Multi Task Learning Framework for Modeling Demographic Disagreements in Annotation Tasks

arXiv自然语言 2026-11-17 01:42 12 阅读 查看原文

背景与问题

Annotators from different demographic backgrounds often disagree in systematic ways due to varying lived experiences, cultural contexts, and their relationship to the content. When such disagreement occurs, one group's perspective may sometimes warrant prioritization because they hold an epistemically advantaged standpoint or bear greater stake in the outcome. For example, when assessing toxicity of posts on social media, it may be important to prioritize a targeted group's viewpoint.

框架与技术

We propose a Stakeholder-Aware Joint Multi-Task Learning (SAJ-MTL) framework to jointly model demographic disagreement both across- and within- annotator groups. The model uses conditional gradient routing to handle sparsity induced by group-conditional labeling, and parameterization supports customized weighting of groups in different task settings. SAJ-MTL improves group-level performance in toxicity detection through better context-aware understanding of demographic-sensitive content.

优势与价值

Only annotator group-membership information is required, which is more practical than existing methods requiring more privacy-invasive individual-level annotator details. Our scalable architecture is also computationally efficient to support large-scale deployment.