标注者立场作为信号:用于检测反自闭症歧视的心理测量加权方法 / Annotator Positionality as Signal: Psychometric Weighting for Anti-Autistic Ableism Detection
1️⃣ 一句话总结
该研究提出了一种基于标注者身份立场的心理测量加权方法,用于更准确地检测文本中的反自闭症歧视语言,发现传统多数投票方法会系统性忽视自闭症群体的观点,而大语言模型往往依赖关键词匹配而非上下文理解,导致对自闭症群体产生有害输出。
Large language models (LLMs) are increasingly used in decision-making tasks where they can amplify or suppress perspectives, raising concerns in high-stakes settings affecting autistic communities. While previous research has identified disability-related biases in LLMs, it remains unclear how they conceptualize ableism or detect it in text. We introduce a bias-aware evaluation framework targeting anti-autistic ableist language with a psychometrically-weighted, community-proximate ground truth anchored in annotator positionality. This framework constitutes a stricter standard than conventional majority-vote aggregation which significantly and consistently underweights autistic and autism-accepting perspectives. We find that LLMs frequently produce harmful outputs, mislabel community-reclaimed language as ableist, and express more negative attitudes toward autistic people when assessment instruments are masked. Our error analysis reveals that models rely on surface-level keyword matching rather than contextual factors such as speaker identity, and whether the language fosters in-group solidarity or inflicts out-group harm.
标注者立场作为信号:用于检测反自闭症歧视的心理测量加权方法 / Annotator Positionality as Signal: Psychometric Weighting for Anti-Autistic Ableism Detection
该研究提出了一种基于标注者身份立场的心理测量加权方法,用于更准确地检测文本中的反自闭症歧视语言,发现传统多数投票方法会系统性忽视自闭症群体的观点,而大语言模型往往依赖关键词匹配而非上下文理解,导致对自闭症群体产生有害输出。
源自 arXiv: 2605.26397