菜单

关于 🐙 GitHub
arXiv 提交日期: 2026-05-26
📄 Abstract - Granuscore: A Reference-Free Measure of Granularity for Text Analysis and Question Answering

Natural language conveys information at varying levels of granularity, from fine-grained references to broad descriptions. While granularity is fundamental to human communication, existing measures mostly capture surface detail or sentence specificity. We introduce Granuscore, a reference-free measure of granularity that leverages structural properties of a hierarchical embedding space. Granuscore reliably recovers hierarchical orderings on the Granola-EQ dataset and captures expected differences in granularity across discourse contexts. Across domains, we further show that Granuscore explains non-linear variation in sentence specificity beyond sentence length. Finally, we apply Granuscore to four question-answering benchmarks and analyze how granularity differs for questions, gold answers, and model outputs across response outcomes. The analysis reveals consistent differences in model behavior and provides a principled lens for characterizing the difficulty of QA datasets. Together, the results position Granuscore as a scalable, broadly applicable tool for analyzing granularity in text.

顶级标签: natural language processing model evaluation benchmark
详细标签: granularity reference-free question answering text analysis hierarchical embedding 或 搜索:

Granuscore:一种无需参考的文本粒度度量方法及其在问答分析中的应用 / Granuscore: A Reference-Free Measure of Granularity for Text Analysis and Question Answering


1️⃣ 一句话总结

本文提出了一种名为Granuscore的新方法,无需人工标注或参考文本,仅通过分析词语在层次化语义空间中的结构关系,就能自动衡量文本描述的细致程度(粒度),并成功应用于分析问答系统中不同问题、答案和模型输出的粒度差异,揭示了模型行为规律和数据集难度特征。

源自 arXiv: 2605.26620