检索增强的语言校准 / Retrieval-Augmented Linguistic Calibration
1️⃣ 一句话总结
本文提出了一种名为RALC的轻量级后处理方法,通过检索并改写文本中的语言信心表达,让AI模型在回答问题时能更准确、更可靠地反映其真实把握程度,从而提升信息传达的诚实度与可信度。
Linguistic cues such as "I believe" and "probably" offer an intuitive interface for communicating confidence, yet a generalisable, principled calibration framework for linguistic confidence expressions remains underexplored. In particular, co-occurring linguistic cues, contextual variation, and subjective audience interpretation pose unique challenges. We therefore model linguistic confidence as a distribution over plausible perceived probability values that a statement is correct, capturing interpretation variability that scalar representations discard. Within this distributional framework, we introduce faithfulness as a complementary evaluation dimension and present Faithfulness Divergence (FD), an information-theoretic metric quantifying the surprise induced in audience beliefs upon truth revelation. Building on these foundations, we present Retrieval-Augmented Linguistic Calibration (RALC), a lightweight post-hoc pipeline that propagates calibrated confidence signals back into natural language via retrieval-augmented rewriting. Across three QA benchmarks and five LLM families, RALC improves in-domain faithfulness and calibration up to 66% and 58%, respectively, outperforming black-box and grey-box calibration baselines.
检索增强的语言校准 / Retrieval-Augmented Linguistic Calibration
本文提出了一种名为RALC的轻量级后处理方法,通过检索并改写文本中的语言信心表达,让AI模型在回答问题时能更准确、更可靠地反映其真实把握程度,从而提升信息传达的诚实度与可信度。
源自 arXiv: 2605.19344