评估多轮医学诊断:过早回答、信息诱导与自我纠正 / Benchmarking Multi-turn Medical Diagnosis: Hold, Lure, and Self-Correction
1️⃣ 一句话总结
这篇论文通过构建一个多轮医学诊断测试集,发现大语言模型在逐步获取信息时存在过早下结论、容易被关键信息诱导以及具备自我纠正潜力等问题,并提出了推迟提问和关键信息后置等实用方法来显著提升诊断的准确性和可靠性。
Large language models (LLMs) achieve high accuracy in medical diagnosis when all clinical information is provided in a single turn, yet how they behave under multi-turn evidence accumulation closer to real clinical reasoning remains unexplored. We introduce MINT (Medical Incremental N-Turn Benchmark), a high-fidelity, multi-turn medical diagnosis benchmark comprising 1,035 cases with clinically labeled evidence shards, controlled turn granularity, and information-preserving decomposition. Through systematic evaluation of 11 LLMs on MINT, we uncover three persistent behavioral patterns that significantly impact diagnostic decisions: (1) intent to answer, models rush to answer before sufficient evidence has been observed, with over 55% of answers committed within the first two turns; (2) self-correction, incorrect-to-correct answer revisions occur at up to 10.6 times the rate of correct-to-incorrect flips, revealing a latent capacity for self-correction that premature commitment forecloses; and (3) strong lures, clinically salient information such as laboratory results trigger premature answering even when models are explicitly instructed to wait. We translate these findings into clinically actionable guidance: deferring the diagnostic question to later turns reduces premature answering and improves accuracy at the first point of commitment by up to 62.6%, while reserving salient clinical evidence for later turns prevents a catastrophic accuracy drop of up to 23.3% caused by premature commitment. Our work provides both a controlled evaluation framework and concrete recommendations for improving the reliability of LLMs in multi-turn medical diagnosis.
评估多轮医学诊断:过早回答、信息诱导与自我纠正 / Benchmarking Multi-turn Medical Diagnosis: Hold, Lure, and Self-Correction
这篇论文通过构建一个多轮医学诊断测试集,发现大语言模型在逐步获取信息时存在过早下结论、容易被关键信息诱导以及具备自我纠正潜力等问题,并提出了推迟提问和关键信息后置等实用方法来显著提升诊断的准确性和可靠性。
源自 arXiv: 2604.04325