菜单

关于 🐙 GitHub
arXiv 提交日期: 2026-06-20
📄 Abstract - Rebuttals Move Peer-Review Scores, but Initial-Review Structure Bounds the Movement

Author rebuttals are the main post-submission window in peer review, but their effect on reviewer scores remains hard to measure because score updates mix rebuttal content with initial score position, paper-level consensus, reviewer confidence, and discussion dynamics. We study ICLR 2024-2025 using 73,000 reviewer trajectories with externally archived pre- and post-rebuttal scores, and use LLMs only as measurement instruments. Gemini Flash 3.0 predicts implied pre-rebuttal scores from score-stripped review text. The resulting text-score offset predicts later movement, with score-increase rates rising from 8.3% when text reads below the assigned score to 31.9% when it reads above. Claude Opus 4.6 induces, and outcome-blinded Gemini Flash 3.0 validates, a 44-feature taxonomy of resolved reviewer-author exchanges, where 23 features replicate across model and held-out year under Bonferroni correction. In the rebuttal-engaged benchmark (n=6,705), initial-review structure already predicts much score movement (AUC=0.747, minimal AUC=0.696), while adding the resolved exchange raises AUC to 0.804. Rebuttals can move scores, but measurable movement is bounded by initial-review structure, and robust exchange signals are mostly rebuttal failure modes.

顶级标签: llm machine learning model evaluation
详细标签: peer review rebuttal score prediction llm-as-measurement iclr 或 搜索:

作者的回应能改变同行评审的分数,但初始评审结构限制了改变的程度 / Rebuttals Move Peer-Review Scores, but Initial-Review Structure Bounds the Movement


1️⃣ 一句话总结

这篇论文通过分析ICLR 2024-2025年间7.3万次评审记录,发现作者对评审意见的回应确实能提升评分,但这种提升的幅度受到初始评审结果和论文本身质量的制约,而且回应中最能影响分数的信号往往只出现在那些回应本身存在问题的案例中。

源自 arXiv: 2606.22166