提示词架构决定推理质量:关于洗车问题的变量隔离研究 / Prompt Architecture Determines Reasoning Quality: A Variable Isolation Study on the Car Wash Problem
1️⃣ 一句话总结
这篇论文通过实验发现,在解决需要推断隐含物理约束的‘洗车问题’时,让AI使用STAR(情境-任务-行动-结果)这种结构化思考框架,比单纯给它更多背景信息更能大幅提升推理准确率,从0%提升到了85%。
Large language models consistently fail the "car wash problem," a viral reasoning benchmark requiring implicit physical constraint inference. We present a variable isolation study (n=20 per condition, 6 conditions, 120 total trials) examining which prompt architecture layers in a production system enable correct reasoning. Using Claude 3.5 Sonnet with controlled hyperparameters (temperature 0.7, top_p 1.0), we find that the STAR (Situation-Task-Action-Result) reasoning framework alone raises accuracy from 0% to 85% (p=0.001, Fisher's exact test, odds ratio 13.22). Adding user profile context via vector database retrieval provides a further 10 percentage point gain, while RAG context contributes an additional 5 percentage points, achieving 100% accuracy in the full-stack condition. These results suggest that structured reasoning scaffolds -- specifically, forced goal articulation before inference -- matter substantially more than context injection for implicit constraint reasoning tasks.
提示词架构决定推理质量:关于洗车问题的变量隔离研究 / Prompt Architecture Determines Reasoning Quality: A Variable Isolation Study on the Car Wash Problem
这篇论文通过实验发现,在解决需要推断隐含物理约束的‘洗车问题’时,让AI使用STAR(情境-任务-行动-结果)这种结构化思考框架,比单纯给它更多背景信息更能大幅提升推理准确率,从0%提升到了85%。
源自 arXiv: 2602.21814