菜单

关于 🐙 GitHub
arXiv 提交日期: 2026-05-20
📄 Abstract - "I didn't Make the Micro Decisions": Measuring, Inducing, and Exposing Goal-Level AI Contributions in Collaboration

As large language models (LLMs) increasingly shape how users form, refine, and extend their goals, attributing contributions in human-AI collaboration becomes critical for users calibrating their own reliance and for evaluators assessing AI-assisted work. Yet existing methods focus on final artifacts, missing the process through which goals themselves are jointly shaped. We introduce a goal-level attribution framework, CoTrace, that decomposes explicit goals into verifiable requirements and traces both direct contributions and indirect influences across dialogue turns. Applying CoTrace to 638 real-world collaboration logs, we find that while models account for only 11-26% of goal-shaping contribution, they contribute substantially more on introducing lower-level concrete requirements, and make various kinds of indirect contributions. Through controlled simulations, we show that interaction design choices significantly affect model goal-shaping behavior. In a user study, exposing participants to goal-level analyses shifts their perceived contributions by nearly 2 points on a 5-point scale, revealing systematic miscalibration in how users understand their own AI-assisted work.

顶级标签: llm human-ai interaction evaluation
详细标签: goal attribution co-trace framework collaboration analysis user calibration interaction design 或 搜索:

“我没有做出微观决策”:衡量、引导和揭示人机协作中的目标级AI贡献 / "I didn't Make the Micro Decisions": Measuring, Inducing, and Exposing Goal-Level AI Contributions in Collaboration


1️⃣ 一句话总结

本文提出了一种名为CoTrace的新框架,能够像“显微镜”一样追踪AI在与人类协作中如何逐步影响和塑造目标,而不仅仅是评估最终成果。通过分析真实对话记录和用户实验,研究发现AI虽然只承担了11%-26%的目标塑造任务,但特别擅长引入具体细节需求,并且交互方式的设计会显著改变AI的贡献方式;更重要的是,向用户展示这些分析结果后,他们对AI贡献的认知会发生近2分(5分制)的显著变化,说明人们往往错误估计了自己在AI辅助工作中的实际主导程度。

源自 arXiv: 2605.21363