菜单

关于 🐙 GitHub
arXiv 提交日期: 2026-07-05
📄 Abstract - Learning to Control LLM Agent Harnesses with Offline Reinforcement Learning

Large language model (LLM) agents are usually improved by changing prompts, models, or hand-written workflows, while the execution harness around the model is treated as fixed infrastructure. We argue that this harness is itself a learnable control layer. We formalize harness operation as a finite-horizon Harness MDP, where a lightweight controller selects structural execution actions while the LLM executor remains frozen. The controller is trained from offline rollouts using advantage-weighted regression with only terminal task-rubric rewards. We also separate final task quality from a post-hoc Harness Maturity Score, which measures whether the harness follows reliable execution patterns rather than only whether the final answer is correct. This separation gives a finite-buffer view of harness learning: final-quality gains require high-return support in the offline buffer, while process behavior can shift whenever it aligns with advantage-weighted actions. Across six controlled domains and two public-benchmark adapters, the learned controller consistently improves verification behavior and selectively improves final task quality, with the largest gains on adapted tau-bench retail, adapted AgentBench DB-Bench, and coding with a calibrated structural verifier. Ablations against behavior cloning and Forced CHECK show that the gains are not explained by imitation or by simply adding checks. These results identify harness control as a learnable layer for frozen LLM agents, while showing that offline support limits when better process control becomes better final answers.

顶级标签: llm agents reinforcement learning
详细标签: offline rl harness control advantage-weighted regression execution actions evaluation 或 搜索:

利用离线强化学习控制大型语言模型智能体的执行框架 / Learning to Control LLM Agent Harnesses with Offline Reinforcement Learning


1️⃣ 一句话总结

本文提出将大型语言模型(LLM)智能体的外部执行框架视为一个可学习的控制层,通过离线强化学习训练一个轻量级控制器来优化执行流程,从而在保持LLM本身不变的前提下,提升智能体的任务完成质量和行为可靠性。

源自 arXiv: 2607.05458