技能奖励模型:通过智能体技能统一异构评价标准 / Skill-RM: Unifying Heterogeneous Evaluation Criteria via Agent Skill
1️⃣ 一句话总结
本文提出Skill-RM,一种将奖励建模转化为可复用“奖励评估技能”的统一框架,通过智能体动态选择和整合多种评价证据(如规则、参考答案、检查表等),从而更灵活、透明地评估大语言模型输出,在多项测试中表现优于传统方法。
Reward models (RMs) provide critical feedback signals for LLM post-training, notably in reinforced fine-tuning (RFT) and reinforcement learning (RL) pipelines. However, current reward evaluation relies on heterogeneous criteria such as rule-based verifiers, ground-truth references, procedural checklists, and complex rubrics, where a unified mechanism to integrate all types of evidence remains unexplored. To this end, we propose Skill Reward Model (Skill-RM), a unified framework that reformulates reward modeling as the execution of a reusable Reward-Evaluation Skill. By treating reward computation as a structured agentic task, Skill-RM provides a consistent interface to orchestrate heterogeneous resources, dynamically selecting and aggregating evidence tailored to the specific requirements of each input. This approach enables the reward model to move beyond static evaluation, ensuring consistency and transparency across diverse tasks. Extensive experiments on reward benchmarks and downstream applications, including best-of-N selection and reinforcement learning, demonstrate that Skill-RM consistently outperforms traditional judge baselines. Our findings suggest that Skill-RM not only provides a unified solution for reward modeling but also achieves superior performance through the strategic and dynamic orchestration of evidence. The code is at this https URL.
技能奖励模型:通过智能体技能统一异构评价标准 / Skill-RM: Unifying Heterogeneous Evaluation Criteria via Agent Skill
本文提出Skill-RM,一种将奖励建模转化为可复用“奖励评估技能”的统一框架,通过智能体动态选择和整合多种评价证据(如规则、参考答案、检查表等),从而更灵活、透明地评估大语言模型输出,在多项测试中表现优于传统方法。
源自 arXiv: 2606.03980