解耦任务与行为:机器人强化学习中的两阶段奖励课程 / Decoupling Task and Behavior: A Two-Stage Reward Curriculum in Reinforcement Learning for Robotics
1️⃣ 一句话总结
这篇论文提出了一种两阶段训练方法,先让机器人学会基本任务,再引导它优化行为细节(如节能),从而更简单有效地训练出性能更好、更稳定的机器人控制策略。
Deep Reinforcement Learning is a promising tool for robotic control, yet practical application is often hindered by the difficulty of designing effective reward functions. Real-world tasks typically require optimizing multiple objectives simultaneously, necessitating precise tuning of their weights to learn a policy with the desired characteristics. To address this, we propose a two-stage reward curriculum where we decouple task-specific objectives from behavioral terms. In our method, we first train the agent on a simplified task-only reward function to ensure effective exploration before introducing the full reward that includes auxiliary behavior-related terms such as energy efficiency. Further, we analyze various transition strategies and demonstrate that reusing samples between phases is critical for training stability. We validate our approach on the DeepMind Control Suite, ManiSkill3, and a mobile robot environment, modified to include auxiliary behavioral objectives. Our method proves to be simple yet effective, substantially outperforming baselines trained directly on the full reward while exhibiting higher robustness to specific reward weightings.
解耦任务与行为:机器人强化学习中的两阶段奖励课程 / Decoupling Task and Behavior: A Two-Stage Reward Curriculum in Reinforcement Learning for Robotics
这篇论文提出了一种两阶段训练方法,先让机器人学会基本任务,再引导它优化行为细节(如节能),从而更简单有效地训练出性能更好、更稳定的机器人控制策略。
源自 arXiv: 2603.05113