LSRIF:用于指令遵循的逻辑结构化强化学习 / LSRIF: Logic-Structured Reinforcement Learning for Instruction Following
1️⃣ 一句话总结
这篇论文提出了一种名为LSRIF的新训练框架,它通过显式地建模指令中的逻辑结构(如顺序、条件和并行关系),并设计对应的结构化奖励方法,显著提升了大语言模型遵循复杂指令和进行逻辑推理的能力。
Instruction-following is critical for large language models, but real-world instructions often contain logical structures such as sequential dependencies and conditional branching. Existing methods typically construct datasets with parallel constraints and optimize average rewards, ignoring logical dependencies and yielding noisy signals. We propose a logic-structured training framework LSRIF that explicitly models instruction logic. We first construct a dataset LSRInstruct with constraint structures such as parallel, sequential, and conditional types, and then design structure-aware rewarding method LSRIF including average aggregation for parallel structures, failure-penalty propagation for sequential structures, and selective rewards for conditional branches. Experiments show LSRIF brings significant improvements in instruction-following (in-domain and out-of-domain) and general reasoning. Analysis reveals that learning with explicit logic structures brings parameter updates in attention layers and sharpens token-level attention to constraints and logical operators.
LSRIF:用于指令遵循的逻辑结构化强化学习 / LSRIF: Logic-Structured Reinforcement Learning for Instruction Following
这篇论文提出了一种名为LSRIF的新训练框架,它通过显式地建模指令中的逻辑结构(如顺序、条件和并行关系),并设计对应的结构化奖励方法,显著提升了大语言模型遵循复杂指令和进行逻辑推理的能力。
源自 arXiv: 2601.06431