菜单

关于 🐙 GitHub
arXiv 提交日期: 2025-12-17
📄 Abstract - Nemotron-Math: Efficient Long-Context Distillation of Mathematical Reasoning from Multi-Mode Supervision

High-quality mathematical reasoning supervision requires diverse reasoning styles, long-form traces, and effective tool integration, capabilities that existing datasets provide only in limited form. Leveraging the multi-mode generation ability of gpt-oss-120b, we introduce Nemotron-Math, a large-scale mathematical reasoning dataset containing 7.5M solution traces across high, medium, and low reasoning modes, each available both with and without Python tool-integrated reasoning (TIR). The dataset integrates 85K curated AoPS problems with 262K community-sourced StackExchange-Math problems, combining structured competition tasks with diverse real-world mathematical queries. We conduct controlled evaluations to assess the dataset quality. Nemotron-Math consistently outperforms the original OpenMathReasoning on matched AoPS problems. Incorporating StackExchange-Math substantially improves robustness and generalization, especially on HLE-Math, while preserving accuracy on math competition benchmarks. To support efficient long-context training, we develop a sequential bucketed strategy that accelerates 128K context-length fine-tuning by 2--3$\times$ without significant accuracy loss. Overall, Nemotron-Math enables state-of-the-art performance, including 100\% maj@16 accuracy on AIME 2024 and 2025 with Python TIR.

顶级标签: llm model training data
详细标签: mathematical reasoning dataset distillation long-context training tool-integrated reasoning instruction tuning 或 搜索:

Nemotron-Math:基于多模式监督的高效长上下文数学推理知识蒸馏 / Nemotron-Math: Efficient Long-Context Distillation of Mathematical Reasoning from Multi-Mode Supervision


1️⃣ 一句话总结

这篇论文通过利用大模型生成多种解题思路和工具使用方式,构建了一个大规模、高质量的数学推理数据集,并开发了高效的训练方法,使AI模型在数学竞赛和实际应用中的解题能力达到了顶尖水平。


源自 arXiv: 2512.15489