Achieving fully autonomous driving systems requires learning rational decisions in a wide span of scenarios, including safety-critical and out-of-distribution ones. However, such cases are underrepresented in real-world corpus collected by human experts. To complement for the lack of data diversity, we introduce a novel and scalable simulation framework capable of synthesizing massive unseen states upon existing driving logs. Our pipeline utilizes advanced neural rendering with a reactive environment to generate high-fidelity multi-view observations controlled by the perturbed ego trajectory. Furthermore, we develop a pseudo-expert trajectory generation mechanism for these newly simulated states to provide action supervision. Upon the synthesized data, we find that a simple co-training strategy on both real-world and simulated samples can lead to significant improvements in both robustness and generalization for various planning methods on challenging real-world benchmarks, up to +6.8 EPDMS on navhard and +2.9 on navtest. More importantly, such policy improvement scales smoothly by increasing simulation data only, even without extra real-world data streaming in. We further reveal several crucial findings of such a sim-real learning system, which we term SimScale, including the design of pseudo-experts and the scaling properties for different policy architectures. Our simulation data and code would be released.
SimScale：通过大规模真实世界仿真学习驾驶 / SimScale: Learning to Drive via Real-World Simulation at Scale
这篇论文提出了一个名为SimScale的新型仿真框架，它能够利用现有的真实驾驶数据，通过神经渲染和反应式环境生成大量高保真、多样化的模拟驾驶场景，并配合一种伪专家轨迹生成机制来提供训练监督，从而显著提升自动驾驶规划模型在安全关键和罕见场景下的鲁棒性与泛化能力，且其性能提升仅需增加模拟数据即可平滑扩展。