We study how to predict the downstream closed-loop performance of a learned latent world model from validation-time diagnostics alone. Choosing the right checkpoint from a world-model training run is difficult: validation loss and multi-step prediction RMSE keep improving long after closed-loop performance has collapsed. We present a suite of structural validation-time diagnostics drawn from optimal-control theory and apply them to Gymnasium's LunarLander v3, which features shaped rewards. We train an RSSM [5, 4] world model on it and treat per checkpoint CEM-MPC return as the oracle for closed-loop quality. By evaluating 40 metrics against this oracle, we find that the strongest single predictor is the Reward Observability Fraction (ROF), which measures the reward predictor's dependence on the observable subspace. We combine ROF with three structural regularizers into a single-number offline checkpoint selection score, the Composite Reward Observability Fraction (CROF). The CROF-selected world model trains a model-based A2C policy that beats a fairly evaluated model-free A2C baseline by ~24.5 return points while using ~65x fewer real-environment interactions, and the same world model also drives a strong zero-shot CEM-MPC policy. Code and data: this https URL.
潜在世界模型的闭环性能预测：非马尔可夫奖励下月面着陆器的离线检查点选择与MPC及基于模型的强化学习 / Predicting Closed-Loop Performance of Latent World Models: Offline Checkpoint Selection for MPC and Model-Based RL Under Non-Markovian Rewards in LunarLander
本文提出一种名为CROF的离线评分方法，通过分析奖励预测器对可观测状态的依赖程度等结构指标，仅凭验证阶段的诊断数据就能筛选出训练过程中性能最佳的模型检查点，从而在月面着陆器任务中让基于模型的强化学习策略比传统无模型方法提升约24.5分，同时减少65倍的与环境交互次数。
源自 arXiv: 2607.01736
