菜单

关于 🐙 GitHub
arXiv 提交日期: 2026-06-11
📄 Abstract - MoVerse: Real-Time Video World Modeling with Panoramic Gaussian Scaffold

We present MoVerse, a real-time video world model that creates an interactively navigable scene from a single narrow-field-of-view image. This setting is challenging because the input observes only a small fraction of the environment, while interactive roaming requires a complete surrounding world, persistent geometry, controllable camera motion, and temporally coherent high-fidelity observations. MoVerse addresses this problem by separating world construction from observation rendering. It first expands the input into a gravity-aligned 360$^\circ$ panorama with topology-aware diffusion, closing the missing field of view before 3D reasoning. It then lifts the panorama into a persistent 3D Gaussian scaffold using panoramic geometry-aware residual prediction, yielding a dense and directly renderable spatial memory. Finally, a Gaussian-conditioned video renderer translates scaffold renderings along user-specified camera trajectories into photorealistic video. To make this renderer practical for interaction, we train a bidirectional diffusion teacher for high-quality conditional rendering and distill it into a causal autoregressive student for bounded-latency streaming. This design combines the controllability and long-range consistency of explicit 3D representations with the perceptual quality of generative video models. MoVerse supports real-time scene roaming at 8~FPS on a single NVIDIA RTX~4090 GPU, demonstrating a practical path toward single-image world creation with interactive video output.

顶级标签: computer vision video generation multi-modal
详细标签: panoramic gaussian scaffold real-time rendering 3d scene generation diffusion model interactive navigation 或 搜索:

MoVerse:基于全景高斯支架的实时视频世界建模 / MoVerse: Real-Time Video World Modeling with Panoramic Gaussian Scaffold


1️⃣ 一句话总结

该论文提出了一种仅凭单张窄视野照片就能快速生成360度全景交互场景的实时视频世界模型,通过先补全全景图、再构建可渲染的3D高斯记忆、最后用高效的因果视频生成器实时渲染用户操控的连续视频,实现了在普通显卡上以8帧/秒流畅浏览虚拟世界。

源自 arXiv: 2606.13376