DiffST:面向真实世界时空视频超分辨率的时空感知扩散模型 / DiffST: Spatiotemporal-Aware Diffusion for Real-World Space-Time Video Super-Resolution
1️⃣ 一句话总结
本文提出了一种高效的扩散模型框架DiffST,通过单步采样、整视频处理、跨帧上下文聚合和视频级全局特征引导,同时提升了视频空间分辨率放大和时间插值的性能与速度,比现有扩散方法快约17倍。
Diffusion-based models have shown strong performance in video super-resolution (VSR) and video frame interpolation (VFI). However, their role in the coupled space-time video super-resolution (STVSR) setting remains limited. Existing diffusion-based STVSR approaches suffer from two issues: (1) low inference efficiency and (2) insufficient utilization of spatiotemporal information. These limitations impede deployment. To address these issues, we introduce DiffST, an efficient spatiotemporal-aware video diffusion framework for real-world STVSR. To improve efficiency, we adapt a pre-trained diffusion model for one-step sampling and process the entire video directly rather than operating on individual frames. Furthermore, to enhance spatiotemporal information utilization, we introduce cross-frame context aggregation (CFCA) and video representation guidance (VRG). The CFCA module aggregates information across multiple keyframes to produce intermediate frames. The VRG module extracts video-level global features to guide the diffusion process. Extensive experiments show that DiffST obtains leading results on real-world STVSR tasks. It also maintains high inference efficiency, running about 17$\times$ faster than previous diffusion-based STVSR methods. Code is available at: this https URL.
DiffST:面向真实世界时空视频超分辨率的时空感知扩散模型 / DiffST: Spatiotemporal-Aware Diffusion for Real-World Space-Time Video Super-Resolution
本文提出了一种高效的扩散模型框架DiffST,通过单步采样、整视频处理、跨帧上下文聚合和视频级全局特征引导,同时提升了视频空间分辨率放大和时间插值的性能与速度,比现有扩散方法快约17倍。
源自 arXiv: 2605.13182