菜单

关于 🐙 GitHub
arXiv 提交日期: 2026-03-16
📄 Abstract - Learning Latent Proxies for Controllable Single-Image Relighting

Single-image relighting is highly under-constrained: small illumination changes can produce large, nonlinear variations in shading, shadows, and specularities, while geometry and materials remain unobserved. Existing diffusion-based approaches either rely on intrinsic or G-buffer pipelines that require dense and fragile supervision, or operate purely in latent space without physical grounding, making fine-grained control of direction, intensity, and color unreliable. We observe that a full intrinsic decomposition is unnecessary and redundant for accurate relighting. Instead, sparse but physically meaningful cues, indicating where illumination should change and how materials should respond, are sufficient to guide a diffusion model. Based on this insight, we introduce LightCtrl that integrates physical priors at two levels: a few-shot latent proxy encoder that extracts compact material-geometry cues from limited PBR supervision, and a lighting-aware mask that identifies sensitive illumination regions and steers the denoiser toward shading relevant pixels. To compensate for scarce PBR data, we refine the proxy branch using a DPO-based objective that enforces physical consistency in the predicted cues. We also present ScaLight, a large-scale object-level dataset with systematically varied illumination and complete camera-light metadata, enabling physically consistent and controllable training. Across object and scene level benchmarks, our method achieves photometrically faithful relighting with accurate continuous control, surpassing prior diffusion and intrinsic-based baselines, including gains of up to +2.4 dB PSNR and 35% lower RMSE under controlled lighting shifts.

顶级标签: computer vision model training multi-modal
详细标签: single-image relighting diffusion models physical priors latent representation dataset creation 或 搜索:

学习潜在代理以实现可控的单图像重光照 / Learning Latent Proxies for Controllable Single-Image Relighting


1️⃣ 一句话总结

这篇论文提出了一种名为LightCtrl的新方法,它通过从少量物理渲染数据中学习稀疏但关键的材质与几何线索,并结合光照感知掩码来引导扩散模型,从而实现了对单张图片进行更精确、更可控的光照效果调整。

源自 arXiv: 2603.15555