菜单

关于 🐙 GitHub
arXiv 提交日期: 2026-03-04
📄 Abstract - Dual Diffusion Models for Multi-modal Guided 3D Avatar Generation

Generating high-fidelity 3D avatars from text or image prompts is highly sought after in virtual reality and human-computer interaction. However, existing text-driven methods often rely on iterative Score Distillation Sampling (SDS) or CLIP optimization, which struggle with fine-grained semantic control and suffer from excessively slow inference. Meanwhile, image-driven approaches are severely bottlenecked by the scarcity and high acquisition cost of high-quality 3D facial scans, limiting model generalization. To address these challenges, we first construct a novel, large-scale dataset comprising over 100,000 pairs across four modalities: fine-grained textual descriptions, in-the-wild face images, high-quality light-normalized texture UV maps, and 3D geometric shapes. Leveraging this comprehensive dataset, we propose PromptAvatar, a framework featuring dual diffusion models. Specifically, it integrates a Texture Diffusion Model (TDM) that supports flexible multi-condition guidance from text and/or image prompts, alongside a Geometry Diffusion Model (GDM) guided by text prompts. By learning the direct mapping from multi-modal prompts to 3D representations, PromptAvatar eliminates the need for time-consuming iterative optimization, successfully generating high-fidelity, shading-free 3D avatars in under 10 seconds. Extensive quantitative and qualitative experiments demonstrate that our method significantly outperforms existing state-of-the-art approaches in generation quality, fine-grained detail alignment, and computational efficiency.

顶级标签: computer vision multi-modal model training
详细标签: 3d avatar generation diffusion models multi-modal guidance dataset creation text-to-3d 或 搜索:

用于多模态引导三维虚拟人生成的双扩散模型 / Dual Diffusion Models for Multi-modal Guided 3D Avatar Generation


1️⃣ 一句话总结

这篇论文提出了一个名为PromptAvatar的新框架,它利用文本和图像提示,通过两个专门的扩散模型快速生成细节丰富、无需迭代优化的高保真3D虚拟人,解决了现有方法在细节控制、速度和数据依赖上的难题。

源自 arXiv: 2603.04307