菜单

关于 🐙 GitHub
arXiv 提交日期: 2026-06-24
📄 Abstract - Variational Inference via Entropic Transport Descent

Particle-based variational inference (ParVI) methods approximate an intractable target distribution by evolving an ensemble of interacting samples. Existing approaches rely predominantly on kernel-based repulsion (e.g., SVGD), which suffers from variance collapse in high dimensions and mode collapse on multimodal targets -- pathologies caused by the absence of global transport structure. We introduce entropic transport descent (ETD), a ParVI family that frames each particle update as an entropy-regularized optimal transport problem. Derived from the JKO proximal scheme by lifting to the space of couplings and relaxing via the KL chain rule, each ETD iteration reduces to a Sinkhorn computation. The resulting transport plan provides global coordination, guiding each particle to nearby high-density proposals and naturally preserving multimodal structure. ETD can operate entirely score-free, requiring only pointwise evaluations of the unnormalized target density. Experiments on variance-collapse diagnostics, Bayesian logistic regression, neural networks, and molecular Boltzmann distributions show that ETD matches or outperforms SVGD, AGF-SVGD, and SGLD, with the largest gains in high-dimensional and multimodal settings.

顶级标签: machine learning model training
详细标签: variational inference optimal transport particle-based methods sinkhorn algorithm mode collapse 或 搜索:

基于熵输运下降的变分推断 / Variational Inference via Entropic Transport Descent


1️⃣ 一句话总结

本文提出了一种名为熵输运下降(ETD)的新型粒子变分推断方法,通过将每个粒子的更新构建为熵正则化的最优输运问题,有效解决了传统方法在高维和多模态分布上出现的方差崩溃与模式坍塌问题,仅需目标密度的点值即可实现全局协调的粒子演化。

源自 arXiv: 2606.25265