菜单

关于 🐙 GitHub
arXiv 提交日期: 2026-02-11
📄 Abstract - Search or Accelerate: Confidence-Switched Position Beam Search for Diffusion Language Models

Diffusion Language Models (DLMs) generate text by iteratively denoising a masked sequence, repeatedly deciding which positions to commit at each step. Standard decoding follows a greedy rule: unmask the most confident positions, yet this local choice can lock the model into a suboptimal unmasking order, especially on reasoning-heavy prompts. We present SOAR, a training-free decoding algorithm that adapts its behavior to the model's uncertainty. When confidence is low, SOAR briefly widens the search over alternative unmasking decisions to avoid premature commitments; when confidence is high, it collapses the search and decodes many positions in parallel to reduce the number of denoising iterations. Across mathematical reasoning and code generation benchmarks (GSM8K, MBPP, HumanEval) on Dream-7B and LLaDA-8B, SOAR improves generation quality while maintaining competitive inference speed, offering a practical way to balance quality and efficiency in DLM decoding.

顶级标签: natural language processing model evaluation machine learning
详细标签: diffusion language models decoding algorithm beam search reasoning text generation 或 搜索:

搜索还是加速:用于扩散语言模型的置信度切换位置束搜索 / Search or Accelerate: Confidence-Switched Position Beam Search for Diffusion Language Models


1️⃣ 一句话总结

这篇论文提出了一种名为SOAR的新型解码算法,它能根据模型自身的不确定度动态切换搜索策略:在模型信心不足时进行更广泛的搜索以避免错误决策,在信心充足时则并行解码以加速生成,从而在数学推理和代码生成任务上,以接近的速度实现了更高质量的文本输出。

源自 arXiv: 2602.10953