注意力频率调制:扩散交叉注意力的免训练频谱调制 / Attention Frequency Modulation: Training-Free Spectral Modulation of Diffusion Cross-Attention
1️⃣ 一句话总结
这篇论文提出了一种名为‘注意力频率调制’的新方法,它通过分析并调整AI文生图模型中‘注意力’信号在不同频率上的分布,无需重新训练就能精细地控制生成图像的细节风格和构图,比如让画面更聚焦于整体轮廓或局部细节。
Cross-attention is the primary interface through which text conditions latent diffusion models, yet its step-wise multi-resolution dynamics remain under-characterized, limiting principled training-free control. We cast diffusion cross-attention as a spatiotemporal signal on the latent grid by summarizing token-softmax weights into token-agnostic concentration maps and tracking their radially binned Fourier power over denoising. Across prompts and seeds, encoder cross-attention exhibits a consistent coarse-to-fine spectral progression, yielding a stable time-frequency fingerprint of token competition. Building on this structure, we introduce Attention Frequency Modulation (AFM), a plug-and-play inference-time intervention that edits token-wise pre-softmax cross-attention logits in the Fourier domain: low- and high-frequency bands are reweighted with a progress-aligned schedule and can be adaptively gated by token-allocation entropy, before the token softmax. AFM provides a continuous handle to bias the spatial scale of token-competition patterns without retraining, prompt editing, or parameter updates. Experiments on Stable Diffusion show that AFM reliably redistributes attention spectra and produces substantial visual edits while largely preserving semantic alignment. Finally, we find that entropy mainly acts as an adaptive gain on the same frequency-based edit rather than an independent control axis.
注意力频率调制:扩散交叉注意力的免训练频谱调制 / Attention Frequency Modulation: Training-Free Spectral Modulation of Diffusion Cross-Attention
这篇论文提出了一种名为‘注意力频率调制’的新方法,它通过分析并调整AI文生图模型中‘注意力’信号在不同频率上的分布,无需重新训练就能精细地控制生成图像的细节风格和构图,比如让画面更聚焦于整体轮廓或局部细节。
源自 arXiv: 2603.28114