ALIGNBEAM:通过跨词汇表对数混合实现推理时的安全对齐迁移 / ALIGNBEAM : Inference-Time Alignment Transfer via Cross-Vocabulary Logit Mixing
1️⃣ 一句话总结
本文提出ALIGNBEAM方法,无需重新训练模型,通过在每次解码时将一个安全模型的预测信号翻译并融入目标模型,从而在推理过程中将安全对齐能力从一种语言模型家族迁移到另一种,有效提升了微调后模型对有害指令的拒答率。
Domain fine-tuning degrades the safety of large language models: fine-tuned specialists readily comply with harmful prompts framed in domain language. Existing inference-time defenses that mix logits from a safe anchor model require both models to share a vocabulary, which rules them out for the cross-family specialists where safety is most degraded. We present ALIGNBEAM, a training-free method that lifts this restriction by translating anchor logits into the target model's vocabulary token-by-token at each decoding step; a small LLM judge then selects the safest among K candidate continuations. No weights are changed, and the safety-utility trade-off can be tuned at deployment without retraining. Across both cross-vocabulary and same-vocabulary evaluation pairs, ALIGNBEAM substantially raises refusal on adversarial benchmarks while keeping task accuracy and inference overhead within practical bounds. The results show that safety alignment can be transferred between model families at inference time, without touching either model's weights.
ALIGNBEAM:通过跨词汇表对数混合实现推理时的安全对齐迁移 / ALIGNBEAM : Inference-Time Alignment Transfer via Cross-Vocabulary Logit Mixing
本文提出ALIGNBEAM方法,无需重新训练模型,通过在每次解码时将一个安全模型的预测信号翻译并融入目标模型,从而在推理过程中将安全对齐能力从一种语言模型家族迁移到另一种,有效提升了微调后模型对有害指令的拒答率。
源自 arXiv: 2606.12342