📄
Abstract - The Labyrinth and the Thread: Rethinking Regularizations in Sequential Knowledge Editing for Large Language Models
Sequential editing of structured knowledge in large language models allows targeted factual updates without retraining, yet existing methods often rely on complex regularization or constraint mechanisms whose necessity remains unclear. In this work, we systematically investigate the mechanisms underlying effective and stable sequential editing. Specifically, we first analyze the empirical success of AlphaEdit and establish, via a rigorous optimization analysis, the formal equivalence between one-time and sequential editing. Building on this insight, we generalize the equivalence to a broader class of editing objectives, demonstrating that stability emerges naturally from properly accounting for accumulated editing constraints, rather than from specialized regularization or null-space operations. We empirically confirm that many commonly used regularization strategies are unnecessary for reliable sequential updates. Furthermore, we extend our framework to handle conflicting edits, ensuring robust and consistent behavior under contradictory updates. Ultimately, our work provides Ariadne's thread through the labyrinth of sequential editing, charting a path toward simpler, more interpretable, and dependable knowledge updates. Our code is available at this https URL.
迷宫与线:重新思考大型语言模型顺序知识编辑中的正则化机制 /
The Labyrinth and the Thread: Rethinking Regularizations in Sequential Knowledge Editing for Large Language Models
1️⃣ 一句话总结
本文通过严格的数学分析发现,大型语言模型进行多次知识更新时,许多常用的复杂正则化技巧其实是不必要的,只要正确累积每次修改的约束条件就能自然保证更新稳定,从而为简化模型知识编辑提供了更清晰、可靠的指导。