菜单

关于 🐙 GitHub
arXiv 提交日期: 2026-02-10
📄 Abstract - Knowledge Integration Decay in Search-Augmented Reasoning of Large Language Models

Modern Large Language Models (LLMs) have demonstrated remarkable capabilities in complex tasks by employing search-augmented reasoning to incorporate external knowledge into long chains of thought. However, we identify a critical yet underexplored bottleneck in this paradigm, termed Knowledge Integration Decay (KID). Specifically, we observe that as the length of reasoning generated before search grows, models increasingly fail to integrate retrieved evidence into subsequent reasoning steps, limiting performance even when relevant information is available. To address this, we propose Self-Anchored Knowledge Encoding (SAKE), a training-free inference-time strategy designed to stabilize knowledge utilization. By anchoring retrieved knowledge at both the beginning and end of the reasoning process, SAKE prevents it from being overshadowed by prior context, thereby preserving its semantic integrity. Extensive experiments on multi-hop QA and complex reasoning benchmarks demonstrate that SAKE significantly mitigates KID and improves performance, offering a lightweight yet effective solution for knowledge integration in agentic LLMs.

顶级标签: llm natural language processing model evaluation
详细标签: knowledge integration reasoning retrieval-augmented generation inference-time strategy multi-hop qa 或 搜索:

大语言模型搜索增强推理中的知识整合衰减 / Knowledge Integration Decay in Search-Augmented Reasoning of Large Language Models


1️⃣ 一句话总结

这篇论文发现,当大语言模型在搜索外部知识前进行过长的推理时,会逐渐忘记使用搜到的信息,导致性能下降,并提出了一种无需额外训练、在推理时就能将关键知识固定在推理过程首尾的方法来有效解决这个问题。

源自 arXiv: 2602.09517