Large language models (LLMs) increasingly store user preferences in persistent memory to support personalization across interactions. However, in third-party communication settings governed by social and institutional norms, some user preferences may be inappropriate to apply. We introduce BenchPreS, which evaluates whether memory-based user preferences are appropriately applied or suppressed across communication contexts. Using two complementary metrics, Misapplication Rate (MR) and Appropriate Application Rate (AAR), we find even frontier LLMs struggle to apply preferences in a context-sensitive manner. Models with stronger preference adherence exhibit higher rates of over-application, and neither reasoning capability nor prompt-based defenses fully resolve this issue. These results suggest current LLMs treat personalized preferences as globally enforceable rules rather than as context-dependent normative signals.
BenchPreS：一个用于评估持久记忆大语言模型上下文感知个性化偏好选择性的基准 / BenchPreS: A Benchmark for Context-Aware Personalized Preference Selectivity of Persistent-Memory LLMs
这篇论文提出了一个名为BenchPreS的新基准，用于测试大语言模型能否根据不同的社交和制度情境，智能地选择应用或抑制存储在记忆中的用户个性化偏好，结果发现即使是顶尖模型也常常错误地将偏好当作普适规则来使用。
源自 arXiv: 2603.16557
