菜单

关于 🐙 GitHub
arXiv 提交日期: 2026-03-15
📄 Abstract - Emotional Cost Functions for AI Safety: Teaching Agents to Feel the Weight of Irreversible Consequences

Humans learn from catastrophic mistakes not through numerical penalties, but through qualitative suffering that reshapes who they are. Current AI safety approaches replicate none of this. Reward shaping captures magnitude, not meaning. Rule-based alignment constrains behaviour, but does not change it. We propose Emotional Cost Functions, a framework in which agents develop Qualitative Suffering States, rich narrative representations of irreversible consequences that persist forward and actively reshape character. Unlike numerical penalties, qualitative suffering states capture the meaning of what was lost, the specific void it creates, and how it changes the agent's relationship to similar future situations. Our four-component architecture - Consequence Processor, Character State, Anticipatory Scan, and Story Update is grounded in one principle. Actions cannot be undone and agents must live with what they have caused. Anticipatory dread operates through two pathways. Experiential dread arises from the agent's own lived consequences. Pre-experiential dread is acquired without direct experience, through training or inter-agent transmission. Together they mirror how human wisdom accumulates across experience and culture. Ten experiments across financial trading, crisis support, and content moderation show that qualitative suffering produces specific wisdom rather than generalised paralysis. Agents correctly engage with moderate opportunities at 90-100% while numerical baselines over-refuse at 90%. Architecture ablation confirms the mechanism is necessary. The full system generates ten personal grounding phrases per probe vs. zero for a vanilla LLM. Statistical validation (N=10) confirms reproducibility at 80-100% consistency.

顶级标签: agents ai safety systems
详细标签: emotional cost functions qualitative suffering irreversible consequences agent character anticipatory dread 或 搜索:

AI安全的情感代价函数:教导智能体感受不可逆后果之重 / Emotional Cost Functions for AI Safety: Teaching Agents to Feel the Weight of Irreversible Consequences


1️⃣ 一句话总结

这篇论文提出了一种名为‘情感代价函数’的新AI安全框架,它让智能体通过建立‘定性痛苦状态’来深刻理解并内化其行为的不可逆后果,从而获得类似人类的、基于具体情境的智慧,而非仅仅是数字惩罚或行为约束。

源自 arXiv: 2603.14531