菜单

关于 🐙 GitHub
arXiv 提交日期: 2026-06-23
📄 Abstract - The Gentle Collapse: Distributional Metrics for Continual Learning

Accuracy degradation is the standard metric for Catastrophic Forgetting (CF), however, it records only whether forgetting occurred or not. It saturates at the extremes and collapses discretely at task boundaries, hiding the internal structure of what is being forgotten. We introduce six softmax-derived metrics spanning true-label rank (TLR), predictive confidence, and distributional divergence that characterize forgetting continuously, each normalized to [0, 1] with no modification to training. On CIFAR-100, these metrics carry information where accuracy does not: at 0% accuracy, the Confusion Margin spans an IQR of [0.32, 0.50] across classes that accuracy treats identically. We demonstrate that this richer signal is actionable in mitigating catastrophic forgetting. Per-sample metric scores used as loss weights reduce forgetting by 1.3 percentage points over uniform experience replay (ER) on CIFAR-100. Furthermore, the slope of a metric over a small window provides a stable sampling criterion: at a small-window size (e.g. 3 epochs), accuracy-trend degrades to 34.79% (std. = 2.32) while log-TLR achieves 41.07% (std. = 0.57). This gap is structural since reliable small-window trend estimation requires a continuous signal. On TinyImageNet, log-TLR trend sampling reduces forgetting by 7.7 percentage points over the ER baseline.

顶级标签: machine learning model evaluation
详细标签: continual learning catastrophic forgetting distributional metrics softmax analysis experience replay 或 搜索:

温和的崩塌:用于持续学习的分布度量 / The Gentle Collapse: Distributional Metrics for Continual Learning


1️⃣ 一句话总结

本文提出了一套基于softmax输出的连续度量指标(如真实标签排名、预测置信度和分布散度),能在分类准确率饱和或降为零时仍持续反映遗忘的内部结构,并利用这些指标的样本级分数和趋势斜率,有效缓解持续学习中的灾难性遗忘。

源自 arXiv: 2606.25165