流式持续学习中的时间任务化:评估不稳定的来源 / Temporal Taskification in Streaming Continual Learning: A Source of Evaluation Instability
1️⃣ 一句话总结
该论文指出,在流式持续学习中,将连续数据流切分成离散任务的时间划分方式并非中性预处理,而是一个会显著影响模型评估结果的主动变量,不同的切分方式会导致截然不同的性能结论,因此应将其视为评估中必须考虑的关键因素。
Streaming Continual Learning (CL) typically converts a continuous stream into a sequence of discrete tasks through temporal partitioning. We argue that this temporal taskification step is not a neutral preprocessing choice, but a structural component of evaluation: different valid splits of the same stream can induce different CL regimes and therefore different benchmark conclusions. To study this effect, we introduce a taskification-level framework based on plasticity and stability profiles, a profile distance between taskifications, and Boundary-Profile Sensitivity (BPS), which diagnoses how strongly small boundary perturbations alter the induced regime before any CL model is trained. We evaluate continual finetuning, Experience Replay, Elastic Weight Consolidation, and Learning without Forgetting on network traffic forecasting with CESNET-Timeseries24, keeping the stream, model, and training budget fixed while varying only the temporal taskification. Across 9-, 30-, and 44-day splits, we observe substantial changes in forecasting error, forgetting, and backward transfer, showing that taskification alone can materially affect CL evaluation. We further find that shorter taskifications induce noisier distribution-level patterns, larger structural distances, and higher BPS, indicating greater sensitivity to boundary perturbations. These results show that benchmark conclusions in streaming CL depend not only on the learner and the data stream, but also on how that stream is taskified, motivating temporal taskification as a first-class evaluation variable.
流式持续学习中的时间任务化:评估不稳定的来源 / Temporal Taskification in Streaming Continual Learning: A Source of Evaluation Instability
该论文指出,在流式持续学习中,将连续数据流切分成离散任务的时间划分方式并非中性预处理,而是一个会显著影响模型评估结果的主动变量,不同的切分方式会导致截然不同的性能结论,因此应将其视为评估中必须考虑的关键因素。
源自 arXiv: 2604.21930