多任务学习的渐近行为:隐式正则化与双下降效应 / Asymptotic Behavior of Multi--Task Learning: Implicit Regularization and Double Descent Effects
1️⃣ 一句话总结
这篇论文通过理论分析和实验证明,多任务学习之所以能提升模型泛化能力,是因为它在数学上等效于给模型添加了额外的正则化项,从而能够延缓甚至消除训练过程中常见的‘双下降’过拟合现象。
Multi--task learning seeks to improve the generalization error by leveraging the common information shared by multiple related tasks. One challenge in multi--task learning is identifying formulations capable of uncovering the common information shared between different but related tasks. This paper provides a precise asymptotic analysis of a popular multi--task formulation associated with misspecified perceptron learning models. The main contribution of this paper is to precisely determine the reasons behind the benefits gained from combining multiple related tasks. Specifically, we show that combining multiple tasks is asymptotically equivalent to a traditional formulation with additional regularization terms that help improve the generalization performance. Another contribution is to empirically study the impact of combining tasks on the generalization error. In particular, we empirically show that the combination of multiple tasks postpones the double descent phenomenon and can mitigate it asymptotically.
多任务学习的渐近行为:隐式正则化与双下降效应 / Asymptotic Behavior of Multi--Task Learning: Implicit Regularization and Double Descent Effects
这篇论文通过理论分析和实验证明,多任务学习之所以能提升模型泛化能力,是因为它在数学上等效于给模型添加了额外的正则化项,从而能够延缓甚至消除训练过程中常见的‘双下降’过拟合现象。
源自 arXiv: 2603.05060