反向传播作为幂零线性系统 / Backpropagation as a Nilpotent Linear System
1️⃣ 一句话总结
本文提出一种全新的数学视角,将深度学习中的反向传播过程重新表述为一个基于幂零算子的全局线性系统,从而揭示其与上三角矩阵直接求解的等价性,并深入解释了残差网络和迁移学习中的梯度传导机理。
Backpropagation is the computational engine of deep learning, yet its mathematical structure is typically treated as a procedural traversal of computational graphs. We present a global operator theory of the \emph{F-adjoint} framework, which reformulates the layerwise backward recursion of an $L$-depth feedforward network into a single linear system $(I-\cB)\Xs=\bG$, where $\bG$ is a source vector. We prove that the global backward operator $\cB$ is strictly block upper-triangular and nilpotent of index at most $L$. This nilpotency guarantees the exact termination of the Neumann series solution after at most $L$ terms, revealing classical backpropagation to be mathematically equivalent to block back-substitution on an upper bidiagonal system. We formalise \emph{F-symmetry} -- the condition in which the backward pass perfectly mirrors the forward pass -- identifying orthogonal weight matrices as canonical examples. Through worked numerical examples, we demonstrate how this operator perspective exposes the single-path collapse of strictly feedforward networks and its breakdown in residual architectures. Finally, we leverage this compositional structure to rigorously derive the mechanics of residual networks (gradient highways) and transfer learning (gradient truncation). This framework elevates backpropagation from an algorithmic recipe to a global nilpotent-operator formulation.
反向传播作为幂零线性系统 / Backpropagation as a Nilpotent Linear System
本文提出一种全新的数学视角,将深度学习中的反向传播过程重新表述为一个基于幂零算子的全局线性系统,从而揭示其与上三角矩阵直接求解的等价性,并深入解释了残差网络和迁移学习中的梯度传导机理。
源自 arXiv: 2607.11289