作为局部到全局计算的神经网络 / Neural Networks as Local-to-Global Computations
1️⃣ 一句话总结
这篇论文提出了一种将前馈神经网络重新解释为一种称为“层状结构”的数学对象的新视角,从而揭示了网络计算本质上是一种从局部约束到全局协调的调和过程,并基于此开发了无需反向传播的局部训练方法和新的网络行为诊断工具。
We construct a cellular sheaf from any feedforward ReLU neural network by placing one vertex for each intermediate quantity in the forward pass and encoding each computational step - affine transformation, activation, output - as a restriction map on an edge. The restricted coboundary operator on the free coordinates is unitriangular, so its determinant is $1$ and the restricted Laplacian is positive definite for every activation pattern. It follows that the relative cohomology vanishes and the forward pass output is the unique harmonic extension of the boundary data. The sheaf heat equation converges exponentially to this output despite the state-dependent switching introduced by piecewise linear activations. Unlike the forward pass, the heat equation propagates information bidirectionally across layers, enabling pinned neurons that impose constraints in both directions, training through local discrepancy minimization without a backward pass, and per-edge diagnostics that decompose network behavior by layer and operation type. We validate the framework experimentally on small synthetic tasks, confirming the convergence theorems and demonstrating that sheaf-based training, while not yet competitive with stochastic gradient descent, obeys quantitative scaling laws predicted by the theory.
作为局部到全局计算的神经网络 / Neural Networks as Local-to-Global Computations
这篇论文提出了一种将前馈神经网络重新解释为一种称为“层状结构”的数学对象的新视角,从而揭示了网络计算本质上是一种从局部约束到全局协调的调和过程,并基于此开发了无需反向传播的局部训练方法和新的网络行为诊断工具。
源自 arXiv: 2603.14831