自适应信号复苏:面向稀疏视觉网络的通道级剪枝后修复 / Adaptive Signal Resuscitation: Channel-wise Post-Pruning Repair for Sparse Vision Networks
1️⃣ 一句话总结
本文提出了一种无需重新训练、仅需少量校准数据的通道级后剪枝修复方法,通过对每个输出通道独立估计并自适应抑制不可靠的方差匹配校正,有效解决了高稀疏度下全局剪枝导致的精度崩溃问题。
One-shot magnitude pruning can cause severe accuracy collapse in the high-sparsity regime, even when the pruning mask preserves the largest weights. We argue that this failure reflects a granularity mismatch in post-pruning repair. Under global magnitude pruning, nearly collapsed channels can coexist with channels that retain informative activation variance within the same layer. Existing layer-wise activation repair methods apply a single correction to the whole layer, and can therefore over-amplify damaged channels while trying to restore the layer-level signal. We propose Adaptive Signal Resuscitation (ASR), a training-free channel-wise repair method that matches the granularity of repair to the granularity of damage. ASR estimates a variance-matching correction for each output channel and stabilizes it with a data-driven shrinkage rule, suppressing unreliable corrections for channels with weak post-pruning signal while preserving corrections for healthier channels. Applied before BatchNorm recalibration, ASR requires only forward passes on a small calibration set and no retraining. Across three datasets, four convolutional architectures, and both unstructured and structured sparsity settings, ASR generally improves over layer-wise repair, with the clearest gains in high-sparsity regimes. On ResNet-50 at 90% sparsity, ASR recovers 55.6% top-1 accuracy on CIFAR-10, compared with 41.0% for layer-wise repair and 28.0% for BatchNorm-only recalibration. Ablations show that naive channel-wise variance matching is insufficient, and that shrinkage stabilizes post-pruning repair.
自适应信号复苏:面向稀疏视觉网络的通道级剪枝后修复 / Adaptive Signal Resuscitation: Channel-wise Post-Pruning Repair for Sparse Vision Networks
本文提出了一种无需重新训练、仅需少量校准数据的通道级后剪枝修复方法,通过对每个输出通道独立估计并自适应抑制不可靠的方差匹配校正,有效解决了高稀疏度下全局剪枝导致的精度崩溃问题。
源自 arXiv: 2605.21426