专家乘积训练减少自然语言推理中的数据集伪影 / Product-of-Experts Training Reduces Dataset Artifacts in Natural Language Inference
1️⃣ 一句话总结
这篇论文提出了一种名为“专家乘积”（PoE）的训练方法，通过降低模型对数据集中虚假相关性的依赖，有效减少了自然语言推理任务中常见的偏见，在不显著损失准确率的前提下提升了模型的真实推理能力。
arXiv 提交日期: 2026-04-21
Abstract - Product-of-Experts Training Reduces Dataset Artifacts in Natural Language Inference
Neural NLI models overfit dataset artifacts instead of truly reasoning. A hypothesis-only model gets 57.7% in SNLI, showing strong spurious correlations, and 38.6% of the baseline errors are the result of these artifacts. We propose Product-of-Experts (PoE) training, which downweights examples where biased models are overconfident. PoE nearly preserves accuracy (89.10% vs. 89.30%) while cutting bias reliance by 4.71% (bias agreement 49.85% to 45%). An ablation finds lambda = 1.5 that best balances debiasing and accuracy. Behavioral tests still reveal issues with negation and numerical reasoning.
源自 arXiv: 2604.19069