菜单

关于 🐙 GitHub
arXiv 提交日期: 2026-06-09
📄 Abstract - Do Transformers Actually Help Intrusion Detection? A Temporal Sequence Evaluation on CIC-IDS2017

Recent deep learning approaches for network intrusion detection increasingly incorporate temporal architectures such as recurrent networks and Transformers, often reporting near-perfect performance on CIC-IDS2017. However, many existing studies neither supply their temporal modules with genuine sequence inputs nor evaluate under realistic, leakage-free conditions, making it unclear whether reported gains arise from true sequence-modeling capability. In this work, we reformulate CIC-IDS2017 as a temporal intrusion-detection task by constructing ordered flow sequences from network conversations and benchmarking nine classical and deep learning architectures under a random split, two leakage-free splits, and a padding-scheme ablation. The central finding is that padding convention, not architecture, determines the Transformer's performance: on genuinely sequential (non-padded) windows the Transformer achieves the highest macro-F1 of any model in the experiment (0.89); under zero-pad+mask evaluation it drops markedly (-0.24 macro-F1), while LSTM, GRU, and 1D-CNN remain stable. Under leakage-free group evaluation the Random Forest is the most robust model (+0.009), while the Transformer's false-alarm rate grows from 0.04% to 2.7%, a 67-fold increase invisible under conventional protocols. These findings demonstrate that evaluation methodology -- specifically padding convention and split protocol -- has a larger effect on reported performance than architectural choice, and that widely used random splits with repeat-last padding can overestimate model robustness by up to 0.24 macro-F1. We advocate leakage-free splits, explicit padding disclosure, and sequence-aware benchmarking as standard practice in future IDS research. Code and implementation details are available at this https URL.

顶级标签: systems machine learning model evaluation
详细标签: intrusion detection temporal sequence padding scheme leakage-free evaluation benchmark 或 搜索:

Transformer真的有助于入侵检测吗?基于CIC-IDS2017的时间序列评估 / Do Transformers Actually Help Intrusion Detection? A Temporal Sequence Evaluation on CIC-IDS2017


1️⃣ 一句话总结

本文通过严格的时间序列评估发现,在入侵检测任务中,Transformer的性能提升主要源于数据填充方式而非其模型架构,传统评估方法因使用随机拆分和重复填充而高估了模型效果,因此建议采用无泄漏的数据拆分并明确报告填充策略。

源自 arXiv: 2606.11098