Hybrid Interpretable and Deep Learning Models for Intrusion Detection in Large-Scale Network Traffic

An Intelligent and Scalable Approach for Detecting Complex and Evolving Network Threats

Abstract

The fast growth of cyber-attacks and network traffic, have put forward the requirement of autonomous and scalable IDSs that can accurately discern among normal and malicious activities. In this paper, a hybrid machine learning (ML)-based IDS model, DTCNN-IDS, is presented by combining Decision Tree (DT), Convolutional Neural Network (CNN), and TabTransformer. The framework is tested against the KDD99 data set, containing 4,898,431 network records with continuous and categorical fields. A uniform pipeline with preprocessing, encoding, normalization, and multi-class supervised learning (M2A approach) allows for robust model evaluation. DT produces high accuracy (99.99%) but biased results on minority attacks (U2R recall = 0.72, R2L recall = 0.76) as a result of class imbalance. CNN enhances the nonlinear feature learning and achieves an accuracy of 99.7% with the precision, recall and F1-score of 0.996. The best-performing model is TabTransformer, achieving accuracy of 99.8%, precision of 0.997, recall of 0.998 and F1-score of 0.997, which also significantly improves detection of minority attacks. The improved sensitivity and stability are further confirmed by the Precision–Recall, scalability analyses and statistical testing (p < 0.05) validates the significance of results.