Statistical methods for feature selection: unlocking the key to improved accuracy

Abstract

The ever-growing amount of data generated by modern networks poses significant challenges for intrusion detection systems (IDS) in effectively analyzing and classifying security risks. Therefore, it is crucial to identify the most biased characteristics for building efficient and effective IDS algorithms. However, not all features are equally informative or relevant for intrusion detection. In response to these problems, this study proposes a Hybrid approach that uses traditional and advanced statistical techniques. The proposed method effectively validates the features generated from the hybrid model and set-operation theorem to provide the best optimal subset of features for IDS. Various machine learning methods are used to test the proposed model on three popular IDS datasets: NSL-KDD, UNSW NB15, and CIC-DDoS2019. The experimental findings show that the suggested hybrid technique improves IDS performance effectively and efficiently, providing a viable answer to the issues that intrusion detection systems confront.

KEYWORDS:

• Intrusion detection system (IDS)
• filter method
• wrapper method
• Anderson–Darling test

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The proposed model employs four publicly available datasets, which can be accessed and downloaded via the following link:

1. NSL-KDD dataset: link: http://205.174.165.80/CICDataset/NSL-KDD/Dataset/.

2. UNSW NB15 dataset: link: https://research.unsw.edu.au/projects/unsw-nb15-data-set.

3. CIC-DDoS2019 dataset: link: http://205.174.165.80/CICDataset/CICDDoS2019/Dataset/.

Additional information

Notes on contributors

Bidyapati Thiyam

Ms. Bidyapati Thiyam is currently a Ph.D. student at National Institute of Technology Nagaland, India. She received a Master's degree in Information Science and Engineering from B.M.S. College of Engineering, Bangalore, India in 2015. Her research interest includes IoT, Network Security and Intrusion Detection System (IDS). She can be reached at [email protected].

Shouvik Dey

Shouvik Dey is presently Associate Professor in the Department of Computer Science & Engineering at National Institute of Technology Nagaland, India. He received his Ph.D. degree in 2012 from Jadavpur University, India. His research interests are Internet of Things, Distributed Systems. He can be reached at [email protected]