Detecting and mitigating botnet attacks using deep learning in software-defined networks

• Main Author: Muhammad Waqas, Nadeem
• Format: Final Year Project / Dissertation / Thesis
• Published: 2023
• Subjects: H Social Sciences (General); HM Sociology; T Technology (General); TA Engineering (General). Civil engineering (General)
• Online Access: http://eprints.utar.edu.my/6246/1/CCA_MWN_2023.pdf; http://eprints.utar.edu.my/6246/

Software-Defined Networking (SDN) is a newly emerging network architecture separating control and data planes. It provides easy and flexible organization, management, and communication of complex or large-scale networks. Its programmable and centralized interfaces facilitate making complex and intelligent network decisions seamlessly and dynamically and can address the requirements of the data centers for managing the entire network. It also provides opportunities for individuals and businesses to build custom network applications based on their requirements and enhance their network services. Although SDN architecture offers high benefits, it introduced a new array of security and privacy challenges (i.e., single point of failure) that can preclude the wide adoption of SDNs. The SDN controller is a crucial element that attracts attackers to launch malicious attacks or activities on the controller (s) through OpenFlow switches. Distributed Denial of Service (DDoS) and botnet attacks are considered dangerous threats for networks such as IoT, SDNs, cloud computing, etc. If the attacker accesses the SDN controller, it can reroute the network traffic, causing severe damage to the whole network. So, Network Intrusion Detection Systems (NIDSs) have become important tools to protect networks against malicious attacks. Deep learning (DL)- based network applications are trending and have shown promising results in detecting and mitigating potential threats with fast response. In this research, we analyze and show the classification performance in terms of detecting and real-time performance of various DL methods based on Recurrent Neural Networks (RNNs), Convolution Neural Networks (CNNs), Multilayer Perceptron (MLP), Deep Neural Networks (DNNs), and Long Short Term Memory (LSTM) for botnet-based DDoS attacks in an SDN environment. A new simulation-based dataset is developed and used to train deep learning methods. We also used feature weighting and threshold tuning methods to derive the significant features required for detection. The simulation outcomes and measurements are verified using a simulation-based dataset and a real-time testbed environment. The aim of comparative analysis among the DL methods is to find the lightweight DL method with baseline hyperparameters, features and data that can be easily acquired to detect botnet-based DDoS attacks. The performance of the methods is evaluated using different metrics such as accuracy, detection rate, training and detection times, precision, F1 score, True Positive Rate (TPR), and False Positive Rate (FPR). The outcomes proved that the DL methods produced good results using optimal features. Finally, based on the simulation results, we observed that the CNN method outperforms using the simulated dataset and in real testbed settings. The detection rate of CNN reaches 97% for attack flows and 99% for normal flows. We also adopted graph theory and dynamic flow deletion-based mitigation strategy to protect the SDN environment against botnet attacks.