Advancing DDoS attack detection with hybrid deep learning: integrating convolutional neural networks, PCA, and vision transformers

In a DDoS attack, various compromised systems are used to exploit a targeted system, service, or network with a massive volume of incoming traffic, overwhelming its computational resources and making it inaccessible to actual users. DDoS attacks aim to interfere with a specific online service or website, making it unavailable to authorized users and harming the victim organization’s finances and reputation. DDoS attacks are usually organized by a network of seized devices or bots, generating a coordinated and dispersed attempt to overwhelm the target with traffic, making it challenging to stop the attack and identify its origin. DDoS attacks can be carried out for several purposes, such as extortion, retaliation, gaining an advantage over competitors, or ideological convictions. With advancements in technology, DDoS attacks have become easier to execute using automated platforms. Attackers can use quick automated tools to attack devices with weak protection. Notable examples include a massive DDoS attack on GitHub in early 2018, utilizing self-learning algorithms based on AI, which caused significant disruption with a capacity of 1.35 Tbps [16]. Additionally, an attack on Amazon Web Services (AWS) in 2019 resulted in an 8-hr outage, impacting the accessibility of cloud services, websites, and applications for numerous customers [17]. In February 2020, the most extensive DDoS attack to date affected Amazon cloud services, leading to 3 days of heightened threat. Another example is a data breach that allowed hackers to obtain the data of more than 533 million Facebook users, found by a white hat security organization in 2021 [18].

When attempting to counteract a DDoS attack, the most crucial thing to keep in mind is the difference between genuine and malicious traffic. There are various forms of DDoS attacks on the Internet. A Multi-vector DDoS attack uses many attack vectors to overload a target in different ways, which might divert attention away from mitigation efforts focused on a single trajectory. It takes a range of strategies to oppose various trajectories in a multi-vector DDoS attack. DDoS attack prevention can be challenging, but there are numerous effective techniques as follows.

Network breakdown. Networks can be divided into smaller, easier-to-manage segments to lessen the effect of DDoS attacks. Virtual local area networks (VLANs) may be constructed to do this, and firewalls can prevent an attack from spreading. The optimum approach is zero-trust micro-segmentation. Adding a firewall at the device level and hiding devices outside of the operating system continue to be the most dependable ways to prevent DDoS attacks.

DDoS attacks attempt to render a service unusable by saturating it with traffic from numerous sources. The ability to detect complex and sophisticated DDoS attacks is constrained by traditional detection techniques including signature-based and statistical-based approaches. Due to their capacity to automatically identify patterns and features from big datasets, to identify DDoS attacks DL-based techniques are efficacious. Following are a few steps to implement the proposed model:

Data collection. A large dataset of network traffic that includes both benign and DDoS attack traffic should be gathered. The dataset has to be varied and includes a range of DDoS attacks.

It is important to remember that the performance of DL-based DDoS attack detection is dependent on the data and model’s architecture. Thus, it is crucial to adhere to best practices at every stage of the procedure and continually assess and improve the system.

Anderson J.P. first introduced an IDS in 1980. He described an intrusion or threat as any effort to access information without authorization or change data in a way that renders the system inaccessible or unreliable. The intrusion detection expert system (IDES) was an IDS prototype that Denning [10] suggested in 1985. It is either a hardware or a software used to identify harmful activity on computer networks. An IDS’s main goal is to protect computer systems by identifying and recognizing various unsafe network traffic. To keep network confidentiality, integrity, availability, and functioning intact, it is essential to implement IDS into network security [3]. The general architecture of IDS in any computer network is shown in Figure 2.

Figure 2:

Network diagram depicting components involved in DDoS defense, including internet access points, firewalls, switches, IDS, and end-user devices. DDoS, distributed denial of service; IDS, intrusion detection systems.

There are several categories in which intrusion detection methods can be classified, primarily based on network infrastructure. An IDS can be classified on the basis of deployment into network and host-based [19]. Whereas on the basis of detection, it is classified as anomaly and signature-based. Figure 3 provides an overview of an IDS.

Anomaly based. Detection of anomalies in any network involves creating a standard profile that represents normal network behavior. It identifies any notable deviations from this established behavior, indicating potential anomalies. However, the disadvantage of utilizing an IDS based on anomalies is the possibility of generating a high number of false positive (FP) detections [19].

Figure 3:

Architectural overview of an IDS, detailing its core components, detection methodologies, data integration points, and response mechanisms. IDS, intrusion detection systems.

A CNN is an artificial neural network (ANN) with fully connected, pooling, and convolutional layers. These layers work together to turn the raw data into visually appealing representations. In order to find important patterns in the data, the convolutional and pooling layers in Figure 4 analyze the input first and produce various feature maps. The output of these layers is received by a fully connected layer, which classifies it. During training, the convolutional and fully connected layers’ weight parameters are optimized by the use of gradient descent. CNN provides the advantage of automated feature extraction, which has gained popularity in recent research. When using CNN in the context of 2D networks, additional preprocessing could be required to make the 1D input compatible with the network’s 2D structure, such as photos. For example, Jia et al. [20] presented a unique CNN architecture created especially for DDoS detection on datasets like KDD99, private datasets, and CICDoS2019.

Convolutional layer. In each convolutional layer there are sets of filters that convolve with input to produce feature maps. Each element (i, j) in the feature map is computed by applying the activation function to the sum of element-wise products of W and input x, plus the bias: h(i, j) = ReLU(b + ((W × x))).

Figure 4:

CNN. CNN, convolutional neural network.

The curse of dimensionality is common and large datasets are hard to interpret. PCA is used for dimensionality reduction of the dataset by creating new uncorrelated variables having maximum variance [1]. In PCA, the original dataset is projected in the direction of high variance, which is orthogonal to each other. We can reduce the computational costs by reducing the number of features. After standardizing the data, eigenvectors and eigenvalues are extracted through the covariance matrix. The number of steps for PCA is as follows : standardize the data, calculate the covariance matrix, find eigenvalues and eigenvectors, take the first K-eigenvectors having the highest eigenvalues, and in the end, project the dataset into k eigenvectors.

Class imbalance issues occur when a model is trained on an unbalanced dataset; it is a serious concern that causes biased outputs with a high false positive rate (FPRs) in ML and DL models. Techniques such as undersampling and oversampling are frequently utilized to address these issues. The SMOTE method is used in this research to address the class imbalance issues. Yizhen Jia et al. initially suggested SMOTE in Ref. [20]. It is very helpful when working with datasets where the majority class (DDoS attacks) outnumbers the minority class (normal traffic). The primary goal of SMOTE is to create synthetic samples for the minority class in order to balance the dataset and provide the model more representative training data. SMOTE is a useful tactic for handling imbalance class sizes.

The well-known DDoS-specific CICDoS2019 dataset, released by the Canadian Institute for Cybersecurity and publicly available on their website [33], is being utilized in this research. The CICDDoS2019 dataset detail is shown in Table 1 [34], which includes benign and wide-ranging DDoS attacks. Intrusion detection techniques in DL heavily rely on the availability of datasets. However, the scarcity of such datasets is due to privacy and regulatory concerns regarding the sensitive information present in network traffic. To overcome this challenge, researchers often resort to creating simulated data. However, these simulated datasets often lack completeness and do not adequately cover the range of application behaviors. Several public domain datasets have been widely used for IDS to overcome this issue.

There are two types of DDoS attacks. Response-based authentication that adheres to a standard protocol worldwide is called reflection-based authentication. It consists of domain name system (DNS), TFTP, and NetBIOS types. Another type of attack known as “exploitation-based” attack occurs when a hacker tries to harm oneself by taking advantage of a vulnerable system. It consists of SYN, UDP, and UDPLag. This dataset was selected mainly due to its high potential for future research applications and the fact that it is one of the most recent in the IDS field for detecting DDoS attacks. The dataset contains around 50 million data points; 2.2 million records (2 lac from each class) were selected due to the limitations of computational resources.

The removal of null values and outliers, normalization, dimensionality reduction, and resolving class imbalance issues are all included in data preprocessing. Certain features, such as SimilarHTTP, Flow ID, Source Port, Unnamed: 0, Source IP, Destination IP, Timestamp, Destination Port, and Label, were found redundant and removed from the dataset in order to prepare it for model training. The following standard procedures were used to preprocess the dataset.

Data cleaning. Removing redundant, NaN, or unnecessary data points from the dataset is one of the most crucial phases in lowering noise and raising the overall quality of the data. Moreover, the extremely low variance was reduced for 12 features: Idle Std, Max, Mean, Min, Inbound, min seg size forward, act data pkt fwd, Active Std, Min, Mean, Max, and Init Win bytes backward. Removing these data improves the model’s efficiency and reduces computational complexity.

Based on the variance and eigenvalues, the dataset’s dimensions were trimmed down to 20 features as per cumulated explained variance shown in Figure 5. PCA is used to reduce dimensionality and select the top 20 features. Models with reduced dimensionality perform better in terms of accuracy and sensitivity rates while consuming less processing power.

Dataset split. Because it might impact the outcomes, keeping the dataset balanced is essential for both training and testing the model. The Sklearn library is used to partition data: 70% of the data is utilized for training, while 30% is used for testing and validation.

Figure 5:

Proposed architecture processing network traffic data through CNN for DDoS attack detection, incorporating PCA and SMOTE for data preparation. CNN, convolutional neural network; DDoS, distributed denial of service; PCA, principal component analysis; SMOTE, Synthetic Minority Oversampling Technique.

In order to detect DDoS attacks using the CICDDoS2019 dataset for binary class classification, this research presents a DL-based CNN model. Initially, the CNN layer receives the data. Filters in the convolution layer extract the most significant features, producing a feature map. This is followed by a max pooling layer to reduce dimensionality. To prevent overfitting, a dropout layer is employed, followed by a sigmoid activation function to distinguish between DDoS and normal traffic. The proposed DL-based CNN model, shown in Figure 5, consists of two convolutional layers with 32 and 64 filters followed by max pooling layers for downsampling. Flattening and dense layers are added with L2 regularization to avoid overfitting. The final output layer uses a sigmoid activation function for binary classification shown in Figure 5. The hyperparameter values of the model are as follows:

Activation function: Relu, Sigmoid

In the study, a dual-path architecture was proposed to enhance the detection of DDoS attacks, utilizing both CNN and ViT. Initially, the network traffic data, characteristic of DDoS and benign activities, underwent a rigorous preprocessing phase in which features were normalized and encoded into a format suitable for image-like data processing. This preprocessed data was then fed simultaneously into two distinct model pathways: the CNN and ViT. The CNN pathway focused on leveraging spatial hierarchies through convolutional layers, which have been the primary focus of this paper. Concurrently, the ViT pathway processed the same data through its transformer layers, designed to capture complex patterns via self-attention mechanisms. Although the primary focus of the research remained on the CNN model, ViT was introduced to compare its performance against traditional CNN architectures. The comparative analysis revealed that ViT not only complemented CNN’s capabilities but also demonstrated superior performance in recognizing intricate patterns indicative of DDoS attacks, thereby affirming its potential as a powerful tool in cybersecurity defenses. The process of the methodology is mentioned in Figure 6.

Figure 6:

Proposed architecture diagram illustrating the dual-path approach for DDoS attack detection, featuring both CNN and ViT layers. CNN, convolutional neural network; DDoS, distributed denial of service; ViT, vision transformers.

The Kaggle open-source platform was utilized for CNN experiment in this research to evaluate the proposed hybrid DL-based model with the CICDDoS2019 dataset. Kaggle was chosen for its open-source design, cloud-based Jupyter Notebook environment, and free availability. On Kaggle, users have the opportunity to collaborate, access, and exchange datasets, utilize notebooks with graphics processing unit (GPU) support, and participate in competitions with fellow data scientists to tackle various data science challenges.

The Hugging Face Transformers library was utilized to apply a ViT model to the DDoS dataset, which had been previously transformed into an image-like format. The selected ViT model, initially pretrained on large-scale image datasets, was fine-tuned on the synthetic images derived from network traffic features characteristic of DDoS and benign traffic. This fine-tuning process involved adjusting the model’s top layers to suit the classification tasks, specifically to distinguish between DDoS attacks and normal activities. During the training phase, the model’s deeper layers were fine-tuned while the initial layers were kept frozen to leverage the generic features learned from natural images. The training was optimized through meticulous adjustments of learning rates and batch sizes, tailored to the specific patterns and anomalies present in network data. Upon completion, ViT achieved an improved accuracy rate, demonstrating the model’s efficacy in capturing complex, nuanced patterns in DDoS attack behaviors, which markedly enhanced the detection capabilities.

This study employs a number of criteria, including accuracy, precision, F1 score, sensitivity or recall, and duration, to assess the model’s efficacy. Table 2 offers a thorough synopsis of the classification findings along with a full explanation of the confusion matrix. The evaluation metrics are as follows.

True positive (TP): Represents the correctly detected attacks.

The following metrics are calculated using these mathematical equations:

Precision: It measures the ratio of correctly predicted attacks by all data categorized as attacks: Precision=TPTP+FP {\rm{Precision}} = {{{\rm{TP}}} \over {{\rm{TP}} + {\rm{FP}}}}

This section includes the results obtained by the CNN model and comparison with other state-of-the-art models. The proposed CNN model achieves superior results in classifying DDoS attacks. The CNN design consists of multiple layers, including pooling layers, convolution layers, and fully connected layers. The Sigmoid and Relu activation functions are used to classify objects by assigning probabilistic values. Each layer has a specific purpose and is responsible for extracting abstract features that are not easily visible to humans. The performance of the model was assessed using PCA as a dimensionality reduction method on the open-source Kaggle platform, using 30, 20, and 10 features; 2.2 million data points are selected from the dataset. Since the data were initially unbalanced, we employed the SMOTE approach, which produces 4.4 million data points by balancing the dataset’s normal and DDoS traffic.

As shown in Figure 7, the model’s highest accuracy was attained with 20 features using PCA and binary classification at 99.72%. The graph presents a detailed comparison of three CNN model configurations, CNN-30, CNN-20, and CNN-10, highlighting their performance on various metrics and training times. All models show a high level of accuracy, precision, recall, and F1 score, maintaining values around 99.6%–99.8%, indicating robust performance across configurations. The CNN-20 model slightly outperforms the others in test accuracy at 99.72%, although the differences are minimal, suggesting that all models are similarly effective. Notably, the training time decreases significantly from CNN-30 to CNN-10, from over 600 s to just about 323 s, illustrating a marked improvement in efficiency with simpler or reduced models. This decrease in training time without a significant drop in performance metrics suggests that less complex models may provide a more efficient solution while maintaining high accuracy and precision, making them suitable for environments where computational resources or time are constraints. Figure 8 also shows the confusion matrix.

Figure 7:

Performance and efficiency of CNN models: This graph shows key performance metrics—Test Accuracy, Training Accuracy, Precision, Recall, and F1 Score—across three CNN configurations (CNN-30, CNN-20, CNN-10). It also illustrates the reduction in training time for each model, emphasizing the balance between model accuracy and computational efficiency. CNN, convolutional neural networks.

Figure 8:

Confusion matrix. DDoS, distributed denial of service.

ViT represented a shift from CNN by applying the transformer architecture, originally designed for natural language processing, to image analysis. Instead of processing the entire image at once, ViTs divided the image into patches and treated each patch as a token similar to words in a sentence. This allowed the model to learn spatial hierarchies and apply self-attention mechanisms across patches, facilitating a deeper understanding of the relationships and contextual relevance within the image.

For applying a ViT to a DDoS dataset, the data had been transformed into a format that mimicked visual data. This involved encoding network traffic features – such as packet size, frequency, type, source/destination IPs — into image-like grid formats where each “pixel” or patch represented different aspects or combinations of network features. These images then served as input for ViT, allowing it to learn and identify patterns indicative of DDoS activity. The self-attention mechanism of ViT was particularly advantageous in this context, as it could highlight unusual patterns or anomalies in network traffic that were characteristic of DDoS attacks.

ViT outperforms the CNN models across several key performance metrics. It achieves a test accuracy of 99.99%, which is significantly higher compared with the highest among the CNN models (CNN-20) at 99.72%. This improvement in accuracy indicates ViT’s superior capability in handling complex pattern recognition tasks, potentially due to its attention mechanisms which allow it to focus more effectively on important features within the data. Furthermore, ViT also shows a notable improvement in Recall, at 99.95%, which is crucial for minimizing false negatives (FNs) — a critical aspect in DDoS detection where missing an attack can be costly. The F1 Score of ViT is also higher at 99.85%, suggesting a better balance between precision and recall compared with the CNN models. This efficiency, combined with superior performance, underscores ViT’s potential as a powerful tool in network security contexts, particularly in detecting and analyzing DDoS attacks. Further details on the implementation and technical specifications of ViT are beyond the scope of this paper and will be explored in-depth in a subsequent publication.