<abstract xmlns="http://www.w3.org/1999/xhtml">

<p>In the rapidly-evolving landscape of software development, the detection of vulnerabilities in source code has become of paramount importance. Our study introduces a novel knowledge distillation (KD) technique aimed at enhancing vulnerability detection in software codebases. Using benchmark datasets such as SARD, SeVC, Devign, and D2A, we assess the prowess of the KD method when applied to different classifiers, specifically GPT2, CodeBERT, and LSTM. The empirical results are revealed a marked improvement in the performance of these classifiers upon the implementation of the KD technique, particularly with the GPT-2 model demonstrating the most promising outcomes. This work underscores the potential of integrating transformer-based learning models, like GPT-2, with knowledge distillation for more efficient and accurate vulnerability detection.</p>
</abstract>

In the rapidly-evolving landscape of software development, the detection of vulnerabilities in source code has become of paramount importance. Our study introduces a novel knowledge distillation (KD) technique aimed at enhancing vulnerability detection in software codebases. Using benchmark datasets such as SARD, SeVC, Devign, and D2A, we assess the prowess of the KD method when applied to different classifiers, specifically GPT2, CodeBERT, and LSTM. The empirical results are revealed a marked improvement in the performance of these classifiers upon the implementation of the KD technique, particularly with the GPT-2 model demonstrating the most promising outcomes. This work underscores the potential of integrating transformer-based learning models, like GPT-2, with knowledge distillation for more efficient and accurate vulnerability detection.

From text to threats: A language model approach to software vulnerability detection

College of Computing, Illinois Institute of Technology

International Journal of Mathematics and Computer in Engineering

This work is licensed under the Creative Commons Attribution 4.0 International License.

In the rapidly-evolving landscape of software development, the detection of vulnerabilities in source code has become of paramount importance. Our study...

From text to threats: A language model approach to software vulnerability detection

Article Category: Original Study

Published Online: Oct 31, 2023

Page range: 23 - 34

Received: Sep 04, 2023

Accepted: Oct 26, 2023

DOI: https://doi.org/10.2478/ijmce-2024-0003

Keywords
Knowledge distillation, DistilVulBERT, software vulnerability detection, language models, codebase analysis

© 2024 Marwan Omar et al., published by Sciendo

This work is licensed under the Creative Commons Attribution 4.0 International License.

From text to threats: A language model approach to software vulnerability detection

Article Category: Original Study

Published Online: Oct 31, 2023

Page range: 23 - 34

Received: Sep 04, 2023

Accepted: Oct 26, 2023

DOI: https://doi.org/10.2478/ijmce-2024-0003

KeywordsKnowledge distillation, DistilVulBERT, software vulnerability detection, language models, codebase analysis

© 2024 Marwan Omar et al., published by Sciendo

This work is licensed under the Creative Commons Attribution 4.0 International License.

Keywords
Knowledge distillation, DistilVulBERT, software vulnerability detection, language models, codebase analysis