<abstract xmlns="http://www.w3.org/1999/xhtml">

<p>Identifying vulnerabilities within source code remains paramount in assuring software quality and security. This study introduces a refined semi-supervised learning methodology that capitalizes on pattern-exploiting training coupled with cloze-style interrogation techniques. The research strategy employed involves the training of a linguistic model on the SARD and Devign datasets, which are replete with vulnerable code fragments. The training procedure entails obscuring specific segments of the code and subsequently prompting the model to ascertain the obfuscated tokens. Empirical analyses underscore the efficacy of our method in pinpointing vulnerabilities in source code, benefiting substantially from patterns discerned within the code fragments. This investigation underscores the potential of integrating pattern-exploiting training and cloze-based queries to enhance the precision of vulnerability detection within source code.</p>
</abstract>

Identifying vulnerabilities within source code remains paramount in assuring software quality and security. This study introduces a refined semi-supervised learning methodology that capitalizes on pattern-exploiting training coupled with cloze-style interrogation techniques. The research strategy employed involves the training of a linguistic model on the SARD and Devign datasets, which are replete with vulnerable code fragments. The training procedure entails obscuring specific segments of the code and subsequently prompting the model to ascertain the obfuscated tokens. Empirical analyses underscore the efficacy of our method in pinpointing vulnerabilities in source code, benefiting substantially from patterns discerned within the code fragments. This investigation underscores the potential of integrating pattern-exploiting training and cloze-based queries to enhance the precision of vulnerability detection within source code.

SecuGuard: Leveraging pattern-exploiting training in language models for advanced software vulnerability detection

International Journal of Mathematics and Computer in Engineering

This work is licensed under the Creative Commons Attribution 4.0 International License.

Identifying vulnerabilities within source code remains paramount in assuring software quality and security. This study introduces a refined semi-supervised...

Line	Examples	Method	SARD	D2A	REVEAL	Devign

1.	\| T \|= 0	unsupervised (avg)	38.8±9.6	69.5±7.2	44.0±9.1	39.1±4.3
2.	\| T \|= 0	unsupervised (max)	42.8±0.0	79.4±0.0	56.4±0.0	43.8±0.0
3.	\| T \|= 0	iPet	66.7±0.2	89.5±0.1	73.7±0.1	63.6±0.1

4.	\| T \|= 15	supervised	32.1±1.6	25.0±0.1	10.1±0.1	34.2±2.1
5.	\| T \|= 15	Pet	52.9±0.1	87.5±0.0	63.8±0.2	41.8±0.1
6.	\| T \|= 15	iPet	57.6±0.0	89.3±0.1	70.7±0.1	43.2±0.0

7.	\| T \|= 60	supervised	44.8±2.7	82.1±2.5	52.5±3.1	45.6±1.8
8.	\| T \|= 60	Pet	60.0±0.1	86.3±0.0	66.2±0.1	63.9±0.0
9.	\| T \|= 60	iPet	64.7±0.1	88.4±0.1	69.7±0.0	67.4±0.3

10.	\| T \|= 200	supervised	53.0±3.1	86.0±0.7	62.9±0.9	47.9±2.8
11.	\| T \|= 200	Pet	61.9±0.0	88.3±0.1	69.2±0.0	74.7±0.3
12.	\| T \|= 200	iPet	62.9±0.0	89.6±0.1	71.2±0.1	78.4±0.7

13.	\| τ \|= 1000	supervised	63.0±0.5	86.9±0.4	70.5±0.3	73.1±0.2
14.	\| τ \|= 1000	Pet	68.8±0.1	89.9±0.2	72.7±0.0	85.3±0.2

Ex.	Method	SARD	D2A	REVEAL	Devign

\| T \|= 15	VulDeBERT	40.45	72.6	36.7	34.7
\| T \|= 15	VulBERTa	43.23	81.1	320.6	32.9
\| T \|= 15	Pet	49.60	84.1	59.0	39.5
\| T \|= 15	iPet	54.60	87.5	67.0	42.1

\| T \|= 60	VulDeBERT	46.6	83.0	60.2	40.8
\| T \|= 60	VulBERTa	39.5	84.8	61.5	34.8
\| T \|= 60	Pet	55.3	86.4	63.3	55.1
\| T \|= 60	iPet	57.7	87.3	69.6	56.3

SecuGuard: Leveraging pattern-exploiting training in language models for advanced software vulnerability detection

Article Category: Original Study

Pubblicato online: 02 giu 2024

Pagine: -

Ricevuto: 28 ott 2023

Accettato: 16 gen 2024

DOI: https://doi.org/10.2478/ijmce-2025-0005

Parole chiave
Language models, software vulnerabilities, vulnerability detection, cloze-style questions, pattern-exploiting training, RoBERTa

© 2025 Mahmoud Basharat et al., published by Sciendo

This work is licensed under the Creative Commons Attribution 4.0 International License.

Fig. 1

Fig. 2

The average accuracy and the standard deviation for BERT base on SARD, D2A, REVEAL and DEVIGN over 5 training set sizes.

A comparison of PET with VulBERTa and VulDeBERT methods using BERT (base).

SecuGuard: Leveraging pattern-exploiting training in language models for advanced software vulnerability detection

Article Category: Original Study

Pubblicato online: 02 giu 2024

Pagine: -

Ricevuto: 28 ott 2023

Accettato: 16 gen 2024

DOI: https://doi.org/10.2478/ijmce-2025-0005

Parole chiaveLanguage models, software vulnerabilities, vulnerability detection, cloze-style questions, pattern-exploiting training, RoBERTa

© 2025 Mahmoud Basharat et al., published by Sciendo

This work is licensed under the Creative Commons Attribution 4.0 International License.

Fig. 1

Fig. 2

The average accuracy and the standard deviation for BERT base on SARD, D2A, REVEAL and DEVIGN over 5 training set sizes.

A comparison of PET with VulBERTa and VulDeBERT methods using BERT (base).

Parole chiave
Language models, software vulnerabilities, vulnerability detection, cloze-style questions, pattern-exploiting training, RoBERTa