<abstract xmlns="http://www.w3.org/1999/xhtml">

<p>In this study, we undertake a comprehensive examination of zero-shot text classification and its associated implications. We propose the adoption of the BERT model as a method for text feature representation. Subsequently, we utilize the Pointwise Mutual Information (PMI) metric to adjust the weight values within a graph convolutional neural network, thereby facilitating the construction of a text graph. Additionally, we incorporate an attention mechanism to transform this text graph, enabling it to represent the output labels of zero-shot text classification effectively. The experimental environment is set up, and the comparison and ablation experiments of the text classification model based on BERT and graph convolutional neural network with the baseline models are carried out in several different types of datasets, and the parameter settings of <italic>λ</italic> are adjusted according to the experimental results, and the convergence of the BERT model is compared to test the robustness of the model performance and the classification effect. When <italic>λ</italic> was set to 0.60, the model achieved the best results in each dataset. When the task is set to 5-way-5-shot, the convergence rate of the model for the Snippets dataset using the penultimate layer of features can reach 74%-80% of the training accuracy at the 5,000th step. The training accuracy gradually flattens out in the first 10,000 steps, and the model achieves classification accuracy in all four learning scenarios, with good stability.</p>
</abstract>

In this study, we undertake a comprehensive examination of zero-shot text classification and its associated implications. We propose the adoption of the BERT model as a method for text feature representation. Subsequently, we utilize the Pointwise Mutual Information (PMI) metric to adjust the weight values within a graph convolutional neural network, thereby facilitating the construction of a text graph. Additionally, we incorporate an attention mechanism to transform this text graph, enabling it to represent the output labels of zero-shot text classification effectively. The experimental environment is set up, and the comparison and ablation experiments of the text classification model based on BERT and graph convolutional neural network with the baseline models are carried out in several different types of datasets, and the parameter settings of λ are adjusted according to the experimental results, and the convergence of the BERT model is compared to test the robustness of the model performance and the classification effect. When λ was set to 0.60, the model achieved the best results in each dataset. When the task is set to 5-way-5-shot, the convergence rate of the model for the Snippets dataset using the penultimate layer of features can reach 74%-80% of the training accuracy at the 5,000th step. The training accuracy gradually flattens out in the first 10,000 steps, and the model achieves classification accuracy in all four learning scenarios, with good stability.

Zero-sample text classification algorithm based on BERT and graph convolutional neural network

School of Computer Science and Software Engineering

PetroChina Changqing Oilfield Company Oil Production PLANT NO.7

Applied Mathematics and Nonlinear Sciences

This work is licensed under the Creative Commons Attribution 4.0 International License.

In this study, we undertake a comprehensive examination of zero-shot text classification and its associated implications. We propose the adoption of the BERT...

Zero-sample text classification algorithm based on BERT and graph convolutional neural network

Published Online: Jul 02, 2024

Page range: -

Received: Mar 09, 2024

Accepted: May 30, 2024

DOI: https://doi.org/10.2478/amns-2024-1560

Keywords
BERT model, Graph convolutional neural network, Attention mechanism, Baseline model, Text classification

© 2024 Ying Qiao et al., published by Sciendo

This work is licensed under the Creative Commons Attribution 4.0 International License.

Zero-sample text classification algorithm based on BERT and graph convolutional neural network

Published Online: Jul 02, 2024

Page range: -

Received: Mar 09, 2024

Accepted: May 30, 2024

DOI: https://doi.org/10.2478/amns-2024-1560

KeywordsBERT model, Graph convolutional neural network, Attention mechanism, Baseline model, Text classification

© 2024 Ying Qiao et al., published by Sciendo

This work is licensed under the Creative Commons Attribution 4.0 International License.

Keywords
BERT model, Graph convolutional neural network, Attention mechanism, Baseline model, Text classification