Research on Text Classification of Power Work Orders Based on RoBERTa-RCNN and Channel Attention Mechanism

Given the scarcity of algorithms for classifying power grid work order texts, coupled with challenges such as sparse features, lack of structure, and large data scale, this paper proposes a text classification model based on the Roberta-RCNN framework that incorporates a channel attention mechanism. The model employs the Roberta model for word vectors, masks entities and word semantic units, and links them to the Transformer encoding layer to encode the word embedding vectors output by the Roberta layer. This approach addresses model overfitting and enhances generalization ability. The RCNN model employs a channel attention mechanism to optimize feature extraction, allowing the model to focus more on learning feature representations from various channels, followed by softmax classification. Experimental results on the THUCNews short text dataset demonstrate improved evaluation metrics compared to existing methods. Comparative experiments on a self-compiled power grid work order dataset show that the proposed algorithm model achieves higher accuracy compared to other models. This indicates its effectiveness in extracting feature information from text and addressing the challenges of power grid work order text classification with promising classification performance.