1. bookVolume 5 (2021): Issue 3 (July 2021)
    Special Issue: Extraction and Evaluation of Knowledge Entities from Scientific Documents
Journal Details
License
Format
Journal
First Published
30 Mar 2017
Publication timeframe
4 times per year
Languages
English
access type Open Access

Automatic Subject Classification of Public Messages in E-government Affairs

Published Online: 14 Apr 2021
Page range: 336 - 347
Received: 30 Oct 2020
Accepted: 10 Feb 2021
Journal Details
License
Format
Journal
First Published
30 Mar 2017
Publication timeframe
4 times per year
Languages
English
Abstract

Public messages on the Internet political inquiry platform rely on manual classification, which has the problems of heavy workload, low efficiency, and high error rate. A Bi-directional long short-term memory (Bi-LSTM) network model based on attention mechanism was proposed in this paper to realize the automatic classification of public messages. Considering the network political inquiry data set provided by the BdRace platform as samples, the Bi-LSTM algorithm is used to strengthen the correlation between the messages before and after the training process, and the semantic attention to important text features is strengthened in combination with the characteristics of attention mechanism. Feature weights are integrated through the full connection layer to carry out classification calculations. The experimental results show that the F1 value of the message classification model proposed here reaches 0.886 and 0.862, respectively, in the data set of long text and short text. Compared with three algorithms of long short-term memory (LSTM), logistic regression, and naive Bayesian, the Bi-LSTM model can achieve better results in the automatic classification of public message subjects.

Keywords

Introduction

Since the beginning of the 21st century, the use of information and communication technologies (ICT) in government has been called “e-government” (Cegarra-Navarro, Pachon, & Cegarra, 2012). Durrant (2002) defined it as the government’s long-term commitment to improving the relationship between citizens and the public sector through the provision of cost-effective and efficient services and information. Holmes (2001) thinks that e-government is to develop a citizen-centered government environment, which uses ICT to serve citizens whenever and wherever possible. Benefiting from the rise of new technologies such as the mobile network, the communication and dissemination of public affairs can be easily realized through the network in China. The public message refers to the opinions and suggestions of the public to participate in social affairs through microblogs, WeChat, e-mail, government websites, and other network political platforms. With more and more channels available for expressing opinions and suggestions, now people have more opportunities to participate in social construction. Because of this feature, government departments also need to understand the opinions and suggestions of the public, and feedback of the public on the online political inquiry platforms. The example of public messages is shown in Fig. 1.

Figure 1

Example of public messages.

Most of the public messages are saved or converted into written text, and then archived in digital format. On this basis, the government staff needs to classify the public messages and send those messages to the concerned functional departments for reply messages (Jeong, Lee, & Hong, 2017). Public message classification is a key step in the above process, however, when dealing with the public messages on the network political platform, the staff should classify the messages according to a certain classification standard, and rely on manual processing according to experience, which has the problems of heavy workload, low efficiency, and high error rate (Kim & Hong, 2021). The sharp increase of text data makes manual classification more and more difficult, and the high cost and low efficiency of information processing hinder the improvement of government governance.

Text classification is a process of labeling text documents into one or more pre-specified categories, and natural language processing (NLP) technology provides the possibility for automatic text classification in e-government affairs (Knutsson, Sneiders, & Alfalahi, 2012). In view of the above problems, this study hopes to apply a bi-directional long short-term memory (Bi-LSTM) network model based on attention mechanism to the scene of e-government text classification, which will bring some help for the related work in the future. First, the use of a deep learning algorithm for automatic classification can effectively save the time of manual classification. Second, the deep learning model can automatically learn some internal relations between data and tags. Apart from tagging work, it does not need any manual intervention and also saves the complicated process of feature definition (Minaee et al., 2020). At the same time, the public message involves a wide range of fields, and the generalization ability of deep learning in this classification problem is stronger than machine learning, which is more conducive to further expand the application. Third, Bi-LSTM fully considers the meaning of words in context, which overcomes the disadvantage that long short-term memory (LSTM) network can only consider one-way information of words. Finally, the research introduces the attention mechanism into the model. Different weights are given to different words, which can effectively improve the importance of words expressing the theme, and find out the key information in the message.

The rest of our paper is structured as follows: Section 2 reviews the related work; Section 3 focuses on the model presented in this paper; Section 4 presents the comparative experiments performed and also the description and analysis of the experimental results; Section 5 discusses the advantages and limitations of the model; and Section 6 is the summary and future prospects.

Literature Review
Application of Recurrent Neural Network in Chinese Text Classification

A recurrent neural network (RNN) is a common neural network in deep learning, which is mainly used in text classification, speech recognition, machine translation, emotion classification, video behavior recognition, and other fields. LSTM networks, a type of RNN, were proposed by Hochreiter and Schmidhuber (1997). Different from general RNN, as an improved cyclic neural algorithm, LSTM can not only solve the common problems, such as gradient explosion or gradient disappearance in neural networks, but also solve the long-distance dependence problem that the general RNN cannot handle. RNN was named with short-term memory. When the input text is long, the RNN network tends to forget the previous input text, and the information cannot be effectively transmitted, which has a negative impact on the classification in the later stage. The LSTM algorithm introduces the “gate” structure to realize the forgetting or retention of text information and the long-term memory of important information (Palangi et al., 2016).

At present, many scholars have introduced LSTM model into the study of Chinese text classification. Tao, Li, Liu and Liu (2019) proposed an end-to-end classification model of short essays based on bidirectional LSTM to solve the problems of short essays and sparse features. Good results have been achieved in the classification of Chinese news and other data sets. Li et al. (2020) proposed a sentiment analysis model based on deep learning, lexicon-integrated dual-channel CNN-LSTM family model, which combined CNN and LSTM/BiLSTM branches in parallel, and when tested found to be superior to many baseline methods in experiments on Chinese comment text data sets. Xie et al. (2020) proposed the LSTM Chinese text classification algorithm based on attention mechanism and feature enhancement fusion, which not only increased the weight of important text features but also enhanced the difference between them and other text features, significantly improving the recognition ability of Chinese text feature.

The successful application of LSTM algorithm in Chinese text classification provides strong support for relevant researches in the field of e-government.

Application of Text Classification in E-government

Text classification refers to the automatic and efficient text information classification technology by using computer technology. Currently, scholars have carried out related researches in the field of e-government. Zablith and Osman (2019) used a neural network to realize text classification and text emotion analysis of e-government platforms. The F-value of classification reached 85.16%, and emotion analysis and artificial emotion assessment had a high correlation (71.44%). Zhang, Wang and Zhu (2020) used the LSTM algorithm to conduct emotion analysis and discussed the influencing mechanism of government information release strategy on the evolution of negative emotional contagion. Ku and Leroy (2014) developed a government decision support system that combines NLP techniques, similarity measures, and Naive Bayes classifiers to classify electronic reports of different types of crimes with an accuracy rate of 94.82%. Kano, Fujita and Tsuda (2019) applied text mining technology to classify citizen reports based on text content, and the results showed that the automatic classification based on the content of citizen reports was more accurate than the classification based on the default category.

Researches indicated that text classification is widely applied in e-government. However, there are still few researches on the application of text mining technology in e-government with Chinese as the main text content. There is no relevant research on the application of Bi-LSTM network algorithm based on attention mechanism, an improved network algorithm for long-term and short-term memory.

Classification Model of Public Message Data of E-government
Technical Route of Model

The technical route of this model is shown in Fig. 2, which mainly includes three parts: data preprocessing, model training and testing, and model evaluation and result analysis.

Figure 2

Technical route of model.

The Usage of Word2vec in Classification of Public Message

Text representation is the basic work in NLP. The quality of text representation directly affects the performance of the whole NLP system. Text vectorization is the main way of text representation. This study adopts the Word2vec model proposed by Mikolov, Chen, Corrado and Dean (2013). Word2vec model is essentially a three-layer BP neural network and there is only one hidden layer. Among the commonly used Word2vec models, skip-gram model performs well in a large corpus so that the author will use this model to realize the Word2vec training.

Bi-LSTM Based on Attention Mechanism
Long Short-term Memory Network

The LSTM model is shown in Fig. 3. The box represents an identical timing module and the LSTM model consists of a series of identical timing modules. The LSTM model realizes the forgetting or retention of text information through the “gate” structure which consists of three gates: forgetting gate, input gate, and output gate.

Figure 3

Long short-term memory network module (Zhang, Zheng, Hu, & Yang, 2015).

① Forgetting Gate: The input of the forgetting gate is the output hi−1 of the front layer and the public message text xi, which are inputted in this layer. To make information pass selectively, the gates use sigmoid as the activation function and will output a calculation result Fi between 0 and 1. Fi represents the probability that the LSTM model state of the front layer is forgotten. 1 represents “completely reserved” and 0 represents “completely abandoned”. The relevant formulas are as follows: σ=sigmod(x)=11+ex {\rm{\sigma }} = {\rm{sigmod}} ({\rm x}) = {1 \over {1 + {{\rm{e}}^{ - {\rm{x}}}}}} Fi=σ(Wf[xi,hi1]+bf) {{\rm{F}}_{\rm{i}}} = {\rm{\sigma }}\left( {{{\rm{W}}_{\rm{f}}}\left[ {{{\rm{x}}_{\rm{i}}},{{\rm{h}}_{{\rm{i}} - 1}}} \right] + {{\rm{b}}_{\rm{f}}}} \right)

② Input Gate: The input gate consists of two parts. The first part uses sigmoid and the output is Ii. The second part uses tanh as the activation function and the output C˜i {{\rm{\tilde C}}_{\rm{i}}} is the output of this layer. The value of Ii is between 0 and 1, and Ii represents the extent to which the public message information in this layer is retained. Based on this, we can update the cell state of this layer with the new information from the public message data of e-government. The relevant formulas are stated as follows: Ii=σ(Wl[xi,hi1])+bl {{\rm{I}}_{\rm{i}}} = {\rm{\sigma }}\left( {{{\rm{W}}_{\rm{l}}}[{{\rm{x}}_{\rm{i}}},{{\rm{h}}_{{\rm{i}} - 1}}]} \right) + {{\rm{b}}_{\rm{l}}} C˜i=tanh(Wc[xi,hi1])+bc {{\rm{\tilde C}}_{\rm{i}}} = \tanh \left( {{{\rm{W}}_{\rm{c}}}\left[ {{{\rm{x}}_{\rm{i}}},{{\rm{h}}_{{\rm{i}} - 1}}} \right]} \right) + {{\rm{b}}_{\rm{c}}} Ci=Fi×Ci1+Ii×C˜i {{\rm{C}}_{\rm{i}}} = {{\rm{F}}_{\rm{i}}} \times {{\rm{C}}_{{\rm{i}} - 1}} + {{\rm{I}}_{\rm{i}}} \times {{\rm{\tilde C}}_{\rm{i}}}

③ Output Gate: The output gate is used to control how much the status of the unit is output to the current output value of LSTM. First, the SIGMOD function is used to get a value Oi between 0 and 1. Then the LSTM model state Ci is processed by tanh activation function and multiplied by Oi. The result Hi is the output of this layer. After Hi of every layer were weighted and summed, the type of public message will be obtained. The relevant formulas are stated as follows: Oi=σ(WO[xi,hi1])+bO {{\rm{O}}_{\rm{i}}} = {\rm{\sigma }}\left( {{{\rm{W}}_{\rm{O}}}\left[ {{{\rm{x}}_{\rm{i}}},{{\rm{h}}_{{\rm{i}} - 1}}} \right]} \right) + {{\rm{b}}_{\rm{O}}} Hi=Oi×tanh(Ci) {{\rm{H}}_{\rm{i}}} = {{\rm{O}}_{\rm{i}}} \times \tanh \,({{\rm{C}}_{\rm{i}}})

In the classification of the public message data of e-government, we need to analyze the long text. LSTM solves the problem of gradient vanishing in the traditional RNN by introducing the memory cell and gate mechanism, which makes it finally realize the protection of important information of the message. These gates determine the information that flows in and out of the current time steps.

Bi-LSTM Based on Attention Mechanism

During the process of using the LSTM model to classify, this study found that the relationship in the context of the message is bidirectional. Generally, LSTM can understand the following text according to the previous text, but cannot understand the previous content by the following message. At the same time, the content that has nothing to do with the topic in the message will affect the training effect. To improve the training effect of the model, the model should be focused on the critical information. Based on the above two reasons, this study adopts the Bi-LSTM model based on attention mechanism to get better classification effect in the public message data of e-government (Zhou et al., 2016).

Bi-LSTM

Bi-LSTM is a combination of forward LSTM and backward LSTM. The forward LSTM can obtain the past data information of the input sequence and the backward LSTM can obtain the future data information of the input sequence. In this way, the semantic relationship between the front and the back of a sentence is enhanced, and the accuracy of the model is improved.

Attention Mechanism

Encoder-decoder is a common model framework in deep learning. When predicting each output, its corresponding semantic code is the same, i.e., each word in the input text has the same influence on each word in the output. There are two disadvantages: one is that the semantic vector cannot completely represent the whole sequence of information. The other is that the information carried by the first input content will be covered by the later input information.

The attention mechanism in deep learning simulates the attention model of the human brain. It adds an “attention area” to the classification process. It represents the parts of the input sequence that should be focused on, so that the machine can pay attention to the important information and then generate the next output according to the “attention area”. In the above encoder-decoder framework, the combination of attention layer can effectively solve the above problems. The details are shown in Fig. 4.

Figure 4

Encoder-decoder framework with attention mechanism.

In the attention layer, the formulas of semantic coding are as follows: ci=j=1Txαijhj {{\rm{c}}_{\rm{i}}} = \sum\limits_{{\rm{j}} = 1}^{{{\rm{T}}_{\rm{x}}}} {{{\rm{\alpha }}_{{\rm{ij}}}}{{\rm{h}}_{\rm{j}}}} αij=exp(eij)k=1Txexp(eik) {{\rm{\alpha }}_{{\rm{ij}}}} = {{\exp \left( {{{\rm{e}}_{{\rm{ij}}}}} \right)} \over {\sum\nolimits_{{\rm{k}} = 1}^{{{\rm{T}}_{\rm{x}}}} {\exp \left( {{{\rm{e}}_{{\rm{ik}}}}} \right)} }} eij=a(si1,hj) {{\rm{e}}_{{\rm{ij}}}} = {\rm{a}}\left( {{{\rm{s}}_{{\rm{i}} - 1}},{{\rm{h}}_{\rm{j}}}} \right)

Take the sentence “洪山公园施工吵” as an example. In the above formulas, Tx is the length of the sentence, hj is the output of the hidden layer of the Encoder Layer at time j, and si−1 is the output of the hidden layer of the Decoder Layer at time i-1. In the text classification of public message data of e-government, the model learns different weight coefficients αij at different times through the attention mechanism, and the corresponding input feature is weighted by introducing the attention mechanism into the neural network. For the sentence “洪山公园施工吵,” we give the following probability distribution value: (洪山公园, 0.2), (施工, 0.5), and (吵, 0.3). The probability of each word represents the size of attention assigned to different words by the attention mechanism. As the word “施工” has the greatest impact on the type of message, it is given greater weight, thus improving the accuracy of classification. In this way, the attention mechanism identifies the main part to express the theme of each message and the input of the model is weighted to highlight the effective characteristics through the attention mechanism.

To sum up, the Bi-LSTM model based on the attention mechanism is shown in Fig. 5.

Figure 5

Bi-LSTM based on attention mechanism (Zhou et al., 2016). Bi-LSTM, Bi-directional long short-term memory.

Calculation of Classification Result

To carry out the text classification, this study transfers the output vector v of the full connection layer which is combined with the Attention Layer to the softmax activation function of seven categories in the output layer to predict the results of text classification. The formula of the prediction result of classification is explained as follows. Here, the cross-entropy loss function is used as the objective function, and the backpropagation optimization is carried out by the Adam algorithm. The parameters in the text classification model are trained and updated to minimize the cross-entropy of known message categories and predicted message categories. Y=softmax(Wcv+b) {\rm{Y}} = {\rm{softmax}} \left( {{{\rm{W}}_{\rm{c}}}{\rm{v + b}}} \right) Where Y is the prediction result matrix of message classification, while Wc is the same as Wc in the Input Gate, and b is the offset term.

Evaluation Index

This study focuses on the classification problem of seven categories with a single label. After the precision (p), recall (r) and F-score of the seven categories are calculated and the macro average value is calculated, which is the arithmetic mean value of each classification index. In this study, these three macro average values are used to evaluate the performance of the classification model.

Macro_precision=1ni=1nPi {\rm{Macro}}\_{\rm{precision}} = {1 \over {\rm{n}}}\sum\nolimits_{{\rm{i}} = 1}^{\rm{n}} {{{\rm{P}}_{\rm{i}}}} Macro_recall=1ni=1nri {\rm{Macro}}\_{\rm{recall}} = {1 \over {\rm{n}}}\sum\nolimits_{{\rm{i}} = 1}^{\rm{n}} {{{\rm{r}}_{\rm{i}}}} Macro_F_score=1ni=1nFli {\rm{Macro}}\_{\rm{F}}\_{\rm{score}} = {1 \over {\rm{n}}}\sum\nolimits_{{\rm{i}} = 1}^{\rm{n}} {{{\rm{F}}_{{\rm{li}}}}}
Empirical Analysis
Data Sources

The experimental data sets are from the BdRace Data Mining Competition Platform of China University Big Data Education Innovation Alliance. The platform collects the online government affairs public message data for the period November 2010 to January 2020 from the Internet. It selects and generates the Chinese text classification data sets, and the corpus uses UTF-8 coding. Data sets have seven categories including urban and rural construction, environmental protection, transportation, education and sports, labor and social security, business tourism, and health and birth control are selected. The amounts of data of each category are shown in Table 1 and there are 9210 texts in total. Specific examples of public messages are shown in Table 2.

Data Sets

Category Amounts of data
Urban and rural construction 2009
Environmental protection 938
Transportation 613
Education and sports 1589
Labor and social security 1969
Business and tourism 1215
Health and birth control 877

Specific Examples of Public Messages*

Message number User ID Message topics Time Message Details Category
24 A00074011 Xihu Construction Group in a city occupies the road for construction, which has potential safety hazards. (Original Text: A市的西 湖建筑集团霸占道路进 行施工有安全隐患。) 2020/1/6 12:09:38 On the road to the west of A3 Avenue, the sidewalk and street lamp are included in the construction enclosure of Yanzishan resettlement housing project of Xihu construction group. Every day, especially during the commuting period, there are a lot of people and traffic on this road, which has a great potential safety hazard. I strongly request civilized City A to rectify this extremely uncivilized road section as soon as possible. (Original Text: 在A3区大道向西方向的道路, 西湖建筑集团燕子山安置房项目将人行道和路 灯纳入在施工围墙内。每天尤其上下班期间, 这条路上人流车流极多,安全隐患非常大。 强烈请求文明城市A市,尽快整改这个极不文 明的路段。) Urban and Rural Construction
161600 A00015711 Shengchang glass company in Xidi province has been emitting high sulfur for many years, seriously polluting the atmosphere. (Original Text: 西地省盛 常玻璃公司多年高硫排 放,严重污染大气层。) 2014/8/14 11:53:40 There is a Shengchang glass company in Hekou Town, G5 County, which produces more than 190 tons of glass every day. Since it was put into operation in June 2010, it has been using high sulfur petroleum coke as fuel, with sulfur content as high as 3.5%. It melts 195 tons of glass every day, uses 53 tons of coke, and emits 1.8 tons of sulfur dioxide every day. So far, it has emitted more than 2600 tons of sulfur dioxide, seriously polluting the atmosphere. Since the use of petroleum coke, the company has always directly discharged the flue gas into the atmosphere without any desulfurization equipment and facilities. (Original Text: G5县合口镇有个盛常玻璃公 司,每天产玻璃190吨以上,自2010年6月投 产以来,一直用高硫石油焦为燃料,含硫高达 3.5%,每天熔化玻璃195吨,用焦量53吨,每 天排放二氧化硫1.8吨,至今累计排放2600吨 以上,严重污染大气层。该公司自用石油焦以 来始终将烟气直接排放到大气中,无任何脱硫 设备和设施。) Environmental Protection
160660 A00039781 In City A, many fishing boats stop in the middle of the channel on the Chujiang River, affecting the passage of cargo ships. (Original Text: 在A市, 楚江上有很多渔船停在 航道中间影响货船通 行。) 2017/9/12 16:47:13 Every night between the first bridge and the second bridge of the Chujiang River in City A, there are many small boats fishing in the middle of the river, and some fishing boats even stop in the middle of the channel, which seriously affects the passage of the incoming and outgoing cargo ships. I hope the relevant departments will take charge of it. (Original Text: 每天晚间在楚江A市的楚江一桥至二桥之 间有多小船在河中间钓鱼,有的渔船甚至停在 航道中间,严重影响来往的货船通行,希望有 关部门管管。) Transportation
1957 A00012120 Is the art training institution for the college entrance examination in Yujiang village, Hanpu Town, zone A3 legal? (Original Text: A3区含浦 镇玉江村高考美术培训 机构是否合法?) 2019/6/20 10:37:18 In Yujiang village, Hanpu Town, A3 Area, there is a college entrance examination art training institution called „Tongxing Tianyi”. It runs schools without legal qualifications. I hope the relevant departments can crackdown on these illegal training institutions and protect the legitimate rights and interests of law-abiding citizens. (Original Text: A3区含浦镇玉 江村有一个叫“同行添艺”的高考美术培训机 构,在没有取得合法办学资质的情况下非法办 学,我希望有关部门能够打击这些非法培训机 构,保护守法公民的合法权益。) Education and Sports
116962 A000111166 Consultation on maternity leave payment. (Original Text: 咨询产假 工资发放问题。) 2019/7/16 10:18:22 How does maternity leave pay? What materials need to be submitted to apply for maternity leave salary? Where can I do it? Is it a one-time payment? Is it paid before maternity leave or monthly? How to calculate the amount of maternity leave salary? (Original Text: 产假工 资怎么发放?申请产假工资需要提交哪些材 料?在哪里办理?是一次性发放吗?是在休产 假之前发放还是按月发放?产假工资的金额是 怎么计算的?) Labor and Social Security
78300 A00068226 The gasoline quality of the petrochemical company in F7 county is too poor. (Original Text: F7县的 石化公司的汽油质量太 差。) 2017/5/25 16:50:55 Hello! The quality of the gasoline in F7 county is too poor. Every time I refuel, the engine fault light will be on immediately. When I go to the place where I repair the car, it shows that the quality of the fuel is poor. After filling oil in other places, the fault light will disappear naturally. Dear leader, for the benefit of thousands of car owners, please send someone to check. (Original Text: 您好!我们F7 县的汽油质量太差,我每次加油过后发动机故 障灯立马就亮,去修车的地方检查显示燃油质 量差。在其他地方加完油后,故障灯会自然消 失。尊敬的领导,为了千千万万车主的利益, 请您派人来查查。) Business and Tourism
81060 A1046074052 The family planning office has yet to deal with the internal bleeding caused by a ligation operation in F8 City. (Original Text: F8市的 一位市民因做了结扎手 术,导致内出血,计生 办还没对这起事故做个 处理。) 2012/7/24 22:51:04 Hello, mayor. I’m an ordinary citizen in F8 city. My wife had a ligation operation in s hospital last year. On the same day, she had internal bleeding. After arriving at the people’s Hospital of F8 City, she was transferred to Chuya Hospital of a city. After the rescue, she saved her life. But the Family Planning Office has not dealt with the accident up to now, so I have no choice but to ask for help. (Original Text: 市长您好,我是F8市的一个普通老百 姓,我老婆去年在s医院做了个结扎手术,当 天就内出血,到了F8市人民医院后又转到A市 楚雅医院,经过抢救,保住了生命。但是计生 办到现在还没有对这起事故做个处理,我无 奈求帮助。) Health and Birth Control
Text Preprocessing

In this stage, the following preprocessing work is carried out for text data.

Data Cleaning: There are a lot of spaces before and after the message details column of the data sets, and the non-text information such as punctuation in the text is not needed to establish the classification model. Therefore, first, non-text information such as punctuation is removed;

Word Segmentation: There is no space between Chinese words and more than one Chinese word can be expressed as words with clear meaning. The purpose of Chinese word segmentation is to divide a coherent sentence into words with independent meaning according to certain segmentation criteria. In this study, Jieba Chinese word segmentation tool in Python is used to segment the text data sets;

Remove Stop Words: Modal auxiliary words, conjunctions, and other words have high frequency and low information content, which have no impact on the subsequent analysis. They belong to noise data. In this study, we use the stop words list of Harbin Institute of technology to remove stop words from the text data;

Text Vectorization: To extract the semantic information of words in the vectorization stage, Word2vec method is selected for vectorization. Word2vec uses vectors with the same dimension to represent the semantic information of text, and the distance between vectors can effectively represent the similarity of the meaning between words, which is an important step in the digitization of text data. In the Word2vec training module provided by gensim database, it is necessary to adjust the parameters repeatedly to carry out different training to find the best parameters for the corpus. After repeated training, the study set 8 as the value of training window, set 5 as the value of the minimum word frequency threshold, and set 250 as the value of the word vector dimension. The skip-gram model is used for word2vec training.

Oversampling: We can find that the sample is imbalances in the data set from the amount of text data of each category. Here, the sample balance is achieved by increasing the number of minority samples in the classification. The smote algorithm is called for oversampling, and some random noise and interference data are added at the same time to prevent the model from overfitting.

Model Parameter Setting

The choice of model structure parameters is critical to a good training model. The parameters obtained after adjustment and comparison in the process are the best parameters. The specific workflow of the Bi-LSTM network algorithm based on the attention mechanism is illustrated as in following Algorithm 1.

Input: the text data that has been preprocessed
Output: accuracy, recall, and F-score
Begin
At the ratio of 4:1, the text data sets are divided into training data set and test data set:

Training data set: x_train, Tag: y_train

Test data set: x_test, Tag: y_test

Bi-directional long short-term memory network model based on attention mechanism was established:

Set 1200 as the value of input nodes, set 64 as the value of hidden layer nodes and set 7 as the value of output layers which means seven categories will be outputted.

20% dropout was used.

Set softmax as the activation function and set categorical_crossentropy as the loss function. The optimizer of training is Adam.

Use Bi-directional long short-term memory network to calculate the word vector to get the higher level sentence vector.

Attention Layer: the results of bi-directional long short-term memory network were weighted by attention layer while ATT_SIZE = 100.

Call the fit function, set the iteration number of the model to epochs = 15, and specify that each batch contains 128 samples during gradient descent.

Input x_train to the model for training and x_test to the model for predicting. Accuracy, recall, and F-score are used as indicators to reflect the effect of the model.

End
Analysis of Experimental Results

The data sets provide two types of message text: “message details” and “message topic”, in which “message details” is long text and “message topic” is a short text. After training and testing the two types of text by Bi-LSTM network classifier based on attention mechanism, the accuracy, recall, and F-score results are shown in Table 3. The accuracy, recall, and F-score of LSTM algorithm, logistic regression algorithm, and naive Bayesian algorithm on the long text and short text data sets are also shown in Table 3.

Comprehensive Analysis of Classification Model Results

Classification model Types of message text Accuracy Recall (r) F-score
Bi-LSTM based on attention mechanism Long text 0.884 0.887 0.886
Short text 0.863 0.859 0.862
LSTM network Long text 0.833 0.826 0.829
Short text 0.842 0.850 0.845
Logistic regression Long text 0.839 0.841 0.840
Short text 0.772 0.789 0.780
Naive bayesian Long text 0.812 0.803 0.807
Short text 0.731 0.756 0.743

Bi-LSTM, Bi-directional long short-term memory; LSTM, Long short-term memory.

It can be seen from Table 3 that the accuracy, recall, and F-score of the models established by the four algorithms are all above 0.8, which has a good effect. Among them, the Bi-LSTM network algorithm based on attention mechanism performs best, which is 6.88% higher than the F-score of an ordinary LSTM algorithm and reaches 0.886. In short text classification, logistic regression algorithm and naive Bayesian algorithm have poor classification effect, while Bi-LSTM network algorithm based on attention mechanism performs best, which is 2.01% higher than the F-score of ordinary LSTM algorithm and reaches 0.862. In conclusion, the Bi-LSTM network algorithm based on attention mechanism has the best training effect and it is the most suitable one for the classification of public messages of e-government.

Discussion
The Advantages of Bi-LSTM based on Attention Mechanism in the Task of Classification of Public Message

First, the Bi-LSTM network algorithm based on the attention mechanism belongs to the deep learning algorithm. Compared with the normal machine learning algorithm, it does not need the complicated feature definition process (Li, Cao, Wang, & Xiao, 2017). At the same time, the government message involves a wide range of fields. Due to the limitation of feature engineering, machine learning has weak generalization ability in this classification problem, while deep learning can better achieve text classification in large text data, and the model has better generalization ability. What’s more, this model can better capture the long-distance dependence because it can learn which information to remember and which information to forget through the training process, which is the advantage that machine learning does not have.

Second, on the one hand, Bi-LSTM solves the problem of gradient disappearance or gradient explosion in traditional RNN. On the other hand, the Bi-LSTM network algorithm based on attention mechanism can process input data from front to back and from back to front, i.e., it can better capture bidirectional semantic dependency which overcomes the disadvantage that LSTM can’t encode the information from back to front.

Finally, the introduction of attention mechanism enables the model to give higher weight to important information, instead of giving the same weight to all input word vectors as usual, thus improving the accuracy of classification. In previous studies, the effectiveness of attention mechanism has also been verified. Zhao and Wu (2016) proposed a neural network with attention mechanism to improve the performance of sentence classification, which can capture the information of each word without any external features. Sun (2019) introduced the attention mechanism based on Gated Recurrent Unit (GRU) model, which improved the classification effect to a certain extent. It can be seen that the attention mechanism used in this study can play a good role in classification problems by weighting important information, and it is suitable for text classification of the public message of e-government.

The Limitations of Bi-LSTM Based on Attention Mechanism in the Task of Classification of Public Message

First, according to previous studies, Bi-LSTM is more accurate for sentences containing multiple aspects, while LSTM performs better for sentences with only one aspect (Park, Song, & Shin, 2020). In this study, there is no special distinction between sentences with different levels of meaning, which may lead to some deviation in the results.

Second, the high dimension of the word vector will affect the performance of Bi-LSTM. The representation of text is usually a high-dimensional vector. If it is used as the input of Bi-LSTM, the parameters of the network will increase sharply and it is difficult to optimize the network. To solve this problem, Liu and Guo (2019) proposed a model called attention-based Bi-LSTM with convolution layer, because they believe that convolution can be used to extract the features of text vectors and reduce the dimension of vectors. The same method is also used in the research of Guo, Zhao and Cui (2020). They use CNN to reduce the dimension of the word vector matrix formed by the original data, and then fuse the Bi-LSTM model for emotional analysis, so as to further improve the operation efficiency and prediction accuracy of the model. However, this study did not use this way to optimize the model. Because the word vector dimension of the data set in this paper is less, this method is not used to improve the model. However, in the future, when facing a larger sample size of the public message text, the above method may be needed to improve the model.

Finally, another problem of the Bi-LSTM network algorithm based on attention mechanism is that it requires a lot of hardware for training, and the linear layer needs a lot of memory bandwidth to calculate, which makes the process of model training very time-consuming.

Conclusion

In this study, the Bi-LSTM network algorithm based on attention mechanism is used to establish a multi-classification model of text based on deep learning, aiming to realize the classification of various messages in e-government more accurately. First, this study uses the Bi-LSTM algorithm to strengthen the relevance of messages before and after the training process, combines the characteristics of the attention mechanism, and strengthens the semantic attention to important text features. Finally, through the full-connection layer fusion feature weight, the classification calculation is carried out. The experimental results on the Internet government message corpus show that the message classification model proposed here can achieve a higher F-score and accurate precision, which has certain advantages compared with three algorithms of LSTM and logistic regression and naive Bayesian. In addition, the performance of the four algorithms on the long text and short text data sets is analyzed and clarified in detail.

At present, further research can be done based on two aspects. The first one is to recognize the number of aspects that sentences contain. Bi-LSTM can be used for the classification of sentences with multiple aspects, and LSTM can be used for the classification of sentences with a single aspect, so as to further improve the classification effect; second, the granularity of text can be further optimized, and the attention mechanism can be used at different levels, such as words, words and sentences.

Figure 1

Example of public messages.
Example of public messages.

Figure 2

Technical route of model.
Technical route of model.

Figure 3

Long short-term memory network module (Zhang, Zheng, Hu, & Yang, 2015).
Long short-term memory network module (Zhang, Zheng, Hu, & Yang, 2015).

Figure 4

Encoder-decoder framework with attention mechanism.
Encoder-decoder framework with attention mechanism.

Figure 5

Bi-LSTM based on attention mechanism (Zhou et al., 2016). Bi-LSTM, Bi-directional long short-term memory.
Bi-LSTM based on attention mechanism (Zhou et al., 2016). Bi-LSTM, Bi-directional long short-term memory.

Data Sets

Category Amounts of data
Urban and rural construction 2009
Environmental protection 938
Transportation 613
Education and sports 1589
Labor and social security 1969
Business and tourism 1215
Health and birth control 877

Comprehensive Analysis of Classification Model Results

Classification model Types of message text Accuracy Recall (r) F-score
Bi-LSTM based on attention mechanism Long text 0.884 0.887 0.886
Short text 0.863 0.859 0.862
LSTM network Long text 0.833 0.826 0.829
Short text 0.842 0.850 0.845
Logistic regression Long text 0.839 0.841 0.840
Short text 0.772 0.789 0.780
Naive bayesian Long text 0.812 0.803 0.807
Short text 0.731 0.756 0.743

Specific Examples of Public Messages*

Message number User ID Message topics Time Message Details Category
24 A00074011 Xihu Construction Group in a city occupies the road for construction, which has potential safety hazards. (Original Text: A市的西 湖建筑集团霸占道路进 行施工有安全隐患。) 2020/1/6 12:09:38 On the road to the west of A3 Avenue, the sidewalk and street lamp are included in the construction enclosure of Yanzishan resettlement housing project of Xihu construction group. Every day, especially during the commuting period, there are a lot of people and traffic on this road, which has a great potential safety hazard. I strongly request civilized City A to rectify this extremely uncivilized road section as soon as possible. (Original Text: 在A3区大道向西方向的道路, 西湖建筑集团燕子山安置房项目将人行道和路 灯纳入在施工围墙内。每天尤其上下班期间, 这条路上人流车流极多,安全隐患非常大。 强烈请求文明城市A市,尽快整改这个极不文 明的路段。) Urban and Rural Construction
161600 A00015711 Shengchang glass company in Xidi province has been emitting high sulfur for many years, seriously polluting the atmosphere. (Original Text: 西地省盛 常玻璃公司多年高硫排 放,严重污染大气层。) 2014/8/14 11:53:40 There is a Shengchang glass company in Hekou Town, G5 County, which produces more than 190 tons of glass every day. Since it was put into operation in June 2010, it has been using high sulfur petroleum coke as fuel, with sulfur content as high as 3.5%. It melts 195 tons of glass every day, uses 53 tons of coke, and emits 1.8 tons of sulfur dioxide every day. So far, it has emitted more than 2600 tons of sulfur dioxide, seriously polluting the atmosphere. Since the use of petroleum coke, the company has always directly discharged the flue gas into the atmosphere without any desulfurization equipment and facilities. (Original Text: G5县合口镇有个盛常玻璃公 司,每天产玻璃190吨以上,自2010年6月投 产以来,一直用高硫石油焦为燃料,含硫高达 3.5%,每天熔化玻璃195吨,用焦量53吨,每 天排放二氧化硫1.8吨,至今累计排放2600吨 以上,严重污染大气层。该公司自用石油焦以 来始终将烟气直接排放到大气中,无任何脱硫 设备和设施。) Environmental Protection
160660 A00039781 In City A, many fishing boats stop in the middle of the channel on the Chujiang River, affecting the passage of cargo ships. (Original Text: 在A市, 楚江上有很多渔船停在 航道中间影响货船通 行。) 2017/9/12 16:47:13 Every night between the first bridge and the second bridge of the Chujiang River in City A, there are many small boats fishing in the middle of the river, and some fishing boats even stop in the middle of the channel, which seriously affects the passage of the incoming and outgoing cargo ships. I hope the relevant departments will take charge of it. (Original Text: 每天晚间在楚江A市的楚江一桥至二桥之 间有多小船在河中间钓鱼,有的渔船甚至停在 航道中间,严重影响来往的货船通行,希望有 关部门管管。) Transportation
1957 A00012120 Is the art training institution for the college entrance examination in Yujiang village, Hanpu Town, zone A3 legal? (Original Text: A3区含浦 镇玉江村高考美术培训 机构是否合法?) 2019/6/20 10:37:18 In Yujiang village, Hanpu Town, A3 Area, there is a college entrance examination art training institution called „Tongxing Tianyi”. It runs schools without legal qualifications. I hope the relevant departments can crackdown on these illegal training institutions and protect the legitimate rights and interests of law-abiding citizens. (Original Text: A3区含浦镇玉 江村有一个叫“同行添艺”的高考美术培训机 构,在没有取得合法办学资质的情况下非法办 学,我希望有关部门能够打击这些非法培训机 构,保护守法公民的合法权益。) Education and Sports
116962 A000111166 Consultation on maternity leave payment. (Original Text: 咨询产假 工资发放问题。) 2019/7/16 10:18:22 How does maternity leave pay? What materials need to be submitted to apply for maternity leave salary? Where can I do it? Is it a one-time payment? Is it paid before maternity leave or monthly? How to calculate the amount of maternity leave salary? (Original Text: 产假工 资怎么发放?申请产假工资需要提交哪些材 料?在哪里办理?是一次性发放吗?是在休产 假之前发放还是按月发放?产假工资的金额是 怎么计算的?) Labor and Social Security
78300 A00068226 The gasoline quality of the petrochemical company in F7 county is too poor. (Original Text: F7县的 石化公司的汽油质量太 差。) 2017/5/25 16:50:55 Hello! The quality of the gasoline in F7 county is too poor. Every time I refuel, the engine fault light will be on immediately. When I go to the place where I repair the car, it shows that the quality of the fuel is poor. After filling oil in other places, the fault light will disappear naturally. Dear leader, for the benefit of thousands of car owners, please send someone to check. (Original Text: 您好!我们F7 县的汽油质量太差,我每次加油过后发动机故 障灯立马就亮,去修车的地方检查显示燃油质 量差。在其他地方加完油后,故障灯会自然消 失。尊敬的领导,为了千千万万车主的利益, 请您派人来查查。) Business and Tourism
81060 A1046074052 The family planning office has yet to deal with the internal bleeding caused by a ligation operation in F8 City. (Original Text: F8市的 一位市民因做了结扎手 术,导致内出血,计生 办还没对这起事故做个 处理。) 2012/7/24 22:51:04 Hello, mayor. I’m an ordinary citizen in F8 city. My wife had a ligation operation in s hospital last year. On the same day, she had internal bleeding. After arriving at the people’s Hospital of F8 City, she was transferred to Chuya Hospital of a city. After the rescue, she saved her life. But the Family Planning Office has not dealt with the accident up to now, so I have no choice but to ask for help. (Original Text: 市长您好,我是F8市的一个普通老百 姓,我老婆去年在s医院做了个结扎手术,当 天就内出血,到了F8市人民医院后又转到A市 楚雅医院,经过抢救,保住了生命。但是计生 办到现在还没有对这起事故做个处理,我无 奈求帮助。) Health and Birth Control

j.dim-2021-0004.tab.004

Input: the text data that has been preprocessed
Output: accuracy, recall, and F-score
Begin
At the ratio of 4:1, the text data sets are divided into training data set and test data set:

Training data set: x_train, Tag: y_train

Test data set: x_test, Tag: y_test

Bi-directional long short-term memory network model based on attention mechanism was established:

Set 1200 as the value of input nodes, set 64 as the value of hidden layer nodes and set 7 as the value of output layers which means seven categories will be outputted.

20% dropout was used.

Set softmax as the activation function and set categorical_crossentropy as the loss function. The optimizer of training is Adam.

Use Bi-directional long short-term memory network to calculate the word vector to get the higher level sentence vector.

Attention Layer: the results of bi-directional long short-term memory network were weighted by attention layer while ATT_SIZE = 100.

Call the fit function, set the iteration number of the model to epochs = 15, and specify that each batch contains 128 samples during gradient descent.

Input x_train to the model for training and x_test to the model for predicting. Accuracy, recall, and F-score are used as indicators to reflect the effect of the model.

End

Cegarra-Navarro, J. G., Pachon, J. R. C., & Cegarra, J. L. M. (2012). E-government and citizen’s engagement with local affairs through e-websites: The case of Spanish municipalities. International Journal of Information Management, 32(5), 469–478. doi: 10.1016/j.ijinfomgt.2012.02.008 Cegarra-NavarroJ. G. PachonJ. R. C. CegarraJ. L. M. 2012 E-government and citizen’s engagement with local affairs through e-websites: The case of Spanish municipalities International Journal of Information Management 32 5 469 478 10.1016/j.ijinfomgt.2012.02.008 Open DOISearch in Google Scholar

Durrant, F. (2002). e-Government and the Internet in the Caribbean: An initial assessment. In R. Traunmüller & K. Lenk (Eds.), Lecture Notes in Computer Science: Vol. 2456. Electronic Government (pp. 101–104). Berlin, Heidelberg: Springer. doi:10.1007/978-3-540-46138-8_15 DurrantF. 2002 e-Government and the Internet in the Caribbean: An initial assessment In TraunmüllerR. LenkK. (Eds.), Lecture Notes in Computer Science: Vol. 2456. Electronic Government 101 104 Berlin, Heidelberg Springer 10.1007/978-3-540-46138-8_15 Open DOISearch in Google Scholar

Guo, X. D., Zhao, N., & Cui, S. Z. (2020). Consumer reviews sentiment analysis based on CNN-BiLSTM. Systems Engineering-Theory & Practice, 40(03), 653–663. doi:10.12011/1000-6788-2018-1890-11 GuoX. D. ZhaoN. CuiS. Z. 2020 Consumer reviews sentiment analysis based on CNN-BiLSTM Systems Engineering-Theory & Practice 40 03 653 663 10.12011/1000-6788-2018-1890-11 Open DOISearch in Google Scholar

Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780. doi: 10.1162/neco.1997.9.8.1735 HochreiterS. SchmidhuberJ. 1997 Long short-term memory Neural Computation 9 8 1735 1780 10.1162/neco.1997.9.8.1735 Open DOISearch in Google Scholar

Holmes, D. (2001). eGov: eBusiness strategies for government. Boston, MA: Nicholas Brealey Publishing. HolmesD. 2001 eGov: eBusiness strategies for government Boston, MA Nicholas Brealey Publishing Search in Google Scholar

Jeong, H. Y., Lee, T. H., & Hong, S. G. (2017). A corpus analysis of electronic petitions for improving the responsiveness of public services: Focusing on Busan petition. The Korean Journal of Local Government Studies, 21(1), 423–436. doi: 10.20484/klog.21.1.17 JeongH. Y. LeeT. H. HongS. G. 2017 A corpus analysis of electronic petitions for improving the responsiveness of public services: Focusing on Busan petition The Korean Journal of Local Government Studies 21 1 423 436 10.20484/klog.21.1.17 Open DOISearch in Google Scholar

Kano, E., Fujita, Y., & Tsuda, K. (2019). A method of extracting and classifying local community problems from citizen-report data using text mining. Procedia Computer Science, 159, 1347–1356. doi: 10.1016/j.procs.2019.09.305 KanoE. FujitaY. TsudaK. 2019 A method of extracting and classifying local community problems from citizen-report data using text mining Procedia Computer Science 159 1347 1356 10.1016/j.procs.2019.09.305 Open DOISearch in Google Scholar

Kim, N., & Hong, S. (2021). Automatic classification of citizen requests for transportation using deep learning: Case study from Boston city. Information Processing & Management, 58(1), 1–13. doi: 10.1016/j.ipm.2020.102410 KimN. HongS. 2021 Automatic classification of citizen requests for transportation using deep learning: Case study from Boston city Information Processing & Management 58 1 1 13 10.1016/j.ipm.2020.102410 Open DOISearch in Google Scholar

Knutsson, O., Sneiders, E., & Alfalahi, A. (2012, October). Opportunities for improving egovernment: Using language technology in workflow management. Proceedings of the 6th International Conference on Theory and Practice of Electronic Governance, 495–496. doi:10.1145/2463728.2463833 KnutssonO. SneidersE. AlfalahiA. 2012 October Opportunities for improving egovernment: Using language technology in workflow management Proceedings of the 6th International Conference on Theory and Practice of Electronic Governance 495 496 10.1145/2463728.2463833 Open DOISearch in Google Scholar

Ku, C. H., & Leroy, G. (2014). A decision support system: Automated crime report analysis and classification for e-government. Government Information Quarterly, 31(4), 534–544. doi: 10.1016/j.giq.2014.08.003 KuC. H. LeroyG. 2014 A decision support system: Automated crime report analysis and classification for e-government Government Information Quarterly 31 4 534 544 10.1016/j.giq.2014.08.003 Open DOISearch in Google Scholar

Li, J., Cao, Y., Wang, Y., & Xiao, H. (2017). Online learning algorithms for double-weighted least squares twin bounded support vector machines. Neural Processing Letters, 45(1), 319–339. doi: 10.1007/s11063-016-9527-9 LiJ. CaoY. WangY. XiaoH. 2017 Online learning algorithms for double-weighted least squares twin bounded support vector machines Neural Processing Letters 45 1 319 339 10.1007/s11063-016-9527-9 Open DOISearch in Google Scholar

Li, W., Zhu, L., Shi, Y., Guo, K., & Cambria, E. (2020). User reviews: Sentiment analysis using lexicon integrated two-channel CNNLSTM family models. Applied Soft Computing, 94, 1–11. doi: 10.1016/j.asoc.2020.106435 LiW. ZhuL. ShiY. GuoK. CambriaE. 2020 User reviews: Sentiment analysis using lexicon integrated two-channel CNNLSTM family models Applied Soft Computing 94 1 11 10.1016/j.asoc.2020.106435 Open DOISearch in Google Scholar

Liu, G., & Guo, J. (2019). Bidirectional LSTM with attention mechanism and convolutional layer for text classification. Neurocomputing, 337(APR.14), 325–338. doi: 10.1016/j.neucom.2019.01.078 LiuG. GuoJ. 2019 Bidirectional LSTM with attention mechanism and convolutional layer for text classification Neurocomputing 337 APR.14 325 338 10.1016/j.neucom.2019.01.078 Open DOISearch in Google Scholar

Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space. ArXiv Preprint. arXiv:1301.3781. MikolovT. ChenK. CorradoG. DeanJ. 2013 Efficient estimation of word representations in vector space ArXiv Preprint. arXiv:1301.3781 Search in Google Scholar

Minaee, S., Kalchbrenner, N., Cambria, E., Nikzad, N., Chenaghlu, M., & Gao, J. (2020). Deep learning based text classification: A comprehensive review. ArXiv Preprint. arXiv:2004.03705. MinaeeS. KalchbrennerN. CambriaE. NikzadN. ChenaghluM. GaoJ. 2020 Deep learning based text classification: A comprehensive review ArXiv Preprint. arXiv:2004.03705 Search in Google Scholar

Palangi, H., Deng, L., Shen, Y., Gao, J., He, X., Chen, J., . . . Ward, R. (2016). Deep sentence embedding using long short-term memory networks: Analysis and application to information retrieval. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 24(4), 694–707. doi:10.1109/TASLP.2016.2520371 PalangiH. DengL. ShenY. GaoJ. HeX. ChenJ. WardR. 2016 Deep sentence embedding using long short-term memory networks: Analysis and application to information retrieval IEEE/ACM Transactions on Audio, Speech, and Language Processing 24 4 694 707 10.1109/TASLP.2016.2520371 Open DOISearch in Google Scholar

Park, H. J., Song, M., & Shin, K. S. (2020). Deep learning models and datasets for aspect term sentiment classification: Implementing holistic recurrent attention on target-dependent memories. Knowledge-Based Systems, 187, 104825. doi: 10.1016/j.knosys.2019.06.033 ParkH. J. SongM. ShinK. S. 2020 Deep learning models and datasets for aspect term sentiment classification: Implementing holistic recurrent attention on target-dependent memories Knowledge-Based Systems 187 104825 10.1016/j.knosys.2019.06.033 Open DOISearch in Google Scholar

Sun, M. M. (2019). Chinese text classification based on gru-attention. Modern Information Technology, 3(03), 10–12. SunM. M. 2019 Chinese text classification based on gru-attention Modern Information Technology 3 03 10 12 Search in Google Scholar

Tao, Z., Li, X., Liu, Y., & Liu, X. (2020). Classifying short texts with improved-attention based bidirectional long memory network. Data Analysis and Knowledge Discovery, 3(12), 21–29. doi: 10.11925/infotech.2096-3467.2019.0267 TaoZ. LiX. LiuY. LiuX. 2020 Classifying short texts with improved-attention based bidirectional long memory network Data Analysis and Knowledge Discovery 3 12 21 29 10.11925/infotech.2096-3467.2019.0267 Open DOISearch in Google Scholar

Xie, J., Hou, Y., Wang, Y., Wang, Q., Li, B., Lv, S., & Vorotnitsky, Y. I. (2020). Chinese text classification based on attention mechanism and feature-enhanced fusion neural network. Computing, 102(3), 683–700. doi: 10.1007/s00607-019-00766-9 XieJ. HouY. WangY. WangQ. LiB. LvS. VorotnitskyY. I. 2020 Chinese text classification based on attention mechanism and feature-enhanced fusion neural network Computing 102 3 683 700 10.1007/s00607-019-00766-9 Open DOISearch in Google Scholar

Zablith, F., & Osman, I. H. (2019). Reviewmodus: Text classification and sentiment prediction of unstructured reviews using a hybrid combination of machine learning and evaluation models. Applied Mathematical Modelling, 71, 569–583. doi: 10.1016/j.apm.2019.02.032 ZablithF. OsmanI. H. 2019 Reviewmodus: Text classification and sentiment prediction of unstructured reviews using a hybrid combination of machine learning and evaluation models Applied Mathematical Modelling 71 569 583 10.1016/j.apm.2019.02.032 Open DOISearch in Google Scholar

Zhang, S., Zheng, D., Hu, X., & Yang, M. (2015, October). Bidirectional long short-term memory networks for relation classification. Proceedings of the 29th Pacific Asia conference on language, information and computation, 73–78. ZhangS. ZhengD. HuX. YangM. 2015 October Bidirectional long short-term memory networks for relation classification Proceedings of the 29th Pacific Asia conference on language, information and computation 73 78 Search in Google Scholar

Zhang, W., Wang, M., & Zhu, Y. C. (2020). Does government information release really matter in regulating contagion-evolution of negative emotion during public emergencies? From the perspective of cognitive big data analytics. International Journal of Information Management, 50, 498–514. doi: 10.1016/j.ijinfomgt.2019.04.001 ZhangW. WangM. ZhuY. C. 2020 Does government information release really matter in regulating contagion-evolution of negative emotion during public emergencies? From the perspective of cognitive big data analytics International Journal of Information Management 50 498 514 10.1016/j.ijinfomgt.2019.04.001 Open DOISearch in Google Scholar

Zhao, Z., & Wu, Y. (2016). Attention-based convolutional neural networks for sentence classification. INTERSPEECH, 705–709. doi:10.21437/Interspeech.2016-354 ZhaoZ. WuY. 2016 Attention-based convolutional neural networks for sentence classification INTERSPEECH 705 709 10.21437/Interspeech.2016-354 Open DOISearch in Google Scholar

Zhou, P., Shi, W., Tian, J., Qi, Z., Li, B., Hao, H., & Xu, B. (2016). Attention-based bidirectional long short-term memory networks for relation classification. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 207–212. doi: 10.18653/v1/P16-2034 ZhouP. ShiW. TianJ. QiZ. LiB. HaoH. XuB. 2016 Attention-based bidirectional long short-term memory networks for relation classification Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 207 212 10.18653/v1/P16-2034 Open DOISearch in Google Scholar

Recommended articles from Trend MD

Plan your remote conference with Sciendo