<abstract xmlns="http://www.w3.org/1999/xhtml"><sec id="j_jdis-2019-0020_s_006_w2aab3b7b4b1b6b1aab1c18b1Aa"><h3>Purpose</h3><p>Move recognition in scientific abstracts is an NLP task of classifying sentences of the abstracts into different types of language units. To improve the performance of move recognition in scientific abstracts, a novel model of move recognition is proposed that outperforms the BERT-based method.</p></sec><sec id="j_jdis-2019-0020_s_007_w2aab3b7b4b1b6b1aab1c18b2Aa"><h3>Design/methodology/approach</h3><p>Prevalent models based on BERT for sentence classification often classify sentences without considering the context of the sentences. In this paper, inspired by the BERT masked language model (MLM), we propose a novel model called the masked sentence model that integrates the content and contextual information of the sentences in move recognition. Experiments are conducted on the benchmark dataset PubMed 20K RCT in three steps. Then, we compare our model with HSLN-RNN, BERT-based and SciBERT using the same dataset.</p></sec><sec id="j_jdis-2019-0020_s_008_w2aab3b7b4b1b6b1aab1c18b3Aa"><h3>Findings</h3><p>Compared with the BERT-based and SciBERT models, the F1 score of our model outperforms them by 4.96% and 4.34%, respectively, which shows the feasibility and effectiveness of the novel model and the result of our model comes closest to the state-of-the-art results of HSLN-RNN at present.</p></sec><sec id="j_jdis-2019-0020_s_009_w2aab3b7b4b1b6b1aab1c18b4Aa"><h3>Research limitations</h3><p>The sequential features of move labels are not considered, which might be one of the reasons why HSLN-RNN has better performance. Our model is restricted to dealing with biomedical English literature because we use a dataset from PubMed, which is a typical biomedical database, to fine-tune our model.</p></sec><sec id="j_jdis-2019-0020_s_010_w2aab3b7b4b1b6b1aab1c18b5Aa"><h3>Practical implications</h3><p>The proposed model is better and simpler in identifying move structures in scientific abstracts and is worthy of text classification experiments for capturing contextual features of sentences.</p></sec><sec id="j_jdis-2019-0020_s_011_w2aab3b7b4b1b6b1aab1c18b6Aa"><h3>Originality/value</h3><p>T he study proposes a masked sentence model based on BERT that considers the contextual features of the sentences in abstracts in a new way. The performance of this classification model is significantly improved by rebuilding the input layer without changing the structure of neural networks.</p></sec></abstract>

PurposeMove recognition in scientific abstracts is an NLP task of classifying sentences of the abstracts into different types of language units. To improve the performance of move recognition in scientific abstracts, a novel model of move recognition is proposed that outperforms the BERT-based method.Design/methodology/approachPrevalent models based on BERT for sentence classification often classify sentences without considering the context of the sentences. In this paper, inspired by the BERT masked language model (MLM), we propose a novel model called the masked sentence model that integrates the content and contextual information of the sentences in move recognition. Experiments are conducted on the benchmark dataset PubMed 20K RCT in three steps. Then, we compare our model with HSLN-RNN, BERT-based and SciBERT using the same dataset.FindingsCompared with the BERT-based and SciBERT models, the F1 score of our model outperforms them by 4.96% and 4.34%, respectively, which shows the feasibility and effectiveness of the novel model and the result of our model comes closest to the state-of-the-art results of HSLN-RNN at present.Research limitationsThe sequential features of move labels are not considered, which might be one of the reasons why HSLN-RNN has better performance. Our model is restricted to dealing with biomedical English literature because we use a dataset from PubMed, which is a typical biomedical database, to fine-tune our model.Practical implicationsThe proposed model is better and simpler in identifying move structures in scientific abstracts and is worthy of text classification experiments for capturing contextual features of sentences.Originality/valueT he study proposes a masked sentence model based on BERT that considers the contextual features of the sentences in abstracts in a new way. The performance of this classification model is significantly improved by rebuilding the input layer without changing the structure of neural networks.

Masked Sentence Model Based on BERT for Move Recognition in Medical Scientific Abstracts

Journal of Data and Information Science

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

PurposeMove recognition in scientific abstracts is an NLP task of classifying sentences of the abstracts into different types of language units. To improve the...

Masked Sentence Model Based on BERT for Move Recognition in Medical Scientific Abstracts

Article Category: Research Paper

Published Online: Dec 27, 2019

Page range: 42 - 55

Received: Sep 27, 2019

Accepted: Nov 05, 2019

DOI: https://doi.org/10.2478/jdis-2019-0020

Keywords
Move recognition, BERT, Masked sentence model, Scientific abstracts

© 2019 Gaihong Yu, Zhixiong Zhang, Huan Liu, Liangping Ding, published by Sciendo

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

Masked Sentence Model Based on BERT for Move Recognition in Medical Scientific Abstracts

Article Category: Research Paper

Published Online: Dec 27, 2019

Page range: 42 - 55

Received: Sep 27, 2019

Accepted: Nov 05, 2019

DOI: https://doi.org/10.2478/jdis-2019-0020

KeywordsMove recognition, BERT, Masked sentence model, Scientific abstracts

© 2019 Gaihong Yu, Zhixiong Zhang, Huan Liu, Liangping Ding, published by Sciendo

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

Keywords
Move recognition, BERT, Masked sentence model, Scientific abstracts