Acceso abierto

Masked Sentence Model Based on BERT for Move Recognition in Medical Scientific Abstracts

Gaihong Yu
Gaihong Yu
, 
Zhixiong Zhang
Zhixiong Zhang
, 
Huan Liu
Huan Liu
 y 
Liangping Ding
Liangping Ding
   | 27 dic 2019
Journal of Data and Information Science's Cover Image
Acerca de este artículo
Artículo anterior
Artículo siguiente
Cite
Article Category: Research Paper
Publicado en línea: 27 dic 2019
Páginas: 42 - 55
Recibido: 27 sept 2019
Aceptado: 05 nov 2019
DOI: https://doi.org/10.2478/jdis-2019-0020
Palabras clave
© 2019 Gaihong Yu, Zhixiong Zhang, Huan Liu, Liangping Ding, published by Sciendo
This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

Figure 1

An example of an abstract.
An example of an abstract.

Figure 2

Sentence representations.
Sentence representations.

Figure 3

The architecture of the masked sentence model based on BERT.
The architecture of the masked sentence model based on BERT.

The results of Exp3: based on MSM integrated information.

LabelPRF1Support
Background75.2681.1878.113,077
Objectives78.0861.9869.102,333
Methods92.9897.4895.179,884
Results96.0293.7494.879,713
Conclusions94.7094.5194.604,571
Avg / Total91.2291.3091.1529,578

Data format for integrating sentence content and context.

LabelThe content & context of the sentence
MethodsWe selected the major journals (11 journals) collecting papers (more than 7,000) over the last five years from the top members of the research community, and read and analyzed the papers (more than 200) covering the topics.
MethodsThis survey aims at reviewing the literature related to Clinical Information Systems (CIS), Hospital Information Systems (HIS), Electronic Health Record (EHR) systems, and how collected data can be analyzed by Artificial Intelligence (AI) techniques. aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa. Then, we completed the analysis using search engines to also include papers from major conferences over the same five years. We defined a taxonomy of major features and research areas of CIS, HIS, EHR systems. We also defined a taxonomy for the use of Artificial Intelligence (AI) techniques on healthcare data. In the light of these taxonomies, we report on the most relevant papers from the literature. We highlighted some major research directions and issues which seem to be promising and to need further investigations over a medium- or long-term period.

The results of Exp2: based on the context of sentences.

LabelPRF1Support
Background72.2779.7275.823,077
Objectives70.5160.2764.992,333
Methods90.7089.8090.259,884
Results87.7189.2088.459,713
Conclusions90.1989.3089.744,571
Avg / Total86.1386.1586.0929,578

Data format of sentence content.

LabelThe content of the sentence
MethodsWe selected the major journals (11 journals) collecting papers (more than 7,000) over the last five years from the top members of the research community, and read and analyzed the papers (more than 200) covering the topics.

The results of Exp1: based on the content of sentences.

LabelPRF1Support
Background64.3775.8569.643,077
Objectives73.5556.9764.202,333
Methods92.4294.9793.689,884
Results92.0891.0991.589,713
Conclusions84.9581.3883.134,571
Avg / Total86.7586.6186.5329,578

Comparison of the results of the experiments.

LabelExp1Exp2Exp3Exp3-Exp1Exp3-Exp2
F1F1F1+F1+F1
Background69.6475.8278.118.472.29
Objectives64.2064.9969.104.94.11
Methods93.6890.2595.171.494.92
Results91.5888.4594.873.296.42
Conclusions83.1389.7494.6011.474.86
Avg / Total86.5386.0991.154.625.06

Data format of the sentence’s context.

LabelThe context of the sentence
MethodsThis survey aims at reviewing the literature related to Clinical Information Systems (CIS), Hospital Information Systems (HIS), Electronic Health Record (EHR) systems, and how collected data can be analyzed by Artificial Intelligence (AI) techniques. aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa. Then, we completed the analysis using search engines to also include papers from major conferences over the same five years. We defined a taxonomy of major features and research areas of CIS, HIS, EHR systems. We also defined a taxonomy for the use of Artificial Intelligence (AI) techniques on healthcare data. In the light of these taxonomies, we report on the most relevant papers from the literature. We highlighted some major research directions and issues which seem to be promising and to need further investigations over a medium- or long-term period.

PubMed 20k RCT results.

ModelsF1 (PubMed 20k RCT)
Our ModelMaskedSentenceModel_BERT91.15
OthersHSLN-RNN (Jin and Szolovits, 2018) (SOTA)92.6
BERT-Base (Beltagy et al., 2018)86.19
Sci BERT (SciVocab) (Beltagy et al., 2018)86.80
Sci BERT (BaseVocab) (Beltagy et al., 2018)86.81
eISSN:
2543-683X
Idioma:
Inglés
Calendario de la edición:
4 veces al año
Temas de la revista:
Computer Sciences, Information Technology, Project Management, Databases and Data Mining
RSS Feed de revista