<abstract xmlns="http://www.w3.org/1999/xhtml">

<p>Although Czech rule-based tools for automatic punctuation insertion rely on extensive grammar and achieve respectable precision, the pre-trained Transformers outperform rule-based systems in precision and recall (Machura et al. 2022). The Czech pre-trained RoBERTa model achieves excellent results, yet a certain level of phenomena is ignored, and the model partially makes errors. This paper aims to investigate whether it is possible to retrain the RoBERTa language model to increase the number of sentence commas the model correctly detects. We have chosen a very specific and narrow type of sentence comma, namely the sentence comma delimiting vocative phrases, which is clearly defined in the grammar and is very often omitted by writers. The chosen approaches were further tested and evaluated on different types of texts.</p>
</abstract>

Although Czech rule-based tools for automatic punctuation insertion rely on extensive grammar and achieve respectable precision, the pre-trained Transformers outperform rule-based systems in precision and recall (Machura et al. 2022). The Czech pre-trained RoBERTa model achieves excellent results, yet a certain level of phenomena is ignored, and the model partially makes errors. This paper aims to investigate whether it is possible to retrain the RoBERTa language model to increase the number of sentence commas the model correctly detects. We have chosen a very specific and narrow type of sentence comma, namely the sentence comma delimiting vocative phrases, which is clearly defined in the grammar and is very often omitted by writers. The chosen approaches were further tested and evaluated on different types of texts.

Is it Possible to Re-Educate Roberta? Expert-Driven Machine Learning for Punctuation Correction

Department of Czech Language, Faculty of Arts

Department of Cybernetics, Faculty of Applied Sciences

Journal of Linguistics/Jazykovedný casopis

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Although Czech rule-based tools for automatic punctuation insertion rely on extensive grammar and achieve respectable precision, the pre-trained Transformers...

Is it Possible to Re-Educate Roberta? Expert-Driven Machine Learning for Punctuation Correction

Published Online: Dec 25, 2023

Page range: 357 - 368

DOI: https://doi.org/10.2478/jazcas-2023-0052

Keywords
comma, Czech, vocative, machine learning, RoBERTa

© 2023 Jakub Machura et al., published by Sciendo

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Is it Possible to Re-Educate Roberta? Expert-Driven Machine Learning for Punctuation Correction

Published Online: Dec 25, 2023

Page range: 357 - 368

DOI: https://doi.org/10.2478/jazcas-2023-0052

Keywordscomma, Czech, vocative, machine learning, RoBERTa

© 2023 Jakub Machura et al., published by Sciendo

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Keywords
comma, Czech, vocative, machine learning, RoBERTa