<abstract xmlns="http://www.w3.org/1999/xhtml">

<p>The emergence of the ChatGPT language model has sparked significant interest in educational circles, thanks to its capabilities in generating text, understanding context, and engaging in logical reasoning and conversation. This study explores the impact of ChatGPT on English writing, integrating statistical language models and TF-IDF word frequency analysis to develop a method for identifying and scoring deviations in English essays. Our findings demonstrate the model’s high accuracy in detecting digressions across three categories (88%, 73.33%, and 60%), and its scoring closely aligns with human assessments, with a mean error of just 5.17 and a Pearson correlation coefficient of 0.8024. This evidence supports the model’s reliability and significant potential for application in English writing evaluation.</p>
</abstract>

The emergence of the ChatGPT language model has sparked significant interest in educational circles, thanks to its capabilities in generating text, understanding context, and engaging in logical reasoning and conversation. This study explores the impact of ChatGPT on English writing, integrating statistical language models and TF-IDF word frequency analysis to develop a method for identifying and scoring deviations in English essays. Our findings demonstrate the model’s high accuracy in detecting digressions across three categories (88%, 73.33%, and 60%), and its scoring closely aligns with human assessments, with a mean error of just 5.17 and a Pearson correlation coefficient of 0.8024. This evidence supports the model’s reliability and significant potential for application in English writing evaluation.

ChatGPT Analysis of Strengths and Weaknesses in English Writing and Their Implications

Applied Mathematics and Nonlinear Sciences

This work is licensed under the Creative Commons Attribution 4.0 International License.

The emergence of the ChatGPT language model has sparked significant interest in educational circles, thanks to its capabilities in generating text,...

ChatGPT Analysis of Strengths and Weaknesses in English Writing and Their Implications

Published Online: May 03, 2024

Page range: -

Received: Apr 01, 2024

Accepted: Apr 15, 2024

DOI: https://doi.org/10.2478/amns-2024-0912

Keywords
Statistical language modeling, TF-IDF word frequency statistics, ChatGPT, Off-topic detection, English writing

© 2024 Zheng Li et al., published by Sciendo

This work is licensed under the Creative Commons Attribution 4.0 International License.

ChatGPT Analysis of Strengths and Weaknesses in English Writing and Their Implications

Published Online: May 03, 2024

Page range: -

Received: Apr 01, 2024

Accepted: Apr 15, 2024

DOI: https://doi.org/10.2478/amns-2024-0912

KeywordsStatistical language modeling, TF-IDF word frequency statistics, ChatGPT, Off-topic detection, English writing

© 2024 Zheng Li et al., published by Sciendo

This work is licensed under the Creative Commons Attribution 4.0 International License.

Keywords
Statistical language modeling, TF-IDF word frequency statistics, ChatGPT, Off-topic detection, English writing