New Challenges in Scientific Publications: Referencing, Artificial Intelligence and ChatGPT

During the COVID-19 pandemic, the scientific community saw a surge in articles submitted to various journals for publication. Some of these journals have bypassed the usual peer-review process to meet the increased demand, resulting in the publication of papers that they would not normally accept (e.g., small, uncontrolled studies, etc.) (1). Other authors have then cited these studies, which has led to almost any claim being supported by a reference, no matter how absurd. At about the same time we have seen the emergence of artificial intelligence tools, especially the language model ChatGPT, which has already changed the way many people write articles (2). Additionally, artificial intelligence (AI) will without doubt enhance the writing of scientific papers, as it makes such writing easier and less time consuming.

References are a mandatory part of any scientific and professional article. They serve to prove that the authors’ claims are based on sound prior knowledge (3). However, with the rapid rise in published articles and other sources of information, both the purpose and need for references are being increasingly questioned (4). Science is making more information freely available, but the number of articles published far exceeds the increase in knowledge. New types of quasi-scientific publications have become a serious issue for scientific work (5). All this has led to a situation in which it is now possible to claim anything and provide a reference to support that claim. This problem was particularly evident in the toxic debates during the COVID-19 pandemic (6, 7).

The quality of journals is almost exclusively measured by their impact factor. Although the fact that many experts consider this to be an inadequate approach (8), the practice of relying solely on quantitative measures of quality is still widespread. As a consequence, the selection of references by the authors can be done in such a way that it resembles gathering likes on Facebook more than a rigorous review of existing knowledge (9).

Quantitative evaluation of the impact of an article and the scientific value of a researcher’s work carries the risk that researchers tend to choose a research idea that will most likely generate favourable metrics. Therefore, rather than choosing an interesting and important topic, the likelihood of being cited in other articles is the main driver for developing new knowledge and insights. This could have serious consequences for the development of humanity, as science will not be focused on improving the world and human life, but on maintaining the artificial evaluation of the impact of research.

All this requires a rethinking of how to evaluate the quality and scientific value of articles. Most authors, peer reviewers, and editors act in good faith and do their jobs to the best of their abilities. Pre-publication peer review – coordinated by journal editors - tries to separate the wheat from the chaff. A lot of journals are now reducing the number of references and some journals have already limited the number to 30. Moreover, new trends in research assessment show a clear move away from a purely quantitative approach (10).

Critical evaluation of cited articles, including the references, is becoming increasingly important. It is no longer enough for a reviewer or editor to limit the assessment to evaluate the research question, methodology, and discussion.

Theoretically, the “Introduction and Discussion” part of the manuscript should be based on a systematic review of the existing literature, and therefore only such articles should be cited.

AI has been around for some time, though many authors may not have been aware of it. Today, however, we are witnessing a sudden surge in AI tools for literature search, analysis, synthesis, and even writing.

Just three years ago, we had to manually search PubMed using Boolean operators, prioritize searches, mark phrases, search for synonyms and word roots for truncation, and set search fields. Now, however, AI algorithms do this for us.

The newest AI-based tools for finding relevant and authoritative articles are able to search for articles not only by analysing metadata; they use citation information and natural language processing and machine learning algorithms to analyse articles and prioritize them based on their relevance and scientific impact to a specific query. They also visually represent links between connected/cited articles (and even cited statements; eg. scite) and topics (Semantic Scholar, Connected Papers, Research Rabbit, Litmaps), and generate accurate in-text citations and bibliographies through integrated reference manager tools. Some of these tools (scite) can also be used for research articles assessment (11).

AI-based literature search, analysis and synthesis tools (Elicit, Scolarcy, ChatGPT, Bing AI) can help authors to synthesize article knowledge quickly, easily and clearly. Previously, authors would spend days sorting through articles to collect the most important findings. By automatically analysing and summarizing data, AI tools can now generate summaries of key article information, present them in easy-to-read tables, generate titles and abstracts (Writefull, Scolarcy, Abstract Generator), and even sections of a manuscript (ChatGPT). Major scientific publishers offer authors the possibility to search for an appropriate journal for manuscript submission (Springer Journal Suggester, Elsevier Journal Finder) by providing the title and abstract of the manuscript.

Plagiarism-detection tools help editors and authors by verifying the degree of similarity between a focal manuscript and other publications. Authors can also refine their manuscripts linguistically before submitting them to the chosen journal, as AI-based writing tools can help authors with grammar, spelling, and formatting, suggesting changes in vocabulary and punctuation, as well as more accurate and appropriate words and phrases or paraphrase the text. In the past, authors had to rely on online dictionaries and paid proofreaders, while today computer tools (Grammarly, Writefull) can correct English while authors are writing, not just based on the semantics and syntax of the language but also on the statistics of word-sequence usage in a large collection of texts.

With the release of ChatGPT in November 2022, the possibilities of artificial intelligence became even more evident. This large language model makes possible things we had not expected. It is now not necessary to search through document links to find the desired information, as instead authors are able to receive the information they need in their native language and converse about it with a chatbot in order to refine their answers. The model easily analyses and synthesizes our natural language. Often, the model opens our eyes to the breadth and depth of a problem, making it very useful for thinking about new research questions. Within five days, one million users had used ChatGPT, which is something that Facebook took ten months to achieve (12), and within two months some 100 million users had already tried it. There is, of course, a need for caution. A large language model can only be used as personal assistant, and its suggestions should always be verified since they are not always reliable. They can easily mislead the author and reader because the model is known to make factual errors, generate non-existing references and stubbornly and convincingly defend statements that may be false. It is also extremely important to use it ethically and for the benefit of humanity.

ChatGPT can also be used for the writing of scientific articles (2). It can provide prompts or topics related to research and generate text based on its understanding of the subject matter. However, it cannot create new, original ideas. It is important to note that as an AI language model, it generates text based on patterns it has learned from its training data. While it can create coherent and grammatically correct sentences, it may not always generate accurate or precise scientific content. Hence, it is essential to review and edit the text carefully to ensure accuracy and clarity. Additionally, it is important to acknowledge any sources or references used in the article to avoid plagiarism.

Many AI tools are in the process of development and training. They learn from users also and many of them are currently free. Unfortunately, at the moment there is no general AI tool for researchers or editors. They thus have to use many different AI tools at specific stages of their work and keep abreast of developments to ensure they are using the most appropriate ones.

The best advice to authors is not to waste time debating whether to try out AI tools or not, but to take the first step towards the future and learn about their potential and limitations. Currently, optimism is high for what AI tools can offer this community, and the wider world.

With AI tools that are available for scientific literature search, analysis, synthesis, writing and assessment, new opportunities are emerging for authors, reviewers and editors that will probably change scientific publishing as we know it. This requires also a rethinking of the research assessment methodology. Researchers and editors of scientific journals should be aware of the opportunities and limitations that modern technology is bringing to this context, and discussions are now vital in order to suggest improvements and develop an appropriate way of publishing and evaluating scientific findings in this new and evolving environment.