<abstract><title style='display:none'>Abstract</title><sec id="j_jdis-2017-0006_s_006_w2aab2b8c32b1b7b1aab1c15b1Aa"><title style='display:none'>Purpose</title>We present a systematic review of the literature concerning major aspects of science mapping to serve two primary purposes: First, to demonstrate the use of a science mapping approach to perform the review so that researchers may apply the procedure to the review of a scientific domain of their own interest, and second, to identify major areas of research activities concerning science mapping, intellectual milestones in the development of key specialties, evolutionary stages of major specialties involved, and the dynamics of transitions from one specialty to another.</sec><sec id="j_jdis-2017-0006_s_007_w2aab2b8c32b1b7b1aab1c15b2Aa"><title style='display:none'>Design/methodology/approach</title>We first introduce a theoretical framework of the evolution of a scientific specialty. Then we demonstrate a generic search strategy that can be used to construct a representative dataset of bibliographic records of a domain of research. Next, progressively synthesized co-citation networks are constructed and visualized to aid visual analytic studies of the domain’s structural and dynamic patterns and trends. Finally, trajectories of citations made by particular types of authors and articles are presented to illustrate the predictive potential of the analytic approach.</sec><sec id="j_jdis-2017-0006_s_008_w2aab2b8c32b1b7b1aab1c15b3Aa"><title style='display:none'>Findings</title>The evolution of the science mapping research involves the development of a number of interrelated specialties. Four major specialties are discussed in detail in terms of four evolutionary stages: conceptualization, tool construction, application, and codification. Underlying connections between major specialties are also explored. The predictive analysis demonstrates citations trajectories of potentially transformative contributions.</sec><sec id="j_jdis-2017-0006_s_009_w2aab2b8c32b1b7b1aab1c15b4Aa"><title style='display:none'>Research limitations</title>The systematic review is primarily guided by citation patterns in the dataset retrieved from the literature. The scope of the data is limited by the source of the retrieval, i.e. the Web of Science, and the composite query used. An iterative query refinement is possible if one would like to improve the data quality, although the current approach serves our purpose adequately. More in-depth analyses of each specialty would be more revealing by incorporating additional methods such as citation context analysis and studies of other aspects of scholarly publications.</sec><sec id="j_jdis-2017-0006_s_010_w2aab2b8c32b1b7b1aab1c15b5Aa"><title style='display:none'>Practical implications</title>The underlying analytic process of science mapping serves many practical needs, notably bibliometric mapping, knowledge domain visualization, and visualization of scientific literature. In order to master such a complex process of science mapping, researchers often need to develop a diverse set of skills and knowledge that may span multiple disciplines. The approach demonstrated in this article provides a generic method for conducting a systematic review.</sec><sec id="j_jdis-2017-0006_s_011_w2aab2b8c32b1b7b1aab1c15b6Aa"><title style='display:none'>Originality/value</title>Incorporating the evolutionary stages of a specialty into the visual analytic study of a research domain is innovative. It provides a systematic methodology for researchers to achieve a good understanding of how scientific fields evolve, to recognize potentially insightful patterns from visually encoded signs, and to synthesize various information so as to capture the state of the art of the domain.</sec></abstract>

Science Mapping: A Systematic Review of the Literature

<abstract><title style='display:none'>Abstract</title><sec id="j_jdis-2019-0001_s_006_w2aab3b7b1b1b6b1aab1c17b1Aa"><title style='display:none' id="d4944091e3411_w2aab3b7b1b1b6b1aab1c17b1aAa">Purpose</title>The purpose of this study is to assess the coverage of the scientific literature in Scopus and Web of Science from the perspective of research evaluation.</sec><sec id="j_jdis-2019-0001_s_007_w2aab3b7b1b1b6b1aab1c17b2Aa"><title style='display:none' id="d4944091e3419_w2aab3b7b1b1b6b1aab1c17b2aAa">Design/methodology/approach</title>The academic communities of Norway have agreed on certain criteria for what should be included as original research publications in research evaluation and funding contexts. These criteria have been applied since 2004 in a comprehensive bibliographic database called the Norwegian Science Index (NSI). The relative coverages of Scopus and Web of Science are compared with regard to publication type, field of research and language.</sec><sec id="j_jdis-2019-0001_s_008_w2aab3b7b1b1b6b1aab1c17b3Aa"><title style='display:none' id="d4944091e3427_w2aab3b7b1b1b6b1aab1c17b3aAa">Findings</title>Our results show that Scopus covers 72 percent of the total Norwegian scientific and scholarly publication output in 2015 and 2016, while the corresponding figure for Web of Science Core Collection is 69 percent. The coverages are most comprehensive in medicine and health (89 and 87 percent) and in the natural sciences and technology (85 and 84 percent). The social sciences (48 percent in Scopus and 40 percent in Web of Science Core Collection) and particularly the humanities (27 and 23 percent) are much less covered in the two international data sources.</sec><sec id="j_jdis-2019-0001_s_009_w2aab3b7b1b1b6b1aab1c17b4Aa"><title style='display:none' id="d4944091e3435_w2aab3b7b1b1b6b1aab1c17b4aAa">Research limitation</title>Comparing with data from only one country is a limitation of the study, but the criteria used to define a country’s scientific output as well as the identification of patterns of field-dependent partial representations in Scopus and Web of Science should be recognizable and useful also for other countries.</sec><sec id="j_jdis-2019-0001_s_010_w2aab3b7b1b1b6b1aab1c17b5Aa"><title style='display:none' id="d4944091e3444_w2aab3b7b1b1b6b1aab1c17b5aAa">Originality/value</title>The novelty of this study is the criteria-based approach to studying coverage problems in the two data sources.</sec></abstract>

A Criteria-based Assessment of the Coverage of Scopus and Web of Science

<abstract><title style='display:none'>Abstract</title>Due to the recent explosion of big data, our society has been rapidly going through digital transformation and entering a new world with numerous eye-opening developments. These new trends impact the society and future jobs, and thus student careers. At the heart of this digital transformation is data science, the discipline that makes sense of big data. With many rapidly emerging digital challenges ahead of us, this article discusses perspectives on iSchools’ opportunities and suggestions in data science education. We argue that iSchools should empower their students with “information computing” disciplines, which we define as the ability to solve problems and create values, information, and knowledge using tools in application domains. As specific approaches to enforcing information computing disciplines in data science education, we suggest the three foci of user-based, tool-based, and application-based. These three foci will serve to differentiate the data science education of iSchools from that of computer science or business schools. We present a layered Data Science Education Framework (DSEF) with building blocks that include the three pillars of data science (people, technology, and data), computational thinking, data-driven paradigms, and data science lifecycles. Data science courses built on the top of this framework should thus be executed with user-based, tool-based, and application-based approaches. This framework will help our students think about data science problems from the big picture perspective and foster appropriate problem-solving skills in conjunction with broad perspectives of data science lifecycles. We hope the DSEF discussed in this article will help fellow iSchools in their design of new data science curricula.</abstract>

Big Data and Data Science: Opportunities and Challenges of iSchools

<abstract><title style='display:none'>Abstract</title><sec id="j_jdis-2017-0012_s_006_w2aab3b8c28b1b7b1aab1c15b1Aa"><title style='display:none'>Purpose</title>The purpose of the paper is to provide a framework for addressing the disconnect between metadata and data science. Data science cannot progress without metadata research. This paper takes steps toward advancing the synergy between metadata and data science, and identifies pathways for developing a more cohesive metadata research agenda in data science.</sec><sec id="j_jdis-2017-0012_s_007_w2aab3b8c28b1b7b1aab1c15b2Aa"><title style='display:none'>Design/methodology/approach</title>This paper identifies factors that challenge metadata research in the digital ecosystem, defines metadata and data science, and presents the concepts <italic>big metadata</italic>, <italic>smart metadata</italic>, and <italic>metadata capital</italic> as part of a metadata <italic>lingua franca</italic> connecting to data science.</sec><sec id="j_jdis-2017-0012_s_008_w2aab3b8c28b1b7b1aab1c15b3Aa"><title style='display:none'>Findings</title>The “utilitarian nature” and “historical and traditional views” of metadata are identified as two intersecting factors that have inhibited metadata research. Big metadata, smart metadata, and metadata capital are presented as part of a metadata <italic>lingua franca</italic> to help frame research in the data science research space.</sec><sec id="j_jdis-2017-0012_s_009_w2aab3b8c28b1b7b1aab1c15b4Aa"><title style='display:none'>Research limitations</title>There are additional, intersecting factors to consider that likely inhibit metadata research, and other significant metadata concepts to explore.</sec><sec id="j_jdis-2017-0012_s_010_w2aab3b8c28b1b7b1aab1c15b5Aa"><title style='display:none'>Practical implications</title>The immediate contribution of this work is that it may elicit response, critique, revision, or, more significantly, motivate research. The work presented can encourage more researchers to consider the significance of metadata as a research worthy topic within data science and the larger digital ecosystem.</sec><sec id="j_jdis-2017-0012_s_011_w2aab3b8c28b1b7b1aab1c15b6Aa"><title style='display:none'>Originality/value</title>Although metadata research has not kept pace with other data science topics, there is little attention directed to this problem. This is surprising, given that metadata is essential for data science endeavors. This examination synthesizes original and prior scholarship to provide new grounding for metadata research in data science.</sec></abstract>

Big Metadata, Smart Metadata, and Metadata Capital: Toward Greater Synergy Between Data Science and Metadata

<abstract><title style='display:none'>Abstract</title><sec id="j_jdis-2017-0002_s_007_w2aab2b8b7b1b7b1aab1c15b1Aa"><title style='display:none'>Purpose</title>First, to review the state-of-the-art in patent citation analysis, particularly characteristics of patent citations to scientific literature (scientific non-patent references, SNPRs). Second, to present a novel mapping approach to identify technology-relevant research based on the papers cited by and referring to the SNPRs.</sec><sec id="j_jdis-2017-0002_s_008_w2aab2b8b7b1b7b1aab1c15b2Aa"><title style='display:none'>Design/methodology/approach</title>In the review part we discuss the context of SNPRs such as the time lags between scientific achievements and inventions. Also patent-to-patent citation is addressed particularly because this type of patent citation analysis is a major element in the assessment of the economic value of patents. We also review the research on the role of universities and researchers in technological development, with important issues such as universities as sources of technological knowledge and inventor-author relations. We conclude the review part of this paper with an overview of recent research on mapping and network analysis of the science and technology interface and of technological progress in interaction with science. In the second part we apply new techniques for the direct visualization of the cited and citing relations of SNPRs, the mapping of the landscape around SNPRs by bibliographic coupling and co-citation analysis, and the mapping of the conceptual environment of SNPRs by keyword co-occurrence analysis.</sec><sec id="j_jdis-2017-0002_s_009_w2aab2b8b7b1b7b1aab1c15b3Aa"><title style='display:none'>Findings</title>We discuss several properties of SNPRs. Only a small minority of publications covered by the Web of Science or Scopus are cited by patents, about 3%–4%. However, for publications based on university-industry collaboration the number of SNPRs is considerably higher, around 15%. The proposed mapping methodology based on a “second order SNPR approach” enables a better assessment of the technological relevance of research.</sec><sec id="j_jdis-2017-0002_s_010_w2aab2b8b7b1b7b1aab1c15b4Aa"><title style='display:none'>Research limitations</title>The main limitation is that a more advanced merging of patent and publication data, in particular unification of author and inventor names, in still a necessity.</sec><sec id="j_jdis-2017-0002_s_011_w2aab2b8b7b1b7b1aab1c15b5Aa"><title style='display:none'>Practical implications</title>The proposed mapping methodology enables the creation of a database of technology-relevant papers (TRPs). In a bibliometric assessment the publications of research groups, research programs or institutes can be matched with the TRPs and thus the extent to which the work of groups, programs or institutes are relevant for technological development can be measured.</sec><sec id="j_jdis-2017-0002_s_012_w2aab2b8b7b1b7b1aab1c15b6Aa"><title style='display:none'>Originality/value</title>The review part examines a wide range of findings in the research of patent citation analysis. The mapping approach to identify a broad range of technology-relevant papers is novel and offers new opportunities in research evaluation practices.</sec></abstract>

Patent Citations Analysis and Its Value in Research Evaluation: A Review and a New Approach to Map Technology-relevant Research

<abstract><title style='display:none'>Abstract</title><sec id="j_jdis.201623_s_005_w2aab2b8b6b1b7b1aab1c15b1Aa"><title style='display:none'>Purpose</title>To address the under-reporting of research results, with emphasis on the under-reporting/distorted reporting of adverse events in the biomedical research literature.</sec><sec id="j_jdis.201623_s_006_w2aab2b8b6b1b7b1aab1c15b2Aa"><title style='display:none'>Design/methodology/approach</title>A four-step approach is used: (1) To identify the characteristics of literature that make it adequate to support policy; (2) to show how each of these characteristics becomes degraded to make inadequate literature; (3) to identify incentives to prevent inadequate literature; and (4) to show policy implications of inadequate literature.</sec><sec id="j_jdis.201623_s_007_w2aab2b8b6b1b7b1aab1c15b3Aa"><title style='display:none'>Findings</title>This review has provided reasons for, and examples of, adverse health effects of myriad substances (1) being under-reported in the premiere biomedical literature, or (2) entering this literature in distorted form. Since there is no way to gauge the extent of this under/distorted-reporting, the quality and credibility of the ‘premiere’ biomedical literature is unknown. Therefore, any types of meta-analyses or scientometric analyses of this literature will have unknown quality and credibility. The most sophisticated scientometric analysis cannot compensate for a highly flawed database.</sec><sec id="j_jdis.201623_s_008_w2aab2b8b6b1b7b1aab1c15b4Aa"><title style='display:none'>Research limitations</title>The main limitation is in identifying examples of under-reporting. There are many incentives for under-reporting and few dis-incentives.</sec><sec id="j_jdis.201623_s_009_w2aab2b8b6b1b7b1aab1c15b5Aa"><title style='display:none'>Practical implications</title>Almost all research publications, addressing causes of disease, treatments for disease, diagnoses for disease, scientometrics of disease and health issues, and other aspects of healthcare, build upon previous healthcare-related research published. Many researchers will not have laboratories or other capabilities to replicate or validate the published research, and depend almost completely on the integrity of this literature. If the literature is distorted, then future research can be misguided, and health policy recommendations can be ineffective or worse.</sec><sec id="j_jdis.201623_s_010_w2aab2b8b6b1b7b1aab1c15b6Aa"><title style='display:none'>Originality/value</title>This review has examined a much wider range of technical and non-technical causes for under-reporting of adverse events in the biomedical literature than previous studies.</sec></abstract>

Under-reporting of Adverse Events in the Biomedical Literature

<abstract>
<title style='display:none'>Abstract</title>
<sec>
<title style='display:none'>Purpose</title>
This study aims to evaluate the accuracy of authorship attributions in scientific publications, focusing on the fairness and precision of individual contributions within academic works.
</sec>
<sec>
<title style='display:none'>Design/methodology/approach</title>
The study analyzes 81,823 publications from the journal <italic>PLOS ONE</italic>, covering the period from January 2018 to June 2023. It examines the authorship attributions within these publications to try and determine the prevalence of inappropriate authorship. It also investigates the demographic and professional profiles of affected authors, exploring trends and potential factors contributing to inaccuracies in authorship.
</sec>
<sec>
<title style='display:none'>Findings</title>
Surprisingly, 9.14% of articles feature at least one author with inappropriate authorship, affecting over 14,000 individuals (2.56% of the sample). Inappropriate authorship is more concentrated in Asia, Africa, and specific European countries like Italy. Established researchers with significant publication records and those affiliated with companies or nonprofits show higher instances of potential monetary authorship.
</sec>
<sec>
<title style='display:none'>Research limitations</title>
Our findings are based on contributions as declared by the authors, which implies a degree of trust in their transparency. However, this reliance on self-reporting may introduce biases or inaccuracies into the dataset. Further research could employ additional verification methods to enhance the reliability of the findings.
</sec>
<sec>
<title style='display:none'>Practical implications</title>
These findings have significant implications for journal publishers, highlighting the necessity for robust control mechanisms to ensure the integrity of authorship attributions. Moreover, researchers must exercise discernment in determining when to acknowledge a contributor and when to include them in the author list. Addressing these issues is crucial for maintaining the credibility and fairness of academic publications.
</sec>
<sec>
<title style='display:none'>Originality/value</title>
This study contributes to an understanding of critical issues within academic authorship, shedding light on the prevalence and impact of inappropriate authorship attributions. By calling for a nuanced approach to ensure accurate credit is given where it is due, the study underscores the importance of upholding ethical standards in scholarly publishing.
</sec>
</abstract>

Beyond authorship: Analyzing contributions in <italic>PLOS ONE</italic> and the challenges of appropriate attribution

<abstract>
<title style='display:none'>Abstract</title>
<sec>
<title style='display:none'>Purpose</title>
The goal of this study is a comparative analysis of the relation between funding (a main driver for scientific research) and citations in papers of Nobel Laureates in physics, chemistry and medicine over 2019-2020 and the same relation in these research fields as a whole.
</sec>
<sec>
<title style='display:none'>Design/Methodology/Approach</title>
This study utilizes a power law model to explore the relationship between research funding and citations of related papers. The study here analyzes 3,539 recorded documents by Nobel Laureates in physics, chemistry and medicine and a broader dataset of 183,016 documents related to the fields of physics, medicine, and chemistry recorded in the Web of Science database.
</sec>
<sec>
<title style='display:none'>Findings</title>
Results reveal that in chemistry and medicine, funded researches published in papers of Nobel Laureates have higher citations than unfunded studies published in articles; vice versa high citations of Nobel Laureates in physics are for unfunded studies published in papers. Instead, when overall data of publications and citations in physics, chemistry and medicine are analyzed, all papers based on funded researches show higher citations than unfunded ones.
</sec>
<sec>
<title style='display:none'>Originality/Value</title>
Results clarify the driving role of research funding for science diffusion that are systematized in general properties: a) articles concerning funded researches receive more citations than (un)funded studies published in papers of physics, chemistry and medicine sciences, generating a high Matthew effect (a higher growth of citations with the increase in the number of papers); b) research funding increases the citations of articles in fields oriented to applied research (e.g., chemistry and medicine) more than fields oriented towards basic research (e.g., physics).
</sec>
<sec>
<title style='display:none'>Practical Implications</title>
The results here explain some characteristics of scientific development and diffusion, highlighting the critical role of research funding in fostering citations and the expansion of scientific knowledge. This finding can support decisionmaking of policymakers and R&amp;D managers to improve the effectiveness in allocating financial resources in science policies to generate a higher positive scientific and societal impact.
</sec>
</abstract>

Research funding and citations in papers of Nobel Laureates in Physics, Chemistry and Medicine, 2019-2020

<abstract>
<title style='display:none'>Abstract</title>
Research data infrastructures form the cornerstone in both cyber and physical spaces, driving the progression of the data-intensive scientific research paradigm. This opinion paper presents an overview of global research data infrastructure, drawing insights from national roadmaps and strategic documents related to research data infrastructure. It emphasizes the pivotal role of research data infrastructures by delineating four new missions aimed at positioning them at the core of the current scientific research and communication ecosystem. The four new missions of research data infrastructures are: (1) as a pioneer, to transcend the disciplinary border and address complex, cutting-edge scientific and social challenges with problem- and data-oriented insights; (2) as an architect, to establish a digital, intelligent, flexible research and knowledge services environment; (3) as a platform, to foster the high-end academic communication; (4) as a coordinator, to balance scientific openness with ethics needs.
</abstract>

New roles of research data infrastructure in research paradigm evolution

<abstract>
<title style='display:none'>Abstract</title>
<sec>
<title style='display:none'>Purpose</title>
The notable increase in retraction papers has attracted considerable attention from diverse stakeholders. Various sources are now offering information related to research integrity, including concerns voiced on social media, disclosed lists of paper mills, and retraction notices accessible through journal websites. However, despite the availability of such resources, there remains a lack of a unified platform to consolidate this information, thereby hindering efficient searching and cross-referencing. Thus, it is imperative to develop a comprehensive platform for retracted papers and related concerns. This article aims to introduce “Amend,” a platform designed to integrate information on research integrity from diverse sources.
</sec>
<sec>
<title style='display:none'>Design/methodology/approach</title>
The Amend platform consolidates concerns and lists of problematic articles sourced from social media platforms (e.g., PubPeer, For Better Science), retraction notices from journal websites, and citation databases (e.g., Web of Science, CrossRef). Moreover, Amend includes investigation and punishment announcements released by administrative agencies (e.g., NSFC, MOE, MOST, CAS). Each related paper is marked and can be traced back to its information source via a provided link. Furthermore, the Amend database incorporates various attributes of retracted articles, including citation topics, funding details, open access status, and more. The reasons for retraction are identified and classified as either academic misconduct or honest errors, with detailed subcategories provided for further clarity.
</sec>
<sec>
<title style='display:none'>Findings</title>
Within the Amend platform, a total of 32,515 retracted papers indexed in SCI, SSCI, and ESCI between 1980 and 2023 were identified. Of these, 26,620 (81.87%) were associated with academic misconduct. The retraction rate stands at 6.64 per 10,000 articles. Notably, the retraction rate for non-gold open access articles significantly differs from that for gold open access articles, with this disparity progressively widening over the years. Furthermore, the reasons for retractions have shifted from traditional individual behaviors like falsification, fabrication, plagiarism, and duplication to more organized large-scale fraudulent practices, including Paper Mills, Fake Peer-review, and Artificial Intelligence Generated Content (AIGC).
</sec>
<sec>
<title style='display:none'>Research limitations</title>
The Amend platform may not fully capture all retracted and concerning papers, thereby impacting its comprehensiveness. Additionally, inaccuracies in retraction notices may lead to errors in tagged reasons.
</sec>
<sec>
<title style='display:none'>Practical implications</title>
Amend provides an integrated platform for stakeholders to enhance monitoring, analysis, and research on academic misconduct issues. Ultimately, the Amend database can contribute to upholding scientific integrity.
</sec>
<sec>
<title style='display:none'>Originality/value</title>
This study introduces a globally integrated platform for retracted and concerning papers, along with a preliminary analysis of the evolutionary trends in retracted papers.
</sec>
</abstract>

Amend: an integrated platform of retracted papers and concerned papers

<abstract>
<title style='display:none'>Abstract</title>
<sec>
<title style='display:none'>Purpose</title>
Recently, global science has shown an increasing open trend, however, the characteristics of research integrity of open access (OA) publications have rarely been studied. The aim of this study is to compare the characteristics of retracted articles across different OA levels and discover whether OA level influences the characteristics of retracted articles.
</sec>
<sec>
<title style='display:none'>Design/methodology/approach</title>
The research conducted an analysis of 6,005 retracted publications between 2001 and 2020 from the Web of Science and Retraction Watch databases. These publications were categorized based on their OA levels, including Gold OA, Green OA, and non-OA. The study explored retraction rates, time lags and reasons within these categories.
</sec>
<sec>
<title style='display:none'>Findings</title>
The findings of this research revealed distinct patterns in retraction rates among different OA levels. Publications with Gold OA demonstrated the highest retraction rate, followed by Green OA and non-OA. A comparison of retraction reasons between Gold OA and non-OA categories indicated similar proportions, while Green OA exhibited a higher proportion due to falsification and manipulation issues, along with a lower occurrence of plagiarism and authorship issues. The retraction time lag was shortest for Gold OA, followed by non-OA, and longest for Green OA. The prolonged retraction time for Green OA could be attributed to an atypical distribution of retraction reasons.
</sec>
<sec>
<title style='display:none'>Research limitations</title>
There is no exploration of a wider range of OA levels, such as Hybrid OA and Bronze OA.
</sec>
<sec>
<title style='display:none'>Practical implications</title>
The outcomes of this study suggest the need for increased attention to research integrity within the OA publications. The occurrences of falsification, manipulation, and ethical concerns within Green OA publications warrant attention from the scientific community.
</sec>
<sec>
<title style='display:none'>Originality/value</title>
This study contributes to the understanding of research integrity in the realm of OA publications, shedding light on retraction patterns and reasons across different OA levels.
</sec>
</abstract>

A comparative study on characteristics of retracted publications across different open access levels

<abstract>
<title style='display:none'>Abstract</title>
<sec>
<title style='display:none'>Purpose</title>
Assess whether ChatGPT 4.0 is accurate enough to perform research evaluations on journal articles to automate this time-consuming task.
</sec>
<sec>
<title style='display:none'>Design/methodology/approach</title>
Test the extent to which ChatGPT-4 can assess the quality of journal articles using a case study of the published scoring guidelines of the UK Research Excellence Framework (REF) 2021 to create a research evaluation ChatGPT. This was applied to 51 of my own articles and compared against my own quality judgements.
</sec>
<sec>
<title style='display:none'>Findings</title>
ChatGPT-4 can produce plausible document summaries and quality evaluation rationales that match the REF criteria. Its overall scores have weak correlations with my self-evaluation scores of the same documents (averaging r=0.281 over 15 iterations, with 8 being statistically significantly different from 0). In contrast, the average scores from the 15 iterations produced a statistically significant positive correlation of 0.509. Thus, averaging scores from multiple ChatGPT-4 rounds seems more effective than individual scores. The positive correlation may be due to ChatGPT being able to extract the author’s significance, rigour, and originality claims from inside each paper. If my weakest articles are removed, then the correlation with average scores (r=0.200) falls below statistical significance, suggesting that ChatGPT struggles to make fine-grained evaluations.
</sec>
<sec>
<title style='display:none'>Research limitations</title>
The data is self-evaluations of a convenience sample of articles from one academic in one field.
</sec>
<sec>
<title style='display:none'>Practical implications</title>
Overall, ChatGPT does not yet seem to be accurate enough to be trusted for any formal or informal research quality evaluation tasks. Research evaluators, including journal editors, should therefore take steps to control its use.
</sec>
<sec>
<title style='display:none'>Originality/value</title>
This is the first published attempt at post-publication expert review accuracy testing for ChatGPT.
</sec>
</abstract>

Can ChatGPT evaluate research quality?

<abstract>
<title style='display:none'>Abstract</title>
<sec><title style='display:none'>Purpose</title>Digital literacy and related fields have received interests from scholars and practitioners for more than 20 years; nonetheless, academic communities need to systematically review how the fields have developed. This study aims to investigate the research trends of digital literacy and related concepts since the year of 2000, especially in education.</sec>
<sec><title style='display:none'>Design/methodology/approach</title>The current study analyzes keywords, co-authorship, and cited publications in digital literacy through the scientometric method. The journal articles have been retrieved from the WoS (Web of Science) using four keywords: “Digital literacy,” “ICT literacy,” “information literacy,” and “media literacy.” Further, keywords, publications, and co-authorship are examined and further classified into clusters for more in-depth investigation.</sec>
<sec><title style='display:none'>Findings</title>Digital literacy is a multidisciplinary field that widely embraces literacy, ICT, the Internet, computer skill proficiency, science, nursing, health, and language education. The participants, or study subjects, in digital literacy research range from primary students to professionals, and the co-authorship clusters are distinctive by countries in America and Europe.</sec>
<sec><title style='display:none'>Research limitations</title>This paper analyzes one fixed chunk of a dataset obtained by searching for all four keywords at once. Further studies will retrieve the data from diverse disciplines and will trace the change of the leading research themes by time spans.</sec>
<sec><title style='display:none'>Practical implications</title>To shed light on the findings, using customized digital literacy curriculums and technology is critical for learners at different ages to nurture digital literacy according to their learning aims. They need to cultivate their understanding of the social impact of exploiting technology and computational thinking. To increase the originality of digital literacy-related studies, researchers from different countries and cultures may collaborate to investigate a broader range of digital literacy environments.</sec>
<sec><title style='display:none'>Originality/value</title>The present study reviews research trends in digital literacy and related areas by performing a scientometric study to analyze multidimensional aspects in the fields, including keywords, journal titles, co-authorship, and cited publications.</sec>
</abstract>

A Scientometric Study of Digital Literacy, ICT Literacy, Information Literacy, and Media Literacy

<abstract><title style='display:none'>Abstract</title>The “Norwegian Model” attempts to comprehensively cover all the peer-reviewed scholarly literatures in all areas of research in one single weighted indicator. Thereby, scientific production is made comparable across departments and faculties within and between research institutions, and the indicator may serve institutional evaluation and funding. This article describes the motivation for creating the model in Norway, how it was designed, organized and implemented, as well as the effects and experiences with the model. The article ends with an overview of a new type of bibliometric studies that are based on the type of comprehensive national publication data that the Norwegian Model provides.</abstract>

The Norwegian Model in Norway

Editorial and Peer Review Process

Instructions for Authors

Publication Ethics Statement

Publication Agreement