<abstract xmlns="http://www.w3.org/1999/xhtml">

<sec>
<h3>Purpose</h3>
<p>The aim of this article is to explore up to seven parameters related to the methodological quality and reproducibility of thematic bibliometric research published in the two most productive journals in bibliometrics, Sustainability (a journal outside the discipline) and Scientometrics, the flagship journal in the field.</p>
</sec>
<sec>
<h3>Design/methodology/approach</h3>
<p>The study identifies the need for developing tailored tools for improving the quality of thematic bibliometric analyses, and presents a framework that can guide the development of such tools. A total of 508 papers are analysed, 77% of Sustainability, and 23% published in Scientometrics, for the 2019-2021 period.</p>
</sec>
<sec>
<h3>Findings</h3>
<p>An average of 2.6 shortcomings per paper was found for the whole sample, with an almost identical number of flaws in both journals. Sustainability has more flaws than Scientometrics in four of the seven parameters studied, while Scientometrics has more shortcomings in the remaining three variables.</p>
</sec>
<sec>
<h3>Research limitations</h3>
<p>The first limitation of this work is that it is a study of two scientific journals, so the results cannot be directly extrapolated to the set of thematic bibliometric analyses published in journals from all fields.</p>
</sec>
<sec>
<h3>Practical implications</h3>
<p>We propose the adoption of protocols, guidelines, and other similar tools, adapted to bibliometric practice, which could increase the thoroughness, transparency, and reproducibility of this type of research.</p>
</sec>
<sec>
<h3>Originality/value</h3>
<p>These results show considerable room for improvement in terms of the adequate use and breakdown of methodological procedures in thematic bibliometric research, both in journals in the Information Science area and journals outside the discipline.</p>
</sec>
</abstract>

Purpose
The aim of this article is to explore up to seven parameters related to the methodological quality and reproducibility of thematic bibliometric research published in the two most productive journals in bibliometrics, Sustainability (a journal outside the discipline) and Scientometrics, the flagship journal in the field.


Design/methodology/approach
The study identifies the need for developing tailored tools for improving the quality of thematic bibliometric analyses, and presents a framework that can guide the development of such tools. A total of 508 papers are analysed, 77% of Sustainability, and 23% published in Scientometrics, for the 2019-2021 period.


Findings
An average of 2.6 shortcomings per paper was found for the whole sample, with an almost identical number of flaws in both journals. Sustainability has more flaws than Scientometrics in four of the seven parameters studied, while Scientometrics has more shortcomings in the remaining three variables.


Research limitations
The first limitation of this work is that it is a study of two scientific journals, so the results cannot be directly extrapolated to the set of thematic bibliometric analyses published in journals from all fields.


Practical implications
We propose the adoption of protocols, guidelines, and other similar tools, adapted to bibliometric practice, which could increase the thoroughness, transparency, and reproducibility of this type of research.


Originality/value
These results show considerable room for improvement in terms of the adequate use and breakdown of methodological procedures in thematic bibliometric research, both in journals in the Information Science area and journals outside the discipline.

The need to develop tailored tools for improving the quality of thematic bibliometric analyses: Evidence from papers published in Sustainability and Scientometrics

Journal of Data and Information Science

This work is licensed under the Creative Commons Attribution 4.0 International License.

Purpose
The aim of this article is to explore up to seven parameters related to the methodological quality and reproducibility of thematic bibliometric...

The need to develop tailored tools for improving the quality of thematic bibliometric analyses: Evidence from papers published in Sustainability and Scientometrics

Article Category: Research Paper

Published Online: Sep 22, 2023

Page range: 10 - 35

Received: Jun 20, 2023

Accepted: Sep 01, 2023

DOI: https://doi.org/10.2478/jdis-2023-0021

Keywords
Thematic bibliometric analyses, Sustainability, Scientometrics, Reproducibility, Methodological quality

© 2023 Alvaro Cabezas-Clavijo et al., published by Sciendo

This work is licensed under the Creative Commons Attribution 4.0 International License.

The need to develop tailored tools for improving the quality of thematic bibliometric analyses: Evidence from papers published in Sustainability and Scientometrics

Article Category: Research Paper

Published Online: Sep 22, 2023

Page range: 10 - 35

Received: Jun 20, 2023

Accepted: Sep 01, 2023

DOI: https://doi.org/10.2478/jdis-2023-0021

KeywordsThematic bibliometric analyses, Sustainability, Scientometrics, Reproducibility, Methodological quality

© 2023 Alvaro Cabezas-Clavijo et al., published by Sciendo

This work is licensed under the Creative Commons Attribution 4.0 International License.

Keywords
Thematic bibliometric analyses, Sustainability, Scientometrics, Reproducibility, Methodological quality