<abstract xmlns="http://www.w3.org/1999/xhtml">

<p>Cancer research is occasionally described as being in a reproducibility crisis. The cancer literature has ample papers retracted due to misconduct, including the use of paper mills, invalid authorship, or fake data. The objective of this paper was to gain an appreciation of the balance of retractions and associated retraction notices of 23 retracted <italic>Cancer Biotherapy and Radiopharmaceuticals</italic> papers associated with paper mills. By 23 March 2023, these retracted papers had already accumulated 287 citations according to Web of Science Core Collection, 253 according to Scopus, and 365 according to Google Scholar, i.e., metrically speaking, they were highly rewarded. All authors had an affiliation (71% being a hospital) in China. Most (12/21; 57%) of corresponding authors had emails with a @163.com suffix. Four of the retraction notices (i.e., 17%) explicitly indicated paper mills as a reason for retraction although, in general, the retraction notices lacked details and background that could assist readers’ understanding of the retractions.</p>
</abstract>

Cancer research is occasionally described as being in a reproducibility crisis. The cancer literature has ample papers retracted due to misconduct, including the use of paper mills, invalid authorship, or fake data. The objective of this paper was to gain an appreciation of the balance of retractions and associated retraction notices of 23 retracted Cancer Biotherapy and Radiopharmaceuticals papers associated with paper mills. By 23 March 2023, these retracted papers had already accumulated 287 citations according to Web of Science Core Collection, 253 according to Scopus, and 365 according to Google Scholar, i.e., metrically speaking, they were highly rewarded. All authors had an affiliation (71% being a hospital) in China. Most (12/21; 57%) of corresponding authors had emails with a @163.com suffix. Four of the retraction notices (i.e., 17%) explicitly indicated paper mills as a reason for retraction although, in general, the retraction notices lacked details and background that could assist readers’ understanding of the retractions.

<div xmlns="http://www.w3.org/1999/xhtml">
<sec id="j_jdis-2023-0009_s_001">
<label>1</label>
<div>Introduction</div>
<p>Cancer and oncology researches are stated as being in a replication and trust crisis (<a ref-type="bibr" href="#j_jdis-2023-0009_ref_007">Errington et al., 2021</a>; <a ref-type="bibr" href="#j_jdis-2023-0009_ref_017">Teixeira da Silva, 2022</a>). Plagued by issues related to replication, fraud, misconduct, paper mill-derived research and cross-publication forgeries, as well as failed peer review and editorial handling, the number of retracted papers related to cancer research has continued to climb (<a ref-type="bibr" href="#j_jdis-2023-0009_ref_011">Pantziarka &amp; Meheus, 2019</a>; <a ref-type="bibr" href="#j_jdis-2023-0009_ref_013">Retraction Watch, 2022</a>). A salient example, <italic>Tumor Biology</italic>, had one of the highest retraction rates among biomedical journals, many as a result of fake peer review (<a ref-type="bibr" href="#j_jdis-2023-0009_ref_001">Bhatt, 2021</a>). Fake peer review (<a ref-type="bibr" href="#j_jdis-2023-0009_ref_014">Rivera &amp; Teixeira da Silva, 2021</a>) and paper mill-derived fraud (<a ref-type="bibr" href="#j_jdis-2023-0009_ref_006">Else &amp; Van Noorden, 2021</a>) threaten the integrity of and trust in the literature and industry (COPE &amp; STM, 2022). Authors citing retracted literature might not always be aware that they are doing so (<a ref-type="bibr" href="#j_jdis-2023-0009_ref_005">De Cassai et al., 2022</a>).</p>
<p>The objective of this study was to appreciate some metrics- and indexing-related factors of papers that have been retracted from an indexed cancer journal, <italic>Cancer Biotherapy and Radiopharmaceuticals</italic> (<italic>CB&amp;R</italic>), published by Mary Ann Liebert, Inc., and to gain an appreciation of the weight of “paper mills” in this body of retractions. We were spurred to analyze <italic>CB&amp;R</italic> due to the public comments by Susan Jensen, the director of production and editorial operations at Mary Ann Liebert, Inc., who indicated that <italic>CB&amp;R</italic> was identifying, and rejecting, papers that may have been derived from paper mills<sup><a ref-type="fn" href="#j_jdis-2023-0008_fn_001">①</a></sup>. This study attempts to better appreciate <italic>CB&amp;R</italic> retractions associated with paper mills, and the “quality” of their retraction notices (RNs).</p>
</sec>
<sec id="j_jdis-2023-0009_s_002">
<label>2</label>
<div>Data and methods</div>
<p>The <italic>CB&amp;R</italic> website was searched between July 5 and 10, 2022, to identify retractions in this journal. Details were confirmed manually, and against the Retraction Watch database (<a ref-type="bibr" href="#j_jdis-2023-0009_ref_013">Retraction Watch, 2022</a>). Scopus and Web of Science were also consulted (July 10, 2022) to try and draw any additional information of potential interest that could not be gleaned from either the <italic>CB&amp;R</italic> website, or the Retraction Watch database. Background data for all of these aspects of the retracted papers can be found in the Supplementary Table.</p>
</sec>
<sec id="j_jdis-2023-0009_s_003">
<label>3</label>
<div>Results and discussion</div>
<sec id="j_jdis-2023-0009_s_003_s_001">
<label>3.1</label>
<div>Properties of 23 <italic>CB&amp;R</italic> retractions</div>
<p>We identified 23 retractions in <italic>CB&amp;R</italic>, all original articles whose authors had an affiliation in China (i.e., 100%), and 52/73 (71%) of those affiliations were associated with hospitals (<a ref-type="table" href="#j_jdis-2023-0009_tab_001">Table 1</a>). This finding closely mirrors a trend observed by <a ref-type="bibr" href="#j_jdis-2023-0009_ref_022">Zhao et al. (2021)</a>. None of the RNs are pay-walled, and three of the RNs have a correction. Four of the RNs explicitly indicate paper mills as a reason for retraction, but the vast majority of papers were retracted for integrity-based issues related to raw data, data manipulation, or breach of data-related policies (<a ref-type="table" href="#j_jdis-2023-0009_tab_001">Table 1</a>). In Scopus, the retracted status of 17% of the 23 retracted papers is not indicated. For another four papers, the RN is available only when viewing the full description of the document, which might be confusing for Scopus users.</p>
<table-wrap id="j_jdis-2023-0009_tab_001" position="float">
<label>Table 1.</label>
<caption><p><italic>Cancer Biotherapy and Radiopharmaceuticals</italic> retractions (until July 10, 2022), including reason(s) for retraction and country of authors, according to <a ref-type="bibr" href="#j_jdis-2023-0009_ref_013">Retraction Watch (2022)</a>. In addition, retraction notices were consulted to appreciate if retractions were induced by the authors, or by the editor/publisher. Finally, the number of affiliations associated with a hospital was noted.</p></caption>
<table frame="void" rules="none">
<thead>
<tr>
<th>DOI</th>
<th>Reason(s) for retraction<sup><a ref-type="table-fn" href="#j_jdis-2023-0009_table-fn_001">1</a></sup></th>
<th>Explicit author-induced?</th>
<th># Hospital affiliations<a ref-type="table-fn" href="#j_jdis-2023-0009_table-fn_003">**</a></th>
</tr>
</thead>
<tbody>
<tr>
<td><ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://doi.org/10.1089/cbr.2019.3275">https://doi.org/10.1089/cbr.2019.3275</ext-link></td>
<td>Irreproducible results</td>
<td align="center">Yes</td>
<td align="center">4/5</td>
</tr>
<tr>
<td><ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://doi.org/10.1089/cbr.2019.3520">https://doi.org/10.1089/cbr.2019.3520</ext-link></td>
<td>Concerns/issues about data/results; paper mill<a ref-type="table-fn" href="#j_jdis-2023-0009_table-fn_002">*</a></td>
<td align="center">Yes</td>
<td align="center">4/5</td>
</tr>
<tr>
<td><ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://doi.org/10.1089/cbr.2014.1766">https://doi.org/10.1089/cbr.2014.1766</ext-link></td>
<td>Concerns/issues about data/image; investigations/objection by third party</td>
<td align="center">No</td>
<td align="center">1/1</td>
</tr>
<tr>
<td><ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://doi.org/10.1089/cbr.2014.1723">https://doi.org/10.1089/cbr.2014.1723</ext-link></td>
<td>Concerns/issues about data/image; investigations/objection by third party; error in text</td>
<td align="center">No</td>
<td align="center">1/1</td>
</tr>
<tr>
<td><ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://doi.org/10.1089/cbr.2019.3535">https://doi.org/10.1089/cbr.2019.3535</ext-link></td>
<td>Image falsification/fabrication; miscommunication by third party; original data not provided; paper mill<a ref-type="table-fn" href="#j_jdis-2023-0009_table-fn_002">*</a></td>
<td align="center">No</td>
<td align="center">4/4</td>
</tr>
<tr>
<td><ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://doi.org/10.1089/cbr.2020.3563">https://doi.org/10.1089/cbr.2020.3563</ext-link></td>
<td>Concerns/issues about third party involvement; image duplication; data falsification/fabrication; miscommunication by third party</td>
<td align="center">No</td>
<td align="center">4/6</td>
</tr>
<tr>
<td><ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://doi.org/10.1089/cbr.2019.3299">https://doi.org/10.1089/cbr.2019.3299</ext-link></td>
<td>Breach of policy by author; original data not provided; paper mill</td>
<td align="center">No</td>
<td align="center">1/1</td>
</tr>
<tr>
<td><ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://doi.org/10.1089/cbr.2014.1698">https://doi.org/10.1089/cbr.2014.1698</ext-link></td>
<td>Data/text duplication; paper mill<a ref-type="table-fn" href="#j_jdis-2023-0009_table-fn_002">*</a></td>
<td align="center">No</td>
<td align="center">5/9</td>
</tr>
<tr>
<td><ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://doi.org/10.1089/cbr.2014.1728">https://doi.org/10.1089/cbr.2014.1728</ext-link></td>
<td>Data/text duplication; paper mill<a ref-type="table-fn" href="#j_jdis-2023-0009_table-fn_002">*</a></td>
<td align="center">No</td>
<td align="center">1/1</td>
</tr>
<tr>
<td><ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://doi.org/10.1089/cbr.2014.1759">https://doi.org/10.1089/cbr.2014.1759</ext-link></td>
<td>Data/text duplication; paper mill<a ref-type="table-fn" href="#j_jdis-2023-0009_table-fn_002">*</a></td>
<td align="center">No</td>
<td align="center">1/2</td>
</tr>
<tr>
<td><ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://doi.org/10.1089/cbr.2014.1778">https://doi.org/10.1089/cbr.2014.1778</ext-link></td>
<td>Data/text duplication; paper mill<a ref-type="table-fn" href="#j_jdis-2023-0009_table-fn_002">*</a></td>
<td align="center">No</td>
<td align="center">2/3</td>
</tr>
<tr>
<td><ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://doi.org/10.1089/cbr.2017.2306">https://doi.org/10.1089/cbr.2017.2306</ext-link></td>
<td>Breach of policy by author; paper mill</td>
<td align="center">No</td>
<td align="center">1/1</td>
</tr>
<tr>
<td><ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://doi.org/10.1089/cbr.2017.2386">https://doi.org/10.1089/cbr.2017.2386</ext-link></td>
<td>Misconduct by third party; paper mill</td>
<td align="center">Yes</td>
<td align="center">2/2</td>
</tr>
<tr>
<td><ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://doi.org/10.1089/cbr.2018.2625">https://doi.org/10.1089/cbr.2018.2625</ext-link></td>
<td>Unreliable results; paper mill</td>
<td align="center">No</td>
<td align="center">1/1</td>
</tr>
<tr>
<td><ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://doi.org/10.1089/cbr.2018.2626">https://doi.org/10.1089/cbr.2018.2626</ext-link></td>
<td>Breach of policy by author; paper mill</td>
<td align="center">No</td>
<td align="center">1/1</td>
</tr>
<tr>
<td><ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://doi.org/10.1089/cbr.2018.2664">https://doi.org/10.1089/cbr.2018.2664</ext-link></td>
<td>Breach of policy by author; paper mill</td>
<td align="center">No</td>
<td align="center">2/2</td>
</tr>
<tr>
<td><ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://doi.org/10.1089/cbr.2018.2749">https://doi.org/10.1089/cbr.2018.2749</ext-link></td>
<td>Paper mill</td>
<td align="center">No</td>
<td align="center">2/2</td>
</tr>
<tr>
<td><ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://doi.org/10.1089/cbr.2019.2858">https://doi.org/10.1089/cbr.2019.2858</ext-link></td>
<td>Paper mill</td>
<td align="center">No</td>
<td align="center">5/5</td>
</tr>
<tr>
<td><ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://doi.org/10.1089/cbr.2019.2983">https://doi.org/10.1089/cbr.2019.2983</ext-link></td>
<td>Paper mill</td>
<td align="center">No</td>
<td align="center">2/4</td>
</tr>
<tr>
<td><ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://doi.org/10.1089/cbr.2019.3070">https://doi.org/10.1089/cbr.2019.3070</ext-link></td>
<td>Breach of policy by author; paper mill</td>
<td align="center">No</td>
<td align="center">2/4</td>
</tr>
<tr>
<td><ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://doi.org/10.1089/cbr.2018.2545">https://doi.org/10.1089/cbr.2018.2545</ext-link></td>
<td>Concerns/issues about data/image; original data not provided; paper mill</td>
<td align="center">No</td>
<td align="center">3/3</td>
</tr>
<tr>
<td><ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://doi.org/10.1089/cbr.2017.2432">https://doi.org/10.1089/cbr.2017.2432</ext-link></td>
<td>Concerns/issues about data/image; original data not provided; paper mill</td>
<td align="center">No</td>
<td align="center">2/3</td>
</tr>
<tr>
<td><ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://doi.org/10.1089/cbr.2015.1952">https://doi.org/10.1089/cbr.2015.1952</ext-link></td>
<td>Concerns/issues about authorship/data/image/results/third party involvement; paper mill<a ref-type="table-fn" href="#j_jdis-2023-0009_table-fn_002">*</a></td>
<td align="center">No</td>
<td align="center">1/2</td>
</tr>
</tbody>
</table>
<table-wrap-foot>
<fn-group>
<fn id="j_jdis-2023-0009_table-fn_001" symbol="1"><label>1</label><p>Reasons, and wording of reasons, modified from the original statements at Retraction Watch after close examination of the reasons and the papers/websites.</p></fn>
<fn id="j_jdis-2023-0009_table-fn_002" symbol="2"><p>* The term “paper mill” does not appear in the original retraction notice.</p></fn>
<fn id="j_jdis-2023-0009_table-fn_003" symbol="3"><p>** All affiliations from all papers were from China.</p></fn>
</fn-group>
</table-wrap-foot>
</table-wrap>
</sec>
<sec id="j_jdis-2023-0009_s_003_s_002">
<label>3.2</label>
<div>Emails and identities of authors of retracted <italic>CB&amp;R</italic> papers</div>
<p>In addition to the association between paper mill-derived cancer research and affiliations that are Chinese hospitals, the use of web-based emails is also characteristic (<a ref-type="bibr" href="#j_jdis-2023-0009_ref_006">Else &amp; Van Noorden, 2021</a>; <a ref-type="bibr" href="#j_jdis-2023-0009_ref_010">Liu &amp; Chen, 2021</a>; <a ref-type="bibr" href="#j_jdis-2023-0009_ref_016">Teixeira da Silva, 2021</a>; <a ref-type="bibr" href="#j_jdis-2023-0009_ref_022">Zhao et al., 2021</a>). Corresponding authors’ emails were missing from 4/23 papers on the publisher’s website, one paper had three corresponding authors, but none had institutional emails, 12/21 (57%) emails were @163.com (5/21 were @126.com; 4/21 were @sina.com), and 9/21 (43%) emails had prefixes or names that were unrelated to the corresponding author’s name (Suppl. <a ref-type="table" href="#j_jdis-2023-0009_tab_001">Table 1</a>).</p>
<p>The <italic>CB&amp;R</italic> instructions for authors (IFA) made no specific mention of email-related limitations (<italic>CB&amp;R</italic>, 2022). Another cancer journal, <italic>Tumor Biology</italic>, published by IOS Press, explicitly prohibited the use of web-based emails for submission, stating: “The journal does not accept submissions from authors using nondescript, anonymous, email addresses (e.g., <ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://yahoo.com">yahoo.com</ext-link>, <ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://yahoo.com">gmail.com</ext-link>, <ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://163.com">163.com</ext-link>, <ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://rediffmail.com">rediffmail.com</ext-link>, <ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://sina.com">sina.com</ext-link>, <ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://126.com">126.com</ext-link>, <ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://hotmail.com">hotmail.com</ext-link>, etc.).” (<italic>Tumor Biology</italic>, 2022). No ORCID (<underline>O</underline>pen <underline>R</underline>esearcher and <underline>C</underline>ontributor <underline>ID</underline>) for any corresponding author was identified in all 23 retracted papers, even though the <italic>CB&amp;R</italic> IFA indicated that ORCID is mandatory for corresponding authors: “All submitting authors are required to complete their submissions using an ORCID identifier” (<italic>CB&amp;R</italic>, 2022). The existence of blank or disposable (one-time use) ORCIDs, as are sometimes observed in paper mill-derived papers, degrade trust in the validity of author-based identifiers (<a ref-type="bibr" href="#j_jdis-2023-0009_ref_018">Teixeira da Silva, 2022b</a>).</p>
</sec>
<sec id="j_jdis-2023-0009_s_003_s_003">
<label>3.3</label>
<div>The citation of retracted literature</div>
<p>According to Web of Science Core Collection Citations, the 23 retracted papers accumulated (until March 23, 2023) 287 citations (253 according to Scopus and 365 according to Google Scholar). At the same time, these papers continue to receive citations after the retraction: 54 citations according to Web of Science Core Collection, 54 according to Scopus, and 77 according to Google Scholar. Some retracted cancer papers continue to accrue positive (i.e., supporting) citations, although these tend to wane after the second year, and dip to near-zero citations after about a decade (<a ref-type="bibr" href="#j_jdis-2023-0009_ref_008">Hamilton, 2019</a>). Just over 55% of 12,231 Web of Science-indexed retracted papers published between 1981 and 2020 were cited at least once, about a quarter of which were self-citations (<a ref-type="bibr" href="#j_jdis-2023-0009_ref_015">Sharma, 2021</a>). <a ref-type="bibr" href="#j_jdis-2023-0009_ref_015">Sharma (2021)</a> also noted that <italic>Cancer Research</italic> had accrued 106 retractions that were cited on average 46 times. The reasons why authors cite retracted papers are not always clear (<a ref-type="bibr" href="#j_jdis-2023-0009_ref_004">Dal-Ré &amp; Ayuso, 2021</a>), although the citation of older retracted papers tends to wane over time (<a ref-type="bibr" href="#j_jdis-2023-0009_ref_009">Hsiao &amp; Schneider, 2022</a>).</p>
</sec>
<sec id="j_jdis-2023-0009_s_003_s_004">
<label>3.4</label>
<div>Limited information in retraction notices limits their usefulness</div>
<p>We believe that more detailed information in the RNs would have made them more informative. Missing information from a RN, incomplete RNs, or RNs that are opaque or unclear do not benefit academia, and are a poor instrument of information (<a ref-type="bibr" href="#j_jdis-2023-0009_ref_019">Teixeira da Silva &amp; Vuong, 2022</a>; <a ref-type="bibr" href="#j_jdis-2023-0009_ref_021">Xu &amp; Hu, 2022</a>). In our view, a complete and informative RN – irrespective of its size – would have all of the following components to be both informative and transparent, holding the relevant parties accountable: 1) global appreciation of each concern and how it impacts the integrity of the paper, or its data; 2) an understanding of how the concerns came to light; 3) dates of any and all communication between all relevant parties; 4) outcomes of queries and – where possible – perspectives and responses of authors and editors.</p>
</sec>
</sec>
<sec id="j_jdis-2023-0009_s_004">
<label>4</label>
<div>Study limitation</div>
<p>This research note has a limitation, having focused on only one cancer-related journal (<italic>CB&amp;R</italic>), and only on 23 retractions and their associated RNs.</p>
</sec>
</div>

Assessment of retracted papers, and their retraction notices, from a cancer journal associated with “paper mills”

Journal of Data and Information Science

This work is licensed under the Creative Commons Attribution 4.0 International License.

Cancer research is occasionally described as being in a reproducibility crisis. The cancer literature has ample papers retracted due to misconduct, including...

Assessment of retracted papers, and their retraction notices, from a cancer journal associated with “paper mills”

Article Category: Research Note

Published Online: Jun 07, 2023

Page range: 118 - 125

Received: Feb 02, 2023

Accepted: Mar 29, 2023

DOI: https://doi.org/10.2478/jdis-2023-0009

Keywords
Indexing, Oncology, Publishing history, PubMed, Retraction Notices, Transparency

© 2023 Jaime A. Teixeira da Silva et al., published by Sciendo

This work is licensed under the Creative Commons Attribution 4.0 International License.

Assessment of retracted papers, and their retraction notices, from a cancer journal associated with “paper mills”

Article Category: Research Note

Published Online: Jun 07, 2023

Page range: 118 - 125

Received: Feb 02, 2023

Accepted: Mar 29, 2023

DOI: https://doi.org/10.2478/jdis-2023-0009

KeywordsIndexing, Oncology, Publishing history, PubMed, Retraction Notices, Transparency

© 2023 Jaime A. Teixeira da Silva et al., published by Sciendo

This work is licensed under the Creative Commons Attribution 4.0 International License.

Keywords
Indexing, Oncology, Publishing history, PubMed, Retraction Notices, Transparency