Scientific Value Weights more than Being Open or Toll Access: An analysis of the OA advantage in Nature and Science

For measuring hot topics (Banks, 2006; Ye, 2013), we can follow the definition of original h-index to define a hot-degree of a topic (including subjects, keywords, and so on) in sciences, as follows.

Simply, H of a topic means that there are H publications at most, each with H citations at least, on the topic. In WoS, we can easily search H because it is just the h-index of a topic.

For comparing the OA and TA, we can introduce the ratio of hot-degree as a concise indicator (1) R=HOAHTA R = {{{H_{OA}}} \over {{H_{TA}}}} where H_OA is the hot-degree of OA and H_TA is that of TA, for the same topic. When R > 1, i.e. OA > TA, it means open advantages. When R < 1, i.e. TA > OA, it indicates toll advantages.

To clarify the topic-related properties of published papers, we used classical Latent Dirichlet Allocation (LDA, c.f. Blei, Ng, & Jordan, 2003) to cluster the topics of Nature and Science. We divided the Nature dataset into three parts according to its disciplines, namely biomedicine, physics, and others. We performed the same operation on the Science dataset. Taking the Nature biomedicine dataset as an example, we show the data processing flowchart in Figure 1.

Figure 1

The algorithm includes the following steps.

Divided Nature biomedicine dataset into two parts, OA and TA.

The OA topic t_oi and the TA topic t_tj represent the same topic, so the R-index can be calculated from the Pair_{OA – TA}.

We collected data from the Web of Science (WoS) by searching and downloading both OA and TA articles, letters, and reviews published in Nature and Science for the period of 2010–2019. In our research, we only chose papers published in Nature or Science. The sub journals of Nature or Science like Nature Communications, Scientific Reports were not taken into account. As the most prominent comprehensive scientific magazines, both Nature and Science receive significant recognition among colleagues and may open the way to greater media visibility. Science and Nature publish only 10 percent of received papers, thus creating what has been called a situation of “artificial scarcity” (a paradox in the age of digital information). The data from both magazines are comparable, according to the OA classification of the WoS.

As more and more academic journals provide the choice of open access, the OA status of one paper is also provided across the WoS platform, which can be one of the following choices: legal Gold, Bronze (free contents at a publisher's website) and Green(e.g. author self-archived as a repository) OA versions. In our research, papers with any of the above open access statuses will be classified into OA dataset. Otherwise, they will be classified into TA dataset. Since WoS does not provide the disciplinary classification for Nature and Science, we collected the disciplinary data from www.nature.com and www.sciencemag.org directly and classified the data into three broad disciplines, which are biomedicine, physics, and others. Table 1 shows the characteristics of the dataset. As the data was distributed unequally, we calculated median values rather than average values.

When summing both OA and TA items together, we found that 56% of the papers belong to biomedicine, 19% physics, and 25% others in Nature; while 43% of the papers are in biomedicine, 24% physics, and 33% others in Science. That is, publications in Nature and Science are skewed to biomedicine.

Furthermore, we used classical LDA to quantitatively cluster all papers (which mean average scientific discoveries) into 100 topics and compared them with the qualitative dataset, as well as 2011–2019 Nature remarkable papers and 2010–2019 Science breakthroughs (which mean important scientific discoveries) qualitatively. For comparative analysis, we selected three topics to manifest in both Nature and Science, CRISPR/Cas9, quantum computing & quantum entanglement, and astrophysics. CRISPR/Cas9 and quantum computing & entanglement are hot topics in biology and physics, respectively, and astronomy belongs to a frontier topic of physics. CRISPR/Cas9 was selected as one of Science breakthroughs in 2013 (Genetic Microsurgery for the Masses, 2013), 2015 (Travis, 2015), and 2017 (Stokstad et al., 2017) and among Nature remarkable papers in 2019 (Robots, hominins, and superconductors: 10 remarkable papers from 2019, 2019). Quantum computing is a hot topic of 2019 (Pennisi et al., 2019) Science breakthroughs and 2016 (2016 Editors’ choice, 2016) Nature remarkable paper, and quantum entanglement is a 2015 (Runners-up, 2015) Science breakthrough and a 2015 (2015 Editors’ choice, 2015) Nature remarkable paper. There are 5 Nature remarkable papers and 9 Science breakthroughs in astronomy, mainly involving celestial mechanics, astrometry, and astrophysics.

Using designed indicators – hot-degree and R-index, we concluded the representative results as shown in Table 2, including biomedical, physical, and other records as well as p-value of OA-TA related t-test, where p < 0.05 indicates significant effectiveness.

Except for Science Other category, we can see a significant difference between the hot-degree of OA and TA at the level of p < 0.01 (high significant effectiveness) in the related t-test. The hot-degree of OA and TA in different disciplines show differences. For example, the hot-degree of OA in biomedical is greater than that of TA and the hot-degree of OA in physical is less than TA. This conclusion can be drawn from both Nature and Science datasets. For other disciplines, the difference of hot-degree between OA and TA is significant in the Nature dataset and not significant in the Science dataset, but the hot-degree of OA is less than TA in both Nature and Science datasets. The combined dataset shows that there is a significant difference between the hot-degree of OA and TA, and the hot-degree of OA is greater than TA.

Obviously, there exist disciplinary differences. Annual disciplinary distributions of biomedicine and physics suggest distribution curves of total and median citations in Nature and Science as Figure 2.

Figure 2

As OA > TA means open advantages and TA > OA indicates toll advantages, we cannot say that OA is always advantageous over TA in any field and under any condition. According to Figure 2, median citations show OA advantages in both biomedicine and physics. In the view of total citations, the discipline difference between biology and physics is slightly obvious. The total citation of biomedical OA papers is greater than that of TA papers, but in physics total citation of TA papers is at a disadvantage compared to that of OA ones.

After clustering the publications in both magazines using LDA, where hot topics are clustered by LDA algorithm in total papers of Nature and Science and 100 different topics are extracted in 10 years, we obtained the distributions of hot-degree, which shows biomedical and physical topics with OA or TA respectively in Figure 3.

Figure 3

Figure 3 shows that different topics have different distributions with different hot-degree. The distribution of OA hot-degree in biomedicine is significantly greater than that of TA, while physics shows the opposite. The distribution of hot-degree of OA is significantly less than TA.

Meanwhile, we conclude R-index distribution on LDA topics as Figure 4. The pie chart in each distribution plot depicts the topic proportions of r > 1, r = 1 or r < 1 in different datasets.

Figure 4

In Figure 4, obviously, OA advantages do exist in biomedicine. However, TA advantages occupy most portion in physics.

Therefore, we see that the way a paper is published in Science and Nature, OA or TA, does not necessarily affect the dissemination of average scientific discoveries in all fields. The above analysis also indicates that OA may benefit and promote hot topics in biomedicine. Therefore, we selected typical qualitative evidence for further discussion.

Quantum computing & quantum entanglement and CRISPR/CAS9 are two hot sub-fields of physics and biology, respectively, which can be used for comparison. The former has a total of 3 papers in the qualitative dataset, so we selected the three most important papers of CRISPR/Cas9. Jennifer Doudna and Emmanuelle Charpentier firstly demonstrated that the CRISPR/Cas9 system could be used to edit genes accurately in 2012 (Jinek et al., 2012). Zhang Feng's team proved the ease of programming and wide applicability of RNA-guided nuclease technology based on human and mouse cells (Cong et al., 2013). George Church engineered the type II bacterial CRISPR system to function with custom guide RNA (gRNA) in human cells. These are pioneering papers in CRISPR/CAS9 (Mali et al., 2013). Since there are 17 papers (7 OA and 10 TA papers, access at https://www.nature.com and https://www.sciencemag.org) of astronomy covering several branches, they are used separately to further compare OA and TA.

From the representative qualified papers, we found that CRISPR/Cas9 papers were more likely to go down the OA route, quantum computing papers the TA route, and astronomy likes both OA and TA. These papers were all published in 2010–2019 Nature or Science, of which 3 CRISPR/CAS9 papers were published in Science, 3 Quantum papers were published in Nature, and Astronomy had 9 Nature papers and 8 Science papers, respectively, with representative results shown in Figure 5.

Figure 5

In Figure 5, the upper (a) part shows trends of citations to CRISPR/Cas9 and quantum computing & quantum entanglement papers, and the lower (b) part shows the median citation trends of the astronomy papers in OA and TA. In Figure 5 (a), the three curves with higher citations are CRISPR/CAS9, and the three curves with lower citations are quantum computing & quantum entanglement. CRISPR/Cas9 papers prefer the OA route and quantum topic the TA route, which is consistent with the above quantitative analysis on the disciplinary differences of biomedicine and physics. Zooming on the topic of quantum computing & quantum entanglement, we further found that there were both OA and TA papers. Although one OA paper (the green curve) was published in 2019, it obtained a higher number of citations compared with the other two TA papers. In Figure 4 (b), the median citation curves of OA always keep above the TA-median curve, which might indicate an OA advantage in astronomy. However, whether the publication is Nature or Science, the median citation of Physics TA is higher than that of OA in Table 1. This result indicates that OA can promote the spread of important scientific discoveries more than TA.

These findings reveal that disseminating important scientific discoveries may follow either the open or close route, but scientific breakthroughs are never decided accordingly. OA or TA may affect the citations to papers, leading to either OA or TA advantages, where OA papers may receive high citations while TA may not, which merely suggests that OA promotes the spread of scientific discoveries. However, real scientific discoveries have become known not because of the channel through which the papers about these discoveries are published.

More and more academic journals became hybrid journals, where the author(s) could choose OA or TA publishing route. Therefore, the OA and TA records in database WoS became a research example in this article. As we ignored the effect of other open information sources such as arXiv and bioArxiv, we may partly lose the numbers of citations, which could cause limitations. Another flaw concerns that Nature truly employs a number of strong measures for access-promoting subscription-based articles. For example, since December 2, 2014, all research papers from Nature have been made free to read in a proprietary screen-view format that can be annotated but not copied, printed, or downloaded (Nature, 2014), so that the boundary between OA and TA articles may become fuzzy.