Document Type Profiles in Nature, Science, and PNAS: Journal and Country Level

In bibliometric assessments, Article and Review are document types of scientific communication that are usually taken into account, while other types, like Editorial Material, are often excluded. Earlier studies have dealt with the issue of which document types should be included in the assessments. Some of these studies treat a certain document type and argue that the type should be included in the assessments, e.g. Book Review (Zuccala & van Leeuwen, 2011), Editorial Material (van Leeuwen et al., 2013), and Proceedings Paper (Michels & Fu, 2014). Research that concerns document type classification problems in Web of Science (WoS) has been done, where the corresponding studies treat document types such as Research Letter (van Leeuwen et al., 2007) and Proceedings Paper (Michels & Fu, 2014), and the validity of the used classification (Harzing, 2010; 2013; van Leeuwen et al., 2007). Lewison (2009) investigated the usage of document type as an indicator in research output assessment, whereas Zhang, Rousseau, & Glänzel (2011) studied country profiles in relation to document types. Citation impact of document types has been studied (Braun, Glänzel, & Schubert, 1989b; Campanario et al., 2011; Frandsen, 2008; Garfield, 1987; van Leeuwen et al., 1999; Rousseau, 2009; Sigogneau, 2000), as well as document types used in social media (Haustein, Costas, & Larivière, 2015).

Research on document types has mainly focused on all journals covered by WoS. In this study, we restrict our attention to the three journals Nature, Science, and Proceedings of the National Academy of Sciences of the United States (PNAS). In view of the prestige these three journals have within the scientific community, and the important role they play in various research assessments, we believe it is of importance to study them separately. Earlier research on Nature, Science, and PNAS has compared their internationality (e.g. Kaneiwa et al., 1988), their publication patterns and citation impact (e.g. Braun, Glänzel, & Schubert, 1989a; Wang et al., 2008), and their dynamic usage history (e.g. Wang et al., 2014). However, in this study, we examine the document types of the three journals using two levels of analysis: journal and country.

A total of 35,816 publications, appearing in the three journals, were retrieved from WoS on November 15, 2015. News Item is one of the main document types of Nature and Science, but it was not formally indexed in WoS until 1996. In view of this, we do not use publications published before 1996 in order to avoid comparisons, between years, which might be affected by the absence/presence of News Item publications in the journals. We use a 5-year time gap, year 1999 as the start year and year 2014 as the end year.

The number of publications of each journal for the four years is shown in Table 1. At the journal level of the study, we used all publications appearing in the three journals. There exist, however, publications without addresses in our dataset. Therefore, at the country level of the study, we only used all publications with addresses appearing in the three journals. At this level, we made use of address fractionalization. Clearly, due to co-authorship, a publication might involve more than one country among its addresses. If, for instance, a publication has two USA addresses and one France address, USA contributes 2/3 and France 1/3 to the publication.

Table 1

Number of publications of the journals.

Table 2 reports the 10 most productive countries, regarding fractional publication output, in the three journals in the publication year 2014. The countries are ordered descending according to the total number of publication fractions in the three journals. In addition to the top 10 countries, we included the other (relative to People’s Republic of China) four BRICS countries as cases for comparison, i.e Brazil, Russia, India, and South Africa.

Table 2

Number of publication fractions of the 10 most productive countries and four BRICS countries in year 2014.

In the bibliographic records of WoS, the document type of each publication is recorded. Based on this information, we calculated, for a given journal, a given document type and a given year the number of publications for the combination. In order to map the document type profile for each of the three journals across time, the raw numbers were used to obtain the relative contribution of publications to each document type, and for each considered year, with respect to all publications published in the journal.

For journal i, document type j, and publication year y, we define P_ijy as the number of publications of journal i of document type j in year y, and R_ijy as the percentage of publications of document type j of journal i in year y relative to the total number of publications of journal i in year y. Formally: Rijy=Pijy/Piy×<!-- ? -->100,$$\begin{array}{} R_{ijy}=P_{ijy}/P_{iy}\times100, \end{array} $$(1)

where P_iy is the total number of publications of journal i in year y.

Then, for journal i in year y, the (relative) document type distribution is represented by the vector D_iy, defined as in Equation (2): Diy=(Rily,Ri2y,…<!-- ? -->,Riny),$$\begin{array}{} D_{iy}=(R_{ily},R_{i2y},\ldots,R_{iny}), \end{array} $$(2)

where n is the number of document types of journal i in year y.

For a country c, we define P_ijyc as the number of c publication fractions in journal i of document type j in year y, and R_ijyc as the percentage of c publication fractions in journal i of document type j in year y relative to P^′_ijy, the number of publications, with at least one address, in journal i of document type j in year y. Formally: Rijyc=Pijyc/P′ijy×<!-- ? -->100.$$\begin{array}{} R_{ijyc}=P_{ijyc}/P'\,_{ijy}\times100. \end{array} $$(3)

Since fractionalization is used, the sum of the percentages across all countries in journal i, with regard to document type j and in year y, is equal to 100. For country c in journal i in year y, the (relative) document type distribution is represented by the vector D_iyc, defined as Equation (4): Diyc=(Rilyc,Ri2yc,…<!-- ? -->,Rinyc),$$\begin{array}{} D_{iyc}=(R_{ilyc},R_{i2yc},\ldots,R_{inyc}), \end{array} $$(4)

where n is the number of document types of journal i in year y.

By application of the cosine measure, we obtained similarities, for each journal, for pairs of years within a country. The similarity between years y and y+5 for country c in journal i is defined as Equation (5): Siy(y+5)c=∑<!-- ? -->j=1nRijyc×<!-- ? -->Rij(y+5)c∑<!-- ? -->j=1n(Rijyc)2×<!-- ? -->∑<!-- ? -->j=1n(Rij(y+5)c)2.$$\begin{array}{} \displaystyle S_{iy(y+5)c}=\frac{\displaystyle \sum\nolimits_{j=1}^{n}R_{ijyc}\times R_{ij(y+5)c}}{\displaystyle \sqrt{\sum\nolimits_{j=1}^{n}\Big(R_{ijyc}\Big)^2}\times\sqrt{\sum\nolimits_{j=1}^{n}\Big(R_{ij(y+5)c}\Big)^2}}. \end{array} $$(5)

In this study, we have treated the document type profiles in the three journals Nature, Science and PNAS. We analyzed the profiles at two levels, journal and country. The results for the journal level show that Nature and Science mainly publish Editorial Material, Article, News Item and Letter. The publications of PNAS, though, are heavily concentrated on the type Article, regardless of year. This outcome mirrors the difference in aims and scope of the journals. Nature and Science aim at publishing remarkable scientific research and important scientific news related to such research. PNAS is a very academic journal, which almost exclusively publishes high-level frontier research. Interestingly, the percentages for the type Article for Nature and Science are decreasing slightly from 1999 to 2014, while the corresponding percentages for Editorial Material are increasing. Van Leeuwen (2013) explored the possibilities to include editorial materials in research assessment procedures. With regard to citation impact, the difference between articles and editorial materials turned out to be relatively small. Partly based on this outcome, the authors concluded that some editorial materials can be included in research assessment procedures.

At the country level, the results show, with regard to year 2014, that most studied countries focus on Article and Letter in Nature, but on Letter in Science and PNAS. The BRICS countries, except People’s Republic of China, have rather unbalanced document type profiles, and this is the case also for Japan. The publication output of Brazil, Russia, India, and South Africa in the three journals is small, a fact that might explain the unbalanced profiles of these countries. For the comparison of profiles between different years, there are BRICS countries for which the profiles change to a quite large extent.

Large differences are found when comparing the country document type profiles of the three journals with the corresponding profiles in all WoS journals (Zhang, Rousseau, & Glänzel, 2011). For example, with respect to all WoS journals, it was shown that People’s Republic of China published almost exclusively documents of the type Article, whereas the publishing emphasis of China in Nature, Science, and PNAS is on Letter. A similar pattern can be observed for several other countries of this study. This suggests that the country document type profiles of Nature, Science, and PNAS are different from the corresponding profiles of other WoS journals.

For future research, we would like to compare the citation impact of different document types in Nature, Science, and PNAS with their impact across all WoS journals.