Does Success Breed Success? A Study on the Correlation between Impact Factor and Quantity in Chinese Academic Journals

Huang (2016) proposed a new method to show the relationship between the IF and the number of journal papers. We also use this method because it is intuitive and convenient for comparing this study with others. To thoroughly demonstrate the correlation between the number of articles and the IF, the following Lorenz curve formula (1) is employed to compute the accumulated number of articles, as in Huang (2016). In detail, (1) yi=∑j=1iNj∑j=1MNjand xi=iM,i=1,2,⋯,M,xi∈(0,1] {y_i} = {{\sum\nolimits_{j = 1}^i {{N_j}} } \over {\sum\nolimits_{j = 1}^M {{N_j}} }}and\;{x_i} = {i \over M},i = 1,2, \cdots ,M,{x_i} \in (0,1] where M denotes the number of journals included in a discipline; N denotes the number of journal papers; and i = 1, 2, 3,..., M. The curve y(x) is continuous and monotonically increasing, and the point (x, y) starts from (0, 0) and ends at (1, 1). If the number of journal papers is independent of the IF, the result would be a straight line y=x; if the number of published articles is positively or negatively related to the IF, the straight line would bend upwards or downwards and become convex or concave, respectively. Thus, a positive or negative correlation is demonstrated by a convex or concave region. Huang (2016) proposed a parametrization formula: (2) y=x+αxβ(1−x)γ y = x + \alpha {x^\beta }{\left( {1 - x} \right)^\gamma } where α, β, and γ are the associated parameters. Among these parameters since x^β (1 − x)^γ ≥ 0, x ∈ (0, 1], a positive or negative α represents a positive or negative relationship between the IF and the number of journal papers, respectively. This will affect the bending direction of the straight line y=x.

The data used in this paper were all collected from the 2007–2017 Chinese Science and Technology Journal Citation Reports (extended edition) released by the Chinese Science and Technology Information Institute. More than 7,000 Chinese science and technology journals, which can be divided into more than 150 disciplines or research fields, are included in this database. In addition, we exclude journals with incomplete data from 2007 to 2017 in this database because the lack of data for a certain year or several years in these journals makes calculations impossible. As most disciplines contain fewer than 100 journals, we reclassify the journals according to the classification methods of the Essential Science Indicators (ESIs) disciplines and SCI/SSCI disciplines, and their corresponding Chinese journal disciplines are listed in Appendix. The ESI disciplines are Agricultural Sciences; Economics & Business; Plant & Animal Science; Geosciences; Materials Science; Engineering; Social Sciences, general; and Clinical Medicine. The SCI/SSCI disciplines are Economics and Education & Educational Research. Notably, Library and Information Science is included in Social Sciences, general.

Chen (2018) mentioned that some journals publish many articles to obtain publication fees, which may significantly influence the relationship between the IF and the number of journal papers. However, it was found that removing journals that require publication fees is hard to realize, for it is almost impossible to collect data on publication fees. First, it is very difficult to obtain data on publication fees of so many journals for many years. Second, those journals that garner expensive publication fees are not willing to mention it. However, most journals that require publication fees for making money have an obvious feature: they publish a larger number of papers that are significantly more than the average level of journals in the same research field. Only by publishing a large number of papers can they obtain huge profits. Nevertheless, we must admit that it may also exclude some journals that have no purpose of making money, and journals that publish a larger number of papers may reveal some interesting findings.

In this paper, we only exclude journals that publish an excessively larger number of papers. This method excludes journals that are more than two standard deviations from the average number of articles in the discipline to which they belong until outliers cannot be detected. Additionally, we compare and analyze the data before and after outliers are excluded. The corresponding calculation formula is a s follows: y={keep,x≤μ+2σremove,x>μ+2σ y = \left\{ {\matrix{ {keep,x \le \mu + 2\sigma } \hfill \cr {remove,x > \mu + 2\sigma } \hfill \cr } } \right. where y symbolizes whether a journal is retained, x is the number of publications of a journal, μ is the mean, and σ is the standard deviation.

The number of journals in each discipline before and after removing outliers is presented in table 1.

Before removing outliers, the AIF of all disciplines maintained a growth trend from 2007 to 2017, and the growth rate of the AIF of social sciences disciplines was generally larger than that of natural sciences disciplines. Among the disciplines, Education & Educational Research had the largest increase, which was 190.10%, while Materials Science had the smallest increase, which was 28.94%. Moreover, the AIF of social sciences disciplines was generally larger than that of natural sciences disciplines, which is vastly different from international journals. Among the disciplines, Economics & Business had the highest AIF value of 1.601, and Materials Science had the lowest AIF value of 0.499 in 2017. The related data are presented in table 2.

After removing outliers, the AIF in all disciplines also maintained a growth trend from 2007 to 2017, and the growth rate of the AIF of social sciences disciplines was also generally larger than that of natural sciences disciplines. Among the disciplines, Education & Educational Research had the largest increase, which was 193.46%, while Geosciences had the smallest increase, which was 28.21%. Additionally, the AIF of social sciences disciplines was also generally larger than that of natural sciences disciplines. Among the disciplines, Economics & Business had the highest AIF value of 1.759, and Materials Science had the lowest AIF value of 0.483 in 2017. The related data are presented in table 3.

In the past 20 years, the number of scientific researchers in China has been increasing (according to the Statistical Yearbook of China), the level of scientific research has been continuously improving, the literature reviews have been more detailed, the analysis has been being more comprehensive, and the number of references has also been increasing (according to the Chinese Science & Technology Journal Citation Report). There is no doubt that the AIFs of all disciplines are increasing.

However, the AIF of Materials Science and Engineering is lower in most years from 2007 to 2017 after removing outliers, which means that the removal of outliers from journals with a large number of journal papers may not be applicable to all disciplines. Thus, the robustness of the conclusions can be better observed when combined with the analysis before and after removing the outliers. The related data are presented in table 4.

Before removing outliers, the growth trend of the average number of journal papers is not the same as that of the AIF. From 2007 to 2017, the average number of journal papers in social sciences disciplines decreased, and that of natural sciences disciplines increased. Among the disciplines, Agricultural Sciences had the largest increase, which was 67.60%, and Agricultural Sciences had the largest average number of journal papers, which was on average 426.399, while Geosciences had the smallest average number of journal papers, which was on average 151.529. The related data are presented in table 5.

By contrast, after removing outliers, from 2007 to 2017, the average number of journal papers on social sciences disciplines also decreased, and that of natural sciences disciplines also increased. Among the disciplines, Agricultural Sciences had the largest increase, which was 27.85%, while Economics had the largest decrease, which was −21.84%. In addition, in 2017, Clinical Medicine had the largest average number of journal papers, which was on average 258.094, while Geosciences had the smallest average number of journal papers, which was on average 110.987. The related data are presented in table 6.

Based on the aforementioned analysis, we drew the following conclusions. The AIFs of the journals in all disciplines maintained a growth trend from 2007 to 2017. Whether before or after removing outliers, from 2007 to 2017, the AIF and its growth rate for social sciences disciplines were generally larger than those of natural sciences disciplines, the average number of journal papers on social sciences disciplines decreased, and the average number of journal papers on natural sciences disciplines increased. After removing outliers, the yearly average number of papers per journal decreased.

Due to the large number of disciplines, we use Geosciences as an example. Figures 1 and 2 show the situation before and after removing outliers, respectively. We plot data from 11 years (2007–2017) in one graph and use different gray colors, which gradually darken as the year progresses, for each year's data. Geosciences include 174 and 150 journals before and after removing outliers, respectively. The vertical axis y is the cumulative number of journal papers, and the horizontal axis x is the ranking of the IF after normalization. The α values of Geosciences are presented in tables 7 and 8.

Figure 1

Figure 2

In figure 1, the dark data are on the top, and the light data are on the bottom, indicating that the relationship between the number of journal papers and the IF becomes increasingly more positively correlated. In addition, the data are distributed on both sides of y=x, which shows that the relationship between the IF and the number of journal papers changed from negative to positive.

In figure 2, the dark data are on the top, and the light data are on the bottom, indicating that the relationship between the number of journal papers and the IF also becomes increasingly more positively correlated. The data are all distributed above y=x, which means that the relationship in Geosciences has always remained positively correlated after removing outliers, indicating that outliers may have a substantial influence on the relationship. The outliers garner excessive publication fees and rely on publishing many papers, which greatly affects their quality, lowers their IF, and may cause a negative correlation in certain years.

Based on the above analysis, we continue to discuss the situation both before and after removing outliers.

We calculate the value of α in each discipline, according to formula (2), before and after removing outliers; the results are presented in tables 7 and 8, respectively.

Before removing outliers, the α of 7 disciplines was negative in 2007, and 2 became positive in 2017. Additionally, the α of 5 disciplines increased from 2007 to 2017. Among the disciplines, Clinical Medicine, Geosciences, and Plant & Animal Science changed from negative to positive, while Education & Educational Research changed from positive to negative. After removing outliers, the α of 3 disciplines was negative in 2007, and the α of 3 disciplines remained negative in 2017. Additionally, the α of 8 disciplines increased from 2007 to 2017. Among the disciplines, “Social Sciences, general” changed from negative to positive, and Education & Educational Research changed from positive to negative.

Before removing outliers, the number of disciplines in which α increased and the number of disciplines in which α decreased is equal. However, after removing outliers, the number of disciplines in which α increased is larger than the number of disciplines in which α decreased. Moreover, before or after removing outliers, Materials Science and Engineering maintained positive correlations; Economics & Business and Economics maintained negative correlations, while Education & Educational Research maintained a change from positive to negative.

In 2017, the α of Economics & Business, Economics, and Education & Educational Research remained negative either before or after removing outliers, indicating that the negative correlation between the IF and the number of journal papers in these disciplines is robust. Additionally, the α of Clinical Medicine, Materials Science, Engineering, Geosciences, and Plant & Animal Science remained positive either before or after removing outliers, indicating that the positive correlation between the IF and the number of journal papers in these disciplines is robust. However, the α of Agricultural Sciences and Social Sciences generally changed from negative to positive after removing the outliers, indicating that there are journals that have many articles published but low IF in these two disciplines. These journals may publish many papers to garner excessive publication fees while ignoring the quality of these papers.

The aforementioned analysis demonstrates that the relationship between the IF and the number of journal papers in natural sciences disciplines tends to be positively correlated and that in the social sciences disciplines tends to be negatively correlated. Moreover, the α of natural sciences disciplines tends to be larger, and the α of social sciences disciplines tends to be smaller.

One possible explanation for the SBS principle is that journals with high IFs receive more submissions than journals with low IFs, and there are correspondingly more publishable papers that fulfill certain criteria. However, different phenomena are discovered in Chinese journals. The relationship between the IF and the number of journal papers in Chinese journals on natural sciences disciplines is positive and that for Chinese journals on social sciences disciplines is negative. This may be caused by the influence of the international publishing system according to our interviews with some journal editors and surveys of the natural sciences evaluation system in many universities, which is also partly reflected in Chen (2018).

Because Chinese universities mainly implement SCI-based evaluation standards in the natural sciences until recently, Chinese natural science journals are not highly recognized and have lost their academic power. Except for a few journals, for example, Science of China and Science Bulletin, Chinese natural sciences journals have received little attention from researchers. Moreover, researchers who gain funding from the Natural Science Foundation of China are ordered to publish papers in Chinese journals to ensure that these journals remain in business. Thus, few Chinese submissions are increasingly concentrated in a few well-recognized journals. This may also explain why the AIF of Materials Science and Engineering is lower after removing the outliers.

By contrast, CSSCI journals remain highly recognized by Chinese universities. Because of the higher language requirements of SSCI journals and the regional characteristics of the research objects in the social sciences, only a few Chinese researchers publish articles in SSCI journals. For example, the impact of publishing an article in Social Sciences in China is similar to that of publishing in American Economic Review in some universities. In addition, the CSSCI list is adjusted every two years, causing Chinese social sciences journals to be in a highly competitive situation and many journals with a higher IF to consciously control the number of articles published in order to improve or maintain the quality. When the Journal of Wuhan University and the Journal of Tongji University were removed from the CSSCI list in 2017, they issued a statement expressing anger but had to allow authors to withdraw their accepted but unpublished manuscripts

Available online: https://www.sohu.com/a/124768409_488760 (accessed on 14 June 2020).

In addition, Chinese natural sciences journals are losing their academic power, which can also be partly corroborated by the fact that the AIF and increase in the AIF of Chinese natural sciences disciplines are smaller than those of Chinese social sciences disciplines (see the details in table 2 and table 4) from 2007 to 2017, which is rather opposite in the international publishing system. According to the China Education Statistics Yearbook, the share of full-time social sciences teachers of and doctoral students studying social sciences was only 75.79% of the number of full-time natural sciences teachers of and doctoral students studying natural sciences in 2017, the number of full-time natural sciences teachers of and doctoral students studying the natural sciences in China increased by 40.0% from 2007 to 2017, and the number of full-time teachers of and doctoral students studying social sciences increased by 14.0% over the same period.

In summary, the difference in the relationship between the IF and the number of journal papers for Chinese natural sciences disciplines and Chinese social sciences disciplines may be due to the influence of the international publishing system. However, the α of “Social Sciences, general” fluctuated from 2007 to 2017 and was larger than 0 after removing the outliers, indicating that there may be differences between various disciplines that belong to social sciences, such as the α of Economics and Education & Educational Research being increasingly negative. This phenomenon indicates that journals in Economics and Education & Educational Research are becoming increasingly competitive, and the other social science disciplines may experience weaker competition. For example, according to the China Education Statistics Yearbook, the number of full-time teachers of literature and philosophy decreased by 15.8% from 2007 to 2017, and the proportion of those teachers in all the social sciences decreased from 47.4% to 35.9%.