Open Access

Are University Rankings Statistically Significant? A Comparison among Chinese Universities and with the USA

Loet Leydesdorff
Loet Leydesdorff
, 
Caroline S. Wagner
Caroline S. Wagner
 and 
Lin Zhang
Lin Zhang
   | Mar 02, 2021
Journal of Data and Information Science's Cover Image
About this article
Previous Article
Next Article
Cite
Article Category: Research Paper
Published Online: Mar 02, 2021
Page range: 67 - 95
Received: Nov 01, 2020
Accepted: Jan 25, 2021
DOI: https://doi.org/10.2478/jdis-2021-0014
Keywords
© 2021 Loet Leydesdorff et al., published by Sciendo
This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Figure 1

Potentially overlapping confidence intervals of the PP-Top10% for three leading Chinese universities.
Potentially overlapping confidence intervals of the PP-Top10% for three leading Chinese universities.

Figure 2

Grouping of significantly different and non-different values for PP-top 10% among three Chinese universities; z-values (provided in the figure) are used for sizing the nodes.
Grouping of significantly different and non-different values for PP-top 10% among three Chinese universities; z-values (provided in the figure) are used for sizing the nodes.

Figure 3

Grouping of 205 Chinese universities in terms of z-values (p <.01); VOSviewer used only for the visualization; modularity Q = 0.165. The figure can be web-started from here (or with a black background from here).
Grouping of 205 Chinese universities in terms of z-values (p <.01); VOSviewer used only for the visualization; modularity Q = 0.165. The figure can be web-started from here (or with a black background from here).

Figure 4

Development of the values of P, P-top10%, and PP-Top10% for Fudan University during 2016–2020.
Development of the values of P, P-top10%, and PP-Top10% for Fudan University during 2016–2020.

Figure 5

Grouping of 205 Chinese universities based on overlapping confidence intervals; VOSviewer used for the decomposition and clustering; font- and node-sizes based on z-values. (The map can be web-started from here.)
Grouping of 205 Chinese universities based on overlapping confidence intervals; VOSviewer used for the decomposition and clustering; font- and node-sizes based on z-values. (The map can be web-started from here.)

Figure 6

Grouping of 205 Chinese and 197 US universities in terms of z-values; differences among groups are significant at the 1% level; VOSviewer used for the decomposition and clustering. (The map can be web-started from here.)
Grouping of 205 Chinese and 197 US universities in terms of z-values; differences among groups are significant at the 1% level; VOSviewer used for the decomposition and clustering. (The map can be web-started from here.)

Figure 7

Distribution of z values in decreasing order for 205 Chinese and 197 American universities.
Distribution of z values in decreasing order for 205 Chinese and 197 American universities.

Figure 8

The participation of Fudan University in the top 10% class of papers using the Leiden Rankings for subsequent years as a time series versus the reconstruction using the 2020-model.
The participation of Fudan University in the top 10% class of papers using the Leiden Rankings for subsequent years as a time series versus the reconstruction using the 2020-model.

75 top universities with overlapping confidence values.

University Z rank within group
Tsinghua University 11.005 1 1
Hunan University 10.193 2 2
Hong Kong University of Science and Technology 6.566 3 3
University of Science and Technology of China 6.482 4 4
City University of Hong Kong 6.454 5 5
Shandong University of Science and Technology 6.444 6 6
Hong Kong Polytechnic University 6.406 7 7
South China University of Technology 6.049 8 8
Chinese University of Hong Kong 5.993 9 9
Nankai University 4.860 10 10
University of Hong Kong 4.418 11 11
Shenzhen University 4.089 12 12
Zhejiang University 3.800 13 13
Qufu Normal University 3.724 14 14
Fuzhou University 3.487 15 15
Beihang University 3.432 16 16
University of the Chinese Academy of Sciences 3.354 17 17
Harbin Institute of Technology 3.226 18 18
Central China Normal University 3.090 19 19
University of Macau 3.046 20 20
Northwestern Polytechnical University 2.915 21 21
Wuhan University of Technology 2.914 22 22
Huazhong University of Science and Technology 2.907 23 23
Peking University 2.689 24 24
Southern University of Science and Technology 2.332 25 25
Beijing University of Chemical Technology 2.273 26 26
GuangZhou University 2.120 27 27
Hong Kong Baptist University 2.094 28 28
Nanjing University 1.973 29 29
Xiamen University 1.954 30 30
Wuhan University 1.765 31 31
Tianjin University 1.738 32 32
Dalian University of Technology 1.553 33 33
Shandong Normal University 1.478 34 34
East China Normal University 1.333 35 35
Jiangsu Normal University 1.313 36 36
Soochow University 1.310 37 37
Zhejiang Normal University 1.238 38 38
University of Jinan 1.195 39 39
University of Electronic Science and Technology of China 1.154 40 40
Nanjing University of Science and Technology 1.140 41 41
Nanjing University of Information Science and Technology 1.093 42 42
Southwest Jiaotong University 1.088 43 43
China University of Geosciences 1.035 44 44
Huazhong Agricultural University 1.030 45 45
Tongji University 0.954 46 46
University of Science and Technology Beijing 0.932 47 47
China Agricultural University 0.704 48 48
Beijing Institute of Technology 0.587 49 49
Sun Yat-sen University 0.572 50 50
Hangzhou Dianzi University 0.449 51 51
North China Electric Power University 0.414 52 52
China University of Petroleum Beijing 0.389 53 53
Guangdong University of Technology 0.332 54 54
Shanghai University 0.332 55 55
Fourth Military Medical University 0.257 56 56
Beijing Normal University 0.178 57 57
Chongqing University 0.000 58 58
Hefei University of Technology 0.000 59 59
Xi’an Jiaotong University 0.000 60 60
Nanjing University of Posts and Telecommunications −0.110 61 61
Heilongjiang University −0.158 62 62
Nanjing Agricultural University −0.171 63 63
China Pharmaceutical University −0.259 64 64
Central South University −0.280 65 65
China University of Petroleum East China −0.407 66 66
Qingdao Agricultural University −0.422 67 67
Beijing Jiaotong University −0.448 68 68
Chang’an University −0.465 69 69
Hubei University −0.498 70 70
Dalian Medical University −0.643 72 71
Chongqing University of Posts and Telecommunications −0.656 73 72
Huaqiao University −0.795 74 73
Anhui Agricultural University −0.862 75 74
Army Medical University −0.952 80 75

z-values for nodes and links among three leading Chinese universities.

Peking University Tsinghua University Zhejiang University
Peking University 2.689 **
Tsinghua Univ. 8.460 *** 11.005 ***
Zhejiang University 0.638 8.533 *** 3.800 ***

Standardized residuals of the chi-square values in Table 4.

Top 10% Non-top
Tsinghua 5.84 −2.19
Zhejiang −5.37 2.01

Bottom group of 102 Chinese universities.

University z rank within-group
Hubei University −0.498 70 1
Northwest University −1.609 100 2
South China Agricultural University −1.631 101 3
Zhejiang University of Technology −1.676 102 4
China University of Mining and Technology −1.678 103 5
Nanjing University of Aeronautics and Astronautics −1.816 106 6
Second Military Medical University −1.844 108 7
Zhejiang Sci-Tech University −1.886 110 8
Taiyuan University of Technology −1.934 111 9
Shanghai University of Traditional Chinese Medicine −1.950 112 10
Qingdao University of Science and Technology −1.961 113 11
Nanjing University of Chinese Medicine −1.981 114 12
Harbin Engineering University −2.033 115 13
Lanzhou University −2.110 117 14
Northeast Normal University −2.128 118 15
China Jiliang University −2.130 119 16
Jiangnan University −2.163 120 17
Jinan University −2.227 121 18
Hangzhou Normal University −2.245 122 19
Southern Medical University −2.263 123 20
Hunan Normal University −2.282 124 21
Henan University −2.328 125 22
Beijing University of Chinese Medicine −2.346 126 23
Tianjin Medical University −2.348 127 24
Shantou University −2.383 128 25
Xi’an University of Architecture and Technology −2.384 129 26
Hainan University −2.417 130 27
Hohai University −2.453 131 28
Beijing University of Posts and Telecommunications −2.458 132 29
Qingdao University −2.465 133 30
Jiangsu University −2.467 134 31
Jiangxi Normal University −2.486 135 32
Shaanxi Normal University −2.505 136 33
Shandong University of Technology −2.506 137 34
Nanchang University −2.528 138 35
Henan Agricultural University −2.584 139 36
Guizhou University −2.675 140 37
Guangxi Normal University −2.709 141 38
Northeast Forestry University −2.751 142 39
Northeastern University −2.761 143 40
Henan Normal University −2.775 144 41
Shandong Agricultural University −2.783 145 42
Anhui Normal University −2.834 146 43
Lanzhou University of Technology −2.835 147 44
Ningbo University −2.838 148 45
Southwest University −2.855 149 46
Yunnan University −2.940 150 47
Shanghai Ocean University −2.954 151 48
Northwest Normal University −2.962 152 49
Inner Mongolia University −2.968 153 50
Beijing University of Technology −2.969 154 51
Fudan University −3.065 155 52
Yanshan University −3.100 156 53
Tianjin University of Science and Technology −3.150 157 54
Beijing Forestry University −3.151 158 55
People's Liberation Army University of Science and Technology −3.152 159 56
Capital Normal University −3.188 160 57
Southwest University of Science and Technology −3.433 161 58
Hebei University of Technology −3.558 162 59
Shenyang Pharmaceutical University −3.713 163 60
Xinjiang University −3.741 164 61
Kunming University of Science and Technology −3.832 165 62
Nantong University −3.935 166 63
Xiangtan University −4.012 167 64
Northeast Agricultural University −4.025 168 65
Wenzhou Medical University −4.062 169 66
Chongqing Medical University −4.106 170 67
Guangxi University −4.119 171 68
Xi’An University of Technology −4.203 172 69
Zhengzhou University −4.244 173 70
Ocean University of China −4.246 174 71
Henan Polytechnic University −4.348 175 72
Shihezi University −4.400 176 73
Kunming Medical University −4.506 177 74
Xuzhou Medical College −4.604 178 75
Tianjin Polytechnic University −4.775 179 76
South China Normal University −4.827 180 77
Nanjing Medical University −4.941 181 78
Chengdu University of Technology −4.958 182 79
Xinxiang Medical University −4.962 183 80
Harbin University of Science and Technology −5.001 184 81
North University of China −5.068 185 82
Yangtze University −5.093 186 83
Guangzhou Medical University −5.200 187 84
Sichuan Agricultural University −5.212 188 85
Sichuan University −5.247 189 86
National University of Defense Technology −5.257 190 87
Harbin Medical University −5.331 191 88
Tianjin Normal University −5.358 192 89
Shanxi Medical University −5.378 193 90
Henan University of Science and Technology −5.416 194 91
Shandong University −5.619 195 92
Anhui Medical University −5.951 196 93
Hebei University −6.307 197 94
Guangxi Medical University −7.267 198 95
Peking Union Medical College −7.621 199 96
Xinjiang Medical University −7.703 200 97
China Medical University −8.064 201 98
Fujian Medical University −8.169 202 99
Hebei Medical University −8.966 203 100
Jilin University −9.607 204 101
Capital Medical University −12.944 205 102

Number of research universities included in LR 2020 for respective countries.

Country (top 10 of 65) N of universities
China 205
United States 198
United Kingdom 58
Germany 54
Japan 53
South Korea 44
Italy 41
Spain 41
India 36
Iran 36 %
sum 766 65.1%
<55 other countries> = (1,176 – 766) = 410 34.9%
65 countries 1,176 100%

Top group of 32 Chinese universities.

University Z overall rank within-group rank
Tsinghua University 11.005 1 1
Hunan University 10.193 2 2
Hong Kong University of Science and Technology 6.566 3 3
University of Science and Technology of China 6.482 4 4
City University of Hong Kong 6.454 5 5
Shandong University of Science and Technology 6.444 6 6
Hong Kong Polytechnic University 6.406 7 7
South China University of Technology 6.049 8 8
Chinese University of Hong Kong 5.993 9 9
Nankai University 4.860 10 10
University of Hong Kong 4.418 11 11
Shenzhen University 4.089 12 12
Qufu Normal University 3.724 14 13
Fuzhou University 3.487 15 14
Beihang University 3.432 16 15
University of the Chinese Academy of Sciences 3.354 17 16
Central China Normal University 3.090 19 17
University of Macau 3.046 20 18
Northwestern Polytechnical University 2.915 21 19
Wuhan University of Technology 2.914 22 20
Southern University of Science and Technology 2.332 25 21
Beijing University of Chemical Technology 2.273 26 22
GuangZhou University 2.120 27 23
Hong Kong Baptist University 2.094 28 24
Shandong Normal University 1.478 34 25
Jiangsu Normal University 1.313 36 26
Zhejiang Normal University 1.238 38 27
University of Jinan 1.195 39 28
Nanjing University of Information Science and Technology 1.093 42 29
Hangzhou Dianzi University 0.449 51 30
Heilongjiang University −0.158 62 31
Qingdao Agricultural University −0.422 67 32

Chi-square for the comparison of Tsinghua and Zhejiang University in LR 2020.

Chi-square top 10% non-top
Tsinghua 34.10 4.79 38.89
Zhejiang 28.87 4.05 32.92
χ2 = 71.80

Middle group of 69 Chinese universities.

University z overall rank within-group
Zhejiang University 3.800 13 1
Harbin Institute of Technology 3.226 18 2
Huazhong University of Science and Technology 2.907 23 3
Peking University 2.689 24 4
Nanjing University 1.973 29 5
Xiamen University 1.954 30 6
Wuhan University 1.765 31 7
Tianjin University 1.738 32 8
Dalian University of Technology 1.553 33 9
East China Normal University 1.333 35 10
Soochow University 1.310 37 11
University of Electronic Science and Technology of China 1.154 40 12
Nanjing University of Science and Technology 1.140 41 13
Southwest Jiaotong University 1.088 43 14
China University of Geosciences 1.035 44 15
Huazhong Agricultural University 1.030 45 16
Tongji University 0.954 46 17
University of Science and Technology Beijing 0.932 47 18
China Agricultural University 0.704 48 19
Beijing Institute of Technology 0.587 49 20
Sun Yat-sen University 0.572 50 21
North China Electric Power University 0.414 52 22
China University of Petroleum Beijing 0.389 53 23
Guangdong University of Technology 0.332 54 24
Shanghai University 0.332 55 25
Fourth Military Medical University 0.257 56 26
Beijing Normal University 0.178 57 27
Chongqing University 0.000 58 28
Hefei University of Technology 0.000 59 29
Xi’an Jiaotong University 0.000 60 30
Nanjing University of Posts and Telecommunications −0.110 61 31
Nanjing Agricultural University −0.171 63 32
China Pharmaceutical University −0.259 64 33
Central South University −0.280 65 34
China University of Petroleum East China −0.407 66 35
Beijing Jiaotong University −0.448 68 36
Xidian University −0.529 71 37
Dalian Medical University −0.643 72 38
Chongqing University of Posts and Telecommunications −0.656 73 39
Huaqiao University −0.795 74 40
Anhui Agricultural University −0.862 75 41
Wuhan University of Science and Technology −0.899 76 42
Nanjing Tech University −0.919 77 43
Donghua University −0.923 78 44
East China University of Science and Technology −0.943 79 45
Army Medical University −0.952 80 46
Anhui University −1.004 81 47
Tianjin University of Technology −1.047 82 48
Renmin University of China −1.064 83 49
Nanjing Normal University −1.189 84 50
Southeast University −1.213 85 51
Fujian Agriculture and Forestry University −1.218 86 52
Northwest Agriculture and Forestry University −1.226 87 53
University of Shanghai for Science and Technology −1.232 88 54
North China University of Science and Technology −1.249 89 55
Wenzhou University −1.306 90 56
Southwest Petroleum University −1.311 91 57
Shaanxi University of Science and Technology −1.314 92 58
China Three Gorges University −1.333 93 59
University of South China −1.345 94 60
Shanghai Normal University −1.369 95 61
Dalian Maritime University −1.380 96 62
Yangzhou University −1.392 97 63
Changzhou University −1.544 98 64
Nanjing Forestry University −1.581 99 65
Guangzhou University of Chinese Medicine −1.685 104 66
Fujian Normal University −1.781 105 67
China Academy of Chinese Medical Sciences −1.840 107 68
Shanghai Jiao Tong University −1.876 109 69

Expected values of top 10% cited papers for Tsinghua and Zhejiang University.

Expected values top 10% non-top total p
Tsinghua 2,449.01 17,452.99 19,902.00
Zhejiang 2,892.99 20,617.01 23,510.00
5,356.09 38,055.91 43,412.00

Observed values of top 10% cited papers for Tsinghua and Zhejiang University during the period 2015–2019 (fractional counting, all sciences).

Observed values top 10% non-top total p
Tsinghua 2,738 17,164 19,902
Zhejiang 2,604 20,906 23,510
5,342 38,070 43,412

Top-20 universities in each of the three clusters. (See for a full list in the Annexes.)

top group (top 20) z middle group (top 20) z bottom group (top 20) z
1 Tsinghua Univ. 11.005 Zhejiang Univ. 3.800 Hubei Univ. −0.498
2 Hunan Univ. 10.193 Harbin Institute of Technology 3.226 Northwest Univ. −1.609
3 Hong Kong Univ. of Science and Technology 6.566 Huazhong Univ. of Science and Technology 2.907 South China Agricultural Univ. −1.631
4 Univ. of Science and Technology of China 6.482 Peking Univ. 2.689 Zhejiang Univ. of Technology −1.676
5 City Univ. of Hong Kong 6.454 Nanjing Univ. 1.973 China Univ. of Mining and Technology −1.678
6 Shandong Univ. of Science and Technology 6.444 Xiamen Univ. 1.954 Nanjing Univ. of Aeronautics and Astronautics −1.816
7 Hong Kong Polytechnic Univ. 6.406 Wuhan Univ. 1.765 Second Military Medical Univ. −1.844
8 South China Univ. of Technology 6.049 Tianjin Univ. 1.738 Zhejiang Sci-Tech Univ. −1.886
9 Chinese Univ. of Hong Kong 5.993 Dalian Univ. of Technology 1.553 Taiyuan Univ. of Technology −1.934
10 Nankai Univ. 4.860 East China Normal Univ. 1.333 Shanghai Univ. of Traditional Chinese Medicine −1.950
11 Univ. of Hong Kong 4.418 Soochow Univ. 1.310 Qingdao Univ. of Science and Technology −1.961
12 Shenzhen Univ. 4.089 Univ. of Electronic Science and Technology of China 1.154 Nanjing Univ. of Chinese Medicine −1.981
13 Qufu Normal Univ. 3.724 Nanjing Univ. of Science and Technology 1.140 Harbin Engineering Univ. −2.033
14 Fuzhou Univ. 3.487 Southwest Jiaotong Univ. 1.088 Lanzhou Univ. −2.110
15 Beihang Univ. 3.432 China Univ. of Geosciences 1.035 Northeast Normal Univ. −2.128
16 Univ. of the Chinese Academy of Sciences 3.354 Huazhong Agricultural Univ. 1.030 China Jiliang Univ. −2.130
17 Central China Normal Univ. 3.090 Tongji Univ. 0.954 Jiangnan Univ. −2.163
18 Univ. of Macau 3.046 Univ. of Science and Technology Beijing 0.932 Jinan Univ. −2.227
19 Northwestern Polytechnical Univ. 2.915 China Agricultural Univ. 0.704 Hangzhou Normal Univ. −2.245
20 Wuhan Univ. of Technology 2.914 Beijing Institute of Technology 0.587 Southern Medical Univ. −2.263

Descriptive statistics of the comparison among 197 American and 205 Chinese universities.

Row Labels low middle High Isolates Grand Total
China 116 67 21 1 205
USA 36 60 99 2 197
Grand Total 152 127 120 402

Highest ranked universities in top-groups in the combined American-Chinese set of 402 universities in LR 2020 with their respective z-scores.

rank Top 20 Chinese universities z Top 20 American Universities z
1 Tsinghua Univ. 11.005 Harvard Univ. 36.632
2 Hunan Univ. 10.193 Stanford Univ. 26.028
3 Hong Kong Univ. of Science and Technology 6.566 Massachusetts Institute of Technology 24.504
4 Univ. of Science and Technology of China 6.482 Univ. of California, Berkeley 20.319
5 City Univ. of Hong Kong 6.454 Yale Univ. 16.576
6 Shandong Univ. of Science and Technology 6.444 Univ. of California, San Francisco 16.524
7 Hong Kong Polytechnic Univ. 6.406 Princeton Univ. 16.522
8 South China Univ. of Technology 6.049 Columbia Univ. 16.348
9 Chinese Univ. of Hong Kong 5.993 Univ. of California, San Diego 16.061
10 Nankai Univ. 4.860 Univ. of Pennsylvania 15.975
11 Univ. of Hong Kong 4.418 Cornell Univ. 14.934
12 Shenzhen Univ. 4.089 Univ. of Washington, Seattle 14.620
13 Qufu Normal Univ. 3.724 Johns Hopkins Univ. 14.540
14 Fuzhou Univ. 3.487 Northwestern Univ. 14.488
15 Central China Normal Univ. 3.090 Univ. of California, Los Angeles 14.450
16 Univ. of Macau 3.046 Univ. of Michigan 13.951
17 Wuhan Univ. of Technology 2.914 California Institute of Technology 13.695
18 Southern Univ. of Science and Technology 2.332 Univ. of Chicago 13.561
19 GuangZhou Univ. 2.120 Duke Univ. 12.605
20 Hong Kong Baptist Univ. 2.094 Washington Univ. in St. Louis 12.123
eISSN:
2543-683X
Language:
English
Publication timeframe:
4 times per year
Journal Subjects:
Computer Sciences, Information Technology, Project Management, Databases and Data Mining
Journal RSS Feed