<abstract xmlns="http://www.w3.org/1999/xhtml">

<sec><h3>Purpose</h3>
<p>In recent decades, with the availability of large-scale scientific corpus datasets, difference-in-difference (DID) is increasingly used in the science of science and bibliometrics studies. DID method outputs the unbiased estimation on condition that several hypotheses hold, especially the common trend assumption. In this paper, we gave a systematic demonstration of DID in the science of science, and the potential ways to improve the accuracy of DID method.</p>
</sec>
<sec><h3>Design/methodology/approach</h3>
<p>At first, we reviewed the statistical assumptions, the model specification, and the application procedures of DID method. Second, to improve the necessary assumptions before conducting DID regression and the accuracy of estimation, we introduced some matching techniques serving as the pre-selecting step for DID design by matching control individuals who are equivalent to those treated ones on observational variables before the intervention. Lastly, we performed a case study to estimate the effects of prizewinning on the scientific performance of Nobel laureates, by comparing the yearly citation impact after the prizewinning year between Nobel laureates and their prizewinning-work coauthors.</p>
</sec>
<sec><h3>Findings</h3>
<p>We introduced the procedures to conduct a DID estimation and demonstrated the effectiveness to use matching method to improve the results. As a case study, we found that there are no significant increases in citations for Nobel laureates compared to their prizewinning coauthors.</p>
</sec>
<sec><h3>Research limitations</h3>
<p>This study ignored the rigorous mathematical deduction parts of DID, while focused on the practical parts.</p>
</sec>
<sec><h3>Practical implications</h3>
<p>This work gives experimental practice and potential guidelines to use DID method in science of science and bibliometrics studies.</p>
</sec>
<sec><h3>Originality/value</h3>
<p>This study gains insights into the usage of econometric tools in science of science.</p>
</sec>
</abstract>

Purpose
In recent decades, with the availability of large-scale scientific corpus datasets, difference-in-difference (DID) is increasingly used in the science of science and bibliometrics studies. DID method outputs the unbiased estimation on condition that several hypotheses hold, especially the common trend assumption. In this paper, we gave a systematic demonstration of DID in the science of science, and the potential ways to improve the accuracy of DID method.

Design/methodology/approach
At first, we reviewed the statistical assumptions, the model specification, and the application procedures of DID method. Second, to improve the necessary assumptions before conducting DID regression and the accuracy of estimation, we introduced some matching techniques serving as the pre-selecting step for DID design by matching control individuals who are equivalent to those treated ones on observational variables before the intervention. Lastly, we performed a case study to estimate the effects of prizewinning on the scientific performance of Nobel laureates, by comparing the yearly citation impact after the prizewinning year between Nobel laureates and their prizewinning-work coauthors.

Findings
We introduced the procedures to conduct a DID estimation and demonstrated the effectiveness to use matching method to improve the results. As a case study, we found that there are no significant increases in citations for Nobel laureates compared to their prizewinning coauthors.

Research limitations
This study ignored the rigorous mathematical deduction parts of DID, while focused on the practical parts.

Practical implications
This work gives experimental practice and potential guidelines to use DID method in science of science and bibliometrics studies.

Originality/value
This study gains insights into the usage of econometric tools in science of science.

Practical operation and theoretical basis of difference-in-difference regression in science of science: The comparative trial on the scientific performance of Nobel laureates versus their coauthors

Department of Statistics and Data Science

Southern University of Science and Technology

Journal of Data and Information Science

This work is licensed under the Creative Commons Attribution 4.0 International License.

Purpose
In recent decades, with the availability of large-scale scientific corpus datasets, difference-in-difference (DID) is increasingly used in the science...

	DID regression Model Results
Model	Model I	Model II	Model III
Matching Techniques	-	CEM	PSM

ATT	−0.1488	−0.0780	−0.1481
SE	0.0998	0.1307	0.1317

Common trend assumption	Fail	Pass	Pass
p-value	0.0999	0.6836	0.1582

Fixed Effect Controls
Individual	Yes	Yes	Yes
Year	Yes	Yes	Yes

	Prize year	Total #Publication	Total #Citation
Minimum	1934	10	443
Maximum	2011	1,627	194,896
Mean	1987	245.9	25,662.6
Median	1991	157	13,061
Standard deviation	18.6	266.1	31,571.4
Count	335	335	335

Average citation index ${\bar{c}}_{i} = \frac{1}{11} \sum_{t = - 10}^{0} c_{it}$ {\bar c_i} = {1 \over {11}}\sum\nolimits_{t = - 10}^0 {{c_{it}}} before prizewinning

Matching Techniques	-		CEM		PSM

Group	Control	Treated	Control	Treated	Control	Treated
Observations	227	108	80	80	84	108
Mean	4.7083	5.2350	5.0701	5.2869	5.1353	5.2350
Std. dev.	1.4886	1.5034	1.7627	1.6595	1.5821	1.5034
Minimum	0.3519	0.1997	0	0	1.3754	0.1997
Maximum	8.4902	8.2194	8.3668	8.4933	8.4902	8.2194

	Treatment	Control	Treatment-Control
Prior	E(Y⁰¹)	E(Y⁰⁰)	E(Y⁰¹)–E(Y⁰⁰)
Posterior	E(Y¹¹)	E(Y¹⁰)	E(Y¹¹)–E(Y¹⁰)
Posterior-Prior	E(Y¹¹)–E(Y⁰¹)	E(Y¹⁰)–E(Y⁰⁰)	[E(Y¹¹)–E(Y⁰¹)]–[E(Y¹⁰)–E(Y⁰⁰)]

Practical operation and theoretical basis of difference-in-difference regression in science of science: The comparative trial on the scientific performance of Nobel laureates versus their coauthors

Article Category: Research Paper

Data publikacji: 05 mar 2023

Zakres stron: 29 - 46

Otrzymano: 30 gru 2022

Przyjęty: 04 sty 2023

DOI: https://doi.org/10.2478/jdis-2023-0003

Słowa kluczowe
Science of Science, Bibliometrics, Difference-in-Difference, CEM, PSM, Nobel Prize

© 2023 Yurui Huang et al., published by Sciendo

This work is licensed under the Creative Commons Attribution 4.0 International License.

Figure 1

Regression results and CTA tests by applying different matching techniques.

Descriptive statistics for the entire sample (Nobel laureates and their prizewinning-work collaborators).

Matching results by applying different matching techniques.

Statistics for the treatment effects.

Practical operation and theoretical basis of difference-in-difference regression in science of science: The comparative trial on the scientific performance of Nobel laureates versus their coauthors

Article Category: Research Paper

Data publikacji: 05 mar 2023

Zakres stron: 29 - 46

Otrzymano: 30 gru 2022

Przyjęty: 04 sty 2023

DOI: https://doi.org/10.2478/jdis-2023-0003

Słowa kluczoweScience of Science, Bibliometrics, Difference-in-Difference, CEM, PSM, Nobel Prize

© 2023 Yurui Huang et al., published by Sciendo

This work is licensed under the Creative Commons Attribution 4.0 International License.

Figure 1

Regression results and CTA tests by applying different matching techniques.

Descriptive statistics for the entire sample (Nobel laureates and their prizewinning-work collaborators).

Matching results by applying different matching techniques.

Statistics for the treatment effects.

Słowa kluczowe
Science of Science, Bibliometrics, Difference-in-Difference, CEM, PSM, Nobel Prize