An author credit allocation method with improved distinguishability and robustness

Collaboration has become a common practice in scientific production (Hara et al., 2003; Hu et al., 2020; Newman, 2004; Regalado, 1995). While scientific collaboration promotes publication output and the work’s impact (Dong et al., 2017; Pravdić et al., 1986; Wu et al., 2019), it brings difficulties in determining who gets how much credit for the team’s work (Hodge et al., 1981; Zeng et al., 2017). This gives rise to a new problem of credit allocation, which aims to quantitatively calculate the share of the credit by each author (Allen et al., 2014; Kennedy, 2003). However, as most important decisions in science, such as job appointment, funding application, tenure promotion, and award nomination, are based on the achievements of the individual (Jones, 2011), it is more important for credit allocation methods to accurately and confidently identify the leading author or the owner of a collaborated work (Rennie et al., 1994; Smolinsky et al., 2020), the one who gets the most credit among all authors.

In practice, the science community has developed several ways to solve the credit allocation problem. One is to consider all co-authors in the team get the same credit for the work (Burrell et al., 1995; Oppenheim, 1998; Price, 1981; Rennie et al., 1994). Equivalently, any co-author can claim himself/herself as the owner of the work. This is most common in fields like Mathematics, Economics, and Particle Physics, where the author’s name is usually listed alphabetically in the paper (Endersby, 1996; Frandsen et al., 2010; Waltman, 2012).

The other way takes the byline order of authors into consideration (Beveridge et al., 2007; Riesenberg et al., 1990; Zuckerman, 1968). The last author, who is usually the corresponding author, and the first author take a greater share of the credit. Considering the observation that the first and last authors usually conduct most jobs of the work (Tscharntke et al., 2007), it seems fair to make them the owner of the paper. While these two approaches have their pros and cons (Das et al., 2020; J. Xu et al., 2016), they are widely applied in the science community for their simplicity and feasibility.

Recently, an algorithm-based approach is proposed that tackles the credit allocation problem from a new angle (Shen & Barabási, 2014). The leading author of a paper is not determined by factors on that paper, such as the byline order of authors, but is determined by the perception of the scientific community. By analyzing how the target paper and papers by the team members are co-cited, the leading author can be quantitatively identified, who is the one most appreciated by peers in the community and recognized as the leader of the teamwork. The original method and other variants (Bao et al., 2017; F. H. Wang et al., 2019; J. P. Wang et al., 2019; Xing et al., 2021) are tested in empirical data and demonstrate high accuracy in determining Nobel Prize laureates from the team that uncovers the scientific discover together.

However, existing methods present several limitations that can be further addressed. First, it is important to determine the weights of different papers in the citation network (H. Huang et al., 2022; Waltman et al., 2013). the number of citations, which has been widely accepted as a measure of a paper’s awareness (Eysenbach, 2006; L. L. Xu et al., 2021; Waltman, 2016; Radicchi et al., 2008; Sinatra et al., 2016), is considered by some methods and improves their performance. But given the scale-free nature of the citation distribution (Barabási, 2009; Barabási et al., 2001; Barabási et al., 2003), the weights of papers can be very screwed towards some highly cited ones. Second, the robustness of the method needs to be further verified and strengthened. As one can publish papers that intentionally cite the target paper and ignore other related co-publications, the methods have to be robust to such manipulations. Finally, to identify the leading author of the paper, we would prefer that the best candidate is distinguishable from others. If the methods give close or tied scores to candidates in the first and second positions, the leading author may be indistinguishable. This issue can be critical in dealing with large-scale data. For instance, if one wants to investigate the correlation between the leading author of a paper and his/her byline order in the author list (Lu et al., 2022), the method has to ensure that the identified owner of a paper is well separated from other co-authors.

To address the aforementioned issues, we propose a new allocation algorithm called NCCAS. We use a modified Sigmoid function that transfers citation counts to weights of different papers, which successfully solved the issue of the fat-tail distributed citation numbers. Furthermore, we remove the target paper in the credit calculation, which brings improved distinguishability of the identified leading author. Following the testing framework of previous studies, we validate the accuracy of NCCAS using Nobel Prize-winning papers and their citation networks in the American Physical Society (APS) and the Microsoft Academic Graph (MAG) dataset. In addition, we use 654,148 articles published in the field of computer science from 2000 to 2009 in the MAG dataset to validate the distinguishability and robustness of NCCAS. Compared with other baseline methods, NCCAS not only identifies Nobel laureates in their collaborated papers more accurately, but also assigns higher scores to them, making the leading author of the work more distinguishable from other candidates. In addition, NCCAS is more robust against additional citations in hypothetical malicious manipulation experiments. Given NCCAS’s capability in handling large-scale publication data and its nice performance, it has the potential to be a useful tool in other studies related to different roles or contributions of authors.

The first collective credit allocation (CCA) algorithm is introduced by Shen and Barabási (2014). As shown in Figure 1, assume a target paper p₀ co-authored by author A and author B. To identify which author is recognized more by the community, we need to identify all papers that cite the target paper. Through the set of citing papers, we further identify the set of papers that are co-cited with the target paper and are co-authored by at least one of the two authors. From the set of co-cited papers, we get an authorship involvement matrix A that counts the fractional share of authorship of the two authors in each co-cited paper. Furthermore, we get a strength vector s that assigns different weights to co-cited papers. In CCA, the element of s is the number of co-citations one co-cited paper receives with the target paper. The product of the matrix A and the vector s gives the total score of authors A and B, which represents the overall appearance of each author in related co-cited literatures weighted by their importance.

Figure 1.

Two extreme cases may help us understand the idea behind the CCA algorithm. If author A does not participate in any co-cited papers, the algorithm will assign author B as the leading author of the target paper. If author A and author B always appear together in co-cited papers, the algorithm will designate both authors as the owners of the target paper. In general, the ownership identified falls somewhere between these two extreme examples. Author A and author B have some research collaborations, as well as collaborations with other authors on similar research topics.

Since the development of CCA, other variants are introduced aiming to improve the allocation accuracy from different perspectives. All of these algorithms focus on the modification of the involvement matrix A and the strength vector s. CCA considers that all citing papers have the same importance. Hence, the element in vector s is the number of citing papers that cite the co-cited papers, or equivalently the number of co-citations. In the NCCA algorithm, F. H. Wang et al. (2019) consider that citing papers with more citations is more important. The element value of s is calculated by summing up citations of citing papers that cite the co-cited paper. In the DCA algorithm, Bao et al. (2017) adjust the involvement matrix A by multiplying it by a residual influence Φ. The idea is that the importance of a paper would gradually decrease with time, which is captured by Φ. DCA uses the same strength vector s as that in CCA. The final credits of the co-authors are determined by the product of the author involvement matrix A, residual influence Φ, and the strength vector s. J. P. Wang et al. (2019) propose IDCA method that uses the same residual influence Φ as that in DCA but modifies the strength vector s. The element value of s is calculated by summing over the product of the PageRank value and the citation number of the citing paper. Xing et al. (2021) propose the CoCA method that modifies the collection of papers in matrix A and the value of the strength vector s. CoCA focuses on the subsequent works by co-authors of the target paper (papers authored by at least one of the co-authors). These subsequent papers generate the involvement matrix A. For each subsequent paper, its weight in s is calculated as the number of times that the subsequent paper and target paper use the same reference paper.

While all these methods demonstrate reasonable accuracy in determining the Nobel laureates, they present several limitations that can be addressed. First, the number of citations (or co-citations) is commonly utilized to calculate the strength vector s. But the number of citations usually follows a scale-free distribution. Therefore, s can be skewed towards some extreme values. Furthermore, most of these methods are tested in citation networks composed of APS publications. It is important to extend the analyses to a bigger and more comprehensive dataset such as the MAG dataset. These methods usually give very close credit scores of different co-authors, making the leading author of the paper not very distinguishable from others. Finally, as citations can be intentionally added to the target paper, it is important to perform the robustness test, which is not well emphasized in past studies.

We utilize two mainstream scholar data sets in this paper. The APS dataset is from 11 Physical Review journals launched by the American Physical Society, covering the period from 1893 to 2009, with a total of 463,332 papers, 4,710,547 citation pairs, and 247,974 author information. The Microsoft Academic Graph (MAG) dataset, covers 71 academic fields, 224,876,396 papers, 1,492,784,521 citation pairs, and 236,021,246 author information. To avoid author name ambiguity, we apply Rule-matching Name-disambiguation (Jia et al., 2017; Sinatra et al., 2016) to disambiguate the author information in the APS dataset. We extract the full name of their last name, first name, and middle name initials for uniform replacement. For the MAG dataset, we directly use the disambiguated author names in the data.

We select the Nobel Prize-winning paper in Physics used in previous studies (Bao et al., 2017; F. H. Wang et al., 2019; J. P. Wang et al., 2019; Shen et al., 2014; Xing et al., 2021) as the validation dataset to verify the accuracy of our algorithm. Totally, there are 30 Nobel Prize-winning papers considered by past studies. After removing papers with only one author and papers that all co-authors receive laureates, 24 multi-author papers were retrieved in the APS dataset and 22 in the MAG dataset. Details can be found in Appendix Tables S1 and S2.

At the same time, we extract 654,148 papers in the field of computer science in the MAG dataset to validate the distinguishability of identified paper owners.

3.1

Method

In this work, we propose the Nonlinear Collective Credit Allocation with a modified Sigmoid (NCCAS) algorithm (Figure. 2). NCCAS is similar to NCCA, which takes the citation number of the citing paper into consideration. Compared with other related methods, NCCAS introduces two major modifications.

Figure 2.

First, to handle extreme values in citation counts and to make the distribution of element values in s bounded, we introduce a modified Sigmoid function (Eq. (1)) that transfers the citation counts to the weight of a paper. The parameter α in Eq. (1) is the average number of citations that the citing papers receive. In the ablation experiment of this paper, we show that the average value gives the most accurate identification of the Nobel laureates compared with the use of median and mode values. The parameter n is the average number of citations of all papers considered. The value varies with different tasks and datasets used. For example, for the task of identifying Nobel laureates, we have n = 20.813 in the APS dataset and n = 22.383 in the MAG dataset. When applying NCCAS to 654,148 papers in computer science, we have n = 13.086. In general, the parameter α sets the scale of citing papers that cites the same target paper and n sets the scale of the whole data considered. 1 S(x)=11+e−xa*Factor,Factor={ 1α,x=0,orx≥nxα,0<x<n \[S(x)=\frac{1}{1+{{e}^{-\frac{x}{a}}}}*Factor,\,Factor=\{\begin{array}{*{35}{l}} \frac{1}{\alpha },x=0,\,or\,x\ge n \\ \frac{x}{\alpha },0<x<n \\ \end{array}\]

With Eq. (1) that rescales the citation counts, the element value of the strength vector s can be calculated as si=∑j∈JS(xj)${{s}_{i}}=\underset{j\in J}{\mathop \sum }\,S({{x}_{j}})$, where i represents a co-cited paper, J is the set of citing papers that cite the co-cited paper i, and x_j is the number of citations of the citing paper j.

The second modification is the removal of the target paper in the involvement matrix A and the strength vector s. This is because that all citing papers are associated with the target paper. Hence, the target paper will have the largest value in strength vector s. Consequently, keeping the target paper in A will dilute the contributions of co-cited papers, making the credit score C less sensitive to co-citing papers. In the ablation experiment, we show that the removal of the target paper itself in calculating C will make the identified leading author more distinguishable from others.

Taken together, as shown in Figure 2, the procedure of NCCAS is as follows:

Find all the papers that cite the target paper p₀, forming the set of citing papers set D=(d₀, d₁, …, d_m).

Find all the papers that are authored by at least one author of the target paper. Among them, find the set of co-cited papers, P=(p₁,p₂,…,p_k), that are cited with the target paper by the citing paper.

Calculate the strength vector s for the co-cited papers. The element value s_i is the sum of the transformed weight of citing papers that cite the co-cited paper i. For example, for the co-cited paper p₂ in Figure 2, we have s₂=S(x₁)+S(x₂)+S(x₃)+S(x₅), as four papers (d₁, d₂, d₃, d₅) cite both the target paper p₀ and the co-cited paper p₂.

Generate the authorship involvement matrix A by calculating the authorship credit of an author in co-cited papers. The authorship credit value is calculated using the fractional counting method. If author a, one of the authors in the target paper, appears in a co-cited paper with a total of three authors, the authorship credit of author a in this co-cited paper is 1/3.

The credit shares of all co-authors in the target paper p₀ are given by vector C=As. By normalizing vector C, we can obtain the credit shares of co-authors in fractional form.

In this part, we perform different experiments to explain the reason for the modifications that NCCAS makes, and the reason for the choice of some parameters.

To summarize, we propose the NCCAS method to allocate credit for each co-author and identify the leading author of a multi-author publication. We introduce a modified Sigmoid function to rescale the citation number of the citing papers. We remove the target paper in the calculation of the credit share. Compared with other credit allocation methods, NCCAS gives the best performance in identifying Nobel laureates in both the APS and the MAG datasets. In addition, the leading author identified is well separated from other co-authors in terms of their credit scores, providing an improved distinguishability compared with other methods. NCCAS is also more robust under manipulations, which acquires the largest number of added papers to invert the leading author. These features make NCCAS very applicable to large-scale publication data.

Future applications include analyzing the role of a paper’s leading author. It is interesting to check if the leading author tends to be the first author, the last author, or the author with the longest academic age (Drenth, 1998; Hundley et al., 2013; Sekara et al., 2018), and how this pattern differs in different disciplines. The identification of a paper’s leading author would help in the study of a scientist’s research agenda (Huang et al., 2023; S. Huang et al., 2022; X. Yu et al., 2021). Taking only publications led by a scientist will reduce the influence of papers that he/she participates in with less significant roles. Finally, identifying the leading author can also help understand the formation and the structure of a scientific team (Milojević, 2014; Yu et al., 2022). NCCAS can be a useful tool to address these questions.