Delayed recognition: recent developments and a proposal to study this phenomenon as a fuzzy concept

A publication suffering from delayed recognition is a publication that received very little attention shortly after publication, but received recognition later. Stephen Cole proposed to use citations as a proxy for recognition (Cole, 1970). Although recognition can be given in many ways—receiving tenure is another important way in which scientists are recognized for their achievements—collecting received citations is the most practiced way to operationalize the notion of delayed recognition. This contribution is not meant as a review of the topic, but we concentrate on a few recent developments. Yet, among the many papers written by colleagues on delayed recognition we single out for mention: (Bornmann et al., 2018; Burrell, 2005; Du & Wu, 2016; El Aichouchi & Gorry, 2018; Garfield, 1980; Glänzel et al., 2003; Ke et al., 2015; Li & Ye, 2012; van Raan, 2004, 2015, 2017).

In this short paper we will discuss three aspects: naming of the phenomenon, recent methods based on a cumulative citation curve and re-interpretation of delayed recognition as a fuzzy concept.

The concept of delayed recognition in relation to persons or articles has also been described as premature discovery, suffering from Mendel’s syndrome, being late bloomers or being ahead of one’s time. Mendel’s work on the rules of heredity is often considered as the prototype case. Yet, Mendel’s work was not totally unknown before the 20^th century as mentioned by Garfield (1970), giving reference to Zirkle (1964).

In an article published in 2004, Ton van Raan proposed the name “sleeping beauty” for an article suffering delayed recognition (van Raan, 2004). This catchy term took on immediately: on June 3, 2018 van Raan’s article had already received 176 citations in the Web of Science (WoS). When the value, importance or usefulness of such a “sleeping beauty” is finally recognized in another article, denoted here as P, serving as a wake-up call for the scientific community (leading to general recognition of the “sleeping beauty”), article P is referred to as the Prince, continuing the metaphor of the story of the Sleeping Beauty. The act of “awakening” the sleeping beauty is then sometimes referred to as “the kiss.”

Sugimoto and Mostafa (2018) recalled that, in the context of sleeping beauties, Braun et al. (2010, p. 198) discussed the “ideal couple” and further sexualized the metaphor by discussing male and female dominance and “absolute superiority”: a measurement of the relative citations achieved by the prince and the sleeping beauty. Finally, they introduced the notion of chastity of sleeping beauties, in terms of the number of articles that awoke the dormant article and mentioned the possible unfaithful behavior of princes. Clearly a form of sexualization of citation trajectories has been—and is still—going on.

It is clear that these types of metaphors, continuing with “brave girls” for articles which are immediately recognized (Ye & Bornmann, 2018) have the tendency to become more and more gender-loaded. For this reason Sugimoto & Mostafa (2018) wrote an editorial, decrying this “clear violation of sociocultural norms”. They made a plea to future authors that the use of any such terms, despite connections to historical roots in the literature, should be avoided. As a consequence they stated that JASIST’s author guidelines will be adapted to make this policy explicit and clear.

As a reaction Hu et al. (2018) proposed the metaphor of gender-neutral terms “hibernator” and “awakener” to replace the terms “sleeping beauty” and “prince”. It is, of course, an open question if any metaphor is really useful.

Now we propose a framework to study delayed recognition of an article at a given moment in time, say T. More precisely, we consider the question: does this article suffers delayed recognition or has it in the past (while now it, perhaps, behaves like a normal article, already receiving a declining number of citations).

Studying this question we consider three aspects: “delayed,” “recognition” and fuzzy membership.

When it comes to the “delayed” part, this implies that one must wait a certain period before one may say that there is a delay. In this study we wait at least ten years (see further for details), but further investigations are needed to study the influence of this starting time. Does it matter if one starts investigations 10 years after publication or is 15 or 20 years better?

Next we come to the “recognition” part. We propose to concentrate on the 1% most cited publications in the same publication year as the publication under investigation. A choice must further be made to include all publication types in this 1% or only normal articles (or normal articles and reviews). We think that here all choices are valid, i.e., have some scientific value, but the choice must be stated clearly.

Finally we come to the most difficult part: constructing a framework to come to a fuzzy membership value. This value, between zero and one, must in a meaningful way express to which extent an article can be said to belong to the fuzzy set of publications with delayed recognition. This membership function, as calculated at time T, is denoted as DR(T). If an article is not “recognized”, i.e. it does not belong to the 1% most cited, it is not ahead of its time and its DR(T) value is set equal to 0 for any T. Our approach is based on ideas from Ke et al. (2015) and Du and Wu (2018).

To the best of our knowledge Ke et al. (2015) were the first to state that suffering delayed recognition is not a yes-no situation. They introduced a parameter-free measure that quantifies the extent to which a specific paper can be considered to suffer delayed recognition. Papers with citations growing linearly with time have B = 0. B is non-positive for papers whose citation trajectory is a concave function of time and positive for papers with a convex citation curve. Du and Wu (2017, 2018) proposed a similar measure, but based on the cumulative citation curve.

We will calculate a partial membership function, denoted as K(t), for each time t between 10 and T. The final DR(T) value is then equal to:

DR(T)=Max10≤<!-- = -->t≤<!-- = -->T{0,K(t)}$$\begin{array}{} \displaystyle DR(T)=Max_{10 \leq t \leq T}\ \{0, K(t)\} \end{array}$$(3)

The use of the maximum function in formula (3) avoids that the DR coefficient diminishes over time, which is against the definition of the concept of delayed recognition: once an article is accepted to have suffered delayed recognition this cannot be undone.

When determining K(t) for given t, we define C(n) equal to the cumulative number of received citations at the beginning of year n (where year 0 is the publication year) and hence C(0) = 0; c(1) = C(1) denotes the number of citations received during the publication year. If C(t) = 0 then K(t) is set equal to 0. If now C(t) ≠ 0, we consider the line y(n) connecting the origin (0, 0) with the point (t, C(t)). This line, which we call the recognition line at time t, has equation

y(n)=C(t)tn$$\begin{array}{} \displaystyle y(n)=\frac{C(t)}{t}n \end{array}$$(4)

Now we calculate the sum of the differences in each n, 0 ≤ n ≤ t, between the line y(n) and the cumulative citation curve C(n). This sum is denoted as S(t).

S(t)=∑<!-- ? -->n=0tC(t)tn−<!-- - -->C(n)$$\begin{array}{} S(t)=\sum^{t}_{n=0}\left(\frac{C(t)}{t}n-C(n)\right) \end{array}$$(5)

The largest possible value of S(t) occurs when all C(n) are zero except C(t). This happens if the publication receives its first citation in the year t. Yet, we are not interested in that year, but just use this value as a reference. For this case, S(t)=∑<!-- ? -->n=0t−<!-- - -->1C(t)tn=C(t)t(t−<!-- - -->1)t2=(t−<!-- - -->1)C(t)2.$\begin{array}{} S(t)=\sum^{t-1}_{n=0}\left(\frac{C(t)}{t}n\right)=\frac{C(t)}{t}\frac{(t-1)t}{2}=\frac{(t-1)C(t)}{2}. \end{array}$ Finally K(t) is defined as the ratio of the observed S(t) value over the largest possible one:

K(t)=2(t−<!-- - -->1)C(t)S(t)$$\begin{array}{} \displaystyle K(t)=\frac{2}{(t-1)C(t)}S(t) \end{array}$$(6)

leading to a value between -1 and +1.

K(t) is negative if C(n) is always situated above the line y(n) and certainly positive when C(n) is always situated under this line. Yet, K(t) may also be positive when parts of C(n) are above the recognition line y(n).

We note that if an article receives its first citation in year 10 and is ‘recognized’ then, based on equation (3), its DR(T) value is equal to one, for all T ≥ 10.

We reviewed recent developments related to the study of delayed recognition, leading to the idea to consider delayed recognition as a fuzzy concept. We proposed a method to obtain fuzzy membership values. One of the requirements for suffering delayed recognition, is that the article must belong to the 1% most-cited ones. This means that at most 1% of the articles under consideration have a non-zero fuzzy membership value, and probably much less than 1%. The value 0.333 for linear growth in citations can be considered a benchmark for comparisons.

Besides proper hibernators (sleeping beauties) who have a long period with no or few citations, articles suffering delayed recognition may have a convex cumulative citation curve, such as in the case of linear growth in citations. Examples of these two types are shown in this contribution: Romans (1986) being a proper hibernator and Leakey et al. (1964) and Otte & Rousseau (2002) being examples of the second type.

We made the important observation that using citations to study delayed recognition is just a—convenient—operationalization of the concept, but that experts may agree on delayed recognition long before this is shown by citations. This is illustrated by the case of Leakey et al. (1964). This leads to the question: How good (adequate) is citation analysis for detecting premature discoveries?

As this contribution is just a feasibility study, many questions are left unanswered, such as:

What are typical values for membership functions?

These questions are left as topics for further research.

Finally we mention the obvious limitation: as all citation studies also this one is database dependent.