A Bibliometric Framework for Identifying “Princes” Who Wake up the “Sleeping Beauty” in Challenge-type Scientific Discoveries

As early as in the 1960s, some scholars noted the phenomenon that quite a few major scientific discoveries went unnoticed for a long time before they were discovered or rediscovered, and hence were referred to as “resisted discoveries” (Barber, 1961;Cole, 1970), “premature discoveries” (Stent, 1972), or “delayed recognition” studies (Garfield, 1970, 1989, 1990). Studies in this period were mostly case studies from the viewpoint of the sociology of science, and thus gave no quantitative definition about how abrupt a citation boost must be to suggest delayed recognition (Garfield, 1980).

The quantitative concept of “delayed recognition,” i.e. “sleeping beauties” (SBs) in science, was proposed by van Raan (2004), who defined SB as a publication that goes unnoticed (“sleeps”) for a long time and then, suddenly, attracts a lot of attention (“is awakened by a prince”). He suggested three criteria for distinguishing SBs: (1) deeply asleep (receive at most 1 citation per year on average), or less deeply asleep (between 1 and 2 citations per year on average) for a few years after publication; (2) sleep at least five years; and (3) awakened by over 20 citations during the four years following the sleeping period. Since van Raan’s article, scientometrics interest in sleeping beauties has increased considerably.

Recent studies focus on estimating the frequency of occurrence of delayed recognition measured by citation analysis and a few shared features (Glänzel & Garfield, 2004; Glänzel, Schlemmer, & Thijs, 2003); SBs in a specific journal (Kozak, 2013) and in specific subject categories, such as in psychology (Lange, 2005), physics (Marx, 2014; Redner, 2005), mathematics (van Calster, 2012), and medical sciences (Gorry & Ragouet, 2015; Ohba & Nakao, 2012); the awakening probability of SBs based on a well-known stochastic model (Burrell, 2005) or the citation distribution within the sleeping period (Li et al., 2014); the citation lifecycle of SBs (Lachance & Larivière, 2014); a new type of SB with a short leap immediately after publication (Li, 2014; Li & Ye, 2012); the application-oriented tendency of the SBs in physical and engineering sciences (van Raan, 2015). All the above studies identified SBs in terms of average-based criteria directly or indirectly referring to van Raan’s definitions.

Different from average-based criteria, Costas, van Leeuwen, and van Raan (2010) used quartile-based criteria, in which papers with delayed recognition are those that have not received 50% of their citations when 75% of other documents in their fields have already received 50% of their citations. These studies concluded that potential cases of delayed recognition are rare in general, and thus have almost no effect on research evaluations for both individual scientists and research groups (Costas, van Leeuwen, & van Raan, 2011, 2013).

Subsequent developments are the redefinition of SBs by referring to average criteria, quartile criteria, and the “boost factor” of citations (Mazloumian et al., 2011). The new method is especially suitable for distinguishing exponential SB citation curves (Li & Shi, 2015). Sun, Min, and Li (2015) proposed a vector for measuring obsolescence of scientific articles. It is designed as O = (Gs, A-, n), where n is the age of an article, and Gs and A- are parameters for revealing the shape of citation curves. The obsolescence vector yielded more reasonable classifications than did the average-based and quartile-based criteria. Nevertheless, the number of SBs identified is heavily dependent on the adopted rules, which are based on arbitrary threshold parameters for sleeping time and citations.

Previous studies all resulted in the conclusion that SBs are rare and exceptional events in science. To overcome these limitations, Ke et al. (2015) introduced a parameter-free approach, i.e. a “beauty coefficient,” to quantify the extent to which a given paper can be considered an SB. The beauty coefficient value B for a given paper is based on a comparison between its citation history and a reference line that is determined by its publication year, the maximum number of citations received in a year, and the year when such maximum is achieved. Their results revealed that the fraction of SBs may be well within 6.5% of all papers indexed in Web of Science. They concluded that the SB phenomenon is far less exceptional than previously thought. Yet this approach fails to cover the citation curve after the paper receives its maximum annual citations (Du & Wu, 2015). The average-based, quartile-based and parameter-free criteria have both advantages and disadvantages in defining and identifying SBs in science, and no consensus about precise definitions of an SB and related concepts has been reached by the scientometric community.

Every sleeping beauty has her prince(s), PR for short, who wakes up the SB by citing the SB document. Following the PR’s example, other scientists begin to cite the SB more frequently. There are two kinds of understandings to the identification of the PRs. The first argument focuses on the direct citation relationship between SB and PR, i.e., the PR must cite the SB. According to van Raan (2004), the PR is the first paper to quote the SB (self-citing excluded) before a “citation boom.” Braun, Glänzel, and Schubert (2010) sought the candidate PR articles among those that first cited the SB article. Such candidates were supposed to be highly or at least fairly-well cited and to have a considerable number of co-citations with the SB. The second viewpoint emphasizes co-citations between the SB and the PRs by subsequent scholars. Ohba and Nakao (2012) scanned the reference list of articles citing the SB and searched for PRs that were published around the time of the SB’s awakening, which prompted the authors of subsequent related works to refer to the SB.

The strength of co-citations was determined by the frequency with which the SB-PR pair is cited together in reference lists. Articles with a co-citation frequency of more than 30% were regarded as PRs for the SB. Li et al. (2014) claimed that the PR is not necessarily the first article that cited the SB. Instead, the PR may just have cited the SB earlier, and—more importantly—caused the SB to receive wide attention and thus awaken. The PRs and the SB are supposed to have a high co-occurrence in the citation network of the SB. For finding the true PRs objectively, they recommended to analyze not only direct citations to the SB, but also consider indirect citations, including using the SB’s viewpoint without explaining the original paper, citing knowledge of the SB but not indexing it in references, and so on.

New discoveries and theories are crucial for the development of science, but they are often initially resisted or neglected, and encounter skepticism from the scientific community (Campanario, 2009; Fang, 2014). So the mechanism underlying delayed recognition is always relevant to major scientific progress or groundbreaking scientific revolutions.

Daniel E. Koshland, a former Science editor-in-chief, proposed the “cha-cha-cha” typology of scientific discoveries (Koshland, 2007). “Challenge” discoveries are a response to an accumulation of facts or concepts that remained unexplained by, or were incongruous with, scientific theories of the time; “Chance” discoveries suddenly appear and are often referred to as serendipitous or “unsought findings”; “Charge” discoveries solve known problems within an existing theoretical framework which can be considered. According to Kuhn’s “paradigm” (Kuhn, 1962), “charge” and “challenge” discoveries relate to two types of innovative research, cumulative processes and revolutionary breakthroughs, respectively. Koshland’s “cha-cha-cha” theory was explained to the informetric community in Rousseau (2007).

It is important to avoid delayed recognition and to detect SBs as early as possible, in order to promote potentially valuable but not readily accepted innovative research. To accomplish this goal, it is imperative to find the PRs who awaken SBs and figure out why this happens. However, finding the true PR article is a difficult task. In this paper we propose a set of criteria for discerning papers as PRs who brought challenge-type SBs to the attention of the scientific community. In the next section, we will describe a framework for identifying PRs who woke up SBs, and validate the criteria in terms of a case study of a landmark paper on super-resolved fluorescence microscopy.

Currently, there are two approaches to studying the relationship between SBs and candidate PRs. One argues that the PR is the SB’s first citing document; the other claims that the SB and the PR have substantial co-citations, without considering whether the PR directly cited the SB.

We consider the definition of the PR as the first paper that cited the SB suitable only for the very rare cases of “coma sleep”, i.e. there is no attention at all in terms of citations, as acknowledged in van Raan (2015). Instead, the SB-PR co-citation relationship should be given priority in finding the PR(s), based on the following considerations.

First, compared with the 19^th and 20^th centuries, and especially since the introduction of the Internet, the SBs who are sleeping are rarely uncited. Rather, they tend to be cited sporadically by a few publications, including some self-citations.

Second, an alarming trend has been noted towards the citation of review articles rather than the primary research papers in which the original findings were described (Knottnerus & Knottnerus, 2009; Marks et al., 2013). It seems that busy authors do not cite the original work but more recent and perhaps derivative work and reviews. Such behavior renders the papers “cited but not read.” Moreover, as the number of papers rises inexorably, authors often have to make a choice about what to cite, and ideas persisting in authors’ minds may not be included in the bibliography. Such papers are “read but not cited” (Florence, 2015). The San Francisco Declaration on Research Assessment (DORA) recommends that researchers “wherever appropriate, cite primary literature in which observations are first reported, rather than reviews, in order to give credit where credit is due.”

http://www.ascb.org/dora/

Third, the “real” awakening of the SB is often not a “one PR only” action; hence, observing only the first citation is a limited view.

In order to discover the PRs who wake up SBs, we propose that candidates for being a PR meet the following four criteria:

Be published near the year in which the SB began to attract a lot of citations;

The first three criteria are easy to understand, and are in accordance with previous viewpoints ( Braun, Glänzel, & Schubert, 2010; Ohba & Nakao, 2012). Recently developed construal-level theory (Trope & Liberman, 2010) explains the mechanism of people appraising the value of an object through psychological distance. It has been found that if an object is far from one’s experiences, he/she will form more abstract representations, or higher level construal, of the object (Liberman et al., 2007). In other words, people are at a large psychological distance from an unfamiliar object.

Novelty and creativity, which have close relationships with scientific discovery, have been shown to be related to high-level construals (Förster, Friedman, & Liberman, 2004). Consequently, people will always estimate a lower probability than the actual value to novelty and creativity in science. Since breakthroughs proposed by an SB are always ahead of their time and outside of the current scope of knowledge, or even violate previous theories, the scientific community may not know of their existence, or dare not to cite the SB, as the new idea has not been observed experimentally, or evidence about the new theory has not been reported. Once the PR occurred, the scientific community discovered and understood the SB because the PR re-introduced or explained the new idea, and thus provided clues for people to know about it. The scientific community then began to cite the SB publication in their work because they were assured now that the work of the SB is correct.

People are at a larger psychological distance from the SB than from the PR. So, at the moment before the SB’s citation boom, the number of annual citations of the PR is larger than that of the SB. The PR and the SB are either concept-related or citation-related. In fact, this criterion is also suitable for cases in which the SB is uncited during the sleeping period. In the next section, we will test the usefulness of our bibliometric framework through a key publication of the 2014 chemistry Nobel laureate Stefan W. Hell, who is credited with breaking Ernst Abbe’s diffraction resolution limit in fluorescence microscope, although Abbe’s theory was one of the most prominent paradigms in the natural sciences.

In our previous work (Du & Wu, 2015), we concluded that by examining the reference list of articles citing the SB, one can discern the PR articles characterized by the following features: (1) they were published not long after the SB’s awakening; (2) they are highly co-cited with the SB; 3) they have almost the same co-citation speed and “co-delayed recognition index” with the SB. But it seems a tough task when we first tentatively calculate the value of every document’s co-citation speed and “co-delayed recognition index” with the SB, and then compare them with those of the SB. Since the scientific community regards the discovery in Hell (1994) as transformative innovation, in the present study, we try to develop and validate a simplified framework to identify the PRs.

In this paper, we not only found that Hell (2000) is the first PR, but also detected another PR, — i.e. Hell (2003) — and two “retinues” of the PR, Betzig (2006) and Rust (2006). It seems that these four documents (two PRs and two retinues) introduced Hell (1994) to subsequent scholars and thus made her an SB. By examining the reference list of articles citing SBs, one can discern PR articles by means of our bibliometric framework.

Our study also confirmed and improved the characteristics of PRs summarized by Braun, Glänzel, and Schubert (2010). First, they are published around the SB’s awakening. Second, they are highly cited papers and highly co-cited with the SB. Third, they are supposed to demonstrate an obvious pulling effect on the SB, especially in case of transformative innovations. Using the concept of Kuhn’s “paradigm,” there are two kinds of innovative research — cumulative processes and revolutionary breakthroughs. The bibliometric framework is more suitable for the latter. Publications belonging to so-called transformative research, when less-cited, may be given special attention, because they may suddenly attract a lot of attention after a period of time, as reflected in our case study.

Finally, an SB may need a PR plus other agents to wake her up, and documents with potential awakening functionality tend to be published in multidisciplinary and prestigious journals with higher impact and wider scope than the journals publishing SBs. For example, PNAS, Nature Biotechnology, Nature Methods, and Science contrast with Optical Letters in our case study. The rapid application of super-resolution microscopy developed by physics scientists are already applied on a large scale in major fields of biology and life sciences, such as cell biology, microbiology, and neurobiology (Huang, Babcock, & Zhuang, 2010). Now, scientists are active in introducing new developments of other disciplines into their own research. Interdisciplinary research promotes the evolution of science (Fang, 2014). In many cases, the SB phenomenon occurs when research finds application in a field outside of its own, such as statistical methods that become useful in biology.

A sleeping beauty may need one or more “princes” and even “retinues” to be awakened. The PRs may be either conceptually related or direct-citation-related to the SB. In this study, the PRs include two documents of Hell himself, who believed and insisted on his theory, and finally succeeded. Although the other two PRs did not cite the SB in the first generation, they cited other documents of Hell, and these citations induced the citation leap of the SB. Hell did not co-author with Betzig’s group or Zhuang’s group; it is through a knowledge relationship (citation) that they contributed to the citation leap of the SB.

The study of the SB and her PRs has been considered “a rather useful and instructive model in studying the mechanisms of scientific information flow through citations” (Braun, Glänzel, & Schubert, 2010), and is likely to contribute to the understanding of citation dynamics in general. The key scientific issues of the study towards SBs is to figure out the mechanisms for awakening, in order to promote potentially valuable but not readily accepted innovative research (Fang, 2014; Wang et al., 2012). Trapido (2015) discussed why novelty is sometimes an advantage and sometimes a liability, and what is required for highly novel knowledge to earn evaluating audiences’ recognition, rather than neglect or skepticism. If the scientific community could learn more lessons from previous resistance to discoveries, the dilemma of science opposing innovation will be at least partially solved (Fang, 2014). The above conclusions are drawn from a detailed analysis of a typical SB and her awakening process. It requires further empirical study to investigate whether the conclusions can be expanded to other SBs or fields besides chemistry and physics.