Scientometric engineering: Exploring citation dynamics via arXiv eprints

Scientists stand on the shoulders of giants by citing their predecessors’ works in their own works¹. Simultaneously, the scientific community often assumes, either explicitly or implicitly, that the more citations, the higher the scientific value. This assumption has long made the citation count and its various derivatives influential in research evaluation and administrative decision-making, offering practical, albeit imperfect, proxy metrics of scientific quality or importance (Garfield, 1972; Hirsch, 2005; Waltman, 2016). While such use of the citation-based metrics is controversial and often criticized (Aksnes, Langfeldt, & Wouters, 2019; Garfield, 1979, 2006; Lehmann & Lautrup, 2006; Line, 1993; Nicolaisen, 2007; Radicchi, Weissman, & Bollen, 2017; Seglen, 1998), they are nevertheless relevant to quantify the attention of other scientists working on related issues, embodying the scientific knowledge transfer and information flow in the academic sphere. The life of scientific literature in terms of citations varies, experiencing widely different temporal patterns (He, Lei, & Wang, 2018; Ke, Ferrara et al., 2015; Redner, 2005; van Raan, 2004; Wang, Song, & Barabási, 2013). Some papers remain continually cited and even become “immortal giants,” while others show fluctuating patterns of citation collection, and, in fact, a significant portion of papers apparently end their lives with little or no citational impact. Nevertheless, once aggregated and averaged across papers in a given research field, the average citation history curves typically follow a “jump–decay” pattern (Barnett, Fink, & Debus, 1989; Glänzel & Schoepflin, 1995); it increases up to a few years after publication to reach a peak and subsequently decreases to some asymptotic value.

The citation pattern has long fascinated researchers and practitioners alike. They have introduced the concept of literature obsolescence (aging) (Gross & Gross, 1927; Line, 1993; Line & Sandison, 1974)—the process of becoming no longer cited or less used—to investigate how quickly a scientific publication moves in and out of the focus of researchers’ attention. Here, obsolescence does not necessarily mean deterioration in its intrinsic value but is a natural consequence of scientific development or technological advancement over time. Scientific discovery presented in a paper and the value it creates are eventually integrated into subsequent papers, thereby contributing to the shoulders to stand on. Having said that, an interesting question is whether the obsolescence rate has changed over time. Some studies (Larivière, Archambault, & Gingras, 2008; Martín-Martín, Orduna-Malea et al., 2016; Verstak, Acharya et al., 2014) have suggested that researchers are increasingly relying on older publications, while others (Evans, 2008; Parolo, Pan et al., 2015) have suggested that the obsolescence of scientific knowledge has accelerated over the past decades. Although results and implications from these previous studies vary without consensus (see also Pan, Petersen et al., 2018; Sinatra, Deville et al., 2015; Zhang & Glänzel, 2017), a common feature was the use of the citation data on papers published in peer-reviewed journals. The bibliometric data used were commonly obtained from publishers or major bibliographic databases, such as Web of Science² and Scopus³.

Nowadays, the study of literature growth and obsolescence has embarked on an entirely new phase. This change is due to the new digital technologies, resources, and online interfaces that have enormously increased the speed and volume of scientific knowledge production and consumption. With the ongoing rapid and large-scale digitization of information, researchers increasingly rely on digital research documents, or eprints, available on a free and open-access platform. These eprints mainly include preprints, not yet peer-reviewed and accepted in scientific journals or other traditional means. Still, they are citable as they are publicly disclosed and accessible on the internet via the preprint servers, which have become an integral part of today’s scholarly publishing system. arXiv⁴, the most prominent and oldest eprint archive, launched in August 1991, has served as an indispensable research platform (document submission and retrieval system) for physics, mathematics, and computer sciences (arXiv.org, 2021; Ginsparg, 2016). For the past 30 years, it has functioned as a primary source of current and ongoing research, accelerating recognition and dissemination of research findings and facilitating rapid scholarly communication. The number of eprints posted on arXiv has grown dramatically, from 30,601 (2000) to 155,866 (2019), increasing more than five-fold during the past 20 years (Supplementary Figure S1a), and the number of citations received by the arXiv eprints has also snowballed with time (Supplementary Figure S1b). Previous studies have investigated how arXiv has impacted traditional publication practices and the researchers’ citation activities (Aman, 2013; Feldman, Lo, & Ammar, 2018; Gentil-Beccot, Mele, & Brooks, 2010; Larivière, Sugimoto et al., 2014; Moed, 2007; Okamura, Yoda et al., 2020; Wang, Chen, & Glänzel, 2020). For instance, evidence has been provided that arXiv has an advantage in accelerating citation (Aman, 2013; Moed, 2007; Okamura, Yoda et al., 2020) (see also Supplementary Figure S2). Recently, the preprint mode of scholarly communication has seen an exponential rise during the COVID-19 global pandemic, driven by the vital need for early and rapid dissemination of COVID-19-related research results. Two growing preprint servers are bioRxiv⁵ (launched in November 2013) and medRxiv⁶ (launched in June 2019), which have offered indispensable platforms for researchers in biomedical disciplines (Abdill & Blekhman, 2019; Fraser, Momeni et al., 2020; Fraser, Brierley et al., 2021; Fu & Hughey, 2019; Kirkham, Penfold et al., 2020; Sevryugina & Dicks, 2021). Scholarly communications in today’s digitalized academic sphere have thus been rapidly shifting from the conventional journal-centered publishing styles toward a new mode of communication via preprint servers and social media (e.g., blogging, Facebook, and Twitter), per Open Science practices (Berg, Bhalla et al., 2016; Fraser et al., 2021; Gentil-Beccot et al., 2010; Ginsparg, 2016; Shuai, Pepe, & Bollen, 2012; Thelwall, Haustein et al., 2013; Wang et al., 2020).

Facing this publishing paradigm shift, what constitutes the growth and obsolescence of scientific knowledge has also been changing and diversifying. In many research areas and subfields, nowadays, it is quite common that eprints posted on preprint servers or repositories are not yet or never to be published in journals indexed in the commercial bibliometric databases (Abdill & Blekhman, 2019; Larivière et al., 2014; Okamura, Yoda et al., 2020; Sevryugina & Dicks, 2021). Even if the preprints are to be published eventually in a database-indexed journal, the citations received during the preprint duration (in their preprint forms) may not be fully covered in major bibliometric databases, leading to bias in the citation analysis. In addition, there exists a field-dependent bias resulting from the database coverage (Waltman, 2016). For instance, computer science has a unique conference-centric publishing culture, in which researchers have a tendency to value top-tier conferences as a publication venue rather than high-impact journals (Feldman et al., 2018; Kim, 2019). Also, the main research outputs in humanities and social sciences studies have been books rather than journal papers. For these disciplines, the citational impact of a scientific work tends to be underrepresented in many commercial bibliometric databases. Consequently, research methods and measurement based on the conventional publishing systems may often not be desirable to grasp the whole volume and dynamics of scientific attention, or citations, in this new era of Open Science.

With the aim of obtaining a better understanding of the real dynamics underlying the citation network beyond the traditional journal-centered scholarly communication, we investigate the citation data of eprints posted on arXiv. The data contain the information on various types of eprints, regardless of which journal (or none) eventually published (or will publish) them. A disadvantage of using the arXiv data compared to the commercial databases would be the limited variety in the subject categories; for instance, arXiv’s research disciplines do not include biomedical disciplines. Still, the arXiv data cover various subject categories focusing on physics, mathematics, and computer sciences, and the long-term citation data (since 1991) provide a unique test bed for modeling and visualizing the distinct disciplinary patterns of scientific attention. Making full use of the advantages, we investigate two dimensions of citation dynamics: the “quantitative” dimension and the “temporal” dimension. The quantitative dimension regards the degree distribution in the citation network (Price, 1965, 1976). We consider both the power law model (Peterson, Pressé, & Dill, 2010; Price, 1976; Redner, 1998) and the lognormal model (Radicchi, Fortunato, & Castellano, 2008; Redner, 2005; Sheridan & Onodera, 2018), and discuss the extent to which each model explains the empirical data. We conclude that, by and large, the arXiv citation data obey the lognormal law. The other, temporal dimension concerns the time dependence of citation accumulation at the discipline-average level (Barnett et al., 1989; Glänzel & Schoepflin, 1995; Gross & Gross, 1927). The entire span of the growth and obsolescence pattern is modeled by a simple nonlinear function of time, which fits the empirical data remarkably well for all disciplines.

The revealed characteristics of the citation dynamics open up some interesting research directions; we discuss two directions in this paper. The first direction concerns the “fair” evaluation of scientific attention cast on individual papers. Although citation-based metrics have increasingly been applied in research evaluation and science policymaking, various sources of bias have limited their validity and utility (Aksnes et al., 2019; Lehmann & Lautrup, 2006; Seglen, 1998; Waltman, 2016). A particularly significant issue is the disciplinary difference, including the size of the research community, the typical time to publication, the average level of per-paper citations, the growth and obsolescence rates, and the degree to which the preprint (eprint) mode of communication is adopted⁷. The knowledge of citation dynamics revealed in this paper can mitigate some of these biases in both the quantitative and temporal dimensions, through a new index of citational attention called the γ-index, as will be demonstrated. The second direction is more theory oriented, which is an effort to reveal the underlying mathematical model that consistently explains the observed evolution pattern of citations, both quantity-wise and time-wise. Unlike the commonly accepted model of preferential attachment (Barabási & Albert, 1999; Barabási, Jeong et al., 2002; Newman, 2001), which generates networks with a power law, our stochastic model naturally reproduces the observed lognormal law in the quantitative dimension while also reproducing the observed evolution pattern (“jump–decay” plus “constant attention”) of the per-paper average citations in the temporal dimension. We hope this paper serves to provoke further discussions and investigations of the related issues, thereby shedding new light on the nature of collective attention dynamics on scientific knowledge.

We first describe the bibliometric data used throughout this study; see Appendix A in the Supplementary Materials for the data analysis and visualization platforms. Subsequently, we analyze the degree distribution in the arXiv citation network, providing evidence for the lognormal law in the quantitative dimension of the citation dynamics.

Our bibliometric analyses were based on a data set of 1,589,006 eprints posted on arXiv from its launch in 1991 until the end of 2019—hereafter referred to as “the arXiv eprints”—plus the data on their citations. This raw data set is the same as that used in Okamura, Yoda et al., (2020), which can be found in the Zenodo repository (Okamura & Koshiba, 2021). Here, eprints include preprints, conference proceedings, book chapters, data sets and commentary (i.e., all electronic material that has been posted on arXiv). The content and metadata of the arXiv eprints were retrieved from the arXiv API⁸ as of January 21, 2020, where the metadata included data about the eprint’s title, author, abstract, subject category, and arXiv ID, which is arXiv’s original eprint identifier. In addition, the associated citation data were derived from the Semantic Scholar API⁹ from January 24, 2020 to February 7, 2020, containing the citation information in and out of the arXiv eprints and their published versions (if applicable). Here, whether an eprint has been published in a journal or other means is assumed to be inferrable, albeit indirectly, from the status of the digital object identifier (DOI) assignment. It is also assumed that if an arXiv eprint received c_pre and c_pub citations until the data retrieval date (February 7, 2020) before and after it is assigned a DOI, respectively, then the citation count of this eprint is recorded in the Semantic Scholar data set as c_pre + c_pub. Both the arXiv API and the Semantic Scholar data sets contained the arXiv ID as metadata, which served as a key variable to merge the two data sets.

Our classification of research disciplines was based on that described in the arXiv.org (2021) website. There, the 153 arXiv subject categories are aggregated into several disciplines, of which we restrict our attention to the following six disciplines: Astrophysics (astro-ph), Computer Science (CS) (comp-sci), Condensed Matter Physics (cond-mat), High Energy Physics (HEP) (hep), Mathematics (Math) (math) and Other Physics (oth-phys), which collectively accounted for 98% of all the eprints¹⁰. Those eprints that are tagged to multiple arXiv disciplines (Supplementary Figures S3a, b) were counted independently for each discipline. Due to this overlapping feature, the final data set contained a cumulative total of 2,011,216 eprints. See Table 1 for the numbers of eprints by discipline (see Supplementary Table S1 for a detailed description). A notable difference was observed between the four physics-based disciplines and the nonphysics disciplines (CS and Math) in the distribution of the time elapsed without being assigned a DOI (Supplementary Figure S4), in line with the findings of previous studies (Aman, 2013; Larivière et al., 2014). Specifically, while the majority of physics-based eprints acquire a DOI within approximately 2 years and exhibit a steep slope down toward the third year, the majority of nonphysics-based eprints are never assigned a DOI (CS: 77%, Math: 69%) to the data retrieval date, exhibiting a heavy-tailed histogram. This is highly suggestive of the empirical fact that journal publication is not necessarily the venue of choice for researchers of CS and Math, and that they heavily rely on arXiv for tracking new research findings (Kim, 2019; Wang et al., 2020).

To investigate which (or both, or neither) of the models (y_PL and y_LN) explains the empirical distribution of the arXiv citation data better, we created the quantile plot of y_k against each model, using the subdata sets of eprints classified according to the year of first submission (1991–2019). The fitness of each model was checked by visual inspection of the linearity between y_k and ln(1 − q_k) (power law; Supplementary Figure S5), and between y_k and Φ⁻¹(q_k) (lognormal; Supplementary Figure S6). First, let us look into the results for the power law model; see Figure 2a for the quantile plot based on the year-2010 data set. An approximately linear relationship can be seen in the region −ln(1 − q) ≳ 4, or q ≳ 0.98 in terms of the quantile rank. However, this model does not fit well with the vast majority of the observations (i.e., below approximately the 98th percentile eprints). To cure this situation, we also considered a shifted power law model (Eom & Fortunato, 2011; Thelwall, 2016), P_sPL(c) ∼ ((a − 1)/θ)(c/θ + 1)^−a, by introducing a shift parameter θ > 0 (Supplementary Figure S7). Indeed, the approximately linear region could be expanded by fine-tuning the level of θ (see, e.g., the plot with θ = 10). However, without any justification for its dynamical origin, the shift parameter (θ) represented nothing more than an arbitrary threshold set by hand. Also, the value of θ was quite sensitive to the slope of the linear region. Consequently, the estimated value of the exponent (a) was crucially dependent on the choice of the arbitrary parameter.

By contrast, the lognormal model was shown to fit the empirical data remarkably well; see Figure 2b for the quantile plot based on the year-2010 data set. An approximately linear relationship between y_k and Φ⁻¹(q_k) is observed for the entire region of c_k > 0, without any arbitrary thresholds as in the shifted power law model case. Applying an ordinary least squares regression model to the nonzero-citation data, the two lognormal parameters (b, m) were estimated with a very high coefficient of determination (the adjusted R-squared > 0.99; Supplementary Table S3), with which the degree distribution was estimated as y ∼ 𝒩(⁠$bˆ$⁠, $mˆ$²). This feature was also true for each data set year and each discipline (Supplementary Figure S6). These results lead us to conclude that the arXiv citation distribution is better explained by the lognormal model than the power law model¹¹. Note that for both the power law and the lognormal models, the empirical data deviate from the law in the region 1 − q ≲ 𝒪(10⁻⁵) (i.e., exceptionally highly cited eprints).

Next, we move on to the temporal dimension of the citation dynamics. As the following argument applies equally to journal papers and eprints, we use the term paper instead of eprint for the moment.

There have been two approaches known in the literature to investigate the scientific literature growth and obsolescence. The first approach is called the retrospective (or synchronous, backwards-looking, citations from) approach (Barnett et al., 1989; Burrell, 2002; Glänzel, 2004; Larivière et al., 2008; Nakamoto, 1988; Pan et al., 2018; Price, 1965; Redner, 2005; Verstak et al., 2014; Yin & Wang, 2017; Zhang & Glänzel, 2017), which looks at the distribution of the age of papers cited by papers published in a given year. The second approach is called the prospective (or diachronous, forward-looking, citations to) approach (Bouabid, 2011; Bouabid & Larivière, 2013; Burrell, 2002; Glänzel, 2004; Nakamoto, 1988; Pan et al., 2018; Parolo et al., 2015; Redner, 2005; Wang et al., 2013; Yin & Wang, 2017), which looks at the distribution of citations acquired over time by papers published in a given year. Our methodology was based on the latter, prospective approach, investigating the citation history of papers until the data retrieval date. Also, our analyses here are conducted at the discipline-average level, in which the citation dynamics is driven by collective scientific attention. The discipline-average trajectory of citation accumulation—average citation history curve—is indicative of the typical growth and obsolescence patterns of papers associated with the research discipline.

As the number of papers increases, the contributions from this anomalous component, (iii), tend to be suppressed, and the average citation history curve will become approximately the regular part, (i) + (ii). However, cases of very highly cited papers, including some giant Sleeping Beauties, could still be impactful enough to deform the average citation history curve from the regular pattern. In fact, previous studies have found that highly cited publications tend to follow varied citation history curves (Redner, 2005; Wang et al., 2013). To mitigate such anomalous effects on our analysis of the average citation history curve, we set the threshold percentile of the citation distribution as the 99th percentile, and focus on the eprints below this threshold (hereafter referred to as the below-99th percentile eprints or the below-99th citation data)¹². The rest, the top 1% highest cited eprints, were removed from analyses, as the average citation history curve of such a highest cited cluster of eprints no longer follows the regular time-dependence of citation accumulation (Supplementary Figures S8a, b).

Figure 3 shows the time distribution of the empirical data, {(t_i, u_i)|i = 0, …, 20}, overlaid by the fitted regression curves based on Eq. 5 (i.e., û(t) = $Aˆ$f(t; $μˆ$⁠, $σˆ$⁠) + $Bˆ$g(t; $λˆ$⁠), 0 ≤ t ≤ 20), for each arXiv discipline and for the below-pth percentile eprints (p = 99, 95, 90, 75, 50). Here, the estimates of parameters are indicated by a “hat” symbol. The coefficient of determination (the adjusted R-squared) was higher than 0.99 for all disciplines. Note that, considering the extremely skewed distribution of citations (Figure 1), the vertical axis may be better thought of as shown by an arbitrary scale, which nevertheless is useful for a cross-discipline comparative analysis. A wide variety of differences can be seen among the fitted curves across both disciplines and percentile sections. For each discipline panel, the vertical gaps between the regression curves reflect the variance of the quantitative citation distribution (cf. Supplementary Figure S9). Regarding the percentile section, focusing on the below-99th citation data, Table 2 summarizes the estimated regression coefficients with the derivative metrics, and Figure 4 shows the predicted discipline-average citation history curves (see Supplementary Table S4 and Supplementary Figure S10 for the results regarding the lower percentile data). As can be seen, CS exhibits the highest peak in a short time interval, indicating the highest growth rate of citation counts; its peak value (û_p = 2.08) is more than double that of any other discipline. Also, Math exhibits the heaviest-tailed curve profile with the highest constant attention component (⁠$Bˆ$ = 0.452)¹⁴. These characteristics are in sharp contrast to the remaining four physics-based disciplines. Differences among the physics-based disciplines are also evident concerning the peak height, the mode year, and the degree of stretching in the time direction. For example, Astrophysics has the lowest peak value (û_p = 0.450) and the lowest constant attention (⁠$Bˆ$ = 0.158). Also, the jump–decay phase of HEP occurs in a much shorter time interval than the other physics-based disciplines. All these characteristics reflect different citation, publication, and communication practices intrinsic to each research discipline (Björk & Solomon, 2013; Schubert & Braun, 1996; Zitt, Ramanana-Rahary, & Bassecoulard, 2005).

Having obtained the average citation history curve with quantitative characteristics, we now conduct a cross-discipline comparative analysis of the obsolescence rate. To characterize the obsolescence phenomenon, previous studies commonly used measures such as “half-life” (which often also refers to the median citation age) (Burton & Kebler, 1960; Larivière et al., 2008; Parolo et al., 2015; Zhang & Glänzel, 2017), average citation age (Larivière et al., 2008; Redner, 2005; Zhang & Glänzel, 2017), “life-expectancy” (or “life-time”) (Bouabid, 2011; Bouabid & Larivière, 2013), and Price Index (Larivière et al., 2008; Zhang & Glänzel, 2017)¹⁵. An alternative way to conceptualize the obsolescence rate may be to use the magnitude of the down-slope (gradient) of the citation history curve after its peak, which involves both the quantitative and temporal dimensions. Whichever of the above indicators is used, it is subject to discipline-specific biases arising from the disciplinarily different publication practices, systems and processes (Björk & Solomon, 2013; Schubert & Braun, 1996; Zitt et al., 2005). For instance, regarding the quantitative bias, 10 citations received by a Life Science paper would not represent the same citational impact as that received by a Math paper (e.g., Schubert & Braun, 1996, Table 1; Okamura, 2019, Supplementary Table S2). This means that the drop in the raw number of yearly citations would not be a well-suited indicator of obsolescence. Also, regarding the temporal bias, what a year means to the academic life-cycle can be very different between disciplines. For instance, some disciplines have a high turnover rate of citation with a fast-paced research environment compared to other disciplines, as already evidenced in our arXiv data (Figure 4 and Supplementary Figure S10). Indeed, an investigation of the predicted cumulative citation history curves revealed that, on average, more than half of the citations throughout the life-to-date of an HEP eprint has already been received within the first 3 years since posted on arXiv, whereas the corresponding figure for a Math eprint is below 30%. Besides, publishing delay also varies across disciplines (Aman, 2013; Björk & Solomon, 2013). Therefore, a comparative analysis of the obsolescence rates can be conducted only after first gauging the relevant bibliometric coordinates against a quantitatively and temporally normalized reference standard.

It is intriguing to know how the obsolescence rate has changed over time (Evans, 2008; Larivière et al., 2008; Martín-Martín et al., 2016; Pan et al., 2018; Parolo et al., 2015; Verstak et al., 2014; Zhang & Glänzel, 2017). The findings from the previous studies have been inconclusive or inconsistent, as they use different sources of the citation data with different field categorizations and time windows. We revisited this issue by using the arXiv data with the proposed internal obsolescence rate of Eq. 10. We constructed a series of 10 consecutive subdata sets from the entire arXiv data set such that the first, the second, and the nth subdata set respectively contain the eprints posted on arXiv before and including the year 2019, 2018, and 2020 − n, with n running from 1 to 10. Subsequently, the regression model of Eq. 5 was applied to each subdata set. The estimated regression coefficients were used to obtain δ₁ and δ₂ for each data set year (2010–2019), with which the internal obsolescence rate (𝒮) is obtained by Eq. 10. Figure 5 shows the resultant trends in 𝒮 by discipline. During the past decade, the most remarkable increase is observed for CS, for which 𝒮 has increased from 0.383 (2010; the lowest of all) to 1.37 (2019; close to the highest). It implies that for CS researchers today, knowledge at the research front has increasingly become obsolete with the exponential growth in the CS eprints (Supplementary Figure S1). Condensed Matter Physics and Other Physics also follow an increasing trend with relatively similar levels of 𝒮. Math shows a less steep increase from 0.440 (2010) to 0.749 (2019), remaining at the lowest level since 2012. Astrophysics stays relatively flat around slightly above 𝒮 = 1, suggesting that the speed of obsolescence does not vary much over time. By contrast, HEP exhibits a decreasing trend, for which 𝒮 has steadily dropped from 1.85 (2010) to 1.45 (2019). Still, HEP kept at the highest level throughout the period, suggesting a highly competitive situation among researchers.

Based on the revealed knowledge of the quantitative and temporal dimensions of citation dynamics, this section extends our results in two directions. First, we develop a new citation-based index that can be used to compare the citational impact of papers published in different disciplines at different times. Second, we provide a mathematical (stochastic) model of citation dynamics that reproduces both the quantitative and temporal patterns observed in the previous sections. The first part will be of interest to practitioners of research evaluation and science policymakers who wish to improve the informed peer-review process, while the second part is mainly for interested researchers working on the intersection of scientometrics and mathematical physics/economics.

Previous studies have developed a number of citation-based indices to quantify the research impact of scientific publications. These indices are often “field-normalized,” reflecting the fact that the average citations per paper vary widely between fields (Schubert & Braun, 1996; Waltman, 2016; Zitt et al., 2005). However, the normalization scales used there have mostly been based on the bibliometric data on journal papers. Considering that scholarly communications, including citations, have already been or are being heavily dependent on eprint (preprint) systems in many research disciplines (see Section 1), the normalization scales are necessarily biased in both the quantitative and temporal dimensions without consideration of nonjournal-based citations¹⁷. In addition, an appropriate citation window varies between disciplines due to the different bibliodynamic clock discussed earlier. Towards a less biased evaluation and comparative analysis of the scientific attention, we propose a new, quantitatively and temporally normalized citation index based on the revealed characteristics of the average citation history curve.

For easy reference, Table 3 presents a “ready reckoner” for the γ-index precalculated based on the below-99th citation data¹⁸. Due to the monotonically increasing property of the cumulative density function, H(T), the γ-index for each citation count monotonically decreases as the elapsed years (T) after being posted on arXiv becomes longer. Note that we have omitted the negative values of γ from the table because, as discussed, the γ-index for the below-99th citation data could be conservatively valid for γ ≳ 0. As an illustration, 10 citations at T = 2 in the CS discipline (γ = 1.73) is comparable to 50 citations at T = 6 in the same discipline (γ = 1.71). As another illustration, 10 citations at T = 3 in the HEP discipline (γ = 1.68) is seen as more impactful than the same number of citations at the same elapsed time in the CS discipline (γ = 0.96). Also, 50 citations at T = 9 in the Astrophysics discipline (γ = 2.87) is comparable to the same number of citations at T = 4 in the HEP discipline (γ = 2.91), or 100 citations at T = 4 in the CS discipline (γ = 2.85).

We have demonstrated how the idea of the quantitatively and temporally normalized index of the citational impact works by applying the γ-index to the real world (arXiv) data. However, it should be kept in mind that this index would still produce a significantly biased result if applied without an appropriate classification of research disciplines or subfields. For instance, even within the same discipline of HEP, papers on theoretical HEP and experimental HEP could show quite distinct characteristics regarding the typical citation history profile. The same situation can also occur for the Math discipline (e.g., between papers on pure mathematics and applied mathematics). We also note that, depending on the research discipline, the γ-index may not be a beneficial indicator of the citational impact for particularly “young” papers produced within a couple of years (cf. Wang, 2013). With all these caveats, the γ-index (or the γ*-index) can offer a useful, albeit approximate and imperfect, measure to evaluate and compare the citational attention on scientific publications across disciplines and time periods.

So far, we have investigated the macroscopic, discipline-average picture of citation distribution and evolution pattern. Indeed, the regression model of Eq. 5 achieved a remarkable fit to the empirical arXiv data. However, the underlying microscopic mechanism that governs the quantitative and temporal dimensions of citation dynamics at the level of individual papers, consistent with the revealed macroscopic picture, is yet to be uncovered. To address this issue, we provide mathematical modeling of the citation dynamics through a stochastic differential equation (SDE). A key observation is that the lognormal law observed for the degree distribution is related to geometric Brownian motion. This stochastic model plays a central role in many mathematical models, most notably in Black–Scholes option pricing (Black & Scholes, 1973; Merton, 1973) known in the financial engineering literature. Recall that the Black–Scholes model involves the deterministic and stochastic parts, reminding us of some structural similarity to the citation dynamics. In the real world of scholarly communication via citations, when an author of scientific papers determines which literature to cite, the author’s attitudes and behaviors are often influenced—whether consciously or unconsciously—by the recent citation performance of the literature. However, that does not solely determine the paper’s reference list; the author’s finite cognitive capacity, resource limitations, time constraints, and pursuit of novelty introduce some random, nondeterministic factors in the citation phenomenon. Other random events would include the discovery of unexpected relationships to other works and the subsequent development of research. These observations lead us to speculate that the citation dynamics is also described by a deterministic plus stochastic model, as in the Black–Scholes model. Then, intuition suggests that the deterministic part is encoded in the average citation history function, whereas the stochastic part introduces the statistical variability in the trajectory distribution of citations. It will be an extended Black–Scholes model, where the “drift” and “volatility” parameters are now time dependent, encoding the nontrivial temporal characteristics of the citation dynamics. Below, we show that it is indeed the case.

This study investigated the citations of more than 1.5 million eprints on arXiv to model and analyze the quantitative and temporal characteristics of collective attention on scientific knowledge. The eprints included not only papers published in journals but also their preprint versions and other open electronic materials. The developed conceptual and technical framework allowed us to explore the citation dynamics beyond the traditional journal-centered scholarly communication, making the previously invisible visible. By applying a nonlinear regression model to the long-term discipline-average citation data, some interesting facts were revealed regarding how quickly and how many, on average, an eprint in each discipline acquires citations and how quickly it becomes obsolete. Apart from very highly cited papers, it was uncovered that while the jump–decay plus constant attention patterns were consistent across disciplines, the quantitative characteristics such as the average peak height, the average time to reach the peak, and the curve’s skewness varied among the disciplines. In particular, CS exhibited the steepest gradient with the highest peak in the growth phase, while Math exhibited the heaviest-tailed curve profile in the obsolescence phase. The regression results were used to analyze the trends in the obsolescence rate, in both the quantitative and temporal dimensions. The temporal obsolescence rate, or the internal obsolescence rate, was quantified by 𝒮 = δ₁/δ₂, where δ₁ and δ₂ represented the typical time interval of the growth phase and the obsolescence phase, respectively, identified in the average citation history curve. The quantity-wise obsolescence rate, or the retention rate, was quantified by 𝓡 = $Bˆ$/û_p, where $Bˆ$ and û_p represented the asymptotic value and the peak value of the average citation history curve, respectively. Both obsolescence metrics exhibited varied characteristics across disciplines, reflecting the different publication and citation practices. Further, the revealed characteristics of the discipline-average citation history curves were used to develop a new citation index, called the γ-index, which was normalized across disciplines and time periods. When applied to the arXiv citation data, the distribution of γ was fairly close to the standard normal distribution (except its left tail). This fact suggested that the γ-index could be used as an improved, less biased alternative to those citation-based impact measures widely used in the current academic or government practice. Moreover, a stochastic modeling of the citation dynamics is presented, which successfully reproduced both the observed lognormal law for the cumulative citation distribution—the quantitative dimension—and the observed time behavior of the average citation history curve—the temporal dimension—in a unified formalism.

The new conceptual and methodological framework developed in this paper to explore the dynamics of collective attention on science would be of interest to a wide range of research communities, including academic researchers, practitioners, and policymakers. However, as with any bibliometric research, this study also faced various limitations that may have impacted the general validity of the findings. First, papers not being posted on arXiv, including those directly submitted to journals, were not considered due to the data availability. Although a growing number of research outputs in the fields of physical, mathematical, and computer sciences have relied on arXiv (Supplementary Figures S1a, b), those citations outside the current arXiv data could have impacted some quantitative aspects of the findings. Second, the results should be assessed with caution, as they are likely to be highly dependent on the research discipline classification scheme (Schubert & Braun, 1996; Waltman, 2016; Zitt et al., 2005). Although this study relied on the same classification as that used on the arXiv.org (2021) website, different specifications based on different subject areas or specialities could also have been applied, leading to quantitatively different implications on the citation dynamics. The validity and utility of the γ-index also wholly depends on the discipline classification scheme. Third, the citation data would have been incomplete due to the limitation of the external source and its connectivity to the arXiv system. There have not been consistent tools to measure citations in the sphere of eprints, and arXiv’s citation-tracking algorithm may also be imperfect. If a significant number of papers or related citations have not been correctly recorded in the Semantic Scholar API data, then the quantitative results could have been different from—albeit qualitatively similar to—the findings of this paper. Fourth, and finally, as also emphasized in the discussion on the γ-index, it should be noted that there are inherent limitations in using citation-based methods to evaluate the research impact. Citation is not necessarily a measure of the absolute quality or value of scientific works; its primary criterion is utility in research (Garfield, 1979). Therefore, our quantitative approaches must not solely be used to evaluate individual research outcomes and must always be combined with expert judgements or trustworthy peer reviews.

There are many directions in which this work may be extended or applied. For instance, the quantitative information on the discipline-average citation history curve can be used to adjust the citation window to calculate various popular citation-based metrics, including the journal impact factor (Garfield, 1972, 2006) and the h-index (Hirsch, 2005), alleviating the problem of biased assessment. Such knowledge will also be helpful to better detect “atypical” citation patterns for individual papers, including the cases of multiple humps and Sleeping Beauties (He et al., 2018; Ke et al., 2015; van Raan, 2004). It would also be interesting to investigate the determinants of various bibliometric indicators considered in this paper, including the internal obsolescence rate, the retention/inflation rate, and the γ-index (γ*-index). Our compiled arXiv data set contained the information of the arXiv ID, which can be used to identify various information about each eprint, including title, author names, affiliations and the number of authors, institutions to which the authors belong, researcher ID, journal titles, and DOI (if applicable). The researcher ID can also be used to draw information about the authors’ research records and various achievements. By integrating these information resources into a more comprehensive data set, we will be able to conduct causal inference with both quantitative and qualitative methods for a deeper understanding of the citation dynamics. In so doing, it is important to extend the research area beyond the arXiv disciplines, including the areas of not only other natural sciences and biomedical sciences but also humanities and social sciences. The yet-to-be-seen landscape of interdisciplinary research driven by open eprints would be different from that seen through the lens of journal papers (e.g., Okamura, 2019). Finally, it would be interesting to investigate further how the networks of citations (Barabási et al., 2002; Golosovsky & Solomon, 2017; Pan et al., 2018; Wang et al., 2013), or more broadly, collective attention (Lorenz-Spreen et al., 2019) is formed and disseminated through time and social space, based on the framework developed in this paper. The increasing availability of large-scale data will enable us to explore new frontiers in scientometrics.

We close this paper with a remark on what “citation” would represent in the coming new era of Open Science. As ever more scientific knowledge is posted, published and made available to the public through various open digital materials and platforms, those who “cite” the source of knowledge will be diversified across the stakeholders of science and society. Simultaneously, the reasons and ways for citation will also be diversified. The scientific landscape and the scenery, which researchers have long seen over the shoulder of giants, will necessarily change as Open Science provides a new way to see the world from high above the data clouds. It will enable people to access and experience collaborative knowledge production and consumption processes through open, transparent and shared platforms. Accordingly, what the term “citation”—or scientific attention, attitudes, and behavior—has meant and means today in scientific communities will not be the same tomorrow. A deeper understanding of the who, what, why, and how of “citation” in a much broader context will be required to build a balanced and sustainable relationship between science and the future society.

The author would like to thank Ryo Suzuki and two anonymous reviewers for their valuable comments on the manuscript. The views and conclusions contained herein are those of the author and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of any of the organizations to which the author is affiliated.

The author has no competing interests.

The author did not receive any funding for this research.

The data sets generated and/or analyzed during this study can be found in the Zenodo repository at https://doi.org/10.5281/zenodo.5803962.