Recently, Wu, Wang, and Evans (2019) proposed a new family of indicators, which measure whether a scientific publication is disruptive to a field or tradition of research. Such disruptive influences are characterized by citations to a focal paper, but not its cited references. In this study, we are interested in the question of convergent validity. We used external criteria of newness to examine convergent validity: In the postpublication peer review system of F1000Prime, experts assess papers whether the reported research fulfills these criteria (e.g., reports new findings). This study is based on 120,179 papers from F1000Prime published between 2000 and 2016. In the first part of the study we discuss the indicators. Based on the insights from the discussion, we propose alternate variants of disruption indicators. In the second part, we investigate the convergent validity of the indicators and the (possibly) improved variants. Although the results of a factor analysis show that the different variants measure similar dimensions, the results of regression analyses reveal that one variant ( DI 5 ) performs slightly better than the others.

Cocitation measurements can reveal the extent to which a concept representing a novel combination of existing ideas evolves towards a specialty. The strength of cocitation is represented by its frequency, which accumulates over time. Of interest is whether underlying features associated with the strength of cocitation can be identified. We use the proximal citation network for a given pair of articles ( x , y ) to compute θ , an a priori estimate of the probability of cocitation between x and y, prior to their first cocitation. Thus, low values for θ reflect pairs of articles for which cocitation is presumed less likely. We observe that cocitation frequencies are a composite of power-law and lognormal distributions, and that very high cocitation frequencies are more likely to be composed of pairs with low values of θ , reflecting the impact of a novel combination of ideas. Furthermore, we note that the occurrence of a direct citation between two members of a cocited pair increases with cocitation frequency. Finally, we identify cases of frequently cocited publications that accumulate cocitations after an extended period of dormancy.

Citation analysis of the scientific literature has been used to study and define disciplinary boundaries, to trace the dissemination of knowledge, and to estimate impact. Co-citation, the frequency with which pairs of publications are cited, provides insight into how documents relate to each other and across fields. Co-citation analysis has been used to characterize combinations of prior work as conventional or innovative and to derive features of highly cited publications. Given the organization of science into disciplines, a key question is the sensitivity of such analyses to frame of reference. Our study examines this question using semantically themed citation networks. We observe that trends reported to be true across the scientific literature do not hold for focused citation networks, and we conclude that inferring novelty using co-citation analysis and random graph models benefits from disciplinary context.