A multidimensional perspective on the citation impact of scientific publications

The citation impact of scientific publications is usually seen as a one-dimensional concept. We introduce a three-dimensional perspective on the citation impact of publications. In addition to the level of citation impact, quantified by the number of citations received by a publication, we also conceptualize and operationalize the depth and dependence of citation impact. This enables us to make a distinction between publications that have a deep impact concentrated in one specific field of research and publications that have a broad impact scattered over different research fields. It also allows us to distinguish between publications that are strongly dependent on earlier work and publications that make a more independent scientific contribution. We present a large-scale empirical analysis of the level, depth, and dependence of the citation impact of publications. In addition, we report a case study focusing on publications in the field of scientometrics. Our three-dimensional citation impact framework provides a more detailed understanding of the citation impact of a publication than a traditional one-dimensional perspective.


Introduction
Measuring the citation impact of scientific publications is an important topic in bibliometric and scientometric research.Many different citation impact indicators, calculated based on the citations received by a publication, have been proposed (Waltman, 2016).The most basic citation impact indicator is the raw citation count of a publication.Although this indicator is easy to calculate, it has often been criticized and many alternatives have been proposed.
Normalization is a commonly used approach to construct more sophisticated citation impact indicators (Waltman & Van Eck, 2018).Several attributes of a publication have been used for normalization, in particular a publication's scientific field and its age (e.g., Radicchi, Fortunato, & Castellano, 2008;Waltman, Van Eck, Van Leeuwen, Visser, & Van Raan, 2011).Another prominent line of research on citation impact indicators focuses on PageRank-inspired approaches (Waltman & Yan, 2014).For instance, Chen, Xie, Maslov, and Redner (2007) proposed a PageRank approach for quantifying the citation impact of a publication.This approach favors older publications over more recent ones.A correction for the age of a publication was introduced by Walker, Xie, Yan, and Maslov (2007).Attributes derived from the full text of citing publications, such as the number of times a publication is cited in the full text of a citing publication and the location in the full text where the publication is cited, have also been suggested as useful features for constructing citation impact indicators (e.g., Ding, Liu, Guo, & Cronin, 2013;Wan & Liu, 2014;Zhu, Turney, Lemire, & Vellino, 2015).
The approaches discussed above still regard the citation impact of a publication as a one-dimensional concept.In this paper, we propose a multidimensional perspective on the citation impact of a publication.We argue that, in addition to the level of citation impact, there are other interesting aspects of the citation impact of a publication that can be derived from the citations received by a publication.
To illustrate this, consider two publications,  and .As shown in Figure 1, these publications have both received five citations.If we just count the citations received by the two publications, the publications will be considered to have the same citation impact.However, the publications citing  also cite each other and therefore seem to be closely related, while the publications citing  do not cite each other and therefore seem to be quite unrelated from each other.Hence,  and  have the same level of citation impact, but  seems to have a relatively deep citation impact in a narrow research area, while  seems to have a broader citation impact.To distinguish between the different ways in which  and  have an impact on other publications, we propose an approach for quantifying the depth of the citation impact of a publication.We are also interested in the dependence of a publication's citation impact on earlier publications.For instance, consider two publications,  and .As shown in Figure 2, these publications have both received five citations, and they both have three references.All publications citing  also cite each of 's references, while the publications citing  do not cite 's references.Hence, the citation impact of  seems to depend strongly on earlier publications, namely the ones cited by .It is likely that  is a follow-up study of these earlier publications.On the other hand,  seems to have a much more independent citation impact, since publications citing  do not cite the references of .
In this paper, we propose to conceptualize and operationalize the citation impact of a publication in a three-dimensional framework that focuses on the level, depth, and dependence of citation impact.In a traditional one-dimensional perspective on citation impact, only the level of citation impact is considered.Beyond the level of citation impact, no insights are obtained into the way in which a publication has an impact on other publications.By introducing the concepts of depth and dependence, our proposed framework aims to offer a more detailed understanding of the citation impact of a publication. of the references of , which is not the case for publications citing .Therefore  has a citation impact that is strongly dependent on earlier publications, while  has an independent citation impact.
The idea of analyzing citation relations between publications that cite a focal publication is not new.Clough, Gollings, Loach, and Evans (2015) compared the number of citations given to a publication in a citation network with the number of citations given to the same publication in the transitive reduction of the citation network.According to Clough et al., the transitive reduction can be used to get "an indication that results in a paper were used across a wide number of fields".Huang, Bu, Ding, andLu (2018a, 2018b) analyzed so-called citing cascades, defined as the citation network of a focal publication and its citing publications.In particular, they studied citation relations between citing publications.The citation impact framework proposed in the current paper partly builds on the ideas explored by Huang et al.The notion of dependence introduced in our citation impact framework is also related to the concepts of development and disruption recently proposed by Wu, Wang, and Evans (in press).
The organization of this paper is as follows.In Section 2, we introduce our threedimensional framework for characterizing the citation impact of publications.In Section 3, we present an empirical analysis based on our proposed framework.
Finally, in Section 4, we provide a discussion of our framework and we summarize our conclusions.

Level, depth, and dependence of the citation impact of a publication
In this section, we present our three-dimensional framework for characterizing the citation impact of publications.We first discuss the level of citation impact, followed by the depth of citation impact and finally the dependence of citation impact.
Throughout this section, we focus on a publication  that has  references and that has been cited by  other publications, denoted by  1 ,  2 , … ,   .

Level of citation impact
The level of citation impact of a publication reflects how much impact the publication has had on other publications.We operationalize this by the number of citations a publication has received.Hence, the level of citation impact of publication , denoted by level  , equals level  = . (1) The larger the number of citations received by a publication, the higher the level of citation impact of the publication.The level of citation impact coincides with the traditional way in which the citation impact of a publication is conceptualized and operationalized.

Depth of citation impact
To understand the notion of the depth of the citation impact of a publication, we consider an example involving two publications,  and .In this example, it is clear that  and  have an impact on other publications in very different ways.We consider  to have a deep citation impact, while we consider  to have a broad citation impact.Hence, if the impact of a publication is strongly concentrated within a single research field, the publication has a deep citation impact.
On the other hand, if the impact of a publication is scattered over many different research fields, the publication has a broad citation impact.We treat depth and breadth of citation impact as opposite concepts.Consequently, a high depth implies a low breadth, and the other way around.In our operationalization, we focus on the depth of citation impact.
The depth of the citation impact of publication , denoted by depth  , is given by (2) where  , denotes the number of co-citation links (Small, 1973) between  and   , that is, the number of publications that cite both  and   .In other words, the depth of the citation impact of publication  equals the average number of co-citation links between publications citing  and  itself.For instance, in Figure 1, the depth of the citation impact of  and  equals 10 / 5 = 2 and 0 / 5 = 0, respectively.This means that  has a deep citation impact, while  has a broad citation impact.
Hence, the upper bound of depth  is determined by the number of citations  has received, suggesting there may be a positive correlation between depth  and level  .If depth  = 0, there are no citation relations between publications that cite .Publications citing  then do not seem to be related to each other and  has a broad citation impact.On the other hand, if , all publications that cite  are connected to each other by citation relations.In other words, publications citing  are strongly related and  has a deep citation impact.We note that depth  is undefined if  has not been cited (i.e.,  = 0).
We do not intend to make a normative judgment by quantifying the depth of the citation impact of a publication.From our point of view, a deep citation impact is not necessarily 'better' than a broad citation impact, or the other way around.However, we do believe that the depth vs. breadth distinction is useful to get a more detailed understanding of the way in which a publication has an impact on other publications.

Dependence of citation impact
Finally, we introduce the notion of the dependence of the citation impact of a publication.Publications may have a similar level and a similar depth of citation impact, but nevertheless there may be an important difference in how they have an impact on other publications.Some publications may have an impact by building on earlier publications and by contributing new scientific knowledge in a cumulative way.It is likely that these publications will usually be cited together with the publications on which they build.We consider the citation impact of these publications to have a high dependence.Other publications may have an impact without relying strongly on earlier publications.These publications may introduce new ideas that have been developed relatively independently from earlier literature.
These publications will not be cited together with other publications on which they have a strong dependence.We consider these publications to have a relatively independent citation impact.
Our operationalization of the dependence of the citation impact of a publication mirrors the operationalization of the depth of citation impact provided above, with co-citation links being replaced by bibliographic coupling links.The dependence of the citation impact of publication , denoted by dependence  , is given by (3) where  , denotes the number of bibliographic coupling links (Kessler, 1963) between  and   , that is, the number of references that  and   have in common.In other words, the dependence of the citation impact of  equals the average number of bibliographic coupling links between publications citing  and  itself.For instance, in Figure 2, the dependence of the citation impact of  and  equals 15 / 5 = 3 and 0 / 5 = 0, respectively.
Eq. ( 3) implies that dependence  ∈ [0, ].Hence, the upper bound of dependence  is given by the number of references of .This suggests that publications with longer reference lists, such as review articles, may tend to have a higher dependence.If dependence  = 0, none of the publications citing  cites any of the references of .There then seem to be no common dependencies of  and publications citing  on earlier literature, and  is therefore considered to have an independent citation impact.On the other hand, if dependence  = , each of the publications citing  also cites all references of .This indicates that the citation impact of  depends strongly on the references of .We note that dependence  is undefined if  has not been cited (i.e.,  = 0).

Data
Our empirical analysis was carried out using data extracted from the in-house version of the Web of Science (WoS) database available at the Centre for Science and Technology Studies (CWTS) at Leiden University.We made use of the Science Citation Index Expanded, the Social Sciences Citation Index, and the Arts & Humanities Citation Index.The data covers nearly 50 million publications that appeared between 1980 and 2017 and over 720 million citation relations between these publications.The analysis focuses on highly cited publications in the period 2000-2017.We defined a highly cited publication as a publication that has received at least 100 citations.We did not impose any document type restrictions.In total, 637,237 highly cited publications in the period 2000-2017 were identified.We calculated the level, depth, and dependence of the citation impact of these publications.The calculation of dependence required the identification of bibliographic coupling links.A bibliographic coupling link was identified between two publications if there is a third publication that is included in our data and that is cited by the other two publications.If two publications both cite the same publication but this publication is not included in our data, this did not result in the identification of a bibliographic coupling link. 1

Overview
Figure 3 shows the cumulative distribution functions (CDFs) of the level, depth, and dependence of the citation impact of our 637,237 highly cited publications.The well-known skewness of citation distributions is reflected in the CDF of the level of citation impact.The CDFs of the depth and dependence of citation impact are surprisingly similar.The mean and median depth equal 2.8 and 2.4, respectively.The maximum depth equals 25.3. 2 The mean and median dependence equal 2.7 and 2.3, respectively, while the maximum dependence equals 29.5. 3able 1 reports the Pearson and Spearman correlations between the level, depth, and dependence of the citation impact of our highly cited publications.The Pearson and Spearman correlations yield similar results.As expected (see Subsection 2.2), there is a positive correlation between level and depth, although this correlation is weak.Depth and dependence are also positively correlated.Despite the positive 1 This is the reason why our analysis focuses on highly cited publications in the period 2000-2017 and why highly cited publications in the period 1980-1999 are not considered.The calculation of dependence for older publications is affected by the fact that many references in these publications point to literature that appeared before 1980 and that is not included in our data.
10 correlations between level and depth and between depth and dependence, there is a weak negative correlation between level and dependence. 4 11 different values of depth.The results are in agreement with the correlations reported in Table 1.On average, the higher the level of citation impact, the higher the depth and the lower the dependence of citation impact.In addition, the higher the depth of citation impact, the higher the dependence.Figure 4 clearly shows how the most highly cited publications are characterized by a high depth and a low dependence.
Overall, however, depth and dependence are positively correlated, and the publications with the highest depth therefore also tend to have a high dependence.We speculate that our findings may be explained by distinguishing between on the one hand publications contributing to research areas that are of a strongly cumulative nature and on the other hand publications making a more independent scientific contribution.In general, the former publications can be expected to have a relatively high depth and dependence, while the latter publications can be expected to have a lower depth and dependence.This would explain the positive correlation between depth and dependence.Cumulative research areas may start from highly influential 'breakthrough' publications.These publications have a high level of citation impact and a high depth.However, because they are at the start of a new research area, they can be expected to have a low dependence.This would explain why the most highly cited publications tend to have a high depth and a low dependence.Clusters are non-overlapping.Each publication belongs to only one cluster.The 4,047 clusters were grouped into the following five broad scientific disciplines: • Biomedical and health sciences (BHS) • Life and earth sciences (LES) • Mathematics and computer science (MCS) • Physical sciences and engineering (PSE) • Social sciences and humanities (SSH) We now present a comparison of these five disciplines in terms of the level, depth, and dependence of the citation impact of publications.We again focus on highly cited publications, defined as publications that have received at least 100 citations.Table 2 reports for each of the five disciplines the number of highly cited publications and the mean, median, and maximum level, depth, and dependence of the citation impact of these publications. 5The average level of citation impact is quite similar in the different disciplines.For the depth and dependence of citation impact, more substantial disciplinary differences can be observed.On average, the highest values of both depth and dependence can be found in BHS, LES, and PSE.This may reflect the cumulative nature of research in these disciplines.MCS and SSH are characterized by lower values of depth and dependence, suggesting that scientific contributions are more independent from each other in these disciplines.Especially the low values of dependence in MCS are noteworthy.However, the low values of dependence in MCS and, to a lesser extent, SSH may partly be an artifact of our data.
When two publications both cite the same third publication, this is counted as a bibliographic coupling link only if the third publication is included in our data.
Especially in the case of MCS and SSH, this is not always the case, since our data does not include conference proceedings and books, which play an important role in MCS and SSH, Figure 5. Cumulative distribution functions per discipline of the level, depth, and dependence of the citation impact of highly cited publications.
Figure 5 shows for each of our five disciplines the CDFs of the level, depth, and dependence of the citation impact of highly cited publications.The results are in line with the statistics presented in Table 2.The distribution of the level of citation impact almost coincides for the five disciplines.The values of depth and dependence tend to be substantially lower in MCS and SSH than in BHS, LES, and PSE.

Case study of the field of scientometrics
To provide a more detailed demonstration of our three-dimensional framework for characterizing the citation impact of publications, we now present a case study in which the framework is applied to publications in the field of scientometrics.As explained in Subsection 3.3, 4,047 clusters of publications were obtained using an algorithmic methodology.One of these clusters can be considered to represent the field of scientometrics.We selected all 182 highly cited publications (i.e., publications with at least 100 citations) in this cluster.For these publications, we calculated the level, depth, and dependence of their citation impact.
Figure 6 presents the depth and dependence of the 182 publications in a scatter plot.The dashed vertical and horizontal lines indicate the median depth (1.71) and median dependence (1.24), respectively.Based on the dashed lines, four quadrants were obtained.In each of these quadrants, we selected a publication for a more detailed analysis.Hence, we selected a publication with a low depth and a low dependence (referred to as P1), a publication with a high depth and a low dependence (P2), a publication with a low depth and a high dependence (P3), and a publication with a high depth and a high dependence (P4).We selected publications with which we are sufficiently familiar ourselves, so that we are able to offer a detailed interpretation of the citation impact of the selected publications.Table 3 lists the four selected publications and reports the level, depth, and dependence of their citation impact and their number of references.In addition, Figure 7 shows for each of the selected publications the distribution of the number of co-citation links and the number of bibliographic coupling links between the selected publication and the publications citing the selected publication.Publications citing P1 therefore tend to refer only sparsely to other publications on bibliometric visualization.This is reflected by the relatively low depth and dependence of P1.The depth of P1 equals 1.19, which indicates that a publication citing P1 on average has 1.19 co-citation links with P1.In other words, when a publication cites P1, there will on average be 1.19 publications in which the publication is cited together with P1.This means that publications citing P1 are only weakly connected to each other by citation relations.It shows that P1 has a broad but not so deep citation impact.P1 has a dependence of 1.07, indicating that the average number of bibliographic coupling links between each of P1's citing publications and P1 itself is 1.07.Hence, when a publication cites P1, there will on average be 1.07 publications that are cited both by this publication and by P1. Figure 7 shows that there are a limited number of publications citing P1 that have a more substantial number of co-citation links or bibliographic coupling links with P1.Unlike most publications citing P1, these are likely to be publications that contribute to the methodological literature on bibliometric visualization.
Table 3.The four selected publications, the level, depth, and dependence of their citation impact, and their number of references.research, but that it can be considered a foundational publication in this subfield.
However, it is important to be aware that P2 has only a very limited number of references (see Table 3), which means that almost by necessity its dependence will be low.The small number of references of P2 may be seen as additional evidence of the foundational role of this publication, but alternatively it could also be argued to reflect a lack of generosity in the referencing behavior of the author of P2.P3, it will on average also cite 8.26 references of P3.As can be seen in Figure 7, some publications citing P3 even cite more than 20 of P3's references.The high dependence of P3 indicates that P3 builds on a large body of literature and that the citation impact of P3 is strongly dependent on this literature.This reflects that, as a review article, P3 does not make an original scientific contribution.It is sometimes suggested that researchers have the tendency to cite review articles instead of citing the underlying original works, but the high dependence of P3 shows that this is not the case for P3.
The depth of P3 equals 1.34, which is just below the median depth of the 182 highly cited publications in the field of scientometrics.Publications that cite P3 on average have 1.34 co-citation links with P3, indicating that publications citing P3 are only relatively weakly connected to each other by citation relations.This may be due to the gradual decline in the interest of the scientometric community in the h-index.It also shows that P3 has not developed into a canonical reference for publications dealing with the h-index.This may partly be explained by the fact that around 2010 a number of review articles about the h-index were published more or less at the same time.
Publication P4 is about the extraction of macroscopic information from Web links.
This publication deals with a topic in field of webometrics, which partly overlaps with the field of scientometrics.P4 was published in 2001.It has received 107 citations in our data.As can be seen in Figure 6, P4 is a rather unique publication in the scientometric literature, because it combines a high depth (6.75) with a high dependence (5.68).This means that publications citing P4 have lots of citation relations both with each other and with the references of P4.As can be seen in Table 3, the number of references of P4 is not exceptionally large, making P4's high dependence even more noteworthy.The high depth of P4 suggests that P4 makes an important contribution to a relatively narrow but densely connected area of research.
On the other hand, the high dependence of P4 seems to indicate that P4 should not be regarded as a pioneering publication.The citation impact of P4 is strongly dependent on earlier publications.Hence, P4 can be considered to make an important incremental contribution but not a highly innovative one.

Discussion and conclusion
We propose a three-dimensional framework for characterizing the citation impact of scientific publications.The proposed framework makes a distinction between the level, depth, and dependence of the citation impact of a publication.The level of citation impact is operationalized by the number of citations a publication has received.The depth of the citation impact of a publication is quantified by calculating the average number of co-citation links between the publication and its citing publications.The more strongly a publication's citing publications are connected to each other by citation relations, the higher the depth of the citation impact of the publication.The dependence of the citation impact of a publication is quantified by calculating the average number of bibliographic coupling links between the publication and its citing publications.The more a publication and its citing publications refer to the same earlier publications, the higher the dependence of the citation impact of the publication.
In a traditional one-dimensional perspective on citation impact, the number of citations received by a publication is used as an indicator of the impact of the publication on later publications.Our three-dimensional framework offers a more detailed understanding of the citation impact of a publication.It enables us to make a distinction between publications that have a deep impact concentrated in one specific field of research and publications that have a broad impact scattered over different research fields.It also allows us to distinguish between publications that are strongly dependent on earlier work and publications that make a more independent scientific contribution.
In the field of scientometrics, we find that the article in which the h-index was introduced has a high depth and a low dependence.This reflects the role of this article as the starting point of a new subfield of research within the field of scientometrics.
On the other hand, a review article on the h-index has a high dependence, which shows the strong reliance of this article on earlier works.A high dependence can be expected to be a typical feature of review articles.The article in which the VOSviewer software was introduced has a low depth, reflecting its broad but not so deep impact.Finally, an article in the field of webometrics has a high depth and a high dependence, indicating that this article contributes to a strongly cumulative research area, but that it does not play a pioneering role in this area.
There are various directions for future research.In future work, our citation impact framework could be analyzed in more detail, for instance by studying its mathematical properties and by carrying out additional case studies.Also, the effect of author self-citations in our citation impact framework could be analyzed.In addition, different ways in which the indicators provided by our citation impact framework are presented to users could be tested and compared.More generally, ideas similar to the ones proposed in this paper could be explored at the level of researchers or research groups rather than at the individual publication level.Finally, the distinction between cumulative research and more independent research could be studied in alternative ways.Research areas that are of a strongly cumulative nature for instance could be identified by searching for densely connected subnetworks in a citation network of publications.

Figure 1 .
Figure 1.Deep vs. broad citation impact.Nodes represent publications and edges represent citation relations.Publications  and  have both received five citations.All publications citing  also cite each other, which is not the case for publications citing .Therefore  has a deep citation impact, while  has a broad citation impact.

Figure 2 .
Figure 2. Dependent vs. independent citation impact.Nodes represent publications and edges represent citation relations.Publications  and  have both received five citations, and they both have three references.All publications citing  also cite each

Figure 3 .
Figure 3. Cumulative distribution functions of the level, depth, and dependence of the citation impact of highly cited publications.

Figure 4
Figure 4 provides more detailed insight into the relations between the level, depth, and dependence of citation impact.The figure shows CDFs of depth and dependence for different levels of citation impact.In addition, it shows CDFs of dependence for

Figure 4 .
Figure 4. Cumulative distribution functions showing the relations between the level, depth, and dependence of the citation impact of highly cited publications.
Using the algorithmic methodology introduced byWaltman and Van Eck (2012), publications in our WoS database in the period 2000-2017 were clustered based on citation relations.Only publications of the document types 'article', 'letter', and 'review' were included in the clustering.4,047 clusters of publications were obtained.

Figure 6 .
Figure 6.Scatter plot of the depth and dependence of the citation impact of 182 highly cited publications in the field of scientometrics.The dashed vertical and horizontal lines indicate the median depth and median dependence, respectively.In each quadrant, one publication was selected, marked by a star.

Figure 7 .
Figure 7. Distribution of the number of co-citation links and the number of bibliographic coupling links between a selected publication and the publications citing the selected publication.The dashed vertical line indicates the mean of the distribution.The mean of the distribution of the number of co-citation links corresponds to the depth of the citation impact of a selected publication.The mean of the distribution of the number of bibliographic coupling links corresponds to the dependence of the citation impact of a selected publication.
Publication P3, published in 2010, is a review article about the h-index and other related bibliometric indices.P3 has been cited 116 times in our data.It includes 256 references, of which 175 point to publications included in our data.The large number of references reflects the voluminous literature on the h-index published between 2005 and 2010.P3 has a high dependence of 8.26.Hence, when a publication cites

Table 1 .
Pearson (upper-right triangle)andSpearman (bottom-left triangle)correlations between the level, depth, and dependence of the citation impact of highly cited publications.

Table 2 .
Number of highly cited publications per discipline and the mean, median,