Abstract

Citations can be used in evaluative bibliometrics to measure the impact of papers. However, citation analysis can be extended by considering a multidimensional perspective on citation impact which is intended to receive more specific information about the kind of received impact. Bornmann, Wray, and Haunschild (2020) introduced the citation concept analysis (CCA) for capturing the importance and usefulness certain concepts (explained in publications) have in subsequent research. In this paper, we apply the method by investigating the impact various concepts introduced in Robert K. Merton’s book Social Theory and Social Structure has had. This book was to lay down a manifesto for sociological analysis in the immediate postwar period, and retains a major impact 70 years later. We found that the most cited concepts are “self-fulfilling” and “role” (about 20% of the citation contexts are related to one of these concepts). The concept “self-fulfilling” seems to be important especially in computer sciences and psychology. For “role,” this seems to be additionally the case for political sciences. These and further results of the study could demonstrate the high explanatory power of the CCA method.

1. INTRODUCTION

Traditional citation analyses simply count how often a paper is cited in later citing publications. Thus, in traditional citation analyses each citation is counted equally (with a weight of 1). This equally counting approach is based on the premise of the normative theory of citing that publications are cited to appreciate the contribution of the cited author (or his or her paper) to research (Merton, 1973). However, overviews of studies investigating the citation process reveal that publications are cited because of many reasons and fulfilling many functions (Tahamtan, Safipour Afshar, & Ahamdzadeh, 2016). For example, certain reputable authors can be cited to persuade the reader of a citing document that those authors may confirm the ideas, concepts, and results in the citing document. The approach of assuming many reasons and functions in the citation process is rooted in the social-constructivist theory of citations. It questions the equally counting approach in citation analyses.

In the past, many different approaches have been proposed to advance traditional citation analyses. Many of these approaches can be subsumed under the headings “citation context” and “citation content” studies. For citation context studies, the texts around the citations in a document are analyzed. This approach of studying the citation process can be used—among other things—to study whether the author of the document is giving credit or is citing for rhetorical reasons. In addition, it can be inspected in which sections most of the citations appear and how often certain publications are cited in the document. With the other approach—citation content analysis (which is used in this study)—the content of the text around a citation is analyzed to characterize the cited document. In other words, one investigates the publications citing a certain document to learn about the content of the cited document. If one is interested in learning about the concepts proposed in a focal document and cited (used) in various citing publications, Bornmann, Wray, and Haunschild (2020) name this citation content analysis approach “citation concept analysis” (CCA).

Concepts might refer to ground-breaking ideas, theories, explanations, mechanisms, etc. published in a citable document. The concepts must have unambiguous labels to be clearly identifiable in CCAs. Bornmann et al. (2020) introduced the CCA method based on analyses of classic books by Kuhn (1962) and Popper (1934, 1959, 1962). Both researchers have proposed various ground-breaking concepts (e.g., “paradigm,” “scientific revolution,” and “falsification”). Bornmann et al. (2020) studied what the impact of the various concepts was and how the impact differs depending on the fields and publication years of the citing papers. The authors used an approach proposed by Small (2018) and Small, Boyack, and Klavans (2019) to additionally measure uncertainty associated with concepts based on the presence or not of hedging words (e.g., “may”).

In this study, we follow the approach of Bornmann et al. (2020), undertaking a CCA of another classic book in social sciences: Social Theory and Social Structure (ST&SS) by Robert K. Merton (RKM) (Merton, 1968). This book was to lay down a manifesto for sociological analysis in the immediate postwar period, and retains a major impact seventy years later. We were interested in the influential concepts in the book (e.g., “self-fulfilling” or “anomie”) which are cited in subsequent publications and how often these concepts are cited.

2. CITATION CONTENT ANALYSES—A SHORT LITERATURE OVERVIEW

The CCA introduced by Bornmann et al. (2020) has been developed based on some precursor studies investigating the influence of authors, single publications, or other entities on citing publications. These citation content analyses investigated the context of citations in citing publications to characterize the cited author, publication, etc. Since these analyses were time consuming (the analyses were mostly done manually), the number of available studies is relatively small. The studies are described rather briefly in the following, since more detailed summaries of the papers can be found in the recent literature overview published by Tahamtan and Bornmann (2019).

One of the first citation content studies is Anderson (2006). Here, the influence of Karl Weick’s book The Social Psychology of Organizing on its citing publications is investigated. The set of citing publications was restricted to journals in organization studies (e.g., the Academy of Management Review). The most frequently cited concepts from the book were “enactment,” “equivocality,” and “refutational.” Anderson (2006) also found that the concepts are differently cited in US-based journals than in European-based journals:

First, while Organizing [The Social Psychology of Organizing] has been highly cited in journals from both regions, there appear to be substantial differences in terms of the specific content it has been cited for. The finding that OS [the journal Organization Studies] authors cite the difference between “organizing” versus “organization” much more frequently than do the American journal authors supports the contention that European scholars are more concerned about process and interpretive perspectives than their North American counterparts (p. 1686).

In a later study, the same author (in collaboration with a coauthor) investigated the impact that a highly cited paper (James P. Walsh’s and Gerardo Rivera Ungson’s paper “Organizational memory”) in the field of organizational memory received (Anderson & Sun, 2010). This paper consolidated various existing directions of thinking about organizational memory. The authors found that the paper was mainly cited from “management discipline” and the area of “information technology.” The paper has been cited for “one of four reasons: (1) for their notion of six organizational memory ‘storage bins’, (2) for a general reference to organizational memory, (3) for the issue of the use, misuse, and abuse of organizational memory, and (4) for their definition of organizational memory” (Anderson & Sun, 2010, p. 142).

McCain and Salvucci (2006) focused in their citation content study on the publications citing Frederick P. Brooks’s book The Mythical Man-Month. Essays on Software Engineering. Useful concepts—especially for areas such as “software engineering” and “computer science”—were “project management issues” and “building the system.” In another study, Danell (2012) analyzed not only the citation content of publications citing one paper, but three (highly cited) papers (in the area of complementary and alternative medicine). The author revealed that many citations came from the Web of Science subject categories “Rehabilitation,” “Medicine, General & Internal,” “Orthopedics,” “Integrative & Complementary Medicine,” “Substance Abuse,” and “Psychology Clinical.” Bornmann, Haunschild, and Hug (2018) compared keywords of citing papers and keywords of cited papers with terms in the citation context of the citing papers using the impressive oeuvre of Eugene Garfield (EG, who conceptualized the scientific citation index). The comparison suggests that “papers of EG and citation contexts of papers citing EG are semantically more closely related to each other than to titles and abstracts of papers citing EG” (p. 427). Chang (2013) did not investigate several publications, but several editions of the same book: de Solla Price’s book Little Science, Big Science and Beyond—a classic book in scientometrics. The analysis of the content surrounding citation anchors shows that the book is most frequently cited for three concepts: “science growth patterns,” “scientific communication,” and “scientific productivity.”

As the study by Sieweke (2014) reveals, not only single publications can be the basis for citation content studies, but also the oeuvre of a certain researcher. The author studied the impact of the well-known French sociologist Pierre Bourdieu on citing papers from leading journals in management and organization studies. Sieweke (2014) identified as most influential concepts “capital,” “habitus,” and “field,” whereby within the “capital” concept “social capital” seems to be most important. Liu, Ding, et al. (2015) conducted a citation content analysis of Nobel Prize laureate John O’Keefe’s work about the discovery of cell placement. Frequent terms in citing sentences were “cell placement,” “hippocampus,” and “environment.” In the most recent study, Gonzalez-Teruel and Abad-Garcia (2018) analyzed the impact of Elfreda Chatman’s theories (“information poverty theory,” “life in the round theory,” and “normative behavior theory”) on citing papers. The results show that the theories were most important in “social sciences,” “computer science,” and “medicine” (but not in “arts and humanities”).

3. ROBERT K. MERTON’S SOCIAL THEORY AND SOCIAL STRUCTURE

In the late 1940s, Columbia professor RKM, then in his late 30s, assembled much of his then published material into ST&SS. At that stage his oeuvre included “a masterful dissertation that became a foundation work in the sociology of science, an additional 15 well-received articles and book-chapters, plus 50 or so book reviews” (Sica, 1998, p. 113).

ST&SS was first published in 1949, with subsequent editions in 1957 and 1968. Each edition retains a four part structure:

  • • 

    on theoretical sociology (on relations between theory and research, and functional analysis),

  • • 

    studies in social and cultural structure (anomie, bureaucracy, and reference groups),

  • • 

    sociology of knowledge and mass communications, and

  • • 

    sociology of science.

The 1957 “revised” edition includes four extra essays (two of which are “continuities” that attempt to update analyses included in the first edition), which expand its volume by one third, and it also incorporates some revisions. The 1968 “expanded” edition differs from its predecessor only in a considerable expansion of its introduction into two chapters, with the whole part 1 “on theoretical sociology” being published separately in 1967. The first edition was subtitled “towards the codification of theory and research,” but this was dropped for later editions.

Seen more analytically, ST&SS is a tri-partite book:

  • • 

    methodologically, it sketches a program about how sociology might best advance;

  • • 

    theoretically, it provides a theoretical orientation and also a wealth of concepts that can be used in sociological explanations; and

  • • 

    substantively, it provides a glittering array of particular insights and specific studies that illuminate approach and concepts.

Each of the three editions emphasizes one or other of these three major contributions. The first edition indicates the approach and the theoretical orientation, but places greatest weight on substantive applications. The second broadens considerably the theoretical apparatus. The third revisits the broader methodological or metatheoretical concerns.

Since most of the chapters in ST&SS were published previously, we considered not only ST&SS in this study but also the following previous publications: “social structure and anomie,” “the self-fulfilling prophecy,” “bureaucratic structure and personality,” “science and the social order,” “role of the intellectual in public bureaucracy,” “the bearing of empirical research upon the development of social theory,” and “the machine, the worker, and the engineer.” However, the key essay in ST&SS on functional analysis was not previously published. Furthermore, considerable fresh editorial material was included in the book, not only in the introduction but also as introductions to each of the four parts. Later editions included more material that was previously unpublished: the two long essays on “continuities.” Some of the writing (e.g., the essays on the “self-fulfilling prophecy” and the “machine, the worker, and the engineer) was aimed at a lay audience (Merton, 1949, p. 120). Coauthors of chapters include Paul Lazarsfeld (on media research) and Alice Rossi (on reference group theory; Merton, 1957). Interestingly, the chapters tend to be included from newest to oldest.

Not all of his previously published material was included. The most surprising omission is his essay on the “unanticipated consequences of purposive action” (Merton, 1936) which would have strengthened the theoretical core of the book. The exclusion of this essay is likely related to the shift in sociological thinking in this period away from the interests in the social psychology of “social action,” which had been more influential over the previous decade, to a more systems-interest in functional analysis in which the place of social psychology was more problematic.

ST&SS famously begins with a masthead quotation culled from scattered (and unidentified) pages from Whitehead’s (1917) “the organisation of thought” which leads to discussion in which RKM steers a complex path between natural sciences and humanities.

Although the introduction eschews the task of integrating the book as a whole, the introduction to each part spells out the connections that its material has to the remainder of the book.

On page 11, Merton (1949) brings in the main character in his plot. The main thrust of ST&SS undoubtedly was seen by RKM as an enunciation of a program of work to extend functional analysis. This task is seen as being as much methodological as theoretical, thus sitting easily alongside the two clearly methodological essays in ST&SS, which address the mutual interrelations of theory and research.

While RKM argues for a structural-and-functional analysis approach to understanding cultural and social structures, this approach is extended in the book by a more thorough structural treatment and the deployment of a considerable array of social structural concepts, which are offered to his sociological colleagues to invite further conceptual development and empirical exploration. Finally, in the substantive material gathered in the book, RKM draws attention to the sociological need to study the various institutions and social forms of modern societies, including bureaucrats, scientists, and social science experts.

RKM tended to eschew the classic “research report” format for his papers and he admits that his book is a compilation of essays, yet is “reluctant to believe that the book lacks altogether the logical, and not merely literary, graces of coherence, unity and emphasis” (Merton, 1949, p. 3).

Although in this period it was not always easy to get sociological books published, RKM (and some other sociologists) had the active interest of Jeremy Kaplan, who had recently founded the Free Press and was keen to purvey sociological books (Lipset, 1993). Not that this was a straightforward offer: Kaplan needed RKM’s work more than RKM needed him. Indeed to help launch Kaplan’s enterprise ST&SS was published on faith and a prayer, and with no hard promise of royalties.”… Merton, whose book sold well over 100,000 copies, never received more than a 10 percent royalty” (Lipset, 1993, p. 7). The title is evocative, with a catchy symmetry: two two-word terms/phrases, “social theory” and “social structure” are joined together, and with repetition of “social.” The title broadly indicates RKM’s concern with an approach and a subject matter. His purpose is further indicated by the subtitle.

Translations were not mere carbon copies of the American original, but several differed in content. Sztompka (1996) notes that “for many of the multiple foreign translations of STSS, RKM adduces special introductions which relate to the sociological tradition of the given country” (p. 265). Indeed, some of the later translations have given rise to more debate than earlier versions—such as the review symposium on the German edition in the Berliner Journal für Soziologie (from 1996), which included the point, derived from RKM’s own middle-range doctrine of the transitoriness of theories, that the translation was decades late.

The range of published reviews of the original texts is limited, at least as far as mainstream sociology journals are concerned. No review of the original volume appeared in either the American Journal of Sociology or the American Sociological Review, although Social Forces and the British Journal of Sociology reviewed the first edition and the American Sociological Review reviewed both the second and third editions. The book was clearly welcomed in these reviews, with fervor rising over time: But some critical comments are also included.

Nor are RKM’s own comments on his work much help. When ST&SS was declared a citation classic, Merton (1980) merely enumerated some of the areas of work it contains—reference groups; local and cosmopolitan influentials; the self-fulfilling prophecy; unanticipated consequences; the paradigm of the sociology of knowledge; and sociology of science—without reference to any underlying theme.

In a chapter on RKM’s work, Sica (1998) provides an argument that the material in ST&SS tends to sound dated and reflects rather an earlier era where social life was more ordered and with less social change, although it would be reasonable to suggest that the prose retains a considerable freshness and does not read as a fusty classic.

These reactions can be extended through citation studies. ST&SS has been recorded as one of the most cited sociological books of all times. Moreover, what is particularly interesting about its pattern of use is that it has been taken up not merely by sociologists, and more broadly social scientists, but also by humanists and natural scientists. RKM has received 39,000 citations in those social science journals covered by the Social Science Citation Index (SSCI) and 35,000 (obviously overlapping) in natural science journals, as measured by the Science Citation Index (SCI). Of these, 12,000 (30%) of those in the SSCI are to ST&SS. As Clark, Modgil, and Modgil (1990) remark in reflecting on a much earlier bibliographical analysis:

this staggering figure is even more remarkable when it is remembered (a) that the citation index excludes citations in books (although it does include citations to books) and (b) it is now well established that the vast majority of citations of a paper normally appear in the first few years immediately after publication. The period under discussion (1969–1989) was between twenty and forty years after the publication of RKM’s most cited work, ST&SS, and between thirty and fifty years after the publication of his celebrated papers on “unanticipated consequences and anomie” (Clark, Modgil, & Modgil, 1990, p. 23).

At the turn of the century, the International Sociological Association organized a limited email exercise to nominate the best work of the 20th century (an exercise flawed, inter alia, by the difficulty that many classical authors span the divide to the 19th century). In this “competition,” RKM (and ST&SS) was “voted” as the third most significant author and third most significant book: especially among men rather than women, and among the older generation of sociologists. Another indicator of the book’s continuing popularity was its placement at rank 42 among the top 50 sociology bestsellers on the amazon.co.uk website (October 1999)—even though it was at that time out of stock.

4. METHODS

4.1. Concepts Used from Social Theory and Social Structure

As noted above ST&SS is a tri-partite book:

  • • 

    methodologically, it sketches a program about how sociology might best advance;

  • • 

    theoretically, it provides a theoretical orientation and also a wealth of concepts that can be used in sociological explanations; and

  • • 

    substantively, it provides a glittering array of particular insights and specific studies that illuminate approach and concepts.

Given RKM’s broad development of concept-terms it is difficult to pin down a set wide enough to do justice to his work. After consideration of the texts against a wish-list of key concepts, a dozen were able to be explored empirically given the limitations of the database used. Although it is not unproblematic as an indicator, the number of pages listed alongside each of the terms in the index of ST&SS was used. Since RKM is usually a careful writer, there is a close fit between his concepts and the terms used to name each, and this close fit is often signaled in his commentary by use of the combination “concept-term.” Each will be briefly defined and related to the three-thrust model of STSS. The concepts used in this study are printed in bold.

Middle-range theory is a key (and a particularly strategic) concept in RKM’s promulgated sociological methodology program. RKM discusses various meanings and formats that theory might adopt, with middle-range theories providing the key format. There is much discussion around these, but a synthetic statement provided by RKM summarizes their intent and also points to examples:

Middle-range theory is principally used in sociology to guide empirical inquiry. It is intermediate to general theories of social systems which are too remote from particular classes of social behavior, organization and change to account for what is observed and to those detailed orderly descriptions of particulars that are not generalized at all. Middle-range theory involves abstractions, of course, but they are close enough to observed data to be incorporated in propositions that permit empirical testing. Middle-range theories deal with delimited aspects of social phenomena, as is indicated by their labels. One speaks of a theory of reference groups, of social mobility, or role-conflict and of the formation of social norms just as one speaks of a theory of prices, a germ theory of disease, or a kinetic theory of gases (Merton, 1968, p. 39).

A second broad concept was that of an analytical paradigm, which for RKM is a framework for organizing a field of study: The analytical paradigm consists of several questions or issues that arise in understanding a field of social phenomena, together with a delineation of the optional positions that might be taken in relation to these issues. He developed several paradigms in STSS and also drew attention to their generic characteristics and methodological importance. Paradigms can be conceptualized as broad frameworks within which a variety of—partially competing—middle-range theories can be nested. They might also show patterns through which middle-range theories might be added to each other to cumulate theory coverage. It is important to remember that RKM’s notion of paradigm long predates the more famous and broadly similar subsequent usage of the term by Kuhn (1962).

As argued above, two sets of theoretical conceptions are offered. Functional (later structural-functional or even structure-and-functional) theory or analysis is the general theme (concept functions). Several social scientists from the 1930s had developed a formal approach that suggested that society was best studied by considering the functions that social phenomena fulfilled for wider social units—particular whole societies. While RKM was an early proponent of the functional approach he also uncovered its assumptions and charted its limitations. In sum, he advocated a more complex and sophisticated version, which was open-ended and also pushed toward empirical analysis rather than reliance on observatorial fiat.

But STSS also developed the structural aspect of the structure-function pairing. This began with reference group theory, which postulates that people’s behavior is orientated toward key stakeholders (sometimes formally sometimes more subjectively). A narrower version of this involves the conception of reference or role model wherein a person’s behavior is guided by their orientation to the model. But RKM also built more widely on status-role theory which held that people occupy social positions within social structures (named as status) and that they orientate their behaviors toward those in other social positions that face them (which refers more to the outward-facing aspect of statuses). For example, teachers are related to principals/headmasters, other teachers, students, students’ parents, etc. RKM’s work revealed how complex social structures are and provided an ever-expanded set of concepts as tools for their analysis.

In his empirical studies, RKM noted that people (in particular thought leaders) either tended to be orientated toward outside reference groups (cosmopolitans) or were more embedded in matrices of local belonging (locals). So he coined the matched/paired concepts of locals and cosmopolitans to capture these alternative orientations. Since then, more complex interweavings of these two orientations have been suggested.

RKM saw social structures as providing or raising barriers to opportunities. For example, the chances of upward mobility are shaped by possibilities opened up by the existing social structure. Opportunities also refers (ambivalently) to other positions that people might potentially move to, and/or the resources available to a person in his or her social position.

In their everyday life, people’s actions (which are shaped by their positions) in the social structure have consequences for themselves in the future and also for other people. RKM pointed out that some of these consequences are intended and some are unintentional. The latter are sociologically more interesting.

Another key line of investigation for RKM was reflexivity. This involves, for example, how people’s beliefs guide their behaviors. But such beliefs may be incorrect, although ironically this can lead to them becoming self-fulfilling, which generates them as true. For example, a run on a bank can be generated by an original false belief that that bank is unsound.

RKM’s concept of anomie is broadly derived from one of sociology’s founding fathers, Emile Durkheim, although he provides a much tighter version. RKM argues that (particularly in changing societies) lack of adequate access to required resources or means to achieve well-accepted cultural norms (e.g., for upward mobility or monetary success) can structurally induce those more stressed by the poor linkage of the values they hold and the inadequate resources at hand for achieving them toward innovation, deviance, and crime. Thus, these alternative approaches can be used to achieve success in the goals of that society.

4.2. Dataset Used

In this study, we used citation context data from Microsoft Academic (MSA) (Sinha, Shen, et al., 2015); see https://aka.ms/msracad. We searched for the book title: A single database entry belongs to the different editions of RKM’s book ST&SS. We also searched for previously published works that introduced the concepts of the book before the book was published: “social structure and anomie,” “the self-fulfilling prophecy,” “bureaucratic structure and personality,” “science and the social order,” “role of the intellectual in public bureaucracy,” “the bearing of empirical research upon the development of social theory,” and “the machine, the worker, and the engineer.” These were cited 14,377 times inside MSA. A total of 1,912 distinct citances (a set for words around the citation symbol, usually a sentence) are available (Small, Boyack, & Klavans, 2019). For 1,904 distinct citances, a level 0 (the highest hierarchical disciplinary level) field of study (FOS) is available for the statistical analyses. These numbers show that for most citing papers, citation context information is missing in MSA. For only 9.9% of the citing papers is citation context information available to us.

We searched certain terms—derived from key concepts from RKM’s book—in the citances in the following manner. Punctuation characters (, . ; :) were removed from the citances, and all citances were converted to lowercase characters. The concepts and the corresponding search terms are shown in Table 1. We removed the titles of RKM’s publications from the citances when any of the search terms is contained in the title to avoid dubious hits.

Table 1. 
Concepts and corresponding search terms
ConceptSearch term(s)
Consequence “consequence*” 
Middle-range theory “middle*range*” 
Status “status*” 
Social structure “social*structur*” 
Self-fulfilling “selffulfilling*” and “self_fulfilling*” 
Role “role*” 
Anomie “anomie” 
Function “*function*” 
Cosmopolitan “cosmopolitan*” 
Opportunity “opportunit*” 
Analytical paradigm “paradigm*” 
Reference group “reference*group*” 
ConceptSearch term(s)
Consequence “consequence*” 
Middle-range theory “middle*range*” 
Status “status*” 
Social structure “social*structur*” 
Self-fulfilling “selffulfilling*” and “self_fulfilling*” 
Role “role*” 
Anomie “anomie” 
Function “*function*” 
Cosmopolitan “cosmopolitan*” 
Opportunity “opportunit*” 
Analytical paradigm “paradigm*” 
Reference group “reference*group*” 

Note: The asterisk is a truncation symbol. An underscore denotes a single arbitrary character.

In this study, we also measured uncertainty associated with concepts. Small et al. (2019) propose the use of certain hedging words (e.g., “may”) to measure the uncertainty that might be associated with cited papers. The authors explain that “hedging does not assert that the paper is wrong, but only suggests that uncertainty surrounds some aspect of the ideas put forward” (p. 1086). Hyland (1996) proposes to distinguish between reader-motivated (e.g., “believe,” “suggest,” and “analogy”) and content-motivated (e.g., “generally,” “almost,” “might,” and “probable”) hedging words.

Henry Small provided us with an initial list of hedging words for measuring uncertainty, which he used in Small et al. (2019). The list is as follows: “not clear,” “no clear,” “appears,” “possibility,” “seems,” “speculated,” “to some extent,” “impression,” “sometimes,” “perhaps,” “not known,” “seem,” “apparently,” “tends,” “not necessarily,” “preliminary,” “contingent,” “could,” “doubt,” “explore,” “feel,” “hope,” “hopeful,” “hopefully,” “likely,” “may,” “might,” “nevertheless,” “nonetheless,” “not known,” “opportunity,” “plausible,” “possible,” “possibly,” “potential,” “potentially,” “probable,” “probably,” “projected,” “promise,” “promising,” “questions,” “risky,” “speculative,” “suspect,” “uncertain,” “unclear,” “unknown,” “unsolved,” “whether,” and “yet to be determined.”

Bornmann et al. (2020) checked in the citances of their data set how frequently the terms occur and how frequently they were used to express uncertainty. The final set with the most frequent terms they focused on is as follows: “may,” “could,” “questions,” “might,” “potential,” “seems,” “perhaps,” “likely,” and “sometimes.” This set has also been used in this study to measure uncertainty. We assume that these terms are frequently used terms in general.

4.3. Statistics

In this study, it was counted how often certain concepts (e.g., “paradigm”) are mentioned in citances of citing publications. In the statistical analyses, the possibility was considered that concepts are mentioned multiple times (more than one concept is frequently mentioned in a citing text). Since there is information in MSA available about the FOS and publication year of publications, we analyzed differences between fields and time periods in citing certain concepts. The relationships between concepts and fields or concepts and publication years can be represented in contingency tables. In these tables, citations of concepts are dependent variables; fields and time periods are independent variables.

We applied the Stata command mrtab (Jann, 2005) to analyze multiple mentions of concepts depending on publication year and FOS. In the interpretation of the results, we focused on two statistics: (a) as indications of effect sizes (Cumming & Calin-Jageman, 2016), we analyzed differences in percentages of concept mentions between different FOSs and publication years; and (b) we calculated statistical significance tests in r × c tables. To investigate the overall relationship between concepts and FOSs (or publication years), we performed an overall chi-square test. However, the results of the test could not be used when the following assumptions for chi-square tests on contingency tables were not fulfilled: “no more than 20% of the expected counts are less than 5 and all individual expected counts are 1 or greater” (Yates, Moore, & McCabe, 1999, p. 734). In addition, we performed a series of separate chi-square tests for each concept. For these tests, the p values were adjusted correspondingly to account for simultaneous calculations of many tests.

5. RESULTS

In section 5.1, we report our results on the citation impact received by the concepts—overall and in various FOSs and time periods. In section 5.2, the results relating to the perceived uncertainty of the concepts are presented.

5.1. First Empirical Part: Citations of Concepts in Various Fields of Study and Time Periods

Table 2 shows how frequently the concepts have been cited in various FOSs. The row “total” refers to the total numbers of concept mentions in the citing papers’ citances (broken down by FOS). The row “cases” includes the total numbers of the citing papers’ citances (broken down by FOS). For many citing papers of the book (and the previously published papers), citation context information is available; however, it does not contain any concept mentions considered here. The table reports column percentages based on all citances of the FOS (containing concept mentions or not). Thus, we assume that the concept mentions are dependent on the FOS.

Table 2. 
Citation concept analysis of Social Theory and Social Structure by Robert K. Merton and his related publications. How frequently have concepts (the concepts are decreasingly sorted by the column “total”) been cited in various fields of study?
Concept ArtBiologyBusinessChemistryComputer sciencesEconomicsEngineeringGeographyHistoryMathematicsMedicinePhilosophyPolitical sciencesPsychologySociologyTotal
Self-fulfilling n 17 18 20 94 13 179 
20 0.00 14.89 100.00 30.36 18.18 14.29 33.33 0.00 0.00 13.89 0.00 16.67 36.15 7.56 21.75 
Role n 15 29 53 17 137 
20 50.00 14.89 0.00 26.79 8.08 14.29 0.00 0.00 50.00 5.56 0.00 24.17 20.38 9.88 16.65 
Middle-range theory n 16 13 19 17 10 31 117 
20 0.00 34.04 0.00 23.21 19.19 7.14 33.33 33.33 0.00 16.67 100.00 14.17 3.85 18.02 14.22 
Function n 17 13 24 31 109 
14.89 12.5 17.17 28.57 25 13.89 10.83 9.23 18.02 13.24 
Anomie n 11 11 16 18 31 93 
2.13 7.14 11.11 33.33 30.56 13.33 6.92 18.02 11.3 
Social structure n 13 20 19 72 
20 50 8.51 8.93 6.06 25 5.56 10.83 7.69 11.05 8.75 
Status n 10 12 19 16 65 
6.38 3.57 10.1 8.33 10 7.31 9.3 7.9 
Opportunity n 19 14 51 
6.38 6.06 7.14 5.56 7.31 8.14 6.2 
Consequence n 18 13 49 
4.26 1.79 3.03 14.29 25 2.78 6.67 6.92 7.56 5.95 
Reference group n 17 39 
7.07 33.33 2.78 6.54 4.07 4.74 
Cosmopolitan n 20 
20 2.13 1.79 2.02 14.29 33.33 1.67 1.54 3.49 2.43 
Analytical paradigm n 12 
2.13 1.01 3.33 0.77 2.33 1.46 
Total N 52 65 108 14 38 146 298 202 943 
100 100 110.64 100 116.07 109.09 100 100 100 125 105.56 100 121.67 114.62 117.44 114.58 
Cases   47 56 99 14 36 120 260 172 823 
Concept ArtBiologyBusinessChemistryComputer sciencesEconomicsEngineeringGeographyHistoryMathematicsMedicinePhilosophyPolitical sciencesPsychologySociologyTotal
Self-fulfilling n 17 18 20 94 13 179 
20 0.00 14.89 100.00 30.36 18.18 14.29 33.33 0.00 0.00 13.89 0.00 16.67 36.15 7.56 21.75 
Role n 15 29 53 17 137 
20 50.00 14.89 0.00 26.79 8.08 14.29 0.00 0.00 50.00 5.56 0.00 24.17 20.38 9.88 16.65 
Middle-range theory n 16 13 19 17 10 31 117 
20 0.00 34.04 0.00 23.21 19.19 7.14 33.33 33.33 0.00 16.67 100.00 14.17 3.85 18.02 14.22 
Function n 17 13 24 31 109 
14.89 12.5 17.17 28.57 25 13.89 10.83 9.23 18.02 13.24 
Anomie n 11 11 16 18 31 93 
2.13 7.14 11.11 33.33 30.56 13.33 6.92 18.02 11.3 
Social structure n 13 20 19 72 
20 50 8.51 8.93 6.06 25 5.56 10.83 7.69 11.05 8.75 
Status n 10 12 19 16 65 
6.38 3.57 10.1 8.33 10 7.31 9.3 7.9 
Opportunity n 19 14 51 
6.38 6.06 7.14 5.56 7.31 8.14 6.2 
Consequence n 18 13 49 
4.26 1.79 3.03 14.29 25 2.78 6.67 6.92 7.56 5.95 
Reference group n 17 39 
7.07 33.33 2.78 6.54 4.07 4.74 
Cosmopolitan n 20 
20 2.13 1.79 2.02 14.29 33.33 1.67 1.54 3.49 2.43 
Analytical paradigm n 12 
2.13 1.01 3.33 0.77 2.33 1.46 
Total N 52 65 108 14 38 146 298 202 943 
100 100 110.64 100 116.07 109.09 100 100 100 125 105.56 100 121.67 114.62 117.44 114.58 
Cases   47 56 99 14 36 120 260 172 823 

Notes. Results of chi-square tests for single concepts (rows): self-fulfilling: χ2 = 66.74, p = 0.000; role: χ2 = 32.36, p = 0.043; middle-range theory: χ2 = 55.73, p = 0.000; function: χ2 = 14.79, p = 1; anomie: χ2 = 36.73, p = 0.01; social structure: χ2 = 12.00, p = 1; status: χ2 = 6.41, p = 1; opportunity: χ2 = 6.97, p = 1; consequence: χ2 = 10.75, p = 1; reference group: χ2 = 15.60, p = 1; cosmopolitan: χ2 = 30.23, p = 0.085; analytical paradigm: χ2 = 6.83, p = 1. The result of the overall chi-square test is not presented because the assumptions are not fulfilled.

The most cited concept in Table 2 is “self-fulfilling.” About 20% of the citances are related to this concept. The concept “role” follows with about 17%. The concepts “cosmopolitan” (2.4%) and “analytical paradigm” (1.5%) are those with only a few mentions (compared to the other concepts) in our data set. The results of the chi-square tests for the single concepts reveal three statistically significant results: “self-fulfilling,” “role,” “anomie,” and “middle-range theory” seem to be differently used in the various FOSs. The concept “role” seems to be important especially in computer sciences (26.79%), political sciences (24.17%), and psychology (20.38%) (considering the low case numbers in certain FOSs such as biology). For “self-fulfilling,” this seems to be the case in psychology (36.15%) and for “middle-range theory” in business (34.04%).

The heightened use in psychology may be because of an increasing movement from large sections of psychology toward more social psychological and sociological concerns. Rather than studying only psychological and even physiological processes, many psychologists are more interested in locating their study among interacting groups of people and hence find the analytical tool of “role” useful. Over recent decades the field of business studies has been concerned to establish itself as a more “scientific” discipline and hence has indulged in various methodological discussions designed to establish scientific credentials and to outline scientific ways of proceeding, hence drawing on RKM’s concept of middle-range theories. Indeed, management scientists have published a considerable number of publications devoted to middle-range theory (Moore & Pinder, 1980).

Table 3 shows the results for concept mentions depending on time. The results point out scarcely any time-dependent patterns; the chi-square tests for all concepts except one are statistically not significant. The exception is “anomie,” which has been increasingly mentioned in citances over time. Presumably, since ST&SS has long been available for scholars to read, they seem to draw broadly on its offerings and no particular aspect is emphasized.

Table 3. 
Citation concept analysis of Social Theory and Social Structure by Robert K. Merton and his related publications: How frequently have concepts (the concepts are decreasingly sorted by the column “total”) been cited in various time periods (publication years)?
Concept <20002000–20052006–20102011–2017Total
Self-fulfilling n 25 26 50 78 179 
21.74 25 19.53 22.41 21.75 
Role n 24 19 46 48 137 
20.87 18.27 17.97 13.79 16.65 
Middle-range theory n 14 11 42 50 117 
12.17 10.58 16.41 14.37 14.22 
Function n 23 10 38 38 109 
20 9.62 14.84 10.92 13.24 
Anomie n 27 56 93 
2.61 6.73 10.55 16.09 11.3 
Social structure n 30 28 72 
5.22 7.69 11.72 8.05 8.75 
Status n 13 36 65 
6.96 7.69 5.08 10.34 7.9 
Opportunity n 11 18 15 51 
6.09 10.58 7.03 4.31 6.2 
Consequence n 16 16 49 
6.96 8.65 6.25 4.6 5.95 
Reference group n 11 15 39 
6.96 4.81 4.3 4.31 4.74 
Cosmopolitan n 20 
2.61 1.92 3.13 2.01 2.43 
Analytical paradigm n 12 
1.74 0.96 1.95 1.15 1.46 
Total N 131 117 304 391 943 
113.91 112.5 118.75 112.36 114.58 
Cases   115 104 256 348 823 
Concept <20002000–20052006–20102011–2017Total
Self-fulfilling n 25 26 50 78 179 
21.74 25 19.53 22.41 21.75 
Role n 24 19 46 48 137 
20.87 18.27 17.97 13.79 16.65 
Middle-range theory n 14 11 42 50 117 
12.17 10.58 16.41 14.37 14.22 
Function n 23 10 38 38 109 
20 9.62 14.84 10.92 13.24 
Anomie n 27 56 93 
2.61 6.73 10.55 16.09 11.3 
Social structure n 30 28 72 
5.22 7.69 11.72 8.05 8.75 
Status n 13 36 65 
6.96 7.69 5.08 10.34 7.9 
Opportunity n 11 18 15 51 
6.09 10.58 7.03 4.31 6.2 
Consequence n 16 16 49 
6.96 8.65 6.25 4.6 5.95 
Reference group n 11 15 39 
6.96 4.81 4.3 4.31 4.74 
Cosmopolitan n 20 
2.61 1.92 3.13 2.01 2.43 
Analytical paradigm n 12 
1.74 0.96 1.95 1.15 1.46 
Total N 131 117 304 391 943 
113.91 112.5 118.75 112.36 114.58 
Cases   115 104 256 348 823 

Notes. Results of chi-square tests for single concepts (rows): self-fulfilling: χ2 = 1.48, p = 1; role: χ2 = 4.04, p = 1; middle-range theory: χ2 = 2.54, p = 1; function: χ2 = 7.97, p = 0.56; anomie: χ2 = 18.95, p = 0.003; social structure: χ2 = 4.99, p = 1; status: χ2 = 5.81, p = 1; opportunity: χ2 = 5.87, p = 1; consequence: χ2 = 2.74, p = 1; reference group: χ2 = 1.51, p = 1; cosmopolitan: χ2 = 0.91, p = 1; analytical paradigm: χ2 = 0.91, p = 1. The result of the overall chi-square test is not presented because the assumptions are not fulfilled.

5.2. Second Empirical Part: Uncertainty Associated with Concepts

Table 4 focuses on the uncertainty which might be associated with concepts. The results reveal that “self-fulfilling” seems to be the most “uncertain” concept: 34.51% of the uncertainty responses account for this concept and 27.37% of the concept mentions reflect some kind of uncertainty. One reason why this concept is used in a precautionary way is that it is subjective and it is not always clear what actors have in mind that might cause this effect.

Table 4. 
Counts (n) and percentage of citing texts—referring to the concepts in Social Theory and Social Structure by Robert K. Merton—reflecting uncertainty (all fields of study; the concepts are decreasingly sorted by the last column)
ConceptNumber of uncertainty mentionsPercentage of uncertainty mentions (n = 142)Number of concept mentionsPercentage of concept mentions with uncertainty
Self-fulfilling 49 34.51 179 27.37 
Consequence 10 7.04 49 20.41 
Status 12 8.45 65 18.46 
Reference group 4.93 39 17.95 
Opportunity 6.34 51 17.65 
Cosmopolitan 2.11 20 15.00 
Anomie 12 8.45 93 12.90 
Middle-range theory 12 8.45 117 10.26 
Role 13 9.15 137 9.49 
Function 10 7.04 109 9.17 
Social structure 3.52 72 6.94 
Analytical paradigm 0.00 12 0.00 
Total 142 100.00 943 15.06 
ConceptNumber of uncertainty mentionsPercentage of uncertainty mentions (n = 142)Number of concept mentionsPercentage of concept mentions with uncertainty
Self-fulfilling 49 34.51 179 27.37 
Consequence 10 7.04 49 20.41 
Status 12 8.45 65 18.46 
Reference group 4.93 39 17.95 
Opportunity 6.34 51 17.65 
Cosmopolitan 2.11 20 15.00 
Anomie 12 8.45 93 12.90 
Middle-range theory 12 8.45 117 10.26 
Role 13 9.15 137 9.49 
Function 10 7.04 109 9.17 
Social structure 3.52 72 6.94 
Analytical paradigm 0.00 12 0.00 
Total 142 100.00 943 15.06 

6. DISCUSSION

Citations can be used in evaluative bibliometrics to measure the impact of papers (Narin, 1976). However, citation analysis can be extended by considering a multidimensional perspective on citation impact (Bu, Waltman, & Huang, 2019) which is intended to receive more specific information about the kind of received impact. Bornmann et al. (2020) introduced the CCA method for capturing the importance and usefulness of certain concepts (introduced in specific publications) for subsequent research. The authors introduced the method based on measuring the impact of certain concepts in the classic books by Kuhn (1962) and Popper (1934, 1959, 1962). In this study, we investigated the impact that various concepts published in ST&SS have.

ST&SS captured much of RKM’s work up to 1949, some of his additional work through the 1950s, and a small sliver of extended work in the 1960s, but his oeuvre is much wider. As other studies show (see above), ST&SS remains the prime source of RKM’s work, but it has been extended by an array of other articles and indeed books. Moreover, since (as with several other books) RKM assembled these from already published pieces, citers have a choice of alternative sources to draw on (which we considered in this study). And of course, other scholars may be influenced by RKM indirectly through publications not authored by RKM. This establishes ST&SS as an uneven platform for those wishing to build on his work.

As histories of sociology well establish (Crothers, 1987), the functional framework which was the main thrust of the first edition (and of course subsequent editions) faded in its power in guiding sociological work by the 1960s and this depressed this aspect of his writing and perhaps had a broader limiting impact as well.

It is not surprising that “paradigm” is seldom attended to, as Kuhn’s (1962) famous study, in which this concept was central, swamped RKM’s earlier usage. Another concept that has not fared so well is “cosmopolitan” (part of the “local”/“cosmopolitan” concept-pair). While this is a useful distinction, it is not one that is often analyzed, and to some extent this too is the victim of available synonyms.

The general array of RKM’s concepts obtains a broadly similar level of attention. This draws attention to the wide range of RKM’s analytical toolkit and the availability of many terms within it. Given the complexity of social structures it is necessary to use a large toolkit, and although different analysts will subscribe to several of the tools available, all are potentially important. This relative evenness of attention may flow from RKM’s “failure” to provide a more general theoretical framework, which would have provided some guidance about the relative importance of various concepts. From RKM’s perspective he emphasizes “middle range theories” and so preferred not to provide more general theory.

Although “status” and “role” are paired concepts for RKM, it is probable that the latter is more popular in general, and has less of a “structural” and abstract tone often implying vertical differentiation. Again, after a decade-long burst of popularity role theory is no longer so popular.

It was argued above that RKM joined in debates about the direction of sociology and the means for moving along the path indicated: It is not surprising then that “middle-range theory” became a popular concept, which has continued to resonate across subsequent decades as the best strategy for developing sociology is a continuing concern.

Similar limitations to those mentioned in Bornmann et al. (2020) apply to our current study: The selection of concepts and search terms for the concepts and the uncertainty detection is a much less severe limitation. Here also the wording and length of the citation context determines our ability to properly assign citation contexts to concepts and determine their uncertainty.

Finally, the least severe limitation is the fact that some publications are not assigned to FOSs in MSA. As 99.6% of the citances could be matched to a level 0 FOS, this limitation seems negligible. However, the assignment of FOS to papers by MSA is not transparent. MSA algorithmically assigns FOS on the paper-basis. The quality and details of the algorithm of the FOS assignment are unclear (the same holds true for MSA’s criteria for indexing publications). Algorithmic FOS assignments may or may not be accurate. For example, the accuracy of an algorithm based on direct citation relations has been questioned (Haunschild, Marx, et al., 2018; Haunschild, Schier, et al., 2018a, 2018b). Although a case study on computer science publications reported promising results regarding the MSA FOSs (Scheidsteger, Haunschild, et al., 2018), a large-scale comparison is still missing. Despite these limitations, we are confident that CCA is an interesting method for enriching citation analyses.

AUTHOR CONTRIBUTIONS

Charles Crothers: Conceptualization, Supervision, Validation, Writing—original draft, Writing—review & editing; Lutz Bornmann: Conceptualization, Formal analysis, Investigation, Methodology, Project administration, Supervision, Validation, Writing—original draft, Writing—review & editing; Robin Haunschild: Conceptualization, Data curation, Investigation, Methodology, Project administration, Supervision, Validation, Writing—original draft, Writing—review & editing.

COMPETING INTERESTS

The authors have no competing interests.

FUNDING INFORMATION

There was no specific funding for this study.

DATA AVAILABILITY

The data is available at https://doi.org/10.5281/zenodo.3690144.

REFERENCES

Anderson
,
M. H.
(
2006
).
How can we know what we think until we see what we said? A citation and citation context analysis of Karl Weick’s The Social Psychology of Organizing
.
Organization Studies
,
27
(
11
),
1675
1692
.
Anderson
,
M. H.
, &
Sun
,
P. Y. T.
(
2010
).
What have scholars retrieved from Walsh and Ungson (1991)? A citation context study
.
Management Learning
,
41
(
2
),
131
145
. https://doi.org/10.1177/1350507609341091
Bornmann
,
L.
,
Haunschild
,
R.
, &
Hug
,
S. E.
(
2018
).
Visualizing the context of citations referencing papers published by Eugene Garfield: a new type of keyword co-occurrence analysis
.
Scientometrics
,
114
(
2
),
427
437
. https://doi.org/10.1007/s11192-017-2591-8
Bornmann
,
L.
,
Wray
,
K. B.
, &
Haunschild
,
R.
(
2020
).
Citation concept analysis (CCA)—A new form of citation analysis revealing the usefulness of concepts for other researchers illustrated by two exemplary case studies including classic books by Thomas S. Kuhn and Karl R. Popper
.
Scientometrics
,
122
,
1051
1074
. https://doi.org/10.1007/s11192-019-03326-2
Bu
,
Y.
,
Waltman
,
L.
, &
Huang
,
Y.
(
2019
).
A multidimensional perspective on the citation impact of scientific publications
.
Retrieved February 6, 2019, from https://arxiv.org/abs/1901.09663
Chang
,
Y. W.
(
2013
).
A comparison of citation contexts between natural sciences and social sciences and humanities
.
Scientometrics
,
96
(
2
),
535
553
. https://doi.org/10.1007/s11192-013-0956-1
Clark
,
J.
,
Modgil
,
C.
, &
Modgil
,
S.
(
1990
).
Robert K. Merton: Consensus and controversy
.
London
:
Taylor & Francis
.
Crothers
,
C.
(
1987
).
Robert K. Merton. Key sociologists series
.
London
:
Tavistock Publications
.
Cumming
,
G.
, &
Calin-Jageman
,
R.
(
2016
).
Introduction to the New statistics: Estimation, open science, and beyond
.
London
:
Taylor & Francis
.
Danell
,
J. A. B.
(
2012
).
Representation and Negotiation of Complementary and Alternative Medicine: A Citation Context Analysis
.
Science Communication
,
34
(
3
),
299
333
. https://doi.org/10.1177/1075547011413140
Gonzalez-Teruel
,
A.
, &
Abad-Garcia
,
F.
(
2018
).
The influence of Elfreda Chatman’s theories: A citation context analysis
.
Scientometrics
,
117
(
3
),
1793
1819
. https://doi.org/10.1007/s11192-018-2915-3
Haunschild
,
R.
,
Marx
,
W.
,
French
,
B.
, &
Bornmann
,
L.
(
2018a
).
Relationship between field-normalized indicators calculated with different approaches of field-categorization
.
Paper presented at the International Conference on Science and Technology Indicators (STI 2018)
,
Leiden, The Netherlands
. https://openaccess.leidenuniv.nl/handle/1887/65267
Haunschild
,
R.
,
Schier
,
H.
,
Marx
,
W.
, &
Bornman
,
L.
(
2018b
).
Algorithmically generated subject categories based on citation relations: An empirical micro study using papers on overall water splitting
.
Journal of Informetrics
,
12
(
2
),
436
447
. https://doi.org/10.1016/j.joi.2018.03.004
Hyland
,
K. E. N.
(
1996
).
Talking to the academy: Forms of hedging in science research articles
.
Written Communication
,
13
(
2
),
251
281
. https://doi.org/10.1177/0741088396013002004
Jann
,
B.
(
2005
).
Tabulation of multiple response
.
The Stata Journal
,
5
(
1
),
92
122
.
Kuhn
,
T. S.
(
1962
).
The structure of scientific revolutions
.
Chicago, IL
:
University of Chicago Press
.
Lipset
,
S. M.
(
1993
).
Jeremiah Kaplan, The Free Press, and post-war sociology
.
American Sociological Association Footnotes
,
7
.
Liu
,
S.
,
Ding
,
K.
,
Wang
,
B.
,
Tang
,
D.
, &
Qu
,
Z.
(
2015
).
The research of paper influence based on citation context – A case study of the Nobel Prize winner’s paper
. In
A. A.
Salah
,
Y.
Tonta
,
A. A. A.
Salah
,
C.
Sugimoto
&
U.
Al
(Eds.),
The 15th Conference of the International Society for Scientometrics and Informetrics
(pp.
241
248
).
Istanbul, Turkey
:
ISSI, Boaziçi University Printhouse
.
McCain
,
K. W.
, &
Salvucci
,
L. J.
(
2006
).
How influential is Brooks’ Law? A longitudinal citation context analysis of Frederick Brooks’ The Mythical Man-Month
.
Journal of Information Science
,
32
(
3
),
277
295
. https://doi.org/10.1177/0165551506064397
Merton
,
R. K.
(
1936
).
The Unanticipated Consequences of Purposive Social Action
.
American Sociological Review
,
1
(
6
),
894
904
. https://doi.org/10.2307/2084615
Merton
,
R. K.
(
1949
).
Social theory and social structure: Toward the codification of theory and research
.
University of California, CA
:
Free Press
.
Merton
,
R. K.
(
1957
).
Social theory and social structure
.
University of Iowa, IA
:
Free Press
.
Merton
,
R. K.
(
1968
).
Social theory and social structure
.
New York, NY
:
Free Press
.
Merton
,
R. K.
(
1973
).
The Sociology of science: Theoretical and empirical investigations
.
Chicago, IL
:
University of Chicago Press
.
Merton
,
R. K.
(
1980
).
This week’s citation classic
.
Current Contents
,
21
,
May 12
.
Moore
,
L. F.
, &
Pinder
,
C. C.
(Eds.). (
1980
).
Middle range theory and the study of organizations.
.
Heidelberg
:
Springer
.
Narin
,
F.
(
1976
).
Evaluative Bibliometrics: The use of publication and citation analysis in the evaluation of scientific activity
.
Cherry Hill, NJ
:
Computer Horizons
.
Popper
,
K. R.
(
1934
).
Logik der Forschung: Zur Erkenntnistheorie der modernen Naturwissenschaft
.
Vienna
:
Springer
.
Popper
,
K. R.
(
1959
).
The logic of scientific discovery
(2nd ed.).
London
:
Routledge
.
Popper
,
K. R.
(
1962
).
Conjectures and Refutations: The growth of scientific knowledge
.
University of Michigan, MI
:
Basic Books
.
Scheidsteger
,
T.
,
Haunschild
,
R.
,
Hug
,
S.
, &
Bornmann
,
L.
(
2018
).
The concordance of field-normalized scores based on Web of Science and Microsoft Academic data: A case study in computer sciences
.
Paper presented at the International Conference on Science and Technology Indicators (STI 2018)
,
Leiden, The Netherlands
. https://openaccess.leidenuniv.nl/handle/1887/65358
Sica
,
A.
(
1998
).
Robert K. Merton
. In
R.
Stones
(Ed.),
Key sociological thinkers
,
111
123
,
310–314
.
London
:
Macmillan Publishing
(revised edition, 2008, pp. 151–167, 397–401, Basingstoke: Palgrave Macmillan)
.
Sieweke
,
J.
(
2014
).
Pierre Bourdieu in management and organization studies – A citation context analysis and discussion of contributions
.
Scandinavian Journal of Management
,
30
(
4
),
532
543
. https://doi.org/10.1016/j.scaman.2014.04.004
Sinha
,
A.
,
Shen
,
Z.
,
Song
,
Y.
,
Ma
,
H.
,
Eide
,
D.
,
Hsu
,
B.-J. P.
, &
Wang
,
K
. (
2015
).
An overview of Microsoft Academic Service (MAS) and applications
.
Paper presented at the 24th International Conference on World Wide Web (WWW ’15 Companion)
,
Florence, Italy
.
Small
,
H.
(
2018
).
Characterizing highly cited method and non-method papers using citation contexts: The role of uncertainty
.
Journal of Informetrics
,
12
(
2
),
461
480
. https://doi.org/10.1016/j.joi.2018.03.007
Small
,
H.
,
Boyack
,
K. W.
, &
Klavans
,
R.
(
2019
).
Citations and certainty: a new interpretation of citation counts
.
Scientometrics
,
118
(
3
),
1079
1092
. https://doi.org/10.1007/s11192-019-03016-z
Sztompka
,
P.
(Ed.). (
1996
).
Robert K. Merton. On social structure and science
.
Chicago, IL
:
University of Chicago Press
.
Tahamtan
,
I.
, &
Bornmann
,
L.
(
2019
).
What do citation counts measure? An updated review of studies on citations in scientific documents published between 2006 and 2018
.
Scientometrics
,
121
(
3
),
1635
1684
. https://doi.org/10.1007/s11192-019-03243-4
Tahamtan
,
I.
,
Safipour Afshar
,
A.
, &
Ahamdzadeh
,
K.
(
2016
).
Factors affecting number of citations: A comprehensive review of the literature
.
Scientometrics
,
107
(
3
),
1195
1225
. https://doi.org/10.1007/s11192-016-1889-2
Whitehead
,
A. N.
(
1917
).
The organisation of thought, educational and scientific
.
London
:
Williams and Norgate
.
Yates
,
D.
,
Moore
,
D.
, &
McCabe
,
G.
(
1999
).
The practice of statistics
.
New York, NY
:
W. H. Freeman
.

Author notes

Handling Editor: Ludo Waltman

This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. For a full description of the license, please visit https://creativecommons.org/licenses/by/4.0/legalcode.