Abstract
Research that is not explicitly bound to a distinct discipline has not yet gained much acknowledgment with regard to research impact assessment and mapping of the respective research field. In this article, we provide a suggestion for new impact metrics taking the example of migration research as a phenomenon-oriented research field. Therewith, research merit is made comparable and is calculated irrespective of discipline. We show how the field of migration studies evolved, apply our new metrics and give insight into impact factors, numbers of citations of articles, and authors, as well as journals. Further, we present a field-related collaboration network that indicates a rather disconnected community. However, collaborations between researchers are on the rise. In our conclusion, we argue that there is a need for further assessment of research impact within other phenomenon-oriented research fields.
1. INTRODUCTION
Scientific knowledge graphs (SKG) and the impact of research have been built and measured for various disciplines (e.g., Kajikawa, Ohno et al. (2007) for sustainability science, Marshakova (1981) for information science, and Popp, Kovács et al. (2016) for agricultural economics). Big impact assessors such as the SCImago ranking portal furnish researchers from all over the world who work in (sub-)disciplines both big and small with numbers and measurements such as impact factors (for journals) and h-indices (for individual researchers or groups of researchers). The Science Journal Ranking (SJR) indicator “represents a bibliometric measure, based on a diffusion algorithm, for the quantification of the prestige of scientific journals” (Radicchi, Fortunato, & Vespignani, 2012, p. 250), relying on Elsevier’s Scopus database. Citation counts allow for an “evaluative measure of the scientific productivity of authors and the status of scientific journals” (Marshakova, 1981, p. 13), assuming that “citations represent a proxy for the quantification of scientific relevance” (Radicchi et al., 2012, p. 251; see also Moed, 2005). However, phenomenon-related research receives much less attention and some areas are entirely neglected.
“Citation analysis has already been gaining popularity in information science and in the science of science” noted Marshakova (1981, p. 13) four decades ago. Using citation counts we decide about the relevance and quality (though somewhat falsely) of a paper and its author(s). The “link between two papers as the number of identical documents cited by the two papers” (Marshakova, 1981, p. 21) is called bibliographic coupling, whereas “the co-citation method (or prospective coupling) unites the papers cited by the same documents” (Marshakova, 1981, p. 21). Cocitation analysis was introduced by Small (1973) and has gained importance since then. It is by citing a colleague, as well as by collaborating with them, that networks develop in the sense that a (formal) relation between one author and another is established as they include the paper in their work, thereby showing a connection to or similar interest in the same research topic. Granovetter (1973, p. 1377) claims that such network analyses highlight micro-macro linkages in that they show how individuals are “bound up with larger-scale aspects of social structure.” The status of an actor can then be described by their centrality in the network. These analyses, too, often disregard research that is not directly affiliated with a distinct discipline.
Our article has four main sections. By using the field of migration research, we first make an argument that measuring research impact for phenomenon-oriented research fields that do not appear in the well-known indices is relevant and explain why a different measurement is needed. Furthermore, we provide a short insight into the field of migration research. Second, we set up a framework describing how to acknowledge scientific merit in this area by suggesting a new calculation for research impact assessment for internally diverse fields. Third, we provide a broad range of citation and collaboration measures and SKGs to give an overview so that we can select the most relevant measures for building a novel sophisticated citation factor. We then, lastly, discuss strengths and limitations of our work and obstacles we encountered, before giving an outlook on future research in this area.
With regard to the SCImago rankings (cf. https://www.scimagojr.com/), the options offered on the website let the researcher sort results according to disciplines (such as chemistry, dentistry, and engineering) and according to subject categories (such as small animals, pollution, horticulture, and ocean engineering). However, this is of no use for migration researchers: Their field is multidisciplinary, with researchers coming from ethnology, political science, communication studies and many others; plus, the subject of migration is not included in the subject list, perhaps because it is a comparatively novel topic. Migration research belongs to what Schwarz and Stensaker (2016) call “PDR” (phenomenon-driven research). This research does not exclusively belong to a specific discipline but “is problem-oriented research that focuses on capturing, documenting, and conceptualizing an observed phenomenon of interest in order to facilitate knowledge creation and advancement” (Schwarz & Stensaker, 2016, p. 245). This leads us to ask the following research question: How can we measure and visualize research impact in phenomenon-oriented research fields? This article aims to provide an answer to this question.
1.1. Relevance
In the broader field of the science of science (Fortunato, Bergstrom et al., 2018) most big bibliometric studies concentrate on the major disciplines (e.g., Redner (2005) for physics) that often provide trustable databases from which to retrieve relevant information such as authors’ names, affiliation, keywords, abstracts, and citation lists. Unfortunately, a field such as migration research does not have its own database that lists topic-related contributions (such as the CMMC [Communication and Mass Media Complete] for communication research or the APS database for physics). Furthermore, the number of papers as well as the citations and impact factors vary greatly across disciplines. They reach their largest numbers, for example, in internal medicine and biochemistry but are lowest in disciplines such as literature and dance (Patience, Patience et al., 2017). If the science of science wants to step beyond the provision of SKGs and research impact assessment for single disciplines, we are in need of fine-grained, context-sensitive representations of scholarly knowledge in these inter- and multidisciplinary fields. This is why we want to provide a framework of how to better conceptualize scholarly knowledge and research impact assessment by using the field of migration research as an appropriate example.
Due to a lack of ready-made data, we will utilize different metrics to allow us to better control for the various disciplinary backgrounds. Only then will it be possible to compare research impact beyond discipline- and journal-bound impact factors and to generate synergies between currently divided research communities. However, we do not propose our approach as a substitute but rather as a valuable complement to hitherto existing impact assessment. With regard to funding, new indicators seem relevant, as these phenomenon-oriented research communities lack “advocates” in the funding institutions that still organize decision processes according to distinct disciplines. A law from Radicchi, Fortunato, and Castellano (2008) says that if “the number of citations of a paper is divided by the average number of citations collected by papers in the same discipline and year, the distribution of the resulting score is essentially indistinguishable for all disciplines.” However, what if there is no concrete “discipline”? And how, for instance, can a political scientist or psychologist merit a great impact in migration research when in their core discipline migration researchers are viewed as a peripheral group? Hence, one needs to pay attention to field-specific evolvements. This is why in the following section we will give an insight into how migration research developed, but we will keep this section short as the focus of this article lies in developing a general approach to measuring research impact in phenomenon-oriented research. In the results section we present a basic overview using descriptive statistics, then turn to the most influential journals, authors, institutions, and countries before presenting collaboration networks. We believe that looking at domain-specific figures other than those that are only citation-related (e.g. dominance of countries) can be useful for validation purposes of the whole approach.
1.2. The Field of Migration Studies
The field of migration research covers diverse topical clusters: from mobility to development, from education to identity, from gender to racism (cf. Figure 1).
Even though migration and flight are “anthropological constants” (Strukelj, 2020, p. 174), they have not yet received as much scholarly attention as other areas of research, especially in comparison to topics in the natural sciences. This might be due to the fact that migration studies is not considered a distinct research discipline and many researchers stick to their own disciplinary approaches when dealing with the issue of migration. Whereas some researchers have called the late twentieth century “the age of migration” (e.g., Castles & Miller, 1993), there is evidence that the refugee crisis in 2015 has led to an increase in scholarly interest in the general topic of migration (Pisarevskaya, Levy et al., 2020, p. 9). Zlotnik (1998, p. 430) defines international migrants as “people who depart from their country of origin (or citizenship) to reside in another country and those who return to their country of origin after residing abroad.” Both international migrants such as refugees, migrant workers, or marriage immigrants, as well as internally displaced persons, must be considered when studying migration. Zlotnik (1998) provides an overview of migration flows starting from 1965. (Previous to 1965, there are only scarce records.) Since then, the number of international migrants has been constantly growing and similarly the internationalization of research in this field has also been increasing (Pisarevskaya, Levy et al., 2020). Scholarly interest has rapidly increased since the so-called migrant crisis in 2015; however, some of the flagship journals of the field were founded in the 1960s (e.g., International Migration in 1961 and International Migration Review in 1964). The field of migration studies has been multidisciplinary since its inception, starting primarily in the humanities and anthropology, with more recent approaches based in the social sciences. Of course, migration research is not solely published in migration studies journals; yet, the core of migration studies is found in these journals and researchers who aim to position themselves as experts in the field strive to publish in these journals. This directly leads us to explain how we chose an appropriate sample of journals to elucidate the development of migration research from its infancy to present day, cutting-edge migration research.
2. METHODS
2.1. Sampling
The SCImago metrics are based on Scopus data; this is why we suggest using Scopus as a data source. Furthermore, the Web of Science as a direct competitor does not list as many databases and is more oriented towards the natural sciences. As migration research has lexical overlaps with unrelated fields (e.g., “migration distance in sea turtles” (Hays & Scott, 2013)), it is difficult to generate an appropriate corpus by only looking for migration-related keywords, for example in article titles. Even though we acknowledge that journal-based delineations exclude all publications relevant to the field that are published in other journals or in generalist journals (e.g. PLOS ONE, Science, Nature), we state with Sweileh, Wickramage et al. (2018) that search via keywords in titles and abstracts “is known to retrieve many false positive results” because the keywords “might be also used in other scientific disciplines such as molecular biology, genetics, botany, and veterinary.” Hence, we opt to accumulate a list of journals relevant in the field. Therefore, we relied on the work of Pisarevskaya et al. (2020), who pursued an inductive approach to publications from 40 English-language migration journals and four book series to conduct computer-based topic modeling in migration studies, tracing changes over time. Pisarevskaya et al. (2020) collected metadata originating from Scopus (94%) and Web of Science (1%), and the remaining (5%) were gathered manually, resulting in 29,844 articles. Hence, relying only on Scopus will certainly cover the bulk of the articles (we provide the full curated list of journals plus number of articles for each journal in a repository: https://doi.org/10.5281/zenodo.5113133). Pisarevskaya et al. (2020) observed the period from 1985 to 2017, for two reasons: starting from the mid-1980s, the field of migration studies became more international, complex and diversified; and there were more than 10 active journals and a sufficient number of them provided abstracts. Hassan, Visvizi, and Waheed (2019), however, assert that the international migration debate had already started in the 1960s. Following this argument, we, too, relied on data from 1960 until November 21, 2020.
For this study, we used the Scopus Search API as well as the Semantic Scholar API for data extraction. There are many other options available: Microsoft Academic Graph, Elsevier ICSR Lab, BigQuery Lab, and Crossref are among them1. We preferred the selected APIs based on ease of reproducibility. Both APIs do not require any approval process and can be easily accessible through a simple signup. The scripts used to extract data are available at https://doi.org/10.5281/zenodo.5113133.
2.1.1. Data extraction, cleaning, and integration
We extracted the data for our corpus in two steps: first, from the Scopus database and then from the Semantic Scholar database. We extracted all articles with basic information using the Scopus Search API. The API allows the user to search all articles by specifying the journal name (using EXACTSRCTITLE). However, we found that it returns articles for other journals if the provided journal is a subpart of any other journal name. For example, “IDENTITIES” can also be part of other journal titles. We excluded all those journals in the data cleaning process. For papers from Scopus, we were able to extract articles with titles, first author, affiliation of the first author, and Digital Object Identifier (DOI); but due to the unavailability of an institutional account, we were unable to get data for other authors, abstracts, and citations. To compensate for this, we used the Semantic Scholar API, which provides this information. We used the DOI to map the records for both databases. Figure 2 shows our high-level architecture for data extraction.
2.1.2. Network generation
For the generation of networks, we used the data from both sources. The affiliation of a paper with institution and country is decided through the affiliation data of the first author retrieved from the Scopus database. The Semantic Scholar database was used for authors, publication year, citations, and references. We used the iGraph library to generate these networks, as the library is available in Python and R languages. Gephi software was used for visualization of most of the graphs.
We generated the following networks from the data:
The Collaboration Network shows who is collaborating with whom. Each node represents an author, and an edge between two authors shows that the authors have generated a collaborated work where the weight of an edge depicts the number of such collaborations.
In the Citation Network each node is either an article published in the selected journals or a paper not published in the selected journals but citing an article published in the selected journals. Each edge from an article x to another article y represents that x cited y. We generated another citation network in which we only considered articles published in the selected journals. Hence, this network shows who is citing whom within the core-migration journals. We call the former network extended articles and the latter base articles.
The Journal Citation Network shows journals citing articles of other journals. An edge in the network from node J1 to node J2 (representing two journals) depicts an article published in journal J1 that has cited an article published in J2. The network also contains self-loops showing citations from articles of a journal to articles published in the same journals. The weight of the edge shows the number of such citations between the two journals.
The Country Citation Network is similar to the journal citation network. We generated the country citation network in which, instead of journals, each node represents a country. An edge from node C1 to node C2 shows articles published by the first author belonging to a country C1 citing another article whose first author belongs to a country C2. The weight of the edge shows the number of such citations between the two countries.
The use of multiple data sources also has its disadvantages. One of the major disadvantages is the unavailability of similar data in both sources. We were able to extract 35,405 articles from Scopus databases but out of these only 32,450 articles were available in the Semantic Scholar database. Similar to other studies (e.g., Sweileh et al., 2018), we separated certain types of publications such as reviews, editorials, and conference proceedings. We included only those described as “journal articles” by the Scopus database and that included an author and references, which resulted in 27,433 articles—the most extensive corpus yet with regard to a quantitative analysis of authors, papers, and citations (to the best of our knowledge) while avoiding false positives (i.e. articles from other fields). This big corpus also distinguishes this study from former ones (e.g., Hassan et al., 2019) that analyzed around 12,000 articles.
We refer to these 27,433 articles as “base articles”; they build the basis for a number of information graphs generated such as the collaboration network, citation network, and country citation network. Yet, to generate the citation network, we also included articles not published in the list of journals we used to extract the “base articles,” but these articles cited at least one paper included in the base articles; we refer to these as extended articles (level 1 citations). However, we did not include citations of the “extended articles” (level 2 citations) as they may extend beyond the rim of the field’s center.
2.2. A Novel Framework for Impact Metrics in Interdisciplinary Phenomenon-Oriented Research Fields
Due to the comparison with the most influential papers in their subject field of origin, the resultant metrics will have low absolute values, but these values are fit for direct comparison between different papers, authors, or journals in migration studies. We also calculate the average values for migration studies. For authors from a field whose average citation count is below the average count of migration studies, venturing into the field may be advisable career-wise. Moreover, collaborating with authors from fields with higher average citation count may also be a good idea from that standpoint.
Papers with a higher citation count will ceteris paribus exhibit a higher MigImp to acknowledge the overall citation numbers of a paper. At the same time, the specific form of the MigImp yields especially high values for papers with a high proportion of their citations from within migration studies. Hence, the proposed metric emphasizes within-field citations so that those papers are acknowledged that concentrate on furthering migration research at the cost of their own general visibility (e.g., by providing specific theories, methods, or data). The Pearson Correlation between the MigImp of a paper and its citation count for our corpus is 0.52 and between MigImp and PageRank (Chen, Xie et al., 2007; Yan & Ding, 2011) it is 0.58. This rather moderate correlation (explored in detail in the next section) means that our metric grasps an additional quality of a paper beyond these approaches: It focuses specifically on the impact in an interdisciplinary field and can thus complement the aforementioned analyses. Together MigCite and MigImp can uncover whether the relative success of a paper/author is fueled by its/their importance in migration studies or citation practices in the author’s field of origin. MigCite is meant for close inspection of a given author and MigImp characterizes migration studies as a research field. We identified the field of origin via the authors’ self-definition on the web and if we encountered disciplinary overlaps we decided to use the field in which most of the author’s papers were published.
3. RESULTS
3.1. Basic Overview: Articles, Citations, Journals, Papers
Publications on migration date back to the 1960s, when the journal International Migration, the outlet publishing the fourth highest number of migration studies articles was founded. However, in those early days, only a few articles tackled this topic, as shown in Figure 3 (we are showing data since 1986 for better visualization, as 90% of the articles were published after 1985). Contrary to the linear growth in the number of publications over time identified by Sweileh et al. (2018, p. 10), we find an exponential increase in the number of articles. In general, the growth of publications mirrors the general growth of global research activities.
This growth was potentially driven by the establishment of additional influential journals, both in terms of published articles on migration (cf. Table 1) and number of citations in migration articles (cf. Table 2). These journals were founded after 1960, with the majority in the 1970s and 1990s after the fall of the Iron Curtain. We observe that 34% of articles are published in the four most prominent journals. This seems to be true also for other subdisciplines, such as agricultural economics (Popp et al., 2016, p. 157). On average, there are 810 articles published each year, with a maximum of 1,562 in 2017. This, of course, depends on the frequency of journal issues per year and the average number of articles integrated into each issue and volume. Citations mirror the exponential development in published papers, delayed by about two decades—this delay becomes shorter and shorter as the numbers of citations and the number of papers keep rising.
Rank . | Journal . | Total articles . | Total citations . | Avg. authors per article . | Avg. citations per article . | Total MigImp . | Avg. MigImp per article . |
---|---|---|---|---|---|---|---|
1 | Journal of Ethnic and Migration Studies | 2,889 | 76,054 | 1.58 | 26.3 | 1,788.0 | 0.619 |
2 | Ethnic and Racial Studies | 2,508 | 88,911 | 1.41 | 35.5 | 1,042.0 | 0.416 |
3 | Journal of Black Studies | 1,684 | 27,671 | 1.38 | 16.4 | 48.1 | 0.028 |
4 | International Migration | 1,674 | 40,491 | 1.63 | 24.2 | 505.0 | 0.302 |
5 | International Journal of Intercultural Relations | 1,624 | 82,238 | 2.39 | 50.6 | 430.0 | 0.265 |
Rank . | Journal . | Total articles . | Total citations . | Avg. authors per article . | Avg. citations per article . | Total MigImp . | Avg. MigImp per article . |
---|---|---|---|---|---|---|---|
1 | Journal of Ethnic and Migration Studies | 2,889 | 76,054 | 1.58 | 26.3 | 1,788.0 | 0.619 |
2 | Ethnic and Racial Studies | 2,508 | 88,911 | 1.41 | 35.5 | 1,042.0 | 0.416 |
3 | Journal of Black Studies | 1,684 | 27,671 | 1.38 | 16.4 | 48.1 | 0.028 |
4 | International Migration | 1,674 | 40,491 | 1.63 | 24.2 | 505.0 | 0.302 |
5 | International Journal of Intercultural Relations | 1,624 | 82,238 | 2.39 | 50.6 | 430.0 | 0.265 |
Rank . | Journal . | Total articles . | Total citations . | Avg. authors per article . | Avg. citations per article . | Total MigImp . | Avg. MigImp per article . |
---|---|---|---|---|---|---|---|
1 | Population and Development Review | 1,050 | 93,155 | 1.82 | 88.72 | 150.0 | 0.143 |
2 | International Migration Review | 1,146 | 77,207 | 1.74 | 67.37 | 1,021.0 | 0.891 |
3 | Journal of Studies in International Education | 539 | 28,994 | 1.82 | 53.79 | 67.5 | 0.125 |
4 | International Journal of Intercultural Relations | 1,624 | 82,238 | 2.39 | 50.64 | 430.0 | 0.265 |
5 | Cultural Diversity and Ethnic Minority Psychology | 934 | 43,474 | 3.53 | 46.55 | 68.9 | 0.073 |
Rank . | Journal . | Total articles . | Total citations . | Avg. authors per article . | Avg. citations per article . | Total MigImp . | Avg. MigImp per article . |
---|---|---|---|---|---|---|---|
1 | Population and Development Review | 1,050 | 93,155 | 1.82 | 88.72 | 150.0 | 0.143 |
2 | International Migration Review | 1,146 | 77,207 | 1.74 | 67.37 | 1,021.0 | 0.891 |
3 | Journal of Studies in International Education | 539 | 28,994 | 1.82 | 53.79 | 67.5 | 0.125 |
4 | International Journal of Intercultural Relations | 1,624 | 82,238 | 2.39 | 50.64 | 430.0 | 0.265 |
5 | Cultural Diversity and Ethnic Minority Psychology | 934 | 43,474 | 3.53 | 46.55 | 68.9 | 0.073 |
Table 1 shows that, for instance, the articles that appeared in the International Journal of Intercultural Relations are often cited (in general), but that their impact within the distinct field of migration research is not as high, for example, as the MigImp of the Journal of Ethnic and Migration Studies.
Overall, our data shows that the field is growing rapidly, not only because of general advances in academia, but moreover potentially caused by the rising number of migrants (Sweileh et al., 2018), a rising public and political awareness of migration issues, soaring globalization, and international relations. Thus, migration papers receive comparatively great attention, as evidenced by their citations. NCite(Migration), which is the average citations of the 31st to 500th most cited papers, as proposed by Patience et al. (2017), is 322.97, whereas the overall average of citations in migration studies is 30.28 and the average of the second and third quartiles in citation ranks is 12.70.
Table 3 lists the most cited papers: The work from 1993 topping that list may have inspired future research on migration that is mirrored in the other influential papers published in the late 1990s or 2000s. This nicely shows that an increase in empirical studies, the rise of the concept of transnationalism, and the increasing institutionalization of migration studies at the beginning of the 1990s (cf. Vertovec, 2020) led to some important conceptual contributions at the end of the 1990s. The foundational works assembled in the table are not single case studies but provide summarizing reviews or theoretical approaches to migration.
Rank . | Title . | Year . | First Author . | Total citations . | MigCite . | MigImp . |
---|---|---|---|---|---|---|
1 | Theories of international migration: A review and appraisal | 1993 | D. S. Massey | 3,710 | 83.33 | 23.77 |
2 | Whose culture has capital? A critical race theory discussion of community cultural wealth | 2005 | T. J. Yosso | 3,203 | 72.80 | 2.20 |
3 | Super-diversity and its implications | 2007 | S. Vertovec | 2,872 | 87.03 | 22.64 |
4 | The Internationalization of higher education: Motivations and realities | 2007 | P. Altbach | 2,152 | 48.91 | 2.08 |
5 | Gender and nation | 1997 | N. Yuval-Davis | 2,146 | 51.10 | 4.29 |
6 | Acculturation: Living successfully in two cultures | 2005 | J. Berry | 2,145 | 26.48 | 6.60 |
7 | The study of transnationalism: Pitfalls and promise of an emergent research field | 1999 | A. Portes | 1,894 | 45.09 | 45.01 |
8 | Conceptualizing simultaneity: A transnational social field perspective on society | 2004 | P. Levitt | 1,854 | 44.14 | 37.59 |
9 | Rethinking assimilation theory for a new era of immigration | 1997 | R. Alba | 1,791 | 42.64 | 24.15 |
10 | Conceiving and researching transnationalism | 1999 | S. Vertovec | 1,510 | 45.76 | 19.82 |
Rank . | Title . | Year . | First Author . | Total citations . | MigCite . | MigImp . |
---|---|---|---|---|---|---|
1 | Theories of international migration: A review and appraisal | 1993 | D. S. Massey | 3,710 | 83.33 | 23.77 |
2 | Whose culture has capital? A critical race theory discussion of community cultural wealth | 2005 | T. J. Yosso | 3,203 | 72.80 | 2.20 |
3 | Super-diversity and its implications | 2007 | S. Vertovec | 2,872 | 87.03 | 22.64 |
4 | The Internationalization of higher education: Motivations and realities | 2007 | P. Altbach | 2,152 | 48.91 | 2.08 |
5 | Gender and nation | 1997 | N. Yuval-Davis | 2,146 | 51.10 | 4.29 |
6 | Acculturation: Living successfully in two cultures | 2005 | J. Berry | 2,145 | 26.48 | 6.60 |
7 | The study of transnationalism: Pitfalls and promise of an emergent research field | 1999 | A. Portes | 1,894 | 45.09 | 45.01 |
8 | Conceptualizing simultaneity: A transnational social field perspective on society | 2004 | P. Levitt | 1,854 | 44.14 | 37.59 |
9 | Rethinking assimilation theory for a new era of immigration | 1997 | R. Alba | 1,791 | 42.64 | 24.15 |
10 | Conceiving and researching transnationalism | 1999 | S. Vertovec | 1,510 | 45.76 | 19.82 |
Applying our metric MigCite to put paper citations in the context of the first author’s field of origin, the ranking among top papers changes because these top papers receive interdisciplinary attention as migration studies papers and not as papers from their authors’ fields of origin. Top papers received only a fraction of their citations from within the field of migration. Thus, the total number of citations is not necessarily a reliable indicator for how much a paper has shaped the migration studies field. This becomes obvious when applying our MigImp metric, which would put the paper with seventh most overall citations on top of the list. However, ranking all papers according to MigImp reveals that the most cited papers retain their top positions with only four less-frequently cited papers entering the top 10 list.
3.2. Influential Journals, Authors, Institutions, and Countries
When examining the number of papers published as well as the number of citations in our corpus, the most influential journals in migration studies tend to stand out. For example, a rather high share of self-citations of Language, Culture and Curriculum might be also due to the fact that it is a highly specialized topic. Concentration on the aspect of language stands in contrast to other migration journals with a wider scope. Thus, referring to articles with this special focus and thereby citing from the journals’ previous articles might be necessary. The same might apply for the second ranked journal Race, Ethnicity and Education, which focuses on educational aspects, whereas for instance the Journal of Ethnic and Migration Studies as well as Ethnic and Racial Studies (cf. Table 1) do not have this limitation.
To identify the most influential people in the field, we studied productivity in terms of not only papers that an author contributes to (cf. Table 4) but also the number of citations that an author’s papers receive (cf. Table 5). Overall, these two aspects are moderately correlated (rPearson = 0.54), indicating that writing more articles strengthens one’s overall recognition in the field of migration. At the same time, publishing a groundbreaking work that receives many individual citations also greatly helps to boost an author’s recognition and overall citation count, as the most cited authors’ high level of average citations shows. Consider for example S. Vertovec, who wrote the fifth most influential paper in terms of MigImp (cf. Table 3) and received the highest number of overall citations in the field. This feature may be especially salient in migration studies because the field is relatively young and because some foundational texts and authors are well recognized among researchers of different disciplinary backgrounds. Some of the most distinguished persons in the field also act(ed) as editors of migration journals (e.g., S. Schwartz and S. Vertovec). Notably, several highly cited researchers have already passed away or are emeriti. This correlates to the list of highly cited papers of which several were published in the 1990s. Table 4 further shows that, for example, Graeme Hugo has more total and average citations, offering a broad portfolio of articles that are widely cited (cf. also Table 5), but that Brenda Yeoh outperforms him when it comes to the impact within the migration research community.
Rank . | Name . | Current institution . | Total articles . | Total citations . | Avg. citations per article . | Avg. MigCite . | MigImp . |
---|---|---|---|---|---|---|---|
1 | M. Banton | University of Bristol | 45 | 440 | 9.78 | 0.23 | 15.1 |
2 | M. Verkuyten | European Research Centre on Migration & Ethnic Relations | 42 | 1,224 | 29.14 | 0.86 | 8.98 |
3 | D. Massey | Princeton University | 39 | 4,937 | 126.59 | 3.01 | 32.6 |
4 | B. S. A. Yeoh | National University of Singapore | 36 | 1,616 | 44.89 | 1.00 | 29.4 |
5 | G. Hugo | The University of Adelaide | 32 | 1,767 | 55.22 | 2.91 | 11.3 |
Rank . | Name . | Current institution . | Total articles . | Total citations . | Avg. citations per article . | Avg. MigCite . | MigImp . |
---|---|---|---|---|---|---|---|
1 | M. Banton | University of Bristol | 45 | 440 | 9.78 | 0.23 | 15.1 |
2 | M. Verkuyten | European Research Centre on Migration & Ethnic Relations | 42 | 1,224 | 29.14 | 0.86 | 8.98 |
3 | D. Massey | Princeton University | 39 | 4,937 | 126.59 | 3.01 | 32.6 |
4 | B. S. A. Yeoh | National University of Singapore | 36 | 1,616 | 44.89 | 1.00 | 29.4 |
5 | G. Hugo | The University of Adelaide | 32 | 1,767 | 55.22 | 2.91 | 11.3 |
Rank . | Name . | Current institution . | Total citations . | Total articles . | Avg. citations per article . | Avg. MigCite . | MigImp . |
---|---|---|---|---|---|---|---|
1 | S. Vertovec | Max Planck Institute for the Study of Religious and Ethnic Diversity | 7,266 | 17 | 427.41 | 12.95 | 80.8 |
2 | C. Ward | CQUniversity Noosa | 5,841 | 22 | 265.5 | 3.64 | 45.1 |
3 | A. Portes | University of Miami | 5,161 | 20 | 258.05 | 10.32 | 114.0 |
4 | D. Massey | Princeton University | 4,937 | 39 | 126.59 | 3.01 | 32.6 |
5 | R. Alba | The Doctorate-Granting Institution of the City University of New York | 4,215 | 23 | 183.26 | 4.36 | 63.0 |
Rank . | Name . | Current institution . | Total citations . | Total articles . | Avg. citations per article . | Avg. MigCite . | MigImp . |
---|---|---|---|---|---|---|---|
1 | S. Vertovec | Max Planck Institute for the Study of Religious and Ethnic Diversity | 7,266 | 17 | 427.41 | 12.95 | 80.8 |
2 | C. Ward | CQUniversity Noosa | 5,841 | 22 | 265.5 | 3.64 | 45.1 |
3 | A. Portes | University of Miami | 5,161 | 20 | 258.05 | 10.32 | 114.0 |
4 | D. Massey | Princeton University | 4,937 | 39 | 126.59 | 3.01 | 32.6 |
5 | R. Alba | The Doctorate-Granting Institution of the City University of New York | 4,215 | 23 | 183.26 | 4.36 | 63.0 |
When moving from author to institution and to country level, we always attribute a paper to the first author’s (current) institution. While this approach helps to avoid the disproportionate impact of multiauthored papers (especially for intrainstitutional collaborations) on institution impact, it may neglect crucial contributions of non-first authors. Unfortunately, we could not retrieve data on other authors due to database access limitations (cf. Section 4.2).
There are 4,074 unique institutions out of which 2,249 (55.2%) published only once. On average, an institution published 5.97 migration studies papers in our observation period with a median of 1. This distribution indicates that most institutions only publish occasionally on migration topics rather than having dedicated migration studies research groups. However, there are some highly productive institutions in terms of migration papers published (listed in Table 6). All of these institutions are universities, indicating that most migration studies research is academic. The University of Oxford ranks first with regard to the number of articles as well the as number of citations. It is one of the few universities that offer an MSc and a DPhil in Migration Studies. Further, this program was designed as a cooperation between two departments (School of Anthropology and Oxford Department of International Development) which itself shows that migration research connects different disciplines and often is a joint effort. Rank 2, the UCLA Center for the Study of International Migration, is fostered by several faculties as well and it offers a minor in International Migration Studies. The minor programs of Critical Migration Studies at the University of Toronto and Global Migration at the University of Amsterdam show that there are academic staff (and not only single researchers) interested in the field. However, when investigating the average citation count of papers published by an institution (cf. Table 7), one finds a mention of the World Bank, which most likely supplies statistical figures from its migration data portal to which researchers refer. The same likely holds for the Max Planck Institute for Demographic Research. Interestingly, for instance, in the study by Sweileh et al. (2018) the UN and WHO presented themselves as prominent actors, too.
Rank . | Institution . | Total articles . | Total citations . | Avg. authors per article . | Avg. citations per article . |
---|---|---|---|---|---|
1 | University of Oxford | 316 | 16,854 | 1.5 | 53.3 |
2 | University of California, Los Angeles | 254 | 13,619 | 2.13 | 53.6 |
3 | University of Toronto | 201 | 7,340 | 1.59 | 36.5 |
4 | Universiteit van Amsterdam | 177 | 6,184 | 1.56 | 34.9 |
5 | University of Sussex | 170 | 7,188 | 1.4 | 42.3 |
6 | National University of Singapore | 148 | 6,436 | 1.71 | 43.5 |
Rank . | Institution . | Total articles . | Total citations . | Avg. authors per article . | Avg. citations per article . |
---|---|---|---|---|---|
1 | University of Oxford | 316 | 16,854 | 1.5 | 53.3 |
2 | University of California, Los Angeles | 254 | 13,619 | 2.13 | 53.6 |
3 | University of Toronto | 201 | 7,340 | 1.59 | 36.5 |
4 | Universiteit van Amsterdam | 177 | 6,184 | 1.56 | 34.9 |
5 | University of Sussex | 170 | 7,188 | 1.4 | 42.3 |
6 | National University of Singapore | 148 | 6,436 | 1.71 | 43.5 |
Rank . | Institution . | Total articles . | Total citations . | Avg. authors per article . | Avg. citations per article . |
---|---|---|---|---|---|
1 | University of New Hampshire Durham | 12 | 2,680 | 1.83 | 223 |
2 | University of Canterbury | 15 | 2,874 | 2.4 | 192 |
3 | University of Greenwich | 18 | 2,651 | 2.39 | 147 |
4 | Max Planck Institute for Demographic Research | 13 | 1,499 | 2.46 | 115 |
5 | The World Bank, USA | 27 | 3,010 | 2.22 | 111 |
6 | National University of Singapore | 148 | 6,436 | 1.71 | 43 |
Rank . | Institution . | Total articles . | Total citations . | Avg. authors per article . | Avg. citations per article . |
---|---|---|---|---|---|
1 | University of New Hampshire Durham | 12 | 2,680 | 1.83 | 223 |
2 | University of Canterbury | 15 | 2,874 | 2.4 | 192 |
3 | University of Greenwich | 18 | 2,651 | 2.39 | 147 |
4 | Max Planck Institute for Demographic Research | 13 | 1,499 | 2.46 | 115 |
5 | The World Bank, USA | 27 | 3,010 | 2.22 | 111 |
6 | National University of Singapore | 148 | 6,436 | 1.71 | 43 |
Aggregating impact on a country level reveals a strong dominance of US and UK institutions, such as the University of New Hampshire Durham, University of Greenwich, Boston College, University at Albany, and Duke University, whose members first-authored almost half of the articles in our corpus. Overall, we observe a strong dominance of countries of the Global North also with regard to number of articles, as Figure 4 shows—dominated by the United States with 8,711 and United Kingdom with 4,389. Both countries are also citing articles from each other, as evident in Figure 5. The figure shows the countries’ citation network (i.e., citations from papers in one country’s institution to papers from another country’s institution). We show only those countries having more citations than the average citations among countries. The size of arrows represents the fraction of citations from another country (cf. Figure 5).
Researchers from 147 different countries participated in the production of the retrieved base articles. The Global North’s dominance in the field suggests an underrepresentation of scholars from the Global South. This could be a manifestation of what Strukelj (2020, p. 177) calls the “receiving-country bias,” meaning that Western scholars prevail in taking the perspective of immigration rather than emigration (this, too, is supported by the results of Sweileh et al. (2018)).
We were able to identify countries that did not generate much publication output but did generate quality output (in terms of citations) by calculating the average citations per article for each country. We considered countries with at least 100 publications in the corpus (cf. Table 8). There are two articles published by authors from Singapore in the year 1999. One of the articles (Ward & Kennedy, 1999) received more than 700 citations. Similarly, for New Zealand, out of four articles published in the year 1990, one article (Searle & Ward, 1990) got 1,220 citations, which makes New Zealand a high-level citation country.
Country . | Number of articles . | Total citations . | Average number of authors . | Average citations per article . |
---|---|---|---|---|
Singapore | 202 | 7,940 | 1.62 | 39.31 |
New Zealand | 230 | 8,971 | 1.89 | 39.00 |
United States | 8,711 | 305,439 | 1.96 | 35.06 |
Austria | 141 | 4,441 | 1.85 | 31.50 |
Canada | 1,499 | 4,719 | 1.80 | 31.49 |
United Kingdom | 4,389 | 13,062 | 1.54 | 29.76 |
Belgium | 299 | 8,657 | 2.17 | 28.95 |
Norway | 251 | 7,235 | 1.79 | 28.82 |
Netherlands | 887 | 25,445 | 1.98 | 28.69 |
Australia | 1,489 | 41,742 | 1.81 | 28.03 |
Country . | Number of articles . | Total citations . | Average number of authors . | Average citations per article . |
---|---|---|---|---|
Singapore | 202 | 7,940 | 1.62 | 39.31 |
New Zealand | 230 | 8,971 | 1.89 | 39.00 |
United States | 8,711 | 305,439 | 1.96 | 35.06 |
Austria | 141 | 4,441 | 1.85 | 31.50 |
Canada | 1,499 | 4,719 | 1.80 | 31.49 |
United Kingdom | 4,389 | 13,062 | 1.54 | 29.76 |
Belgium | 299 | 8,657 | 2.17 | 28.95 |
Norway | 251 | 7,235 | 1.79 | 28.82 |
Netherlands | 887 | 25,445 | 1.98 | 28.69 |
Australia | 1,489 | 41,742 | 1.81 | 28.03 |
3.3. Collaboration Network
In this section, we shed light on how often and with whom the authors collaborated. This is a sign of how interconnected the field is. We assume that there will be fewer collaborations than in other (disciplinarily closed) fields. Thus, this measurement can also be an indicator for other phenomenon-oriented research areas. In addition, we find that the number of authors per article rises over time. On average, there are 1.77 authors per paper and 42% of the articles are the product of collaboration between multiple authors. Hence, even though 58% of the papers were written by single authors, studying collaboration networks seems worthwhile. The whole collaboration network of our data includes 35,102 authors and 41,925 collaboration relations (weighted by frequency of collaboration of two authors) between them, but 27% (n = 9,196) of the authors never collaborated at all, and overall only 6% of authors collaborated more than once. The overall network density of collaborating authors is remarkably low at 0.012%, even compared to other networks of interdisciplinary research fields (e.g., 0.34% for electronic market research (Fischbach, Putzke, & Schoder, 2011)).
Furthermore, similar to Hassan et al. (2019), there are more than 200 disconnected communities containing at least 10 authors, 43 of which contain at least 20 authors. Thereby, the giant component (cf. Figure 6) features 5,491 authors and 12,563 collaboration relations. Within these communities, collaboration is ubiquitous as demonstrated by the high clustering coefficient of 0.71. Migration as a diverse phenomenon prompts research from various perspectives that are still bound to their disciplines. Unlike, for example, advertising that is an interesting field based on somewhat “closer” disciplines, such as psychology, economics, and marketing, the scope of migration is much wider and binds the interest of disperse fields such as geography and communication studies that might only very seldom work in interdisciplinary settings but prefer close disciplinary ties. Hence, the clustering is considerably higher than in the larger, more general fields of Biology (0.066), Mathematics (0.15), or Physics (0.43) (Newman, 2004) and on par with more specialized disciplines such as Statistics (.611), Nanoscience (.776) or Pharmacology (.775) (González-Alcaide, Pinargote, & Ramos, 2020). In contrast, the monothematic interdisciplinary field of Electronic Market Research exhibits an even higher degree of clustering (.875) (Fischbach et al., 2011).
Nevertheless, the case of migration studies also demonstrates the value of interdisciplinary collaboration featured in the giant component of the collaboration network (cf. Figure 6) that consists of 15.64% of authors. Three of the five authors with the highest publication count obtain central positions in the network. For example, M. Verkuyten works at an interdisciplinary research center which even bridges subcommunities of different fields. On the other hand, M. Banton, the most productive author, is not part of this interdisciplinary network. Moreover, four of the five most cited authors are also not frequent collaborators. Hence, homophily and intensive collaboration, especially with one’s students (indicated by a high eigenvector centrality (Xu & Chang, 2020)) or repeatedly with similar authors (Abbasi, Altmann, & Hwang, 2010) but also working in isolation seem viable strategies in a multidisciplinary field such as migration studies. To date, it is not yet possible to judge the long-term success of the most frequent collaborator S. Schwartz (83 collaborations) because his works were published only after 2015.
4. DISCUSSION
4.1. Assessing Research Impact of Phenomenon-Oriented Research Fields
Since the emergence of scientometry, the focus has been on the big disciplines trying to look for a closed set of actors that make up a distinct discipline and identify its leading researchers (e.g., Milman and Gavrilova (1993) for chemical engineering). This is why we consider it important not to stick to the respective discipline but to look for research impact assessment within an interdisciplinary phenomenon-oriented research field. In this article, we proposed a novel framework for assessing the research impact of phenomenon-oriented research by considering the example of migration studies, which currently is at the heart of political debates.
Our analysis employed SKGs to unravel the citation and collaboration structure of the migration studies research field. Regarding collaboration, we find a high level of clustering that indicates ubiquitous collaboration despite the majority of papers being written by single authors. Specifically in migration studies, leading interdisciplinary cooperation projects seem especially beneficial to one’s citation success. However, collaborations are based not only on the proximity of research topics but often on the thematic and geographical proximity of research institutions.
Comparison of research impact across disciplines is often difficult due to discipline-specific collaboration and citation patterns. This specifically affects interdisciplinary research areas and obscures the assessment of the most influential works, authors, or journals in that field. To counteract this, as it relates to citations, we proposed the MigCite-Factor and MigImp metric that discount research impact according to the field that the paper, author, or journal come from. Moreover, the MigCite-Factor identifies the most important papers of the field itself by taking both general visibility and within-field citations into account. One can straightforwardly apply this approach to other interdisciplinary fields. For example, the field of terrorism research fulfills many of the characteristics that we have identified for migration research: It is inter- and multidisciplinary, is relatively young, and does not have a coherent fund of definitions, theories, concepts, and methods. Even though there have been first attempts to apply social network analysis to its research objects, (i.e., terrorist groups; Perliger & Pedahzur, 2011), the researchers’ networks have not been the focus of analysis. Thus, it would be interesting to apply our new research impact assessments and citation procedures to terrorism research.
4.2. Strengths and Limitations
Our research was motivated by the question of how we can reliably measure scientific collaboration, productivity, quality, and impact, if specific field-related structures influence the outcome and extent of scientific or even practical impact (e.g., applications in medical procedures or in migration policy institutions). Our findings support what Fortunato et al. (2018) asserted: “Science often behaves like an economic system with a one-dimensional ‘currency’ of citation counts. […] Science can be improved by broadening the number and range of performance indicators.” We propose at least weighting the classical impact metrics by the disciplinary background of the paper, author, or journal that they evaluate to allow for meaningful comparison in an interdisciplinary field such as migration studies. Working with such weighted measures may also inform the decisions of appointment committees or funding bodies.
Despite our comparatively large corpus, our analysis may still miss important clusters of the migration research debate, as we started from a fixed list of journals and retrieved only those additional articles that were somehow connected to works published in those journals. We aimed for the golden mean of including as many relevant works as possible while avoiding irrelevant ones, but one may weight these two goals differently than we did.
Furthermore, the Scopus database is limited: Non-indexed journals or gray literature did not contribute to our sample. Finally, the validity of our corpus depends on the validity of Pisarevskaya et al.’s (2020) list of journals, as we used this list as a baseline for our own sampling. We are well aware that a change in our sampling strategy could have affected our results; nevertheless, we believe them to draw a valid picture of the field of migration research.
Beside these specific issues, we encountered several problems posed by the limitations of bibliometric databases. First, different spellings of authors’ and institutions’ names, or multiple authors with identically spelled names, authors that sometimes publish with or without including their second name, and name changes after marriage might have negatively impacted data integrity despite our cleaning efforts. This is what initiatives such as ORCID or the Web of Science ResearcherID try to overcome. Furthermore, access to authors’ fields of origin via a database would allow for comparing our MigCite-Factor with the traditional citation count not only for the most cited papers and most important authors but for the whole data set. Second, the methodological implementation of our scientometric study is somehow limited in the sense that the delineation of the field of migration research is performed based on the journal list developed by Pisarevskaya et al. (2020). Thus, we condone that it incorporates any limitations of the delineation performed in Pisarevskaya et al. (2020). Further, we could only retrieve the first author of a paper using the Scopus API. As a workaround, we extracted the other authors via the Semantic Scholar API, which does not provide affiliation data. Third, the scattering of our data across the two databases is also a major obstacle that may lead to failed retrieval of some data sets (missing influential papers in our SKGs as a worst-case). This obstacle is especially exigent for interdisciplinary, phenomenon-oriented research data, as papers are scattered over different journals and appear in different disciplinary backgrounds. In this case, it is of no use to look for simple citation counts that quantify the impact of a paper regardless of its disciplinary or topic-related environment.
4.3. Conclusion and Outlook
At present, research data are more accessible and available without cost than ever before and, due to the massive digitalization of older documents, research from past decades is made readily available. Additionally, there is a growing interest throughout all research communities in retrieving reliable data that allows the individual researcher to rank their work and impact in comparison within the community—a community that is shifting depending on the subject of the research. We assumed that, as inter- and multidisciplinary research fields do not have their own databases and because the number of papers as well as the citations and impact factors vary substantially across research fields, a tailored, context-sensitive representation of the impact of knowledge is needed for these fields. In our opinion, the proposed metric has the potential to provide such a representation. With our paper, we hope to contribute to the efforts of creating synergies between hitherto divided research communities. Our analysis leads to several suggestions for future research: First of all, when assessing impact of papers or authors, we do not weight citations other than by discriminating between citations from within and outside migration studies for MigImp. Hence, putting higher weight on citations from more important papers (Chen et al., 2007) seems a logical next step. Applying our approach to another interdisciplinary field such as terrorism research would not only aid in its further validation but also help in understanding both fields in greater detail.
Even though we assume that the field of migration research is male-dominated, we did not conduct an analysis of gender inequalities in authorship because the metadata did not include authors’ gender. In addition, given the limited availability of full names, we were unable to identify gender and could also not resort to face recognition, for example. Regarding homophily, it would be interesting to see the characteristics of authors that lead to forming the mutual bonds of collaboration. Perhaps it is dependent upon the sociocultural background (US and UK institutions) or related to institutional standing (similar levels of hierarchy between collaborating researchers), or is it that collaborations between male vs. female researchers prevail or is it a question of age?
Another area for further exploration would be to couple our analysis of authors with a topic modeling of their research. This could answer questions such as: How far do specific topics influence research activities? Do we find clusters of researchers around specific subtopics? And how are these connected? Can we identify trending subtopics that always rank high in citations and would be profitable career-wise? It is beyond the scope of this paper to give answers to questions such as these. Yet, we hope to have given a starting point and to have initiated further debate about how to measure research impact in phenomenon-oriented research fields.
ACKNOWLEDGMENT
We developed the initial idea for this project at the Summer Institute in Computational Social Science (SICSS) in Bamberg 2019 (https://sicss.io/2019/bamberg/). Thanks to all for the input we received at the Institute. Thank you also to Lucas Sage for commenting on a draft version of this paper. Another “Thank you” goes to the reviewers who have given us very constructive and detailed feedback.
AUTHOR CONTRIBUTIONS
Liane Rothenberger: Conceptualization, Formal analysis, Investigation, Methodology, Validation, Writing—original draft, Writing—review & editing. Muhammad Qasim Pasta: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Software, Validation, Visualization, Writing—original draft, Writing—review & editing. Daniel Mayerhoffer: Conceptualization, Formal analysis, Investigation, Methodology, Validation, Writing—original draft, Writing—review & editing.
COMPETING INTERESTS
The authors have no competing interests.
FUNDING INFORMATION
No funding has been received by Rothenberger or Pasta. Mayerhoffer has received funding from the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation, grant number 430621735), a self-governed research funding organization, funded by the German public purse that did not influence the content of this paper in any way. This work was supported by the German Research Foundation (DFG) within the funding program Open Access Publishing.
DATA AVAILABILITY
Processed data, high-resolution figures, tables and supplementary material along with guidelines for open reusage are available at the following Zenodo repository: https://doi.org/10.5281/zenodo.5113133.
Note
We are thankful to the reviewers for highlighting a number of these resources.