Explicit or implicit digital humanities? An examination of search strategies to retrieve digital humanities publications from large-scale scholarly databases

https://www.webofscience.com/api/gateway/wos/peer-review/10.1162/qss_a_00321

Since its inception in the 1940s, digital humanities (DH) as a research field has witnessed longstanding, charged, and yet still-unsolved debates and discussions of its nature and boundaries (Gold, 2012; Terras, Nyhan, & Vanhoutte, 2013). It has become even harder nowadays to reach such a conclusion or consensus, as central topics of discussion in this field have been constantly shifting over time (Fiormonte, Chaudhuri, & Ricaurte, 2022; Gold & Klein, 2016, 2019). This is partially due to the broadening of the conceptualization of what DH means, and the fact that the scholarly communities of DH have become increasingly diversified, bridging communities and scholarly groups from computer and information sciences, humanities, and other domains (Kaplan, 2015; Ma & Li, 2021). The growth of DH into an interdisciplinary research domain and the complexity of the intellectual landscape, including its questions of concern, approaches, and methods, have muddied the water even more and created new challenges to define disciplinary boundaries, core knowledge, and methodological orientations of DH as a research domain.

While many scholars tend to explicitly use the phrase digital humanities to conceptualize their scholarly contributions and frame their research agendas, many do not use the word yet still conduct research on closely adjacent and related disciplines and topics. We call the latter type of contributions implicit DH in this article to compare against the notion of explicit DH for the purpose of discussion. Such “implicit DH” is an important part of the story to understand the identities of DH. The early development of DH, then called humanities computing, emerged from adjacent, more traditional research domains of literature and computational linguistics and adopted implicit knowledge and conventions from these areas of inquiry (Svensson, 2009). In the 2010s, the rise of digital humanities as a phrase to refer to this broad area of research brought explicit DH to public attention (Svensson, 2010; Terras et al., 2013). And yet, up to this point, not everyone has the same understanding and acceptance of the phrase as a research identity. A large body of research literature has since then made an effort to develop granular understandings of what topics, disciplines, and methods are under the big tent (Luhmann & Burghardt, 2022; Sula & Hill, 2019; Weingart & Eichmann-Kalwara, 2017). Explicit and implicit DH are also embodied in DH journals. Among the DH journal list, only a small number contain the phrase digital humanities explicitly, while most journals do not bear the name but publish a number of articles on this topic (Spinaci, Colavizza, & Peroni, 2022).

To develop a more comprehensive understanding of DH as a field, it is essential to consider both explicit and implicit DH knowledge. Sula and Hill’s (2019) work offers an example of empirical studies to reveal the “alternative histories” of early DH in addition to the explicit story of “humanities computing,” using early DH journal article data. Operationalizing implicit versus explicit DH at an empirical level by building data set samples that represent both can provide a basis for future empirical explorations of DH research landscape. This need to build more comprehensive and diverse query terms used in scholarly databases is not unique to the topic of digital humanities, but has been witnessed in many other emerging research fields or topics, such as big data (Huang, Schuehle et al., 2015). In the case of DH, two strategies are commonly used to construct an empirical data set. The first is to use a list of representative keywords, as demonstrated by a few recent quantitative works (Chansanam, Ahmad, & Li, 2022; Su, Zhang, & Immel, 2021; Tang, Cheng, & Chen, 2017; Wang, 2018). The second, which is more recent, is to establish a core list of journals representing knowledge in DH (Spinaci et al., 2022). However, the effectiveness of these strategies used alone and their impact on empirical results have not been fully evaluated.

To fill this gap, we aim to construct two data samples to represent implicit and explicit DH, respectively—a sample of DH journal publications acquired from journal match (i.e., Journal sample) and a sample from keyword search (i.e., Keyword sample). Based on these two subsamples, we created an Overlap sample that is composed of all publications in both subsamples. More specifically, we aim to compare the samples from the following specific perspectives, as expressed in the three research questions, to establish a larger and potentially more comprehensive DH journal publication sample in the future and gain a more comprehensive understanding of the attributes of the DH field. We chose to conduct this research using the Dimensions database because of its more comprehensive coverage of publications in social sciences and humanities (Martín-Martín, Thelwall et al., 2021; Singh, Singh et al., 2021), and especially in digital humanities (Spinaci et al., 2022). This makes Dimensions one of the few suitable databases for this research.

Scientific breakthroughs and research advances have tremendously benefited from the availability of large scholarly data sets and digital infrastructures. The open data movement further provided conditions to promote transparency and collaboration in scientific research (Gewin, 2016). In the humanities domain, however, the issue of using reusable, high-quality data sets to facilitate empirical, data-driven DH research has only captured the wide attention of researchers recently (Burrows, 2011; Kelli, Mets et al., 2018). One of the major challenges for DH researchers to engage with empirical, data-driven humanities research is the lack of open, reusable, and high-quality data. Data used in humanities and cultural research often possess varying formats and nature; images, archives, texts, and artifacts can all contribute tremendously to humanities and cultural work (Borgman, 2015; Schöch, 2013). These cultural data are often not well indexed in established digital infrastructures, widely available online, well cleaned, or even machine readable, creating barriers for analysis and further use. To accelerate data-driven DH research, DH communities have made an effort to tackle this issue. For example, emerging DH journals (e.g., Journal of Cultural Analytics and Journal of Open Humanities Data) have dedicated sections to data papers that illustrate the creation and curation of humanities and cultural data sets (McGillivray, Marongiu et al., 2022). These studies have the potential to make DH research more transparent and accessible, maintain accountability for both researchers and funders, promote replication and novel use of shared data corpora, and facilitate interdisciplinary research collaboration.

The difficulty of conducting empirical, data-driven DH research is also attributed to the lack of digital infrastructural support. Scholarly databases, such as the Web of Science and Scopus, serve as the data source to various meta-analysis fields, such as science of science and scientometrics, by offering large-scale, empirical data about scholarly activities and outputs (LaRowe, Ambre et al., 2009). However, such databases often have limited utility when used to investigate humanities fields, because of their biases towards STEM domains and, in particular, the limited coverage of publications in humanities (Singh et al., 2021; Sivertsen & Larsen, 2012). Behind this finding is the fact that, compared with sciences and engineering research (and to a lesser degree, social sciences), humanities research has a much heavier reliance on multiple academic languages and venues beyond academic journals, especially books (Archambault, Vignola-Gagné et al., 2006; Engels, Ossenblok, & Spruyt, 2012; Nederhof, 2006; Sivertsen & Larsen, 2012). As a result, researchers are often unable to retrieve a sample of publications that is representative of humanities fields using these English- and journal-centered tools (Archambault & Larivière, 2010).

Some recent efforts strive to develop new research tools to facilitate quantitative studies on DH. Among them, Spinaci, Peroni, and Colavizza (2020) and Spinaci et al. (2022) developed a list of core DH journals and analyzed their coverage in standard scholarly databases, which is the first step in mapping DH from our bibliographic universe. Moreover, a new index of DH conferences has been established to trace DH conferences and their publications, which are considered an important communication venue in this field (Lincoln, Weingart, & Eichmann-Kalwara, 2021). In addition to compiling DH samples with representative journals and conferences, researchers have also used scholarly databases (despite their limitations) and quantitative methods to understand the attributes of DH. Researchers have studied cross-discipline and cross-country collaboration patterns, research themes, and media type in DH publications using standard scientometric methods (Su, 2020; Su et al., 2021; Sula & Hill, 2019; Tang et al., 2017; Wang, 2018). More recently, researchers have also applied topic modeling methods to understand the intellectual structure, and particularly the relationship between DH and other research fields, in a more granular manner (Callaway, Turner et al., 2020; Luhmann & Burghardt, 2022). All these studies have demonstrated the usefulness of scholarly databases and quantitative methods to understand DH from a broader perspective.

However, a limitation remains in such quantitative studies regarding the query strategy to be used, given the different coverages of using journals and keywords to select DH publications. Most existing research often uses a list of keywords to retrieve DH publications from large-scale scholarly databases (Chansanam et al., 2022; Su et al., 2021; Tang et al., 2017; Wang, 2018), without considering their theoretical implications and impact on compiling a representative or comprehensive DH data set. In addition, there has also been a lack of research that systematically evaluates the effectiveness of selected keywords for retrieving DH publications, as well as the approach’s conceptual contributions to our understanding of what DH entails and encompasses. Given the emergence of both journal and keyword search approaches, it is vital to understand the differences between these two methods, particularly the pros and cons of each, in order to develop a strategy for constructing DH publication samples for quantitative research. This is the direct motivation of the present research.

Closely related to the difficulty of identifying high-quality DH data sets is the ambiguity of the intellectual identities and extensions of the phrase digital humanities. The explicit DH field that we know today is deeply rooted in various implicit research traditions, disciplines, and approaches. Such implicit knowledge consists of the nuanced, complex landscape of the digital humanities research domain. Research literature has touched upon the explicit and implicit DH from multiple perspectives of historical development, terminological orientation, topical focus, and scholarly communities.

What is currently called digital humanities emerged from several established, traditional disciplines such as literary studies and history during its earlier period of development (Schreibman, Siemens, & Unsworth, 2004). Building upon the early terminology of humanities computing, which had a strong emphasis on the use of computer assistance and quantitative analysis of text, digital humanities was raised as an umbrella label to capture the expansion of fields and research areas adjacent to humanities computing (Svensson, 2012a). Since the early days, when this phrase was coined, the debates on the meaning of digital humanities have never ceased (Liu, 2013). The diverse and interdisciplinary nature of DH was further captured by the metaphor of the “trading zone” (Fickers & van der Heijden, 2020; Svensson, 2012b), which embraces the exchange of ideas, approaches, and methodologies in the broadly perceived DH scholarship.

During recent years, the knowledge map of DH has been further diversified, with the increasing engagement of scholars coming from a range of domains and disciplines. Information scientists, for example, have become one of the major contributing communities in the DH workforce (Bradley, El-Assady et al., 2018; Ma & Li, 2021). Scholars trained in different disciplinary traditions have different preferences for research approaches and methods and possess various understandings of what “DH” entails, particularly based on different perceptions of the dynamic between the “digital” and the “humanities” (Alvarado, 2012; Ramsay, 2016). Accompanying the shifting connotations of the phrase DH among scholarly communities is the complex and nuanced topical, intellectual landscape of DH. Luhmann and Burghardt’s research (2022) empirically illustrates the intellectual and disciplinary identities of DH with topic modeling methods. Other recent empirical studies also added to this observation from DH journal analysis and the perspective of scientific inscription use (Ma & Li, 2021; Spinaci et al., 2020). These studies have further demonstrated the complexity of DH from the perspective of knowledge agendas and topical focus. The complex dynamics between the explicit and implicit DH research, as outlined, has created difficulty in tracing its presence in the current research system using standard tools. We aim to address this problem and bridge this gap by obtaining two major samples from the existing digital infrastructures that correspond to implicit and explicit DH research. We will compare the differences and similarities between these samples and explore strategies to construct a more representative and comprehensive DH data set, enabling empirical and quantitative analysis of the DH field.

In this research, we collected two samples of DH publications from the Dimensions database (version: June 2022) housed in the Centre for Science and Technology Studies (CWTS) at Leiden University. As stated in Section 1, we selected this data source due to its comprehensive coverage of DH publications. We collected our samples by using the abovementioned two searching strategies: (a) identifying all publications from the core DH journals compiled by Spinaci et al. (2022) (i.e., Journal sample), and (b) searching for the keyword digital humanities in the textual columns of the bibliographic records, including the title, abstract, and concepts extracted by Dimensions (i.e., Keyword sample). We should note that we only considered this umbrella label for the DH community despite some other potential terms to represent this community or its subcommunities, such as computational humanities or computational linguistics. This decision is partly justified by the fact that the other terms return much smaller samples than the two main terms we used (all smaller than 100). As a result, including them would not significantly alter the paper sample. The final two data sets correspond to implicit and explicit DH contributions, respectively, in our conceptualization. From all downloaded publications, we limited our sample to English-language research articles that were published before 2022.

We aim to understand the disciplinarity of DH publications from the following three aspects. First, we used the paper-level classification adopted by the Dimensions database to understand the disciplinarity of DH publications as well as those that are cited and cite these publications. This classification is automatically assigned to each article using a machine-learning approach (Herzog & Lunn, 2018). Their scheme uses the 22 Fields of Research (FOR) from the 2008 version of Australian and New Zealand Standard Research Classification (ANZSRC)¹. One article can be assigned to multiple categories; as a result, we used the fractional counting method to understand how the publications are distributed across knowledge domains. Our analysis focuses on six of these fields that each have more than 2% of all publications in our overall sample, as shown in Table 5 in Section 4.2. Given recent findings concerning the limitations of using journal-level classification to summarize the discipline of publications (Shu, Julien et al., 2019), we regard this classification to be a relatively accurate description of the disciplinary orientations of DH publications.

Moreover, we also manually classified the discipline of top paper authors who contribute at least five publications in our total analytical sample. There are 105 unique authors selected using the above criterion. To classify the discipline of the author, we first applied the 22 divisions in the ANZSRC scheme to the author data set. Following this step, we added two new divisions of Digital Humanities and Non-academic to the scheme to code authors whose affiliations are intrinsically focused on DH (such as The Center for Digital Humanities at Princeton or other DH-focused labs or departments) or outside of academia (such as commercial companies, libraries, museums, and archives).

Additionally, we conducted a topic modeling analysis using Latent Dirichlet Allocation (LDA) method on the textual fields (i.e., titles and abstracts) of publications with such information, to understand the topicality of the publications on a lower level of granularity than the classification assigned by the databases. In this part of analysis, we removed publications that do not have any title or abstract or those whose title and abstract are primarily in a different language other than English, to avoid noises in the results. To conduct this analysis, we removed common English-language stop words and only focused on the lemmatized form of all words. Moreover, we also used two different evaluation matrixes proposed by Cao, Xia et al. (2009) and Deveaud, SanJuan, and Bellot (2014). Both matrixes suggest that 19 is the optimal number of clusters to be taken from our data. We offered a proposed name for each group based on the top keywords, to give more interpretability to our results. The R-language algorithm used for the LDA analysis can be retrieved from our Figshare page (Li & Ma, 2024).

The groups and representative keywords for all 19 clusters are displayed in Table 1. Despite our effort to remove publications not in English, one cluster (#18) is specifically composed of foreign-language terms. This is due to the fact that in the Dimensions database there are abstracts containing both English and non-English-language words. Given the difficulties of identifying these publications, we decided to retain these records and this cluster.

Our final sample contains 4,774 unique publications that meet our criteria, including 3,130 from core journals and 1,875 from keyword search. It should be noted that there are 231 publications covered by both samples. They are counted as a separate Overlap sample, so that we can make direct comparisons between the two venues. The final publication counts of the three samples are 2,899 (Journal), 1,644 (Keyword), and 231 (Overlap), respectively.

Figure 1 shows the temporal distributions of publications in these three samples. The results show a similar rising trend across the three samples since the late 2000s. This corresponds to the use of the umbrella label digital humanities at the beginning of the 2010s to capture the work practices of the field. Despite the increasing usage of this phrase, DH practices have remained diverse, and its diversity has increased during recent years with more active engagement from researchers from various domains.

Moreover, we also examined how other publications cite and are cited in the three samples. Table 2 summarizes key attributes of all documents that are cited in the samples. About 50% of articles in our sample (2,387 out of 4,774 publications) do not have any citation based on the collected data. For all references that are cited in our total sample, 28,346 unique references are cited 37,474 times. The results demonstrate a rather similar pattern across the three samples.

In comparison, 2,858 unique documents in our sample (59.9%) have been cited in other publications in the Dimensions database. These documents have been cited in 16,808 unique publications 24,795 times. Table 3 shows a summary of how different subsamples are cited. The different mean citation ages across groups are largely due to the very different temporal distributions of their publications. After limiting all groups to those published since 2010, the three groups show very similar citation ages (3.83, 3.95, and 3.63, respectively). Moreover, this also distinguishes the Keyword and Journal groups in terms of the citations received: The mean number of citations received by the Journal group reduces to 7.43 after removing papers published before 2010.

We further analyzed top journals in the Keyword sample. Table 4 summarizes all 12 journals that have more than 10 publications. Given how Dimensions processes conference publications, we did not remove the two conference sources (i.e., Septentrio Conference Series and Proceedings of the Association for Information Science and Technology), one in the domain of scholarly publishing and the other in library and information sciences. For the rest of the 10 sources, six of them are not in the list of Spinaci’s DH journals at all and only one of them was classified as “Significantly DH” in Spinaci et al.’s (2020) system, the level directly following “Exclusively DH,” which is the category that is the most central to DH. This shows that many explicit DH publications are widely distributed in journals that may not have a strong DH identity and using keywords is still an important approach to collecting a comprehensive and diverse subset of DH publications.

We further examined how the three subsamples are distributed across the 22 knowledge domains defined in ANZSRC. In our total sample, only 3,475 (72.8%) of all publications have classification information, which are used in this analysis. Table 5 summarizes how the top six categories are used in our overall sample as well as in the two major subsamples. Despite the similar pattern shared by the three samples, the Keyword sample has more publications from humanities domains (especially Language, Communication and Culture and History, Heritage and Archaeology), whereas the Journal sample is more closely connected to Information and Computing Sciences, Philosophy, and Religious Studies. Given the prominence of the first two domains in Table 5, we can also argue that the Overlap subsample also demonstrates a greater level of DH-ness in terms of the disciplinarity of publications. This is supported by the observation that a greater share of publications (despite the small sample size) originate from these two domains. The different orientations of DH publications from the two sources suggest that humanities researchers, especially those in the areas of language and literary studies, are more prone to using the phrase digital humanities directly to capture and conceptualize their work compared with researchers from other domains, which indicates the historical connections between these concepts.

Table 6 summarizes the distribution of citing and cited documents across the top knowledge domains. Overall, compared with the pattern of publications per se, there is a much stronger similarity between the patterns of citing and cited documents. Moreover, compared with the distribution of DH publications, both citing and cited documents are much less likely to be in the domains of Lang and Phil but more likely to be from those minor knowledge domains, most of which are from social sciences and STEM fields. This suggests the diversity of impacts that DH publications receive and give, despite the identity of this research field.

Figure 2 illustrates how the shares of publications as well as citing and cited documents in each domain varies over the publication years. It shows that the patterns in Table 6 are largely consistent over time and the changes of the three samples are mostly parallel to each other. The only exception is the domain of Phil, which contributed to a lower share of publications in our sample over time.

To understand the topicality of publications from a more granular perspective, we conducted the topic modeling analysis as described in Section 3. Figure 3 illustrates the distribution of the 19 topics in the three subsamples. The graph shows that the two major subsamples have distinct features on some of the topics.

The patterns in Figure 3 are further examined in Figure 4, which shows the position of each topic cluster based on their shares of publications (weights) in the two major subsamples. There is a clear trend that the topics more likely to be used in the Keyword sample are concentrated on introductory, humanistic or conceptual aspects of DH, such as #1 (General Digital Humanities as a Field), #2 (Digital Humanities in Libraries), #6 (Critical Digital Humanities), and #10 (Digital History and Culture), whereas clusters on the other side are more focused on technical and domain-specific topics, such as #5 (Text Encoding), #4 (Text Analysis and Linguistics), #8 (Design & Cultural Heritage), and #12 (Modeling and Imaging). This observation suggests that authors who are concerned with conceptual discussions of DH identities and general approaches tend to explicitly apply the phrase digital humanities, while authors engaging more with implicit DH research tend to focus on a specific aspect—whether it be a discipline-oriented or methodological approach. The findings correspond to the longstanding discussions about the field identities of DH in research literature. As previous work has demonstrated, DH as a field is highly interdisciplinary, evolving, and dynamic, thanks to the engagement of researchers and practitioners from various disciplines. One of the core issues at the heart of DH is the complex and changing relationship between the digital and the humanities, and the answers to this question usually account for the cohesion and separation within the field (Luhmann & Burghardt, 2022; Svensson, 2010, 2012a). As disciplines bring their established conventions, approaches, and methods into the DH domain, such cohesion and separation are manifested differently across disciplines (Ma & Li, 2021). Therefore, we can observe a clear two-camp structure in Figure 4 that highlights the general conceptual and domain-specific approaches to DH, respectively. Simultaneously, we can also see that there are topics such as #15 (Big Data & Humanities) and #14 (GIS and Spatial Humanities) that demonstrate a stronger combination of the technical and the humanistic, suggesting a higher level of integration for certain topics and areas within DH.

To further explore the workforce and communities that drive the temporal change and disciplinary orientations of the DH domain, we also investigated the authorship distribution and the collaboration patterns among the authors represented in our samples. Table 7 summarizes the author-level information from the three samples. A few observations can be drawn from the table. First, it shows that the three samples have relatively similar authorship and collaboration patterns. Second, the fact that the Journal sample has more authors per paper and fewer single-authored articles can be attributed to the fact that more Journal articles are from the domain of Information and Computing Sciences. Third, there is only a very small overlap of unique authors across the three samples, as suggested by the last column of the table. For example, only 241 unique authors are shared across the major samples, accounting for 3.1% of all unique authors from our analytical sample. This points to an interesting observation that authors who publish in exclusively DH journals do not always tend to explicitly label their work as digital humanities, either in titles or abstracts. This also relates to the previous literature that explored community identities of DH, which has suggested that the reception and use of the phrase digital humanist, for example, varies across researchers based on their very different imaginations of the phrase. Such imaginations are often impacted by their disciplinary training, individual research projects and orientations, skill sets, and even their attitudes towards digital methods and technology (Alvarado, 2012; Ma, 2022; Ramsay, 2016). Researchers who do not necessarily self-identify as digital humanists or label their research as digital humanities work constitute the implicit DH workforce and ought not to be neglected when analyzing the intellectual landscape of DH or constructing a data set that empirically represent DH.

Building upon this observation, we further investigated the differences between the implicit and explicit DH communities. We examined the knowledge domains as represented by the affiliation of top authors. Table 8 summarizes the domains of all of the 105 top authors (with at least five publications) using the modified ANZSRC classification system being described above. The results show that the majority of authors comes from the Language, Communication and Culture and Information and Computing Sciences domains, the two major domains represented in the publication sample. Moreover, there is a substantial number of authors from institutions dedicated to DH, echoing what was reported in earlier research (Ma & Li, 2021). For all other papers (the Others category in the table), there are three domains that host more than two authors respectively (in total 10 authors): Philosophy and Religious Studies, History, Heritage and Archaeology, and Non-academic. Moreover, even though the distributions of authors between the Journal and Keyword samples are relatively similar, top authors are more likely to be from Language, Communication and Culture in the Journal sample, which is an opposite trend to the distribution of publications discussed in Section 4.2.

Table 9 summarizes how the publications contributed by top authors from different domains are distributed across different domains. It shows strong consistency with the disciplinarity of authors and their publications by using the same classification system. Moreover, authors from DH-centric institutions seem to combine the profiles of Lang and Info authors in terms of how they publish in the top two domains. Similar to the discussions of Table 5, this again could show that these authors represent a higher level of DH-ness in their publications.

Moreover, we also constructed a collaboration network among all top authors using VOSviewer (Figure 5). The graph shows three main clusters, with the color representing the domain of the authors using the augmented classification system. There is a clear trend that Lang authors, particularly Jennifer Edmond, Claire Warwick, and Ted Underwood, play central roles in the collaboration network, showing the special position of this research tradition in the DH domain. Among the top authors, Lang authors demonstrate strong collaborative relationships within the community, in contrast to Info authors, who collaborate more with top Lang authors than among themselves. The collaboration network represents a clear and strong presence of the scholarly structure of DH, especially the implicit DH.

This research is designed to understand the differences between the two approaches to acquiring journal publications in the field of digital humanities (i.e., journal list and keyword search) which also represent the implicit and explicit perspectives of DH. Our results show that the publication samples from these two sampling methods have some notable differences, which has strong methodological implications for future empirical research on DH publications and sheds fresh light on the research communities within the broadly defined DH domain.

The two samples have very similar temporal distributions, which corresponds to the history of DH as a term and a research field. Digital humanities is a phrase coined and becoming popular from the beginning of the 2010s, which emphasized DH as a highly interdisciplinary research domain with diverse disciplinary and methodological traditions. Both samples show the same pattern of increasing number of publications during the 2010s, despite the fact that the Journal sample covers a much longer history than the Keyword sample.

Second, the two samples have only a small overlap of the publications and authors. Only 231 publications are covered by both subsamples, accounting for 4.8% of the overall analytical sample we acquired. Similarly, only 241 unique authors are found across the two major subsamples, among all 7,697 unique authors we identified. Such findings clearly show that the two approaches to DH journal publications represent two relatively independent research communities, which are only connected by a few key authors. The results also indicate that we cannot rely on one query method (journal or keyword) to retrieve a representative sample of the whole research field of DH; instead, future quantitative analysis on DH should try to combine both query methods, as suggested by Huang et al. (2015) in the case of big data, another emerging, cross-disciplinary research topic.

For the disciplinary orientation, Information and Computing Sciences and Language, Communication and Culture are the two dominant domains in our overall sample as well as individual subsamples, which is consistent with the classic view that DH is a field between humanities and information technologies (Clement & Carter, 2017). However, beyond this consistency, we also find that the Journal sample has more publications from the technology domain while the Keyword sample is more focused on the linguistic domain. Our findings expand existing knowledge about the disciplinary orientation of DH journals (Huggett, 2012) by identifying two different approaches to publishing DH publications in academic journals. Moreover, as two centers of DH publications, we also find that both of the above domains are more representative in the Overlap publications (i.e., those covered in both subsamples) as well as authors from DH-centric institutions, which shows their centrality in the field of DH.

There are, however, notable differences in the distributions of citing and cited documents as well as top authors across these domains. We find that our sample is much more likely to cite and be cited in the technology domain comparing with linguistics (as well as Philosophy and Religious Studies), contrary to the distribution of DH publications. The results seem to indicate that the knowledge base of DH is strongly situated in STEM fields. In comparison, citations from and to DH publications are more likely to be sparsely distributed across various knowledge domains, many of which belong to social sciences. This finding is critical for us to understand the relationship between DH and other related knowledge domains, and particularly the diversity of the knowledge base in DH research.

Related to the disciplinary orientation, we also find important differences in authorship and collaboration between the two subsamples. First, the Journal sample tends to have more authors in a publication, which is consistent with the fact that more publications in this sample are from technological fields (Ma & Li, 2021). Second, there is an interesting gap between the number of authors from each discipline and the importance of these authors. Our analysis reveals that authors from language and linguistics are the most represented among the top authors in our sample and assume more prominent roles in the collaboration network. They predominantly collaborate within their scholarly community but also engage with authors from more technologically oriented fields, such as Info authors. In contrast, Info and DH authors do not exhibit strong collaborative relationships within their respective communities.

Our results offer important and timely insights into the two different mechanisms of producing DH knowledge in journal publications and how these mechanisms are connected to the larger scholarly communication system. This study enables more effective sampling strategies for quantitative studies on the field of DH and the construction of larger publication samples about DH in the future.

As a “big tent” domain, DH encompasses various research traditions, methodologies, and theories. As a result, disciplinarity of DH has been a critical research topic attracting more scholarly attention during the past decade (Huggett, 2012; Luhmann & Burghardt, 2022; Ma & Li, 2021). This research contributes to this line of research by offering a more comprehensive sampling strategy to compile digital humanities journal publications from scholarly databases. More specifically, we achieved this goal by acquiring two publication samples from the Dimensions database using keyword search and a core journal list and comparing the two samples from various aspects. We argued that each sample represents a major distinct conceptualization of DH, namely, the explicit expressions of DH as a domain of its own integrated identity, and the implicit engagement with DH work from a disciplinary perspective. Such an approach helped us better understand the complexity of DH as an interdisciplinary domain, particularly in terms of its knowledge agendas, research traditions, and approaches, as well as the scholarly communities.

Our results show a clear gap between these two mechanisms. Even though both samples show an increase in publication numbers over time, consistent with the history of DH, there is a very small overlap between the two samples in both publications and unique authors, and the two subsamples show mildly different disciplinary orientations as well as authorship patterns. Our findings also show unique communities within DH represented by these two sample strategies, which expands existing research on the disciplinarity of DH and cannot be observed by just using one sampling method. As a result, we recommend that researchers should combine both query strategies for quantitative studies on DH using scholarly databases in the future, so that a more representative and comprehensive sample can be acquired.

Based on our findings, we plan to expand existing research along the following paths. First, we are currently only considering a single umbrella label that represents DH. However, other terms, such as computational humanities, humanistic computing, and computational linguistics, are also frequently used in the same research space. Despite the difficulty of compiling a comprehensive list of keywords used by DH researchers, we plan to use machine learning methods to acquire a fuller list of journal publications that are in the space of DH to cover publications that do not directly use the phrase digital humanities. Second, we plan to use the list of journal publications from our proposed method to compare with conference publications to better understand various topics in a broader landscape of DH. Future research along these lines will contribute to quantitative investigation of the DH domain, more specifically from the aspect of contributing to a more diverse and comprehensive data set of DH scholarly publications. In the long term, by experimenting with search keywords and conducting comparative analyses of multiple retrieved DH samples (e.g., journals, keywords, conference samples), we also aim to develop a better strategy for measuring and evaluating the “representativeness” of DH research samples. This will benefit quantitative studies in digital humanities and researchers conducting similar studies.

The authors would like to acknowledge the generous support of the Institute for Advanced Study (IAS) Collaborative Research Award at Indiana University Bloomington for the completion of this work. Dr. Zhichao Fang is funded by the Scientific Research Funding of Renmin University of China (No. 23XNF037).

Kai Li: Conceptualization, Formal analysis, Investigation, Writing—original draft, Writing—review & editing. Rongqian Ma: Conceptualization, Investigation, Writing—original draft, Writing—review & editing. Zhichao Fang: Conceptualization, Resources, Writing—review & editing.

The authors have no competing interests.

The authors would like to acknowledge the generous support of the Institute for Advanced Study (IAS) Collaborative Research Award at Indiana University Bloomington for the completion of this work.

The data and source code of this work are available from: https://doi.org/10.6084/m9.figshare.25962826.v1.