Exploration and exploitation: Which research strategy are you better at?

https://www.webofscience.com/api/gateway/wos/peer-review/10.1162/qss_a_00342

Nowadays, an increasing number of scholars are devoting themselves to scientific research and publish a huge number of papers yearly. Evaluating and understanding scholars’ academic performance is a crucial research direction in Science of Science (Zhou, Wang et al., 2020), which serves academic contexts, such as grant applications, performance evaluation, and personnel promotion (Wang, Zhou, & Zeng, 2023). Fair and diversified scientific evaluation is necessary for promoting scientific progress (Ain, Riaz, & Afzal, 2019; Sinatra, Wang et al., 2016). The exploration and exploitation dilemma inevitably faced by scholars makes exploration-exploitation the crucial dimension by which to evaluate individual scholars (Yang, Hu et al., 2023).

With the development of big data analysis techniques, data-driven bibliometrics-based evaluation methods have become a powerful supplement to peer review. Bibliometrics-based methods mainly analyze scientists’ research performance from number of citations and papers, citation and coauthorship network, and academic texts. However, traditional author-level evaluation metrics generally evaluate scholars from a single-dimensional view of their impact (Zhang, Wang et al., 2023), cannot measure the innovation level of scientists (Petersen, Riccaboni et al., 2012), and neglect the research focus of scholars. Recent studies show that the exploration-exploitation research strategies present significantly different return and risk characteristics (Jia, Wang, & Szymanski, 2017; Yin, Wang et al., 2019), and scholars show distinct preferences for them (Foster, Rzhetsky, & Evans, 2015; Huang, Lu et al., 2022). Inspired by these studies, this study aims to address the following research issues:

1. propose an explanatory model to untangle the effect of individual-level ability and five research strategies proposed by Huang et al. (2022) on research performance;

2. quantitatively examine the return and risk characteristics of these research strategies; and

3. comparatively analyze scientists’ abilities towards different strategies.

In this paper, we propose a Research Strategy Q model (RSQ) that untangles and quantifies the effect of scientists’ research ability (Q_α) and research strategy ability (⁠$Eαπ$⁠) on their research performance. Q_α is defined as the underlying fundamental ability of a scientist to publish high-quality papers (Huang, Lu et al., 2023b; Sinatra et al., 2016), while $Eαπ$ is defined as the scientist’s proficiency in exploration and exploitation strategies. The five research strategies proposed by Huang et al. (2022) are adopted. We generate synthetic data and separately select about 20,000 scholars in the physics, chemistry, and computer science fields as our data sets. Our results show that our model can effectively uncouple the distinct abilities of a scientist and disclose that different research strategies have obviously distinct benefits and risk features. Moreover, a scholar may have separate proficiency levels for different research strategies (e.g., $EαπM$ ≠ $EαπE$⁠), while a number of scholars may have separate proficiency levels for the same specific research strategy (e.g., $Eα1πM$ ≠ $Eα2πM$⁠). Importantly, exploration and exploitation are not contradictory but complementary from the perspective of proficiency, while they are mutually exclusive from the perspective of selection preference. Our code is open and accessible¹.

This study has the following theoretical and practical implications. The RSQ model is presented to explain how individual research ability and proficiency in five research strategies affect scientists’ research performance. On the one hand, the RSQ model discloses the unique characteristics of different research strategies, and sheds light on the topic selection behavior from the perspective of exploration and exploitation. On the other hand, the RSQ model can be utilized as a novel tool to evaluate scientists. Governments, academic institutions, and funding agencies may employ our model to evaluate scholars, by which they can improve their talent assessment mechanism.

The rest of this paper is organized as follows. In Section 2, we review related studies. In Section 3, we present the problem definition and our model. In Section 4, we clarify our data set. In Section 5, we provide the analysis of experimental results. In Section 6, we conclude with theoretical and practical implications, limitations, and points for future research.

Peer review and bibliometrics-based methods are two mainstream methods for evaluating scholars’ research performance. In peer review, scholars and their publications are manually evaluated by domain experts or peers based on their knowledge and experience. This subjective method is effective in evaluating small numbers of scientists. However, with more and more scholars devoting themselves to scientific research and the increasing volume of publications, evaluation purely based on peer review is not feasible because it is both time and effort consuming (Bornmann & Daniel, 2009a). In addition, peer review is inevitably influenced by subjective factors, such as cognitive distortions (Protasiewicz, Pedrycz et al., 2016). Hence, peer review is often combined with bibliometrics-based methods.

Bibliometrics-based methods generally evaluate the productivity and impact of scholars based on their bibliometric information, such as citation count, number of papers, h-index, and variants of the h-index (Ain et al., 2019). The h-index, proposed by Hirsch (2005), simultaneously measures the productivity and impact of scholars, and shows a certain degree of consistency with peer review (Johnson & Lovegrove, 2008). However, the h-index suffers from some obvious problems. For example, it cannot completely reflect the citation distribution of a scientist’s papers (Bornmann & Daniel, 2009b); it is affected by self-citation (Bartneck & Kokkelmans, 2011); it puts new scholars at a disadvantage; and it is insensitive to changes in research performance (Rousseau & Leuven, 2008). To solve these issues, several variants of the h-index have been proposed. For instance, Egghe (2006) proposed the g-index, defined as the largest number such that the top g papers received at least g² citations. The g-index is more able to take into account highly cited papers. Alonso, Cabrerizo et al. (2010) computed the hg-index by taking the geometric mean of the h-index and g-index. Schreiber (2018) developed a timely h-index, h_t, that only considers a scientist’s publication and citation records of the previous t years. h_t alleviates the time bias caused by the dominance of rather old papers in the h-index. Khurana and Sharma (2022) presented h_c by adding the weight of the highest cited paper to the h-index, which improves the evaluation performance for low-ranked scholars. Ain et al. (2019) examined the differences between the h-index and its citation intensity-based variants, and found that a high correlation exists among most of them. However, these metrics overlook the impact of research topics and strategies on scientists’ research performance.

PageRank-based algorithms have been utilized to rank scientists, in which citation networks and coauthorship networks are constructed. For example, Sidiropoulos and Manolopoulos (2006) ranked scholars based on the average PageRank score of their papers in the citation network. Gao, Wang et al. (2016) combined the h-index and PageRank algorithm to present the PR-index, which provides a more balanced impact measure than existing indices. Yan and Ding (2011) introduced a weighted PageRank algorithm based on a coauthorship network to measure author impact. Zhang et al. (2023) proposed a RelRank algorithm based on the PageRank algorithm, in which a coauthorship network consisting of the author–venue relationship is developed. RelRank can identify high-impact scholars who are devoted to specific publication outlets. However, the above author-level evaluation indicators still evaluate scientists from a single-dimensional view of their impact (Zhang et al., 2023) and cannot evaluate exploration-exploitation performance of scientists.

Sinatra et al. (2016) proposed a novel Q model, which uncouples the effects of productivity, research ability, and luck on scholars’ research performance. The Q parameter estimated by the probabilistic model is a time-invariant indicator that can truly account for a scientist’s individual ability. However, the Q model neglects the effect of research topics. Yang et al. (2023) presented the disruptive h-index and consolidating h-index based on disruption metrics (Funk & Owen-Smith, 2017) to comprehensively evaluate the disruptive and consolidating impact of scientists. However, the disruptive/consolidating h-index is hard to interpret, because the h-index simply combines two numbers of two distinct units. Inspired by their studies, I employ an interpretable probabilistic graphical model to evaluate the exploration-exploitation performance of scholars.

The conflicting demands between productive tradition and risky innovation bring an “essential tension” to scientists (Kuhn, 1977), and scientists inevitably face a dilemma between exploration and exploitation (Cohen, McClure, & Yu, 2007). Consolidating studies (i.e., exploitation) tend to be more easily accepted by the current scientist evaluation system, and therefore have a greater probability of being published (Jia et al., 2017). Innovating studies (i.e., exploration) generally have extremely high time cost and sunk cost, but are more likely to have greater impact in the long run (Yin et al., 2019). Many studies have studied the choice of exploration and exploitation in terms of scientists’ topic selection behavior.

Jia et al. (2017) disclosed three fundamental properties dominating changes in research interests (i.e., heterogeneity, recency, and subject proximity), and also proposed a random-walk-based to reproduce the empirical observations, by which they shed light on the interplay between exploitation and exploration. Huang, Huang et al. (2023a) developed a random walk-based model based on reinforcement learning theory to examine the role of productivity and impact in the evolution of scientists’ research interests, and found that scientists prefer to select topics that help them improve their productivity and impact. Franco, Malhotra, and Simonovits (2014) showed that scholars prefer to submit papers that reject the null hypothesis, to ensure publication. Liu, Wang et al. (2018) and Liu, Dehmamy et al. (2021) examined the career histories of artists, film directors, and scientists, and showed that successful individuals generally undergo extensive exploration before engaging in sufficient exploitation. Zeng, Shen et al. (2019) analyzed the topic-switching dynamics of physicists, and showed that they shift more frequently among topics than they did in the past. Chen and Ding (2023) analyzed the publication records of 117 Nobel laureates in physics, and found that they generally explore about three topics alternately in different periods and focus on exploiting their core topics throughout their careers. Foster et al. (2015) introduced five research strategies (i.e., jump, new consolidation, repeat consolidation, new bridge, repeat bridge) based on chemical entities networks. They demonstrated that the aggregate distribution of research strategies remains extremely stable, and risky innovation strategies are especially rare. Recently, Huang et al. (2022) proposed five novel research strategies under exploration and exploitation behavior; that is, mature topic selection strategy (π_M), popular topic selection strategy (π_P), diverse topic selection strategy (π_D), emerging topic selection strategy (π_E), and combinatorial innovation topic selection strategy (π_C). They also presented a series of metrics to identify the research strategies adopted by a paper. They found that impactful scientists frequently execute π_D, π_P, π_E, π_C but execute π_M less often. The five strategies and corresponding metrics are summarized in Table 1.

Although related studies have achieved fruitful results in understanding exploration and exploitation, to the best of our knowledge, no relevant studies have tried to quantify the exploration and exploitation strategy abilities of scholars. As mentioned, exploration and exploitation strategies have significantly different return and risk characteristics (Jia et al., 2017; Yin et al., 2019), and scholars show distinct preferences for them (Foster et al., 2015; Huang et al., 2022). This study employs five strategies presented by Huang et al. (2022), and proposes an explanatory model by which scholars’ proficiency in terms of different strategies can be decoupled and analyzed.

Previous studies demonstrate that the research ability of a scientist, α, directly or indirectly affects α’s research performance (Li, Zhang et al., 2022; Sinatra et al., 2016), and the research strategy, π, adopted by α also has an impact on his or her research performance (Foster et al., 2015; Huang et al., 2022). To our minds, a scholar may present different proficiency levels for distinct research strategies. For example, some scholars may be better at exploring new topics, while others may be better at consolidating mature topics. In this paper, the research ability of a scientist, α, denoted as Q_α, is defined as a relatively stable underlying ability of α to publish high-quality papers by using their available knowledge (Huang et al., 2023b; Sinatra et al., 2016). High-quality papers are those that made contributions in a certain field and inspired subsequent research, thus usually with high scientific impact. The proficiency level of a research strategy, π, of α is defined as the research strategy ability, denoted as $Eαπ$⁠, which represents the ability of α to make good use of π. Hence, Q_α is mainly related to individual ability, while $Eαπ$ is related to individual ability and the characteristics of the research strategy. Q_α and $Eαπ$ jointly affect scholars’ research performance. Decoupling the relationship among them and quantifying them are interesting and meaningful research questions.

Specifically, the current study aims to untangle and analyze the impact of research ability (Q_α) and research strategy ability (⁠$Eαπ$⁠) on α’s research performance, by which we provide a novel approach for evaluating scholars from exploration and exploitation and shed light on the benefits and risk features of different research strategies. Notably, Q_α and $Eαπ$ are both seen as unobserved inherent attributes of α, and therefore the probabilistic graphical model is a suitable tool to analyze them.

We propose the Research Strategy Q (RSQ) model based on the Q model proposed by Sinatra et al. (2016) and five research strategies (π_M, π_P, π_D, π_E, π_C) proposed by Huang et al. (2022). The generative process of the RSQ model is to model the process of scholars publishing scientific papers through adopting different research strategies. Specifically, for each scientist, α, individual publication records, citation records, and research strategy execution records are generated in the following steps.

• Step 1. For each scientist (α), α’s number of publications (N_α) is sampled from a Poisson distribution, N_α ∼ Poisson(λ), in which λ represents the average productivity of scholars in a specific field.

• Step 2. For each scientist (α), α’s research ability (Q_α) is sampled from a lognormal distribution, Q_α ∼ lognormal(μ_Q, $σQ2$⁠), in which μ_Q represents the average research ability of scholars in a specific field.

• Step 3. For each scientist (α), α’s research strategy ability on a research strategy, π (⁠$Eαπ$⁠), is also sampled from a lognormal distribution, $Eαπ$ ∼ lognormal(μ_π, $σπ2$⁠), in which μ_π and σ_π indicate the benefit and risk features of π. Five research strategies are studied, and therefore there are five $Eαπ$⁠, π ∈ π, π = {π_M, π_P, π_D, π_E, π_C}.

• Step 4. For each paper (i) authored by α, whether a research strategy, π, is executed or not is sampled from a Bernoulli distribution, $1iπ$ ∼ Bernoulli(⁠$pαπ$⁠), in which $pαπ$ indicates the probability of executing π. The Bernoulli distributions of different research strategies are independent of each other.

• Step 5. For each paper (i) authored by α, the quality of the paper (C_α,i) is simultaneously determined by Q_α, $Eαπ$⁠, $1iπ$⁠, and p_i based on a multiplicative process, as shown in Eq. 1. p_i indicates the random factor in i’s quality (i.e., luck), and is sampled from a lognormal distribution, p_i ∼ lognormal(μ_P, $σP2$⁠). $Cαi$ represents the quality of the paper and is indirectly measured by its citations.

  $Cαi=piQα∏π∈πEαπ1iπ1/5$

  (1)

The above generative process is shown in Figure 1. We assume that the productivity (N_α) follows a Poisson distribution, which is suitable for count data. The citations (⁠$Cαi$⁠) follows a lognormal distribution, which has been proven to be reasonable in related studies (Huang et al., 2023b; Sinatra et al., 2016). The research strategy execution variable (⁠$1iπ$⁠) is a binary variable, and therefore the Bernoulli distribution is employed. We follow the hypothesis of the Q model that the research ability (Q_α) and the luck (p_i) follow a lognormal distribution. To make Eq. 1 reasonable, research strategy ability (⁠$Eαπ$⁠) is also assumed to follow a lognormal distribution. Notably, five research strategies—mature topic selection strategy (π_M), popular topic selection strategy (π_P), diverse topic selection strategy (π_D), emerging topic selection strategy (π_E), and combinatorial innovation topic selection strategy (π_C)—are adopted in the RSQ model. The definitions and quantitative indicators of these strategies have been clearly introduced in Table 1. In a nutshell, π_E and π_C respectively represent studying emerging topics and combinatorial innovation of topics, and are exploration strategies. π_M, π_P, and π_D respectively represent studying mature topics, popular topics, and diversified combination of topics, and are exploitation strategies. The identification method proposed by Huang et al. (2022) is used to identify $1iπ$⁠. In short, if the metrics of π calculated based on a paper, i, in Table 1 exceed the median value in the field, it indicates that π has been executed in i; that is, when $1iπ$ equals 1, a scientist, α, publishes i by executing the research strategy, π. Different from the Q model, the quality of a paper is determined by Q_α, $Eαπ$⁠, $1iπ$⁠, and p_i simultaneously, where $Eαπ$ and $1iπ$ are newly introduced random variables. The joint distribution of the above generative process is shown in Eq. 2. Finally, consistent with Sinatra et al.’s (2016) experiments, we also utilize the number of citations of a paper 10 years after publication (⁠$Cαi$⁠) to gauge its scientific impact, which indirectly measures its quality. Other metrics, such as PageRank score (Chen, Xie et al., 2007), novelty score (Luo, Lu et al., 2022), and F1000 (Bornmann & Haunschild, 2015) can also be employed to gauge $Cαi$⁠.

In the RSQ model, there are the observed data (X), hidden variables (Z), and model parameters (θ). The observed data include N_α, C_α,i, and $1iπ$⁠. The hidden variables include Q_α, p_i, and $Eαπ$⁠. The model parameters include λ, $pαπ$⁠, μ_Q, σ_Q, μ_π, σ_π, μ_p, and σ_p. Importantly, N_α and $1iπ$ are independent of the hidden variables, and can be seen as ancillary variables (Blei, Ng, & Jordan, 2003). Hence, their randomness can be ignored. λ and $pαπ$ represent the average productivity of scientists in a specific field and α’s research strategy preference for π. λ, $pαπ$ is not our focus. μ_Q, σ_Q indicate the average and standard deviation of research ability of scientists in a specific field. μ_π, σ_π suggest the benefits and risk features of π. μ_p, σ_p represent the randomness of the quality of a paper, in which μ_p is set to 0. Moreover, p_i can be substituted by Eq. 1, and therefore Q_α and $Eαπ$ are hidden variables that need to be inferred (gray zone in Figure 1). These variables and parameters are summarized in Table 2.

To solve the RSQ model, we employ the BBVI-EM algorithm proposed by Huang et al. (2023b) to approximate the posterior distribution (Eq. 3). The BBVI-EM is a generic variational inference algorithm, in which the variational parameters (ϕ) and model parameters (θ) are iteratively updated by maximizing the Evidence Lower Bound (ELBO) of the log likelihood (Eq. 4).

The actual Q_α and $Eαπ$ of a scholar can never be known in advance, as they are unobserved individual abilities. Similar to our recent study (Huang et al., 2023b), we generate the synthetic data to evaluate whether the RSQ model can untangle the effect of the research ability and research strategy on the quality of papers and effectively quantify Q_α and $Eαπ$⁠. To achieve this aim, the generative process of the RSQ model is used to create the synthetic data, by which the true value of the hidden variables and model parameters can be known in advance.

As shown in Table 3, we introduce three experimental setups for the synthetic data. In simulation 1, the number of authors (∣𝒜∣) is set to 500, and the average number of publications of scientists (λ) is set to 50. Thus, there are about 25,000 papers. For each scientist α ∈ 𝒜, Q_α is sampled from LogNormal(0, 1). For each paper i ∈ 𝒩, p_i is sampled from LogNormal(0, e⁻¹). There is only one research strategy, π₁, and $pαπ1$ is 0.5. Because μ_π1 > 0, this indicates that adopting π₁ tends to improve the quality of the paper $Cαi$⁠. In simulation 2, in addition to π₁, there is another research strategy, π₂ with μ_π2 < 0, which means that adopting π₂ tends to stifle $Cαi$⁠. Finally, in simulation 3, in addition to π₁ and π₂, the research strategy π₃ with μ_π3 = 0 is introduced. π₃ tends to have a neutral impact on $Cαi$⁠. The remaining configurations in simulation 2 and simulation 3 are consistent with those in simulation 1. In addition, we also generate synthetic data by sampling Q_α and $Eαπ$ from student’s t distribution, the chi-square distribution, and the F distribution in our Supplementary material. Under each simulation setting, 10 groups of synthetic data are generated. Overall, simulation 1 is the simplest inferential task, with one strategy, while simulation 3 is the most difficult inferential task, with three strategies. The estimated Q_α and $Eαπ$ through the BBVI-EM algorithm can be compared with their true value.

We employ the Microsoft Academic Graph (MAG)² (Zhao, Bu, & Li, 2021) as our data source, which contains over 200 million papers as of 2020 and generates six levels of “field of study” (i.e., FoS_L0, FoS_L1, …, FoS_L5) (Shen, Ma, & Wang, 2018). We extract papers in the physics, chemistry, and computer science fields to construct three empirical data sets.

Specifically, we use FoS_L0 (“physics,” “chemistry,” and “computer science”) created in MAG to extract 11,714,663, 18,986,736, and 23,315,998 papers, and employ FoS_L2 to represent research topics. Subsequently, we adopt the approach proposed by Huang et al. (2022) to respectively identify the five research strategies (π_M, π_P, π_D, π_E, π_C). Taking a research policy, π_D, as an example, two metrics, Average distance (X_α,i) and Average volume (X_α,i), of a paper, i, are reduced to a scalar, DS_α,i, based on Principal Component Analysis (PCA). If DS_α,i exceeds the median value of DS of papers in a specific field, π_D is adopted in i (⁠$1iπ$ = 1), and otherwise it is not adopted (⁠$1iπ$ = 0). Hence, N_α, $Cαi$⁠, and $1iπ$ are observed data (i.e., empirical data), while Q_α, $Eαπ$ are hidden variables, which we are interested in. The RSQ model connects the observed data with the hidden variables through Eq. 2. We choose scientists who published their first paper between 1990 and 2000 and have published at least 30 papers up till 2010 as our research objects. Hence, for each paper, the number of citations 10 years after publication can be computed in our empirical data sets. The statistics of the three empirical data sets are shown in Table 4. There are 26,992, 26,937, and 17,750 scientists in the physics, chemistry, and computer science fields, respectively. The adoption probability of π indicates $∑i𝒩1iπ$/∣𝒩∣. In the following experiments, we separately conducted experiments on the three empirical data sets.

We use the synthetic data mentioned to evaluate the estimation performance of the RSQ model for quantifying Q_α and $Eαπ$⁠. The variational parameters μ_Q,α and μ_π,α obtained by the BBVI-EM algorithm are used to estimate Q_α and $Eαπ$⁠, respectively. The Q model is employed as a baseline, in which the effect of research strategies is entirely neglected. As shown in Table 5, we report the average results on 10 groups of the synthetic data under each simulation setting. The Pearsonr, R², RMSE, and MAE are utilized to evaluate the estimation performance.

In simulation 1, only one research strategy, π₁, is simulated. Both models give a satisfactory Pearsonr, which means that the estimated value and the real value are highly linearly correlated. However, the RSQ model gets a significantly better estimation performance than the Q model in terms of R², RMSE, and MAE. The t-test is used to test the significant difference, and “***” indicates the significance level (p < 0.001). The bold text indicates the better results.

In simulation 2, two different research strategies, π₁ and π₂, are considered, and our model also gives satisfactory results. The estimated results of $Eαπ1$ and $Eαπ2$ are separated by “/”. For π₁ and π₂ with opposite characteristics, the RSQ model also effectively estimates Q_α and $Eαπ$⁠, and has a significantly better estimation accuracy than the Q model in terms of all metrics.

In simulation 3, three different strategies, π₁, π₂, and π₃, are considered. The RSQ model accurately quantifies $Eαπ$ and Q_α. Specifically, Pearsonr and R² of our model are close to 1, and are obviously greater than those of the Q model. RMSE and MAE of our model are significantly lower than those of the Q model. Notably, as the variety of strategies increases, the performance of the baseline obviously declines, while our model remains almost unchanged.

In our Supplementary material, we report more results on the synthetic data. Our results show that our model can untangle the effect of research ability and different research strategy ability on research performance and accurately estimate them in the synthetic data.

To disclose the general characteristics of the five research strategies (π_M, π_P, π_D, π_E, π_C), we employ the RSQ model to examine three empirical data sets, and use the model parameters (μ_π, σ_π) to quantitatively analyze the benefit and risk features of a research strategy, π. As mentioned above, μ_π represents the average impact of adopting π on the quality of a paper, $Cαi$⁠, and σ_π indicates the standard deviation of this impact. When μ_π > 0, π tends to improve $Cαi$⁠, while, when μ_π < 0, π tends to stifle $Cαi$⁠.

As shown in Table 6, we report the estimated benefits and risk features of π_M, π_P, π_D, π_E, π_C, in three empirical data sets. For mature topic selection strategy (π_M), in the physics, chemistry, and computer science fields, μ_πM is always less than 0, which suggests that studying mature topics is more likely to stifle scientists’ research performance. Indeed, a crucial evaluation criterion for scientific papers is novelty and originality. Mature topics have been thoroughly studied, and therefore limit an individual’s ability to produce breakthrough research (Huang et al., 2022; Liu et al., 2021). Notably, σ_πM gets a large value, which suggests that high-quality studies might also originate from mature topics, such as the revival of deep learning. For diverse topic selection strategy (π_D) in the three fields, μ_πD is also less than 0. Studying topics with diverse knowledge also negatively affects scientists’ research performance. This may be because π_D requires scholars to have rich knowledge reserves and its research process tends to be more challenging. For example, cross-disciplinary research is generally more time-consuming and requires handling high levels of complexity (Schaltegger, Beckmann, & Hansen, 2013; Xu, Ding, & Malic, 2015). For the popular topic selection strategy (π_P) in the three fields, μ_πP always gets the maximum value, which shows that following the academic frontier is the most effective way to enhance academic impact. Moreover, for the emerging topic selection strategy (π_E) and the combinatorial innovation topic selection strategy (π_C), we find that both μ_πE and μ_πC are positive, which suggests that studying emerging topics and conducting combinatorial innovation tend to produce high-quality studies. Previous studies have shown that emerging topics represent research priorities and frontiers (Rotolo, Hicks, & Martin, 2015; Yang, Lu et al., 2022), and combination innovation is a scientific and reasonable innovation mode (Foster et al., 2015; Luo et al., 2022). Notably, σ_πE and σ_πC both take the larger value, which means that the exploration research strategy is a high-risk and high-yield strategy (Liu et al., 2021; Yu, Szymanski, & Jia, 2021).

In conclusion, π_M and π_D are weakened strategies with μ_π < 0, which tend to stifle scientists’ research performance, while π_P, π_E, and π_C are enhanced strategies with μ_π > 0, which tend to improve scientists’ research performance. The model parameters of the RSQ model present a novel approach to disclose the unique characteristics of different research strategies, which helps researchers to understand the topic selection behavior from the perspective of exploration-exploitation.

The model parameters, μ_π and σ_π, reflect the overall characteristics of a research strategy, π, based on all scientists in the empirical data sets. However, for each scientist, α, the research strategy ability $Eαπ$ is a sample. α may have research strategies that α is good at or not good at. The RSQ model presents a new approach to evaluate scientists from the perspective of research ability (Q_α) and research strategy ability (⁠$Eαπ$⁠).

First, we employ Pearsonr to analyze the relationship among Q_α, $Eαπ$ and some traditional author-level evaluation metrics (i.e., N_α, C_α, $Cα*$⁠, H_α). N_α, C_α, $Cα*$⁠, and H_α respectively denote the number of papers authored by α, total number of citations acquired by α’s papers, number of citations of α’s most-cited paper, and h-index of α. As shown in Figure 2, in the three fields, Pearsonr between any $Eαπ$ and Q_α is lower than 0.6. This means that our model, to some extent, untangles the effect of $Eαπ$ and Q_α on α’s research performance, and therefore $Eαπ$ and Q_α measure the individual abilities in different aspects. In addition, there is a certain linear correlation among $Eαπ$ of different research strategies. To be specific, $EαπE$ and $EαπC$ are highly correlated (over 0.70), as they both measure the exploration strategy ability. $EαπM$ has the lowest correlation with $EαπE$ and $EαπC$ (about 0.24 and 0.17), because it mainly measures the exploitation strategy ability. The low positive correlation also shows that exploration and exploitation are not contradictory but complementary from the perspective of proficiency (Zacher, Robinson, & Rosing, 2016). $EαπP$ is highly correlated with $EαπE$ and $EαπC$⁠. This may be because scholars who are good at following research frontiers are better at innovating. There is a certain correlation between $EαπD$ and other $Eαπ$⁠, as diverse topic knowledge involves various types of topics (Huang et al., 2022). Finally, Q_α and N_α are almost unrelated. The Pearsonr values between Q_α and C_α, $Cα*$⁠, and H_α are only about 0.39, 0.30, and 0.40, which also shows that Q_α can serve as a supplementary indicator for traditional metrics (Sinatra et al., 2016). Moreover, the five $Eαπ$ are also unrelated to N_α, and have a moderate correlation with C_α, $Cα*$⁠, and H_α. Therefore, $Eαπ$ and Q_α can be used as the complementary author-level metrics.

Subsequently, we also used Pearsonr to analyze the relationship among Q_α, $Eαπ$⁠, and research strategy preference (⁠$pαπ$⁠), as shown in Figure 3. $pαπ$ denotes the ratio of $Nα,π+$ to N_α, in which $Nα,π+$ indicates the number of times α adopts π in his or her N_α papers. As shown in Figure 4, we find an interesting result; that is, $Eαπ$ is nearly unrelated to $pαπ$⁠. This means that scholars who prefer to execute a specific research strategy π may not be good at executing π, which further shows the unique attribute of $Eαπ$⁠. Q_α is also unrelated to $pαπ$⁠. In addition, consistent with our previous results (Huang et al., 2022), $pαπM$ is negatively related to $pαπE$ and $pαπC$ in the three fields, which suggests that choosing between exploration and exploitation is an inevitable dilemma for scientists.

To further check the predictive power of Q_α and $Eαπ$⁠, we conduct regression analysis, in which Q_α and $Eαπ$ are employed as the independent variables, while traditional author-level metrics (N_α, C_α, $Cα*$⁠, H_α) are respectively employed as the dependent variable. Specifically, in our regression analysis, we only use scientists’ papers published before 2000 to train the RSQ model, and use their papers published before 2010 to calculate N_α, C_α, $Cα*$⁠, and H_α. Thus, this regression analysis examines the predictive ability of Q_α and $Eαπ$⁠. As shown in Tables 7, 8, and 9, we report the regression coefficients in three empirical data sets. When log N_α (the logarithm of N_α) is the dependent variable, R² is especially lower (less than 0.1), which also suggests that Q_α and $Eαπ$ are linearly unrelated to future productivity of α. However, when logC_α, log$Cα*$⁠, and logH_α are the dependent variables, the maximum value of R² can reach 0.48. The coefficients of Q_α and the five $Eαπ$ are nearly always significantly positive. This means that Q_α and $Eαπ$ can be used to predict the future academic impact of α. The negative value of coefficients (e.g., the coefficient of $EαπD$⁠) is caused by the multicollinearity problem. As mentioned above, these research strategy abilities have a certain degree of linear correlation.

Then we randomly select four case scientists (α₁, α₂, α₃, α₄) in each field, and display their $Eαπ$ in the radar chart, as shown in Figure 4. $Eαπ$ have been normalized by $Eαπ−minEαπ/maxEαπ−minEαπ$⁠. As shown in the left subfigure of Figure 4, α₄ (green) is especially good at executing π_M and π_D, while she or he is not good at π_P, π_E, and π_C. Hence, compared to exploration, α₄ is better at exploitation. In contrast to α₄, α₃ (purple) is good at π_P, π_E, and π_C, while she or he is not adept at π_M and π_D. Thus, α₃ is skilled at exploration. However, α₁ (orange) presents a relatively balanced ability in terms of the five research strategies, and α₁’s radar chart is like a pentagon. Hence, α₁ can balance exploration and exploitation. Moreover, α₃’s radar chart (red) is completely wrapped by α₁’s radar chart, which means that α₃’s abilities to conduct the five research strategies are weaker than those of α₁. The values in parentheses of the legend represent the standardized Q_α. We also find that Q_α1 > Q_α3. For the case study in the chemistry and computer science fields, we find similar results. In conclusion, one scholar may have distinct proficiency levels for different research strategies (e.g., $EαπM$ ≠ $EαπE$⁠), while a number of scholars may have distinct proficiency levels for the same research strategy (e.g., $Eα1πM$ ≠ $Eα2πM$⁠). Hence, $Eαπ$ can work as a novel metric to evaluate scholars from the perspective of exploration and exploitation.

Finally, we also present a case for guiding how to use $Eαπ$ and Q_α. Taking scientists in the physics field as an example, as shown in Table 10, scholars are ranked according to Q_α or $Eαπ$ in descending order. The governments, academic institutions, and funding agencies may refer to the ranking list to select scholars who are suitable for completing their research projects. For example, engineering projects requiring scholars to proficiently exploit mature knowledge can be handed over to scholars with a low rank of $EαπM$⁠, $EαπP$⁠, and $EαπD$⁠, while scientific research projects requiring scholars to creatively explore new knowledge can be assigned to scholars with a low rank of $EαπE$ and $EαπC$⁠.

This study analyzes the impact of the research ability (Q_α) and research strategy ability (⁠$Eαπ$⁠) on scientists’ research performance through a novel RSQ model, in which Q_α indicates the fundamental ability of a scientist α, while $Eαπ$ describes the exploration and exploitation strategy ability of α. The BBVI-EM algorithm (Huang et al., 2023b) is utilized to solve the RSQ model. We use synthetic data and three empirical data sets in the physics, chemistry, and computer science fields to check the effectiveness of our model, and disclose the characteristics of five research strategies proposed by our previous study from the individual and overall perspectives. Overall, our model can not only be taken as a novel author-level evaluation tool, but it also sheds light on topic selection behavior in terms of exploration and exploitation. Here we provide a detailed summary of the theoretical and practical implications, as well as the limitations of our research.

This article has the following theoretical implications. First, we propose a novel probabilistic graphical model, the RSQ model, which explain how scientists’ research ability and proficiency in different research strategies affect their research performance. Based on synthetic data and empirical data, we show that our model can untangle the relationships among research ability (Q_α) and different research strategy abilities (⁠$Eαπ$⁠), and effectively estimate them. Compared with the Q model (Sinatra et al., 2016), overlooking the impact of research strategies, our model achieves a better estimation performance of Q_α, which suggests that the coupling relationships among Q_α and $Eαπ$ should be considered.

Second, we employ the model parameters of the RSQ model to disclose the overall characteristics of five research strategies (π_M, π_P, π_D, π_E, π_C) and shed light on the topic selection behavior from the perspective of exploration and exploitation. Specifically, based on our empirical data sets, our experimental results show that studying mature topics (π_M) tends to stifle scientists from producing high-impact papers, because scientific research requires innovation. Studying diverse knowledge topics (π_D) also stifles scientists’ performance, because diverse knowledge generally requires the handling of high levels of complexity (Schaltegger et al., 2013; Xu et al., 2015). However, following the academic frontier (π_P) is the most effective way to enhance the academic impact. Exploration research strategies (π_E and π_C) are high risk and high yield (Liu et al., 2021; Yu et al., 2021). In conclusion, π_M and π_D are weakened strategies which tend to stifle scientists’ research performance, while π_P, π_E, and π_C are enhanced strategies which tend to improve their research performance.

Third, we employ the variational parameters introduced by the BBVI-EM algorithm to analyze the author-level abilities through quantifying Q_α and $Eαπ$⁠. We check the linear correlation among Q_α, $Eαπ$ and some traditional author-level evaluation metrics. We find that Q_α and $Eαπ$ are linearly unrelated to the productivity (N_α), and have a slight correlation with the accumulative citations (C_α), citations of the most-cited paper (⁠$Cα*$⁠), and the h-index (H_α), which means that they can be used as complementary metrics for author-level evaluation. We also analyze the relationship among Q_α, $Eαπ$⁠, and $pαπ$⁠, and find that scholars who prefer to execute a specific research strategy, π, may not necessarily be good at executing π. Importantly, our research thoroughly explains the relationship between exploration and exploitation; that is, exploration ability (⁠$EαπE$⁠, $EαπC$⁠) and exploitation ability (⁠$EαπM$⁠, $EαπD$⁠, $EαπP$⁠) are positively correlated, indicating that scholars who are good at exploiting are often good at exploring, and vice versa. Indeed, only scholars who are good at exploring can understand and exploit cutting-edge knowledge, and only scholars who are good at exploiting have time to explore new knowledge. Therefore, exploration and exploitation are not contradictory but complementary (Zacher et al., 2016). However, the exploration preference (⁠$pαπE$⁠, $pαπC$⁠) and exploitation preference (⁠$pαπM$⁠) are negatively correlated, which suggests that choosing between exploration and exploitation is an inevitable dilemma for scientists (Cohen et al., 2007). This is due to the limited attention of scholars, who need to make a choice between exploration and exploitation during their working hours. In addition, Q_α and $Eαπ$ have predictive power for forecasting scholars’ future research performance. Finally, we show that one scholar may have sperate proficiency levels for different research strategies, and different scholars may have distinct proficiency levels for the same research strategy.