Political hegemony, imitation isomorphism, and project familiarity: Instrumental variables to understand funding impact on scholar performance Open Access

https://www.webofscience.com/api/gateway/wos/peer-review/10.1162/qss_a_00359

In the winter of 1978, the outbreak of the Islamic Revolution in Iran resulted in an oil crisis that spread from Tehran to the rest of the world, causing inflation and economic depression in a wide range of countries, including the United States. A few months later, Franco Modigliani from MIT became one of the first social and economic scholars to receive funding from the National Science Foundation, a $150,000 grant to assist governments and the scientific community in understanding the monetary mechanism and stabilization policy during the crisis. In the subsequent decades, nations have kept investing in social sciences with the expectation that their research will contribute to economic growth and societal advancement. Nonetheless, the public endlessly debates whether investment in academia will eventually contribute to the discovery and dissemination of knowledge. Early in the 20th century, the development of electronic and open access movements (Evans & Reimer, 2009) in the literature increased the possibility of quantifying and assessing the productivity of science, necessitating the theorization and investigation of the beyond correlation relationship between research funding and scientific productivity.

In practice, the role of research funding in the day-to-day research of academics is being debated. Generally, most scholars claim that investment in science affects scientific labor (Crespi & Geuna, 2008). In other words, research funding is expected to free scientists from financial issues, equipment shortages, and academic communication challenges (Zhang, Yin et al., 2024), allowing them to focus more on their intellectual work and affecting their knowledge discovery and innovation progress. Nevertheless, as research funding has become increasingly competitive, it may be having a negative effect on academics (Alkhawtani, Kwee, & Kwee, 2020; Carvan, 2020). For example, the considerable time and effort required by academics to prepare proposals and applications (Bollen, Crandall et al., 2014), with a high risk of not receiving funding, may be dragging down their output. This endless competition and the negative consequences make some researchers resistant to applying for grants. Further, even if they are funded, academics may be trapped with a large number of teaching responsibilities or household duties (Malisch, Harris et al., 2020) while researching and reporting regularly on their progress, exacerbating the stress of their work. Thus, it remains to be determined whether the role of research funding is what governments expect it to be.

Yet, extant empirical results provide controversial conclusions about the relationship between research funding and scholars’ performance. Detailed evidence of the relationship between funding and academic performance, and the controversies between such evidence, is provided in Section A of the Supplementary material. This controversial conclusion is reflected in whether research funding is effective in terms of scientific output and, if so, the directionality of this impact (i.e., positive or negative; Alkhawtani et al., 2020; Beaudry & Larivière, 2016; Benavente, Crespi et al., 2012; Boyack & Jordan, 2011; Ebadi & Schiffauerova, 2016; Hottenrott & Thorwarth, 2011; Jacob & Lefgren, 2011; Leydesdorff, Bornmann, & Wagner, 2019; Pagel & Hudetz, 2015; Sandström & van den Besselaar, 2018; Wagner, Whetsell et al., 2018; Wang, Jones, & Wang, 2019). The reasons for these controversies could be considerably complicated, partly because it is not yet clear which is the cause and which is the effect of research funding and scholar performance. Research funding decisions are routinely driven by the applicant’s past productivity or allocated to scholars and projects with greater potential (van Arensbergen & van den Besselaar, 2012), leading to a reverse causal relationship between research funding and scholar performance. Then, research funding may further proliferate the scientific publications they are funded for, further leading to selection bias (Ebadi & Schiffauerova, 2016; Kaiser, 2019; Li, Azoulay, & Sampat, 2017). Thus, the selection bias and potential reverse causality resulting from such biased and selective allocation rules for research funding make it possible to confound research funding effects when comparing funded and nonfunded scientists. For example, some studies confirm that these endogeneity issues, such as selection bias, can be expressed as potential, inherent productivity gaps between individual scholars (Li, Zhang et al., 2022), or publishing discrepancies due to gaps in the academics’ research areas and affiliations (Shin, Kim, & Kogler, 2022).

Recent evidence suggests that scholars expect to use beyond correlation strategies to capture research funding’s impact on scholar performance. Instrumental variables (IVs) are attracting much attention because of their strength in isolating endogeneity issues caused by research funding. For example, Yu, Dong, and de Jong (2022) use the urban economic dynamism of universities as an IV to isolate the potential endogeneity of research investment in China when assessing the impact of public research funding on the research output and quality of 622 universities in China from 2010–2017. Yet, it is invariably challenging to find exogenous factors that do not directly affect scholar performance but rather indirectly affect scholar performance by shaping the probability of scholars receiving funding, resulting in a rather limited selection of currently available IVs (Andrews, Stock, & Sun, 2019). Moreover, most IVs measuring funding effects are concentrated in non-English-speaking countries (Lawson, Geuna, & Finardi, 2021). Therefore, it is imperative to find or design IVs for research grants in English-speaking countries wherever possible to isolate selection bias, omitted variables, etc. and to ensure cleaner grant validity outcomes.

This paper examines the impact of receiving research funding on scholars’ performance, focusing on competitive funding in English-speaking nations. Using the Social, Behavioral and Economic Sciences’ (SBE), one of the National Science Foundation’s (NSF’s) directorates, grants from 2000 to 2019 as examples, we adopt IV estimations and two-stage least square (2SLS) regressions to isolate and understand the aforementioned endogeneity of research funding that confounds the assessment of effects. These enable us to quantify the effect of research funding on the number of publications and the impact of studies. Specifically, the main contribution of this paper is that we present and introduce three IVs in research funding, namely scholars’ political resources, isomorphic behavior, and scholars’ familiarity with the SBE program application. We explain how we construct these three IVs and provide tests of their validity and robustness, expecting other scholars to consider them when estimating treatment effects related to academic performance. Lawson et al. (2021) examined the utility of the first IV in funding competitions in non-English-speaking nations. We introduce it to English-speaking countries and combine it with the second and third IVs to demonstrate their efficacy in isolating the endogeneity of research funding. As far as we know, two IVs, isomorphic behavior and scholars’ familiarity with program application, are proposed for the first time. Simultaneously, unlike most studies that use complex and varying competing grants as a treatment, this study relies on the most common standard and continuing grants in the U.S. social sciences to further control for possible confounding of grant effectiveness assessment by multi-purpose types of grants. We take into account the differences of initial productivity, gender, institutions, and research fields between scholars, which further reduces the differences between samples caused by reverse causality and missing variables. Empirical evidence suggests that SBE research funding improves the quantity and quality of academics’ research significantly, but hardly causes their work to be published in more prestigious journals.

IVs are considered by many scholars as one of the most effective strategies to address endogeneity issues. Nevertheless, the exogenous requirements of IVs means that their options for exploring the relationship between research funding and scholar performance are limited. By summarizing some of the existing research, we find that the available IVs broadly fall into two categories: lagged variables and exogenous variables designed on the basis of “local conditions” (Berger, Miller et al., 2005). The former is more common in empirical evidence in economics and business, and the core idea is to use lags of endogenous variables as IVs wherever possible. For instance, when measuring the impact of research funding on research output and innovation in 24 public universities in Ethiopia between 2016 and 2020, Abibo, Muchie et al. (2023) use all available lags of research funding as weak instruments. Similarly, Nugent, Chan, and Dulleck (2022) employ lagged values of scholars’ patent activity and grants as IVs to explore the scholarly performance of 2,375 scholars at 36 Australian universities after receiving ARC funding. However, some scholars also express concerns about using such IVs. When the omitted variable that leads to endogeneity is autocorrelated, using the lagged endogenous variable as an IV does not result in consistent estimates of coefficients (Bellemare, Masaki, & Pepinsky, 2017).

Further, some scholars use instruments based on local conditions. Kelchtermans, Neicu, and Veugelers (2022) design the award number of funded colleagues as an IV to estimate research funding effects on the academic performance of 734 scholars at KU Leuven, Belgium. They argue that the number of grants awarded to other academics around the target scholar in the year in which the target scholar received the grant implies the degree of desire or demand for research funding for the university (i.e., local resource scarcity). Political resources of scholars, funding availability of laboratories and economic indicators of university locations are also proposed to address the dilemma of having no available IVs other than lagged endogenous variables (Lawson et al., 2021; Yu et al., 2022).

Scholars’ political resources are frequently seen as an add-on to their ability to obtain more funding resources. When the effectiveness of the evaluation system depends on the decisions of individual reviewers, the authority to allocate resources may be transferred selectively and inexpressively in the form of grants to those specific individuals preferred by the evaluators (Braun, 1998; Groeneveld, Koller, & Mullins, 1975). It may be confusing to quantify the impact of such allocation mechanisms on fairness when exposed to human bias. Yet, there is an argument that internal endorsement by elite groups and cronyism have led, for example, to grants going to certain scholars with political resources, particularly in highly competitive environments for academic resources (Viner, Powell, & Green, 2004). Evidence suggests that academics employed as NSF rotators have access to more individuals outside of academia and receive twice as many research resources, such as grants (approximately $200,000), compared to their colleagues in the same department over a period of 5 years (Hoenen & Kolympiris, 2020).

Despite the effect of political resources on scholars’ resources, such resources may be controversial as an IV for capturing funding effects on scholarly performance. On the one hand, some studies claim that scholars’ political resources can affect not only the probability of scholars receiving grants, but also their publication efficiency and research quality (Liu & Zhou, 2022) (i.e., political resources do not fit the exogenous hypothesis of an IV). On the other hand, the political resources of scholars may be rather vaguely defined. In general, scholars’ political resources could be split by some studies into internal at the research institution and external tenure. For example, an internal administrative position within a university is considered by some scholars to be an academic’s political resource. Political resources are claimed to be external to scholars when they hold senior positions in disciplinary associations off campus or when they serve in leadership positions in government or other nonprofit organizations. Political resources granted to scholars by different governments or funding agencies may manifest themselves differently for scholars (Viner et al., 2004). Therefore, when focusing on particular research issues, the political resources of scholars should be further specified. Blind faith in or exclusion of political resources as an IV for scholars’ academic resources or academic performance may not be based on solid evidence. For example, Cattaneo, Horta, and Meoli (2019) finds that academics receive dual appointments at universities and government or nonprofit institutions and that such dual appointments do not have a significant effect on the academic performance of scholars. The availability of political resources as an IV is also shown to exist in specific countries by Lawson et al. (2021). The validity of political resources as an IV should therefore be further tested in practice based on the target.

Imitation isomorphism is defined as the tendency for an individual or organization to imitate another successful individual or organization when faced with uncertain goals or tasks because the former believes that some of the latter’s behaviors are beneficial. Imitation isomorphism may be used as a potential exogenous variable based more on local conditions. Individual scholars are frequently affiliated with a faculty or department in a university (Berger & Luckmann, 2016). This supports a cultural view that physical organizations often internally need to group individuals to meet the needs of individual roles and related tasks required for organizational development. This is particularly true of organizations and faculty settings within universities (Desimone, 2002). When similar goals such as promotion or resource needs exist between academics within a subgroup within a university, scholars may tend to imitate the successful scholars around them.

To clarify the direct relationship between imitation isomorphism and research funding, we can draw upon established theories of organizational behavior and social learning. Rooted in institutional theory, imitation isomorphism suggests that in resource-constrained environments, individuals are inclined to mimic behaviors they perceive as conferring a competitive advantage (DiMaggio & Powell, 2000). This process is not merely heuristic; it is a recognized pathway for navigating complex academic systems (DiMaggio & Powell, 2000; March, Olsen, & Christensen, 1976). In the highly competitive landscape of academic funding, scholars often model their approaches on the successful strategies of colleagues within the same institution and research fields (Antonelli & Crespi, 2013). This aligns with organizational sociology’s assertion that such imitation can lead to the emergence of similar research agendas and funding strategies within academic departments. Moreover, imitation isomorphism can directly impact scholars’ research funding by facilitating the flow of information within academic communities (Lieberman & Asaba, 2006). As scholars observe their peers successfully obtaining funding, they gain insights into effective grant-writing techniques, funding opportunities, and strategic alignment with funding agencies’ priorities (Velarde, 2018). This enhanced information exchange not only reduces the uncertainty associated with the funding process but also empowers scholars to make informed decisions that increase their likelihood of securing funding (Ordanini, Rubera, & DeFillippi, 2008; Wood, Meek, & Harman, 1992).

Furthermore, the theory of cumulative advantage, often referred to as the Matthew effect, underscores this relationship by highlighting how initial success in obtaining funding can create a self-reinforcing cycle. Scholars who secure funding early on are more likely to continue receiving it, fostering a pattern of success that others may seek to replicate (Bol, de Vaan, & van de Rijt, 2018; Merton, 1968). Consequently, scholars observing this pattern may be encouraged to pursue similar research areas or grant application strategies, thereby increasing their chances of securing funding through established channels. This dynamic is further supported by recent research demonstrating that scientific funding often flows to particular organizations and individuals, creating a snowball effect (Mitchell & McCambridge, 2022). When a university consistently receives funding for a specific research topic, it is more likely to maintain dominance in that area, which reflects a process of preferential attachment or the Matthew effect in research funding (Antonelli & Crespi, 2013).

Further, when imitation isomorphic trends occur, diffusion may lead to a cumulative effect (Seyfried, Ansmann, & Pohlenz, 2019). When isomorphisms among scholars develop, the grants of earlier winners do not seem to directly affect the scientific productivity of other scholars. Some scholars develop new IVs in an imitation isomorphic framework. For example, Kelchtermans et al. (2022) consider the number of research grants received by an academic’s colleagues within the unit for the year as an IV, implying the demand for funding in the institution.

Familiarity may affect the possibility of scholars receiving research grants, along two general pathways. The first path occurs at the wait-and-see stage of a scholar’s grant application. How to choose the right foundation among a wide range of options and the setting of priorities is usually related to the scholar’s cognitive needs (Ashrafian, Mortlock, & Athanasiou, 2010; Grass, Strobel, & Strobel, 2017). Scholars’ subjective cognitive needs may affect their pre-preparation and decision to apply for grants. Ashrafian et al. (2010) emphasize that scholars’ cognition of grant conditions and normativity should be based on those grant opportunities with which they are more familiar, to increase the success probability.

The second path is reflected in the preparation of application materials, which can be understood as a game between the applicant and the funding decision-maker. On the one hand, the accountability of funding agencies requires them to establish restrictive rules for research grants to initially guarantee the quality of the proposals they receive, allowing noncompliant proposals to slip out of the research grant application pipeline early. On the other hand, an applicant’s familiarity with the annual grant topics and accuracy of the application process tends to affect the probability of their application being successful (van den Besselaar & Mom, 2022). In particular, in a more competitive funding environment, where resources are scarce, applicants’ proposals generally tend to be of high research merit or innovation. Therefore, in the evaluation of similar research programs, an applicant’s understanding of the research projects and the application process and documentation is often one of the key reasons why that applicant stands out from the rest (Beleiu, Crisan, & Nistor, 2015).

Despite the efforts of governments and funding agencies to communicate research objectives and guidelines through various channels, scholars remain largely unfamiliar with research funding processes. To address this gap, official NSF training sessions and workshops, which are highly visual and interactive, have become essential tools for enhancing scholars’ understanding of funding opportunities¹. These programs can significantly affect the likelihood of grant applications being successful, whereas traditional announcements and documents often fall short. Moreover, there are distinct differences between the training related to writing research proposals and that for academic journal articles. Research proposals typically focus on crafting compelling narratives that align with funders’ goals, while academic articles emphasize precise presentation of research findings and methodological rigor. According to a survey by Nature Masterclasses, a significant number of scholars encounter difficulties during the funding application process². This underscores the importance of project familiarity as an exogenous factor influencing the link between research funding and academic performance.

The scientific funding data come from the SBE/NSF database. A detailed description of the SBE appears in Section B of the Supplementary material. We keep information on scientific grants from standard and continuing programs and exclude grant types such as charity, student scholarships, and cooperative agreements. The data contains details of 9,501 U.S. university researchers who received SBE funding in the United States between 2000 and 2019. These researchers are supported as principal investigators (PIs) in the SBE’s Social and Economic Sciences (SES), Behavioral and Cognitive Sciences (BCS), and SBE Office of Multidisciplinary Activities (SMA) subdepartments. We obtain the full names of these researchers, the funding amount, the start and end dates of the grant, the type of grant, the grant number, the PIs’ affiliation, the grant title, and the abstract of the grant. Moreover, we identify scholars’ gender through Gender-API, and the identification method and results are shown in Section C of the Supplementary material.

The project names open for application by the SBE are diverse and constantly updated and added to over the years, making it difficult to observe the research topics of the funded projects to which academics belong in a straightforward way. Therefore, we allocate each academic to a research topic using the Latent Dirichlet Allocation (LDA) model (Blei, Ng et al., 2003) to extract topics from the titles and abstracts of all academics’ programs. We obtained a total of 30 SBE-funded research topics, each of which included five keywords. The results of the LDA study topic sampling and the distribution of the project numbers are shown in Section D of the Supplementary material. Descriptive metadata of 359,814 scientific publications by these academics in both the Web of Science (WoS) and Microsoft Academic Graph (MAG) databases was searched, covering the years 1995 to 2019³. Publications included in WoS provided more detailed grant information than other databases, allowing us to attribute grants to authors and publications more accurately. However, some studies confirm that the publications and journals included in WoS are incomplete compared to MAG and Scopus, resulting in a possible underestimation of author output and impact (Martín-Martín, Thelwall et al., 2021). Therefore, we conduct a joint search of WoS and MAG. We take the concatenation of the publications and highest value of citations for that recipient in the two databases. Our process for linking the SBE, WoS and MAG is shown in Section E of the Supplementary material.

To measure the citation impact of a publication, we also obtain the CiteScore for the journal to which it belongs. CiteScore was designed and launched by Elsevier in 2016 to describe the performance of academic journals relative to other journals in the research area to which they belong⁴. While citation counts are frequently used as raw data to reflect the dissemination or popularity of a publication, CiteScore is seen as the citation impact of publications, and is often associated with academic prestige (DeJong & St. George, 2018). CiteScore is calculated by dividing the citation counts by the publication counts published in a given journal over a four-year time window. It covers over 22,800 journals, more than twice the number of journals covered by another metric, the Journal Impact Factor (JIF). It also solves the problem of JIF not being able to calculate citing records and cited lists of inaccessible articles, and is considered to be more transparent and accessible in its calculation (Fernandez-Llimos, 2018).

As all these PIs come from U.S. universities, we add these universities to the U.S. News and World Report rankings. The U.S. News and World Report rankings focus more on teaching quality and faculty resources. Given the significant correlation between university reputation and the output of academics (Viner et al., 2004; Zhai, Xia et al., 2024), we also consider the QS university rankings, which emphasize academic prestige and their university reputation scores. We use the ranking order of the universities in the 2022 rankings⁵.

Table 1 presents the statistical results of the variables included in the data set, covering variables related to academics, research areas, affiliations, etc. At the individual scholar level, we can see that it takes an average of 11.090 years of academic age for scholars to be awarded an SBE, and that SBE scientific funding is dominated by male scholars at a high 69.00%. Considering that some scholars believe that the base of female scholars who apply is originally smaller than that of male scholars, this may not be evidence of gender discrimination in the allocation of SBE funds. On the contrary, the proportion of female SBE grants successfully funded is higher than the proportion of female scholars known to other funding committees.

The first IV is the political resources of academia. Scholars have explored the important role that social elites play in the distribution of competitive funding. Particularly, in environments where resources are scarce or competitive, political hegemony will dominate the allocation of resources (Mitroff & Chubin, 1979). Thus, drawing on Lawson et al. (2021), employing academic national sociopolitical capital as an IV for Italian scholars, we introduce it to the English-speaking countries and expect that it will affect the probability of success in accessing U.S. and international competitive funding. Here, we redefine academic national sociopolitical capital for the NSF (i.e., defining the role of managers, board members, and external experts in the SBE foundation). We count the number of PIs who served in the SBE Division/NSF Board, NSF Directors, Social Sciences and Humanities Division/American Philosophical Society (APS), or American Association for the Advancement of Science (AAAS) from 1990 to 2019⁶. Although a degree of scientific achievement may be required to hold these influential roles, doing so might be largely the result of the accumulation of sociopolitical capital by scholars. For example, numerous U.S. presidents and U.S. Supreme Court justices (e.g., George Washington, Barack Hussein Obama, and Elena Kagan) are elected members of the APS.

In our data set, 58 PIs served or are serving as members of the NSF Board of Directors or as program directors in SBE departments, or as external experts for grant review meetings or programs; 62 PIs are elected as members of the APS’s seven fields of Sociology and Demography, Economics or Linguistics related to SBE funding areas; and 142 PIs are elected as key members of AAAS’s Social and Economics Science and its interdisciplinary areas. The dummy variable measuring political capital is taken as one (1) in the first year of the lead role and in the following four years to indicate its impact on academics over time⁷. Additionally, we employed two further strategies to validate the reliability of academic political hegemony as an instrumental variable. First, we categorized scholars’ political hegemony into different levels, treating these as separate treatments, and examined whether varying degrees of political capital have a statistically significant direct effect on academic performance, relative to scholars with no political capital. This approach tests whether political hegemony might directly affect academic output, potentially compromising the exogeneity assumption of the instrument. Furthermore, we hypothesized that if academic political hegemony directly impacts scholarly productivity, this effect would likely vary according to the scholar’s level of political capital. To test this hypothesis, we conducted separate IV analyses for different levels of political capital and compared their impact on the estimated effect of research funding. Results show that varying levels of political capital did not lead to significant differences in the funding effect estimates, lending further support to the exogeneity assumption of academic political hegemony as an instrumental variable (see details in Section F of the Supplementary material).

The second IV we selected comes from isomorphism among scholars. We draw on imitation isomorphism in organization theory (DiMaggio & Powell, 2000) and semantic isomorphism in mathematics (Moraschini, 2016) to explain the potential imitation isomorphic behavior of academics and the structural framework of isomorphism in which they are positioned. We introduce this imitation isomorphism to individual scholars. However, within the social sciences, the scope and design of research topics by different researchers span a considerable range (Bechhofer & Paterson, 2012). We therefore further narrowed the structure of imitation isomorphism from the faculty to the research topic level. We use the topic of a scholar’s funded project as the main direction of their research contribution within a given phase; the research directions of scholars under the same research topic are more similar within the social science subfield (i.e., the semantic isomorphic results of the research topic symbolization). Suppose that an individual academic belongs to an entity organization (university) U and his or her funded research topic is T. U = [U₁, U₂, …, U_m] indicates that there are m universities after duplication of all scholars’ affiliations; T = [T₁, T₂, …, T_n] indicates the research topic to which the scholar’s research grant belongs. We then count the number of award-winning scholars for each research topic for each university in the m × n matrix.

When scholars within a cell (the same organization and research topic) receive research funding, other scholars within the same cell will imitate the successfully funded scholars when faced with research financial issues, research blocking, or promotion difficulties. This imitation may affect the probability of scholars who imitate others receiving grants, relative to scholars in other cells where there are no scholars to be imitated. Further discussion on the exclusion restriction and design of the imitation isomorphism instrument is given in Section H of the Supplementary material.

A third IV is also chosen, namely PIs’ familiarity with the SBE program and application. The NSF directorates’ official NSF days and workshop events are selected as a key channel to affect scholars’ familiarity with the grant application. NSF days and workshops are held individually or in collaboration with a university selected by different NSF directorates. They are designed to increase the competitiveness of scholars and universities in competing for NSF funding, highlighting and guiding proposal writing, performance review processes and priorities, and include answering questions from academics. The SBE has conducted NSF days and workshops at 98 universities, and it does not tend to select universities that have historically received more funding to host events. The variable approximating researchers’ knowledge of and training in SBE is the number of events such as NSF days hosted by their university in the five years prior to their receiving the SBE grant.

It is important to reiterate our desire to discover the impact of research funding on scholars’ performance. However, historical scholars’ performance may affect the availability of research grants to academics (i.e., a reverse causality accompanies the two). Reverse causality has been demonstrated to cause the independent variable to be associated with the random error term (Malter, 2014), resulting in biased effect estimates. Thus, we use the logarithmic mean of the scholars’ performance of academics in the three years prior to the sample period as initial scholars’ performance, as suggested by Wooldridge (2010). This initial performance represents the path dependence of performance and the permanent heterogeneity of PIs including academic motivation and cognitive ability (Fernández-Zubieta, Geuna, & Lawson, 2016). We then include this control variable in the 2SLS model to complement the predicted effects.

First, and most importantly, we confirm whether there is an endogeneity issue with SBE funding allocation, which is the basis for the IVs to isolate bias (Mutl & Pfaffermayr, 2011). Then, this study tests the validity of the three IVs we selected, including tests of exogeneity of IVs (Hausman, Stock, & Yogo, 2005), tests of under- and overidentification (Ullah, Zaefarian, & Ullah, 2021), and tests of weak IVs (Andrews et al., 2019).

While the IVs selected are reliable, we also need to determine the robustness of the research funding effect. In this study, all individuals in the treatment group are randomly divided into a pseudotreatment group and a pseudocontrol group; the control group is similarly divided into a pseudotreatment group and a pseudocontrol group. In general, we hypothesize that there should be no significant funding effects or scholars’ performance differences between the respective two subgroups of the treatment and control groups. Therefore, we repeat the IVs and 2SLS strategies separately for the two subgroups in the treatment and control groups, as shown in Figure 1, to see if a research funding effect emerges.

Table 2 shows a considerably significant positive correlation between SBE research funding and scholars’ performance, using OLS regression. Controlling for gender, affiliation, research area, and initial scholars’ performance of a scholar, receiving funding resulted in a 68.9% rise in the number of journal articles authored by the scholar; a 46.669 rise in the average number of citations to a single publication; and the ability of the grantee’s research to be published in higher-impact journals, where the average CiteScore rose by 5.524 compared to those scholars who had not received funding.

In 2SLS first stage, as shown in Table 3, the three IVs we selected have a significant effect on the distribution of SBE grants. The coefficient of the isomorphic effect between scholars (coef. = 0.004, p = 0.000) indicates that the more grants scholars in the same research topic at the same university receive, the more likely it is that subsequent scholars will receive grants. The probability of an academic receiving an SBE grant in the subsequent 3 years rises when they occupy an administrative position in academic associations in the social sciences and economics, with a 4.7% increase in probability. This implies that political resources and a social elite culture do exist in the allocation of scientific research resources (Hoenen & Kolympiris, 2020; Lawson et al., 2021). In addition, the greater the number of NSF days and grant application training activities conducted at universities, the greater it negatively affected the probability of the social and economic scholars at these universities receiving funding, decreasing by 0.6%. The SBE’s outreach and training efforts may trap the ideas and efforts of some academics in the project application process, weakening the competitiveness of the proposal.

In contrast to OLS, the 2SLS second-stage analysis shows considerable variability in research funding’s impact on scholar performance. Although the OLS underestimates the impact of scientific funding on the number of journal articles and overestimates the impact on the number of citations to publications, we observe that receiving funding does increase the number and impact of scholars’ publications but does not lead to a greater possibility of publishing in higher-impact journals. We see that receiving funding resulted in 2.816 more journal articles being published by funded scholars, and the average number of citations to these publications is raised by 16.421.

However, research funding does not improve the likelihood of publishing in more prestigious journals. There is no statistically significant relationship (coef. = 2.883, p = 0.067) between scientific funding and journals’ CiteScore. One key driver of this phenomenon could be the short-term performance metrics inherent in academic and funding evaluation systems. Researchers are often under pressure from funding agencies to produce results within tight timeframes and to report progress regularly, which may incentivize a focus on quantity rather than the depth or quality of research (Zacharewicz, Pulido Pavón et al., 2023). This short-term effect, driven by time pressures, may lead scholars to submit their work to journals with shorter review cycles and higher acceptance rates, which are often of lower impact compared to more prestigious journals that require longer review periods and more stringent criteria (Johann, Neufeld et al., 2024). Moreover, the funding system itself may inadvertently encourage this behavior, as researchers, in an effort to secure continued funding, may prefer journals that allow for quicker publication, rather than engaging in the more time-consuming process of submitting to top-tier journals that require greater investment (Butler, 2003). This issue is further exacerbated by broader academic cultural dynamics. Success is often equated with quantity—such as the number of published papers—rather than the quality or impact of those papers (Lawson et al., 2021). This emphasis on short-term results may undermine the cultivation of long-term research quality and innovation (Zacharewicz et al., 2023).

Building on the earlier finding that research funding does not significantly improve the likelihood of publishing in prestigious journals, further analysis reveals its heterogeneous effects across journals with different CiteScore levels. Specifically, while research funding does not significantly influence articles published in the top 10% CiteScore journals (coef. = −0.650, p = 0.547), it has a substantial positive impact on publications in the bottom 10% CiteScore journals (coef. = 3.183, p = 0.002). This suggests that funding plays a crucial role in enhancing the visibility and impact of work published in lower impact journals. One explanation for this pattern could be the short-term performance pressures discussed earlier, which drive scholars to prioritize rapid results and easier publication opportunities. In this context, funding may provide critical support for researchers operating in resource-constrained environments, enabling them to meet these pressures by improving the quality of their submissions to lower impact journals. By alleviating resource limitations, funding helps raise the “floor” of academic performance, even if it does not substantially influence the “ceiling” represented by top-tier journals. This underscores the dual role of funding: While it supports research productivity broadly, its impact is most pronounced in improving the output of lower performing publications rather than elevating work to higher-impact outlets. The regression results for the top 10% and bottom 10% CiteScore journals and the corresponding tests for the validity of the instrumental variables used in these regressions are presented in Section I of the Supplementary material.

Regarding control variables, there are some gender and institutional gaps in scholars’ performance. Receiving funding significantly increases publication counts for male academics compared to female academics, but this gender gap is to a smaller scale. There is no significant relationship between gender and scientific impact. Meanwhile, publication and impact of research areas, university rankings, and employer reputation do have significant associations with scholars’ performance. For example, when winners are male, they publish 0.039 more publications compared to nonwinners; however, the relationship between the gender gap and the citation count of publications and journal impact is not significant, which is consistent with the findings of other research (Benavente et al., 2012; Hottenrott & Thorwarth, 2011; Lawson et al., 2021). The higher the employer reputation score of the university, the more efficient and effective the scientific publications of academics, which further confirms the findings of a number of studies (Medoff, 2006; Way, Morgan et al., 2019).

The use of IV models often requires endogeneity, exogeneity, overidentification, and under-identification tests to confirm the validity and strength of the IVs. The results are shown in row 1 of Table 4, where we first test whether SBE research funding is endogenous using the Hausman test. Therefore, we reject the null hypothesis that SBE funding is exogenous, which underlies the subsequent use of IVs to isolate endogeneity.

We then use Wooldridge’s (1995) robust score test, Hansen J statistics, and Kleibergen-Paap rk LM statistic to determine whether IVs isolating SBE endogeneity are over- or under-identifying, with the results shown in Table 4, rows 2 and 3. We find that both tests show that the null hypothesis that the variables are exogenous cannot be rejected. Meanwhile, the Cragg-Donald Wald test and Stock-Yogo weak ID test are used to test for weak IVs, with the null hypothesis that the IVs are weak IVs. Usually, when the F-statistic of the Cragg-Donald Wald test is greater than the 5% maximal IV relative bias of the Stock-Yogo weak ID test, then we can be confident that there are no weak IVs. The results are shown in row 4 of Table 4, where the test significantly rejects the null hypothesis, implying that the three IVs are strongly correlated with the endogenous variables.

Further, we report the results statistics for the first stage of 2SLS. As shown in row 5 of Table 4, the Robust F-statistic and the minimum eigenvalue statistics for the first stage are 544.236 and 507.350, respectively, both of which are greater than the test values suggested by other scholars that the threshold F should be greater than 10. Or, when both the robust F-statistic and the minimum characteristic statistic are greater than 10% or 15% of the critical values of the 2SLS size of nominal 5% Wald test, we can conclude that there are no weak IVs. Our robustness tests for the funding effect are presented in Section J of the Supplementary material.

In this study, we revisit the relationship between research funding and scholars’ performance using the example of 9,501 PIs supported by the SBE, the largest social science funding agency in the United States. In particular, we introduce three IVs to isolate the endogeneity issues of research funding effect estimates and consider characteristics of individual researchers, affiliations, and research areas in combination with the 2SLS model to explain the effects of SBE support for the social science community. Robustness tests of both these IVs and the estimation of funding effects confirm our key results.

To adequately address the endogeneity issue that is challenging in the existing literature, we highlight the theoretical background and validation process of the three IVs, the political hegemony of academics, imitation of isomorphic behavior, and program familiarity. This is one of the essential contributions of this study. We find that scholars’ political capital significantly increases the probability of scholars receiving funding, consistent with previous findings (Hoenen & Kolympiris, 2020; Lawson et al., 2021; Viner et al., 2004) that political hegemony increases scientists’ competitive advantage. Imitation isomorphic behavior and conditions among scholars similarly contribute to the probability of grant receipt. On the one hand, information symmetry is critical when applying for grants, and the conditions and circumstances under which imitation occurs among scholars may contribute to the information fluidity of grant applications by imitating scholars. On the other hand, as more grant successes arise in an environment where imitation is isomorphic (i.e., where there is a cumulative advantage in research grants for particular research topics at the given university), research grants are more likely to be allocated to those universities with a history of good application performance. This supports the findings of several studies (e.g., Perc, 2014; Steinþórsdóttir, Einarsdóttir et al., 2020) demonstrating a Matthew effect in applying for and receiving research grants.

Yet, worryingly, the SBE’s project writing training and outreach activities at universities appear to have a negative effect on the success rate of grant applications. When SBE institutions hold more of these events at universities, it rather inhibits the possibility of scholars from these universities receiving funding. The fact that funding agencies tend to choose universities with a weak funding history to host such events may further inhibit the ideas of academics at these universities and even mislead them to focus more on the normative aspects of the proposal than on other key factors. Previous research has demonstrated that 19 or more factors have been found to contribute to the success of applications, including teamwork, clear goals, and communication skills (Beleiu et al., 2015). Therefore, researchers from these universities require more holistic support than just training in writing and receiving publicity, as if they need five fingers working together to make a fist for maximum impact, rather than being taught to focus their strength on just one finger.

After using these three IVs and considering reverse causality, our study shows that funding can affect scholar performance, supporting the results from Ebadi and Schiffauerova (2016) and Pagel and Hudetz (2015). However, research funding may have a limited effect—for example, it does not enable recipients’ knowledge to be published in more prestigious journals. This might be attributable to the system of research evaluation and short-termism. Scholars receiving research grants are frequently required to report or evaluate their performance to the funding agency on an annual or regular basis, and are even required to produce results within a short period of time. This “distorted” system of evaluation and requirements for funding or grants has been shown to cause considerable pressure on researchers, who complain about the insufficient time to publish in more challenging journals (Groen-Xu, Boes et al., 2021). Thus, by abandoning highly prestigious journals that tend to have longer review cycles, are more difficult to publish in, and require more of a researcher’s time commitment (Paiva, Araujo et al., 2017), academics may be forced to present their knowledge in journals that are less prestigious or more familiar. More worryingly, O’Regan and Gray (2018) confirmed that the review system and short-termism of performance evaluation systems in funding programs convey this short-termism to academics, resulting in less reproducible research and less exploration of more unknown knowledge. We would thus appeal to governments and funding agencies to continue optimizing their funding evaluation systems by proposing more realistic and feasible funding and evaluation strategies with a stronger focus on the long-termism required for knowledge quality.

We need to acknowledge some limitations. Our study focuses on regular grants, in order to avoid the interference of other research grant types such as student scholarships, infrastructure purchases, and noncompetitive grants in the evaluation. Therefore, our findings may not be applicable to irregular types of research grants. We use the United States, one of the English-speaking countries, as an example, and it remains to be determined whether our IVs and findings are applicable to other English-speaking countries that, in particular, have different funding and award structures than the United States. In addition, our study observes changes in the overall performance of scholars in the 5 years prior to and 5 years following the award, making it challenging to capture the potential impact of research grants on scholars’ further academic careers and the annual dynamics of grant effects.

An early version of this paper was presented at Wuhan University and Sun Yat-sen University in 2023 and 2024, respectively; the authors thank the suggestions from the audiences. The authors are also grateful to the editor and the anonymous reviewers for their constructive comments.

Yang Ding: Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Visualization, Writing—original draft. Yi Bu: Conceptualization, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Writing—review & editing.

The authors have no competing interests.

Yi Bu is supported by the National Natural Science Foundation of China (#72474009, #72104007, and #72174016). Yang Ding is supported by the PhD fellowship (No. s2222886) in Management Science and Business Economics from the University of Edinburgh Business School.

NSF SBE grant data: NSF funding data for the Social, Behavioral, and Economic Sciences (SBE) Division is publicly available at https://www.nsf.gov/funding/opportunities.

Data on scholars’ political hegemony: Information about scholars holding significant positions, such as NSF Board members and SBE division program directors, remains publicly accessible through sources like https://www.nsf.gov/nsb and https://www.nsf.gov/sbe/sbe-advisory-committee#members-ba9. However, information about external experts or scholars involved in major SBE budget decisions and related matters is now restricted (e.g., https://new.nsf.gov/events/iucrc-center-e-design-edesign-iab-meeting-5/2019-11-05). Due to significant updates to NSF’s website and policies in 2022, access to information about earlier events, particularly those before 2019, has become substantially restricted. Since 2019, detailed records on event participants, including internal and external advisors and reviewing scholars, are no longer available for public viewing. This policy change is likely intended to protect sensitive information, including proprietary technical data, financial details, and personal data associated with grant proposals, in compliance with the Government in the Sunshine Act (5 U.S.C. 552b(c), Sections 4 and 6). As an alternative, NSF now provides video records of key meetings, which remain accessible to the public. Scholars interested in NSF events may gather legally available information from these recordings.

Data on membership in relevant fields of the American Philosophical Society (APS) such as Sociology, Demography, Economics, and Linguistics can be accessed via https://search.amphilsoc.org/memhist/search. Data on the American Association for the Advancement of Science (AAAS) members in the fields of Social and Economic Sciences and interdisciplinary areas can be accessed through https://www.aaas.org/membership.

Data on NSF Days: Information about NSF Days can still be accessed through https://www.nsf.gov/od/olpa/eventgroups/nsf-days. However, starting from 2022, due to the relevant exemptions under the Sunshine Act, NSF has hidden the historical records of NSF Days. Nonetheless, the historical information about NSF Days and similar events can still be retrieved through the official websites of various universities.

Bibliographic data sources: Bibliographic data were obtained from Microsoft Academic Graph and Web of Science. Microsoft Academic Graph data is accessible at https://www.microsoft.com/en-us/research/project/open-academic-graph/, though updates ceased following the sale of the platform. Web of Science data is available via institutional subscriptions. We provide detailed guidance on joint searching of Web of Science and Microsoft Academic Graph bibliographic databases to obtain a more comprehensive publication data set for scholars (see Section E of the Supplementary material).