Proposal success in Horizon 2020: A study of the influence of consortium characteristics

Research, technology, and innovation (RTI) policy is increasingly under pressure to legitimize the provision of public subsidies and demonstrate the impact they generate on the research and innovation system and society as a whole. Demand for efficient and well-performing research funding systems has also spurred scientific interest in the study of the funding procedures, selection outcomes, and additionality effects of publicly funded research grants and partnerships (see, for example, Bornmann, Leydesdorff, & Van den Besselaar, 2010; Luukkonen, 2000; Viner, Powell, & Green, 2004). A prominent example in the scientific literature is the research and development (R&D) projects funded by the Framework Programmes for Research and Innovation of the European Union (EU FP) (Balland, Boschma, & Ravet, 2019; Breschi, Cassi, et al., 2009; Defazio, Lockett, & Wright, 2009; Protogerou, Caloghirou, & Siokas, 2010). Scholars have investigated the organization-specific determinants of EU-wide R&D collaborations (Autant-Bernard, Billand, et al., 2007; Lepori, Veglio, et al., 2015; Paier & Scherngell, 2011), the geographical composition and evolution of these partnerships (Balland, 2012; Balland et al., 2019; Chessa, Morescalchi, et al., 2013; Scherngell & Barber, 2009; Scherngell & Lata, 2013), and their actual impact on the research system in different European regions (Hoekman, Scherngell, et al., 2013; Wanzenböck & Piribauer, 2018).

Most EU FP studies only investigate awarded projects: R&D collaborations that successfully passed the proposal stage to enter an operational phase. A central finding of these studies is the oligarchic network structure created by the funding program (Breschi & Cusmano, 2004), characterized by a small core of research-intensive organizations concentrated in Western Europe which participate in many projects and thus take a lion’s share of the project funds (Balland et al., 2019; Protogerou et al., 2010; Wanzenböck, Scherngell, & Lata, 2015). In contrast, organizations positioned at the network periphery are rarely involved in projects and are only loosely connected to the core players. The persistence of this finding across several EU FP additions has fed scientific and political debates about the goals and funding criteria, particularly whether the EU FPs should serve as excellence programs designed primarily to fund the very best researchers, proposals, and project teams across Europe, or whether they should also aim to widen the European research landscape (Caloghirou, Tsakanikas, & Vonortas, 2001; Hoekman et al., 2013; Makkonen & Mitze, 2016).

The composition of the project teams also raises questions of additionality; namely whether the collaborative funding mechanisms are actually able to create new partnerships, or if they simply reproduce patterns already firmly established in the European research landscape. However, investigations on only awarded collaborations cannot deliver sufficient insight into the types of organization or project consortia that are most likely to secure funding. Mostly due to data limitations, empirical studies on rejected project applications are highly underrepresented in the literature (Bornmann et al., 2010). For the EU FPs, Enger and Castellacci (2016) and Enger (2018) are important exceptions, investigating the participation of universities in both funded and nonfunded Horizon 2020 (H2020) project applications. However, these studies consider only a specific country (Enger & Castellacci, 2016), or type of organization (Enger, 2018), and are thus limited in scope. They can provide only partial insights into how the publicly funded R&D network structures spanning Europe came into being.

In this study we focus on project consortia comprising different types of organizations from different European countries to systematically analyze which types of project teams are more successful in receiving EU H2020 funds for collaborative R&D. By viewing R&D collaboration as a dynamic process (Kumar & Nti, 1998; Majchrzak, Jarvenpaa, & Bagherzadeh, 2014), we assume that the social and institutional composition of the project consortium is crucial for partner-specific learning, particularly in the early collaboration phase, and therefore consortium-specific factors have a significant influence on the chances of success of collaborative project applications. Based on arguments from social network theory and previous literature on EU FPs, we theoretically identify and empirically test a specific set of success factors related to (a) the prominence and visibility of the consortium in the H2020 network; (b) the experience of its members with previous FP projects and the degree of acquaintance among them; and (c) the research capabilities and academic excellence or reputation of the members.

Our empirical investigation draws on proposal evaluation data as included in the eCORDA database. We restrict our sample to project proposals submitted to the H2020 Societal Challenges pillar, resulting in a final sample of around 23,000 organizations located across Europe collaborating in about 7,000 consortia (projects) during the period June 2014 to June 2016. To systematically assess how consortium characteristics relate to a distinct proposal outcome, we rely on different types of regression models and outcome variables: First, we use data on the status of the project proposal (main list, above threshold, below threshold) in a multinomial logistic regression (MLR) framework, and second, we make use of proposal evaluation scores in an ordinary least squares (OLS) regression model.

Our study is an important addition to the existing literature on publicly funded R&D, in particular EU FPs, for several reasons: First, we draw on a unique and comprehensive data set including all rejected H2020 project proposals to differentiate between successful project applications, as typically identified in the literature, and unsuccessful ones. Second, with the focus on the project consortia rather than specific organizations, we provide novel evidence of the proposal success factors as they relate to entire project teams participating in H2020. Third, our empirical results are an important contribution to better dissecting the H2020 network structures, the underlying funding patterns, and associated knowledge diffusion structures, as requested by the literature on EU FPs for some time.

The remainder of the paper is structured as follows: Section 2 discusses the importance of consortium-specific characteristics in early-stage collaborations, from which our hypotheses about distinct social and institutional factors and their influence on the success of proposals are derived. Section 3 introduces the scale and scope of the H2020 database and our proposal success measures, before Section 4 explains the empirical modeling strategy. Section 5 presents the regression results, and Section 6 concludes with a synthesis and some policy implications derived from our empirical results.

Project-based collaborative R&D funds are a key element in most public research funding portfolios, and are designed to bring together researchers from different research traditions, institutional backgrounds, and locations. Temporary partnerships in the form of short- to medium-term projects should enable researchers to better meet the challenges posed by the increasing complexity of knowledge and the need to integrate a diversity of technologies, as well as the growing demand for solutions of immediate relevance to society (Ahuja, 2000; Cummings & Kiesler, 2005; Wuchty, Jones & Uzzi, 2007).

Typically, project-based R&D alliances follow an evolutionary process that can be characterized by different stages and collaboration dynamics (Kumar & Nti, 1998; Majchrzak et al., 2014). Initially, interorganizational collaborations start by forming a consortium to jointly develop a research agenda, followed by an operational stage in which the collaboration agreement or research plan is translated into reality. A central finding of studies in innovation and collaboration management is the changing modes and intensities of interaction throughout the collaboration process, characterized by different patterns of learning between partners over time (Das & Kumar, 2007; Majchrzak et al., 2014; Thune & Gulbrandsen, 2014). It is assumed that content-related factors, such as cognitive proximities, the absorptive capacities, and the ability to combine knowledge across partners (content learning), are crucial for the operational stage of a collaboration, while at the start of a project, social and institutional factors appear more important because they initially shape the cooperation and condition its continuation (Thune & Gulbrandsen, 2014). Partners need time to develop a mutual understanding of a phenomenon, to identify complementarities, and to balance organization-specific interests dedicated to the research plan. “Partner-specific learning” (Das & Kumar, 2007) enables the development of mutual trust, planning for the project’s future stages and the management of the partners’ related expectations. For long-distance or cross-sectoral partnerships in particular, the first stage may be complex and time-consuming (Torre, 2008). However, the better the partners manage the early stage, the more effective and successful the outcome of the collaboration.

Focusing on consortium-specific social or institutional factors would seem to be an appropriate anchor point for the investigation of publicly funded R&D collaborations in their proposal phase. Here, the early collaboration stage encompasses the search for partners and complementarities on the input side (i.e., knowledge, skills, interest), the development of a joint research agenda (i.e., the proposal), and task coordination and resource division within the consortium (i.e., financial and time), which eventually leads to the joint project application. The better the cooperation during this early phase, the higher the potential quality of the proposal and the chances of success in acquiring the research funds. Consequently, the probability of receiving funding may not only be determined by the content, thematic orientation, or skill level and expertise match of the individual organizations. Instead, the interplay of social or institutional factors is assumed to facilitate or hamper effective interaction and partner-specific learning.

In this study we investigate whether distinct partner configurations are more likely to lead to proposal success. We define a consortium associated with a proposal as a temporary alliance between organizations, engaging in early-stage collaboration to jointly apply for research funds to later conduct collaborative research within the boundaries of a specific project. Based on previous studies on publicly funded research in general (Abbasi, Hossain, & Leydesdorff, 2012; Bornmann et al., 2010; Viner et al., 2004) and collaborative research and networks within EU FPs in particular (Enger, 2018; Lepori et al., 2015; Paier & Scherngell, 2011), we derive a set of potential consortium-specific success factors for further investigation in our empirical analysis.

Collaborative research is increasingly viewed in terms of a network system. Given the participation in different R&D projects, research performing actors or organizations are directly and indirectly connected in a network (Breschi & Lissoni, 2004; Newman, 2001; Tomasello, Napoletano, et al., 2016). An actor’s embeddedness in a global network structure is supposed to influence the decision with whom to engage in collaboration, the quality of knowledge flows and the value arising from the collaboration (Burt, 2005; Gilsing, Nooteboom, et al., 2008; Granovetter, 1973). It is assumed that highly connected actors have a comparative advantage over others, as they are more deeply embedded, more exposed to novel knowledge, and more likely to receive strategically valuable information (Owen-Smith & Powell, 2004; Tsai, 2001; Wanzenböck et al., 2015).

We argue that, at the start of a joint research endeavor, the advantages of great network visibility become manifest in the search for synergies between an actor’s own competences and the expertise of others. Network visibility facilitates the formation of multidisciplinary or multi-institutional partnerships, and facilitates contact to strategically important partners such as star scientists or organizations, which can boost the likelihood of receiving public funding. Moreover, centrally positioned partners can more easily seek the tacit knowledge needed to align with the criteria of the research call, and to design a more targeted proposal accordingly. Such network effects, often referred to as preferential attachment or accumulative advantage, can explain the emergence of a Matthew effect, as often observed for publicly subsidized research (Bornmann et al., 2010; Enger, 2018; Viner et al., 2004). We therefore propose a positive relationship between network centrality and proposal success and test the following hypothesis empirically:

In addition to external networks beyond the consortium, the social relations between consortium partners and a consortium’s internal cohesion may determine the quality and success of a research proposal. When applying for collaborative research funds, two factors seem to be particularly important: (a) the level of acquaintance among the consortium partners, and (b) the level of managerial experience in coordinating project partners within the consortium. The former relates to the idea of “strong ties” between consortium members who are closely connected and enjoy a certain level of mutual trust (Granovetter, 1973; Krackhardt, 2003). Such strong relationships are usually the result of repeated interactions over time. A shared history allows collaboration partners to draw on partner-specific experiences, established routines, and interaction modes, making it easier for the consortium to tackle the challenges of the early stage (Das & Kumar, 2007). For multidisciplinary research teams in particular, previous relationships may be important in bridging differences in perspective caused by dissimilar work cultures, scientific norms or typical approaches to problem solving, to strengthening the commitment to the joint project and team cohesion, and to developing a shared research vision (Ratcheva, 2009). Knowledge about the partners’ expertise and skills can contribute to a more efficient division of tasks among partners and facilitates the development of a coherent thematic orientation.

A second factor related to experience is the ability to effectively manage a team of diverse project partners. Typically, the management responsibility for a joint R&D project lies in the hands of a single project coordinator who plays a central role in handling the information exchange and knowledge flows between the partners (Enger & Castellacci, 2016; Maggioni, Uberti, & Nosvelli, 2014). In large-scale and complex project constellations, management capabilities are essential to select the right project partners, to establish effective communication structures, and to align the activities and expectations with respect to the project aims. We therefore argue that coordination experience contributes to successful project acquisition due to learning effects (such as those related to the specifics of the funding scheme or the qualities of partners) that facilitate the development of a coherent project proposal. Accordingly, we will test the following hypothesis empirically:

Empirical studies suggest that both the research capabilities and scientific excellence of organizations have an influence on FP participation and the likelihood of receiving funding (Enger, 2018; European Commission, 2017; Lepori et al., 2015). Researchers and organizations with a good past performance and an established knowledge base in the field are likely to be more productive, generate higher impact research, and be more likely to acquire funding for their research idea (Van den Besselaar & Leydesdorff 2009). Moreover, organizations with an established scientific or research reputation can typically select the partners they would prefer to engage with in a joint project. The favorable opportunities for selection may create a tendency towards collaborating with partners of similarly high quality (a phenomenon typically referred to as homophily in social network theory; McPherson, Smith-Lovin, & Cook, 2001), and a bias towards participating only in the most promising consortia or proposals.

Given that a project proposal in the EU FP depends on the quality and research profile of the entire consortium, we can assume that project consortia involving a large number of high-quality and high-impact organizations will achieve a higher evaluation score for their project. We therefore propose the following hypothesis:

With the support of precompetitive, collaborative R&D, the overall aim of the EU FPs has been to strengthen the scientific and technological base of the European scientific community and economy (Barker & Cameron, 2004; Delanghe, Muldur, & Soete, 2009). Even though the fundamental rationale has remained unchanged since its inception in 1984, the most recent FP—H2020—specifically promotes research and innovation on the basis of three objectives: scientific excellence, industrial leadership, and societal challenges, as institutionalized in the three program pillars of H2020 (European Commission, 2011). This study focuses solely on collaborative R&D projects funded under the Societal Challenges pillar, a thematic H2020 framework incorporating seven predefined themes¹. Over the entire funding period, from 2014 to 2020, the total budget for H2020 is €77.2 billion; at 37%, funding for Societal Challenges accounts for the largest share of the total budget.

For this study, we are drawing on the H2020 eCORDA proposal database (publication date: September 2016) containing largely harmonized applicant data² and project data for all evaluated projects (funded and not funded) that apply for H2020 funds. To analyze proposal success in H2020, we restrict the empirical sample as follows: First, to ensure the consistency and comparability of our observations, we extract only projects funded under the Societal Challenges pillar between 2014 and 2016. Second, we focus only on projects involving instruments that finance actual R&D (i.e., innovation actions, research and innovation actions), and exclude funded coordination and networking activities (so-called Coordination and Support Action (CSA), such as policy dialogs or training). Third, to exclude outliers, we consider only projects with a consortium size of more than three and fewer than 16 partners. Our final sample consists of 7,208 project proposals (i.e., consortia). We consider around 69,357 participations by around 22,612 distinct organizations, based on which we construct the H2020 Societal Challenges pillar network. The majority of these participations come from the private (for profit) sector (36%), followed by research organizations (19%) and universities (36%). Public bodies account for 5% and others for 4%.

Each H2020 proposal subjected to the evaluation procedure receives an overall score based on an assessment by experts. This expert score includes individual assessments in the categories of impact, excellence, and implementation, and typically ranges from 0 to 15³. According to the score achieved, the project proposals are ranked and assigned a certain status. Three different project statuses can be distinguished: (a) below threshold: a project proposal is assessed below a certain threshold and rejected; (b) above threshold: a project is assessed above threshold but rejected; and (c) main list: a project is included in the main list and selected for funding. The threshold scores are predefined and may differ for each H2020 Societal Challenges program. Data on both project status and evaluation score are contained in the eCORDA database, and will be used later as dependent variables in our empirical model variants.

Table 1 shows descriptive statistics on the outcome of the proposal assessment (i.e., project status) and its distribution across the seven H2020 programs. We see that only around 11% of the project applications were successful, while 31% were evaluated as being above threshold but not considered for funding, and 58% were below threshold and immediately rejected. Most of the proposals were submitted to the Health program, with the number of submissions almost double the quantity of those submitted to the second largest program (Society). The Food program records the smallest number of submitted proposals. The Society program shows the lowest success rate, with only around 5% being assigned to the main list. The success rate was highest for the Transport program, with more than a quarter of all submitted proposals receiving H2020 funding.

Moreover, we can compare the structure of the interorganizational H2020 networks based on (a) only the successful (main list) project proposals, and (b) all (successful and nonsuccessful) proposals submitted to the Societal Challenges pillar. Table 2 shows that we can observe only minor differences in the global network structure and connectivity of organizations in the two networks. Figure 1 underlines this finding, showing that the main network component is densely knitted for both networks. It should be noted that the construction of both interorganizational networks is based on a two-mode network data structure, in which the organizations are linked by joint membership in H2020 project applications. This means that all project partners are interlinked, even though it is likely that not all of the partners know each other equally well, particularly during the proposal phase or in large consortia (Breschi & Cusmano, 2004). This assumption of fully connected project consortia can at least partly explain the high number of organizations in the main network component and the structural similarities between the two networks. With respect to organization type, we see that high centrality actors are mostly universities and research organizations that are located at the core of the network and surrounded by a second “ring” consisting mostly of private for-profit organizations, such as industrial companies or consultants. In the main component of the successful proposal network, we observe a few densely connected “cliques” with high internal connection but only loose relations to other actors, while such a clique structure cannot be observed in the full network.

A closer look at the participation intensity of the three major organization types in the H2020 Societal Challenges pillar is seen in Table 3, which shows that private commercials (mostly companies) contributed to project proposals almost as often as higher education organizations (mostly universities), whereas the participation rate for research organizations is somewhat lower. We also see that the higher education sector and the private sector requested similar shares of funding, pointing to a fairly balanced participation of the two major sectors across all the proposals (successful and nonsuccessful).

The explanatory variables in x reflect (a) network visibility, (b) experience and acquaintance, and (c) the research capabilities and excellence of a project consortium as discussed in Section 2. Network visibility is defined as the network centrality of the consortium, measured in terms of the sum of the centralities of the individual project partners as calculated on the basis of our full H2020 network for all project proposals in Societal Challenges. Network centrality is considered in terms of degree and eigenvector centrality (Bonacich, 1987; Wasserman & Faust, 1994)⁴. Because network centrality measures are typically highly correlated (Valente et al., 2008), we consider them separately and estimate different model variants.

Experience and acquaintance of partners in the consortium is measured by two variables: First, coordination experience is a dummy variable and takes the value of 1 if the project coordinator has already coordinated a project funded by the 7th Framework Programme (2007–2013), and 0 otherwise. Second, acquaintance is a count variable related to the sum of joint participations of organizations (i.e., project partners) in FP7 projects. To measure such prior FP collaborations at the level of organizations, we match the organizations in our sample with the FP7 project and applicant data as contained in the FP7 eCORDA database⁵.

To reflect the research capabilities and excellence of the consortium, we consider both the application orientation (technological development) and scientific excellence of the project team. The first variable (patents) is measured by the number of patents of the consortium (i.e., all project partners) in societal challenge-relevant technologies for the period 2007 to 2015. For this purpose, we match the organization names in the H2020 eCORDA database with the information on patent applicants as included in the OECD REGPAT database. Patents relevant to Societal Challenges were selected on the basis of the IPC classification suggested by Frietsch, Neuhäusler, et al. (2016). We use fractional counting to construct the variable. Scientific excellence is a count variable reflecting the number of project partners belonging to one of the top 50 universities according to their publication impact as listed in the CWTS Leiden Ranking 2017 (Centre for Science and Technology Studies, 2017). The ranking is based on the number of a university’s publications belonging in the top 1% of most frequently cited publications in their field⁶. As such, scientific excellence is considered as a proxy for a university’s reputation, assuming that especially elite or prestigious universities can be more selective in their choice of proposal or partner.

Next to this core set of variables, we include a number of control variables, which reflect more general consortium member or project-related characteristics. These include the institutional composition of partners, measured as the share of consortium partners belonging to the major sectors; that is higher education organizations (mostly universities) (HES); private commercials (mostly companies, PRC); or research organizations (REC). Furthermore, we construct a CEE country variable, which reflects the number of consortium members from Central and Eastern European Countries (EU-13).

In addition, we include a dummy variable to indicate whether the project is part of a multilateral public-private partnership (PPP) initiative. PPP initiatives⁷ develop strategic plans mostly with the aim of strengthening European economic competitiveness, and should be implemented through specific calls under the H2020 program. Project consortia based on such PPPs are highly experienced, trust based, and thematically focused, and can have a considerable influence on agenda setting in the H2020 program lines. Our variable partnership initiative takes the value 1 if the project is part of such a PPP initiative, and 0 otherwise. To control for size effects, we compute the variables consortium size, which is the number of project proposal members, and project size, which measures the amount of funding requested in the project proposal. Moreover, we include dummy variables for each thematic program in the H2020 Societal Challenges pillar to control for potential heterogeneities in our estimations due to broader project themes and research field-specific characteristics. An overview of all the variables is given in Table 4.

Table 5 presents the estimation results for our proposal success model based on the MLR model as specified in Eq. 1. Due to the high negative correlation between the share of private firms (PRC) and share of universities (HES) in a consortium (see Table A2 in the  Appendix), we were not able to estimate a model including both variables. We therefore estimated two model variants; the first four columns contain the results for the model specification with HES shares, and the last four columns the results for the model with PRC shares. We also estimated different model variants for degree centrality (Models 1 and 3) and eigenvector centrality (Models 2 and 4). Associated marginal effect calculations are reported in the  Appendix (Table A3). The model variants can be interpreted separately as if they were simple logit models. The project status “below threshold” (category 1) is the reference for interpreting the coefficients for all MLR model specifications. Table 6 shows the OLS estimation results of the model based on the expert scores as specified in Eq. 2.

Turning now to the estimation results, we observe a positive relationship between the network visibility of a consortium and the evaluation outcome of its H2020 proposal for the expert score models (Table 6) and all “above threshold” model variants (Table 5). Interestingly, in the “main list” variants the coefficients for degree centrality are insignificant, but slightly significant and positive for eigenvector centrality for the HES model (Model 2). The marginal effects as illustrated in Figure 2 for different levels of degree and eigenvector centrality further explain these results. We see that, while keeping all other variables constant at their mean values, the effect on proposal success does not increase for degree centrality, but increases slightly with higher levels of eigenvector centrality. This result suggests that being more directly connected to core players can indeed increase the likelihood of succeeding in the application for H2020 funding. Our Hypothesis 1 stating a positive relationship between network visibility and proposal success is therefore only partly confirmed by the empirical results.

It is noteworthy that our findings are only partly in line with the Enger (2018) study, which, although it only looks at highly connected universities, points to an unambiguously higher likelihood of proposal success. If we look at the network visibility of entire consortia, it seems that high network embeddedness does indeed positively contribute to submitting a strong proposal (above threshold). However, highly connected consortia are not necessarily more likely to get funded in H2020 (main list) than consortia with a lower centrality. The findings for eigenvector centrality may, at least partly, explain the closed and oligarchic network structure of FP networks revealed in the studies for funded collaborations (e.g., Autant-Bernard et al., 2007; Breschi & Cusmano, 2004).

Regarding our second hypothesis, we observe that coordination experience in the predecessor program (FP7) is a good predictor of proposal success in H2020, the follow-up edition. The likelihood of achieving “main list” or “above threshold” project status is significantly higher for consortia with an experienced coordinator compared to coordinators without FP7 experience. However, as shown in Figure 3 (left plot), we do not observe significant differences between the effects of coordination experience on being ranked on the main list and above threshold.

Moreover, following the results of our project status models, it seems that acquainted project partners also show a slightly higher likelihood of being ranked on the main list, but with a low marginal effect of one additional prior joint participation (see Table A3 in the  Appendix). Also, having “strong ties” within the consortium does not significantly influence the likelihood of achieving an assessment score “above threshold.” The expert score model shows for acquaintance even a significantly negative, but small, effect of one additional prior joint participation. We achieve a similar result for project consortia linked to partnership (PPP) initiatives. These consortia are also more likely to be on the main list, but are not necessarily more likely to achieve the project evaluation “above threshold” (see also Figure 3, right plot).

Based on these slightly contradictory results, we can confirm our Hypothesis 2 only for the main list models. The proposed positive relationship between coordination experience or acquaintance and proposal success is significant when it comes to highly evaluated proposals with successful outcomes that are finally eligible for FP funding. This result may indeed explain the tendency for European research collaboration programs to produce closed clubs of familiar and experienced project partners. However, it seems that knowing the funding procedures and the managerial tasks related to project coordination is more important than prior acquaintance between the project partners (“strong ties”) for proposal success in the early stage of a collaboration.

Regarding the influence of research capabilities, we observe a clear pattern throughout all model variants. The probability of submitting a successful proposal increases significantly with the number of top universities in a consortium (Figure 4). It appears that scientific excellence is indeed a success criterion for H2020. However, it should be noted that we cannot fully rule out the existence of a selection bias here. It may be that so-called star universities are better able to act strategically with respect to their collaboration choices. They can be more selective than universities with a poorer reputation, and may therefore only engage in projects for which they can assess ex ante a high likelihood of success. In contrast to scientific capabilities, consortia with a broad applied knowledge base (as measured by the number of patents) are not significantly more likely to produce successful proposals. On this basis, the empirical results confirm our Hypothesis 3 only for scientific research capabilities, but not for more applied or technology-specific capabilities.

Beyond our main hypotheses, it is also worth discussing the results for the institutional composition of a consortium. The variables for the share of private companies (PRC) and the share of universities (HES) are highly negatively correlated. This finding suggests that most of the H2020 project proposals take one of two forms: They are either more application oriented (with a high share of companies) or more science oriented (with a high share of universities). The regression results show that consortia with a high share of universities are significantly less likely to achieve an evaluation score “above threshold” or to be listed on the main list. In contrast, consortia dominated by private companies (PRC) and research organizations (REC) appear to have a higher probability of proposal success. These findings seem to be in line with the general FP goals and the H2020 orientation, which focuses on application-oriented research with immediate societal relevance for the Societal Challenges pillar.

To further investigate this finding, we also include the ratio of the share of PRC to HES organizations as an alternative measure for the institutional composition of a consortium. This variable is a rough proxy for the balance of PRC and HES, or the relative application orientation of a project. The average ratio of PRC to HES organizations across all projects is 1.5 (Table A1 in the  Appendix). We see that PRC and HES jointly applied for a project in 85% of the proposals, while 1.5% of all proposals were submitted by consortia including only HES organizations and 0.5% by consortia including only PRC organizations (Table A4). The regression results ( Appendix, Table A5) confirm our finding that more application-oriented proposals have high chances of success.

Finally, of interest from a pan-European perspective is the significant and negative coefficient for partners from Central and Eastern Europe (CEE). It suggests that the probability of achieving a good project evaluation significantly decreases with the number of CEE partners in a consortium. A one-unit increase in the number of CEE partners decreases the likelihood of being on the main list by more than 1.5%.

This study sheds light on the types of project consortia that are more successful in acquiring collaborative R&D funds within Horizon 2020, the latest edition of the EU’s largest research funding scheme. The aim was to explore the systematic patterns underlying the allocation of research funds across research organizations and regions in Europe. We hypothesized that, in addition to content-specific factors, a consortium’s characteristics related to the social, network, or institutional compositions of a consortium can significantly influence its success in securing funding for collaborative R&D projects. For the empirical analysis we used evaluation data (score and project status) for all H2020 proposals, including funded and nonfunded proposals, which were submitted to the Societal Challenges program pillar between 2014 and 2016. Our study covers project applications submitted by over 7,000 project consortia involving around 23,000 research organizations across Europe to systematically reveal those specific partner configurations that are more likely to successfully pass the proposal stage.

We found that consortia achieve a significantly better evaluation score or project status if they involve (a) a coordinator with experience in previous Framework Programmes, (b) a larger number of top universities with a high level of high scientific excellence and impact (i.e., established reputation), and (c) a large number of partners from Western Europe (former EU15). This success profile of high experience and high reputation consortia appears to be in line with the general H2020 program objectives, namely to be the primary program for research excellence in Europe. However, when it comes to better integrating all EU member states—often claimed as a core objective for an EU-level research funding program—this “widening” aspect cannot be confirmed by our results. Hence, H2020 funds consortia meeting the excellence criteria, but focuses less on factors related to the cohesion or inclusion of actors across Europe. The fact that high-quality connections between consortium partners (in terms of their eigenvector centrality), rather than the pure number of project partners and linkages (in terms of degree centrality), positively influence the proposal success would appear to be in line with these findings.

In view of H2020’s ambitions to promote excellence, it is interesting that consortia composed of a high share of universities have significantly lower chances of success than application-oriented consortia, even though universities show the highest participation intensity in H2020 Societal Challenges. Only consortia that include top universities outperform other consortia in terms of proposal success. In contrast, H2020 funding criteria appear to discriminate against consortia with a high university share, typically those more oriented towards discovery and frontier research but at a lower Technology Readiness Level (TRL)⁸. Arguably, the Societal Challenges pillar of H2020 appears to maintain the tradition of previous EU FP editions as a funding program primarily designed for applied projects that promise economic or societal returns in the short term. The fact that universities in particular show a higher tendency to collaborate among themselves, paired with the negative correlation between consortia dominated by scientific organizations and those with a high share of private companies, points to a low integration level of science and industry partners in the projects. Although closing the gap between science and industry is one of the objectives of the multidisciplinary and transdisciplinary Societal Challenges pillar, this gap is still evident and seems to be reproduced by the evaluation and funding criteria at the project level. However, similar to predecessor FPs, research organizations involved in applied research seem to fill this gap. Consortia with a higher share of application-oriented research organizations—the so-called FP core organizations—are more successful in acquiring H2020 projects, as these organizations may fulfill the requested bridging role between science and industry.

Our study is among the first to draw on nonsuccessful R&D projects to clearly differentiate between proposals that are not able to pass the early collaboration stage, and successful proposals that result in “‘real” R&D collaborations. Consequently, our results can be seen as an important step towards dissecting and getting to the “core” of the H2020 network, with its associated structures of collaboration and knowledge diffusion across Europe. In this regard, proposal success for policy-funded research could be explained by two factors: a quality factor, which is determined by how well the project team manages to work together and develop a good proposal (scientific capabilities and experience), and more systemic factors, related to the funding criteria and aims, which favor application-oriented consortia constellations.

Although we were able to provide initial insights into both factors, new research strategies involving large-scale control group methods or semantic methods (e.g., text mining procedures) for closer examinations of the proposal content would be necessary to shed more light on the patterns of funding and the existence of a policy-induced bias in awarding research grants. These issues, as well as the identification of a broader set of success factors, will be important topics for future scientific research on the evaluation and impact of public research funding.

Iris Wanzenböck: Conceptualization, Formal analysis, Investigation, Methodology, Supervision, Validation, Visualization, Writing—original draft, Writing—review & editing. Rafael Lata: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Visualization, Writing—review & editing. Doga Ince: Data curation, Formal analysis, Methodology, Validation, Writing—review & editing.

The authors have no competing interests.

Iris Wanzenböck benefited from funding of the INTRANSIT project funded by the Norwegian research council.

In this study we used detailed microdata about proposals including personal data of the applicants. Due to the confidentiality rules for Framework Programme data stored in eCORDA, we are not allowed to publish any personal or sensitive data provided by the applicants.

We want to thank three anonymous referees for their useful comments and suggestions, and are grateful for the valuable feedback during the GeoInno 2018 and Sunbelt 2018 conference.