Abstract

Innovative firms with good ideas may still struggle to fine-tune them to the stage where they can attract outside funding. We conduct a five-country randomized experiment that tests the impact of an investment readiness program. Firms then pitched their ideas to independent judges. The program resulted in a 0.3 standard deviation increase in the investment readiness score. Two years later, the average impacts on firm investment outcomes are positive but small in magnitude and not statistically significant. Larger and statistically significant impacts on receiving outside funding occur for smaller firms and for firms with lower likelihoods of otherwise being funded.

I. Introduction

EVEN when innovative start-ups and SMEs in developing and transition countries have good ideas, they may not have these ideas fine-tuned to the stage where they can attract outside funding. This is the case in the western Balkans, where there is a perceived lack of investment readiness of innovative start-ups to be in a position where they can compete for, and take on, outside equity (Karajkov, 2009). The most common reasons for a lack of investment readiness include entrepreneurs' reluctance to surrender partial ownership and control of their business, lack of knowledge about the availability of external sources of finance, low investability of business development propositions, a lack of understanding of the key factors investors look for in making investment decisions, and presentational failings such as deficiencies in business pitches (Mason & Kwok, 2010).

While historically government assistance to small firms has taken the form of basic business training and loan support, there has been rapid growth in other types of programs designed to spur and support more innovative start-ups and help them attract outside funding. Policymakers seeking to assist potential high-growth firms face a choice along a continuum between high-intensity, individualized programs that can be difficult to scale and less-intense programs that can cater to many firms but potentially lack sufficient intensity to improve them. The most common intensive approach is to support business accelerators and incubators. These often offer firms some seed capital and work space, in addition to training and mentoring. Globally, the number of accelerators and incubators grew more than fivefold between 2009 and 2018, reaching over 2,500 active structures (Berger, 2019). However, the majority of these work only with small cohorts of ten to twenty firms at a time (e.g., Y-combinator and Tech-stars), and they can be expensive to establish and run. For example, Start-Up Chile takes 100 firms a year at a cost of $15 million (Gonzalez-Uribe & Leatherbee, 2018). The contrasting approach has been to offer short courses of classroom-based or online training over short periods.1 While governments and NGOs have spent billions on training programs, the majority of these efforts are aimed at building the basic business skills of aspiring or new entrepreneurs or in teaching start-ups how to write a business plan, not on readying innovative firms to attract equity finance.

Investment readiness programs are a relatively new intervention that provide a middle ground between the intensive and expensive accelerator/incubator approach, and the cheap and quick classroom training approach. They are intended to provide a comprehensive approach to overcome the constraints to firms receiving outside investment through a mix of individualized training, mentoring, and coaching, at an intensity that is sufficient to make firms more investment ready, while maintaining a cost that is low enough to be scalable to large numbers of firms (Mason & Harrison, 2001; Mason & Kwok, 2010). While global data on the prevalence of these programs are not available, appendix 1 provides 31 examples of investment readiness programs being used in a wide range of countries, including in the United States by the National Science Foundation and several government agencies, the European Union in western Europe, Australia, New Zealand, Malaysia, Morocco, and in multiple countries in East Africa and eastern Europe. However, to date there is no causal evidence as to the effectiveness of these programs, only descriptive studies that do not have control groups (Mason & Kwok, 2010).

We conduct a five-country randomized experiment in Croatia, Kosovo, Macedonia, Montenegro, and Serbia to test the effectiveness of an investment readiness program. A sample of 346 innovative SMEs was randomly divided into two groups: a treatment group that received a high-cost and intensive program that involved help developing their financial plans, product pitch, market strategy, and willingness to take equity financing, along with master classes, mentoring, and other assistance; and a control group, which received access to an inexpensive online-only basic investment readiness course. After this program, both groups of firms competed in a pitch event, where they were scored by independent judges (blinded to treatment status) on their investment readiness.

The independent judges scored the pitches on six aspects of investment readiness: team, technology, traction, market, progress, and presentation. We find that firms that went through the investment readiness program received an average of 0.3 SD higher investment readiness scores at this event and were more likely to get selected to proceed to pitch in front of investors. We then tracked firm outcomes over the next two years via a six-month and two-year follow-up survey. We found positive but statistically insignificant impacts on firm survival, three categories of investment readiness, and steps toward receiving external financing. Treated firms were 5 percentage points more likely to receive external financing, but the 95% confidence interval of (-4.7 p.p., +14.7 p.p.) includes 0 and negative impacts.

We explore several possible explanations for this modest average effect of the program. The judges did appear to be measuring something meaningful, with higher scores from judges being significant predictors of firm outcomes two years later. However, the magnitude of the average change in investment readiness, coupled with that of the association between investment readiness and firm outcomes, would predict impacts on firm financing that we lack statistical power to detect. A key reason for this low power is that some of the control groups were more successful in getting outside funding than we had originally anticipated. Heterogeneity analysis then shows that the program increased investment readiness only for firms that were below the median size at baseline and that for these firms, the program led to a statistically significant 15 percentage point increase in their likelihood of getting outside funding. Similarly, applying the Abadie, Chingos, and West (2018) endogenous stratification methodology, we find the program had positive and significant 12 to 14 percentage point impacts on the likelihood of getting external financing for firms that would otherwise have low likelihoods of getting such financing. The modest average effect of the program therefore arises from averaging larger effects for firms that would otherwise struggle to find financing with no effects on firms for which finding financing was relatively easier. This points to the importance of correctly targeting these programs.

II. What Are Investment Readiness Programs, and Why Implement One in the Western Balkans?

A. What Are Investment Readiness Programs?

While much policy attention around the world has been given to efforts to expand the supply of equity finance for innovative start-ups and SMEs, the effectiveness of these efforts can be hampered by a lack of readiness of firms to receive equity investment. Mason and Kwok (2010) highlight three main aspects of this lack of readiness. First, many entrepreneurs are believed to be equity averse, unwilling to surrender any ownership stake in or even partial control of their firms. Second, many businesses that seek external finance are not considered “investible” by external investors due to deficiencies in their team structure, marketing strategy, financial accounts, intellectual property protection, and other business areas. Third, even if entrepreneurs are willing to consider equity and have investible projects, presentational failings mean that many firms are unable to pitch their ideas successfully to investors.

Investment readiness programs are intended to increase the effective demand for equity financing by helping firms overcome the factors that result in a lack of investment readiness, thereby enlarging the size and quality of the pipeline of potential funding opportunities for investors and increasing the likelihood of new equity investments being made. Appendix 1 provides examples of these programs. While there is substantial heterogeneity in the content of these programs, the most comprehensive programs usually cover four dimensions, based on the core reasons that many investment deals do not materialize (Mason & Harrison, 2001; Mason & Kwok, 2010). The first dimension aims at reducing equity aversion by explaining to entrepreneurs the potential advantages that equity can bring to the firm as both a source of funding and because of the knowledge outside investors can bring to the firm. The second dimension addresses the investability of the business by helping to train the entrepreneur to demonstrate that this person has a viable revenue model, can measure market traction, has dealt appropriately with property right issues, and has a competitive strategy, for example. The third dimension works on the presentational skills, teaching the entrepreneur how to effectively pitch business ideas and provide the key information investors are looking for. Finally, some programs also offer a networking dimension, aiming to facilitate the matching process between entrepreneurs and investors through events such as venture forums.

These programs tend to be subsidized by governments, even in developed economies like the United States and United Kingdom. There are several possible reasons to justify subsidies. The first is that the targeted firms are frequently liquidity constrained and therefore unable to pay. Some incubator and accelerator programs like Y-Combinator overcome this constraint by investing seed capital in the firms in exchange for an equity stake in the business. But since equity aversion is one of the key constraints investment readiness programs are trying to overcome, investment readiness programs have typically not required equity stakes in exchange for participation. Second, since many of these programs are new in nature, potential entrepreneurs may find it hard to assess in advance the overall quality of the program, and their payoffs from participation are highly uncertain, making them unwilling to pay the costs of participating. Finally, governments may justify the subsidies in terms of the public benefits (more innovation, higher tax revenues, greater employment) that can come from successful ventures.

B. Why an Investment Readiness Program in the Balkans?

Increasing innovation is a key regional priority in the Balkans region as a way to boost firm productivity and sustain economic growth. While it is generally accepted that debt finance is not the optimal source of funding for early-stage SMEs and start-ups, equity finance has historically been marginally used only in the region. For example, Vizjak and Vizjak (2016) report that in Croatia in 2014, only fifteen start-ups received financing from venture capital funds and business angels, totaling 21.8 million euros, while Gattini et al. (2016) report only one or two transactions per country in Kosovo and Montenegro. A regional report noted that there is a debate as to how much this lack of use of risk capital reflects a lack of supply of equity finance versus a lack of readiness of entrepreneurs to attract and accept this financing (Karajkov, 2009). Based on the viewpoint that action was needed on both the supply and demand sides, the Enterprise Development and Innovation Facility (EDIF) initiative financed by the European Commission includes efforts to increase the supply of private equity to the region, improve the legislative frameworks to better encourage venture capital activity, and undertake efforts to increase investment readiness. This paper provides an evaluation of the investment readiness component.

III. Experimental Design

To implement this intervention, we ran a competitive procurement process where companies specializing in investment readiness programs provided bids. The winning firm was the company Pioneers JFDI GmbH. Founded in 2009 and based in Vienna, it is one of Europe's leading platforms for entrepreneurship, organizing an annual Pioneers Festival (with 3,000 attendees), as well as providing mentoring, pitch training, and opportunities for presentation and networking with European and international founders and investors. It launched a specific investment readiness program, Pioneers of the Balkans, for this project.

A. Generating the Sample

Eligibility criteria for the program were developed by the World Bank and Pioneers team, conditional on the rules of the European Commission. To participate in the program, a firm had to be legally registered in at least one of the five countries: Croatia, Kosovo, Macedonia, Montenegro, or Serbia. The firm had to be a micro, small, or medium-size enterprise, defined as having fewer than 250 employees, and an annual turnover below 50 million euros. It had to be innovative, meaning that “it will in the foreseeable future develop products, services, or processes which are new or substantially improved compared to the state of the art in its industry, and which carry a risk of technological or industrial failure,” and it could not be on a sanctions list or operating in a set of negative activities (e.g., gambling or alcohol production).

To launch the program, the brand Pioneers of the Balkans was created, and a dedicated website was set up. The program was marketed as a competitive program designed especially for innovative entrepreneurs seeking or considering venture financing.2 The main communications therefore promoted a major pan-regional start-up competition due to take place in two stages, with the semifinals in Belgrade and subsequent finals event in Zagreb. It included a preliminary list of investors who had already confirmed their attendance at the finals and noted that selected firms would receive a training and preparation package.

We had set a target of 300 to 350 participating firms. In designing the program, both providers of investment readiness services and experts in the innovation agencies agreed that there was a limit on how many firms potential investors would be willing to listen to pitches from.3 A two-stage process was designed to overcome this issue: the semifinals would be the main phase of our study, with all firms in the study having a chance to present their ideas in the semifinals and get scored by independent judges on their investment readiness. Then only the top fifty would progress to the finals, with these firms selected on merit.

Pioneers aimed to create broad awareness of the program among entrepreneurial firms in the region, launching the program at the start of August 2015 (see the time line in appendix 2) and marketing the program rapidly. It used five major instruments to achieve this goal: public sources of information for applicants, direct electronic and physical mailings, social media marketing, a road show spanning all five target countries, incentives for early applications (a raffle for a dinner with two leading entrepreneurs from the region), and media relations. Applicants had to apply online, with the data from this application form providing the baseline data for this study. More than 1,200 applications were started online, and 584 full applications were received. These were screened for eligibility, resulting in 346 firms being selected as eligible for the program.

This process succeeded in generating a sample of young firms involved in a wide range of innovative activities.4 Table 1 provides descriptive statistics. At the time of application, firms had a mean (median) of 6 (4) employees, with a 10 to 90 percentile range of (1, 12). They had been in business for 2.5 years on average and are involved in high-tech innovative industries such as cloud computing and big data, app development for a wide range of business and personal services, and pharmaceutical products. Half of the founders have postgraduate education, and 60% have a global rather than regional focus as their key market. While 35% of firms had accepted some outside funding, the majority of this was in the form of public grants and loans from family and friends, with only 9% having already accepted funding from an outside investor like a business angel or venture fund.

Table 1.

Descriptive Statistics and Balance Test on Application Data

Full SampleSemifinal Participants
TreatmentControlp-valueTreatmentControlp-value
Variables stratified on       
Incorporated/registered in Croatia 0.25 0.24 0.612 0.25 0.30 0.920 
Incorporated/registered in Serbia 0.46 0.46 0.626 0.48 0.48 0.513 
Baseline Readiness Score 2.95 2.92 0.150 2.99 2.97 0.476 
Has an outside private investor 0.10 0.09 0.178 0.14 0.06 0.170 
Other variables       
Market attractiveness score 3.08 3.05 0.851 3.13 3.18 0.579 
Product technology score 2.47 2.43 0.835 2.56 2.71 0.085 
Traction score 3.34 3.27 0.507 3.28 3.06 0.382 
Team score 3.04 3.05 0.878 3.08 3.02 0.207 
Sector is business and productivity 0.48 0.39 0.107 0.45 0.36 0.436 
Sector is lifestyle and entertainment 0.18 0.23 0.295 0.20 0.27 0.215 
Uses cloud technology 0.20 0.26 0.231 0.20 0.21 0.984 
Uses big data 0.18 0.21 0.642 0.17 0.20 0.915 
Place in value chain is developer 0.61 0.55 0.171 0.60 0.57 0.677 
Place in value chain is service provider 0.59 0.54 0.372 0.60 0.54 0.108 
Age of firm (years) 2.61 2.66 0.887 2.24 2.29 0.346 
Early-stage firm 0.30 0.33 0.475 0.35 0.37 0.554 
Revenues in 2014 178,073 184,760 0.959 37,642 144,012 0.303 
Number of employees 6.47 5.88 0.539 4.65 5.32 0.800 
Age of main founder 38.22 36.81 0.204 38.02 36.67 0.362 
Main founder has postgraduate education 0.49 0.48 0.816 0.54 0.55 0.740 
At least one founder is female 0.16 0.22 0.128 0.16 0.30 0.071 
Company has a global focus 0.60 0.58 0.576 0.59 0.63 0.569 
Have accepted outside financing 0.34 0.37 0.656 0.42 0.40 0.836 
Previously in mentoring/accelerator program 0.15 0.16 0.704 0.18 0.22 0.202 
Sample size 174 172  110 101  
Joint test of orthogonality of treatment p-value   0.621   0.086 
Full SampleSemifinal Participants
TreatmentControlp-valueTreatmentControlp-value
Variables stratified on       
Incorporated/registered in Croatia 0.25 0.24 0.612 0.25 0.30 0.920 
Incorporated/registered in Serbia 0.46 0.46 0.626 0.48 0.48 0.513 
Baseline Readiness Score 2.95 2.92 0.150 2.99 2.97 0.476 
Has an outside private investor 0.10 0.09 0.178 0.14 0.06 0.170 
Other variables       
Market attractiveness score 3.08 3.05 0.851 3.13 3.18 0.579 
Product technology score 2.47 2.43 0.835 2.56 2.71 0.085 
Traction score 3.34 3.27 0.507 3.28 3.06 0.382 
Team score 3.04 3.05 0.878 3.08 3.02 0.207 
Sector is business and productivity 0.48 0.39 0.107 0.45 0.36 0.436 
Sector is lifestyle and entertainment 0.18 0.23 0.295 0.20 0.27 0.215 
Uses cloud technology 0.20 0.26 0.231 0.20 0.21 0.984 
Uses big data 0.18 0.21 0.642 0.17 0.20 0.915 
Place in value chain is developer 0.61 0.55 0.171 0.60 0.57 0.677 
Place in value chain is service provider 0.59 0.54 0.372 0.60 0.54 0.108 
Age of firm (years) 2.61 2.66 0.887 2.24 2.29 0.346 
Early-stage firm 0.30 0.33 0.475 0.35 0.37 0.554 
Revenues in 2014 178,073 184,760 0.959 37,642 144,012 0.303 
Number of employees 6.47 5.88 0.539 4.65 5.32 0.800 
Age of main founder 38.22 36.81 0.204 38.02 36.67 0.362 
Main founder has postgraduate education 0.49 0.48 0.816 0.54 0.55 0.740 
At least one founder is female 0.16 0.22 0.128 0.16 0.30 0.071 
Company has a global focus 0.60 0.58 0.576 0.59 0.63 0.569 
Have accepted outside financing 0.34 0.37 0.656 0.42 0.40 0.836 
Previously in mentoring/accelerator program 0.15 0.16 0.704 0.18 0.22 0.202 
Sample size 174 172  110 101  
Joint test of orthogonality of treatment p-value   0.621   0.086 

Full sample denotes the full experimental sample. Semifinal participants are the sample scored by judges during the semifinal pitch event. Variables stratified on were the variables used in randomized assignment.

To make clear the types of firms involved, it is worth giving some more specific examples of the types of innovation these firms are doing:

  • A firm that is developing virtual reality software that can be used in outdoor interactive missions, with the aim of deploying this in military training exercises and theme park adventures (e.g., a team-based maze/obstacle course where dragons and other objects are flying around)

  • A firm developing an app that geolocates users on ski fields in Europe and provides a way for them to see where all their family members are at any time, and to direct them to common meeting places

  • A biotech firm that has developed a new coating for common medicines that allows the body to better regulate the dose intensity to reduce under- and overdosages of medicines

  • An architecture firm that has developed an innovative luxury “boatel” that runs on an electric motor and can be used on lakes

  • A firm that has developed solar-powered benches for public spaces that can charge phones and also monitor air and noise quality.

A number of the firms were developing apps for the Balkan and global markets, covering a wide range of activities such as making it easier to use public transport, a local version of Uber, an app to connect consumers with producers of organic products, online sports coaching, and an app to manage freight logistics. But there are also firms involved in physical manufacturing of products, such as high-end electrical bicycles, smart vending machines, indoor pet houses, and a USB charger that charges while bicycling.

B. Random Assignment

Applications closed on September 6, 2015, and were then screened to ensure they met the eligibility requirements. All applicants meeting the formal eligibility criteria were accepted into the study. Eligible applications were then scored on four criteria to measure their initial level of investment readiness: market attractiveness, product technology, traction, and team. Appendix 3 describes the scoring methodology. The top ten proposals overall in terms of score were then randomly assigned to five in treatment and five in control in order to ensure that some of the very top proposals were in both groups. Then the remainder of firms were divided into strata based on country (Serbia, Croatia, or the rest), and on whether they already have a private investor. Within these six strata, firms were ranked into groups of four on the basis of their investment readiness score. Within these quartets, two firms were randomly allocated by computer to treatment and two to control. This was done for an initial batch of 333 firms, allocating 167 to treatment and 166 to control. An additional batch took longer to verify their eligibility requirements and were received after this assignment; these were then also randomly allocated and form a separate stratum. This resulted in 346 firms, with 174 treatment and 172 control. A preanalysis plan was registered with the AEA trial registry on October 2, 2015, to prespecify the initial outcomes of interest.5

This process resulted in treatment and control groups that are evenly balanced and comparable in terms of their initial characteristics (see table 1). As a result, any difference in investment readiness at the conclusion of the program can be reliably assessed as the impact of the program and not due to any preexisting differences across groups.

C. Details of the Treatment and Control Offerings

The treatment and control groups were blinded to treatment status, and both were offered a form of investment readiness training—the difference being in the intensity, cost, and medium of the offerings. We summarize both treatment and control programs here. A key issue with understanding the impact of different training programs is that much of the literature does not provide sufficient detail on what was offered, leaving the program as a black box for others seeking to learn or compare. Therefore, in appendixes 4 and 5, we provide much more detailed information on each program.

The treatment group received an investment readiness program provided by Pioneers. This intensive two-month program aims to prepare companies to be willing to consider equity, make key changes if needed to have systems in place that investors are looking for, and put them in a position where they are ready to talk with potential investors. The first phase (“qualification”) was structured around an online training platform, WhatAVenture. Using this tool, individuals are asked to outline and self-critically assess their businesses by describing the problem or need that their product or service addresses and the commercialization concept and expected revenue streams. They conduct a market sizing exercise and describe their competitive positioning. Each business was assigned a lead mentor who supports them through this process and provides feedback and help.

After completing this first phase, firms were then brought into the “acceleration phase.” In this phase, they had individualized mentoring from both their lead mentor and a pool of more than 100 specialized mentors who could help on specific concrete and sector-specific needs. Appendix 4 provides examples of the types of advice received, which ranged from specifics of dealing with regulations, advice on valuation, pricing strategy, financing options, customer segmentation, technology, and other topics. Mentoring took place both on site and by video calls. During this phase, four master class weekends took place every week in October from Friday evening through Sunday afternoon. These classes rotated around the different countries and were recorded so that those who couldn't attend in person could access the contents online. Each workshop followed a similar format but with varying topics. On Friday evenings, the attending entrepreneurs had a chance to introduce themselves and their businesses in just 90 seconds with no presentation materials and also see examples from the mentors, followed by informal discussions. On Saturdays, five to eight lectures or workshops were held, with themes such as sales and marketing, team building and human resources, and investment and finance. On Sundays, all participants and mentors focused on presentational skills as well as pitch deck structure and design. The final phase was a “pitch preparation phase” and took place in the last two weeks in the run-up to the semifinals. This included working on their pitch decks with their mentors, delivering practice pitches, and then on-site training in Belgrade the day before the semifinals performance as a final practice run.

The total cost of the treatment is estimated to be $614,000, or approximately $4,000 per active participant.6 The main component of the cost is the individual mentoring, which averaged $3,072 per beneficiary, with the master classes costing $793 per beneficiary and pitch training $230.

The control group companies were offered an e-learning course developed and distributed by the Global Commercialization Group (GCG) of the University of Texas at Austin. This course is distributed under the label Innovation Readiness Series and was launched in 2011. It is targeted to a broad audience of entrepreneurs, scientists, engineers, and students, with the goal of helping to transform their innovative and technology-based concepts into a viable commercialization plan and a convincing pitch. The content is delivered online through ten modules of 45 to 60 minutes each, with a multiple-choice quiz at the end of each module. Appendix 5 provides descriptions of the content of each module. The modules cover key issues such as how to articulate the benefits of an innovation to customers and investors, intellectual property protection, market validation, comparing to competition, and how to pitch and present. The cost of the course was a one-time $5,000 setup charge to customize to our program and then $153 per firm.

There were several reasons for offering the control group an online investment readiness program rather than not providing any service at all.7 The first was that from a public policy point of view, a key question was whether an expensive and intensive program was needed or whether identical results could be obtained by cheap and accessible online alternatives. This was considered the more interesting policy counterfactual than offering nothing at all. Second, from an evaluation standpoint, offering both groups an investment readiness program lowers the risk of Hawthorne and John Henry effects, since both groups were told they were being provided with an investment readiness program. Finally, we also believed that offering the control group something would minimize the risk of differential attrition compared to the treatment group.

D. Take-Up

Of the 174 firms randomized into treatment, 157 (90.1%) completed the WhatAVenture online training platform and 79.3% received individual mentoring. Conditional on using individual mentoring, entrepreneurs received a median of eight and a mean of eleven hours of individual mentoring from the lead mentor and pool of specialist mentors.8 These take-up rates are high compared to many other business training programs, which average 65% take-up even for courses of only a few days (McKenzie & Woodruff, 2014). Seventy-six of the 174 (43.7%) attended at least one master class in person, typically the master class held in their country (videos of the master classes were also available online, with typically ten to twenty firms watching each). Approximately 1,150 mentoring hours were provided during the master classes—around 390 hours individual mentoring and 760 hours in the form of lectures and presentations. This represents an average of fifteen hours per attendee. In addition, before the semifinals, 76 firms (43.7%) attended a three-hour final pitch presentation training.

Of the 172 participants assigned to the control group, 120 (70%) accessed the online Innovation Readiness Series platform at least once. However, even conditional on accessing the platform, overall usage was relatively low. Conditional on accessing the online platform, 118 participants viewed the modules' section at least once and 55 viewed it at least 10 times; the mean number of views of the modules section was 21 and the median 9. Each module lasts approximately half an hour, so we can approximate that the mean time spent on the modules was 10 hours and the median 4.5 hours. Only 63 (37% of the control group) participated in one of the seven quizzes at the end of a module. Fifty-one control group entrepreneurs passed at least four quizzes, with 45 attaining the threshold of 70% correct answers in all quizzes, necessary to receive a certificate of completion from the IC2 Institute at the University of Texas at Austin. This low usage is common of many online-only programs and has the advantage for our study of making it unlikely that the control program resulted in large improvements in firm outcomes.

IV. Impacts on Investment Readiness as Scored by Judges

A. The Semifinals and Judging Procedure

The semifinals were held in parallel to, and in cooperation with, the Belgrade Venture Forum, an annual venture capital conference that took place from November 12 to 14, 2015. Participants were invited to present in a pitch event that follows the standard format of such events, with firms giving a five-minute pitch of their business case, followed by five minutes of questions from a jury of judges.

Participation required the founder of the firm or a representative to be physically present in Belgrade. To encourage participation, firms received multiple reminders and calls, were sent an invitation letter with a ticket voucher that allowed them one day of free access to the adjoining Belgrade Venture Forum, and were provided with a transport subsidy that was sufficient to cover the cost of bus travel to the event. The travel time was approximately four hours from Croatia, five hours from Macedonia, and six to seven hours from Kosovo and Montenegro. In total, 211 of the 346 invited firms (61%) attended the semifinals: 110 firms from the treatment group (63.8%) and 101 firms from the control group (58.1). The attendance rate was similar for Serbia (64%) and Croatia (67%) and lower for the other three countries (51%). We discuss robustness to this attrition in the next section.

A group of 66 independent judges was used to do the scoring. Panels of five judges were assigned to judge a session of six firms at a time, with judges then being rotated so that they were on panels with different judges for their next sessions. Each batch of six firms consisted of three treatment and three control firms, selected to have a similar range of initial investment readiness scores and grouped according to industry and country of operation. Judges were assigned to batches based on their availability (some were giving talks at the venture forum), industry, and technology used. Appendix 6 provides details of these judges' characteristics. They were a mix of investors, successful business owners, and experts in mentoring and coaching start-ups. Thirty-seven percent lived in one of the five countries taking part in the competition, and two-thirds were based in other countries. Eighty percent of them regularly mentor start-ups, 64 percent were part of companies that make venture investments, and three-quarters had founded their own companies. They were therefore experienced in what outside investors are looking for in terms of investment readiness.

Judges were blinded to treatment status and were not provided with any information about the company in advance of scoring. None of the judges had been involved as mentors in the program. They were briefed and asked to score each firm on six factors:

  1. 1.

    Team: the skills and capabilities of the entrepreneur and his or her team

  2. 2.

    Technology: the degree of innovativeness and technological advancement

  3. 3.

    Traction: indications of measureable market success

  4. 4.

    Market: the commercial market attractiveness and size of the potential market

  5. 5.

    Recent business progress: the amount of progress firms had made during the last three months (the time since initial application)

  6. 6.

    Presentation performance

An aggregate investment readiness score was then formed using the following weights: team, 28%; technology, 21%; traction, 14%; market; 7%; and progress, 30%. These weights were not revealed to the judges but were based on what seed and early-stage investors would commonly focus on (Kaplan & Strömberg, 2004). They tend to emphasize the quality of the team and their technology (Gompers et al., 2020) and the extent to which the business is continually improving. The presentation score was added to allow judges to independently assess how well the firm presented its ideas, and as a “hygiene” factor that could be used if necessary to avoid placing someone unable to present in front of investors at the final. The correlation between this weighted score and an equally weighted score is 0.995, and we show in appendix 6 that our results are robust to this choice of weighting.

Based on these investment readiness scores, the top 54 firms were invited to the finals event.

B. Estimating the Impact on Investment Readiness as Scored by Judges

To estimate the impact of the program on investment readiness as scored by the judges, we use the following (prespecified) base specification for firm i in stratum s:
Outcomei=α+βTreati+s=1Scs1(iεs)+ɛi,
(1)
where 1(iεs) are strata dummy variables. Note that stratification implicitly controls for baseline investment readiness, country, and whether the firm has an outside private investor at baseline. Robust (Eicker-White) standard errors are used. As a robustness check, we also reestimate equation (1) after controlling for judge fixed effects.

The parameter β is then the intention-to-treat effect (ITT). This measures the impact of being assigned to the treatment group and being offered the expensive and intensive investment readiness program rather than the online course offered to the control group. We could also attempt to measure the local average treatment effect (LATE) of actually receiving treatment. Recall that 90.1% of the treatment group completed the WhatAVenture tool. However, all but one of the treatment group firms that attended the semifinals (99.1%) had completed this tool, so the noncompliers to treatment status are firms for which we do not have investment readiness scores. As such, the ITT and LATE are almost identical for the firms attending the semifinals. We therefore just report the ITT results.

The first column of table 2 presents the impact of treatment in our overall measure of investment readiness, as scored by the judges. This is our main outcome in this table, and so our main approach to multiple hypothesis testing for this set of outcomes is to rely on this aggregate. The control group has a mean investment readiness score of 2.9 (SD 0.9). We find that treatment increases this score by 0.284, which is significant at the 5% level. The magnitude is thus equivalent to 0.31 standard deviations. The second row of estimates shows that this impact continues to hold after controlling for judge fixed effects, with a larger magnitude of 0.41. Figure 1 compares the baseline and competition distributions of investment readiness scores for the treatment and control groups and shows a rightward shift in the distribution, so that these gains appear to be occurring everywhere except at the very top.
Figure 1.
Distributions of Baseline and Post-Intervention Competition Investment Readiness Scores by Treatment Status

Baseline scores are for the subset of firms that attended the semifinals. Competition scores are post-treatment. Kolmogorov-Smirnov test of equality of distributions has a p-value of 0.959 at baseline and 0.017 post-intervention.

Figure 1.
Distributions of Baseline and Post-Intervention Competition Investment Readiness Scores by Treatment Status

Baseline scores are for the subset of firms that attended the semifinals. Competition scores are post-treatment. Kolmogorov-Smirnov test of equality of distributions has a p-value of 0.959 at baseline and 0.017 post-intervention.

Table 2.

Impact of Program on Investment Readiness as Scored by Judges

Components of the Overall Score
Overall ScoreTeam ScoreTechnology ScoreTraction ScoreMarket ScoreProgress ScorePresentation ScoreSelected to go to FinalsSD of Judge Scores
Base specification          
Assigned to treatment 0.284** 0.167 0.372** 0.206 0.268* 0.373*** 0.372** 0.115* 0.006 
 (0.126) (0.150) (0.152) (0.130) (0.137) (0.137) (0.164) (0.068) (0.049) 
Including judge fixed effects          
Assigned to treatment 0.409*** 0.369** 0.476*** 0.295** 0.463*** 0.440*** 0.514*** 0.090 -0.017 
 (0.135) (0.158) (0.174) (0.142) (0.139) (0.143) (0.191) (0.076) (0.051) 
Sample size 211 211 211 211 211 211 211 211 211 
Control mean 2.908 3.042 2.970 2.541 3.406 2.794 3.042 0.122 0.723 
Control SD 0.903 1.068 1.031 0.947 0.940 0.937 1.145 0.328 0.317 
Components of the Overall Score
Overall ScoreTeam ScoreTechnology ScoreTraction ScoreMarket ScoreProgress ScorePresentation ScoreSelected to go to FinalsSD of Judge Scores
Base specification          
Assigned to treatment 0.284** 0.167 0.372** 0.206 0.268* 0.373*** 0.372** 0.115* 0.006 
 (0.126) (0.150) (0.152) (0.130) (0.137) (0.137) (0.164) (0.068) (0.049) 
Including judge fixed effects          
Assigned to treatment 0.409*** 0.369** 0.476*** 0.295** 0.463*** 0.440*** 0.514*** 0.090 -0.017 
 (0.135) (0.158) (0.174) (0.142) (0.139) (0.143) (0.191) (0.076) (0.051) 
Sample size 211 211 211 211 211 211 211 211 211 
Control mean 2.908 3.042 2.970 2.541 3.406 2.794 3.042 0.122 0.723 
Control SD 0.903 1.068 1.031 0.947 0.940 0.937 1.145 0.328 0.317 

Robust standard errors in parentheses. Regressions control for randomization strata. Significant at *10%, **5%, and ***1%. Judge fixed effects controls for which 5 of the 65 judges judged a particular firm.

The next five columns of table 2 examine which components of the overall score have improved with treatment. We find positive impacts on all five components (team, technology, traction, market, and progress), with the impacts statistically significant for three of five measures and significant for all five measures after controlling for judge fixed effects. The seventh row then examines the impact on the team's presentation score. Recall this is not included as part of the overall score but was scored separately. We find that treatment resulted in a 0.37 unit (0.32 SD) increase in the team's presentational score, which is statistically significant at the 5% level. Treated firms are therefore more investment ready in terms of being able to present their idea and in the quality of the idea presented. As a result, treatment doubles the likelihood of a firm being selected for the finals (discussed more in appendix 6), from 12% in the control group. This effect is significant at the 10% level. Our treatment, like most other investment readiness programs, is a bundle of different components, including online training, mentoring, and networking, and we do not have independent verification with which to estimate which component mattered most. However, our descriptive evidence suggests that the main channel for improvement was working one-on-one with the mentor9 Treated firms that used more hours of mentoring were the ones that improved their investment readiness scores the most compared to their score at the time of application, whereas we see no association between attending master classes and the change in score, or between talking with other firms from the program and this change in score.10 A limited role for networking is also suggested by the fact that 51% of the treated firms said they did not talk to a single other firm from the program six months after the program, and the modal firm that did talk to other firms talked to only two others.

C. Robustness of the Impact on Judges' Scores

The investment readiness scores are available only for firms that participated in the semifinals. This raises the concern of bias arising from differential participation patterns among treatment and control firms. The last columns of table 1 examine balance on baseline characteristics by treatment status for the firms that participated in the semifinals. We see that overall, the sample still looks balanced on most observable characteristics, although the overall joint orthogonality test has a p-value of 0.086. Most important, the mean of the baseline overall investment readiness differs only by 0.02 between the two groups, and appendix figure 6.1 compares the full distribution of the baseline investment readiness score by treatment group and participation status, and shows the distributions also look similar. Our preanalysis plan specified two approaches to examining the robustness of our results to this attrition: imputing scores for those who did not attend and using Lee (2009) bounds. Appendix 6 shows the results are robust to both approaches and are also robust to using alternative weighting schemes to aggregate the different components of the overall score. The program therefore succeeded in making firms more investment ready, as judged by independent experts.

D. Did the Treatment Make Bad Ideas Clearer or Hasten the Death of Low-Quality Firms?

Wagner (2017) and Clingingsmith and Shane (2018) note that one effect of training firms to better present their ideas can be to increase the signal contained in pitches, making it easier for judges to distinguish good from bad ideas. If this is the case, the positive average effect on judges' scores that we observe may mask a negative effect for those with lower-quality ideas, offset by an even larger positive effect for those with better-quality ideas.

We investigate this possibility using several approaches. First, in the last column of table 2, we consider as an outcome the standard deviation of the individual judge scores for a firm, with a higher standard deviation indicating more divergence among judges in their assessment of the firm If treatment makes the signals in pitches more precise, we would expect to see less divergence in opinion among judges. We find a very small, and not statistically significant impact of treatment on this measure, which provides a first piece of evidence against the hypothesis that our program made it easier for judges to distinguish good ideas from bad. Second, in appendix table 6.3, we examine treatment heterogeneity with respect to the baseline investment readiness score. If our program causes lower-quality firms to present their ideas more clearly, we would expect the treatment impact to be lower for those with below-median baseline readiness scores. Instead, we find positive and not statistically significant interaction effects with treatment on both the overall score and the presentation quality score and no effect on the standard deviation of judge scores even for those with lower initial quality. Finally, in nonprespecified analysis suggested by a referee, in appendix table 6.4 we estimate quantile treatment effects and cannot reject equality of impacts at the 10th, 25th, median, 75th and 90th percentiles.

In addition to the possibility that training helps make lower-quality ideas clearer to outside judges, the literature has also raised the possibility that the feedback associated with programs may make those with lower-quality ideas select out of running their firms faster (Yu, 2020; Howel, 2018). Appendix table 6.5 shows that it is not the case that firms with low quality (either in terms of baseline scores or as assessed by the judges) fail more quickly when assigned to treatment.

There are several potential reasons why these channels found in some of the literature do not apply here. Our firms are less nascent and have had time to get market feedback on their ideas which contrasts with students pitching a hypothetical product in Clingingsmith and Shane (2018). In addition, the mentors and program did more than give negative feedback to those with lower-quality ideas: they provided tangible help to improve. As in Wagner (2017), this improvement effect appears to be larger than any precision effect.

V. Impacts on Firm Outcomes

The immediate impacts on investment readiness are seen in the performance in the semifinals. We then track the firms over time to see whether this short-term improvement in investment readiness translates into longer-term investment readiness and a higher likelihood of receiving external investments.

A. Measuring Firm Outcomes

Our main outcomes come from two rounds of follow-up surveys in which we attempted to interview all firms, not only those that had participated in the pitch competition. The first round, intended to measure short-term effects, was taken between April and August 2016, corresponding to a period of approximately six months after the end of the investment readiness program and judging. The overall survey response rate was 79.2% and does not differ significantly between treatment (79.9%) and control (78.5%). In addition, we collected information on operating status, number of employees, and whether negotiations for an outside investment had occurred for a further 12% of firms,11 resulting in basic data being available for 92.2% of firms.3

The second follow-up survey took place between August 2017 and March 2018, corresponding to an average of two years since the intervention. Catalini, Guzman, and Stern (2017) show that 75% of firms that receive venture capital financing in the United States receive their first financing within the first two years after incorporation. Although we might expect firms to be slower to raise funding in a less-developed capital market, our firms had been in business an average of 2.5 years at the start of the competition, and so a further two years covers a window where we should expect many firms to receive external financing if they will ever do so. The overall survey response rate for this second follow-up was 85.0% and again does not differ significantly between treatment (86.2%) and control (83.7%), with data on firm operating status and receipt of equity available for 94.5% of firms. Appendix 8 shows no significant difference in response rates by treatment status and that treatment and control firms remain balanced on baseline observable data for those responding to the survey.

The follow-up surveys focused on measuring changes in the firm in three domains. The first is whether the firm is still operating (regardless of whether it has been sold to another owner). The second is investment readiness, where we focus on three aspects identified by Mason and Kwok (2009): (a) willingness and interest in taking on equity investment; (b) general investability, as measured whether there is a viable business of interest to investors in terms of employment, sales, and profits12; and (c) whether the firm has put in place specific measures investors want to see before making investments, such as separation of outcomes, revenue projections, knowledge of customer acquisition costs, tracking key metrics of traction, and covering intellectual property. The third and final domain looks at steps toward receiving external funding and then external financing received. Steps toward financing include contacting outside investors, making pitches, working with mentors or experts to help obtain financing, and entering into negotiations. Receipt of external financing considers new debt and equity investments, as well as receipt of incubator and accelerator grants. Our ultimate outcome is then a component of this receipt of external financing index, measuring whether the firm has made a deal with an outside investor.

We asked several questions under each domain and subdomain. Our preanalysis plan then specifies aggregating these measures to form standardized indices. This reduces concerns about multiple hypothesis testing by focusing on one aggregate outcome in each family of questions. Appendix 3 provides the questions used in forming each question, and appendix 9 provides treatment impacts on each specific question used in these aggregate measures.

We supplement our survey measures of firm outcomes with an index measure of media buzz, which captures measures of whether firms are measured in any of more than 250,000 global news sources in 190 countries, and the number of twitter followers and Facebook likes they have attracted (see appendix 3 for further details). This captures whether the firm is gaining attention and traction with customers, and has the advantage of being available for the full sample, with no attrition.

B. Treatment Impacts on Firm Outcomes

Table 3 presents the treatment effects of the investment readiness program on firm outcomes after estimating equation (1). Panel A shows the short-run impacts six months after the intervention and panel B the impacts two years after intervention. Column 1 shows that treated firms attracted more media buzz, with the 0.11 SD increase after two years significant at the 5% level. Appendix table 9.0 shows this largely comes from more mentions of the firm in global media. Column 2 examines firm survival. Ten percent of control firms had died by the first follow-up and 25% by the second follow-up, two years after intervention. These high death rates are higher than the average rates in developing countries and likely reflect the firms' being young and in relatively developed countries (McKenzie & Paffhausen, 2019). Treatment increases survival by 7.2 percentage points after two years; however, this is not statistically significant at conventional levels (p=0.112).

Table 3.

Impacts on Firm Outcomes Six Months and Two Years after Program

Media BuzzFirm SurvivalInterested in EquityGeneral InvestabilitySpecific Needs of InvestorsInvestment StepsExternal InvestmentMade a Deal with Investor
A. Impact at six months         
Assigned to treatment 0.085 0.049 0.051 0.026 0.082 -0.017 -0.152* -0.024 
 (0.053) (0.030) (0.094) (0.085) (0.080) (0.098) (0.087) (0.033) 
Sample size 346 319 278 277 269 240 279 279 
Control mean -0.060 0.898 -0.015 -0.039 -0.059 0.008 0.084 0.083 
Control SD 0.546 0.303 0.764 0.634 0.682 0.720 0.741 0.276 
B. Impact at two years         
Assigned to treatment 0.112** 0.072 0.032 0.089 0.084 0.044 0.003 0.050 
 (0.047) (0.045) (0.084) (0.082) (0.079) (0.092) (0.080) (0.049) 
Sample size 346 340 309 291 298 282 330 330 
Control mean -0.073 0.753 -0.005 -0.058 -0.059 -0.032 0.018 0.244 
Control SD 0.528 0.433 0.783 0.650 0.692 0.760 0.698 0.431 
Media BuzzFirm SurvivalInterested in EquityGeneral InvestabilitySpecific Needs of InvestorsInvestment StepsExternal InvestmentMade a Deal with Investor
A. Impact at six months         
Assigned to treatment 0.085 0.049 0.051 0.026 0.082 -0.017 -0.152* -0.024 
 (0.053) (0.030) (0.094) (0.085) (0.080) (0.098) (0.087) (0.033) 
Sample size 346 319 278 277 269 240 279 279 
Control mean -0.060 0.898 -0.015 -0.039 -0.059 0.008 0.084 0.083 
Control SD 0.546 0.303 0.764 0.634 0.682 0.720 0.741 0.276 
B. Impact at two years         
Assigned to treatment 0.112** 0.072 0.032 0.089 0.084 0.044 0.003 0.050 
 (0.047) (0.045) (0.084) (0.082) (0.079) (0.092) (0.080) (0.049) 
Sample size 346 340 309 291 298 282 330 330 
Control mean -0.073 0.753 -0.005 -0.058 -0.059 -0.032 0.018 0.244 
Control SD 0.528 0.433 0.783 0.650 0.692 0.760 0.698 0.431 

Robust standard errors in parentheses. Significant at *10%, **5%, and ***1%. All regressions control for randomization strata fixed effects. Media buzz is a standardized index of whether the firm is mentioned in the media, the number of media mentions, number of Facebook likes, and number of Twitter followers. Firm survival is a binary variable that takes the value 1 if the firm is operating and 0 otherwise. Interested in equity is a standardized index of whether the firm is interested in equity financing, the maximum equity share it is willing to have owned by outside investors, whether it has specific deal terms for investors, and whether it would consider a royalty-based investment. General investability is a standardized index of number of employees, whether the founders work full time in the business, whether the firm had positive sales in the first quarter of the year, whether total sales exceeded 10,000 euros in that quarter, whether the firm made a profit in the past year, and whether the firm made sales to western Europe or the United States. Specific needs of investors is a standardized index of whether business and personal accounts are separated, whether the firm made a revenue projection for the next year, whether it knows customer acquisition costs, the number of key metrics tracked, whether it has found out if the product or service can be covered by intellectual property protection, and whether it has at least one form of intellectual property protection received or pending. Investment steps is a standardized index of having contacted an outside investor, made a pitch to an outside investor, have a mentor or external expert supporting them to obtain financing, and entered into negotiations with an outside investor. External investment is a standardized index of having taken on new debt, having made a deal with an outside investor, having received at least 25,000 euros in outside financing, and having received an incubator or accelerator grant (all since August 2015). Made a deal with an investor indicates having made a deal with an outside investor since August 2015 (the program start).

Columns 3 through 7 then examine our index measures of investment readiness and investment outcomes. In the short term (six months after intervention), there is a reduction in external investment, significant at the 10% level, which comes through less debt financing, and no other significant impacts. After two years, the treatment effects on all survey outcomes for the firms are positive but not statistically significant, and they are below 0.1 SD in magnitude for all of our index outcomes. Finally, in the last column, we examine whether the firm had made at least one deal with an outside investor since the start of the program (August 2015). In the control group, 24.4 percent made such a deal after two years. The treatment group is 5 percentage points more likely to have made a deal with an outside investor over the two years, but this is not statistically significant, with a 95% confidence interval of (-4.7p.p., +14.7p.p.).

Appendix 9 shows impacts on the individual measures that make up these aggregate indices. The intervention has a large and significant (p=0.013) impact on employment after two years of 4.5 workers, which almost doubles the employment level in the control mean. Employment is often a key policy outcome by itself, and so this program would compare favorably to a number of other programs when judged on employment alone. However, if we correct for testing 25 different outcomes that make up the aggregate indices, this impact is no longer statistically significant (p=0.425).

VI. Why Does the Increase in Investment Readiness Result in Largely Null Effects on Firm Outcomes?

Our results show that firms receiving the investment readiness program that were rated as more investment ready by judges increased their market traction in terms of capturing media attention, and yet we do not find significant effects on our longer-term survey measures of investment readiness or ultimately making a deal with an investor. We consider three potential explanations for these null results: that the change in investment readiness scores does not capture actual changes in investment readiness; that the study is low powered; and that the small average effect masks significant effects for a subgroup of firms that would otherwise find funding harder to get.

A. Do the Judges' Investment Readiness Scores Actually Capture an Increase in Investment Readiness?

A first possible explanation for a lack of significant treatment impact on firm investment outcomes could be that increases in the scores do not actually reflect improved investment readiness. This could arise from either treatment status influencing how a particular pitch is scored, independent of its actual quality, or the scores not capturing aspects of the firm that actually matter for investment outcomes.

We could see a treatment impact on investment readiness scores without any true change in investment readiness if the control firms get discouraged from not receiving the treatment and so perform badly in their pitch, or if the judges know which firms are treated and consciously or subconsciously score treated firms higher. We do not think either effect is likely in our case. The control firms were also told they were selected for the investment readiness program and were offered the online class for preparation. Given the dispersion of firms across cities, countries and sectors, their social networks were not closely intertwined, and we did not see treated firms posting specifics of the content of their interventions on their social media feeds. Firms did not get to watch the pitches of other firms in the competition, and we received only one case of a firm in the control group asking why it had received a different set of services from others in the program. Moreover, the control firms still had the incentive to try their best in the pitches given that it would determine whether they were selected for the finals.

The judges had neither the knowledge of which firms were treated nor any incentive to score treated firms differently had they known. They were told that all firms participating were part of an investment readiness program and were not involved in other parts of the program. Given the length of the pitches, firms focused on describing their firm and its product rather than their participation in the program, and the questions from judges were focused on typical issues like pathways to scale, competitive and regulatory issues, metrics of traction, and how the firm was valuing itself.

Second, it could be that the scores do not actually capture aspects of the firm that matter for being investment ready. To investigate this possibility, we test whether judges' scores are informative about future outcomes for the firm using the control group sample to run the regression:
Outcomei=μ+θInvestmentReadinessi+γi'Xi+ɛi.
(2)
We carry out this estimation first with no additional controls and then with controls X for a large set of baseline information about the firm and owner: country (dummies for Serbia and for Croatia), whether the firm had received funding from an outside investor at baseline, the business sector (dummies for business and productivity, and lifestyle and entertainment sectors), firm age, whether the firm classifies itself as early stage, the number of employees in the firm, and the age and gender of the founder. We estimate this separately by survey to examine results at different time horizons.

Table 4 presents the results. We see that the judges' scores of investment readiness are positively associated with all of our firm outcomes, both with and without the inclusion of these baseline controls. The relationship is strongest for media buzz, taking steps toward investment, the external investment index, and making a deal with an investor. Even after controlling for a range of baseline characteristics, these associations are significant at the 1% level over two-year horizons. The scores also significantly predict being interested in equity and meeting the specific needs of investors over the six-month horizon, although this relationship weakens over the two-year horizon. The magnitudes for these significant associations suggest a one unit change in the judge scores (which had a mean of 2.9 and standard deviation of 0.9) would result in a 0.19 to 0.33 unit increase for our index measures, and a 16.3 percentage point increase in the likelihood of making a deal with an investor.

B. Is the Null Effect Really a Modest Effect with Insufficient Statistical Power?

A second explanation for the lack of statistical significance is that we lack statistical power to detect the effect of the program on firm outcomes. We have that treatment has a causal impact (β^=0.28) on the investment readiness score received from judges, and that this investment readiness score in turn is a significant predictor (with coefficient θ^) of firm outcomes in the control group sample. Combining these two estimates allows us to obtain an estimate of the predicted treatment effect β^θ^. This predicted effect is shown for each outcome in table 4. It assumes that the only impact of the investment readiness program on firm outcomes is captured through the investment readiness score, that the association between score and outcomes observed in the control group is causal, and that the sequential ignorability assumption of Imai et al. (2011) holds.13 Although these assumptions can be questioned, we believe such an exercise is useful in providing a sense of the magnitudes we might expect to see for treatment effects, given how much our program affected investment readiness scores and how much a change in scores in turn predicts future outcomes.14 We see that the predicted treatment effects are small in absolute terms: each of our index measures is predicted to increase by only 0.04 to 0.09 over two years, and the predicted increase in the likelihood of receiving outside funding is 4.6 percentage points. Our estimated treatment effects in table 3 are similar in magnitude to these predicted treatment effects.5

Table 4.

Judges Scores Predict Firm Outcomes Six Months and Two Years after Program

Media BuzzFirm SurvivalInterested in EquityGeneral InvestabilitySpecific Needs of InvestorsInvestment StepsExternal InvestmentMade a Deal with Investor
A: Association at six months 
Without controls         
Score assessed by Judges 0.261*** 0.024 0.201** 0.076 0.336*** 0.222*** 0.213** 0.093** 
 (0.057) (0.037) (0.076) (0.072) (0.065) (0.082) (0.098) (0.038) 
With controls for country, prior funding, sector, firm age and stage, founder gender and education, baseline employment 
Score assessed by Judges 0.220*** 0.017 0.209** 0.080 0.296*** 0.155 0.190* 0.080** 
 (0.052) (0.042) (0.099) (0.080) (0.078) (0.103) (0.113) (0.037) 
Sample size 101 92 83 83 81 73 82 82 
Control mean -0.060 0.898 -0.015 -0.039 -0.059 0.008 0.084 0.083 
Control S.D. 0.546 0.303 0.764 0.634 0.682 0.720 0.741 0.276 
Predicted treatment effect 0.073 0.007 0.056 0.021 0.094 0.062 0.060 0.026 
Panel B: Association at two years 
Without controls         
Score assessed by Judges 0.271*** 0.061 0.153* 0.040 0.136* 0.322*** 0.322*** 0.166*** 
 (0.059) (0.041) (0.088) (0.073) (0.082) (0.100) (0.072) (0.048) 
With controls for country, prior funding, sector, firm age and stage, founder gender and education, baseline employment 
Score assessed by Judges 0.232*** 0.079 0.153 0.061 0.128 0.331*** 0.333*** 0.163*** 
 (0.053) (0.049) (0.097) (0.082) (0.086) (0.103) (0.081) (0.051) 
Sample size 101 100 92 86 88 80 99 99 
Control mean -0.073 0.753 -0.005 -0.058 -0.059 -0.032 0.018 0.244 
Control S.D. 0.528 0.433 0.783 0.650 0.692 0.760 0.698 0.431 
Predicted treatment effect 0.076 0.017 0.044 0.011 0.038 0.090 0.090 0.046 
Media BuzzFirm SurvivalInterested in EquityGeneral InvestabilitySpecific Needs of InvestorsInvestment StepsExternal InvestmentMade a Deal with Investor
A: Association at six months 
Without controls         
Score assessed by Judges 0.261*** 0.024 0.201** 0.076 0.336*** 0.222*** 0.213** 0.093** 
 (0.057) (0.037) (0.076) (0.072) (0.065) (0.082) (0.098) (0.038) 
With controls for country, prior funding, sector, firm age and stage, founder gender and education, baseline employment 
Score assessed by Judges 0.220*** 0.017 0.209** 0.080 0.296*** 0.155 0.190* 0.080** 
 (0.052) (0.042) (0.099) (0.080) (0.078) (0.103) (0.113) (0.037) 
Sample size 101 92 83 83 81 73 82 82 
Control mean -0.060 0.898 -0.015 -0.039 -0.059 0.008 0.084 0.083 
Control S.D. 0.546 0.303 0.764 0.634 0.682 0.720 0.741 0.276 
Predicted treatment effect 0.073 0.007 0.056 0.021 0.094 0.062 0.060 0.026 
Panel B: Association at two years 
Without controls         
Score assessed by Judges 0.271*** 0.061 0.153* 0.040 0.136* 0.322*** 0.322*** 0.166*** 
 (0.059) (0.041) (0.088) (0.073) (0.082) (0.100) (0.072) (0.048) 
With controls for country, prior funding, sector, firm age and stage, founder gender and education, baseline employment 
Score assessed by Judges 0.232*** 0.079 0.153 0.061 0.128 0.331*** 0.333*** 0.163*** 
 (0.053) (0.049) (0.097) (0.082) (0.086) (0.103) (0.081) (0.051) 
Sample size 101 100 92 86 88 80 99 99 
Control mean -0.073 0.753 -0.005 -0.058 -0.059 -0.032 0.018 0.244 
Control S.D. 0.528 0.433 0.783 0.650 0.692 0.760 0.698 0.431 
Predicted treatment effect 0.076 0.017 0.044 0.011 0.038 0.090 0.090 0.046 

Robust standard errors in parentheses. Significant at *10%, **5%, and ***1%. Predicted treatment effect is the treatment effect predicted from association in the control group between the judges' score and the outcome, multiplied by the treatment effect of the program on the judges score. Outcomes are as defined in table 3.

This program is the first randomized experiment of its kind, but like a number of other experiments involving larger firms, the sample size is set by external constraints in terms of the number of firms that the program attracts and caters to rather than being a choice parameter. Given the sample size, our funding proposal calculated that we would have 80% power to detect a 0.23 increase in the investment readiness score, based on the mean and standard deviation of the baseline score measure and not accounting for the power gains from stratification. Our estimated treatment effect of 0.28 exceeds this level. In contrast, our funding proposal assumed that it would be very rare for control group firms to receive outside funding, assuming a mean of 3%, and then estimated a minimum detectable effect size of 8 percentage points at 80% power, not accounting for the power gains from stratified randomization (since we did not know how strongly our strata would be correlated with the end outcome).15

In practice, our estimated impact on receiving outside funding is 5 percentage points (similar in magnitude to the predicted impact β^θ^=0.046), which is less than this minimal detectable effect. But the larger reduction in power comes from the control mean being much higher than anticipated. While we expected very few control firms to receive external financing, in practice 24.4% of control firms had made a deal within two years. It is much harder to detect an 8 percentage point increase from a control mean of 24.4% than from a control mean of 3%: under our baseline assumptions, power would drop to 33.3% at this mean level, and the minimal detectable effect size is now a 13.7 percentage point increase. So a key reason for not being able to detect a treatment effect on external investment is that control firms found it easier to get investment than we had anticipated. Our surveys provide additional information on the types of external financing control firms were able to get. Seventy-four percent of those receiving funding made an equity-sharing deal and 18% a deal for royalties. The main investors were other firm owners (56%), venture capital funds (35%), angel investors (32%), government funds (23%), accelerators (12%) and selling the firm outright (12%).16

Thus while we increased investment readiness scores, we did not increase them by enough to register large enough changes in investment outcomes to be detectable with our sample size. Our confidence intervals enable us to rule out the program having large absolute impacts on these outcomes, but are wide enough to allow for the program to have moderately sized impacts that are commensurate with what we would expect given the change in investment readiness and how investment readiness correlates with firm outcomes.

C. Does a Modest Average Effect Mask Larger Impacts for Some Firms?

Although the average control firm was more likely to have received investment funding than we had initially anticipated, there is considerable heterogeneity in the sample, and some firms found it harder than others to get funding. The firms in our study were heterogeneous in size at the time of applying: 15.6% had only a single worker, another 32.1% only one or two workers, 31.8% four to six workers, and 20.5% six or more workers. In the absence of our program, smaller firms were likely to be less investment ready and found it harder to get external funding: the judges' investment readiness score for control firms averaged 2.7 for firms with one to three workers (below the median size), versus 3.2 for those with four or more workers (p=0.04), and 14% of control firms with one to three workers had received external financing at our twoyear follow-up, versus 35% of those with four or more workers (p=0.002). This raises the possibility that the treatment worked better for smaller firms, which had more scope to improve.

We did not prespecify examining treatment heterogeneity by firm size, but it was suggested by a referee and makes sense in light of the variation in initial firm size and the association between firm size and our key outcomes in the control group. We split the sample by whether baseline size is below the median of four workers, with 47.7% of firms having one to three workers, and then add a dummy variable for median size or more, and its interaction with treatment, to equation (1).

Table 5 reports the results. The first point to note is that the dummy variable for median size or higher is positive and significant for six of our key outcomes and is most significant for external investment and making a deal with an investor. That is, smaller firms are scored by judges as less investment ready, have less media buzz, survive less, score lower in general investability, and are less likely to receive external investments than larger firms in our sample. Second, looking at the treatment dummy, we see that the investment readiness program had positive and significant effects for below-median-sized firms on their investment readiness score (+0.47 points), their media buzz (+0.16 units), their external investment index (+0.19 units), and most critical, their likelihood of making a deal with an investor (15.6 percentage points). In contrast, the interactions with being of median size or above are negative for all of these outcomes, and significantly so in the case of the external investment index and making a deal with an outside investor. The magnitudes of the interaction suggest that the treatment had no impact on improving the investment readiness score for these larger firms.

Table 5.

Heterogeneity in Impacts by Initial Firm Size

Impacts over Two Years
Investment Readiness ScoreMedia BuzzFirm SurvivalInterested in EquityGeneral InvestabilitySpecific Needs of InvestorsInvestment StepsExternal investmentMade a Deal with Investor
Assigned to treatment 0.474*** 0.157** 0.112 0.071 0.034 0.081 0.217 0.194* 0.156** 
 (0.172) (0.080) (0.071) (0.127) (0.112) (0.122) (0.142) (0.108) (0.071) 
Assigned to treatment* -0.450 -0.056 -0.087 -0.073 0.110 0.005 −0.338* −0.389** −0.212* 
Median size or higher (0.291) (0.133) (0.108) (0.193) (0.188) (0.187) (0.203) (0.192) (0.113) 
Median size or higher 0.422* 0.184* 0.145* 0.001 0.250* 0.159 0.135 0.372*** 0.165** 
 (0.216) (0.098) (0.081) (0.153) (0.137) (0.144) (0.163) (0.125) (0.081) 
Sample size 211 346 340 309 291 298 282 330 330 
Control mean small firms 2.683 -0.207 0.701 -0.052 -0.217 -0.140 -0.109 -0.174 0.143 
Control mean larger firms 3.198 0.065 0.807 0.044 0.106 0.021 0.045 0.219 0.350 
Impacts over Two Years
Investment Readiness ScoreMedia BuzzFirm SurvivalInterested in EquityGeneral InvestabilitySpecific Needs of InvestorsInvestment StepsExternal investmentMade a Deal with Investor
Assigned to treatment 0.474*** 0.157** 0.112 0.071 0.034 0.081 0.217 0.194* 0.156** 
 (0.172) (0.080) (0.071) (0.127) (0.112) (0.122) (0.142) (0.108) (0.071) 
Assigned to treatment* -0.450 -0.056 -0.087 -0.073 0.110 0.005 −0.338* −0.389** −0.212* 
Median size or higher (0.291) (0.133) (0.108) (0.193) (0.188) (0.187) (0.203) (0.192) (0.113) 
Median size or higher 0.422* 0.184* 0.145* 0.001 0.250* 0.159 0.135 0.372*** 0.165** 
 (0.216) (0.098) (0.081) (0.153) (0.137) (0.144) (0.163) (0.125) (0.081) 
Sample size 211 346 340 309 291 298 282 330 330 
Control mean small firms 2.683 -0.207 0.701 -0.052 -0.217 -0.140 -0.109 -0.174 0.143 
Control mean larger firms 3.198 0.065 0.807 0.044 0.106 0.021 0.045 0.219 0.350 

Median size or higher is a dummy variable taking the value 1 if the firm has at least the median number of baseline workers and 0 otherwise. Investment readiness score is score as assessed by judges. Robust standard errors in parentheses, significant at *10%, **5%, and ***1%. Outcomes are taken from two-year follow-up survey, and are two years post-intervention.

We further explore this flexibly in figure 2, which shows coefficients from rolling regressions in the number of employees, which take approximately 30% of the sample at a time. For each subsample we regress the outcome on treatment and controls for the stratifying variables of baseline investment readiness score, country, and having a private investor at baseline. The results confirm the pattern above, whereby investment readiness scores improved only for the smaller firms, and they then subsequently were more likely to receive investment from outside sources.
Figure 2.
The Impact of the Program is Higher for Firms That Were Small at Baseline

Graphs show point estimates from rolling regressions that estimate the impact of being assigned to treatment for rolling samples of approximately 30% of the sample at the time, conditioning on the stratifying variables of initial investment readiness score, country, and whether the firm had a private investor to begin with. Ninety percent confidence intervals shown around point estimates.

Figure 2.
The Impact of the Program is Higher for Firms That Were Small at Baseline

Graphs show point estimates from rolling regressions that estimate the impact of being assigned to treatment for rolling samples of approximately 30% of the sample at the time, conditioning on the stratifying variables of initial investment readiness score, country, and whether the firm had a private investor to begin with. Ninety percent confidence intervals shown around point estimates.

This heterogeneity analysis suggests that the intervention did work for smaller firms and that the null average effects comes from averaging this positive effect with much smaller, or even negative, impacts for larger firms. We conjecture that this impact comes from the program being particularly beneficial for the types of firms that would otherwise struggle to attract investor attention.17 To investigate this idea further, we employ the endogenous stratification method of Abadie et al. (2018) to see whether the program worked better for firms with lower predicted likelihoods of making a deal with an investor over the two years in the absence of treatment.18 This uses the control group to predict the likelihood of receiving an investment as a function of baseline characteristics and uses either a leave-one-out or repeated split samples approach to avoid a small sample bias that can arise from observations contributing to their own estimated fitted values. Given the size of our sample, we then split by above or below the median predicted probability of funding in the absence of treatment.

Table 6 reports the results. We see that the investment readiness program is estimated to have a positive and significant impact on receiving external investment for firms that otherwise would be in the bottom half of firms in our sample in terms of likelihood of receiving an investment. The magnitude is between 12.4 percentage points (repeated split samples approach) and 14.3 percentage points. In contrast, for firms in the top half of the likelihood of receiving an investment to begin with, the program had a negative and statistically insignificant effect. Appendix table 10 uses the leave-one-out classification to compare baseline summary statistics of firms by this predicted likelihood of funding absent the intervention. We see that the firms that are helped more by the program tend to be smaller in size, are less likely to have received mentoring or acceleration before, have a less global focus, and are more likely to be run by less-educated founders and teams with at least one female founder than those with higher predicted likelihoods of funding that the program does not help.

Table 6.

Heterogeneity in Treatment by Predicted Likelihood of Making a Deal with an Investor

Impact on Making a Deal within Two Years
Leave-One-Out EstimatorRepeated Split-Sample Estimator
Low predicted likelihood of funding 0.143** 0.124** 
 (0.066) (0.055) 
High predicted likelihood of funding -0.081 -0.049 
 (0.079) (0.074) 
Impact on Making a Deal within Two Years
Leave-One-Out EstimatorRepeated Split-Sample Estimator
Low predicted likelihood of funding 0.143** 0.124** 
 (0.066) (0.055) 
High predicted likelihood of funding -0.081 -0.049 
 (0.079) (0.074) 

Bootstrap standard errors, based on 500 bootstrap replications, are reported in parentheses. Significant at *10%, **5%, and ***1% levels. The repeated split-sample estimator uses 200 splits of the data. Predicted likelihood of funding based on the following baseline characteristics: Employment above the median, initial investment readiness score, country, whether the firm has had a private investor, whether it classifies itself as early stage, sector, firm age, whether the mainfounder has postgraduate education, whether at least one founder is female, and whether the firm has previously received mentoring. Abadie et al. (2018) endogenous stratification approach used.

VII. Conclusion

Investment readiness programs have been offered in a range of developing and emerging markets, based on the idea of a gap between the quality of ideas entrepreneurs have and their readiness to attract and receive outside investment in those ideas. Despite their growing use, there has not been any rigorous study of their effectiveness. Our five-country randomized trial enables measurement of the effect of such a program.

We find that investment readiness increases, measured by scores in a pitch competition, and that these scores are in turn predictive of future investment readiness and outcomes among firms. Nevertheless, despite finding positive point estimates, our estimates of the treatment effects of the investment readiness program on these firm investment outcomes over the next two years are not statistically significant. Our analysis suggests that this modest average effect in part comes from more of the firms being able to obtain financing without the program than was originally anticipated. In the heterogeneity of impacts, the program appears to have succeeded only in increasing investment readiness and the chance of subsequent external financing for smaller firms (those with one to three workers) and those that otherwise were less likely to receive external financing. We believe these results offer lessons for governments deciding whether and how to use such policies. They show that this type of program can be effective at helping smaller and less experienced firms close the financing gap and suggests the need to carefully target these programs. A further area for policy experimentation is to test which components of the overall investment readiness program matter most, something our sample size prevented us from testing.

Notes

1

See McKenzie and Woodruff (2014) for a recent review. Several recent experiments test short training sessions for early-stage firms, but are not able to trace impacts through to the likelihood of receiving financing. Clingingsmith and Shane (2018) provide thirty-minute pitch training to undergraduate students in Ohio, who then deliver ninety-second pitches to judges, and finds training lower scores on average. More promising is a three-day intervention by Chatterji et al. (2018) among 100 growth-stage firms in India, which matches firms with peers in order to receive advice about managing people. They find this has a positive impact on firm growth and survival for those matched to firms with active management styles.

2

This marketing will likely screen out firms completely opposed to equity, but still attract firms that were uncertain about equity, with only 60% of our control group saying they were interested in equity financing in our first follow-up survey.

3

A second, less binding, concern was to avoid having firms from the bottom tail of quality present to investors in case this had negative spillover reputation effects. This meant avoiding having the worst performers pitch to investors, but the limit on investor attention was the binding constraint that meant only fifty could be selected.

4

Although we do not have a random sample of start-ups in the region with which to examine selection into the program, in appendix 7 we benchmark firm characteristics against those of beneficiaries of the main program for start-ups of the Serbian Innovation Fund. We find considerable overlap in firm type, but our sample also includes more established and larger firms than in their program.

6

The exact cost per firm differs in terms of services contracted versus services actually delivered, since not all firms used all the mentoring hours they were allocated. Pioneers retrospectively estimates that the actual services delivered to the firms were approximately $3,000 based on actual hours of mentoring used. Note further that this calculation does not include the costs of advertising the program through roadshows or of putting on the semifinal and final events, which were important in attracting firms to the program. These overhead costs are estimated at approximately $1,500 per firm (in both the treatment and control groups).

7

The sample size means that we did not have sufficient power to have more than two groups, so this prevented adding a pure control group or having treatments that would attempt to work on only one aspect of investment readiness.

8

Firms were eligible to receive up to thirty hours of individual mentoring time, though the majority of teams used considerably fewer hours than allocated to them. Pioneers attributes this in part to the other work and family commitments facing founders of small firms, as well as a desire by mentors to not over-mentor, scheduling follow-up sessions only when there was interest and clear areas to work on.

9

Pioneers notes to us that they believe the mentoring is more effective when firms have first thought through their idea and documented insights and issues to work on, which is why mentoring was preceded by the online WhatAVenture platform.

10

We regress the change in score between baseline and the semi finals for the treated firms on whether they attended masterclasses, the number of mentoring hours they received, and whether they network with other firms from the program. Only the number of mentoring hours has a significant association (p=0.025), with the point estimate suggesting a 1 SD increase in mentoring hours is associated with firms improving 0.24 units in the investment readiness score, which is approximately the mean improvement for the treated group. However, this is only correlational, and it might be that the most ambitious and determined firms were the ones that used their mentors more and also made changes in their businesses to make their firms more investment ready.

11

Firms that refused to take part in the survey were asked if they would answer three questions for us, which is what enabled us to get this additional information.

12

Since the main goal of this intervention was to make firms readier to receive investment, this was the focus of our survey questions. Moreover, at this early stage, many firms were not yet profitable and considered their revenue commercially sensitive, leading us to focus on whether they had positive revenues and profits and whether revenues exceeded 25,000 euros.

13

The sequential ignorability assumption requires that if there are heterogeneous treatment effects, it is not the case that the firms for which treatment increases investment readiness scores are different from the firms for which an increase in investment readiness scores would increase future outcomes.

14

For example, McKenzie and Woodruff (2017) show that this approach yields predicted magnitudes of business training interventions on firm outcomes that are similar to those obtained by experimental studies.

15

We did not have baseline information on our index measures and did not develop ex ante power calculations for them. The ex-post minimal detectable effect (MDE) sizes based on the standard errors in table 3 are 0.22 to 0.25, which are considerably larger than our point estimates.

16

Note that percentages do not add to 100 since firms can receive investments from multiple sources.

17

Note that the differential treatment impact does not come from differences in take-up rates by firm size: smaller firms were as equally likely to complete the WhatAVenture tool, attend master classes, and use a mentor as larger firms.

18

This was also not prespecified, since this is a new method that we were not aware of at the time of designing this experiment. Its use follows naturally as an exploration for the reason for heterogeneity by firm size.

REFERENCES

Abadie
,
Alberto
,
Matthew
Chingos
, and
Martin
West
, “
Endogeneous Stratification in Randomized Experiments
,” this review 100 (
2018
),
567
580
.
Berger
,
Roland
,
Revisiting the Market for Innovation: How Accelerators and Incubators Can Reinvent Themselves
(
2019
), https://www.rolandberger.com/publications/publication_pdf/roland_berger_revisiting_the_market_for_innovation.pdf.
Catalini
,
Cristian
,
Jorge
Guzman
, and
Scott
Stern
, “
Hidden in Plain Sight: Venture Growth with or without Venture Capital
,” MIT mimeograph (
2017
).
Chatterji
,
Aaron
,
Solène
Delecourt
,
Sharique
Hasan
, and
Rembrand
Koning
, “
When Does Advice Impact Startup Performance?
” NBER working paper 24789 (
2018
).
Clingingsmith,
David
, and
Scott
Shane
, “
Training Aspiring Entrepreneurs to Pitch Experienced Investors: Evidence from a Field Experiment,
Management Science
64
(
2018
),
530
552
.
Gattini
,
Luca
,
Dragan
Soljan
,
Peter
Hauser
,
Tatyana
Dolgaya
,
Sorin
Revenko
, and
Michael
Kortenbusch
,
Assessment of Financing Needs of SMEs in the Western Balkans Countries: Synthesis Report
(
Frankfurt
:
European Investment Bank
,
2016
).
Gompers
,
Paul
,
Will
Gornall
,
Steven
Kaplan
, and
Ilya
Strebulaev
, “
How Do Venture Capitalists Make Decisions?
Journal of Financial Economics
135
:
1
(
2020
),
169
190
.
Gonzalez-Uribe
,
Juanita
, and
Michael
Leatherbee
, “
The Effects of Business Accelerators on Venture Performance: Evidence from Start-Up Chile,
Review of Financial Studies
31
(
2018
),
1566
1603
.
Howell
,
Sabrina
, “
Learning from Feedback: Evidence from New Ventures
,” New York University mimeograph (
2018
).
Imai
,
Kosuke
,
Luke
Keele
,
Dustin
Tingley
, and
Teppei
Yamamoto
, “
Unpacking the Black Box of Causality: Learning about Causal Mechanisms from Experimental and Observational Studies,
American Political Science Review
105
(
2011
),
765
789
.
Kaplan
,
Steven
, and
Per
Strömberg
, “
Characteristics, Contracts, and Actions: Evidence from Venture Capitalist Analyses,
Journal of Finance
59
(
2004
),
2177
2210
.
Karajkov
,
Risto
,
Bridging the Gap: Access to Risk Finance for Early-Stage SMEs in Southeast Europe
(
2009
), http://www.incubat.ro/uploads/Publicatii_Investment_SMS_report.pdf.
Lee
,
David
, “
Training, Wages, and Sample Selection: Estimating Sharp Bounds on Treatment Effects,
Review of Economic Studies
76
(
2009
),
1071
1102
.
Mason
,
Colin
, and
Richard
Harrison
, “
`Investment Readiness': A Critique of Government Proposals to Increase the Demand for Venture Capital,
Regional Studies
35
(
2001
),
663
668
.
Mason
,
Colin
, and
Jennifer
Kwok
, “
Investment Readiness Programmes and Access to Finance: A Critical Review of Design Issues,
Local Economy
25
(
2010
),
269
292
.
McKenzie
,
David
, and
Anna Luisa
Paffhausen
, “
Small Firm Death in Developing Countries
,” this review 101 (
2019
),
635
657
.
McKenzie
,
David
, and
Christopher
Woodruff
, “
What Are We Learning from Business Training Evaluations around the Developing World?
World Bank Research Observer
29
(
2014
),
48
82
.
McKenzie
,
David
, and
Christopher
Woodruff
Business Practices in Small Firms in Developing Countries,
Management Science
63
(
2017
),
2967
2981
.
Vizjak
,
Ana
, and
Maja
Vizjak
, “
Activities of Business Angels in Croatia
,”
Journal of US-China Public Administration
13:1
(
2016
),
16
25
.
Wagner
,
Rodrigo
, “
Does Feedback to Business-Plans Impact New Ventures? Evidence from a Field Experiment
” (
2017
), SSRN: http://ssrn.com/abstract=2766566.
Yu
,
Sandy
, “
How Do Accelerators Impact High-Technology Ventures
”?
Management Sciences
64
(
2020
),
530
532
.

Author notes

A supplemental appendix is available online at https://doi.org/10.1162/rest_a_00882.

Supplementary data