The longer term effects of federal subsidies on firmsurvival: evidence from the advanced technologyprogram

Daniel Smith1,2 • Maryann Feldman2 • Gary Anderson3

� Springer Science+Business Media, LLC 2017

Abstract The goal of this paper is to conduct a survival analysis to determine the causal

impact of federal R&D subsidies on firms’ long-term survival. The data are small firms

which applied to the Advanced Technology Program (ATP) in 1998 and 2000. The ATP’s

focus was on ensuring that early stage, high-risk research was eventually commercialized

successfully and resulted in broad economic benefits for society overall. This paper

therefore explores whether the knowledge and benefits the ATP initially provided to a firm

allowed it to more successfully transition future research projects from development and

testing to commercialization. This paper utilizes a variant of the Heckman (Econometrica

47(1):153–161, 1979) research design to control for inherent pre-award differences

between awarded and non-awarded firms. By using administrative data on reviewer scores,

this analysis shows that the impact of ATP on small firm survival is robust to sample

selection. This paper’s findings suggest that recei ving an ATP award can have a significant

and positive causal effect on firm survival.

Keywords Federal � R&D � Subsidies � Firm � Survival � Innovation

JEL Classification H2 � O3

& Daniel Smithdanielsm@live.unc.edu

Maryann Feldmanmaryann.feldman@unc.edu

Gary Andersonganderso@nsf.gov

1 North Carolina Department of Commerce, Raleigh, NC, USA

2 University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

3 National Science Foundation, Alexandria, VA, USA

123

J Technol TransfDOI 10.1007/s10961-017-9633-5

1 Introduction

The basic justification for government subsidies of R&D performed by the private sector is

that there are positive externalities associated with knowledge creation. Consequently, the

private sector will underinvest in R&D (Klette et al. 2000). R&D subsidies potentially

increase firms’ longer term chances of survival by both directly encouraging firms to

innovate and by encouraging firms to collaborate more with others, thereby increasing their

ability to quickly respond to new technological developments in their industries (David

et al. 2000; Cefis and Marsili 2005; Saxenian 1994). R&D subsidies to private firms (as

opposed to universities, non-profit organizations, or government agencies) are important

because economic growth requires a well-balanced mixture of both R&D and business

development (Leyden and Link 2013). In addition, public R&D subsidies can serve as a

signal to private financiers that a given research project is worth investing in, especially

when the company is relatively new (Takalmo and Tanayama 2010; Colombo et al. 2010).

Despite these theoretical benefits, federal subsidization of privately performed R&D

remains a controversial issue. In recent years, federal programs such as the Advanced

Technology Program (ATP) and the Technology Innovation program each administered by

the National Institute of Standards and Technology (NIST) have been eliminated by

Congress (NIST 1998, 2007; NIST n.d.). However, other federal programs which subsidize

the performance of R&D by private entities (such as the SBIR program, the STTR pro-

gram, and the ARPA-E program) remain in existence (National Institutes of Health 2016;

U.S. Department of Energy n.d.; U.S. Department of Energy 2016). The varied fates of

these federal programs demonstrates that the role and impact of federal support for pri-

vately performed R&D is remains unsettled, at least in the United States.

While some of these programs have had aspects of their performance analyzed, there

has been relatively little empirical research on the impact that these programs have had on

long term firm survival, and the research that has been done has focused primarily on the

short term impact of federal R&D support. For instance, Czarnitziki et al.’s (2007) analysis

of the economic effects of federal and provincial Canadian R&D subsidies considers a

3-year impact horizon. Likewise, Einio (2014) examines the effects of R&D subsidies on

firms’ growth up to 3 years after a firm received a subsidy. This is in spite of the fact that

Mansfield (1995) finds that it generally takes 7 years for a firm to move from basic

research to commercialization. In addition, empirical research on this subject has been

hampered due to qualitative differences between firms selected to receive a particular

subsidy and firms that were not, resulting in selection bias (Wallsten 2000; Murnane and

Willett 2011).

This paper examines the longer term impact of firm participation in the ATP. The ATP

was intended to promote high risk research with large potential societal and economic

benefits (National Research Council 2001). The ATP was also specifically intended to

increase the competitiveness of U.S. firms (National Research Council 2001). The ATP did

this by funding particular high-risk high-reward research projects. ATP program managers

remained actively engaged in project management to ensure that firm’s research remained

true to project goals and program criteria. Since a firm’s survival is intertwined with its

innovative output and competitiveness, the ATP both directly and indirectly impacts the

outcomes examined here (Chesbrough 2003).

This paper studies the survival through 2014 of small firms that applied to the ATP in

the 1998 and 2000. This is a much longer time horizon than most studies which examine

the effects of public R&D subsidies received by private corporations (Czarnitziki et al.

2007; Einio 2014). Importantly, this analysis includes all small firm applicants regardless

D. Smith et al.

123

of whether the applicant was actually funded by ATP. Additionally, this paper exploits

ATP proposal reviewer scores to correct for potential selection bias using a variant of the

research method pioneered by Heckman (1979). Reviewer scores provide a rare and

valuable opportunity to control for inherent pre-existing differences between firms that

received an ATP award and those that did not. Failure to control for such differences is a

common issue in the existing literature focused on evaluating the effects of government

subsidies (Wallsten 2000).

This paper is organized as follows. Section 2 gives an overview of the existing literature

on this subject and the ways this paper is expected to contribute to it. Section 3 provides

both an overview of the ATP program and specific details relating to the 1998 and 2000

ATP competitions. Section 4 discusses some of the broad considerations relating to the

research design of this paper, while Sect. 5 describes the specific econometric methods

employed. Section 6 discusses the empirical data and variables being used. Finally, Sect. 7

details the empirical results and Sect. 8 concludes.

2 Literature review—R&D subsidies and firm outcomes

While any subsidy which is explicitly designed to promote firm growth or employment

tends to be focused on the short term, the primary firm level outcome examined by this

paper will be longer term firm survival (Feldman 1994). Theoretically, to the extent that

R&D subsidies increase firms’ innovative output and absorptive capacity, such subsidies

should also have a positive impact on firm survival. Unfortunately, there has been very

little empirical research directly examining these impacts (Bercovitz and Feldman 2007;

Deeds and Decarolis 1999; Chesbrough 2003). Because firm survival is generally closely

related to shorter term impacts of R&D subsidies such as innovative outputs, increased

interfirm collaboration, and firm growth (Musso and Schiavo 2008; Cefis and Marsili

2005), this literature is also discussed in this section.

2.1 Effects of federal R&D subsidies on firms’ innovative output

There are a multitude of theoretical reasons why federal R&D subsidies would be expected

to increase firms’ chances of survival. To begin with, if federal subsidies do boost the

innovative output of firms, then they would also increase the overall lifespan of those firms.

Both Cefis and Marsili’s (2005) survival analysis of Dutch firms and Esteve-Perez and

Manez-Castillejo’s (2008) survival analysis of Spanish manufacturing firms find that

indicators of innovative output increase a firm’s chances of survival. The existing empirical

work done on whether R&D subsidies boost the actual innovative output of a firm gen-

erally suggests that they do. There has been a great deal of empirical research performed on

this issue at various levels. Table 1 provides some examples of this research. This work has

shown that R&D subsidies can positively impact firm patenting and the introduction of

new commercial products.

2.2 Effects of federal R&D subsidies on firms’ ability to collaborateeffectively

However, the benefits a firm gains by receiving a federal subsidy extend beyond increasing

its innovative output. By increasing the amount of internal R&D a firm conducts, subsidies

The longer term effects of federal subsidies on firm survival…

123

would also cause firms to increase their absorptive capacities (David et al. 2000; Bercovitz

and Feldman 2007; Kumar et al. 1999; Aldieri and Cincera 2009). Since absorptive

capacity refers to the ability of firms to make profitable use of external knowledge, this

would make firms more likely to collaborate with other organizations in the future by

raising the potential returns of such collaborations (Bercovitz and Feldman 2007; Dyer and

Singh 1998). In addition, subsidies which explicitly encourage interorganizational col-

laboration (as the ATP does) would be likely to further encourage firms to collaborate with

different organizations by initially assisting firms in establishing social norms which would

make future collaborations less costly (National Research Council 2001; Dyer 1997;

Schrank and Whitford 2011).

Increased interorganizational collaboration would, in turn, be expected to allow com-

panies to more rapidly become aware of and adapt to new technologies, thus improving

their chances of survival and growth (Saxenian 1994). Saxenian (1994) argues that Silicon

Valley outperformed the IT cluster along Route 128 in Massachusetts primarily because

heightened formal and informal collaboration between the firms there allowed the firms

there to better adjust to changes in information technology. Even large, well-run firms

which have invested heavily in internal R&D can be hurt or even fail altogether because of

their inability to adjust quickly enough to new technology (Chesbrough 2003; Christensen

1997). Private firms appear to be becoming more aware of this, as the propensity of firms to

collaborate with organizations such as universities has been increasing over the past few

decades (Hall et al. 2003; Cunningham and Link 2015; Link and Scott 2005; Audretsch

et al. 2012; Link 2015, Loof and Brostrom 2008; Caloghirou et al. 2001; Hanel and St

Pierre 2006; Sachwald 2008). Thus, R&D subsidies would be expected to encourage firms

to collaborate with other organizations, which would improve their chances of survival.

However, there is some debate in the literature about whether federal or regional

governments are better able to promote interorganizational collaboration. Knoben and

Table 1 Empirical literature on R&D subsidies and innovative output

Author(s) Year Geographicarea(s) studied

Unit ofanalysis

Main Findings

Johnstoneet al.

2010 International Country R&D subsidies have significant and positive effects onpatent counts for more costly technologies

Azoulayet al.

2014 United States Researcharea

NIH funding increases the number of patents directlysupported by the NIH, the number of patents citingpatents directly supported by the NIH, and theoverall number of patents in the same research areaas the patents funded by the NIH (Azoulay et al.2014)

Czarnitzikiet al.

2007 Finland;Germany

Firm R&D subsidization had a positive effect on thepatenting activity of Finnish firms but not WestGerman firms

NishimuraandOkamura

2010 Japan Firm R&D subsidization in general had no significant effectbut R&D subsidization in which a firm collaboratedwith a university had a positive and significant effecton patenting activity

Czarnitzikiet al.

2011 Canada Firm R&D subsidization does in fact boost both the numberof new commercial products a firm sells and thenovelty of a firm’s innovations

D. Smith et al.

123

Oerlemans (2006) conduct a literature review of the research on the relationship between

various types of proximity (including geographical proximity) and interorganizational

collaboration, and find that geographic proximity does seem to play a definite role in

determining the success of an interorganizational partnership.

The importance of geographic proximity to successful partnerships suggests that

regional governments might be better suited to enact policies aimed at promoting

interorganizational collaboration than a federal agency. Oughton et al. (2002) argue that

regional governments will be more effective than federal governments at promoting col-

laboration since they are better able to understand and change the institutional environment

of a region. This is supported by the fact that state governments within the United States

are focusing increasingly on developing innovation policies that reflect the specific com-

petitive advantages of the region they are in (Hall and Link 2015). However, it is also true

that regional governments generally do not have access to the same level of resources as

federal governments (Morgan 1997).

The fact that the ATP is a purely federal program without any regional component

makes it ideal for evaluating whether a federal agency can effectively spur interorgani-

zational collaboration and firm survival (National Research Council 2001). This paper thus

provides an indirect contribution to the debate over whether federal governments should be

involved in such efforts. Although it does not directly examine the impact of a 1998 ATP

award on a firm’s number of annual interorganizational collaborations (since the quality of

the collaborations and characteristics of the collaborating partners are generally more

important than the raw number of collaborations a firm engages in), a positive finding in

the survival analysis would provide circumstantial evidence that federal governments

could effectively stimulate beneficial interorganizational collaborations (Stuart 2000;

Baum et al. 2000; Sampson 2007).

2.3 Effects of federal R&D subsidies on firms’ growth

There is very little direct empirical examination of the effect of R&D subsidies on firm

survival. However, R&D subsidies can boost firm productivity both by creating new

innovations that a firm can benefit from and making the firm better able to take advantage

of existing knowledge generated by actors other than the firm of interest (Griffith et al.

2004). Higher firm productivity is, in turn, usually associated with higher firm growth and

thus firm survival (Wagner 2002; Musso and Schiavo 2008). Thus, empirical analyses of

the effects R&D subsidies have on firm growth are germane to the discussion of their

effects on firm survival.

There has been a fair amount of empirical work examining the relationship between

R&D subsidies and firm growth drawing data from a variety of different countries. While

most studies find evidence of a positive and significant relationship between R&D sub-

sidies and firm growth, Wallsten (2000) finds that, once selection bias has been controlled

for, the R&D subsidies they examine have no significant effect on firms’ growth. This

appears to contradict the findings of Bozeman et al. (2008), who find that a lack of early

stage capital is identified by nanotechnology firms in the state of North Carolina in the

United States of America as being one of the primary obstacles to their growth and

competitiveness, as well as Hardin and Link’s (2015) research suggesting that organiza-

tions which attract funding from multiple sources generally create more jobs. However,

Wallsten’s (2000) findings are even more clearly contradicted by those of Einio (2014),

who uses an instrumental variables approach to explicitly control for selection bias and

nevertheless finds that R&D subsidies do have a positive and significant effect on firm

The longer term effects of federal subsidies on firm survival…

123

growth. However, Einio (2014) draws data from Finnish firms while Wallsten (2000)

examines firms in the United States in their analysis. Thus, one of the contributions of this

paper will be to find out whether or not Einio’s (2014) findings also hold true in the United

States. Additionally, Einio (2014) exploits the fact that the Finnish government allocates

R&D subsidies based partially on the population density of a given region to run their

instrumental variables (IV) regression, while Wallsten (2000) bases his IV regression on

the differing budgets that government agencies are required to set aside for SBIR funding.

Also, while Einio’s (2014) study includes all firms which applied for an R&D subsidy,

Wallsten’s (2000) study only uses firms which applied for an SBIR grant with the National

Aeronautics and Space Administration or the Department of Defense as its primary control

group. Table 2 summarizes some of the empirical literature on this issue. Research

demonstrates that under certain conditions R&D subsidies can have a positive impact on

employment, growth, and sales.

2.4 Effects of federal R&D subsidies on firms’ survival prospects

While there has been a fair amount of research relating to factors that could encourage firm

survival, there have been relatively few survival analyses of firms (Cefis and Marsili 2005).

The studies that do exist provide some theoretical reasons for why R&D subsidies might be

Table 2 Empirical literature on R&D subsidies and firm growth

Author(s) Year Geographicarea(s) examined instudy

Main Findings

Lerner 2000 United States There is a positive correlation between receiving an R&Dsubsidy and higher employment at a firm

Wallsten 2000 United States Once selection bias is controlled for there is no significantrelationship between R&D subsidization and firmemployment

Hussinger 2008 Germany Public R&D subsidies have (statistically) the same positiveand significant effect on firm growth as private R&Dspending

Hall andMaffioli

2008 Argentina, Brazil,Chile, and Panama

Public R&D subsidies have positive effects on firm growth

Czarnitzikiet al.

2011 Canada R&D subsidization boosts the sales and generalperformance of manufacturing firms

BecchettiandTrovato

2002 Italy Subsidies of any sort will tend to boost the growth of smalland medium sized Italian firms

HytinnenandToivanen

2005 Finland Government R&D subsidies disproportionately boost thegrowth of firms in industries that tend to be moredependent on some form of external finance (Hytinnenand Toivanen 2005)

Audretschet al.

2002 United States Completing Phase II of the SBIR program does have apositive and significant effect on a firm’s sales of thetechnology it developed through the SBIR program

Einio 2014 Finland After controlling for selection bias, R&D subsidies are stillfound to have positive and significant effects on firms’sales and employment

D. Smith et al.

123

expected to increase recipient firms’ chances of survival. For instance, Cefis and Marsili

(2005) find that firms’ innovative outputs do have a positive and significant effect on their

survival chances, while Esteve-Perez and Manez-Castillejo (2008) find that firms which

perform their own R&D are likely to survive longer. Table 3 summarizes many of the

studies on firm survival.

Significantly, none of the survival analyses which have been conducted thus far have

directly examined the role governments can play in encouraging firm survival, despite calls

for the governments of developed nations to implement such policies in the literature

(Esteve-Perez and Manez-Castillejo 2008; Holmes et al. 2010). This work therefore fills an

important gap in the literature by evaluating what role federal policy can play in increasing

firms’ survival chances.

3 Description of the ATP and the 1998 and 2000 ATP competitions

The ATP was intended to promote high risk research with large potential social and

economic benefits (National Research Council 2001). The ATP was also specifically

intended to increase the competitiveness of U.S. firms (National Research Council 2001).

The ATP did this by funding particular high-risk high-reward research projects. The

program was created as a response to concerns within the U.S. of a loss of global com-

petitiveness on the part of U.S. firms in the face of increasing competitive pressures from

Asian businesses (National Research Council 2001). The overarching goal of the ATP was

to fund high risk research with large potential social and economic benefits (National

Research Council 2001). Additionally, the ATP was intended to foster greater collabora-

tion between firms (National Research Council 2001). Both individual U.S. owned for-

profit businesses and joint ventures between at least two for-profit businesses are eligible to

apply for the ATP (National Institute of Standards and Technology 2004).

It is important to note that ATP grants were not general purpose R&D subsidies but

were instead intended to fund specific projects that met selective funding criteria. To

ensure that the program targeted high-risk high-reward research projects, the program

established funding criteria related to both scientific and technological merit as well as the

Table 3 Empirical literature on R&D subsidies and firm survival

Author(s) Year Geographicarea(s) examined instudy

Main Findings

Klepper 2002 United States Firms which enter industries earlier and thus have moreexperience are more likely to survive

Cefis and Marsili 2005 The Netherlands A firm’s innovative outputs do increase that firm’schances of survival

Musso andSchiavo

2008 France Financial constraints have a negative and significantimpact on a firm’s chances of survival

Esteve-Perez andManez-Castillejo

2008 Spain Firms which conduct their own R&D are likely tosurvive longer than firms which outsource their R&Dactivities

Holmes et al. 2010 United Kingdom Small and medium enterprises are most likely to fail inthe first 8 or 9 years after their establishment

The longer term effects of federal subsidies on firm survival…

123

potential for broad-based economic benefits (National Institute of Standards and Tech-

nology 1998). Table 4 presents the three detailed criteria in each of these two broad

categories (Feldman and Kelley 2006; National Institute of Standards and Technology

2004). Throughout the review and award process, each criterion received equal weight.

Notably, there are significant differences between these selection criteria and typical

business investment criteria. In contrast to standard business investment criteria, projects

that had a high degree of technological innovation as well as high technical risk, given a

credible technical plan to overcome this risk, were viewed favorably by the ATP (National

Institute of Standards and Technology 1998). Similarly, the criteria favored projects that

had significant economic benefits beyond those captured by the proposing firm itself

(National Institute of Standards and Technology 1998). Finally, the criteria rewarded

particular projects that did not have access to capital from external or internal sources

(National Institute of Standards and Technology 1998). Together, these evaluation criteria

are aimed at ensuring that the ATP invested in innovative, high risk research that the

private sector would not have funded on its own yet had the potential to generate large

social and economic benefits. Table 4 provides a summary of the assessment criteria for

ATP projects.

As the first part of the ATP review process, all proposals were reviewed by one or more

independent experts (National Institute of Standards and Technology 1998). These

reviewers assigned equally weighted scores ranging from 0 to 10 for each of the criteria in

Table 4 (Feldman and Kelley 2006; National Institute of Standards and Technology 2004).

Reviewers were instructed to give high scores to proposals that demonstrated a high degree

of technical innovation, had a high degree of technical risk alongside a credible plan to

overcome that risk, and a quality R&D plan (National Institute of Standards and Tech-

nology 1998). Proposals with significant benefits beyond those to the proposing firm, a

demonstrated lack of internal and external funding and a demonstration of a pathway to

market were scored highly (National Institute of Standards and Technology 1998). NIST

Table 4 Assessment criteria for the ATP

Criteria Description

Scientific and technological merit

1. Innovation inTechnology

Projects must display a high degree of innovation. Innovation may be relatedto the objectives of your research or the approach to achieving theseobjectives

2. High Technical Risk andFeasibility

Projects must be high risk—the mission of ATP is to overcome high-risktechnical barriers—and projects must be credible with respect to technicalapproach

3. Quality of R&D Plan Projects must have a detailed technical plan over the life of the project. Initialtime periods must have more details. Decision points and alternativesshould be discussed in the context of R&D strategy

Potential for broad-based economic benefits

4. National EconomicBenefit

Projects must demonstrate significant economic benefits to the Nation

5. Need for ATP funding Project must need public funds. Applicants must demonstrate that full privatefunding is not available that the project does not meet company criteria orprofile for internal funds

6. Pathway to EconomicBenefit

Proposal must demonstrate how technology, if successful, will be brought tomarket or enter use

D. Smith et al.

123

assembled expert panels that considered these reviews alongside other materials in order to

make recommendations to the ATP Selecting Official who ultimately determined which

projects would be funded (Feldman and Kelley 2006; NIST 2004).

Beyond the criteria and review process, NIST assigned technical personnel to actively

manage and monitor each funded ATP project over the duration of the grant (National

Institute of Standards and Technology 1998). These personnel had significant technical and

research experience in areas directly related to the particular funded project. ATP project

managers remained actively engaged in project management to ensure that a firms’

research remained true to project goals and program criteria (National Institute of Stan-

dards and Technology 1998).

Both the 1998 and 2000 ATP competitions consisted of a general competition for all

technologies and several smaller competitions for specific technology areas (Balutis and

Lambis 2001). There were 502 total project proposals sent in for the 1998 competition

which involved a total of 822 organizations (Feldman and Kelley 2003). 741 of these

organizations were for-profit firms (Feldman and Kelley 2003). For the 2000 ATP com-

petition, a total of 417 proposals involving 555 different organizations were submitted

(National Institute of Standards and Technology 2005). 536 of these organizations were

for-profit firms.

A total of 79 ATP awards were given out in the 1998 competition, while only 58 awards

were given out in the 2000 competition (National Institute of Standards and Technology

n.d.; National Institute of Standards and Technology 2005). The funded projects from the

1998 competition involved 171 organizations, while the funded projects from the 2000

competition only involved 85 organizations (National Institute of Standards and Tech-

nology 2005). Out of these organizations 151 were for-profit firms in the 1998 competition

while 74 were for-profit firms in the 2000 competition (National Institute of Standards and

Technology 2005). The dollar value of the average ATP award given out in the 1998

competition was $3,028,714, while the dollar value of the average received 2000 ATP

award was $2,532,493 (National Institute of Standards and Technology n.d.; National

Institute of Standards and Technology 2005). This information is summarized below in

Table 5 Information Concerning the 1998 and 2000 ATP Competitions

Measure Data Source

Number of TotalApplications

919 Feldman and Kelley (2003), National Institute of Standards andTechnology (2005)

Number of TotalApplicants

1377 Feldman and Kelley (2003), National Institute of Standards andTechnology (2005)

Number of Total For-Profit Applicants

1277 Feldman and Kelley (2003), National Institute of Standards andTechnology Administrative Records

Number of FundedProjects

137 National Institute of Standards and Technology n.d., NationalInstitute of Standards and Technology (2005)

Number of Awardees 256 National Institute of Standards and Technology administrativerecords, National Institute of Standards and Technology (2005)

Number of For-ProfitAwardees

225 National Institute of Standards and Technology AdministrativeRecords

Average Amount ofAward in 1998

$3,028,714.37 National Institute of Standards and Technology AdministrativeRecords

Average Amount ofAward in 2000

$2,532,493 National Institute of Standards and Technology AdministrativeRecords

The longer term effects of federal subsidies on firm survival…

123

Table 5. An appendix containing a case study to demonstrate the nature of ATP-funded

research is omitted for the sake of brevity but available upon request.

4 Research design considerations

This section discusses the broad empirical strategy used to isolate the causal effect of

receiving an ATP award on the performance of awardee firms. The treatment group

consists of firms which applied for and received an ATP award, while the control group

consists of firms which applied for but did not receive an ATP award in any year. For this

reason, the basic research design employed by this paper is a posttest-only design that

includes a control group (Shadish et al. 2002). Because the assignment of ATP awards is

non-random, this should be considered a quasi-experimental research design (Murnane and

Willett 2011, p. 31).

The primary methodological issue with this type of research design is that there can be

significant differences in the treatment and control groups prior to treatment (Shadish et al.

2002). To the extent that these differences are correlated with the outcome measure of

interest, estimates of treatment impacts that ignore these differences will be biased

(Shadish et al. 2002). This is likely to be problematic when attempting to evaluate the

effects of government R&D subsidies on firms. Wallsten (2000) points out that government

agencies are more likely to award subsidies to firms that they believe will perform better.

This means that if pre-existing differences between awardee firms and firms that did not

receive awards are not taken into account then the benefits firms receive from subsidies are

likely to be overestimated (Wallsten 2000). This problem is known as selection bias

(Morgan and Winship 2007).

The fact that the control firms examined in this paper are not drawn at random from the

general population of U.S. firms but are drawn from the population of firms that applied for

a 1998 or 2000 ATP award means that they can be considered an internal control group

(Shadish et al. 2002, p. 122). The differences between the control and the treatment firms

prior to treatment are therefore likely smaller than those between the treatment group and a

control group of firms selected from the general population (Shadish et al. 2002, p. 122).

However, the use of an internal control group is not by itself enough to guarantee that

there are no pre-existing differences between treatment and control firms, especially if

better performing firms were selected to receive ATP awards (Shadish et al. 2002; Wallsten

2000). To fully control for pre-treatment differences between treatment and control firms,

it is necessary to utilize a variant of the research design pioneered by Heckman (1979) in

which the predicted probability of a given observation receiving treatment is included in

the regression on the outcome variable of interest. The analysis takes into account Puhani’s

(2000) finding that for this approach to be unbiased it is necessary to include at least one

variable in the regression on the likelihood of receiving treatment which is excluded from

the final regression.

This paper employs the average reviewer scores for one of the ATP criteria related to

the potential for broad-based economic benefits as such a variable. The analysis uses the

average reviewer score for the Need for ATP criteria. Reviewers rated projects highly

when the project had a demonstrated need for public funds (National Institute of Standards

and Technology 1998). Highly scored projects demonstrated that external public and

private funding was not available and that the project did not meet the company criteria or

profile for internal funding (National Institute of Standards and Technology 1998).

D. Smith et al.

123

Because it is one of the three sub-criteria used to determine a firm’s overall business score,

this variable should be significantly correlated with whether or not a firm receives an ATP

award (National Institute of Standards and Technology 1998). However, because this

project-level funding criteria does not directly relate to overall firm performance or

innovativeness, it is likely uncorrelated with overall firm survival. This variable is there-

fore suitable to control for selection bias (Puhani 2000).

5 Empirical methodology

Since this paper employs a modified version of Heckman’s (1979) method for eliminating

selection bias, it is necessary to run a first stage Probit regression featuring an indicator

variable for whether a firm received an ATP award as the dependent variable. The inde-

pendent variables in this regression will be the sub-score for how well a firm demonstrated

that it would only carry out its research project if it received ATP funding and all control

variables employed in the final analysis. This regression will then be used to predict the

probability of each individual firm receiving an ATP award, which will be included in the

final analysis as a control variable. This first stage equation is given below in Eq. 1. A

represents an indicator variable for whether a firm received an ATP award, S represents the

reviewer sub-score for the criteria used only in the first stage, and Z* represents a vector of

the control variables used in a specific analysis.

Equation 1. First Stage Probit Regression Equation of Indicator for Firms’ ATPAwardee Status

PðAi ¼ 1jSi; Z�i Þ ¼ 2pð Þ�1=2

e�

aþb1Siþb�Z�ið Þ2

2

� �ð1Þ

For the survival analysis, a Cox regression is used in accordance with the work of Cefis

and Marsili (2005). The assumption of proportional hazards made by the Cox regression is

tested by calculating the Schoenfeld residuals and determining whether they take on a

nonzero slope when graphed over survival time using the Stata command ‘‘stphtest’’

(Cleves et al. 2004). Equation 2 gives the hazard rate for a firm j in year t as defined by the

Cox regression. P* represents the predicted probabilities for whether a firm received an

ATP award from the stage 1 regression, A represents an indicator variable for whether a

firm received an ATP award, and Z represents a vector of the control variables.

Equation 2. Hazard Rate for Firm J in Year T as Defined by the Cox Regression

h tj P�j þ Z�

J

� �� �¼ h0 tð Þexp b1P

�i þ b2Ai þ b�Z�

i

� �: ð2Þ

6 Data and variables

The primary data set used in these analyses is a cross-sectional set consisting of 302

unique, firm-level observations relating to small for-profit firms that participated in either

the 1998 ATP competition, the 2000 ATP competition, or both. All of these firms employ

fewer than 500 individuals in total, a standard the U.S. Small Business Administration

The longer term effects of federal subsidies on firm survival…

123

often uses to determine whether or not a given firm qualifies as a small business (U.S.

Small Business Administration 2014). This analysis focuses on small firms because they

often lack the resources to conduct major R&D efforts, and are thus generally less likely to

survive than large, multinational firms (U.S. Small Business Administration n.d.). This data

was drawn primarily from administrative records maintained by the National Institute of

Standards and Technology, but it was supplemented with data from some other sources

which are described below.

The dependent variable in all analyses will be the number of full years after a firm

participated in an ATP competition that it survived up until 2014. The competition in the

year 2000 is treated as the starting point for firms that participated in both competitions.

For the purpose of this analysis, only firms that have gone bankrupt are considered to have

died. Firms which merge with or are acquired by other organizations are not considered to

have expired unless the merged or acquiring entity subsequently goes bankrupt. To ensure

that all observations were non-zero (and thus able to be included in the analysis) a value of

one was added to each observation. Data on this variable is drawn from reputable publi-

cally available online sources such as news articles from well-known publications, industry

news websites, and the databases for different state-level government organizations

responsible for keeping track of business registration details. Because firm level ATP

administrative data, particularly data on firms which applied for but did not receive ATP

awards, is highly sensitive and proprietary, it cannot be publicly released.

The primary independent variable of interest used in all analyses will be an indicator

variable for whether or not a firm received an ATP award at any point in its history. Using

binary variables to represent participation in a federal program is common in the literature

(Feldman and Kelley 2006; Blanes and Busom 2004). This data is drawn from adminis-

trative records maintained by the National Institute of Standards and Technology.

As discussed above, to correct for selection bias this paper will utilize the average

reviewer score for the Need for ATP (NATP) criteria. This score relates to how well a firm

is able to convince the proposal evaluators at the National Institute of Standards and

Technology funding for the proposed project is not available from either external or

internal sources and that it will only carry out its proposed research project if it receives

ATP funding (National Institute of Standards and Technology 1998). As with the overall

business and technical scores, the NATP score can range from 0 to 10 with 10 being the

best score (Feldman and Kelley 2006). Each proposal is reviewed by at least one inde-

pendent expert. Where there are multiple reviews for a single proposed project, the analysis

uses the average NATP. Because the analysis is at the company rather than project level,

NATP reviewer scores were also averaged across proposals for those companies with

multiple proposed projects. This data was all derived from the administrative records of the

National Institute of Standards and Technology.

It is important to control for the positive effects that other, non-ATP government

funding could have on firms. Unfortunately, there is currently no reliable source detailing

all of the subsidies made by the U.S. government to firms (Welsh 2014). For this reason, an

indicator variable for whether a given firm had ever received a Small Business Innovation

Research (SBIR) or Small Business Technology Transfer (STTR) award before its

application to the ATP program was created as a proxy for how likely a firm was to receive

government funding from sources other than the ATP program. Taken together, the SBIR

and STTR programs are one of the largest providers of early stage funding to innovative

small businesses in the United States (National Institute of Health 2015). This data was

drawn from the Small Business Administration Technet database.

D. Smith et al.

123

The remaining control variables used in the analyses are mainly based on those used in

the firm level survival analysis employed by Cefis and Marsili (2005). The control variable

for firm size in the year the firm participated in a competition or the year immediately prior

to it is an ordinal variable ranging from 1 to 13, corresponding to employee numbers

ranging from 0 to 499. Link and Rees (1990) have shown that firm size can have a

significant impact on the advantages a firm derives from some of its innovative activities.

A set of three indicator variables corresponding to whether the project proposal a firm

submitted related to biotechnology, electronics, or information technology is included to

control for industry effects. A dollar measure of a firm’s R&D budget in 1997 (for firms

which participated in the 1998 ATP competition) or a firm’s R&D budget in the year 2000

(for firms which participated in the 2000 ATP competition) is included to control for the

overall level of innovative inputs that a firm has. This data is all drawn from two separate

sources: a survey the National Institute of Standards and Technology administered to firms

which had participated in the 2000 ATP competition and the survey Feldman and Kelley

(2006) administered to firms which participated in the 1998 ATP competition.

To control for the overall quality of a firm’s innovative capacity, this analysis will use

the average of the overall technical scores assigned to a firm (generated using the same

process as that used to create the variable relating to a firm’s NATP score). Because the

technical score is an ex ante assessment of a given firm’s project, it is not a perfect measure

of the quality of a firm’s innovative output. Nevertheless, its focus on the level of inno-

vation, feasibility (albeit with high technical risk viewed positively), and quality of a firm’s

R&D plan makes it a reasonable proxy of the overall quality of a firm’s innovation. This

data was drawn from the administrative records of the National Institute of Standards and

Technology.

Finally, as a robustness check an additional regression will be run which will include the

averaged overall business score associated with a firm (generated using the same process as

that used for the NATP and technical scores) instead of the predicted probabilities of a firm

being selected to receive an ATP award as a control variable. It is important to note that the

overall business score refers to the composite of the three sub-scores discussed in Sec-

tion III. Data on this variable is again drawn from the administrative records at the National

Institute of Standards and Technology. Table 6 provides the summary statistics and

sources of the variables discussed above for the applicants considered in this analysis.

7 Results

We examine the impact of ATP on firm survival using a number of alternative econometric

specifications. Results are presented in Table 7. Column (1) presents the results of a Cox

regression which in addition to the standard control variables suggested by the literature

review included a number of variables based on ATP administrative data. Treatment is

measured using an indicator for whether an applicant received an ATP award. Addition-

ally, we control for the overall technical and economic merit of the proposals using average

reviewer scores This regression therefore did not take any explicit steps to control for

selection bias and the estimate of the impact of receiving an ATP award may be biased.

The results indicate that receiving an ATP award did have a positive and significant effect

on a firm’s survival chances at the 5% level. The number of employees at a firm was also

found to have a positive and significant effect on a firm’s chances of surviving at the 5%

level, while a firm’s R&D budget had a positive and significant effect at the 10% level.

The longer term effects of federal subsidies on firm survival…

123

Table 6 Summary statistics

Variable Mean Median SD Skewness Kurtosis Source

Number of fullyearssurvived

13.381 15.000 4.216 - 1.593 4.369 Various sources;see

AveragedNATPbusiness score

3.927 3.500 2.016 0.641 3.064 NationalInstitute ofStandards andTechnology

Indicator forwhether firmreceived anATP award

0.440 0.000 0.497 0.240 1.058 NationalInstitute ofStandards andTechnology

Indicator forwhether afirm receivedSBIR orSTTRfunding

0.387 0.000 0.488 0.462 1.214 U.S. CensusBureauthrough theNationalInstitute ofStandards andTechnology

Firm R&Dbudget

2,438,129.00 600,000.00 7,424,061.00 8.214 81.879 NationalInstitute ofStandards andTechnology;Feldman andKelley (2006)

Number ofemployees

4.424 4.000 2.457 1.237 4.485 NationalInstitute ofStandards andTechnology;Feldman andKelley (2006)

Indicator for anATP projectfocused onbiotechnology

0.245 0.000 0.431 1.186 2.406 NationalInstitute ofStandards andTechnology;Feldman andKelley (2006)

Indicator for anATP projectfocused onelectronics

0.308 0.000 0.462 0.832 1.692 NationalInstitute ofStandards andTechnology;Feldman andKelley (2006)

Indicator for anATP projectfocused oninformationtechnology

0.139 0.000 0.347 2.086 5.352 NationalInstitute ofStandards andTechnology;Feldman andKelley (2006)

Indicator forparticipationin the 1998ATPcompetition

0.354 0.000 0.479 0.609 1.371 NationalInstitute ofStandards andTechnology

D. Smith et al.

123

These findings are in line with those of Cefis and Marsili (2005), who find that firm size

and innovation both boosted a firm’s survival prospects. Of course, to definitively show a

causal link between receiving an ATP award and firm survival it is necessary to explicitly

control for selection bias. The Breslow method for tied failures was employed in this

regression. The test of the Schoenfeld residuals for this regression returned a Chi squared

value of 8.110 and a corresponding p value of 0.703, indicating that the proportional

hazards assumption held and a Cox regression was appropriate.

The results from the Probit regression to predict the likelihoods of firms receiving ATP

awards are in column (2). These results show that the averaged NATP score had a positive

and significant effect, with an estimated coefficient of 0.289 and a corresponding p value

smaller than 0.001. Despite this, however, the correlation between the averaged NATP

score and the number of full years a firm survived was only 0.224. This indicates that the

NATP score is a variable which satisfies the criteria put forward by Puhani (2000). The size

of a firm, averaged technical score associated with a firm, and indicator variable for

whether a firm participated in the 2000 ATP competition were also found to all have

positive and significant coefficients.

Column (3) contains the main result of interest. The results of the Cox regression with

the predicted probabilities derived from the Probit regression as a control variable indicate

that receiving an ATP award does have a positive and significant effect on a firm’s survival

chances. The number of employees at a firm was also found to be positive and significant at

the 5% level, while a firm’s R&D budget was positive and significant at the 10% level.

This is consistent with the findings of Cefis and Marsili (2005). The Schoenfeld residuals

test for this regression yielded a Chi squared value of 7.610 and a corresponding p value of

0.748. This regression used the Efron method to account for tied failures and robust

standard errors.

As a robustness check, column (4) displays the results of a second Cox regression which

was identical to that used in the primary analysis except that it used the Breslow method to

account for tied failures instead of the Efron method. The results of this regression did not

differ from those of the primary regression in any significant way. The Schoenfeld

residuals test for this regression produced a Chi squared value of 7.230 and a corre-

sponding p value of 0.781.

Table 6 continued

Variable Mean Median SD Skewness Kurtosis Source

Indicator forparticipationin the 2000ATPcompetition

0.735 1.000 0.442 - 1.066 2.135 NationalInstitute ofStandards andTechnology

Averagedoveralltechnicalscore

4.282 3.833 2.191 0.623 2.647 NationalInstitute ofStandards andTechnology

Averagedoverallbusiness score

3.805 3.424 2.059 0.568 2.869 NationalInstitute ofStandards andTechnology

The longer term effects of federal subsidies on firm survival…

123

8 Conclusion

This paper has demonstrated that receiving ATP funding has had a positive and significant

causal impact on the lifespan of small firms which participated in either the 1998 or the

2000 ATP competition and received ATP funding. By employing a variant of the research

design first pioneered by Heckman (1979) and exploiting the NATP scores assigned during

the selection process, this paper has addressed the common econometric issue of selection

bias in subsidy evaluation (Puhani 2000; Wallsten 2000; Murnane and Willett 2011). In

doing so, this paper fills a significant hole in the literature relating to firm survival analysis

Table 7 Impact of ATP award on firm survival

(1) (2) (3) (4)Dependent variable

# Yearssurvived

Indicator: receivedATP award

# Years survived # Years survived

Indicator: ATP awardrecipient

0.534**(0.160)

0.501**(0.141)

0.505**(0.141)

Indicator: Prior SBIR or STTRrecipient

0.750(0.189)

0.079(0.175)

0.762(0.191)

0.760(0.187)

Firm R&D Budget 1.000*(1.53e - 08)

- 3.42e - 10(1.59e - 08)

1.000*(1.57e - 08)

1.000*(1.54e - 08)

Number of employees 0.858**(0.054)

0.121**(0.036)

0.860**(0.062)

0.862**(0.061)

Indicator: biotechnologyproject

1.312(0.379)

0.340(0.217)

1.297(0.373)

1.298(0.366)

Indicator: electronics project 0.912(0.266)

0.236(0.207)

0.892(0.256)

0.898(0.254)

Indicator: informationtechnology project

0.549(0.226)

0.210(0.260)

0.538(0.217)

0.542(0.216)

Indicator: 1998 competitionparticipant

1.321(0.719)

- 0.460(0.339)

1.195(0.623)

1.192(0.616)

Indicator: 2000 competitionparticipant

0.793(0.429)

1.303**(0.374)

0.855(0.511)

0.853(0.504)

Average technical score 0.881(0.077)

0.351**(0.077)

0.864(0.113)

0.865(0.111)

Average NATP score 0.289**(0.073)

Average business score 0.912(0.083)

Predicted probability ofreceiving an ATP award

0.795(0.811)

0.802(0.807)

N = 302 N = 302 N = 302 N = 302

CoxRegression

Probit Regression Two-Stage CoxRegression

Two Stage CoxRegression

Standard error is given in parentheses below estimated coefficient

*Indicates a coefficient is significant at the 10% level

**Indicates a coefficient is significant at the 5% level

D. Smith et al.

123

by directly examining the role that government policies can play in boosting firms’ life

spans. By focusing on a 14–16 year timespan, this paper also delivers a much needed

examination of the longer term benefits of R&D subsidies. This paper should be seen as

supporting the idea that Einio’s (2014) findings hold for the United States as well. Of

course, it is important to note that Einio’s (2014) paper and this paper examined related but

different firm-level outcomes. For policy-makers, this paper implies that national gov-

ernments can play a beneficial role in stimulating innovation in industry if they structure

R&D subsidy programs appropriately.

This paper also contributes to the ongoing debate over public sector entrepreneurship,

which is an important factor in technology transfer (Dublin Institute of Technology 2017).

Public sector entrepreneurship has been defined as ‘‘the promulgation of innovative public

policy initiatives that generate greater economic prosperity by transforming a status-quo

economic environment into one that is more conducive to economic units engaging in

creative activities in the face of uncertainty’’ (Link 2016, p. 355). Given its focus on

stimulating the innovative projects of various firms in a way that would not occur without

government intervention, the ATP clearly falls into this category (National Institute of

Standards and Technology 1998). This paper shows that, in the United States, programs

such as the ATP might be most useful to nascent entrepreneurs, who typically have fewer

resources than well-established entrepreneurs (Leyden 2016). In addition, by emphasizing

interorganizational collaboration, the ATP could help nascent entrepreneurs form the

strong ties to other organizations that they need (Leyden 2016). The paper can be seen as

supporting the argument by Richardson et al. (2016) that the ATP provides capital to

entrepreneurial firms which they might not otherwise receive due to the uncertain nature of

entrepreneurship. Thus, this paper provides significant new insights on the question of how

public sector entrepreneurship can help to facilitate technology transfer from the laboratory

to the market.

It is important to acknowledge the limitations of this research. These findings may not

be applicable to firms located outside of the United States, or to firms with more than 500

employees. Additionally, there is some amount of ambiguity regarding what constitutes a

firm’s ‘‘death.’’ While this analysis did not treat a firm’s acquisition as the equivalent of its

bankruptcy, it is at least possible that in some cases a larger organization might not receive

any overall economic benefit from acquiring a smaller firm but would nevertheless con-

tinue to exist. Unfortunately, it is impossible to determine which particular acquisitions

ultimately benefitted the acquiring entity in some way and which did not without an

unfeasibly large data collection effort. However, the fact that most profit-maximizing

businesses would be unwilling to acquire another business without an extensive exami-

nation of what that business had to offer them suggests that this should not be too great of

an issue. Also, this analysis does not directly address the issue of whether federal R&D

funding crowds out private R&D funding (David et al. 2000). In theory, if a firm that

received ATP funding had its own funding crowded out, then it could use its private R&D

funding on another project and thereby boost its survival chances still further. While the

inclusion of the NATP score in the review process was intended to prevent crowding out

(National Institute of Standards and Technology 1998), it would require a causal analysis

of awardees’ innovative output to determine if this was the case.

Also, the conclusions of this paper may not apply to R&D subsidy programs which

differ significantly from the ATP in terms of structure or selection criteria. The ATP was

fairly unique in terms of both the standards it used to evaluate applications and the fact that

it continued to actively monitor and manage (in collaboration with recipient firms) funded

ATP projects (National Institute of Standards and Technology 1998). Thus, this analysis

The longer term effects of federal subsidies on firm survival…

123

does not indicate that any U.S. federal R&D subsidy would be expected to boost firm

survival. Instead, it suggests that the programs with selection criteria and project man-

agement practices similar to the ATP may have a positive and significant causal effect on

the survival chances of recipient firms.

There are two main directions for future research suggested by this paper. The first is to

compare the effects of the ATP to the effects of other R&D subsidy programs with similar

goals but different structures and awardee selection processes. This would be useful in

determining how best to organize R&D subsidies going forward. One such feature of the

ATP which deserves further examination is that, beginning in 1994, businesses were

allowed to play a role in establishing several sub-competitions within the ATP focused on

promoting research in technology areas they felt would yield broad and significant eco-

nomic benefits (Balutis and Lambis 2001). Businesses would submit white papers calling

for research in specific technology areas, and then staff at the National Institute for

Standards and Technology would evaluate these proposals and work with the businesses to

finalize the technology fields included in the competition (Balutis and Lambis 2001). It will

be interesting to see whether this collaboration of government and industry (coupled with

the rigorous project review process of the ATP) is able to overcome the potential problem

of government being unable to determine which R&D projects it would be socially optimal

to invest in (National Research Council 2001; Stiglitz and Wallsten 1999). Comparing the

results of ATP subsidies given out after 1994 to the results of subsidies with similar goals

but with no input from industry would shed light on this question.

Another feature of the ATP which deserves further examination is the specific scores it

uses to evaluate applications. One of these is the NATP score. This score was included in

the review process to prevent federal funding from crowding out private R&D funding for

R&D projects which would have occurred regardless of whether the recipient firm received

a subsidy (David et al. 2000). However, the NATP score only constituted one-third of one

of the two scores used to determine whether an ATP applicant received funding (National

Institute of Standards and Technology 1998). Comparing the causal effects of ATP funding

to the causal effects of both subsidy programs which placed more emphasis on preventing

crowding out and those which placed less emphasis on that issue could provide valuable

insight into how major or minor an issue crowding out is in practice.

Similarly, the fact that one of the sub-scores used to determine the technical score

positively relates explicitly to how high the technical risk of a given project was might

provide a possible research opportunity (National Institute of Standards and Technology

1998). Because businesses typically shy away from R&D projects involving a high level of

technical risk, there is a strong theoretical justification for arguing that businesses which

received an ATP award would not have engaged in the ATP projects they proposed without

ATP funding. It would be interesting to check the empirical validity of this claim by

comparing the performance of ATP recipients with recipients of a similar subsidy which

did not place an emphasis on high technical risk.

Finally, the fact that the ATP is a purely federal program provides an interesting avenue

for further research. As discussed above, there is some disagreement in the literature as to

whether federal or regional agencies would be more effective at promoting the interor-

ganizational partnerships necessary for encouraging innovation (Oughton et al. 2002).

While this paper has shown indirectly that federal agencies do seem to have the ability to

foster these partnerships through R&D subsidies, the question of whether regional agencies

could conduct this task more effectively remains open. Comparing the causal effects of the

ATP on firm outcomes to the causal effects of R&D subsidies administered by regional

agencies would help to answer this important question.

D. Smith et al.

123

The second direction of research suggested by this paper is a further examination of the

benefits produced by the ATP. The ATP was explicitly intended to promote broad eco-

nomic benefits (National Institute of Standards and Technology 1998). Firm survival time

is one metric of such benefits, but it is not the only one. Analyses focused on determining

other potential benefits of the ATP, such as its impact on firms’ patenting behavior or their

successful commercialization of research projects, would shed more light on the overall

desirability of federal R&D subsidies structured similarly to the ATP in the United States.

References

Aldieri, L., & Cincera, M. (2009). Geographic and technological R&D spillovers within the triad: Microevidence from U.S. patents. The Journal of Technology Transfer, 34(2), 196–211.

Audretsch, D., Leyden, D., & Link, A. (2012). Universities as research partners in publicly supportedentrepreneurial firms. Economics of Innovation and New Technology, 21(5–6), 529–545.

Audretsch, D., Link, A., & Scott, J. (2002). Public/private partnerships: Evaluating SBIR-supportedresearch. Research Policy, 31(1), 145–158.

Azoulay, P., Graff-Zivin, J., Li, D., & Sampat, B. (2014). Public R&D Investments and Private SectorPatenting: Evidence from NIH Funding Rules. Kellogg School of Management. Retrieved from http://economics-eng.huji.ac.il/Uploads/Contents/documents/Pierre_Azoulayy.pdf.

Balutis, A., & Lambis, B. (2001). The ATP Competition Structure. In C. Wessner (Ed.), The advancedtechnology program: Assessing outcomes (pp. 175–188). Washington, D.C.: National Academy Press.

Baum, J., Calabrese, T., & Silverman, B. (2000). Don’t go it alone: Alliance network composition andstartups’ performance in Canadian biotechnology. Strategic Management Journal, 21(3), 267–294.

Becchetti, L., & Trovato, G. (2002). The determinants of growth for small and medium sized firms: The roleof the availability of external finance. Small Business Economics, 19(4), 291–306.

Bercovitz, J., & Feldman, M. (2007). Fishing upstream: Firm innovation strategy and university researchalliances. Research Policy, 36(7), 930–948.

Blanes, J., & Busom, I. (2004). Who participates in R&D subsidy programs? The case of Spanish manu-facturing firms. Regional Policy, 33(10), 1459–1476.

Bozeman, B., Hardin, J., & Link, A. (2008). Barriers to the diffusion of nanotechnology. Economics ofInnovation and New Technology, 17(7–8), 749–761.

Caloghirou, Y., Tsakanikas, A., & Vonortas, N. (2001). University-industry cooperation in the context of theEuropean framework programs. The Journal of Technology Transfer, 26(1–2), 153–161.

Cefis, E., & Marsili, O. (2005). A matter of life and death: Innovation and firm survival. Industrial andCorporate Change, 14(6), 1167–1192.

Chesbrough, W. (2003). Open innovation: The new imperative for creating and profiting from technology.Watertown, MA: Harvard Business Press.

Christensen, C. (1997). The innovator’s dilemma: When new technologies cause great firms to fail. Boston,MA: Harvard Business Press.

Cleves, M., Gould, W., & Gutierrez, R. (2004). An introduction to survival analysis using stata (Reviseded.). College Station, TX: Stata Press.

Colombo, M., Mustar, P., & Wright, M. (2010). Dynamics of science-based entrepreneurship. The Journalof Technology Transfer, 35(1), 1–15.

Cunningham, J., & Link, A. (2015). Fostering university-industry collaborations in European Unioncountries. International Entrepreneurship and Management Journal, 11(4), 849–860.

Czarnitziki, D., Ebersberger, B., & Fieri, A. (2007). The relationship between R&D collaboration, subsidies,and R&D performance: Evidence from Germany and Finland. Journal of Applied Econometrics, 22(7),1347–1366.

Czarnitzki, D., Hanel, P., & Rosa, J. (2011). Evaluating the impact of R&D tax credits on innovation: Amicro-econometric study on Canadian firms. Research Policy, 40(2), 217–229.

David, P., Hall, B., & Toole, A. (2000). Is public R&D a complement or a substitute for private R&D? Areview of the econometric evidence. Research Policy, 29(4–5), 497–529.

Deeds, D., & Decarolis, M. (1999). The impact of stocks and flows of organizational knowledge on firmperformance: An empirical investigation of the biotechnology industry. Strategic Management Jour-nal, 20(10), 953–968.

The longer term effects of federal subsidies on firm survival…

123

Dublin Institute of Technology. (2017). Topics and themes: T2S2015. Retrieved from http://www.t2s2015.com/page/9/topics-and-themes/.

Dyer, J. (1997). Effective interfirm collaboration: How firms minimize transaction costs and maximizetransaction value. Strategic Management Journal, 18(7), 535–556.

Dyer, J., & Singh, H. (1998). The relational view: Cooperative strategy and sources of interorganizationalcompetitive advantage. Academy of Management Review, 23(4), 660–679.

Einio, E. (2014). R&D subsidies and company performance: Evidence from the geographic variation ingovernment funding based on the ERDF population-density rule. The Review of Economics andStatistics, 96(4), 710–728.

Esteve-Perez, S., & Manez-Castillejo, J. (2008). The resource-based theory of the firm and firm survival.Small Business Economics, 30(3), 231–249.

Feldman, M. (1994). The university and economic development: The case of Johns Hopkins University andBaltimore. Economic Development Quarterly, 8(1), 67–76.

Feldman, M., & Kelley, M. (2003). Leveraging research and development: Assessing the impact of the U.S.Advanced Technology Program. Small Business Economics, 20(2), 153–165.

Feldman, M., & Kelley, M. (2006). The Ex Ante assessment of knowledge spillovers: Government R&Dpolicy, economic incentives, and private firm behavior. Research Policy, 35(10), 1509–1521.

Griffith, R., Redding, S., & Van Reenen, J. (2004). Mapping the two faces of R&D: Productivity growth in apanel of OECD countries. The Review of Economics and Statistics, 86(4), 883–895.

Hall, M., & Link, A. (2015). Technology-based state growth policies: The case of North Carolina’s greenbusiness fund. Annals of Regional Science, 54(2), 437–449.

Hall, B., Link, A., & Scott, J. (2003). Universities as research partners. The Review of Economics andStatistics, 85(2), 485–491.

Hall, B., & Maffioli, A. (2008). Evaluating the impact of technology development funds in emergingeconomies: Evidence from Latin America. The European Journal of Development Research, 20(2),172–198.

Hanel, P., & St Pierre, M. (2006). Industry-university collaboration by Canadian manufacturing firms. TheJournal of Technology Transfer, 31(4), 485–499.

Heckman, J. (1979). Sample selection bias as a specification error. Econometrica, 47(1), 153–161.Holmes, P., Hunt, A., & Stone, I. (2010). An analysis of new firm survival using a hazard function. Applied

Economics, 42(2), 185–195.Hussinger, K. (2008). R&D and subsidies at the firm level: An application of parametric and semiparametric

two-step selection models. Journal of Applied Econometrics, 23(6), 729–747.Hytinnen, A., & Toivanen, O. (2005). Do financial constraints hold back innovation and growth?: Evidence

on the role of public policy. Research Policy, 34(9), 1385–1403.Johnstone, N., Hascic, I., & Popp, D. (2010). Renewable energy policies and technological innovation:

Evidence based on patent counts. Environmental & Resource Economics, 45(1), 133–155.Klepper, S. (2002). Firm survival and the evolution of oligopoly. RAND Journal of Economics, 33(1),

37–61.Klette, T., Møen, J., & Griliches, Z. (2000). Do subsidies to commercial R&D reduce market failures?

Microeconometric Evaluation Studies. Research Policy, 29(4–5), 471–495.Knoben, J., & Oerlemans, L. (2006). Proximity and inter-organizational collaboration: A literature review.

International Journal of Management Reviews, 8(2), 71–89.Kumar, V., Kumar, U., & Persaud, A. (1999). Building technological capability through importing tech-

nology: The case of Indonesian manufacturing industry. The Journal of Technology Transfer, 24(1),81–96.

Lerner, J. (2000). The government as venture capitalist: The long-run impact of the SBIR program. TheJournal of Private Equity, 3(2), 55–78.

Leyden, D. (2016). Public sector entrepreneurship and the creation of a sustainable innovative economy.Small Business Economics, 46(4), 553–564.

Leyden, D., & Link, A. (2013). Knowledge spillovers, collective entrepreneurship, and economic growth:The role of universities. Small Business Economics, 41(4), 797–817.

Link, A. (2015). Capturing knowledge: Private gains and public gains from university research partnerships.Foundations and Trends in Entrepreneurship, 11(3), 139–206.

Link, A. (2016). Public sector entrepreneurship: Introduction to the special section. Economica e PoliticaIndustriale, 43(3), 355–356.

Link, A., & Rees, J. (1990). Firm size, university based research, and the returns to R&D. Small BusinessEconomics, 2(1), 25–31.

Link, A., & Scott, J. (2005). Universities as research partners in U.S. joint research ventures. ResearchPolicy, 34(3), 385–393.

D. Smith et al.

123

Loof, H., & Brostrom, A. (2008). Does knowledge diffusion between university and industry increaseinnovativeness? The Journal of Technology Transfer, 33(1), 73–90.

Mansfield, E. (1995). Academic research underlying industrial innovations: Sources, characteristics, andfinancing. The Review of Economics and Statistics, 77(1), 55–65.

Morgan, K. (1997). The learning region: Institutions, innovation, and regional renewal. Regional Studies,31(5), 479–490.

Morgan, S., & Winship, C. (2007). Counterfactuals and causal inference. New York, NY: CambridgeUniversity Press.

Murnane, R., & Willett, J. (2011). Methods matter: Improving causal inference in educational and socialscience research. New York, NY: Oxford University Press.

Musso, P., & Schiavo, S. (2008). The impact of financial constraints on firm survival and growth. Journal ofEvolutionary Economics, 18(2), 135–149.

National Institute of Health. (2015). What are SBIR and STTR programs?. Retrieved from https://sbir.nih.gov/.

National Institute of Standards and Technology. (1998). Advanced Technology Program proposal prepa-ration kit. Gaithersburg, MD: National Institute of Standards and Technology.

National Institute of Standards and Technology. (2004). ATP Proposal Preparation Kit. Retrieved fromhttp://www.atp.nist.gov/atp/kit-04/final-kit.pdf.

National Institute of Standards and Technology. (2005). Survey of ATP Applicants 2000: Introduction.Retrieved from http://www.atp.nist.gov/eao/applicants/intro.htm.

National Institute of Standards and Technology. (2007). America COMPETES Act Brings ImmediateChanges to NIST. Retrieved from https://www.nist.gov/news-events/news/2007/08/america-competes-act-brings-immediate-changes-nist.

National Institute of Standards and Technology. (n.d.). Technology Innovation Program. Retrieved fromhttps://www.nist.gov/technology-innovation-program.

National Institutes of Health. (2016). What is SBIR and STTR?. Retrieved from https://sbir.nih.gov/about/what-is-sbir-sttr.

National Research Council. (2001). The Advanced Technology Program: Assessing outcomes. Washington,D.C.: National Academy Press.

Nishimura, J., & Okamuro, H. (2010). R&D productivity and the organization of cluster policy: Anempirical evaluation of the industrial cluster project in Japan. The Journal of Technology Transfer,36(2), 117–144.

Oughton, C., Landabaso, M., & Morgan, K. (2002). The regional innovation paradox: Innovation policy andindustrial policy. The Journal of Technology Transfer, 27(1), 97–110.

Puhani, P. (2000). The Heckman correction for sample selection and its critique. Journal of EconomicSurveys, 14(1), 53–68.

Richardson, A., Audretsch, D., & Aldridge, T. (2016). Motivating entrepreneurship and innovative activity:Analyzing U.S. policies and programs. In D. Audretsch & A. Link (Eds.), Essays in Public SectorEntrepreneurship. Berlin: Springer.

Sachwald, F. (2008). Location choices within global innovation networks: The case of Europe. The Journalof Technology Transfer, 33(4), 364–378.

Sampson, R. (2007). R&D alliances and firm performance: The impact of technological diversity andalliance organization on innovation. Academy of Management Review, 50(2), 364–386.

Saxenian, A. (1994). Regional advantage: Culture and competition in Silicon Valley and Route 128.Cambridge, MA: Harvard University Press.

Schrank, A., & Whitford, J. (2011). The anatomy of network failure. Sociological Theory, 29(3), 151–177.Shadish, W., Cook, T., & Campbell, D. (2002). Experimental and quasi-experimental designs for gener-

alized causal inference. Belmont, CA: Wadsworth.Stiglitz, J., & Wallsten, S. (1999). Public-private technology partnerships: Promises and pitfalls. American

Behavioral Scientist, 43(1), 52–73.Stuart, T. (2000). Interorganizational alliances and the performance of firms: A study of growth and

innovation rates in a high-technology industry. Strategic Management Journal, 21(8), 791–811.Takalmo, T., & Tanayama, T. (2010). Adverse selection and financing of innovation: Is there a need for

R&D subsidies? The Journal of Technology Transfer, 35(1), 16–41.U.S. Department of Energy. (2016). 2016 ARPA-E Energy Innovation Summit. Retrieved from https://arpa-

e.energy.gov/?q=events/2016-arpa-e-energy-innovation-summit.U.S. National Institute of Standards and Technology. (n.d.). ATP fiscal year 1998 competition results.

Retrieved from http://www.atp.nist.gov/comps/index98.htm.U.S. Department of Energy. ta (n.d.). ARPA-E: General questions. Retrieved from https://arpa-e.energy.gov/

?q=faq/general-questions.

The longer term effects of federal subsidies on firm survival…

123

U.S. Small Business Administration. (n.d.). About SBIR. Retrieved from https://www.sbir.gov/about/about-sbir.

U.S. Small Business Administration. (2014). Summary of size standards by industry sector. Retrieved fromhttps://www.sba.gov/content/summary-size-standards-industry-sector.

Wagner, J. (2002). The causal effects of exports on firm size and labor productivity: First evidence from amatching approach. Economics Letters, 77(2), 287–292.

Wallsten, S. (2000). The effects of government-industry R&D programs on private R&D: The case of thesmall business innovation research program. RAND Journal of Economics, 31(1), 82–100.

Welsh, W. (2014). GAO pushes better reporting of federal grants data. Information Week.

D. Smith et al.

123