DIRECT AND INDIRECT EFFECTS OF S&T

POLICIES: QUESTIONS AND CHALLENGES FOR

EVALUATION

Coloquio sobre evaluación de políticas de CTI, 19 de junio 2014

Camino Real Pedregal

Chiara Criscuolo

Structural Policy Division

Science, Technology and Industry Directorate, OECD

The rationale for public support for R&D

Public support to R&D addresses the less-than-socially-optimal private investment stemming from projects with large expected social benefits but inadequate expected private returns (Arrow, 1962).

Government support to business R&D aims to address market failures, and in particular to:

• Align private and social return, and mitigate the “incomplete private appropriability” characterising the production of scientific and technical knowledge (Nelson, 1959; Arrow, 1962);

• Correct for information asymmetries, characterising R&D and innovation activities, which may lead to difficulties in finding external finance.

Supporting R&D: policy tools

Two broad R&D policy instruments exist:

1) Direct procurement and public funding of knowledge-based innovative activities carried out by public entities (universities, research centres, etc.);

2) Public support to commercial R&D, i.e. incentives for a greater amount of private investment, in the form of:

Tax credits: based on firm-level R&D activity. Market based tool.

Directly reduce marginal cost of R&D activities.

Allow private firms to choose projects.

Direct R&D subsidies: project-specific tools.

Allow public bodies to target projects with perceived high marginal social rates of return.

• R&D grants

– Directed at specific projects (government/government agency chooses)

– Often targeted to national needs

– Sometimes targeted to collaborative research (with universities; government; other firms)

– Certification effect ;

• Tax incentives for R&D

– Market based method, affect price of R&D

• Expenditure based

– Accelerated depreciation - usually 100%, i.e., expensing

– Allowances – amounts that can be deducted from income for tax purposes (>100%)

– Credits – amounts deducted from tax liability

– Payroll withholding tax credit for R&D wages

• Income based: tax KBC income at preferential rate

R&D grants vs fiscal incentives

• Close gap between private and social return to R&D by lowering its cost

• Can also be used to target specific areas (more generous):

– Basic research

– Small & medium-sized firms; young firms

– Collaborative research

– Energy

• Widely used in OECD countries (26 out of 34):

• Differences in the R&D support policy mix

More on R&D tax incentives

As a percentage of GDP, 2011

Source: OECD Science, Technology and Industry Scoreboard 2013.

http://dx.doi.org/10.1787/888932891112

Direct funding of business R&D and R&D

tax incentives, 2011

Correlation between total support – R&D intensity, but not necessarily causal

OECD estimates: an additional instrument to test impacts of support policies

Source: OECD Science, Technology and Industry Scoreboard 2013.

http://dx.doi.org/10.1787/888932890143

Business R&D intensity and government

support to business R&D, 2011

…and differences their design

• Differences in generosity; design; target of areas and firms; expenditure vs income based; tax credits Incremental vs. Volume base;

• R&D definition: not always based on OECD Frascati Manual definition

• Qualified R&D expenditure: e.g only wages related to R&D or total current R&D expenditures (wages + consumables); or include current and capital R&D expenditure

• Carry-over provisions and cash refunds:

• Generosity of the scheme, ceilings

• Temporary vs. permanent nature

• Presence of sub-national tax incentive programs

• Implementation: Complexity, administration and compliance costs,

uncertainty on covered expenses, etc

Differences in design

Design of the R&D tax incentive scheme

Volume base R&D tax credit Australia, Austria, Belgium (capital), Canada, Chile, Denmark, France, Norway.

Incremental R&D tax credit United States (mostly)*.

Hybrid system of a volume and an incremental credit Ireland, Italy, Japan, Korea, Portugal, Spain.

R&D tax allowance Belgium (capital), Brazil, China, Chile, Columbia, Czech Republic, Finland, Hungary, India, Netherlands, Russian Federation, Singapore, Slovenia, South Africa, Turkey, United Kingdom.

Payroll withholding tax credit for R&D wages Belgium, Hungary, Netherlands, Spain, Turkey.

More generous R&D tax incentives for SMEs Australia, Canada, France, Hungary, Japan, Korea, Netherlands, Norway, Portugal, United Kingdom.

Targeting

Special for energy United States (volume-based).

Special for collaboration Hungary, Italy, Japan, Norway.

Special for new claimants France.

Special for young firms and start-ups Belgium, France, Netherlands, Portugal.

Ceilings on amounts that can be claimed

Austria, Denmark, France, Ireland, Italy, Japan, Netherlands, Norway, Portugal, Singapore, Spain, United Kingdom, United States.

R&D Income-based R&D tax incentives Austria (individuals), Belgium, China, France, Hungary, Luxembourg, Netherlands, Spain, Switzerland, Turkey, United Kingdom.

Special treatment of technology acquisitions (capital cost)

Poland

No R&D tax incentives Estonia, Germany, Israel, Mexico (repealed), New Zealand (repealed), Sweden.

Table 2. Details of differences in R&D tax incentives schemes across selected countries (2013), source Andrews and Criscuolo, 2013

Source: OECD Science, Technology and Industry Scoreboard 2013.

http://dx.doi.org/10.1787/888932891150

Implied tax subsidy on R&D expenditures, 2013

Source: OECD Science, Technology and Industry Scoreboard 2013.

http://dx.doi.org/10.1787/888932891150

Implied tax subsidy on R&D expenditures, 2013

Evaluation of the effect of R&D support

Since Griliches (1958) and Blank & Stigler (1957), many investigated:

• Input additionality, i.e. the complementarity or substitutability of public and private R&D funds (especially additionality VS crowding out) (David et al,

2000 and Hall and Van Reenen, 2000 for a survey);

• Output additionality, (Klette et al, 2000, for a survey; Moen et al., 2007, Bloom,

Schankerman and Van Reenen, 2013)

• Behavioural additionality

Not many have investigated indirect effects of R&D fiscal incentives on:

• Scientists wages (see Goolsbee, 1998 and Lokshin and Mohnen, 2008)

• Location of R&D activities (Bloom and Griffith, 2001 and Wilson, 2008)

• R&D start-up decisions (Norway Skattefunne programme evaluation)

• Resource Reallocation (Acemoglu et al., 2013)

• Efficiency of allocation in terms of R&D type (Akcigit et al., 2014)

How to assess the effect of R&D support

• Input additionality; deadweight loss; intensive vs extensive margin

• Second-order effects: output additionality; patents; new products

• Third-order effects: productivity growth; survival; employment growth; location of mobile investment

• Full Cost benefit analysis: Spillovers; Administration costs; Compliance costs; Opportunity costs

• General equilibrium analysis: Wage effects; reallocation of resources

Effect of R&D fiscal incentives: input

additionality

•Does the fiscal incentive increase the total amount of R&D investment in the economy, or it crowds out private R&D investment? (e.g., additionality vs. crowding out). •Two metrics:

•R&D price elasticity: Measures the percentage change in R&D investment resulting from tax relief for every percentage change in its after-tax price (also called the user cost of R&D) •Incrementality ratio (bang for the buck): Measures the change in R&D investment per dollar of foregone tax revenue that is spent on R&D fiscal incentives.

• Long-run elasticity of R&D with respect to price is around 1 (but wide range.

• R&D elasticity higher in the long than in the short run (probably due to the adjustment cost associated with R&D investments)

• Smaller firms more responsive to R&D tax incentives than larger firms, consistent with small firms being more credit constrained , since they are less likely to have collateral.

• The incrementality ratio affected by policy design with estimates for incremental R&D tax credits generally above 1, and below 1 for volume-based R&D tax credits.

• Importance of stability (Guellec and van Pottelsberghe de la Potterie, 2003 and Westmore, 2013)

Evidence on input additionality

• If problem is on number of firms performing R&D rather than how much R&D they perform

• new R&D performers to cover sunk entry costs

• effects might be long-lasting because of persistence

• moral hazard problem (start and stop or overinvestment

• Effective? Scarce evidence but all in the same direction (Corchuelo, 2009; Haegeland and Moen, 2007; Arqué-Castells

and Mohnen, 2012)

• Young startups: refundability might be key (Busom et al.,

2012)

Support to new R&D performers

• Not all firms apply for R&D tax credits because of compliance costs and opportunity costs of applying; monitoring risks; knowledge/information (Corchuelo and Martinez-Ros, 2008)

• Firms with higher human and financial capital; a stable financial position and that have been supported before are more likely to apply

• SMEs and in particular startups incur additional obstacles in applying for R&D tax Credits (Busom et al., 2012)

Compliance and opportunity costs

Effect of R&D tax incentives: output

additionality

• R&D investment only an input: R&D fiscal incentives could increase amount of reported R&D without increase in innovative outputs and Productivity growth

– Relabeling: relatively small in the long term (e.g., Hall 1995).

– Scientists wages: in the short term higher wages of scientists and engineers, if supply inelastic, lowering the increase in the volume of R&D performed (even if in the long term it induces an increase in the supply of qualified workers).

– Lower marginal private returns: If there are decreasing marginal returns, the additional R&D induced by an R&D tax incentive might be less productive (evidence on Skattefunn programme).

– Lower marginal social returns (Bloom et al., 2013)

– More development than research (Akcigit et al., 2014)

Evidence on output additionality

• Thus far, the evidence remains scattered.

• Evaluation of these impacts are difficult:

– imperfect measure of innovation output – e.g. patents and available measures of product and process innovations –

– variable time lags between R&D investments in various types of R&D (research versus development, projects, technology areas, etc.) and the resulting innovation output.

• The available evidence suggests a positive effect of R&D tax incentives on innovative sales or the number of new products (e.g. Czarnitzki, Hanel

and Rosa, 2005; de Jong and Verhoeven, 2007).

• However, innovations brought about by R&D tax incentives might not be the same as those funded privately by the firm or by government grants(Cappelen, Raknerud and Rybalka, 2008).

• The assessment of the impact of R&D tax incentives on innovation outcomes is difficult (spillovers also beyond frontiers)

Evidence on impact on productivity growth

and welfare

• The evidence on the effectiveness of R&D tax incentives on productivity growth is scarce and mixed

• Some positive correlation between R&D tax incentives and productivity (Brouwer et al., 2005 and Lokshin and Mohnen, 2007).

• Estimates of the effectiveness of R&D fiscal incentives on welfare require a full cost-benefit analysis (i.e. consider full direct

and indirect effects of the policy, implementation and compliance costs, and impact of distortionary taxes needed to finance the incentives).

• some studies have attempted to provide such estimates and indicate positive net welfare gain from R&D tax incentives, they depend heavily on the assumptions made (Russo, 2004;Parsons and

Phillips, 2007 and Lokshin and Mohnen, 2009).

Role of R&D tax incentives for productivity

and employment growth and reallocation

• If the firms that benefit most from R&D fiscal incentives are actually those for which R&D is less likely to generate large spillovers because they focus on niche markets, R&D fiscal incentives might not be associated with increases in aggregate productivity growth (Bloom et al., 2010).

• R&D tax credit might reduce growth and welfare if it subsidizes the R&D of incumbents, unless the R&D of new entrants (new R&D performers) is also subsidized and exit of inefficient incumbent is allowed (Acemoglu et al., 2014)

• R&D tax incentives have the unintended consequence of protecting incumbents at the detriment of potential entrants, thus slowing down the reallocation process (Bravo-Biosca , Criscuolo and Menon, 2013).

• Fiscal incentives for R&D aim at increasing the volume of R&D investment, but part of these incentives might lead to an increase in the wages of - or the cost to firms of hiring - R&D scientists and engineers. (Gooslbee, 1999)

• Because of inelastic supply of scientists and/or search costs for

scientists and engineers, bargaining power between firms and R&D workers

• Studies that have looked at this issue remain scarce and are strongly constrained by the availability of suitable data (aggregate vs micro; firm vs individual level).

• The available studies tend to find: – increase in R&D wages does not correspond to a change in quality of

researchers, (e.g.Hægeland and Moen, 2007 and Lokshin and Mohnen, 2008).

Effects of R&D tax credits on wages

Effects of R&D tax incentives: attract

internationally mobile R&D investment

– R&D activity increasingly mobile and internationalized.

– Multinational firms account for a substantial fraction of R&D expenditure

– even if R&D support might affect location of MNEs R&D investment, other key fundamentals are more important.

• for basic research: access to local science and technology, proximity to university frontier research, availability of a skilled workforce, scientists and strong IPRs. (Thursby and Thursby, 2006)

• for development: access to local markets and proximity to other corporate activities, such as production sites, and proximity to local customers (Defever, 2006)

– Distortion from affecting location fo R&D activity (Griffith et al., 2004)

24

• Bloom and Griffith (2001)

– Use country-level data over 19 years to show that R&D in one country responds to changes in tax price of R&D in other “competitor” countries.

– Relate R&D expenditure to the domestic user cost of R&D and foreign user cost of R&D.

– R&D increases with a decrease in the domestic user cost and with an increase in the foreign user cost

• Wilson (2009)

– Very similar exercise using data on US states over time

– Nearer states are considered closer competitors

– Evidence of a zero-sum game among US states competing for mobile R&D

– Estimates long-run elasticity of in-state R&D with respect to in-state user cost of -2.5 and long-run elasticity of in-state R&D with respect to out-of-state user cost of 2.5

Findings: effects of R&D tax credits on location

• difficulties in measuring effective tax rates on R&D, details of design, data availability, and estimation problems:

– Self-selection

– endogeneity

– time lags

– indirect effects on firms that did not apply/receive the fiscal incentives.

– Heterogeneous effects

– General equilibrium effects

Challenges in evaluating impact of R&D

fiscal incentives

• Finding a counterfactual.

• Policy makers and econometricians cannot observe what firms that received support would have invested had they not received the support.

• Firms that get into the program are not random draws

• Additional problem: impact on non-treated-firms (difficult to find a control group amongst non-treated firms)

Identification and selection: the problem

• The question of interest:

– What would the performance of firms that participated to the programme if they had not participated?

– Problem: we cannot observe the outcome of not participating for treated firm

• Solution: we need to find a valid counterfactual: what would have happened “but for” the policy

What do we want to estimate?

Solutions to evaluation concerns

Main microeconometric programme evaluation approaches used:

1) Matching of treated and untreated: requires no assumption on functional forms, but strongly relies on the “conditional independence” assumption and on the existence of a common support.

2) Difference-in-Difference

3) Instrumental variables: account for selection on unobservables, but require instrumental variables, i.e. variables correlated with the likelihood / intensity of treatment, but not with individual firm performance.

4) Regression discontinuity design ( SKATTEFUNN evaluation)

5) (Parametric and semiparametric) sample selection models: account for selection on observables and unobservables. More parametric than IV. Require instrumental variables when applied.

6) (General equilibrium) structural models: economic theory guides the construction of the model and suggests included and excluded variables, but functional form, exogeneity assumptions invoked, and sources of parameter identification may be controversial.

• Structural econometric models estimating firm R&D investment demand function useful when trying to do ex-ante policy evaluation:

– Characterize firm R&D investment behaviour

– Support policy scenarios and simulations

• Mairesse and Mulkay, 2013 on the 2008 French Reform

– estimate Error Correction specification of dynamic R&D investment demand regression with SYS-GMM

• Assumptions: 1)consistent estimates of structural parameters

2)coeffs apply after the 2008 reform

• need to complement with ex-post assessment. Also to look at impact on extensive margin

Structural econometric approach and

ex-ante policy evaluation

• Impact of R&D tax credit on R&D expenditure; but not only...think of general equilibrium effects and unintended consequences. Design is key.

• Ex-post assessment of impact difficult but needed:

– Access to data (administrative and design)

• Complemented with ex-ante evaluation based on structural model when data non available

• Importance of including an evaluation phase in the design of the policy (data collection; outcomes of interest; lags; randomization?)

• Importance of making evaluation results open and accessible and use it for future policy design

Summary

• Important of design and level playing field:

– Refundability; carry forward/backward rules; payroll withholding tax credits for R&D wages

– …but size contingent policies problems

• Importance of:

– innovation policy mix (Supply push policies; innovation demand policies; etc.)

• framework conditions:

– Competition; openness; for experimentation: EPL; bankruptcy; access to finance; entrepreneurship policies etc.

• predictability and stability beyond political mandates

Policy discussion