Evaluation of Cluster Evaluation of Cluster Development Development Programs: What Do We Programs: What Do We Know?Know?

Carlo Pietrobellicarlop@iadb.org

Changing Paradigm of Cluster Development:

Learning from Global Experiences

New Delhi, 20th – 22nd February, 2014

Content of typical IDB-supported cluster program

Objective: Promote the competitiveness of firms in clusters by providing sector-specific public/club goods

Activities:•Cluster mapping and prioritization (often underlying objective is to reduce inter-regional development gaps by fostering local economic development)

•Strengthen local governance (actors’ awareness and mobilization)

•Joint participatory design of Plans to Improve Competitiveness (diagnostics, business identification, SWOT, benchmarking, action plan)

•Action Plan may include: Training and capacity building to promote inter-firm cooperation, Creation of public or club goods, other initiatives consistent with strategy e.g. TA, R&D, export promotion, marketing, logistics, etc.

The IDB support to clusters in LAC since 2000s has been remarkable

WARNING: Many other programs have been developed by each country, notably in Brazil, where clusters have represented a common approach to local industrial policy for over a decade now.

Organization No. of clusters

US$ Main characteristics

IDB 180 300 M. (180 M. from IDB)

18 Competitiveness support loans, with Governments (also at sub-national level). Each invests in approx. 10 clusters

IDB 30 70 M. approx. Other more cluster-specific loans (DR, Gua, Hon, Panama, Haiti, Ecua)

Multilateral Investment Fund (MIF)

72+40 US$120 M. approx.

Since 2007, mainly grants to private sector entities. First clusters only (72), then LED (40).

Total IDB Group 322 490 M. approx. Since early 2000, always with variable local co-financing shares

Following Questions immediately arise:

1. Did the program have an impact on the productivity of firms in the cluster? What is the size of the impact?

» Important: We are talking about causality (attribution to the program)

2. What was the process of having an impact?

3. Did networks and better coordination have an effect?

4. Consider the costs and net benefits of the intervention

5. Did the program have an impact on the productivity of the firms-cluster-region? (general equilibrium problem)

Why evaluations

Solid evaluations crucial because:

resources are public and limited, their funding needs to be justified

recipients need to be accountable

most importantly, we need to learn how to improve program effectiveness and contribute to improving livelihoods through enterprise and socio-economic development.

… but evaluations of CDPs are difficult

CDP are complex, they are mainly indirect and facilitative, system-oriented. They do not consist of a standardized “treatment”, but usually include a menu of specific interventions.

Effects require long periods of time to fully manifest (3-5 years) This implies strategic and operational problems.

Effects on linkages require relational data. Inter-firm linkages are an outcome of interest in itself and information about them is hard to collect, requiring specific techniques of analysis (SNA).

Spillover effects need to be measured. Intra-cluster spillovers make difficult to construct appropriate counterfactuals. Spillover process needs to be modeled. Labor mobility and geographical proximity have been used to measure spillovers.

7

The IDB is devoting substantial efforts to improve the evaluation of CDP - A ToolkitToolkit

Econometric work on firm-level performance with Census data, control groups and Propensity Score Matching (PSM)

Analysis of linkages, networks, and their influence on firms and clusters performance through Social Network Analysis

Case studies to explore processes of change and policy effects

Pietrobelli, Maffioli, Stucchi (Eds.), 2014, The Evaluation of Cluster Development Programs, Washington DC: IDB.

Giuliani E., Maffioli A., Pacheco M., Pietrobelli C., Stucchi R., 2013, “Evaluating the Impact of Cluster Development Programs”, IDB TN-551 July. http://www.iadb.org/document.cfm?id=37925857&pubDetail=1&wt_docType=Technical Notes&wt_docnum=37925857&wt_language=en&wt_department=IFD/CTI&lang=en

Some resultsSome resultsIDB Cluster programs in Minas Gerais and Sao Paulo (Brazil): Econometric evaluation with Propensity Score Matching (PSM, define a group fo firms with same probability) and Difference-in-Differences (DD, before-after and with-without). They combined analysis of Cluster firms data with Census data, building control groups with DD and PSM

oPositive direct and significant effects on:

o Employment: about 20% increase in 3-5 years;

o Probability to export: about +5% per year relative to the original proportion of exporting firms;

o Export levels: between 50% and 80% for each exporter for beneficiary firms. Effects persist and grow overtime.

oIndirect effects on firms localized in the area of influence of clusters, especially on the probability to export and less on export levels. In Pietrobelli, Maffioli, Stucchi (Eds.), 2014, Cluster programs Evaluation, Washington DC, BID.

Some resultsSome results

• Programas Asociativos en Chile – PROFOS (Rivas, 2010)

o For each US$ invested in these programs, additional US$ 2.4 were generated.

o Larger fiscal revenues: 3.2 pesos of additional VAT revenue for each peso invested in the program (in net present values)

o Better responses to crises. Between 1996 and 1999 (Asian crisis) non-beneficiary firms decreased their sales at 2.1% per year; those firms participating in PROFOS assessed in 1996 grew at 12.9% per year due to the changes in behaviour induced by the program (Departamento Economía de la Universidad de Chile,

2003).

SNA Methodologies Applied to CDP Evaluation: The Córdoba Electronics cluster in Argentina

• A CDP in electronics industry in Córdoba, Argentina, during 2003-2007, co-funded by MIF/IDB.

• Objectives: stronger local linkages and cooperation; easier local firms’ access to new production technology and organizational innovations; Better access to new markets.

• Information network (IN, to transfer business information), and the Collaboration network (CN, i.e. projects with joint collaboration).

• From 2005 to 2012 networks became less dense and more hierarchical. Central enterprises ensured network connectivity and created links between treated and untreated firms.

• Local firms plan their linkages and select those from which they believe they can obtain tangible benefits.

• CDP strengthened and created new technology-transfer ties between the electronics firms in Córdoba and other institutions.

Case studies also offer advantages• They allow the in-depth study of selected issues and cases.

• They offer an intuitive understanding of the results.

• They provide critical insights into beneficiaries’ perspectives.

• They may permit consistency checks and offer feedback to help interpret the findings of the quantitative analyses.

• Useful when quantitative models risk to remain a black box and miss the core process of cluster development.

• IDB financed many case studies (e.g. Argentina (Río Negro): Fruits-tourism-high tech, Brasil (São Paulo): Shoes and building material, Chile: Fresh fruit (grapes), avocado & higher education, Uruguay: Tourism

& blueberries). Forthcoming Casaburi and Pittaluga, 2014.

Microeconomic interventions to promote cluster development address a complex social phenomenon based on multi-actor coordination

This complexity demands the combination of multiple approaches for its proper evaluation

Recent impact evaluations using limited but rigorous econometric methodologies show positive and significant results

SNA promises to shed light on the complex workings of inter-firm networks

Case studies remain a crucial way to guide policy-makers and implementing agencies on the specifics of the how and what works better in different clusters in very diverse contexts

Conclusions