Evaluation of the EU Support to Research and Innovation in

Partner Countries (2007-2013)

EVA 2011/ LOT1 Contract n°2013/330982

Dissemination Seminar Brussels, 26 May 2016

Evaluation – purpose and methodology

Objectives – evaluate EU support to

R&I for development• Extent of

strategic approach adopted

• Enhance capacity to reach development objectives

Specific interest in: • Capacity building• Transfer of results• Use of

instruments and modalities

• Approaches used: country/regional, R&I integrated in sectors

Questions

EQ1: Development policy objectives

EQ2: Impact on partner country research communities

EQ3: Instruments and modalities

EQ4: DEVCO-RTD complementarity and coherence

EQ5: Transfer of R&I results into development processes

EQ6: EU capacities

Evaluation questions & themes

Four themes to cover1. Food Security, Nutrition and

Agriculture (FSNA)2. Health3. Environment and Climate

Change (EnvCC)4. Science, Information Society

and Space (SISS)

Three instrumentsa. The European Development

Fund (EDF)b. The Development Cooperation

Instrument (DCI)c. The European Neighbourhood

& Partnership Instrument (ENPI)

Ten countries visited:d. EDF: Burkina Faso, Ethiopia,

Kenya, Mauritiuse. DCI: Peru, South Africa,

Vietnam, India f. ENPI: Tunisia, Ukraine

Intervention logic and evaluation questionsOutputs Intermediate impactsSpecific impactsExpected Results Global

impacts

Improved research and innovation capacity in developing countries• More trained researchers• Higher levels of research skills• Capacities to participate

actively in international research networks and projects

Improved policy and regulatory framework for research and innovation• In developing countries• Internationally

Improved infrastructure for research and knowledge exchange

Specific research and innovation in particular fields conducted in developing countries

More transparent and open knowledge economies in developing countries

Global challenges increasingly resolved

Enhanced development processes and outcomes

More sustainable developm

ent, eradication of poverty and achievement of M

DG

s

More innovative solutions to development problems and global challenges

Reduced poverty and inequality

More EU funded research conducted in developing countries

More internationally integrated and transparent research and innovation processes

Research and innovation in developing countries and adjusted to their needs

Research and innovation conducted in developing countries increasingly oriented to their needs and development processes

Conducive policy and regulatory framework for research and innovation

Inputs/Activities

Policy dialogue:

Funding to support dialogue processes

Encourage EU MS and other partners to harmonise approaches

Policy support and knowledge sharing

Technical & financial cooperation:

Individual capacity development: • Mobility• Skills training• Masters & PhD programmes

Fund research for development:Fund pro-poor, demand-led research for development at global, regional and country level in specific areas, fieldsand sectors (DEVCO programme contributions, core and project funding and RTD/DEVCO calls)

Provide expertise on regulatory frameworks to support R&I

FP7 funding & R&I calls

Policy makers more attuned to using and dispersing development results

DG RTD

Institutional capacity & infrastructure development:• Transferring technology• Physical connectivity• Institutional strengthening• Knowledge management• Multi stakeholder participation• Research/ innovation networking• Improving policy and regulatory

frameworks, strategies, approaches, methodologies

Harmonised EU approaches to donor support for research and innovation in developing countries

Developing countries better placed to participate in international research and innovation projects and tenders

Increased mainstreaming of research and innovation results into development processes

EQ2

EQ5

EQ3

Increased coherence of EU policy on R&I with EU development policy

EQ4

Promote innovation, societal uptake and use of research results:• Foster research application and

implementation of innovative approaches and practices

• Support demand-driven development and testing of innovative solutions

• Stimulate and capture innovative, locally-owned sustainable solutions

RTD policy dialogues

More effective mobilisation of European expertise for addressing global challenges

EQ6

EQ1

A generic logic that also works for sectoral approaches

Inventory

• Scope: 2007-2013 / EDF+DCI+ENP (not RTD)• Four themes/sectors: FSNA, EnvCC, Health, SISS • Total contracted amount of EUR 1.14 billion in 2007-2013 (plus

0.28 billion in mobility grants) • Almost half of the total (EUR 511 million) went to FSNA• Contracts at global, regional and national levels

Fig. 4: Sector allocation of commitments (shares of total contracted amount)

Fig. 5: Geographic scope of commitments as determined by benefitting zone

Inventory Regional distribution:

Types of contractors:

Fig. 6: Shares of total commitments (regional plus individual country contracts)

per region

Fig. 9: Distribution of commitments by type of contractor (shares of contracted

amounts)

Fig. 10: Distribution of contracted amounts by sector and type of contractor

Fig. 7: Values of total commitments (regional plus individual country contracts) per region

Evaluation questions

Policy framework

Impact on partner countries

Instruments & Modalities

To what extent has EU support to R&I through DEVCO been successful in promoting the overall development policy objectives of the EU?

To what extent has DEVCO funding of R&I enabled research communities in partner countries to build up and develop their own R&I capacity, including the ability to actively engage in research networks (regional and international)?

To what extent has DG DEVCO in its support to R&I used its available instruments in a way that maximizes their value?

Evaluation questions

DEVCO-RTD complemen-tarity & coherence

Transfer of results

EU

capacities

To what extent has EU support to R&I by DG DEVCO and by DG RTD been complementary and their collaboration promoted PCD?

To what extent has DEVCO support led to the trans-fer of R&I results into processes likely to impact on the achievement of EU development objectives ?

To what extent have the EU external relations services ensured adequate capacities to conduct policy dialogue related to R&I and to support research and innovation in partner countries?

Conclusions and recommendations

Conclusions cluster – Policy & strategic focus

Sectoral approach broadly effective

Possible pillars for a structured strategy

Role of R&I4D hidden &

poorly understood

The sectoral approach has been broadly effective within each sector and the JAES. But, it limits the ability to have a major impact on R&I as a tool to foster economic transformation in the face of rapid scientific and technological change.

The lack of a clear overall strategy (beyond sector strategies) for DEVCO support to R&I means the valuable role this support plays in achieving EU development objectives is poorly understood and not recognised.

Despite the lack of an overall strategy, many of the individual projects with R&I components showed potential as examples of good practice on which to build a wider structured strategy that can also be adapted to the circumstances of countries at different stages of development.

Conclusions cluster – Policy & strategic focus

Partnerships at

different geographic levels are effective

Well-chosen partnerships at different geographical levels have allowed DG DEVCO to support different types of effective R&I initiatives.

Conclusions cluster – Operational approach

Need to supplement

main elements of

support

Use of instruments

& modalities

not strategic

Inadequate capacity

The basic logic of DG DEVCO’s support to R&I is similar from one sector to another and is generally solid, but it has not been deployed in a consistent fashion and could be further developed.

The use of instruments and modalities has shown little strategic thought and created issues regarding flexibility and matching funds with the longer cycles required for research and for innovations to achieve impact.

DG DEVCO capacity dedicated to R&I has been inadequate, particularly in EU Delegations.

Conclusions cluster - Complementarity

Division of labour

could be stronger

Policy coherence has been

well understood

Greater impact could be achieved by a stronger and more consistent effort by DGs DEVCO and RTD to work together. While there is a high-level understanding on a division of labour between the two DGs, in practice it is only loosely respected.

Policy coherence for development is clearly understood as an important concern in both DGs DEVCO and RTD. Efforts have been made to promote PCD by different EU actors involved in support to R&I.

Conclusions cluster – Results

Results achieved

but largely ad-hoc & localised

Societal uptake has

been limited

R&I efforts supported by DG DEVCO have contributed to development outcomes but largely in an ad-hoc and localised manner that did not promote systematic and sustainable progress on creating conducive conditions for R&I.

Innovation and societal uptake of R&I results from DG DEVCO support have been scarce due to inadequate investment in national institutional frameworks for innovation.

Capitalisation of

results has been

inadequate

R&I results have not been capitalised on and hence their potential impact has not reached beyond the specific programmes or immediate researcher networks involved.

DEVCO is not seen as an agent of

R&I4D

DG DEVCO is not perceived as an agent for R&I for development, and few efforts have been made to create such an image for improved visibility.

JAES (Text Box 5)

Regional partner

A platform used to• Conduct policy

dialogue on African priorities and joint initiatives

• To jointly fund research

• To build capacity to manage R&I

• To provide access to European S&T

Main findings

• Joint Africa-EU Strategy – 8th Partnership on Science, Information Society & Space – recognises S&T as essential element engines of socio-economic growth and development

• Policy dialogue on R&I – support to STISA: Science, Technology & Innovation Strategy for Africa

• AU Research Grants – 20 grants of ~EUR 750,000 to Africa-Europe consortia of ROs in

• Capacity Building for AUC S&T Department – to set up and administer system of research grants

• MESA – Monitoring for Environment and Security in Africa – project to make satellite and remote sensing data available to government, commercial and academic users across Africa

The JAES is both a platform for R&I policy dialogue and a framework for implementing regional initiatives with the AU.

SWITCH Asia (Text Boxes 11+14)

Regional level

DEVCO programme to fund innovation in the environmental sector for sustainable growth• Encourage SCP

practices and technologies

• Focus on SMEs• Funds series of

local projects across Asia

• Up to end 2013, 86 projects funded

Main findings

• MEET-BS – promotes sustainable production by developing markets for affordable water and energy products

• SPIN VCL – 11% of assisted projects successful in capturing markets for their products

Despite successes both projects met external and internal obstacles• Lack of strong price signals in energy and

resource prices to adequately reward improved efficiency

• Lack of national standards to support SCP - bottom-up project inadequately supported by top-down measures

• 3-4 year timescales of project funding is too short for fundamental business transformation

SWITCH Asia – key programme to fund R&I in Environment sector largely with private sector.

PASRI Tunisia (Text Box 12)

National level

Programme d’appui au système de recherche et de l’innovation (PASRI)• Implemented by

Government (ANPR) and GIZ

• Works closely with research community, private sector and various GoT ministries

• 3 components• Governance• Interfacing• Networking

Main findings

• Focus on National Innovation System (NIS)Two types of activities• Structural activities – diagnostic studies,

facilitation, and stimulating creation of networks

• Priming activities – pilot projects to catalyse collaboration between NIS actors: MOBIDOCs

Serious attempt to address R&I continuum in challenging circumstances post-Revolution• Space for rethinking and making changes yet

also lack of stability• Focus on innovation, stimulating private

sector, encouraging research community to address industry needs – potentially major impact

• Sustainability not assured at time of visit (Nov 2015)

Highly relevant project with wholistic approach to NIS

Recommendations

Recommendations – Policy & strategic focus

Formulate a strategic approach with a focus on institutional frameworks for R&I

1

2

Implementation

At national level, develop a strategy for R&I that adapts support provided to the needs and level of development of partner countries

• Clear statement and better implement division of labour with RTD

• Strategic focus on assisting partner governments to formulate national S&T policies and establish innovation systems

• Capacity building of ROs

Past experience provides a basis for a rough typology of 3 stages of different types of support

1. Laying the foundations primarily through sectoral funding – with poorer countries

2. Building up the infrastructure / developing a national S&T policy – with somewhat more developed countries

3. Supporting international cooperation in R&I – DGs RTD and DEVCO work jointly – graduating countries

(Details in Table 7)

Recommendations – Operational approach

• Review use of instruments and modalities in terms of impact they have on research

• Analyse difficulties partners have with modalities

• Use modalities that suit • Longer time frames of research• Capacities of partners

3

4

Implementation

DG DEVCO support should focus on seven principal elements

Use instruments & modalities that suit the needs of R&I

1. Networks – to support research & knowledge flows

2. Capacity – individual, institutional, infrastructural3. Partners – at all 3 levels: global, regional &

national4. Policy dialogue – ownership + coordination with

RTD5. Funding of research – important till alternatives

exist6. Capitalisation of results – so far inadequate

attention7. National Innovation Systems – as core of new

strategy

Recommendations - Complementarity

Consolidate and implement an explicit division of labour with DG RTD

6

Implementation

7Maintain the political & practical commitment to promoting PCD

• Ensure division of labour is clear and carried out at all levels

• DEVCO should review all RTD calls at design stage and consider development value

• Continue to use existing PCD tools• Deepened dialogue with partner countries to see

how to promote PCD • Review incentive structure in RTD for INCO work

• Staff to handle overall R&I policy not just each sector

• Adequate capacity to coordinate closely with RTD

• EUDs should have a designated R&I official• RTD should reevaluate plans to cut S&T

Counsellors

5Ensure adequate human resource capacities for support to R&I

Recommendations – Results

Take more deliberate and systematic steps to foster results

8

Develop a clear strategy for the transfer of results

9

Provide explicit support to the capitalisation of results

10

Implementation

Build a visibility strategy on a stated commitment to R&I support

11

• Clearer policy on support to R&I with greater focus will also help foster results

• Promote establishment of national innovation systems to provide more conducive context

• Ensure this strategy is taken up by EUDs and project implementers

• Promoting innovation for development should be principal objective for supporting R&I

• Set aside supplementary funds specifically for this• Consider establishing a system for the collection of

results so they can be retrieved and systematised• Individual capacity building should include practical

issues on innovation: IPR, raising venture capital, marketing, etc.

• Formulate a clear statement of intent • Liaise closely with EEAS and RTD to ensure respective

role are clear for communication• Develop a communication strategy that takes in HQ

and EUDs together

Evaluation of the EU Support to Research and Innovation in

Partner Countries (2007-2013)

Dissemination Seminar Brussels, 26 May 2016

Food Security, Nutrition and Agriculture

FSNA – intervention logic (Vol II, part A)

Intervention logic

R&I support for FSNA is geared towards improving the delivery of global, regional and national public goods from research and innovation.

Multi-stakeholder approaches to AR4D are mainstreamed in R&I support by the EU and its member states.

Main findings

(1)Public goods: Innovative, locally owned and sus-tainable solutions that can be shared, scaled up and mainstreamed to promote sustainable agricultural development of the poorest and most vulnerable.

(2)Three are distinct impact pathways:a) (Multi-donor) coordinated support to

international organisations delivering global and regional public goods to address global development challenges

b) Programme and project support to specific multi-stakeholder R&I projects, aiming directly at pro-poor, locally owned and sustainable solutions to specific problems in developing regions and countries

c) Support to multi-stakeholder research and innovation processes to improve national R&I capacity and competitiveness in (emerging) economies.

Working with global, regional as well as national partners is key to DEVCO approach to supporting R&I in FSNA.

FSNA – CGIAR (Text Box 8)

Global Partner

CGIAR, the Consortium of International Agricultural Research Centres, runs 15 global CG Research Programmes (CRPs).

Main findings (1)

The EU’s coordinated influence on the CGIAR reform has been significant:• With the help of GFAR, CGIAR strengthens consultations

and policy dialogues with national and (sub-)regional stakeholders.

• Multi-stakeholder partnerships, policy outreach and collaborating closely with national institutions, NGOs and farmer organisations are now central features of most CRPs.

CGIAR research has contributed to development outcomes:• The CRPs lay an important foundation for a sustained

delivery of global and regional public goods for sustainable agricultural development.

• Investing in CGIAR research produces ‘good value for money’.

The EU’s choice to directly fund CRPs has proven to be an effective way to strengthen the EU AR4D agenda.

CGIAR is a key global partner for EU support to R&I in FSNA.

FSNA – CGIAR (Text Box 8)

Global Partner

CGIAR (cont’d)

Main findings (2)

Documentation and sharing of results and approaches (capitalization) is not as yet standard practice.EU support to CGIAR flows through multiple channels: Challenging the effective implementation of CRPs.Serious budgetary limitations affect R&I implementation:• Allowed budget lines not necessarily adapted to local

context • Funding periods generally (too) short for purpose• Continuity between funders and funding periods lacks.Involving and strengthening national research and innovation partners requires further improvement: • At present less than 20% of CRP funding reaches

national research and innovation collaborators.For CGIAR, and others, research uptake and up-scaling is strongly conditioned by national innovation systems.

FSNA – IssAndes (Text Box 10)

Regional level

IssAndes is a regional project implemented by the CIP (part of CGIAR) together with 21 regional partners in four countries in the Andean region.

Main findings

• IssAndes had a strong design, linking agricultural development to nutrition and health.

• The impact pathway methodology integrated nutritional and food security objectives and was key to ensure that interventions had a positive impact on families’ health.

• The project built on a previous regional research and innovation project (Papa Andina) and the networks that were created, which has contributed significantly to its success

• The project strengthened capacity of partners and achieved policy impact on local, national and regional level.

But…• The limited scope to support the different

phases of such a complex research and innovation project have created limitations to its success.IssAndes has shown the impact of a strong design,

impact pathway, investing in partnerships and communication strategy.

FSNA – ASARECA (Text Box 6)

Regional partner

ASARECA is a sub-regional organisation of FARA. It aims to strengthen the NARS of its East and Central African member countries and link them regionally.

Main findings

• Through its activities ASARECA promotes and facilitates regional collaborative research and strengthens the network capacities of research communities.

• ASARECA has been effective in coordinating multi-stakeholder research projects at the regional level with national, regional and international partners.

• The multi-donor trust fund through which ASARECA is funded has been effective at simplifying finance, but sustainability is under pressure due to lack of core funds and lack of capacity to mobilise member country support.

ASARECA is a good example of support to a regional network.

FSNA - Ethiopian coffee sector (Text Box 15)

National level

The EU has supported R&I work in the government's Coffee Improvement Programme (CIP) for over 30 years.

The research components were implemented by the Ethiopian Institute of Agricultural Research (EIAR).

Main findings

• The EU support has had a major impact on the success of the CIP programme.

• R&I and uptake of research results by end-users has been central to the EU approach. Results include the development of 11 new varieties with increased resistance to pests.

• However, due to bureaucracy and institutional hurdles, Ethiopia’s absorption capacity of EU support was significantly reduced (50% of the total EUR 15 million budget).

• Government reforms led to EU resuming support

• The continuity of the EU’s commitment and its tailored approach is highly appreciated by stakeholders. R&I is at the core of EC support to the Coffee Sector in

Ethiopia

FSNA - innovation system

Innovation systems (Text Box 10)

National research and innovation systemsInnovation systems provide spaces where multiple stakeholders, from policy and research institutions, from businesses, advisory and user organisations, meet and interact to specify, pilot and put to use innovations they consider helpful in advancing their objectives.

Main findings

Technological innovation does not come alone. Social innovation is generally needed to turn it into a success: • i.e. policy, regulatory, relational and organisational

changes by government, private actors and communities

The ‘innovation pathway’ from research to societal transformation is winding and littered with obstacles and contextual determinants, particularly in DC. The science-driven ‘proposition-validation-adaptation-scaling’ pathway is only one of the possible innovation pathways for development. Others include:• market or migration-driven lateral transfer of ideas,

technologies or practices.• the recuperation of propositions from traditional

knowledge and practices and their validation and adaptation.

Innovation is where science and society meet…

Thank you for your attention

A strategy adapted to varying needs (Table 7)