final report - unicef.org · idea international & idea-solutions . iii . of change. given...
TRANSCRIPT
FINAL REPORT
MAPPING EXERCISE ON UNICEF SUPPORT TO
NATIONAL EVALUATION CAPACITY DEVELOPMENT
(NECD)
Submitted to
UNICEF EVALUATION OFFICE
Presented by
IDEA International
(www.idea-international.org)
30 April 2018
IDEA International & IDEA-Solutions i
EXECUTIVE SUMMARY
1. National Evaluation Capacity Development (NECD) refers to the process of unleashing, strengthening
and maintaining evaluation capacities, so that development actors progressively engage into significant
and quality evaluation practices and adopt an evaluation culture, thereby contributing to improved
development outcomes. Especially in the context of the 2030 Agenda, evaluations can be used to inform
on progress towards SDGs, to know about current budget allocations and better orient future budget
allocations, to understand better current program performance and improve future policy and
programme design, to hold decision-makers and managers accountable on their use of public monies
and their actions, to demonstrate policy and programme effectiveness, and use them for awareness and
advocacy.
2. This mapping exercise aimed at providing: 1) A clear visual picture of UNICEF-supported NECD
interventions conducted alone or in partnerships along with a characterization of those interventions, 2)
An analysis of the performance of the identified UNICEF-supported NECD interventions based mainly on
a self-assessment; 3) A set of recommendations for the future UNICEF-supported NECD interventions; 4)
A database of the identified UNICEF-supported NECD interventions. This exercise was neither an
evaluation, nor an exhaustive listing of UNICEF-supported NECD intervention. Three data collection
methods were used sequentially to identify and characterize UNICEF-supported NECD interventions: 1)
A documentation review to review key reference documents on NECD; 2) An online survey of UNICEF
M&E staff which led to the identification and characterization of 60 NECD interventions around the world
and a first qualitative assessment of their performance based on their perceptions; and 3) A qualitative
study of eleven NECD interventions selected as case studies, using in-depth interviews with key UNICEF
M&E staff, counterparts, and partners: three at global level: EVALPARTNERS, EVALSDGs, and the Global
Parliamentarians Forum for Evaluation (GPFE); one at regional level: the EAPRO UNDP-UNICEF joint
initiative in the Asia-Pacific Region, and seven at country level: Ecuador-Colombia, Kirghizstan, Morocco,
Nigeria, Philippines, Rwanda, and Sri Lanka.
3. Sixty UNICEF-supported NECD interventions were identified by UNICEF M&E staff in the online survey.
From this list, it appears that the majority of NECD interventions took place at country level (87%), which
is to be expected. Let us note that global interventions (5%) and regional interventions (8%) are far from
negligible, especially if one considers their scope. Country level NECD interventions took place mainly in
South Asia (ROSA – 25%), West and Central Africa (WCARO – 18%), East Asia and the Pacific (EAPRO -
18%), Eastern and Southern Africa (ESARO – 17%), somewhat in Americas and Caribbean (LACRO – 5%),
and little in Europe and Central Asia (ECA – 2%) and Middle East and North Africa (MENA – 2%). Most of
the NECD interventions concerned enabling environment and institutional dimensions of capacity
development. 28,3% were multi-level 1 NECD interventions and 78.3% were multi-dimensional 2
1 Global, regional, country, sector, sub-national levels. 2 Enabling environment, institutional, individual dimensions of capacity development.
IDEA International & IDEA-Solutions ii
interventions, reflecting the importance of upstream work for UNICEF and the tendency to approach
NECD at multiple levels and link them. Sixty percent of the NECD interventions were conducted in middle
income countries. All NECD interventions dealt with the development domain and twenty percent dealt
with both development and humanitarian domains. Government was the main target group of NECD
interventions although in general non-government actors were also involved. By far and large, the 60
UNICEF-supported NECD interventions identified in this mapping exercise have been perceived to have
performed well in terms of relevance, effectiveness, efficiency, and this has been confirmed in the eleven
case studies. However, the performance could be improved in terms of sustainability of NECD seen as a
medium-term process.
4. Good practices encountered include first the capacity to engage a wide variety of international and
national actors, even on sensitive issues, in a respectful and evidence-based way, and progressively build
partnerships. Second, successful NECD interventions benefit a lot from work at all three levels (global,
regional, and country) and the interlinkages between those levels for cross-fertilization. The global level
provides a global perspective and an appreciation of commonalities and differences across regions and
there are trickle down effects at regional and country levels. The regional level provides a benchmarking
across countries in the region, an opportunity to learn from more similar countries, and a conduit of
communication between the global and the country levels. The country level provides concrete evidence
to the regional and global levels on the implementation and results of various NECD interventions in
different country contexts. Country Offices have the local knowledge and adapt to the specific country
conditions, e.g. to the selection of the right entry point and seizing opportunities, the identification of
champions, and the use of local experts. Third, NECD interventions suppose competent interveners. Solid
in-house expertise in evaluation and using quality regional and international evaluation experts make a
world of difference for credibility and results. Fourth, the integration of NECD in strategic plans, work
plans, evaluation frameworks and M&E systems goes a long way in making it a central concern, its
systematization and institutionalization. An NECD intervention should not be seen as an isolated project
per se, but a key building block for promoting and solidifying the evaluation function in the whole
management cycle, and this at all levels. Fifth, UNICEF corporate culture of “putting things on paper”
and documenting experiences in quality publications and briefings helps keep an institutional memory,
disseminate good practices, learn lessons and adjust, and advocate more credibly based on evidence.
5. Lessons learnt include first a recurrent theme that the objectives and scope of any NECD intervention
need to be realistic and take into account a diversity of factors: the slowness of bureaucratic processes
in the public sector, the importance of incentives to vanquish vested interests and resistance to change,
the need to accept the legal, political, institutional and cultural dimensions of evaluation and the
complexity of building successful partnerships. Second, NECD cannot be summed up in a short run
intervention; rather it has to be designed and managed as a medium-term process with a progressive
expansion of the scope and sustainability of results through a programme and a portfolio approach and
an institutionalization perspective. Third, success starts at the phase of design and selection of the NECD
intervention. The objectives, the targets and the roadmap need also to clearly defined with a clear Theory
IDEA International & IDEA-Solutions iii
of Change. Given resource constraints, hard choices must be made. Although there is no unique route,
most often, the first element to address in NECD is the enabling environment, then the institutional and
individual dimensions can be strengthened as clearly identified in the Nigeria case study. Fourth, there
are also some specific lessons depending on the level of capacity-building. At global level, it is often
necessary to “break the silos” and push each partner organization to move beyond its mandate and
rather create a common platform with different perspectives. At institutional level, government
ownership is essential for the establishment of an evaluation culture. At individual level,
professionalization of evaluation is a vital component of NECD.
6. UNICEF has been perceived by the vast majority of the respondents of the online survey and the
qualitative study of case studies as one of the leaders on NECD among international organizations. It
possesses a rich experience in NECD and expertise in NECD at all levels: global, regional, and country.
UNICEF has an uncanny capacity to engage and bring together different types of partners into NECD, i.e.
government, parliamentarians, NGOs, academics, private sector, sub-national actors, international
organizations and other development agencies, etc., to provide a platform for interaction and
information-sharing among all those interested in M&E, and to make linkages across different levels of
NECD. UNICEF is a trusted actor at all levels, recognized for its competencies in evaluation and its role as
an honest broker. Its focus on children is welcome by all, imposes a cross-cutting approach, and is totally
aligned with the 2030 Agenda agreed by all countries. UNICEF is also a relatively strong organization in
evaluation able to intervene from global level to sub-national level through its three-tier organization. It
has a capacity to lead or co-lead interventions at all levels. It is a knowledge-based organization which
can easily transfer skills to the locals with high quality intellectual, ethical and professional standards in
M&E. It can count on a set of competent and committed professionals in evaluation within the
organization and can also mobilize a wide network of professionals, organizations, and networks around
the world who are expert in evaluation and NECD. It has accumulated, documented and disseminated
knowledge on NECD through publications, toolkits, guidelines, etc.
7. In conclusion, UNICEF has accumulated good experience in NECD due to its commitment to SDG and
child focus, working at country, regional and global levels, in partnership with key actors and networks
in evaluation. There is a significant demand of continued leadership from UNICEF by interviewed
counterparts and partners in the NECD intervention case studies. UNICEF brings a unique contribution
to NECD through its child-oriented lens (intersectoral, target group of all social policies key to SDGs) and
an ability to work at global, regional, country and subnational levels and engage a variety of partners in
solid partnerships. Work on NECD also brings visibility and credibility to UNICEF and to its Evaluation
Office. The partial evidence gathered through this mapping exercise would argue for a continued and
institutionalized role for UNICEF to support articulated NECD interventions at global, regional, and
country levels with a child focus in the 2030 Agenda to obtain more sizable results in a more sustainable
way.
IDEA International & IDEA-Solutions iv
8. Six specific recommendations are made (relevant UNICEF actors are indicated in parentheses:
Headquarters – HQ, Regional Offices – ROs, Country Offices - COs):
Recommendation 1 (HQ, ROs): Position NECD in relation with the 2030 Agenda in key strategic documents,
including in the forthcoming UNICEF Evaluation Policy and Regional Evaluation Strategies. This should include
among others a clear definition of NECD and its role for the implementation of the 2030 Agenda, guidance
on approaches and selection criteria to be used by UNICEF approaches; a clear description of the linkages
between global, regional and country level NECD interventions, and the generic roles of UNICEF HQ, RO, CO
as a team and its partners and counterparts. This would contribute to institutionalization and facilitate fund
mobilization.
Recommendation 2 (HQ, ROs): Continue and intensify its partnership approach in NECD at global and
regional levels and take part in promising NECD and related initiatives initiated by other organizations, in
particular with UN organizations, but also bilateral donors and other actors, i.e. initiatives on data generation
and statistics, Voluntary National Reviews, etc. UNICEF has spearheaded several important global and
regional partnerships (EVALPARTNERS, EVALSDGs, GPFE, EAPRO UNDP-UNICEF joint initiative in the Asia-
Pacific Region) which should be continued with added commitments and resources. Such partnerships
provide a more integrated approach, institutional traction, help mobilize resources and go through political
cycles, lead to more sustainable results, and invite reciprocity. MoUs could be signed when relevant with key
partners, especially at global level.
Recommendation 3 (HQ, ROs): Have HQ and ROs support NECD interventions at lower levels3 where those
high-level actors can make a substantial difference, i.e. help deliver concrete milestones and where the
global or regional perspective is eye-opening, stimulates positive competition among countries, enables to
seize economies of scale. A two-tier approach is suggested combining basic services to all country offices
(sharing practices, promoting South-South and triangular cooperation, documenting case-studies with
support from Evaluation Office) and more intense coaching of selected ROs and COs with higher expressed
interest (demand) and high potential for growth in evaluation capacity (strategic).
Recommendation 4 (HQ, ROs, COs): Consider a variety of possible NECD delivery mechanisms, combining
involvement and leadership of internal staff with performance contracts (LTA type) to regional and
international organizations, whether public, private or non-for-profit, that have a track record of supporting
NECD at individual, institutional, and enabling environment levels.
Recommendation 5 (HQ, ROs, COs): Continue improving the capacity of UNICEF to support NECD in a
sustainable way. This may mean more training and coaching of M&E staff, more systematic sharing of good
practices and lessons learnt from more experienced ROs and COs to the still significant number of ROs and
COs that are not so advanced. Some tools for better selection and monitoring and evaluation of NECD
3 Lower levels would mean ROs and COs for HQ and COs for ROs.
IDEA International & IDEA-Solutions v
interventions could also be introduced such as a go/no go grid at country level4 to select NECD interventions
to be supported. Those capacity-building initiatives might be more attractive with financial incentives such
as an innovation fund for new promising NECD interventions to award grants on a competitive basis among
COs.
Recommendation 6 (HQ): Put in place a proper monitoring and documentation system of NECD
interventions on a continued basis. The database of NECD interventions developed within this mapping
exercise can be used as a springboard to develop a web-based information system of the UNICEF-supported
NECD interventions portfolio with decentralized access to a large number of stakeholders and business
intelligence (BI) tools to increase its potential for analysis and its user-friendliness. This data base and
supporting BI tools could be made available to other international organizations to share costs and benefit
from economies of scale as well as obtain a more global perspective on NECD interventions.
44 An example of possible go/no go grid at country level is presented in Annex 6.
IDEA International & IDEA-Solutions vi
ACKNOWLEDGMENTS
The consultants would like to give their sincere thanks to all those that contributed their time and ideas for
this mapping exercise. We would like to thank in particular Mr. George Laryea-Adjei, Director of the UNICEF
Evaluation Office, Ms. Ada Ocampo, Manager of the mapping exercise at the Evaluation Office, all the
members of the Reference Group, in particular Mr. Brook Boyer, Ms. Dorothy Lucks, and Mr. Riccardo
Polastro and, more generally all colleagues from UNICEF and partner organizations that participated in data
collection and discussion of results.
IDEA International & IDEA-Solutions vii
VERBATIM FROM RESPONDENTS
“Proactive involvement of UNICEF has led to more focus on UNICEF concerns.”
“UNICEF is the main agency supporting NECD globally, please continue the good work.”
“We would like to express our deep appreciation to UNICEF who has been the sole development agency that
has been consistently providing support to NECD activities over several years, which was instrumental in the
achievement of most of the outcomes and impact and has shaped the NECD situation in the country. “
IDEA International & IDEA-Solutions viii
TABLE OF CONTENTS
SECTION 1: CONTEXT AND OBJECTIVES ................................................................................................... 1
1.1. CONTEXT ...................................................................................................................................... 1
1.2. OBJECTIVES, USERS, AND SCOPE ................................................................................................... 2
SECTION 2: ANALYTICAL FRAMEWORK ................................................................................................... 4
2.1. DEFINITION OF NATIONAL EVALUATION CAPACITY DEVELOPMENT ................................................ 4
2.2. CHARACTERIZATION AND PERFORMANCE OF NECD INTERVENTIONS ............................................. 5
2.3. NECD STAKEHOLDERS ................................................................................................................... 7
SECTION 3: METHODOLOGY AND IMPLEMENTATION ............................................................................ 10
3.1. OVERALL PROJECT IMPLEMENTATION ......................................................................................... 10
3.2. DATA COLLECTION METHODS ..................................................................................................... 10
3.3. DATA ANALYSIS METHODS.......................................................................................................... 13
3.4 LIMITATIONS ............................................................................................................................... 13
SECTION 4: MAJOR FINDINGS ............................................................................................................... 16
4.1. MAPPING OF UNICEF-SUPPORTED NECD INTERVENTIONS ........................................................... 16
4.2. MAJOR CHARACTERISTICS OF UNICEF-SUPPORTED NECD INTERVENTIONS ................................... 18
4.3. PERCEIVED PERFORMANCE OF UNICEF-SUPPORTED NECD INTERVENTIONS ................................. 23
4.4. GOOD PRACTICES AND LESSONS LEARNT ..................................................................................... 26
4.5. COMPARATIVE ADVANTAGE OF UNICEF ...................................................................................... 27
SECTION 5: RECOMMENDATIONS ......................................................................................................... 30
ANNEX 1 – ORGANIZATION OF THE MAPPING EXERCISE ...................................................................................... 33
ANNEX 2: PROPOSED THEORY OF CHANGE FOR NECD ...................................................................................... 35
ANNEX 3 – DATA COLLECTION INSTRUMENTS .................................................................................................. 39
ANNEX 4 – CASE STUDIES OF NECD INTERVENTIONS AT GLOBAL AND REGIONAL LEVELS ........................................... 44
ANNEX 5 – CASE STUDIES AT COUNTRY LEVEL ................................................................................................. 58
ANNEX 6 – EXAMPLE OF POSSIBLE GO/NO GO GRID AT COUNTRY LEVEL .............................................................. 80
IDEA International & IDEA-Solutions ix
ACRONYMS AND ABBREVIATIONS
CD Capacity Development
CO Country Offices
CP Country Programme
CPAP Country Programme Action Plan
CSO Civil Society Organizations
ECOSOC United Nations Economic and Social Council
GA United Nations General Assembly
GPFE Global Parliamentarians Forum for Evaluation
HQ UNICEF Headquarters
IMEP Integrated Monitoring and Evaluation Plan
IOCE International Organization for Cooperation in Evaluation
LIC Low income country
MIC Middle income country
M&E Monitoring and Evaluation
NECD National Evaluation Capacity Development
NGO Non-Governmental Organizations
RO Regional Offices
RBM Results Based Management
TOC Theory of Change
TOR Terms of Reference
UN United Nations
UNDP United Nations Development Programme
UNDG United Nations Development Group
UNEG United Nations Evaluation Group
UNFPA United Nations Population Fund
UNICEF United Nations Children’s Fund
VOPE Voluntary Organization for Professional Evaluation
UN WOMEN United Nations Entity for Gender Equality and the Empowerment of Women
IDEA International & IDEA-Solutions 0
CONTEXT AND
OBJECTIVES 1
IDEA International & IDEA-Solutions 1
SECTION 1: CONTEXT AND OBJECTIVES
1.1. CONTEXT
One major role of the UN is to develop national capacities (UN Charter page 1). Since the early 2000’s,
UNICEF has put significant effort to support internally and externally evaluation and evaluation capacity
development. Let us mention a few of those normative documents. The 2014-2017 Strategic plan
indicates that priority will be given to undertaking strategic evaluations and enhancing the accountability
system, focusing on decision-making and partnerships. Evaluation will support the achievement of
Strategic Plan objectives by generating findings, conclusions and recommendations to inform
management decisions at all levels and across all results, including in humanitarian action
The UNICEF’s revised Evaluation Policy of 2013 indicates that the evaluation function aims to help UNICEF
to continually improve its performance and results, by supporting organizational learning and
accountability. Since the progressive decentralization of the evaluation function, it is carried out at all
levels of the organization, and it is applicable in all contexts, be it in the humanitarian or development
fields. The policy indicates that Evaluation at UNICEF also aims at contributing to National Evaluation
Capacity Development (NECD), in order to strengthen the capacity of a variety of actors to carry out equity
focused evaluation of national development policies and programmes. The Evaluation Policy not only
provided the first corporate framework for NECD but triggered Executive Board Decisions supporting
NECD initiatives. In its session of June 2016, for example, the Executive Board Requested UNICEF, in
collaboration with other entities of the UN to support National Evaluation Capacity Development, in
accordance with the GA Resolution 69/23.
At a more global level, let us also mention United Nations Evaluation Group (UNEG) norms and standards
for evaluation first established in 2005 and revised in 2016. These norms and standards are meant to serve
as key reference for evaluators and strengthen evaluation practices. In particular, Norm 9 on National
Evaluation Capacities states that “The effective use of evaluation can make valuable contributions to
accountability and learning and thereby justify actions to strengthen national evaluation capacities. In line
with the General Assembly Resolution A/RES/69/237 on building capacity for the evaluation of
development activities at the country level, national evaluation capacities should be supported upon the
request of Member States” (page 12).
As part of the agreement on SDGs and the Agenda 2030, it was mentioned that evaluation should be
included in the framework for the post-2015 development agenda from the outset (UNEG). Follow-Up
and Review (FUR) operating at national, regional and global levels are to promote accountability to
citizens, support effective international cooperation in achieving this Agenda and foster exchanges of best
practices and mutual learning. (par 73/ Agenda).
IDEA International & IDEA-Solutions 2
UNICEF has created key success conditions for NECD during the last twenty years at various levels:
• At global level, UNICEF has played and continues to play a key role in EVALPARTNERS, EVALSDGs,
and the Global Parliamentarians Forum for Evaluation (GPFE). UNICEF created EVALPARTNERS
and seats at the management group. UNICEF co-leads EVALSDGs and provides the secretariat to
the GPFE. UNICEF is also engaged into other partnerships such as EvalYouth and global advocacy
activities aimed at highlighting the importance of developing national evaluation capacities as
well as contributing to create enabling environments that will allow to embed evaluation within
national frameworks and systems to achieve the 2030 agenda.
• At regional level, UNICEF Regional Offices have conducted individually and jointly with partners
regional evaluation capacity development initiatives that benefit NECD as well as directly
supporting selected NECD initiatives led by several UNICEF country offices in their region, trying
to do advocacy and promote the alignment of the national efforts to the 2030 agenda.
• At national level, UNICEF Country Offices are supporting governments, civil society organizations,
parliamentarians and other actors to develop national evaluation policies and frameworks, put
together national evaluation agendas, reinforce capacities to advocate for evaluation,
conceptualize and manage country-led evaluations, in alignment with the endorsement of the
commitment to the 2030 agenda.
1.2. OBJECTIVES, USERS, AND SCOPE
This mapping exercise was aimed at generating information on what UNICEF has done and achieved in
terms of NECD mainly since the elaboration of the revised UNICEF evaluation policy in 2013 to 1)
appreciate the range of NECD interventions that UNICEF led or participated in; 2) provide
recommendations for the next phase of UNICEF strategy on NECD. Specifically, this study aimed at
providing:
1) A clear visual picture of UNICEF-supported NECD interventions conducted alone or in partnerships
along with a characterization of those interventions;
2) An analysis of the performance of the identified UNICEF-supported NECD interventions based
mainly on a self-assessment;
3) A set of recommendations for the future UNICEF-supported NECD interventions;
4) A database of the identified UNICEF-supported NECD interventions.
Let us underline that this exercise was neither an evaluation, nor an exhaustive listing of UNICEF-
supported NECD interventions.
The main users of the information generated through the mapping are indicated in Table 1.1.
Table 1.1: Main Users of the Information Generated through the Mapping
Level Main Users
Global Management and staff of the Evaluation Office, Executive Board, partners
Regional Regional Management Teams, regional Level M&E staff, partners
National Country Management Teams, country Level M&E staff, partners
IDEA International & IDEA-Solutions 3
ANALYTICAL
FRAMEWORK 2
IDEA International & IDEA-Solutions 4
SECTION 2: ANALYTICAL FRAMEWORK
2.1. DEFINITION OF NATIONAL EVALUATION CAPACITY DEVELOPMENT
Capacity development is a long-term, endogenous change process that takes place in the context of
ongoing partner and donor efforts to strengthen related systems of management, governance,
accountability and learning to improve development effectiveness (World Bank 1998)5. Although there is
no unique definition of evaluation capacity (Rist et al. 2010 page 1)6, let us define for the purpose of this
report evaluation capacity development as “…a context-dependent, intentional action system of guided
processes and practices for bringing about and sustaining a state of affairs in which high-quality program
evaluation and its appropriate uses are ordinary and ongoing practices within and/or between one or
more organizations/programs/sites” (Stokdill et al. 2002, page 8) 7 . National Evaluation Capacity
Development (NECD) refers to the process of unleashing, strengthening and maintaining evaluation
capacities, so that development actors progressively engage into significant and quality evaluation
practices and adopt an evaluation culture, thereby contributing to improved development outcomes, in
particular those of the 2030 Agenda. This culture implies a greater demand for evaluation perceived by
users as providing useful information as well as a better capacity to select, manage, and conduct
evaluations, and use evaluation results. Evaluations can be used to know about current budget allocations
and better orient future budget allocations, to understand better current program performance and
improve future policy and programme design, to hold decision-makers and managers accountable on their
use of public monies and their actions, to demonstrate policy and programme effectiveness, and use them
for awareness and advocacy. Even though the thrust of NECD is by definition at country level, there are
significant economies of scale and synergies to be obtained from NECD interventions conducted at global
and regional levels.
NECD concerns of course the national government, but also all other development stakeholders, including
Parliament, sub-national governments, NGOs and CSOs, the private sector, academia, international
organizations and donors. NECD is of course conducted at national level, but also at regional and global
levels with strong linkages and synergies between the three levels. It typically involves four interrelated
and dynamic components: developing an Evaluation Policy, setting up the institutional processes, securing
adequate evaluation capabilities, and engaging with partners.
5 World Bank (1988). Evaluation Capacity Development in Africa: Selected Proceedings from a Seminar in Abidjan, OED proceedings, World Bank, Washington, DC. 6 Rist, R., M.H. Boily, and F. Martin (2010). Influencing Change: Evaluation and Capacity Building. World Bank,
Washington, D.C. 7 Stockdill, S.H., Baizerman, M. and Compton, D.W. 2002. “Toward a Definition of Evaluation Capacity Building
Process: A conversation with the Evaluation Capacity Building Literature.” New Directions for Evaluation, No. 93, Spring 2002, pp. 7-26.
IDEA International & IDEA-Solutions 5
It can be useful to differentiate NECD according to its scope: NECD strategy, NECD domains, and NECD
interventions.
Table 2.1: Scope of NECD
Scope Definition
NECD strategy Global UNICEF strategy8 in terms of NECD, outlining objectives and strategies to achieve those
objectives, with the perspective of contributing to the achievement of SDGs at global level
NECD domains Demand, use, leadership, resources, technical capacity, institutional arrangements, values and
standards
NECD
interventions
UNICEF leadership/ participation in advocacy for NECD, NECD actions conducted by UNICEF alone
or in partnership
Source: IDEA except for NECD domains which are based on Lahey’s typology
NECD interventions may apply at global, regional or national, sector and/or sub-national levels. They may
include one or more dimensions of capacity development: enabling environment, institutional, individual.
Annex 2 presents a proposed Theory of Change for each of those dimensions as contextual information.
NECD is related to several other concepts as evaluation is considered as one pillar of the Results-Based
Management (RBM) approach. Planning is essential since evaluation will to a large extent be made as a
function of objectives, performance indicators and targets defined in the planning system. A
programmatic approach allows for tractable evaluations. Performance-informed budgeting means that
future budget allocations will be influenced by the evaluation of future needs and of past performance.
Monitoring is the companion to evaluation, one feeding into another. Finally, evaluation is worthless
without credible evidence, which highlights the importance of statistical systems, data quality, and
functioning information systems, and use of the evaluation results for decision-making, accountability,
and advocacy.
2.2. CHARACTERIZATION AND PERFORMANCE OF NECD INTERVENTIONS
NECD interventions were characterized using the criteria indicated in Table 2.2.
8 Strategy defined as “a careful plan or method for achieving a particular goal usually over a long period of time” (Merriam-
Webster dictionary, http://www.merriam-webster.com/dictionary/strategy).
IDEA International & IDEA-Solutions 6
Table 2.2: NECD Intervention Characterization Criteria
Characterization criterium Categories
Geographic scope of the NECD
intervention
Global, region or country
UNICEF region in which the NECD
intervention took place
Latin America and the Caribbean, Europe and Central Asia, East Asia and the
Pacific, Eastern and Southern Africa, Middle East and North Africa, South
Asia, West and Central Africa Regional Office
Type of country in which the NECD
interventions took place
Middle income country (MIC), low income country (LIC)9, and fragile10)
Target groups of the NECD
intervention
Level of the NECD intervention Enabling environment, institutional, individual11
Domain of work Humanitarian and/or development12
Objectives
Attention to specific cross-cutting
dimensions of development
Human rights, equity, gender equality, and/or SDGs
Cost
Duration
UNICEF role Alone or partnership, leadership or support
Type of partners
The performance of NECD interventions was assessed again the following criteria:
• Extent to which most important NECD interventions are in line with global reference documents;
• Relevance, effectiveness, efficiency, outputs, outcomes, and sustainability
• Factors supporting good performance
• Challenges to good performance
• Lessons learnt
• Good practices
• UNICEF comparative advantage in terms of NECD.
9 As of 1 July 2016, low-income economies are defined as those with a GNI per capita, calculated using the World
Bank Atlas method, of $1,025 or less in 2015; lower middle-income economies are those with a GNI per capita between $1,026 and $4,035; upper middle-income economies are those with a GNI per capita between $4,036 and $12,475 (http://blogs.worldbank.org/opendata/category/tags/news).
10 A fragile state is a low-income country characterized by weak state capacity and/or weak state legitimacy leaving citizens vulnerable to a range of shocks (https://en.wikipedia.org/wiki/Fragile_state).
11 Including possible multi level interventions since linkages across those levels can lead to more synergies. 12 Some interventions cover both humanitarian and development domains.
IDEA International & IDEA-Solutions 7
2.3. NECD STAKEHOLDERS
NECD stakeholders are presented in Table 2.3.
Table 2.3: NECD Stakeholders
Level Stakeholder
category
Stakeholders
Global UNICEF
Headquarters
• Executive Board
• Evaluation Office
• Programme directors
UNICEF global
partners
• United Nations Evaluation Group (UNEG)
• Evaluation offices of other international organizations such as UNDP, UNFPA,
World Bank, and selected bilateral donors
• International private firms, NGOs and internationally recognized academic
institutions and consultants
• Global evaluation associations or networks such as EVALPARTNERS,
EVALSDGs, IOCE, etc. that often regroup a variety of the above-mentioned
actors
Regional UNICEF Regional
Office
• Regional Management Team
• Regional Evaluation Advisors
Other regional
partners
• Regional Voluntary Organizations for Professional Evaluation (VOPEs):
Regions now have one or more VOPEs regrouping a variety of actors, both on
the demand side (evaluation clients) and the supply side (providers of
evaluation services). Let us note that the definition of regions is not
homogeneous and does not necessarily correspond to the UN regional
division, so regional VOPEs might overlap and a country might be part of
several regions.
• A variety of regional NGOs and CSOs may also participate in the design and
the delivery of NECD activities as well as regional branches of international
NGOs and CSOs.
• Other international organizations and bilateral donors: In particular, UNDP,
UNFPA, World Bank, and regional development banks can be involved in
regional NECD activities in partnership or in complementarity with those of
UNICEF.
IDEA International & IDEA-Solutions 8
Table 2.3: NECD Stakeholders (cont’d)
Level Stakeholder
category
Stakeholders
National UNICEF Country
Offices
• Country Management Team
• M&E unit
• Programs
Government13
• The national government, including possibly the Presidency, the Office of the
Prime Minister, central agencies like the ministry of Economy and Finance,
and the ministry of planning as well as a number of line ministries and
agencies.
• The sub-national governments.
• The Parliament which may include a verification body such as the Auditor’s
General Office.
• The Judiciary Branch, which may also include a verification body
Other national
partners
• National Voluntary Organizations of Professional Evaluators (VOPEs): Most
countries now have one or more VOPE regrouping a variety of actors, both
on the demand side (evaluation clients) and the supply side (providers of
evaluation services).
• The private sector, be it national firms, international companies or private
foundations, may support technically and financially NECD activities. National
consulting firms and individual consultants as well as universities may be
directly involved in the design and delivery of NECD activities.
• A variety of national Non-Governmental Organization (NGOs) and Civil
Society Organizations (CSOs) may also participate in the design and the
delivery of NECD activities as well as national branches of international NGOs
and CSOs.
• Other international organizations and bilateral donors: In particular, UN
country team (UNDP, UNFPA, etc.), World Bank, and regional development
banks can be involved in national NECD activities, individually without any
consultation or coordination, in complementarity or in partnership with those
of UNICEF.
13 The composition of government varies from one country to the next.
IDEA International & IDEA-Solutions 9
METHODOLOGY AND
EXECUTION 3
IDEA International & IDEA-Solutions 10
SECTION 3: METHODOLOGY AND IMPLEMENTATION
3.1. OVERALL PROJECT IMPLEMENTATION
This project started with an inception mission to UNICEF headquarters in September 2017, meeting the
project manager and a number of key informants. An inception report was prepared, discussed and
finalized in November 2017 based on the comments of the Project Manager and the Reference Group.
Then the online survey took place late November and early December 2017. In-depth interviews for the
case studies lasted from late December 2017 till early February 2018. Analysis took place from mid-
January till end February 2018. The presentation of major findings was made on March 1st 2018 at UNICEF
headquarters to the Evaluation Office, and on March 2nd, 2018 to the Reference Group. The draft final
report was produced in early April 2018 and the final version in late April 2018. Those dates are beyond
the deadlines set in the initial work plan. Several reasons explain those delays, including among others an
underestimation of the level of effort to conduct the study, the time necessary for participants from all
over the world to respond, the overlapping with other assignments, and the New Year break. The support
of the Project Manager to provide the contacts, foster a high response rate, and provide rapid response
to questions and issues along the way has proved very valuable.
3.2. DATA COLLECTION METHODS
The mapping exercise built upon three data collection methods conducted sequentially and providing
different kinds of data. First, the documentation review on key reference documents on NECD was used
for two purposes: as background information to the mapping exercise and to identify if UNICEF-supported
NECD interventions were aligned with those key reference documents. Second, the online survey of
UNICEF M&E staff was used to identify the UNICEF supported NECD interventions, characterize them, and
obtain a first assessment of the performance of the NECD interventions by UNICEF M&E staff. Third,
selected NECD interventions were the object of case studies based on in-depth interviews with key
stakeholders, including UNICEF M&E staff, national counterparts, and/or partners.
Review of relevant documentation
Relevant documentation was obtained from the Project Manager, RO evaluation advisers, and CO M&E
staff, as well as counterparts and partners contacted. A review of relevant websites was also conducted.
Online survey of UNICEF CO M&E staff
An online survey was administered to all UNICEF M&E staff at global, regional, and national levels from a
data base provided by UNICEF. A closed questionnaire (see Annex 3) was sent to 248 staff to document
up to three UNICEF-supported NECD. 103 staff, i.e. a response rate of 42% responded. Altogether 60
UNICEF-supported NECD interventions were identified.
IDEA International & IDEA-Solutions 11
NECD Intervention Case studies
Case studies of NECD Interventions were documented at global, regional, and country levels. At global
and regional levels, the criterium for selection of the NECD intervention was the importance of this
NECD intervention in terms of its strategic meaning for UNICEF strategy, partnerships, and resources
allocated to it. Three NECD interventions were selected at global level: EVALPARTNERS, EVALSDGS, and
the Global Parliamentarians Forum for Evaluation (GPFE), and one at regional level: the review of
national evaluation systems and capacities in Asia Pacific for evaluating progress towards SDGs- a joint
UNDP UNICEF initiative.
At country level, one country was selected in each UN region which had been particularly active in NECD:
Kirghizstan, Morocco, Nigeria, Ecuador14, Philippines, Rwanda, and Sri Lanka. In each selected country,
the NECD intervention or combination of related NECD interventions that was most significant was
selected from the list indicated of NECD interventions indicated in the online survey of UNICEF staff.
Hence a total of 11 case studies were documented: 3 at global level, 1 at regional level, and 7 at country
level as indicated in Figure 3.1.
Figure 3.1: List of Case studies
14 Mexico had been initially selected in the LAC region, but the online survey revealed that NECD interventions in
Mexico were mainly conducted by federal government entities without UNICEF participation, while UNICEF Ecuador had conducted an interesting South-South intervention.
IDEA International & IDEA-Solutions 12
The respondents were first the UNICEF NECD intervention project manager and then the counterparts
that s/he identified as most knowledgeable about this intervention. There is a possible risk of bias of
selection by UNICEF officers in their choice of partners to be interviewed. However, let us remind that this
is not an evaluation, but a mapping exercise and the choice of main counterpart was more or less self-
evident most of the time.
An in-depth interview questionnaire using a quantitative-qualitative (Q2) approach was administered in
two phases (see Annex 3). First, the questionnaire was sent by e-mail to the respondents who could fill
up the Section on the Description of the NECD intervention and her/his evaluation of the performance of
the NECD intervention on an ordinal scale. Second, the consultant team called each respondent for a 30
to 60’ interview to discuss and fill up the qualitative part of the questionnaire, including the justification
of the evaluation made of the performance and other questions related to the perception of the
respondent on the NECD intervention and UNICEF’s role.
Main meta data are presented in Figure 3.2.
IDEA International & IDEA-Solutions 13
Figure 3.2: Main Metadata on the Mapping Exercise
3.3. DATA ANALYSIS METHODS
For the documentation review, each document was read, analyzed and a synthesis was prepared. The
review of literature is presented in Annex 2.
Data on the 60 NECD interventions from the online survey were compiled in a data base. A series of tables
and graphs presenting characteristics of those intervention were prepared and a simple GIS using Google
Maps was prepared to position the interventions and to provide the reader with basic characteristics on
each intervention.
The qualitative information was analyzed and synthesized in the form of case studies presented in
Annexes 4 and 5. Excerpts of case studies were also used in the main text to illustrate some of the main
findings.
3.4 LIMITATIONS
Given its objectives, scope, time and budget, the mapping exercise did not pretend to be an evaluation of
NECD interventions. It aimed principally at identifying UNICEF-supported interventions and doing a first
characterization. The appreciation of performance by UNICEF M&E staff and, in the case of case studies,
key counterparts and partners, was based on their qualitative appreciation. It is entirely possible that the
answers given entailed some bias. Also, triangulation of the results was not really considered in as far as
the three methods were used sequentially and aimed at providing different kinds of data. The only
possible triangulation is for the 11 in-depth case studies conducted interviewing UNICEF and national
counterparts. By far and large, the in-depth interviews confirmed the results from the online survey and
enriched it with more qualitative information. A full-fledged evaluation of a sample of NECD interventions
in the population identified in the mapping exercise might be considered by UNICEF in a second phase for
a more objective and thorough assessment.
IDEA International & IDEA-Solutions 14
The mapping was not designed to be exhaustive. First it is limited to the UNICEF-supported NECD
interventions. Second, it is based on the responses of the current UNICEF M&E staff and some
interventions might have been forgotten. However, by far and large, we estimate it represents a fair
representation of the kinds of NECD interventions supported by UNICEF. Due to the large coverage of
UNICEF around the world, it took some time to coordinate meetings with respondents of the in-depth
interviews. Another factor to consider was that data collection fell around the end of the year holidays
which delayed further responses. The support of the Evaluation Office, and the Manager of the mapping
exercise in particular, was essential to have access to key UNICEF staff and counterparts.
.
IDEA International & IDEA-Solutions 15
MAJOR FINDINGS 4
IDEA International & IDEA-Solutions 16
SECTION 4: MAJOR FINDINGS
4.1. MAPPING OF UNICEF-SUPPORTED NECD INTERVENTIONS
Figure 4.1: Mapping the NECD Interventions
Source: UNICEF-IDEA online survey (November 2017)15.
15 See https://www.google.com/maps/d/viewer?mid=1zvecA_d2jMebHK82tX8pc408_85jfS3E&ll=0%2C0&z=3 for full information.
IDEA International & IDEA-Solutions 17
This map includes all the UNICEF-supported NECD interventions identified in the online survey. Global
interventions are located at UNICEF headquarters in New York City. Regional interventions are located at
the regional UNICEF regional office. Country interventions are positioned on the capital location. Let us
underline that this mapping does not pretend to be exhaustive.
Figure 4.2: Percentage of NECD interventions per level and region16
Source: UNICEF-IDEA online survey (November 2017)
From the list of identified NECD interventions, it appears that the majority of NECD interventions took
place at country level (87%), which is to be expected. Let us note that global interventions (5%) and
regional interventions (8%) are far from negligible, especially if one considers their scope. Country level
NECD interventions took place mainly in South Asia (ROSA – 25%), West and Central Africa (WCARO –
18%), East Asia and the Pacific (EAPRO - 18%), Eastern and Southern Africa (ESARO – 17%), somewhat in
Americas and Caribbean (LACRO – 5%), and little in Europe and Central Asia (ECA – 2%) and Middle East
and North Africa (MENA – 2%). Several factors related to demand and supply may explain this result:17 (i)
some countries benefit from more support and resources from UNICEF because their needs are higher;
(ii) the openness to an evaluation culture varies depending on the government and the institutional
16 The global and regional NECD interventions are categorized separately. The country NECD interventions are
regrouped by region. Sub-national interventions are included in the country category. 17 The prevalence and importance of those factors might be investigated in a second phase following this mapping
exercise.
IDEA International & IDEA-Solutions 18
culture; (iii) there may be demand but not necessarily UNICEF is willing, or in a position to respond to
those demands: and (iv) not necessarily all staff at UNICEF at CO and RO might be clear on the importance
of NECD.
4.2. MAJOR CHARACTERISTICS OF UNICEF-SUPPORTED NECD INTERVENTIONS
This sub-section presents the major characteristics of the identified UNICEF-supported NECD
interventions.
Levels of NECD interventions18
Figure 4.3: Percentage of NECD interventions per level
Source: UNICEF-IDEA online survey (November 2017)
Eighty seven percent of the NECD interventions were conducted at the national level, which is to be
expected. However, a small number of important NECD interventions were conducted at sector, regional,
and global levels. Interestingly, the number of sub-national interventions was the second largest,
indicating both the needs at this level and the openness of UNICEF to work at this level. Also, worth
underlining, 28,3% were multi-level NECD interventions, pointing out at the capacity of UNICEF to link
several levels.
18 Please note that an NECD intervention may have several elements of a given characteristic. So, the total might
be higher than 100%.
IDEA International & IDEA-Solutions 19
Types of country in which the NECD interventions took place
Figure 4.4: Percentage of NECD interventions per type of country
Source: UNICEF-IDEA online survey (November 2017)
Sixty percent of the NECD interventions supported were conducted in middle income countries, 28% in
low income countries, 15% as fragile countries.19 This is to be expected as low- income countries have
other priority needs to address, such as just keeping peace and order, paying civil servants, and providing
basic public services.
Work domains in which the NECD interventions took place
Figure 4.5: Percentage of work domain in which the NECD took place
Source: UNICEF-IDEA online survey (November 2017)
19 As already mentioned, one NECD intervention can be categorized in more than one category.
IDEA International & IDEA-Solutions 20
All NECD interventions dealt with the development domain and 20% dealt with both development and
humanitarian domains. This reflects the orientation of UNICEF towards development programmes, but
also the significant and increasing share of interventions in the humanitarian domain with the growing
number and intensity of man-made and nature-related crises.
Target groups of NECD interventions
Figure 4.6: Percentage of Target groups of NECD intervention
Source: UNICEF-IDEA online survey (November 2017)
Government was the main target group of NECD interventions. However, CSOs, Parliament, Academia
and NGOs were also largely involved, which reflects the capacity of UNICEF to engage a number of
government and non-government actors and build partnerships.
Dimensions of NECD interventions
Figure 4.7: Percentage of Dimensions of NECD
Source: UNICEF-IDEA online survey (November 2017)
IDEA International & IDEA-Solutions 21
Most of the NECD interventions covered enabling environment and institutional capacity development
while a little less than half dealt with individual capacity development. 78.3% were multi-dimensional
NECD interventions. This reflects the importance of upstream work for UNICEF and the tendency to
approach NECD with multiple dimensions.
Objectives of NECD Interventions
Figure 4.8: NECD interventions by objective
Source: UNICEF-IDEA online survey (November 2017)
NECD interventions were quite varied and had multiple objectives. The four major ones were to:
contribute to the development of NEPs, build up partnerships to conduct NECD, provide training and
professional development opportunities as well as awareness and advocacy activities for NECD.
Duration of the NECD interventions
Figure 4.9: NECD interventions by duration
Source: UNICEF-IDEA online survey (November 2017)
IDEA International & IDEA-Solutions 22
The duration of NECD interventions varied a lot. A significant share had a short duration: 27% last 3
months or less, 39% between 3 and 12 months, and 34% more than one year. UNICEF has been able to
seize short run opportunities, but also build up capacities over the medium run.
Alignment with Reference Documents
Figure 4.10: Extent to which the NECD intervention is in line with global reference documents
Source: UNICEF-IDEA online survey (November 2017)
Most NECD interventions are by far and large aligned with key reference documents. This reflects a good
dissemination of those documents by the UN and UNICEF in particular.
Incorporation of cross-cutting dimensions in NECD interventions
Figure 4.11: Percentage of NECD interventions considering various cross-cutting dimensions
Source: UNICEF-IDEA online survey (November 2017)
IDEA International & IDEA-Solutions 23
Most NECD interventions took into account cross-cutting dimensions of human rights, equity, gender
equality, and SDGs. The dimensions less accounted for in some cases were human rights and gender
equality. This might reflect less relevance and/or less interest on the part of the national counterpart
government.
4.3. PERCEIVED PERFORMANCE OF UNICEF-SUPPORTED NECD INTERVENTIONS
This sub-section presents a first assessment by UNICEF M&E staff of the 60 identified UNICEF-supported
NECD interventions based on their qualitative perceptions as collected in the online survey. This is
complemented, when deemed relevant, with the results of the case studies, also based on the qualitative
perceptions of UNICEF staff and selected counterparts and partners as obtained from in-depth interviews.
Figure 5.1: Perceived performance of UNICEF-supported NECD interventions
Source: UNICEF-IDEA online survey (November 2017)
The overall performance of the 60 identified UNICEF-supported NECD interventions is perceived as high
according to the respondents of the online survey. On an ordinal scale of 0 to 3, the grades are from the
highest to the lowest for relevance (2.7), effectiveness (2.3), efficiency (2.3), and sustainability (2.0).20
This result of the online survey is triangulated by the results of the case studies (see Annexes 4 and 5).
The relevance of UNICEF supported NECD interventions has been systematically rated as high. The global
interventions are directly supportive of NECD. EVALPARTNERS aims at strengthening national evaluation
capacities through partnerships and advocacy activities. EVALSDGs promotes evaluation activities around
the SDGs and orients and supports the review and follow-up process on the SDG 2030 agenda by adding
value through evaluation. The Global Parliamentarians Forum for Evaluation (GPFE) aims at strengthening
the demand and the use of evaluation by parliamentarians, creating enabling environments for nationally
20 The measurement of final outcomes and impacts fell outside of the scope of this mapping exercise, mainly because
many interventions were too recent.
IDEA International & IDEA-Solutions 24
owned, transparent, systematic evaluation processes, and promote national evaluation policies and
systems. The regional interventions also support national capacities directly, but also indirectly by
fostering experience exchanges, dissemination of good practices and lessons learnt, training on evaluation
approaches and methodologies, and advocacy for important evaluation issues. For example, the EAPRO
UNDP-UNICEF joint initiative and country case studies in the Asia-Pacific region21 aim at generating
knowledge to guide NECD for the 2030 Agenda, identifying successes and lessons in terms of national
evaluation systems, fostering peer learning and providing global and regional NECD guidance through the
production of readiness assessments at country level. At country level, NECD interventions have
supported the establishment and strengthening of national Monitoring & Evaluation systems (cases of
Kyrgyzstan and Sri Lanka), pushed for important elements of NECD such as the formulation of a National
Evaluation Policy (NEP) (case of Philippines) or of an Evaluation Plan (case of Ecuador with support from
Colombia), and of course building up the individual competencies and skills of a variety of actors, public
sector officials and professionals, parliamentarians, NGO and CSO managers and professionals,
academics, and consultants. Many NECD interventions supported the creation and strengthening of a
VOPE (cases of Morocco, Rwanda) to strengthen the culture of evaluation, increase national capacities for
policy and programme evaluation, and conduct advocacy for the institutionalization of the evaluation
function.
The effectiveness and to a large extent the efficiency in producing outputs have also been perceived as
high.22 The global and regional NECD interventions resulted in the establishment and support of several
global networks and partnerships, the preparation of various materials and platforms for communication,
learning and advocacy, a number of evaluations, publications and documentation of NECD interventions.
EVALPARTNERS has clearly strengthened the enabling environment for evaluation by increasing the
demand and the use of evaluation and demonstrated the power of partnerships for influencing global
agendas. EVALPARTNERS has proven to be a very good NECD intervention to foster global NECD initiatives
and partnerships for a more coordinated NECD. It has offered a positive ecosystem and institutional
framework for several networks and NECD interventions. EVALSDGs has also produced with little money
a wide variety of outputs helpful for the preparation towards the evaluation of the SDGs, including
important publications, the spotlight initiative, organization of regional consultations, advocacy for the
establishment of frameworks for SDGs evaluation and how to influence the Voluntary National Reviews
(VNRs). Through capacity building, knowledge sharing and advocacy, the GPFE has empowered
parliamentarians to deliver evidence-based policies and thus contributed to good governance, sustainable
development and social equity. At regional level, the recent EAPRO UNDP-UNICEF joint initiative in the
Asia-Pacific Region produced two country diagnostics, but also engaged the participating countries into a
stimulating debate about the role of M&E systems, and evaluations in particular, to monitor, review and
evaluate implementation of 2030 Agenda and recommendations to strengthen national evaluation
capacity for the SDGs with a demonstrating and stimulating effect on other countries of the region.
21 Called EAPRO UNDP-UNICEF initiative in the Asia-Pacific Region in the rest of the report. 22 The details of outputs and immediate outcomes for each global or regional case study are presented in Annex 4.
IDEA International & IDEA-Solutions 25
At country level, the outputs of the NECD varied from one country to another23. However, the most often
encountered outputs in the seven case studies (Ecuador-Colombia, Kyrgyzstan, Morocco, Nigeria,
Philippines, Rwanda, and Sri Lanka) were the creation and strengthening of evaluation institutions such
as a VOPE and a joint evaluation committee, the elaboration and adoption of a National Evaluation Policy
(NEP), national evaluation framework and evaluation plans, the improved evaluation knowledge and skills
of various key actors as a result of training and exchange of experiences, the production of learning and
advocacy material and platforms, evaluation reports, publications, and documentation of NECD
interventions. Beyond the important production of outputs, the participatory process of engaging a
variety of actors at national, sub-national, regional, and international levels is worth mentioning.
Those outputs have translated already into immediate outcomes in a number of cases. The main
outcomes at global and regional levels were the following: putting evaluation on the agenda of work of
many actors, introducing evaluation in global and regional policy-making frameworks, promoting a culture
of evaluation in global and regional policies and programs, and building up global and regional resources,
momentums and synergies for evaluation. At country level again, the outcomes varied from one country
to the next. However, the most often encountered outcomes were the introduction of evaluation in
policy-making frameworks, the promotion of a culture of evaluation in national policies and programs,
the progressive development of a stronger evaluation function and M&E systems, the capacity building of
national resources and further development of partnerships.
Sustainability is probably the lowest rated criteria in the online survey while still rated as rather good (2
on a scale of 3). This concern for sustainability was also noted in most case studies. A number of NECD
interventions are short run, with no guarantee of continued engagement of UNICEF and its partners as
well as availability of resources. Ad hoc interventions can be a way to seize an opportunity, but also to
scatter resources into too many activities, not to mention they complicate follow up and reporting. It is
hard to keep up the momentum both for UNICEF staff and for counterparts and partners’ staff without a
guiding thread and a medium term NECD plan. Several NECD interventions, especially at global level, have
a huge scope, but depend sometimes, for example, on voluntary work of professionals, or the combination
of joint activities of human resources that are involved in other tasks for other initiatives different from
NECD. In this case, a conducive enabling environment supports the sustainability of those initiatives.
Finally, the political cycles are a cause for concern in many countries when new administrations tend to
question automatically the interventions conducted with previous administrations, and change key
counterparts, leading to a loss of institutional memory, a break in the momentum, and the need to move
back a few steps to give the time to the new incumbents to make their own opinion and acquire the
evaluation competences and skills and the time to both UNICEF and counterparts to build trust for a solid
partnership. Actually, the continued presence of UNICEF in nearly all countries, its general appreciation
23 The details of outputs and immediate outcomes for each country case study are presented in Annex 5.
IDEA International & IDEA-Solutions 26
by all parties as an honest broker, and its solid partnerships with a variety of actors, bring an element of
permanence that can facilitate those political transitions.
4.4. GOOD PRACTICES AND LESSONS LEARNT
A number of specific good practices and lessons learnt in UNICEF-supported NECD interventions have
been identified in the case studies.24 25 Among commonly encountered good practices, let us mention
first the capacity to engage a wide variety of international and national actors, even on sensitive issues,
in a respectful and evidence-based way, and progressively build partnerships. For example, at global level,
EVALPARTNERS, EVALSDGs, and GPFE have all been key vehicles to promote partnerships. At country
level, it is clear that national government and non-government actors own the intervention and are
involved at every step of the process. This provides the basis for building large coalitions, focus efforts
and resources, and create a momentum for reform.
Second, successful NECD interventions benefit a lot from work at all three levels (global, regional, and
country) and the interlinkages between those levels. The global level provides a global perspective and an
appreciation of commonalities and differences across regions and there are trickle down effects at
regional and country levels. For example, the spotlights initiative of EVALSDGs had significant trickle-down
effects at the national level. The regional level provides a benchmarking across countries in the region, an
opportunity to learn from more similar countries, and a conduit of communication between the global
and the country levels. The country level provides concrete evidence to the regional and global levels on
the implementation and results of various NECD interventions in different country contexts. Country
Offices have the local knowledge and adapt to the specific country conditions, e.g. to the selection of the
right entry point and seizing opportunities, the identification of champions, and the use of local experts.
The capacity to work at all levels from global to sub-national and relate those levels for cross-fertilization
also adds value as exemplified by the GPFE leading to country parliamentary initiatives or the EAPRO
UNDP-UNICEF NECD intervention leading to more countries of the region jumping on the band wagon of
national M&E system reviews.
Third, NECD interventions suppose competent interveners. Solid in-house expertise in evaluation and
using quality regional and international evaluation experts make a world of difference for credibility and
results as exemplified time and time again in many case studies.
Fourth, the integration of NECD in strategic plans, work plans, evaluation frameworks and M&E systems
goes a long way in making it a central concern, its systematization and institutionalization. An NECD
intervention should not be seen as an isolated project per se, but a key building block for promoting and
solidifying the evaluation function in the whole management cycle, and this at all levels.
24 The details of good practices and lessons learnt for each global or regional case study are presented in Annex 4. 25 The details of good practices and lessons learnt for each country case study are presented in Annex 5.
IDEA International & IDEA-Solutions 27
Fifth, UNICEF corporate culture of “putting things on paper” and documenting experiences in quality
publications and briefings helps keep an institutional memory, disseminate good practices, learn lessons
and adjust, and advocate more credibly based on evidence. For example, EVALSDGs advocacy briefings
have demonstrated to have a wide reach, to be digestible by evaluators, and to be highly influential.
In terms of commonly encountered lessons learnt, a recurrent theme is that the objectives and scope of
any NECD intervention need to be realistic and take into account a diversity of factors: the slowness of
bureaucratic processes in the public sector, the importance of incentives to vanquish vested interests and
resistance to change, the need to accept the legal, political, institutional and cultural dimensions of
evaluation and the complexity of building successful partnerships. For example, the EAPRO UNDP-UNICEF
joint initiative in the Asia-Pacific Region was too optimistic in terms of timeline of output production.
However, the longer time needed to obtain the first outputs was not lost since it helped national
appropriation and dialogue endogenization, thereby building the conditions for more sustainability.
Second, NECD cannot be summed up in a short run intervention; rather it has to be designed and managed
as a medium-term process with a progressive expansion of the scope and sustainability of results through
a programme and a portfolio approach and an institutionalization perspective. The country cases of
Kyrgyzstan and Nigeria demonstrate the value of a multi-faceted and stepwise approach to NECD in which
both evaluation demand and supply are stimulated
Third, success starts at the phase of design and selection of the NECD intervention. The objectives, the
targets and the roadmap need also to clearly defined with a clear Theory of Change. Given resource
constraints, hard choices must be made. Although there is no unique route, most often, the first element
to address in NECD is the enabling environment, then the individual, institutional and ICT dimensions can
be strengthened as clearly identified in the Nigeria case study.
Fourth, there are also some specific lessons depending on the level of capacity-building. At global level, it
is often necessary to “break the silos” and push each partner organization to move beyond its mandate
and rather create a common platform with different perspectives, e.g. the EVALPARTNERS case study. At
institutional level, government ownership is essential for the establishment of an evaluation culture, e.g.
the Philippines case study. At individual level, professionalization of evaluation is a vital component of
NECD, e.g. the Kyrgyzstan, Morocco and Rwanda and Sri-Lanka case studies.
4.5. COMPARATIVE ADVANTAGE OF UNICEF
Notwithstanding the qualitative nature of the data based on perceptions of respondents, there is an
impressive degree of commonality behind the analysis made by respondents on the comparative
advantage of UNICEF, be them from UNICEF staff or from counterpart or partner organizations. Actually,
UNICEF staff tends to be harsher on themselves than their counterparts and partners who all stress the
critical contribution made by UNICEF to NECD. It is obvious to all that UNICEF has been one of the leaders
on NECD among international organizations. It possesses a rich experience in NECD and expertise in NECD
at all levels: global, regional, and country.
IDEA International & IDEA-Solutions 28
Two main comparative advantages rise above the specificities of each NECD intervention26. First, UNICEF
has an uncanny capacity to engage and bring together different types of partners into NECD, i.e.
government, parliamentarians, NGOs, academics, private sector, sub-national actors, international
organizations and other development agencies, etc., and to provide a platform for interaction and
information-sharing among all those interested in M&E. UNICEF is a trusted actor at all levels, recognized
for its competencies in evaluation and its role as an honest broker. Its focus on children is welcome by all,
imposes a cross-cutting approach, and is totally aligned with the 2030 Agenda agreed by all countries.
Second, UNICEF is a relatively strong organization in evaluation able to intervene from global level to sub-
national level through its three-tier organization. It has a capacity to lead or co-lead interventions at all
levels. It is a knowledge-based organization which can easily transfer skills to the locals with high quality
intellectual, ethical and professional standards in M&E. It can count on a set of competent and committed
professionals in evaluation within the organization or, as one counterpart puts it, “a passionate team of
individuals to whom NECD strikes a sensible cord”. It can also mobilize a wide network of professionals,
organizations, and networks around the world who are expert in evaluation and NECD. It has accumulated,
documented and disseminated knowledge on NECD through publications, toolkits, guidelines, etc. By far
and large, UNICEF-supported NECD interventions have been perceived as high performing.
26 Please see Annexes 4 and 5 for the analysis of comparative advantage of UNICEF in each case study.
IDEA International & IDEA-Solutions 29
RECOMMENDATIONS 5
IDEA International & IDEA-Solutions 30
SECTION 5: RECOMMENDATIONS
In conclusion, UNICEF has accumulated good experience in NECD due to its commitment to SDG and child
focus, working at country, regional and global levels, in partnership with key actors and networks in
evaluation. There is a significant demand of continued leadership from UNICEF by interviewed
counterparts and partners in the NECD intervention case studies. UNICEF brings a unique contribution to
NECD through its child-oriented lens (intersectoral, target group of all social policies key to SDGs) and an
ability to work at global, regional, country and subnational levels and engage a variety of partners in solid
partnerships. Work on NECD also brings visibility and credibility to UNICEF and to its Evaluation Office.
The partial evidence gathered through this mapping exercise would argue for a continued and
institutionalized role for UNICEF to support articulated NECD interventions at global, regional, and country
levels with a child focus in the 2030 Agenda to obtain more sizable results in a more sustainable way.
Six specific recommendations are made (relevant UNICEF actors are indicated in parentheses:
Headquarters – HQ, Regional Offices – ROs, Country Offices - COs):
Recommendation 1 (HQ, ROs): Position NECD in relation with the 2030 Agenda in key strategic
documents, including in the forthcoming UNICEF Evaluation Policy and Regional Evaluation Strategies.
This should include among others a clear definition of NECD and its role for the implementation of the
2030 Agenda, guidance on approaches and selection criteria to be used by UNICEF approaches; a clear
description of the linkages between global, regional and country level NECD interventions, and the generic
roles of UNICEF HQ, RO, CO as a team and its partners and counterparts. This would contribute to
institutionalization and facilitate fund mobilization.
Recommendation 2 (HQ, ROs): Continue and intensify its partnership approach in NECD at global and
regional levels and take part in promising NECD and related initiatives initiated by other organizations,
in particular with UN organizations, but also bilateral donors and other actors, i.e. initiatives on data
generation and statistics, Voluntary National Reviews, etc. UNICEF has spearheaded several important
global and regional partnerships (EVALPARTNERS, EVALSDGs, GPFE, EAPRO UNDP-UNICEF joint initiative
in the Asia-Pacific Region) which should be continued with added commitments and resources. Such
partnerships provide a more integrated approach, institutional traction, help mobilize resources and go
through political cycles, lead to more sustainable results, and invite reciprocity. MoUs could be signed
when relevant with key partners, especially at global level.
Recommendation 3 (HQ, ROs): Have HQ and ROs support NECD interventions at lower levels27 where
those high-level actors can make a substantial difference, i.e. help deliver concrete milestones and where
the global or regional perspective is eye-opening, stimulates positive competition among countries,
enables to seize economies of scale. A two-tier approach is suggested combining basic services to all
country offices (sharing practices, promoting South-South and triangular cooperation, documenting case-
27 Lower levels would mean ROs and COs for HQ and Cos for ROs.
IDEA International & IDEA-Solutions 31
studies with support from Evaluation Office) and more intense coaching of selected ROs and COs with
higher expressed interest (demand) and high potential for growth in evaluation capacity (strategic).
Recommendation 4 (HQ, ROs, COs): Consider a variety of possible NECD delivery mechanisms,
combining involvement and leadership of internal staff with performance contracts (LTA type) to regional
and international organizations, whether public, private or non-for-profit, that have a track record of
supporting NECD at individual, institutional, and enabling environment levels.
Recommendation 5 (HQ, ROs, COs): Continue improving the capacity of UNICEF to support NECD in a
sustainable way. This may mean more training and coaching of M&E staff, more systematic sharing of
good practices and lessons learnt from more experienced ROs and COs to the still significant number of
ROs and COs that are not so advanced. Some tools for better selection and monitoring and evaluation of
NECD interventions could also be introduced such as a go/no go grid at country level28 to select NECD
interventions to be supported. Those capacity-building initiatives might be more attractive with financial
incentives such as an innovation fund for new promising NECD interventions to award grants on a
competitive basis among COs.
Recommendation 6 (HQ): Put in place a proper monitoring and documentation system of NECD
interventions on a continued basis. The database of NECD interventions developed within this mapping
exercise can be used as a springboard to develop a web-based information system of the UNICEF-
supported NECD interventions portfolio with decentralized access to a large number of stakeholders and
business intelligence (BI) tools to increase its potential for analysis and its user-friendliness. This data base
and supporting BI tools could be made available to other international organizations to share costs and
benefit from economies of scale as well as obtain a more global perspective on NECD interventions.
28 An example of possible go/no go grid at country level is presented in Annex 6.
IDEA International & IDEA-Solutions 32
ANNEXES
IDEA International & IDEA-Solutions 33
ANNEX 1 – ORGANIZATION OF THE MAPPING EXERCISE
a. Project Manager
• Mrs. Ada Ocampo
Senior Evaluation Specialist
Evaluation Office
UNICEF New York
b. Reference Group
Name Position E-mail
Tetyana Nikitina UNICEF Sri Lanka [email protected]
Hammad Masood UNICEF Angola [email protected]
Justus Kamwesigye UNICEF ESARO [email protected]
Riccardo Polastro UNICEF EAPRO [email protected]
Fabio Sabatini Evaluation Office [email protected]
Ziad Moussa EVALPARTNERS Co-chair [email protected]
Dorothy Lucks EVALSDGs Co-chair [email protected]
Benoit Gauthier IOCE [email protected]
Javier Guarnizo UNIDO [email protected]
Brook Boyer UNITAR [email protected]
c. Consultant Team
• Dr. Frederic Martin
Team Leader
• Mrs. Ana Maria Fernandez
Senior Evaluation Specialist
• Mrs. Giovanna Donado
Senior Evaluation Specialist
• Mr. Jacques Lefevre
Senior IT Specialist
IDEA International & IDEA-Solutions 34
d. List of persons interviewed during the inception phase
Name Position E-mail
Fabio Sabatini Senior Advisor, Research and Evaluation Office [email protected]
Koorosh Raffii Senior Evaluation Specialist, Evaluation Office [email protected]
Isa Achoba Associate Director Field Results Group (FRG) [email protected]
Laura Olsen Evaluation Specialist, Evaluation Office [email protected]
Lovemore
Mhuriyengwe
Knowledge Management Specialist, Evaluation
Office
Matthew Varghese Sr. Evaluation Specialist, Evaluation Office [email protected]
IDEA International & IDEA-Solutions 35
ANNEX 2: PROPOSED THEORY OF CHANGE FOR NECD
A generic Theory of Change (TOC) is presented as contextual information for this mapping exercise in
Figures 2.1, 2.2, and 2.3, respectively for enabling environment, institutional, and individual dimensions
of capacity development
IDEA International & IDEA-Solutions 36
Figure A2.1: Proposed Theory of Change for NECD at Enabling Environment Level
Assumptions/Risk factors
UNICEF budget allocations for enabling environment NECD Trained and experienced UNICEF staff in enabling
environment NECD Committed UNICEF staff
Working strategic alliances and partnerships
Political stability Political and technical leadership in key institutions and partners
Policymaking coordination across institutions Enabling international environment
Limited resistance to change in key institutions
Impacts
Enabling environment
level
Final beneficiaries
Activities
Outputs Immediate Outcomes
Final Outcomes
Technical and financial support and advocacy with right selection of:
• Strategic orientations
• Target groups (institutions, committees, fora, etc.)
• Support areas corresponding to critical NECD gaps and UNICEF comparative advantage
• Cross cutting issues related to child rights such as human rights, gender equality, and equity and SDGs
• Strategic alliances and partnerships
• Support institutions and experts
• Support systems and tools as well as approach
• Duration and timing
• Sequencing and articulation with other NECD activities
• Adequate funding of NECD activities
• Adequate management of NECD activities (planning, support during execution, monitoring and adjustment)
Reduced critical capacity gaps in
terms of formulation and
application of evaluation policies,
institutional processes, evaluation
capabilities, and partnerships, to
promote the implementation of
child rights and SDGs at global,
regional and national levels
Contribution to an increased
quantity, variety, and quality of
evaluations, and the dissemination of an evaluation
culture in policies, programs, and
projects to promote the
implementation of child rights and
the achievement of SDGs at global,
regional, and national levels
Contribution to the
achievement of SDGs at global,
regional and national levels
Contribution to an
increased use of evaluations to formulate
and revise policies,
programs and projects
related to child rights and SDGs at
global, regional, and
national levels
Contribution to the
improvement of the situation of
children and women at
global, regional and national
levels
IDEA International & IDEA-Solutions 37
Figure A2.2: Proposed Theory of Change for NECD at Institutional Target Group Level
Impacts
Institutional level
Final beneficiaries
Activities
Outputs Immediate Outcomes
Final Outcomes
Technical and financial support and advocacy with right selection of:
• Strategic orientations
• Target groups (institutions)
• Support areas corresponding to critical NECD gaps in key institutions and to comparative advantage of UNICEF
• Cross cutting issues related to child rights such as human rights, gender equality, equity, and SDGs
• Partners
• Support institutions and experts
• Support systems and tools as well as approach
• Duration and timing
• Sequencing and articulation with other NECD activities
• Adequate funding of NECD activities
• Adequate management of NECD activities (planning, support during execution, monitoring and adjustment)
Reduced critical NECD gaps of target institutions in terms
of strategic orientations, institutional
arrangements, technical systems for
the evaluation of policies, programs,
and projects to promote the
implementation of child rights and the
achievement of SDGs at global, regional, and
national levels
Assumptions/Risk factors
UNICEF budget allocations for institutional NECD Trained and experienced UNICEF staff in institutional NECD
Committed UNICEF staff Working partnerships
Existence and endurance of champions of change in key target institutions
Limited resistance to change in key target institutions UNICEF leadership
Existence and endurance of champions of change in key target institutions
Enabling environment
Contribution to the
improvement of the situation of
children and women at
global, regional and national
levels
Contribution to an increased use of evaluations by key institutions
in policies, programs and
projects related to child rights and SDGs to
improve client orientation,
service delivery processes, HR
management, FR management, Information
systems management
Contribution to an increased
quantity, variety, and quality of
evaluations, and the dissemination of an evaluation
culture in policies,
programs, and projects to
promote the implementation
of child rights and the achievement of SDGs at global,
regional, and national levels
Contribution to the
achievement of SDGs at
global, regional and
national levels
IDEA International & IDEA-Solutions 38
Figure A2.3: Proposed Theory of Change for NECD at Individual Target Group29 Level
29 The individual target group level refers to key staff in key institutions and to comparative advantage of UNICEF. They can be working for national, regional, or global institutions, including UNICEF
Impacts Activities
Outputs Immediate Outcomes
Individual level
Institutional level
Final beneficiaries
Final Outcomes
Technical and financial support and advocacy with right selection of:
• Target groups (institutions, participants)
• Sensitization and training areas corresponding to critical evaluation capacity gaps of key staff in key institutions and to comparative advantage of UNICEF
• Cross cutting issues related to child rights such as human rights, gender equality, equity, and SDGs
• Partners
• Sensitization and training institutions and trainers
• Sensitization and training material and approach
• Venue and logistics
• Duration and timing
• Sequencing and articulation with other NECD activities
• Adequate funding of NECD activities
• Adequate management of NECD activities (planning, support during execution, monitoring)
Reduced critical capacity gaps in
terms of knowledge, skills,
attitudes, experience and
behaviour of key staff in key
institutions for the evaluation of
policies, programs, and projects to
promote the implementation of
child rights and the achievement of SDGs at global,
regional, and national levels
Contribution to the improvement of the situation of children and
women at global, regional and
national levels
Assumptions/Risk factors
Contribution to an increased use of evaluations by key institutions in
policies, programs and
projects related to child rights and SDGs to
improve client orientation,
service delivery processes, HR
management, FR management, Information
systems management
UNICEF budget allocations for individual NECD Trained and experienced UNICEF staff in individual NECD
Committed UNICEF staff Working partnerships
Limited capacitated target group rotation Managers in key national institutions ready to use
capacitated individuals Limited resistance to change in key institutions
UNCEF leadership
Existence and endurance of champions of change in key institutions
Enabling environment
Contribution to an increased
quantity, variety, and
quality of evaluations,
and the dissemination
of an evaluation culture in policies,
programs, and projects to
promote the implementation
of child rights and the
achievement of SDGs at global, regional, and
national levels
Contribution to the
achievement of SDGs at
global, regional and
national levels
IDEA International & IDEA-Solutions 39
ANNEX 3 – DATA COLLECTION INSTRUMENTS
There are two data collection instruments:
1. The Online survey questionnaire;
2. The In-depth Interview questionnaire.
The basic questionnaire was the same, but there were some differences. However, the online survey,
questionnaire which aimed at identifying NECD interventions, was administered to UNICEF M&E staff at
global, regional and country levels who could identify and document up to three NECD interventions.
There were very few open questions in this questionnaire. The in-depth interview questionnaire, which
aimed at documenting specific NECD interventions considered as case studies, was administered partly
on line, partly in an interview to key actors from both UNICEF, partners and/or counterparts.
Key sections of the basic questionnaire are presented below.
SECTION 1: CHARACTERIZATION OF UNICEF-SUPPORTED NECD INTERVENTION X
NECD X
Name
Responsible agency or unit
Location
Beneficiaries
Partners (if any)
Main objectives
Actual start year
Actual or planned end year
Indicative duration (in months)
Indicative budget (USD000’)
Main outputs
Main immediate outcomes
Levels of NECD intervention NECD X
Global
Regional
National
Sector
Sub-national
IDEA International & IDEA-Solutions 40
Type of country NECD X
Middle income country
Low income country
Fragile
Other (specify):
Work domain in which the NECD took place NECD X
Humanitarian
Development
Target group of NECD intervention NECD X
Government
Parliament
Private sector
NGOs
CSOs
Academia
Other (specify):
Dimension of NECD NECD X
Enabling environment
Institutional level
Individual level
Objectives of NECD NECD X
Contribute to the development of national evaluation policies or frameworks
Support the inclusion of evaluation in constitutions or other legal frameworks
Contribute to crafting national or sectoral evaluation plans and agendas
Support the establishment of evaluation bodies within ministries or other national
institutions
Build up partnerships to conduct NECD
Support the conceptualization and implementation of country-led evaluations
Develop/revise curricular in evaluation in higher/vocational education institutions
Provide training and professional development opportunities on evaluation
Develop and conduct awareness raising and advocacy activities for NECD
Conduct advocacy for budgetary allocations for evaluation
Raise funding for NECD
Other (specify):
IDEA International & IDEA-Solutions 41
Explicit mention of crosscutting dimensions in NECD NECD X
Human rights
Equity
Gender equality
SDGs
Other (specify):
UNICEF role NECD X
Undertaken independently by UNICEF
Undertaken with partner organizations with UNICEF as a leader or coordinator
Undertaken with partner organizations with UNICEF as a supporting role
Partner(s) of UNICEF in NECD intervention NECD X
International organizations
International fora
Government
Parliament
Private sector
NGOs
CSOs
Academia
Other (specify)
Please add any additional information you see relevant for the sake of characterization of NECD
intervention X
SECTION 2: PERCEIVED PERFORMANCE OF UNICEF-SUPPORTED NECD INTERVENTION X
Extent to which NECD intervention X is in line with global reference documents
Possible grades and meaning:
0 No alignment
1 Partial and indirect alignment
2 Partial, but direct alignment
3 Full and direct alignment
NA Not able to answer
IDEA International & IDEA-Solutions 42
Please indicate the relevant grade for each NECD intervention.
Global Reference Document NECD X
UNICEF Evaluation Policy
UNICEF Strategic Plan, 2014-2017
GA resolution on National Evaluation Capacity Development
GA resolution on the Agenda 2030
Global Evaluation Agenda
UNEG Norms and Standards
Other (specify):
Performance criteria of UNICEF-supported NECD intervention X
Possible grades and meaning:
0 Low
1 Somewhat
2 To a significant extent
3 High
NA Not able to answer
Please assess each NECD intervention according to performance criteria indicated below.
Performance criteria NECD X
Relevance to meet NECD critical gaps
Effectiveness
Efficiency
Outputs
Immediate outcomes
Sustainability of increased capacity built by NECD intervention
Why? (the reasons behind the observed performance, for example analysis of factors supporting or
constraining successful outcomes; the context for interventions, including opportunities and risks, etc.)
UNICEF comparative advantage in terms of NECD
Please list major comparative advantages of UNICEF for key UNICEF NECD interventions. Please prioritize
the comparative advantages, with 1 being the most important one.
1.
2.
3.
IDEA International & IDEA-Solutions 43
Good practices and lessons learned on NECD
Please list major good practices from key UNICEF NECD interventions. If the answer differs significantly
among the listed NECD interventions you selected, please say how.
1.
2.
3.
Please list major lessons learnt from key UNICEF NECD interventions. If the answer differs significantly
among the listed NECD interventions you selected, please say how.
1.
2.
3.
Recommendations for future NECD interventions
Please indicate a set of realistic, affordable and practical recommendations for NECD support. If the
recommendation is made to a specific target group, please indicate the relevant target groups.
Recommendation
1.
2.
3.
Please indicate below any other suggestions or comments useful for this mapping exercise on UNICEF’s
support to national evaluation capacity development (NECD)
IDEA International & IDEA-Solutions 44
ANNEX 4 – CASE STUDIES OF NECD INTERVENTIONS AT GLOBAL AND REGIONAL LEVELS
NECD INTERVENTION: EVALPARTNERS
Characterization of the NECD intervention
Responsible agency or unit
(mechanism responsible)
UNEG and IOCE
Location Global (formally registered in Ottawa, Canada)
Beneficiaries
Governments, parliamentarians, Voluntary Organizations of
Professionals in Evaluation (VOPEs), UN organizations, bilateral
organizations.
Partners 60 partners who are at the same time the beneficiaries listed above.
Main objectives 1. Strengthen national evaluation capacities
2. Improve civil society evaluation capacities
Actual start year 2012
Performance of the NECD intervention
This NECD intervention has demonstrated good performance overall. Its relevance is obvious since
EVALPARTNERS is a global partnership to strengthen national evaluation capacities. It has been largely
effective in terms of producing outputs. It has led to several publications such as advocacy briefing notes,
organization of conferences, fora, peer to peer exchanges, e- learning, e-learning platform and initiatives,
partnerships. EVALPARTNERS has influenced several initiatives: together with UNEG, it advocated for and
influenced the GA Resolution on National Evaluation Capacity Building. It worked closely with UNEG to
advocate for the incorporation of Evaluation in the Agenda 2030. It launched the Global Evaluation
Agenda 2016-2020 after a consultative multi-stakeholder process. It helped establish networks supporting
two streams: EVALSDGs and the Global Parliamentarians Forum for Evaluation (GPFE), and through those
networks, EVALPARTNERS has engaged parliamentarians, governments and other key actors in the
promotion of evaluation as an important element for good governance. A growing number of parliaments
have incorporated evaluation in national legislation or constitutions. A number of National Evaluation
Policies (NEP) have been elaborated and adopted at country level. Nowadays, EVALPARTNERS acts as an
umbrella for several global networks: EVALGENDER+, EVALSDGs, the Global Parliamentarians Forum for
Evaluation (GPFE), EvalIndigenous and EvalYouth. EVALPARTNERS has clearly strengthened the enabling
environment for evaluation by increasing the demand and the use of evaluation and demonstrated the
power of partnerships for influencing global agendas.
IDEA International & IDEA-Solutions 45
Its efficiency has been assessed as relatively good, although some respondents said it could be better
given the amount of resources invested. One complicating factor in the analysis is that it is hard to
differentiate between the inputs and the outputs of a given initiative supported by EVALPARTNERS and
the overall inputs and outputs of EVALPARTNERS per se.
UNICEF has played a major role in EVALPARTNERS, among others the promotion of partnerships. UNICEF
created EVALPARTNERS and seats at the management group. This corresponds to UNICEF strategy and it
also is consistent with the mandate from UNEG to push Strategic Objective 4 on Partnerships. Enabling
factors have been the good buy in and support from UNICEF management on the importance in ECD and
resource mobilization for NECD, as well as key contributions of several UNICEF professionals.
Extent to which the NECD intervention is in line with important reference documents
There is a large alignment of the NECD intervention with key reference documents such as the UNICEF
Evaluation Policy, the GA resolution on National Evaluation Capacity Development, the Global Evaluation
Agenda, the UNICEF Strategic Plan 2014-2017, and UNEG Norms and Standards. This is to be expected
from an NECD intervention associating key global stakeholders.
Extent to which the NECD intervention has given attention to human rights, equity, gender equality,
and SDGs
The NECD intervention considers a variety of cross-cutting dimensions such as human rights, equity,
gender equality, SDGs, and indigenous rights.
Good practices on NECD
Good practices noted in this NECD intervention include: i) a corporate culture of “putting things on paper”,
i.e. publications such as books, policy briefs. This practice can be useful for other NECD actors, in particular
several CSOs which tend to be less organized in their documentation of their activities, processes, and
results, ii) a capacity to touch upon and bring to the policymakers’ table a variety of complex, sensitive,
and cross-cutting issues such as employment, youth protection, gender-based violence, and, iii) a
capability to work at all levels and relate them.
Lessons learnt on NECD
Several lessons can be learnt from this NECD intervention. At enabling environment level, EVALPARTNERS
has a key role to play to contribute to “breaking the silos” and pushing each organization to move beyond
its mandate and rather create a common platform with different perspectives. EVALPARTNERS could be
more involved in global multi-stakeholder partnerships where lies its main comparative advantage and
less at national level to avoid spreading out its resources on too many activities. Among global issues it
could push further, alignment with the Agenda 2030 is critical to move the evaluation agenda forward.
More work could also be done to push evaluation in humanitarian and fragile contexts. At institutional
IDEA International & IDEA-Solutions 46
level, government ownership is essential for the establishment of an evaluation culture. At individual
level, professionalization of evaluation is a vital component of NECD.
UNICEF comparative advantage in NECD
The major comparative advantage of UNICEF in term of NECD is its capacity to bring together actors. First,
UNICEF is a trusted actor at all levels, recognized for its competencies in evaluation and its role as an
honest broker. It does engage with a variety of actors: government, private sector, civil society and VOPES,
at all levels, from global level to regional, national and sub-national levels thanks to its wide network of
regional and country offices. It can count on a set of competent and committed professionals in
evaluation. As one counterpart puts it, “a passionate team of individuals to whom NECD strikes a sensible
cord”.
Recommendations to UNICEF
Recommendations are to: i) continue its implication and allocate more resources; ii) position NECD in
relation with the 2030 Agenda in the forthcoming UNICEF Evaluation Policy for more institutionalization
and sustainability; iii) push for the inclusion of NECD in Regional Evaluation Strategies; iv) facilitate peer-
to-peer learning on NECD; and v) promote the institutionalization of all good practices.
Final conclusion
EVALPARTNERS has proven to be a very good NECD intervention to foster global NECD initiatives and
partnerships for a more coordinated NECD. It has offered a positive ecosystem and institutional
framework for several networks and NECD interventions. Efficiency and sustainability could be somewhat
improved. It could focus more at the global level in its advocacy efforts, and at the same time, promote
and support enabling environments for evaluation at regional and national levels especially, especially in
the context of the 2030 Agenda.
IDEA International & IDEA-Solutions 47
NECD INTERVENTION: EVALSDGS
Characterization of the NECD intervention
Responsible agency or unit IOCE and UNICEF/Evaluation Office
Location Global
Beneficiaries Parliamentarians, policy makers, UN, international organization,
academia, VOPEs, National Governments
Partners Global network under the umbrella of EVALPARTNERS
Main objective
1. Promote evaluation activities around the SDGs
2. Orient and support the review and follow-up process on the SDG
2030 Agenda by adding value through evaluation.
Actual start year 2015
Performance of the NECD intervention
This NECD intervention has demonstrated good performance, especially if one takes into account its
limited budget for a huge mandate. It is obvious that relevance is high given the importance of evaluation
and of SDGs. Let us note that the focus is not only on the evaluation of the achievement of SDGs per se,
but on supporting the demand for and the supply of evaluations of policies and programs that contribute
to SDGs, and even further than that contribute to the awareness about the evaluation function and the
dissemination of an evaluation culture.
It is also clear that this intervention has been effective and efficient with a wide variety of outputs
produced for little money: i) several important publications, including guidance documents and eight
briefing advocacy papers launched on evaluation and SDGs with various partners e.g. IEG/World Bank and
UNDP. The Briefing papers are a joint venture between IIED and EVALSDGs and are among the most
downloaded papers from the IIED website; ii) identification of spotlight countries for the preparation
towards the evaluation of the SDGs, provision of technical support to those countries, and documentation;
iii) communication and learning platform with monthly webinars on country case studies; iv) organization
of regional consultations in partnership with the Global Parliamentarians Forum for Evaluation with
orientation provided to countries to develop plans to establish national evaluation policies in line with the
agenda 2030; v) advocacy for the establishment of frameworks for SDGs evaluation and how to influence
the Voluntary National Reviews (VNRs) which are becoming an important part of the SDG evaluation
process.
It is way too early to talk about final outcomes, however EVALSDGs has already demonstrated its potential
of value added to the Agenda 2030 through the orientation on and the promotion of Evaluation, its
IDEA International & IDEA-Solutions 48
platform for dialogue, advocacy and knowledge generation in relation to evaluation and the SDGs and its
contribution to a better appreciation by stakeholders of inter-linkages and interaction between initiatives
to strengthen evaluation and implement and review the SDGs.
The challenge is sustainability. Even if the networking capacity, the voluntary work and the support from
UNICEF are appreciated, the budget is low (USD 30.000-40.000 annually) which reduces the capacity to
conduct more activities while the agenda is large for a small group of voluntary people, which means the
choice of activities needs to strategic.
Extent to which the NECD intervention is in line with important reference documents
This NECD intervention is aligned with important global strategic documents and policy guidelines. This is
to be expected given its focus at global enabling environment level and the importance of SDGs for the
UN organizations, including UNICEF.
Extent to which the NECD intervention has given attention to human rights, equity, gender equality,
and SDGs
The NECD intervention considers human rights, equity, gender equality, SDGs and environment as cross-
cutting dimensions.
Good practices on NECD
Good practices on NECD are: i) advocacy briefings which have demonstrated to have a wide reach, are
digestible by evaluators, and to be highly influential; ii) the partnership of EVALSDGs with the GPFE
enhances results at the national level; iii) the spotlights initiative which had trickle down effects at the
national level.
Lessons learnt on NECD
Lessons learnt from this NECD point to: i) the need to work more in establishing clear linkages between
global, regional and national levels with mutually beneficial effects. The global level provides a global
perspective and an appreciation of commonalities and differences across regions and there are trickle
down effects at regional and country levels. The regional level provides a benchmarking across countries
in the region, an opportunity to learn from more similar countries, and a conduit of communication
between the global and the country levels. The country level provides concrete evidence to the regional
and global levels on the implementation and results of various NECD interventions in different country
contexts; ii) the importance of human and financial resources to carry out interventions at global level.
The scope of outputs and outcomes is obviously limited by the resource constraints.
IDEA International & IDEA-Solutions 49
UNICEF comparative advantage in NECD
UNICEF comparative advantage in terms of NECD are: i) a strong evaluation practice within the
organization. The technical contents of evaluations conducted at UNICEF follow international standards;
ii) its focus on children. The “No one left behind” strategic goal is clearly embedded in the organization
which focuses on the marginalized with an integrated perspective; iii) its capacity to lead or co-lead global
initiatives; and, iv) its wide network of M&E professionals at the global level. UNICEF has provided
invaluable support to this NECD intervention, mainly in the form of professional time and facilities.
Recommendations to UNICEF
Recommendations made to UNICEF are to: i) incorporate NECD and its relation with the 2030 Agenda in
the forthcoming UNICEF Evaluation Policy; ii) assign annual resources to NECD with a focus on reinforcing
capacities for the evaluation of the SDGs; iii) consider bringing child-focus evaluation to the EVALSDGs
initiative by developing work around thematic areas such as how to do you evaluate SDGs from an
integrated point of view focusing on the nexus around children.
Final conclusion
This recent NECD intervention is dealing with a major issue not only for UNICEF, but also for the UN system
and all other development actors. With limited budget, it has contributed significant and useful outputs,
including advocacy briefing papers, identification of spotlight countries for the preparation towards the
evaluation of the SDGs, monthly webinars on country case studies, organization of regional consultations,
and advocacy for the establishment of frameworks for SDGs evaluation and how to influence the
Voluntary National Reviews (VNRs). One main strength of the approach has been to work in partnerships.
UNICEF has provided limited financial support, but critical indirect support in terms of logistics and direct
institutional support.
IDEA International & IDEA-Solutions 50
NECD INTERVENTION: GLOBAL PARLIAMENTARIANS FORUM FOR EVALUATION (GPFE)
Note: the GPFE was initially known as the Parliamentarians’ Forum for Development Evaluation (PFDE).
Characterization of the NECD intervention
Responsible agency or
unit
Network under EVALPARTNERS headed by a Steering Committee
supported by a Secretariat
Location Global (current Chair is in Colombo, Sri Lanka)
Beneficiaries Parliaments, parliamentarians, governments, and civil society
Partners
Parliaments, EVALPARTNERS, EVALSDGs, EvalGender, World Bank IEG,
African Development Bank, CLEAR Centres, UNICEF and other UN
agencies for specific activities.
Main objectives
1. Strengthen the demand and the use of evaluation by
parliamentarians.
2. Create enabling environments for nationally owned, transparent,
systematic evaluation processes.
3. Promote national evaluation policies and systems.
Actual start year • 2013 for the first ever Regional Parliamentarians Forum for South Asia
launched in in Colombo.
• 2015 for the launch of the Global Movement the main event
celebrating EvalYear 2015
Performance of the NECD intervention
This NECD intervention enjoys a high performance. It is obviously relevant given the key role of the
legislative branch in approving the laws, policies, and budget proposed by the executive branch in
democratic countries. Parliamentarians are the ideal actors to perform an important role to ensure budget
allocations are justified by evaluated development results, to trigger the evaluation agenda, to request
specific evaluations and to promote a culture of evaluation at national level. Through capacity building,
knowledge sharing and advocacy, GPFE empowers parliamentarians to deliver evidence-based policies
and thus contributes to good governance, sustainable development and social equity.
The GPFE has been effective in producing a number of outputs, including: 1) in terms of advocacy
activities, the GPFE Launching New Website & Social Media- Parliament of Nepal on 25th November
201530, a video campaign inviting parliamentarians around the globe to create video messages on the
30 https://www.europeanevaluation.org/forums/general-discussion/global-parliamentarians-forum-evaluation-
gpfe-launching-new-website-social and https://globalparliamentarianforum.wordpress.com/
IDEA International & IDEA-Solutions 51
importance of National Evaluation Policies (NEP) which resulted in 38 videos available in multiple
languages, advocacy products e.g. “10 reasons why parliamentarians should be involved in evaluation”
and the “Meet a parliamentarian” webinar series; 2) in terms of creating and sharing knowledge: mapping
on NEPs, regional consultations and partnerships on NEPs, case studies on National M&E systems with
gender and equity focused lens.
There are also already visible outcomes in terms of increased awareness of parliamentarians on the
importance of evaluation which led to the establishment of regional parliamentarians’ fora and changes
in the legal framework of countries to take into account evaluation. Between 2014-2015, regional
parliamentarians’ fora were created in Africa, East Asia, Latin America and MENA regions. The first ever
Parliamentarians’ Forum for Development Evaluation (PFDE) was established in South Asia in early 2013.
The African Parliamentarians Network on Development Evaluation (APNODE) was initiated at the AfrEA
conference held in Yaoundé, Cameroon in March 2014 after one year of initiating PFDE. APNODE is hosted
and supported by the African Development Bank. Morocco and Ivory Coast have included evaluation as a
necessity to implement legislation and national systems in their constitutions. In Nepal, evaluation is
included directly in the legislation. In Tunisia, it is included in the parliament regulation. Other Parliaments
have taken specific actions to consider evaluation in key domains: Pakistan’s Parliament established an
SDG Secretariat to help parliamentarians effectively oversee SDG progress, address the legislative gaps
and ensure their constituents’ rights. In Trinidad and Tobago, the Parliament established a new Joint
Select Committee on the Environment and Sustainable Development. In Zambia, MPs informed a SDGs
caucus. In Sri Lanka, the Parliament established a Select Committee on the SDGs. In Finland, MPs were
consulted during the preparation of the first Voluntary National Review, chair the Finnish Development
Policy Committee that monitors and assesses implementation of Finland’s international commitments,
and they are widely represented in the National Commission of Sustainable Development that governs
implementation of the SDGs. In the United Kingdom, a Parliament Committee has conducted a critical
inquiry that encouraged a shift in government approach to addressing the SDGs. Parliamentarians have
also spearheaded major evaluation events in national and regional parliaments including in Nepal, Sri
Lanka, Tunisia, Kyrgyz Republic and in the Latin American parliament called “Parlatino”. There are
parliamentarians’ groups on evaluation in the Middle East and North Africa, East Asia, Latin America and
Eurasia. National parliamentarians’ fora on evaluation have been launched in Nepal, Uganda, Kenya and
Sri Lanka. NEPS have been elaborated and validated in Parliament in a number of countries. It is too early
to talk about final outcomes, but immediate outcomes are definitely there.
In terms of sustainability, the constitutional nature of the Parliament, the interest and involvement of
MPS, the support of EVALPARTNERs and from UNICEF since the beginning are contributing factors.
However, other factors can negate those factors such as the high mobility of politicians, government
cycles, political crises, and the tendency in some countries towards more autocracy and populism. Also,
the GPFE has been so far a resounding success in developing countries, but there is much less involvement
of Parliaments from industrialized countries.
IDEA International & IDEA-Solutions 52
Extent to which the NECD intervention is in line with important reference documents
This NECD intervention is aligned with important global strategic documents and policy guidelines. This is
to be expected given its focus at global enabling environment level.
Extent to which the NECD intervention has given attention to human rights, equity, gender equality,
and SDGs
This NECD intervention considers human rights, equity, gender equality, and SDGs as cross-cutting
dimensions in the intervention.
Good practices on NECD
Good practices in NECD include: i) the importance of having a strong and committed leader of the forum
to be proactive in terms of activities and create a momentum which brings news MPs and new countries
on board; ii) the establishment of wide partnerships to maximize synergies, resources, and in the end
results, iii) the respect of country-specific ways of working, iv) the knowledge and experience exchanges
among MPs from various countries.
Lessons learnt on NECD
The main lesson learnt from this NECD intervention is that the evaluation community still needs to learn
about the political dimension of evaluation. Some evaluators find it difficult to understand the role of
parliamentarians as they see evaluation only as a technical exercise outside of its legal, political,
institutional and cultural context.
UNICEF comparative advantage in NECD
UNICEF has supported the PFDE since its inception. It supports the Steering Committee of the GPFE as
well as the Secretariat. Currently it takes part in the organizing committee of EvalColombo 2018, the first
ever Global Parliamentarians event to take place in September 2018.
The comparative advantages of UNICEF in term of NECD come from its expertise in evaluation, its wide
network of country offices with channels and access to parliamentarians and its recognition as a non-
partisan honest broker working with MPS from the political parties in power as well as from the opposition
parties. UNICEF has also a capacity to identify “champions” and “wining situations” as a locally present,
but neutral actor in the countries.
IDEA International & IDEA-Solutions 53
Recommendations to UNICEF
Parliamentarians are responsible for making national government accountable and should therefore be
strong advocates for using evaluation in policymaking. UNICEF should therefore continue to play a key
role to support this intervention, in particular to: i) further develop the parliamentarians’ awareness of
the importance of NECD and the need to allocate resources to NECD and expose them to different NECD
strategies; ii) involve parliamentarians in the work UNICEF does to influence legislation for children, be it
the revision of existing policies or the elaboration of new policies; iii) further develop parliamentarians’
capacities to assess programs and policies in the context of the 2030 Agenda. The support of UNICEF
should be opportunistic and be adapted to the demands and capacities of parliamentarians’ fora in
different countries.
Final conclusion
Created as a network of EVALPARTNERS, the PDFE which became the GPFE is a well-respected forum.
Started in a few countries in Asia, it has since mushroomed in Asia, but also in Africa and Latin America.
Parliamentarians are a key stakeholder in democratic systems and can play a major role for evaluation as
well as benefit greatly from evaluation. This NECD intervention has rapidly created products and
generated outcomes. UNICEF has played a key role from the start and should continue its support in a
context, making it tailor-made to the situation in each country and adapted to emerging evaluation issues.
IDEA International & IDEA-Solutions 54
NECD INTERVENTION: EAPRO UNDP-UNICEF JOINT INITIATIVE AND COUNTRY CASE STUDIES IN THE ASIA-PACIFIC
REGION
Characterization of the NECD intervention
Responsible agency or unit
(mechanism responsible)
UNDP and UNICEF
Location Asia and the Pacific
Beneficiaries
Governments of Indonesia, Malaysia, Thailand, and Sri Lanka in
2017; Mongolia, the Philippines and others in 2018
Partners Asian Development Bank and UNWOMAN in peer review
Main objectives
1. Generate knowledge to guide NECD for 2030 Agenda
2. Identify successes and lessons in terms of national evaluation
systems
3. Foster peer learning and provide global and regional NECD
guidance through the production of readiness assessments at
country level
Actual start year 2017
Actual or planned end year 2019
Indicative duration (in months) 36
Performance of the NECD intervention
The initiative originated from Multiple regional and global fora focusing on national evaluation capacity
development including: NEC commitments and NEC Bangkok declaration, Parliamentarians Forum for
Evaluation by EVALPARTNERS in partnership with UN agencies, Global Evaluation Agenda developed by
EVALPARTNERS, other UN agencies, and a regional training workshop on “Developing National Evaluation
Capacities to Support the Implementation of the 2030 Agenda”. All actors perceived it did not make sense
to approach NECD in an isolated way per institution. All institutions shared the diagnostic and the need
to reinforce data, monitoring, and performance budgeting. This common diagnostic and objectives
facilitated the design of a consistent regional intervention by UNDP and UNICEF which is not easy given
the number of actors involved and the differences among country situations. Other facilitating factors
included the buy-in from country counterparts, the in-country liaison between UNICEF CO and
government professionals, and the interest of other agencies such as ADB.
IDEA International & IDEA-Solutions 55
Performance of this NECD intervention is globally high, recognizing that this is an ongoing intervention, so
this is only a preliminary assessment. In term of relevance, the NECD intervention is important to identify
and then reduce critical gaps to a proper evaluation function contributing to good governance. The
effectiveness to meet its objectives is high even though the process took much longer than expected (1
case study was completed out of the 4 planned in Year 1). The first two country diagnostics provide useful
information on (i) What M&E systems are in place to monitor, review and evaluate implementation of
2030 Agenda? (ii) How do evaluations inform adjustments to strategies, resource allocation and policies?
Is this changing in light of 2030 Agenda? (iii) What evaluation methods/ tools are government/ other
stakeholders using to evaluate policy choices to support integrated approaches? What equity-based
evaluation methods/tools are being used? Could they be scaled up to monitor commitment of ‘leaving no
one behind’? Those reports also make useful recommendations to strengthen national evaluation
capacity for the SDGs. Efficiency in terms of value for money is also high with an interesting combination
of work done by local consultants and guidance from regional and international recognized experts.
Although it is too early to tell about immediate and final outcomes, one can already anticipate the benefit
of having a regional synthesis, the value of peer to peer exchanges, and the ripple effect of this
intervention on other countries in the region who want to jump on the wagon and will position
participating countries in a position to evaluate the SDGs.
Extent to which the NECD intervention is in line with important reference documents
There is full and direct alignment of the NECD intervention with relevant documents such as UNICEF
Evaluation Policy, UNICEF Strategic Plan (2014-2017), the GA resolution on National Evaluation Capacity
Development, the Global Evaluation Agenda and UNEG Norms and Standards. This reflects the fact that
UNICEF-UNDP project managers know about those reference documents, understand them, and are able
to implement them in concrete NECD interventions.
Extent to which the NECD intervention has given attention to human rights, equity, gender equality,
and SDGs
Even if the NECD intervention focuses on the generation of knowledge to guide NECD in the perspective
of the 2030 Agenda and the identification of successes and lessons in terms of national evaluation
systems, it considers human rights, equity, gender equality, and SDGs as cross-cutting dimensions in the
NECD intervention.
Good practices on NECD
Good practices include: i) the case study process based on the framework of EvalAgenda 2030 from
EVALPARTNERS; ii) the productive collaboration between UNDP which enjoys strong linkages with
national governments and UNICEF which also enjoys linkages with governments but also with other
stakeholders, strengthens the UN message and credibility to the governments; iii) the integration of NECD
as part of UNICEF CO work plans; iv) the opportunities of entry points related to the country case studies
IDEA International & IDEA-Solutions 56
and other ongoing UNICEF initiatives (e.g. Thailand evaluation policy); v) the hiring of national consultants
to develop national case studies; vi) the documentation of the case study31
Lessons learnt on NECD
Lessons learnt so far are that: i) the scope and timeframe of the NECD intervention were not consistent
with resources available; ii) follow up on existing case studies and elaborating more case studies is very
important. NECD should not be conceived as a one-off activity, but as a stepwise process; iii) it is important
to make NECD part of the evaluation work plan; iv) While work on MDGs used to be done in silos, SDGs
require interoperability of systems and so governments want an integrated approach.
UNICEF comparative advantage in NECD
The comparative advantages of UNICEF in terms of NECD are: i) a stronger focus on NECD than other
international organizations; ii) its presence in the countries, and not only in the capital city, but also in the
regions. UNICEF COs have national linkages and a level of credibility at country level which gets the
message of evaluation on the agenda. This provides an in-depth perspective for the NECD intervention;
iii) its technical capability in evaluation and experience in supporting the elaboration of evaluation policies
and M&E systems. Evaluations conducted by UNICEF are quality products that are relevant and practical
since they produce recommendations at programme level; and iv) its dedicated M&E staff in UNICEF COs.
Recommendations to UNICEF
Recommendations are that UNICEF HQ and ROs engage in key global and regional strategic partnerships
on NECD such as this NECD intervention and do so with a medium term perspective while UNICEF COs
should continue to i) support evidence generation activities of Government thought a variety of surveys
(e.g. LICS, MICS and HICS) and evaluations as CP priorities; ii) identify capacity gaps that prevent from
acting at policy or program level; iii) work with partners that are involved in planning and in budgeting, so
that the use of evaluations for planning and budget be considered and iv) reinforce their sub-national
focus.
Final conclusion
This regional NECD intervention clearly demonstrates the benefits of an integrated approach: first
integrated in terms of partners with UNDP and UNICEF working together and progressively bringing in
other partners such as ADB, WB, JICA, Australia; integrated also in terms of approach to be consistent
with SDGs which require interoperability of systems, a multisectoral perspective and a clear roadmap with
31Polastro, R. and M. Prokop (2017). Review of National Evaluation Systems and Capacities for Evaluating Progress
Towards the Sustainable Development Goals - Emerging Lessons Learnt from the Asia Pacific Region, Paper presented at the NEC Conference, Istanbul
IDEA International & IDEA-Solutions 57
actions, labour division, and coordination; and finally integrated in terms of results which must go beyond
fragmented outputs such as meetings, support to VOPEs, and publications to move toward a clear Theory
of Change of UNICEF-supported NECD interventions.
IDEA International & IDEA-Solutions 58
ANNEX 5 – CASE STUDIES AT COUNTRY LEVEL
CASE STUDY OF NECD INTERVENTION: SOUTH TO SOUTH COOPERATION BETWEEN THE M&E UNIT IN THE COLOMBIAN
AND ECUADORIAN PLANNING MINISTRIES
Characterization of the NECD intervention
Responsible agency or unit UNICEF Ecuador (and participation of UNICEF Colombia)
Location Ecuador and Colombia
Beneficiaries
Secretaria Nacional de Planificación y Desarrollo (National Planning
and Development Ministry)
Consejo Nacional de Planificación (National Planning Commission)
Partners (if any) M&E units in the Colombian and Ecuadorian Planning Ministries
Main objective
Introduce the M&E Unit (Planning Ministry Ecuador) to good practices
from Colombia on how to develop and implement an evaluation
agenda
Actual start year 2015
Actual or planned end year 2015
Indicative duration (in
months)
3
Performance of the NECD intervention
The NECD intervention was relevant since it was a quick way to move the evaluation agenda. Colombia
was the right kind of country to go forward for Ecuador, sharing similar features, but being ahead in terms
of planning and M&E. Also, relevant professionals of key departments were sent on this study tour: the
Monitoring and Evaluation Undersecretary, the National Development Plan Results Monitoring Director,
and the National Budget Monitoring Director.
The NECD intervention was effective and efficient to a significant extent since the participants in the study
tour learnt a lot of relevant information in a short time and at a low cost. Output targets were largely
achieved since they had fruitful discussions and liaisons with counterparts in the Planning Department of
Colombia (Sinergia) to learn how to identify priorities in terms of evaluation and elaborate an evaluation
agenda to propose to the National Planning Board of Ecuador. This short study tour also did not create
problems from a long absence from work.
This NECD initiative translated into immediate outcomes as Ecuador defined a national evaluation plan in
2015 with prioritized evaluations to be carried out that year. This evaluation plan was then validated by
the Planning Commission led by the President. Those immediate outcomes seem to translate into final
outcomes (to the extent the latter are yet to materialize given the short time span) as Ecuador continues
to develop an annual evaluation plan that allows to prioritize evaluations and allocate resources to
IDEA International & IDEA-Solutions 59
evaluations. One challenge though for sustainability is staff rotation as all members of the mission to
Colombia have since moved to occupy other positions in the public sector.
Extent to which the NECD intervention is in line with important reference documents
The NECD intervention with key reference documents is high in general, such as the UNICEF Evaluation
Policy, the UNICEF Strategic Plan, 2014-2017, and the GA resolution on National Evaluation Capacity
Development and UNEG Norms and Standards. But some documents such as the GA resolution on the
Agenda 2030 and the Global Evaluation Agenda are not well-known by the respondents.
Extent to which the NECD intervention has given attention to human rights, equity, gender equality,
and SDGs
The NECD intervention considered all cross-cutting dimensions when discussing evaluation priorities.
Good practices on NECD
Study tours are not costly and not too demanding in terms of organization and coaching. Contacts and
exchange among civil servants facing similar challenges are easy to establish.
The country selected to host the study tour must be well prepared and be relevant to the level of
development of the visiting country. Colombia was the right choice of country, ahead of Ecuador in
planning and M&E, but not too much. The Sinergia team is well organized and works seriously. Finally, the
study tour benefited from the support of UNICEF Country Offices in both countries.
Lessons learnt on NECD
Elaborating an evaluation agenda requires to have a broad vision of evaluation to consider various types
of evaluation depending on the expressed needs. The Ecuador delegation could expand its awareness and
knowledge of a menu of possible evaluations.
It is important to keep the flow of exchanges after the study tour and even more if some staff leave their
current position. UNICEF plays a key role to keep the momentum and do follow-up.
UNICEF comparative advantage in NECD
UNICEF enjoys a comparative advantage in terms of 1) experience in NECD, 2) expertise in NECD, and 3)
teamwork. UNICEF occupies a unique position because it is well respected and can exercise leverage on
other actors to induce them into engaging into evaluation. In this specific case, UNICEF created the
conditions so that key actors in planning and M&E from Ecuador could learn from the experience of their
peers in a similar country, but with more experience. UNICEF brings in external points of views and helps
counterparts to “think out of the box”.
IDEA International & IDEA-Solutions 60
Recommendations to UNICEF
UNICEF should continue its support to NECD in Ecuador over the medium and long run because the
evaluation culture in the public sector is still new in this country. This support should go to a variety of
public institutions, but also other institutions such as universities, research organizations, consultants,
multilateral organizations. In particular UNICEF can play a unique role in creating or supporting events and
spaces related to evaluation where those actors can share their experiences and results and benefit from
those of others in the country and elsewhere in the region and the world
Final conclusion
This study tour of Ecuador high level civil servants to visit their counterparts in Colombia was a low cost
NECD intervention with concrete immediate outcomes. The Colombia case was relevant for Ecuador and
could translate into a direct transfer of technical and institutional capital as the sharing of their approach
and tools to select evaluation priorities and elaborate an evaluation agenda with the corresponding
budget led Ecuador to adopt a similar process but adapted to their political and institutional realities. This
was individual NECD that led to institutional and enabling environment NECD. While a small player in both
Ecuador and Colombia, UNICEF Ecuador demonstrated it could contribute to the institutionalization of
the evaluation function of Ecuador and keep the momentum going in a context of government cycles.
IDEA International & IDEA-Solutions 61
CASE STUDY OF NECD INTERVENTION: CAPACITY BUILDING AND INSTITUTIONAL CONDITIONS TO INTRODUCE AND M&E
SYSTEM IN KYRGYZSTAN
Note: this NECD intervention is a combination of two projects conducted sequentially: “Improvement of
capacity building to establish a Monitoring and Evaluation system in Kyrgyzstan” in 2014 and “Formation
of institutional conditions and capacity to introduce an M&E system in Kyrgyzstan» in 2015.
Characterization of the NECD intervention
Responsible agency or unit National Monitoring and Evaluation Network of the Kyrgyz Republic
through a (legally registered) Institute of Human Projecting (NGO)
Location Bishkek, Kyrgyzstan
Beneficiaries
Members of the National M&E Network of the Kyrgyz Republic (Kyrgyz
VOPE), Office of the Government, National Parliament,
representatives of government bodies responsible for implementation
of M&E, professional evaluation organizations, Central Asian and
international organizations engaged in evaluation
Partners OSCE, UN Women
Main objective
Enhance national capacity to introduce and implement an M&E system
in the public sector
Actual start year 2014
Actual or planned end year 2017
Indicative duration (in
months)
36
Performance of the NECD intervention
This NECD intervention had overall good performance. Its relevance is definitely high since M&E was a
relatively new concept in Kyrgyzstan and there is greater demand for improved performance,
transparency and accountability of the public sector. The intervention was effective with multiple outputs
and high value for money to raise the level of awareness of three important actors: Government,
Parliament, and the VOPE. The organization of an International M&E conference in Bishkek provided the
foundation to raise the awareness of Government and Parliament on the importance of M&E. Then the
intervention supported the participation of one government official and two Kyrgyz VOPE members in the
biannual conference of the European Evaluation Society in Dublin which helped to forge a long-term
partnership and establish contacts that proved useful during later stages of collaboration. Several
publications were produced under the NECD intervention: two analytical papers on an organizational and
legal analysis of the national M&E legislation and on the implementation of the M&E function in several
ministries, and methodological guidelines for the introduction of staff evaluation in the government.
Several joint capacity building events of civil servants and members of the Kyrgyz VOPE were organized.
IDEA International & IDEA-Solutions 62
Then pilot evaluations of a government program and staff appraisal were conducted. And an M&E law
was adopted in 2014
Those outputs have led to several outcomes, in particular an increased awareness of key national
stakeholders of the importance of M&E in the public sector, a greater commitment of the Kyrgyz
Government and Parliament to the SDG Agenda, the establishment of solid partnerships with Parliament,
Government officials, and other national and regional stakeholders, and an increased reputation of the
Kyrgyz VOPE.
However, sustainability remains a challenge. At individual level, there is no systematic M&E training for
civil servants. At institutional level, the capacity of many ministries to actually implement and use the
M&E system is week and there is a need to involve academic institutions to offer the trainings. At enabling
environment level, the unstable political environment and frequent changes in the government can affect
the continuity of processes.
Extent to which the NECD intervention is in line with important reference documents
There is overall alignment of the NECD intervention with major reference documents such as the GA
resolution on National Evaluation Capacity Development, the GA resolution on the Agenda 2030, the
Global Evaluation Agenda and UNEG Norms and Standards and partial alignment with the UNICEF
Evaluation Policy and UNICEF Strategic Plan.
Extent to which the NECD intervention has given attention to human rights, equity, gender equality,
and SDGs
This NECD intervention addressed all those crosscutting dimensions.
Good practices on NECD
The engagement of Government and Parliament and non-government actors like the VOPE proved to be
important for a long-term partnership going beyond short-term donor-supported interventions. The
leading role of national MPs who are also members of the Global Parliamentarians Forum for Evaluation
(GPFE) facilitated the organization of events and ensured a wide interest in both Parliament and
Government. This demonstrated a multiplier effect of combining the activities of the GPFE global initiative
and of the CO support.
Finally, the importance of building individual capacities of evaluators by high-quality international
instructors in areas where there is not enough national capacity in this area proved crucial for progress.
IDEA International & IDEA-Solutions 63
Lessons learnt on NECD
It has been highlighted that one single NECD intervention may not change the evaluation capacity
situation in the short and medium run, hence the need to expand the scope and sustainability of results
through a programme and a portfolio perspective.
UNICEF comparative advantage in NECD
One comparative advantage of UNICEF is related to its openness to collaboration and the inclusion of
NECD as a priority for the SDGs agenda. UNICEF has a strong and sustained relationship with both
Government and Parliament and enjoys a positive reputation in the country. UNICEF proved to be well-
organized, with a demonstrated capacity to lead, support by the HQ Evaluation Office, and backed up by
significant funding.
Recommendations to UNICEF
UNICEF should continue its support to NECD in Kyrgyzstan over the medium and long run because the
evaluation culture in the public sector is still new in this country. UNICEF has strong comparative
advantages in this domain, most notably through its longstanding presence in the country, strong
technical expertise and a close relationship with the key MPs which needs to be considered as a strategic
point for future NECD. As a matter of fact, awareness has been raised to a point where those MPs consider
the evaluation of programmes as a good approach to boost accountability of ministries and governmental
institutions. UNICEF support is needed to address issues of institutionalization and legitimation of the
M&E process, as well as the customization of tools, which still requires time, expertise, and financial
resources.
Final conclusion
The outputs of this NECD intervention have been deeply appreciated as expressed directly by the key
beneficiaries: UNICEF has been “the sole development agency that has been consistently providing support
to NECD activities over several years, which has been instrumental in the achievement of most of the
outcomes and impact and has shaped the NECD situation in the country”. Kirgizstan is a country where
continued actions for NECD at individual, institutional, and enabling environment levels are needed to
obtain sustainable outcomes. The opportunity cost of letting go after critical support to build the
foundations of an M&E system is deemed very high.
IDEA International & IDEA-Solutions 64
CASE STUDY OF NECD INTERVENTION: SUPPORT TO THE CULTURE AND INSTITUTIONALIZATION OF EVALUATION IN
MOROCCO
Characterization of the NECD intervention
Responsible agency or unit Social Inclusion Section of the Moroccan Association of Evaluation
(AME)
Location Rabat
Beneficiaries
Members of the AME, ministerial departments, Parliamentarians,
elected decentralized actors
Partners Ministerial Departments, Parliament and NGOs
Main objectives
1. Strengthen the culture of evaluation
2. Strengthening national capacities for Policy Evaluation
3. Advocacy for the institutionalization of the evaluation
Actual start year January 2013
Actual or planned end year December 2017
Indicative duration (in
months)
60 months
Performance of the NECD intervention
This NECD intervention demonstrates overall good performance. It is relevant since it promotes individual
capacity development of key actors. It has been effective and efficient in producing outputs, i.e. increased
program evaluation capacities of Parliamentarians, elected decentralized actors, and ministerial
departments through the organization of Moroccan Evaluation Week every two years, organized with a
focus on the evaluation of policies targeting children; and South-South cooperation activities to
strengthen the field of evaluation. The outcome is related to the sharing of good practices in evaluation
of social policies targeting children.
Extent to which the NECD intervention is in line with important reference documents
There is overall alignment of this NECD intervention with key reference documents such as UNICEF
Evaluation Policy, GA resolution on NECD, GA resolution on the Agenda 2030 and Global Evaluation
Agenda, the UNICEF Strategic Plan, 2014-2017, and UNEG Norms and Standards.
Extent to which the NECD intervention has given attention to human rights, equity, gender equality,
and SDGs
This NECD initiative took into account human rights, equity, and SDGs.
Good practices on NECD
IDEA International & IDEA-Solutions 65
Good practices noted in this NECD intervention include: i) the organization of national evaluations events
to promote the continuous training and collaboration of stakeholders, ii) the collaboration of additional
actors such as the community of practitioners, Parliamentarians and decentralized actors; and iii) the
integration of SDG evaluation issues during the second francophone forum of evaluation held in Morocco
in 2016. These activities proved to be useful to raise awareness and build upon networking capacities.
Lessons learnt on NECD
One lesson learnt from this NECD intervention is related to the opportunities created through the process
of promoting the exchanges of knowledge, experiences, and improved practices, either South-South or
South-North. UNICEF needs to seize the opportunity to strengthen and reinforce local capacities to unify
the approach and language of evaluation policies.
Another lesson is the limit of working mainly in terms of NECD at individual level. The AME could achieve
broader results by working more at institutional and enabling environment levels. This would imply a
closer partnership with central level national partners such as the Ministry of General Affairs and
Governance (MAGG), the National Observatory of Human Development (ONDH) and Government General
Secretariat.
UNICEF comparative advantage in NECD
The comparative advantage of UNICEF lies in its focus on children, the support in terms of expert
mobilization and networking facilitation, and finally, the support to the advocacy effort for the
institutionalization of public policy evaluation in Morocco.
Recommendations to UNICEF
Recommendations for UNICEF are related to supporting the AME and other actors in Morocco to reinforce
their capacity to communicate about the main results of their work in evaluation to capture the attention
of other actors that might be interested in supporting NECD activities to enhance the networking and to
foster more collaboration and partnerships among Moroccan actors. It may require developing broader
partnerships, including with actors that have a better networking capability with key central government
institutions to enhance the results in advocacy for more institutionalization of evaluation. Further work
with sub-national actors would also enable to address the specific issues in each region of the country.
Finally, UNICEF could also advocate more for the consideration of equity and gender equality in its NECD
activities in a context of rising social issues.
IDEA International & IDEA-Solutions 66
Final conclusion
This NECD intervention has created a real space to bring together the stakeholders, the community of
practices and the government in a common effort to share policy evaluations and to reinforce the capacity
to communicate and work around a common approach in the country. Achieving final outcomes in a
sustainable way to enable a culture of evaluation to permeate the public sector would imply going one
step further than individual evaluation capacity development and working more at institutional and
enabling environment levels and fostering broader partnerships, in particular with key central
government institutions and intensifying work with sub-national actors.
IDEA International & IDEA-Solutions 67
CASE STUDY OF NECD INTERVENTION: NATIONAL EVALUATION CAPACITY DEVELOPMENT IN NIGERIA: A TWO-FOLD
STRATEGY
Note: this NECD intervention includes two components: Regulatory framework for the evaluation of
public sector programmes/projects; the support to the creation of the Nigerian Association of Evaluators
(formerly the Association of Nigeria Evaluators).
Characterization of the NECD intervention
Responsible agency or unit Ministry of Budget and National Planning
Location Abuja and several other cities
Beneficiaries Ministries, civil servants, and services delivery users.
Partners
Ministry of Budget and National Planning, National Bureau of Statistics,
Special Advisors to the President on SDGs, Association of Nigerian
Evaluators UNFPA, ILO, WHO, World Bank, Nigerian Economic Summit
Group, representatives of private sector, academia and NGOs.
Main objectives
1. To strengthen the capacity of Ministry of Budget and National
Planning (MBNP) with the Evaluation Function, Federal MDAs and
State MDAs
2. To support for the establishment of the Nigeria Association of
Evaluators
Actual start year 2014
Actual or planned end year 2018
Indicative duration (in
months)
48
Performance of the NECD intervention
The performance of this NECD intervention is high. It is obviously relevant as it aims at raising the profile
of the evaluation function within the government, working on advocacy and developing a NEP (demand);
and creating a National Evaluation Association (demand and supply). Hence the title of “two-fold
strategy”.
It has been effective in the stepwise provision of outputs, building the basis for sustainable outcomes
through a participatory and country-owned process. UNICEF and the Ministry of Budget and National
Planning (MBNP) worked together to support the unification in a federation of associations known as the
Nigerian Association of Evaluators (NAE) that functions as the national VOPE in Nigeria. The NAE aims to
further develop and promote a community of evaluators and practitioners that will support the
institutionalization of evaluation in Nigeria. It was instrumental in advocating for a National Evaluation
Policy, and for government officials to engage with the professional communities of evaluators. To further
build the evaluation function, UNICEF and MBNP continue to support NEA to organize annual conferences
IDEA International & IDEA-Solutions 68
to raise the importance of evaluation as an important public management function. The NAE platform
was used to organize three conferences on Evaluation Function for Evidences based Policy Making, SDGs
Achievement, and Accountability) and publish three high level declarations: to adopt an Evaluation Policy
and Develop a Result based management framework in 2015, to continue the engagement toward such
goals in 2016, given the change in government, to maintain continuity with the new government agenda,
to obtain a National Agreement towards the Development of a National Accountability Framework to
support the Monitoring and Evaluation functions of government partners in achieving and reporting on
the SDGs In 2017.
Although it is too early to talk about final outcomes, those outputs have clearly already translated into
immediate outcomes with the commitment of the government to adopt the National Evaluation Policy
(NEP) and being more engaged with strengthening the evaluation function. The next steps are to i)
legislate this policy as an executive decree in the short run and as Law in the medium run; and ii) support
the MBNP with a Monitoring Policy.
Extent to which the NECD intervention is in line with important reference documents
The NECD intervention is aligned with major reference documents, including UNICEF Evaluation Policy,
GA resolution on NECD, GA resolution on the Agenda 2030 and Global Evaluation Agenda, the UNICEF
Strategic Plan, 2014-2017, UNEG Norms and Standards as well as the EVALPARTNERS Global Evaluation
Agenda.
Extent to which the NECD intervention has given attention to human rights, equity, gender equality,
and SDGs
The NECD intervention considered equity and SDGs as cross-cutting dimensions.
Good practices on NECD
The main good practice was the unification of the national evaluation leaders under a national VOPE that
serves as an umbrella organization for evaluation associations in Nigeria, demonstrating the benefits of
coming together in a context of a high, but scattered supply of evaluators, with limited or no opportunity
for networking. The NEA, with the support of key actors like UNICEF, will be in a position to further develop
and promote a community of evaluators and practitioners that will support the institutionalization of
evaluation through capacity building activities.
Let us also mention key factors for sustainability which is sometimes a weak point of NECD interventions:
the focus to sustain an enabling environment, advocacy for more country-led evaluations and further
professionalization of the evaluation function, continued engagement and support to the MBNP, regular
engagements with the regional evaluation association in Africa (AFREA) and with global evaluation
associations (IDEAS), as well as peer review mechanisms to rollout the NEP.
IDEA International & IDEA-Solutions 69
Finally, let us mention the importance of documenting the experience of the NECD intervention which has
been the case in Nigeria32.
Lessons learnt on NECD
NECD takes time to be sustainable and country-owned and the kind of stepwise and participatory
approach used by UNICEF has proven fruitful. The endorsement of NEP should be considered as a key
milestone for the enabling environment, but this is not necessarily the only way forward for large
countries like Nigeria with significant pockets of competencies not only at national level, but also at sub-
national level (States). There is a big potential due to the availability of technical organizations working in
the evaluation area and UNICEF has been good in tapping them and helping bring them together. Also,
the strategic partnership and quality of the relationship with the MBNP is very valuable for NECD
interventions.
UNICEF comparative advantage in NECD
UNICEF is appreciated playing a convener role, bringing development partners to work together, and
contributing its global leadership experience, support to networks such as EVALPARTNERS and its support
to conferences and events promoting South-South cooperation. Another advantage is the good rapport
of the CO with the MBNP which facilitated a number of initiatives while keeping a neutral position vis-a-
vis the government and government partners, and its support to government systems.
Recommendations to UNICEF
The obvious recommendation for UNICEF in Nigeria is to continue its crucial support to combine different
aspects under the NECD intervention and merge the efforts of the government, the NAE, and other
stakeholders, contribute to the dissemination of evaluation methodologies and good practices at federal
and state level, and build up the technical capacity of staff in key Ministries, Departments and Agencies
(MDAs) using a Training of Trainers approach, including a hands-on approach of conducting Joint
Evaluation studies with the government and the NAE.
The recommendation to other country offices is to consider using the same two-fold strategy to create
the conditions for more institutionalization of the evaluation function. The recommendation to the
regional offices is to continue and increase experience sharing in the region, promoting exchange
programs among countries that have similar contexts.
Finally, UNICEF could also advocate more for the consideration of human rights and gender equality in its
NECD activities.
32 Jobin, D. and Z. Lawal (2017). “Application of game theory and new institutional economics in establishing a
National Voluntary Organisation for Professional Evaluation in Nigeria”. African Evaluation Journal. https://aejonline.org/index.php/aej/article/view/197
IDEA International & IDEA-Solutions 70
Final conclusion
Nigeria provides an interesting case study of a large country with significant human and financial
resources, but relatively new to a culture of transparency, evaluation, and accountability as well as facing
issues of coordination at national level, between the federal government and the state governments, and
between government and non-government actors. UNICEF has supported a two-fold strategy working the
national government and the evaluators to create a national VOPE which has been instrumental in
promoting the importance of the evaluation function and advocating for a National Evaluation Policy as
well as to engage government officials with the professional communities of evaluators. Major
conferences have been organized on a regular basis, leading to high level declarations. NECD is a long-
term process, but building an enabling environment is an essential ingredient.
IDEA International & IDEA-Solutions 71
CASE STUDY OF NECD INTERVENTION: NATIONAL EVALUATION POLICY IN THE PHILIPPINES
Characterization of the NECD intervention
Responsible agency or unit National Economic and Development Authority (NEDA) and
Department of Budget and Management (DBM)
Location Manila, Philippines
Beneficiaries Public sector
Partners
Main objective Crafting of National Evaluation Policy of the Philippines
Actual start year 2013
Actual or planned end year 2015
Indicative duration (in
months)
36
Performance of the NECD intervention
The performance of this NECD intervention is high in terms of relevance, effectiveness, efficiency and
outputs. The process of elaboration of the draft National Evaluation Policy took some time which is not
unusual given the importance of this orientation document, the number of stakeholders, and the
resistance to change of some stakeholders. The Policy issuance of the National Evaluation Framework of
the Philippines was done in July 2015 in a joint memorandum NEDA-DBM. In addition to this consistency
framework stating clearly the importance of evaluation and the roles of key stakeholders, the
participation of many public sector actors in fora, public hearings, and support task forces also contributed
to improve the enabling environment for the purposive conduct of evaluations in the public sector in
support of good governance, transparency, accountability, and evidence-based decision-making. Along
with the Monitoring, evaluation and reporting policy put forward by DBM, also benefiting from the
support of UNICEF, to provide clear guidelines at operational level between DBM and line ministries,
awareness among government agencies has increased to a significant extent and many agencies have
started instituting evaluation plans.
Although it is too early to analyze final outcomes, there are some concerns about sustainability. First, the
change of administration in 2016 meant a period of flux in policy making with the review of the previous
administration policies. Also, the actual implementation of the policy implies building up the capacities of
line ministries in managing and/or conducting evaluations.
IDEA International & IDEA-Solutions 72
Extent to which the NECD intervention is in line with important reference documents
The NECD intervention had a partial but direct alignment with all relevant documents listed. It is an
intervention at enabling environment level which facilitates the connection with global strategic
documents and policy guidelines.
Extent to which the NECD intervention has given attention to human rights, equity, gender equality,
and SDGs
The NECD intervention took into account equity, gender equality, and SDGs as crosscutting dimensions.
Finally, UNICEF could also advocate more for the consideration of human rights in its NECD activities.
Good practices on NECD
This NECD intervention illustrated ways to minimize the impact of policy cycles and unexpected events.
Good practices included advocating for the government buy-in and engagement at early stages, and
promoting ownership by involving the government agencies at every step of the process. This also
involved the identification of champions within the partner government agencies; and a non-political,
non-partisan engagement when conducting NECD activities. (watch out: the previous sentence said the
opposite of what you wanted to say)
Lessons learnt on NECD
The overall NECD agenda should consider political changes and policy cycles which affect directly the
availability of internal “champions” for full policy implementation. This speaks of the importance to
involve other UN agencies in the country (like a ONE-UN/UNDAF agenda) and build strong strategic
partnerships to improve support. UNICEF managed to promote the objectives of the NECD intervention
apart from political objectives.
The articulation of the NECD intervention with the national strategies and policies such as the National
Development Plan also helps the buy-in of national stakeholders towards the intervention.
NECD is to be considered in the medium run. The implementation of the policy requires further support
and capacity-building of actors, including the community of M&E professionals in Philippines.
UNICEF comparative advantage in NECD
The main comparative advantage of UNICEF is its ability to put together a well-established network which
includes government, academia, other non-public sectors actors, and other development agencies.
UNICEF has a partners-built system for NECD which allows national actors to benefit from this know-how
and accumulated knowledge with capacity development material like toolkits, IEC, guidelines, etc.
IDEA International & IDEA-Solutions 73
Recommendations to UNICEF
UNICEF should continue its support for the implementation of the NEP. This implies building individual
and institutional capacities in many agencies and other actors at national and sub-national level, which is
a huge task. The first proposal is to coordinate with other actors involved in NECD interventions, in
particular UN agencies, for pooling together resources and designing more ambitious complementary
interventions for bigger results. The second proposal is to use pilot agencies, programmes, provinces and
municipalities with progressive scaling up. Finally, UNICEF could share good practices from the region and
from other regions in terms of implementation of NEPs.
Final conclusion
This NECD intervention is a good example showing that working in good partnership with the government
central planning and budgeting agencies such as NEDA and DBM proved essential to push the elaboration
of a government wide policy on evaluation. It also demonstrated how to overcome policy cycles and
unexpected events which can be a major threat for the sustainability of NECD. UNICEF’s approach to
finding and nurturing linkages with the new power holders, its close partnership with a variety of actors
(public agencies, academia, practitioners, evaluation associations, parliamentarians, etc.) in a non-
political approach, its support to make them work together for the NECD in a coordinated way were key
to overcome political cycles and obtain results.
IDEA International & IDEA-Solutions 74
CASE STUDY OF NECD INTERVENTION: ESTABLISHMENT OF THE RWANDA MONITORING AND EVALUATION SOCIETY
Characterization of the NECD intervention
Responsible agency or unit UNICEF Rwanda PME section/ Capacity Development and Employment
Services Board (CESB)
Location Rwanda, Kigali
Beneficiaries
Rwanda M&E Society (RMES) and all persons with professional practice,
knowledge and interest in M&E who subscribe to the objectives of the
RMES
Partners
Rwanda National Capacity Building Secretariat, National Institute of
Statistics of Rwanda (NISR), Ministry of Labour and Ministry of Finance
Main objectives
1) To support an institutional set up of Rwanda M&E Society
2) To establish a functional Rwandan monitoring and evaluation
society (RMES)
3) To cultivate the highest professional standards in Monitoring and
Evaluation practice and theory through highly skilled and
motivated members
Actual start year 2016
Actual or planned end year Still running
Indicative duration (in
months)
Ongoing
Performance of the NECD intervention
This NECD intervention is still in an initial stage and performance cannot yet be fully assessed. However,
it is interesting to document how it emerged and the role of UNICEF. It is obviously relevant to establish
a VOPE and UNICEF has proven quite effective in sizing the opportunity of a heightened awareness
environment at this time to help deliver outputs. Early 2016 UNICEF Rwanda PME section started
advocating for national evaluation and RBM capacity building with the government and the community
of donors, which led to a discussion on the establishment of the country’s first M&E Society with the
Belgium Embassy and the National Capacity Building Secretariat (NCBS). In November 2016, UNICEF, NISR,
the Ministry of Labour and the Ministry of Finance met and agreed on the establishment of the country’s
first M&E society. Since then, UNICEF has been closely supporting NCBS, a custodian of this initiative. As
of the end of 2017, the institutional framework of the Rwanda M&E Society (RMES), i.e. the Statutes and
draft Annual Workplan 2018 had been drafted. The executive committee members were democratically
elected. The capacity of the Executive Committee has been enhanced through the support of the African
Development Bank and the organization of three high level training workshops in Rapid Results Approach
and Managing for Development Results. Office set-up for the M&E Society Secretariat is supposed to take
place in 2018
IDEA International & IDEA-Solutions 75
This NECD has so far been efficient, UNICEF spending little financial resources during the initial setup.
Resource needs are likely to increase with the operational support to be provided to ensure a functioning
RMES and, if not addressed, might be a barrier for the sustainability of the increased evaluation capacity,
participation of M&E practitioners in the overall country development, and increased interaction and
information sharing among all those interested in M&E. However, the engagement with the national
government institutions, civil society, and academics is a reassuring factor.
Extent to which the NECD intervention is in line with important reference documents
There is overall alignment of the NECD intervention with major reference documents such as the GA
resolution on National Evaluation Capacity Development, the GA resolution on the Agenda 2030, the
Global Evaluation Agenda and UNEG Norms and Standards and partial alignment with the UNICEF
Evaluation Policy and UNICEF Strategic Plan. This is perfectly normal since by definition the NECD
intervention concerns only a limited set of activities while the documents cover a large scope of action.
Extent to which the NECD intervention has given attention to human rights, equity, gender equality,
and SDGs
The NECD intervention considers human rights, equity, and SDGs as cross-cutting dimensions.
Good practices on NECD
The obvious good practices are the national ownership of the RMES where 100% of the Committee
members are Rwandan professionals, inclusive of all major government actors, and its participatory and
democratic process in society management. The election of national professionals as executive committee
members of elected provides an anchoring in local know how. Also, once obtained, the strong leadership
of the government provides a major impetus to the organization of NECD activities. Finally, an interesting
practice proposed (not yet implemented) is to track behavioral and institutional changes following NECD
activities through an online survey.
Lessons learnt on NECD
Advocacy for evaluation can be achieved with limited resources, but the sustainability of activities,
outputs, and outcomes requires a mid-term investment. Technical support and mentoring are important
in order to harness the national ownership.
UNICEF comparative advantage in NECD
UNICEF comparative advantage in NECD comes first from its capacity to engage different types of
partners, i.e. government, and non-government - NGOs, academics, private sector, etc., and to provide a
platform for interaction and information-sharing among all those interested in M&E. Second, UNICEF is a
IDEA International & IDEA-Solutions 76
knowledge-based organization which can easily transfer skills to the locals with high quality intellectual,
ethical and professional standards in M&E.
Recommendations to UNICEF
The situation in Rwanda would benefit from a leading support role of UNICEF working through the RMES
and the Rwanda MfDR Community of Practices initiated in October 2017. The timing is adequate to target
actions in line with the implementation the 2030 SDGs global agenda.
One domain of recommendation to UNICEF is related to the enhancement of communications and
documentation of the NECD activities. It is important to keep documenting on a regular basis good
practices and communicate to stakeholders the results arising from the NECD interventions. This is to be
done by the CO, but also by the RO which can facilitate experience exchange among UNICEF COs which
are supporting NECD initiatives for cross-fertilization.
From an operational point of view, the recommendations for the UNICEF CO are to: 1) help set up an NECD
seed fund to increase the scope and sustainability of NECD interventions, including research; 2) choose
carefully strategic NECD activities to be supported, considering building capacities at individual,
institutional, and enabling environment levels; 3) explore other partnerships to coordinate and liaise with
the Capacity Building and Employment Services Board (CESB) to promote a culture of results in Rwanda;
and 4) take advantage of the RMES to engage the government in developing a National Evaluation
Framework and if possible a National Evaluation Secretariat.
Final conclusion
Rwanda is at the initial stage of NECD and it is too early to assess the full performance of NECD
interventions. However, UNICEF has helped this country start on the right foot with strong national
empowerment and leadership, and involvement of key government and non-government actors. The
creation of the RMES has generated a momentum to engage into strategic M&E activities beyond just
individual training, in particular the establishment of a National Evaluation framework or policy. Also,
Rwanda as a latecomer to NECD can learn from the other regional and international experiences from
more advanced countries in NECD, avoid mistakes, grab good practices, and catch up quickly, if not double
those other countries.
IDEA International & IDEA-Solutions 77
CASE STUDY OF NECD INTERVENTION: STRENGTHENING NATIONAL M&E SYSTEM IN SRI LANKA
Characterization of the NECD intervention
Responsible agency or unit UNICEF CO
Location Sri Lanka – national and subnational (district) levels
Beneficiaries Parliamentarians and public officials
Partners
Global Parliamentarians Forum for Evaluation (GPFE, ex PFDE) at global
level and Parliamentarians’ Forum for Evaluation in South Asia and in
Sri Lanka; Organization of Environment and Children Rights
Preservation (OECRP); Department of Project Management and
Monitoring at the Ministry of Development Assignments; Sri Lanka
Evaluation Association; EVALPARTNERS; University of Sri
Jaywardenepura
Main objective
Strengthen the enabling environment for evaluation and build the
evaluation capacity to promote accountability and evidence-based
policy making and facilitate the achievement of SDGs in Sri Lanka
Actual start year 2016
Actual or planned end year Ongoing
Indicative duration (in
months)
18
Performance of the UNICEF intervention
The performance of this NECD intervention has been good overall. It is directly relevant to support the
strengthening the enabling environment and build capacity as the NECD intervention outputs were
targeted properly to reduce the gaps. Despite the fact that Sri Lanka is considered as a well-developed
M&E system, the practice of evaluation has not yet been fully institutionalized. The outputs of this NECD
were strategic and supportive of this institutionalization: i) a formulated National Evaluation Police (since
2003) and equity-focused and gender-responsive Evaluation Guidelines/ Standards; ii) Sri Lanka
Parliamentarians’ Forum for Evaluation established and operational to support active Members of
Parliament’ (MPs) demand; iii) District Coordinating Committees in 12 targeted districts sensitized in SDG
and use of evaluation at subnational level; and, iv) capacity building, in partnership with the University of
Sri Jayawardenepura, with a post-graduate diploma programme in Evaluation launched and fully
operational.
Overall, Sri Lanka is a good example of country-led, systemic national evaluation capacity development.
And the CO has managed to keep the momentum going through strong partnerships (MPs, public officials,
independent evaluators and academia) and benefiting from a strong drive in the Parliament. However, at
outcome level, there are challenges that halt the results and not necessarily are under the range of action
of UNICEF, for example the NEP is still pending for approval at Parliament and institutionalization of
IDEA International & IDEA-Solutions 78
evaluation capacity was affected by changes in ministerial setup. Even if the NECD intervention has
increased individual and institutional capacities, those capacities are not necessarily put into practice at
enabling environment level. This creates challenges for sustainability at this level.
Extent to which current the NECD intervention is in line with important reference documents
There is overall alignment of the NECD intervention with major reference documents such as the GA
resolution on National Evaluation Capacity Development, the GA resolution on the Agenda 2030, the
Global Evaluation Agenda and UNEG Norms and Standards and partial alignment with the UNICEF
Evaluation Policy and UNICEF Strategic Plan.
Extent to which the NECD intervention has given attention to human rights, equity, gender equality,
and SDGs
This NECD intervention addresses all crosscutting dimensions. Let us mention specifically the elaboration
of evaluation guidelines on gender responsive and equity focused evaluations and capacity building of
target groups in the use of those guidelines.
Good practices on NECD
One good practice has been the interconnected approach of the CO. It prepared a good needs assessment
and has directed strategically activities to target the gaps, taking advantage of the level of development
of M&E in the country context. It also targeted strategic partners, government and non-government, in
evaluation; and built long term relationships. Another good practice has been to involve the subnational
level, where potentially better SDG-related services coverage is required. Also, the consultative process
has contributed to dissemination and raised awareness of the importance of NECD.
Finally, let us mention the partnership with the University of Sri Jayawardenepura to develop a post-
graduate diploma programme in Evaluation and the development of a set of user-friendly resource
materials, including guidance on equity-focused/gender-responsive evaluations, resource materials for
MPs on importance and use of evaluation supported by awareness and capacity development sessions
for MPS and the Parliament’s Research Unit.
Lessons learnt on NECD
A first lesson is the importance of strategic and well-built partnerships at all levels as well as multiple entry
points to be able to keep the momentum created by the portfolio of NECD interventions and pass through
more difficult periods linked to political cycles. A second lesson is the need for a good ex ante assessment
of the potential risks and identification of mitigation measures before deciding to go for an NECD
intervention.
IDEA International & IDEA-Solutions 79
UNICEF comparative advantage in NECD
UNICEF Sri Lanka CO comparative advantage is utilizing strong partnerships at global, regional, national
and subnational levels. It used a strategy of strengthening its partnerships with key government and non-
government institutions involved in M&E in the country like: Department of Project Management and
Monitoring (DPMM), Global and Sri Lankan Parliamentarians’ Fora for Evaluation, Department of Census
and Statistics (DCS), Sri Lanka Evaluation Association (SLEVA), UNDP as well as international networks
(EVALPARTNERS, EVALSDGs, etc.). At the same time, new partnerships have been developed – with one
of the leading universities in Sri Lanka and the Government ICT agency mandated for implementation of
all the ICT projects in the country.
The close alliance with the provincial and national governments and partners allow the organization of
capacities and joint efforts, even in financially scarce scenarios.
Recommendations to UNICEF
Institutionalizing successes in NECD would be facilitated by an explicit statement in the UNICEF Evaluation
policy and then corresponding resources allocations for ROs and COs. NECD cannot depend only on
fundraising by CO, otherwise the scope of the interventions will be constrained and the momentum might
be reduced, if not lost. UNICEF would also gain in having a clear vision on: i) what are the most relevant
NECD approaches; and ii) how they can be implemented with available resources together by HQ, RO, CO
as a team. Dissemination of information and experience within UNICEF is also important. The
recommendation is to provide some space and resources for documenting and sharing successful NECD
experiences, good practices, lessons learnt. Finally, the identification of criteria to decide for a “go or not
go “would help be more systematic in the assessment of opportunities, partners, and risks.
Final conclusion
Overall, this NECD intervention has been successful due to the fact that UNICEF and key partners have
used the momentum to permeate the demand of key actors like MP and Academia. Also, they promoted
to put into practice the developed guidelines by the target audience at parliament. Even though the NEP
is still pending for approval, the CO keeps on track with a vision to interconnect efforts. In this roadmap,
In September 2018, Colombo will host the Parliamentarians’ Forum on Evaluation with parliamentarians
from all over world, members of the evaluation community, government champions and development
partners. The office has been documenting the last two years of engagement in strengthening evaluation
in Sri Lanka, also to be used to support Sri Lanka’s National Voluntary Review submission at the High Level
Political Forum in July 2018 as part of the SDGs Follow-up and Review (FUR) Process. Sustainability remains
a challenge to overcome the instability and turnover created by political cycles, but the variety of
government and non-government partners with good long-term relationships, and a better analysis of
success factors for each NECD intervention could help address this challenge.
IDEA International & IDEA-Solutions 80
ANNEX 6 – EXAMPLE OF POSSIBLE GO/NO GO GRID AT COUNTRY LEVEL
The purpose of this grid is for pre-feasibility assessment. For a full ex-ante evaluation, consultants suggest a process like IDEA-Solutions Projects
for Results (P4R®). Among the criteria to be considered are the classical OECD/DAC criteria of relevance, anticipated effectiveness, efficiency,
results, and sustainability, but also the existence of a logic model based on a clear Theory of Change, the specificities of the country (i.e. NECD in
middle-income country vs low income country), the analysis of potential partners, the contribution of the NECD intervention in the NECD process,
the child-focus. Most of the NECD budget should be spent on those selected interventions, but with continued and significant institutional,
technical, and financial support guaranteed over time to get sizable results over the medium run.
NECD INTERVENTION GO/NO GO GRID AT COUNTRY LEVEL
Name of Potential UNICEF Supported NECD Intervention:
Name of the person, position, email filling the grid:
Date of the update:
1. Presentation of the potential NECD intervention
Target group:
Main Objective:
Main Components/Activities:
Anticipated Budget:
Anticipated Outputs:
Anticipated Outcomes:
Anticipated Duration:
Partners (if any):
Location:
IDEA International & IDEA-Solutions 81
2. Assessment grid
Criteria Weight* Evaluation
Justification -2 -1 0 1 2
A. Level of development of evaluation capacities from the countries expected to be part of the NECD 25%
10
NECD alignment with the Country's strategic orientations/strategic plan/etc.
Counterparts are clearly stablished
Favourability of the political cycle
Maturity of legal framework (norms/laws/constitutions)
Maturity of institutional framework (Clear M&E roles, M&E processes, M&E systems)
Subtotal
0 0 0 0 0
% points for A 0
0
Criteria Weight* Evaluation
Justification -2 -1 0 1 2
B. Expressed commitment from Target Group to achieve outputs and outcomes
25% 10
Buy in from head officials/head positions
Buy in from technical officials/intermediate positions
Background and/or previous work with the partner (e.g. previous work in NECD with UNICEF)
Clarity upon expressed and documented outcomes
Willingness to endorse a performance agreement document
Subtotal
0 0 0 0 0
% points for B 0.0%
0
*Note: the weights may vary depending on the type of country (e.g. lower, middle, upper income)
IDEA International & IDEA-Solutions 82
Criteria Weight* Evaluation
Justification -2 -1 0 1 2
C. Clarity in the contribution of NECD 25% 10
The contribution of UNICEF to the NECD intervention is clear
This contribution is feasible from a Human Resources point of view
This contribution is feasible from a material and financial resources point of view
This contribution is consistent with UNICEF mandate and country management plan
Subtotal
0 0 0 0 0
% points for C 0
0
Criteria Weight* Evaluation
Justification -2 -1 0 1 2
D. Resources capacities of the target group 25% 4
The partner has the specific expertise (individual CVs and Organizational CV, i.e. revise project sheets or portfolio)
Level of effort and budget allocated are considered sufficient to deliver the expected outputs (e.g. revised according a timeline and preliminary budget)
Subtotal
0 0 0 0 0
% points for D 0
0
Criteria Weight* Evaluation
Justification -2 -1 0 1 2
E. Other partners' support 25%
Availability of financial and human resources to complement the support
Commitment to the contribution of outputs and outcomes
Subtotal
0 0 0 0 0
% points for E 0 0
Final decision NO GO GO