juha uitto director gef independent evaluation office may 2015 evaluation in the gef
TRANSCRIPT
Juha UittoDirector
GEF Independent Evaluation Office
May 2015
EVALUATION IN THE GEF
Page 2
Monitoring <-> Evaluation
Monitoring Evaluation
Page 3
GEF M&E Policy
• Defines the concepts, role, and use of monitoring and evaluation within the GEF
• Defines the institutional framework and responsibilities.• Indicates the GEF minimum M&E requirements covering:
• project design, • application of M&E at the project level, and • project evaluation.
• Current M&E Policy Approved by GEF Council in November 2010
Page 4
M&E in the GEF: Objectives
• Promote accountability for the achievement of GEF objectives through the assessment of results, effectiveness, processes, and performance of the partners involved in GEF activities.
• Promote learning, feedback, and knowledge sharing on results and lessons learned among the GEF and its partners as a basis for decision making on policies, strategies, program management, programs, and projects; and to improve knowledge and performance.
Page 5
Separate reporting lines for Monitoring (through Secretariat) and Evaluation (through Independent Evaluation Office)
GEF Council
Agency evaluation units
GEF Independent Evaluation Office GEF Secretariat
Agency GEF coordination units
GEF projects and programs
Annual evaluation reportsOverall Performance Study (to Assembly)Annual Work Programand Budget
Annual Monitoring ReportEvaluation Management Response Programming documents and indicatorsResults Based Management
Project and Program Implementation ReportsAgency Portfolio ReportsProject documents with M&E plans
Project and Programevaluations
Corporate evaluationsProject and ProgramIndependent evaluations
Project and Program Implementation ReportsProject and Program monitoring documentationTerminal evaluations
Page 6
M&E Key Responsibilities in GEF
Partner Key Roles and Responsibilities in M&E
GEF CouncilPolicy-makingOversightEnabling environment for M&E
GEF Independent Evaluation Office
Independent GEF evaluationOversight of M&ESetting minimum requirements for evaluation
GEF Secretariat GEF Results Based Management (monitoring and reporting)Review of GEF M&E requirements in project proposals
Agency GEF operational units
Monitoring of the Agency GEF portfolioEnsure M&E at the project level
Agency evaluation units
Project and/or corporate Agency evaluationsMainstreaming GEF into relevant Agency evaluation
STAP Advice on scientific/technical matters in M&ESupport to scientific and technical indicators
Participating Countries Collaboration on M&E at portfolio and project levels
Stakeholders Participation in monitoring activities and mechanismsProviding views and perceptions to evaluations
Page 7
GEF Independent Evaluation Office
• Mission– Enhance global environmental benefits through excellence, independence
and partnership in monitoring and evaluation.
• Principles– Impartiality– Professionalism– Transparency
Created by decision of the May 2003 GEF Council. Added to the GEF Instrument in April 2014.
Page 8
GEF IEO GEF-6 Work Program
• The IEO will continue to provide feedback on the performance, results, and impacts of GEF support.
• The GEF-6 Work Program will be organized around key issues derived from the Sixth Replenishment of the GEF Trust Fund and relevant to the GEF 2020 Strategy, such as:- The extent, mechanisms, and conditions by which GEF support has identified
and delivered integrated solutions and multiple benefits.- The extent, mechanisms, and conditions by which GEF support has addressed
drivers of environmental degradation.- The performance of the GEF, including issues related to the GEF 2020
Strategy core operational principles.
Page 9
IEO Work Program for GEF-6
Mai
nstr
eam
ing
of K
M a
nd G
ende
r in
all e
valu
ation
s
Page 10
Policy, Guidance and KM
• Policy, Guidance, and Methodological Development:– GEF M&E Policy– Terminal Evaluation Guidelines– Impact Evaluation Guidelines and Handbook– Guidance on Evaluating Gender in GEF Programming
• KM and Learning:– KM needs assessment.– Develop knowledge, learning, and communications strategy for GEF-6.– Review the GEF-6 KM strategy and work plan of the GEF Secretariat.
Page 11
Office Initiatives
• Climate-Eval Community of Practice
• Participation in Conferences:– Evaluating Sustainable Development and NRM– 4th International Conference on National Evaluation Capacities
• Participation in GEF Meetings and Evaluation Networks:– Continue developing instructional and learning products to facilitate lessons learned
from evaluations. – Develop new training materials to be distributed during ECWs in coordination with
the GEF Country Support Program.– Maintain an active role in networks.
Aaron ZazuetaChief Evaluation Officer
May, 2015
The GEF M&E Policy and
Terminal Evaluations Guidelines
Page 14
Overview
1. Minimum M& E requirements in the GEF M & E Policy
2. Terminal Evaluation Guidelines and Reviews
3. Management response and management action record
A dynamic process.
Full Agency M&E requirements are expected to be applied to GEF projects
Page 15
M&E: Minimum Requirement 1
Design of M&E PlansConcrete and fully budgeted M&E plan by CEO endorsement for FSP and CEO approval for MSP. Project logical frameworks should align with GEF focal area results frameworks. M&E Plan should include:• SMART indicators
• Specific• measurable, • achievable and attributable, • relevant and realistic, (OPS 5 found this is critical)• time-bound, • trackable and targeted
• Baseline data for M&E by CEO endorsement• Mid Term Reviews (where required or foreseen) and Terminal
Evaluations included in plan• Organizational set up and budget for M&E
Page 16
M&E: Minimum Requirement 2
Implementation of M&E Plans Project/program monitoring and supervision will include execution of the M&E plan:
• Use of SMART indicators for process and implementation• Use of SMART indicators for results• Baseline for the project is fully established and data are compiled
to review progress• Organizational set up for M&E is operational and its budget is
spent as planned
Page 17
M&E: Minimum Requirement 3
Project/Program Evaluations: • All full sized projects and programs will be evaluated at the end of
implementation• Evaluations should:
Be independent of project management or reviewed by GEF Agency evaluation unit
Apply evaluation norms and standards of the GEF Agency Assess, as a minimum, outputs and outcomes, likelihood of sustainability,
compliance with Minimum Requirements 1 & 2 Contain basic project data and lessons on the evaluation itself (including TORs) Should be sent to GEF IEO within 12 months of completion of project/program
Guidelines for Terminal Evaluations
Page 18
M&E: Minimum Requirement 4
Engagement of Operational Focal Points
• M&E plans should include how OFPs will be engaged• OFPs to be informed on M&E activities, including Mid Term
Reviews and Terminal Evaluations, receiving drafts for comments and final reports
• OFPs invited to contribute to the management response (where applicable)
• GEF Agencies keep track of the application of this requirement in their GEF financed projects and programs
• OFPs can if they wish under take monitoring or evaluation of projects of country portfolio.
Page 19
Terminal Evaluation GuidelinesPurpose of terminal evaluations
Track performance of GEF portfolio
Provide feedback on GEF portfolio performance
Synthesize lessons that may help improve GEF functioning
Page 20
Terminal Evaluation GuidelinesEvaluation Criteria
• Relevance. The extent to which the activity is suited to local and national development priorities and organizational policies, including changes over time
• Effectiveness. The extent to which an objective has been achieved or how likely it is to be achieved
• Efficiency. The extent to which results have been delivered with the least costly resources possible; also called cost effectiveness or efficacy
• Results/Impact. The positive and negative, and foreseen and unforeseen, changes to and effects produced by a development intervention
• Sustainability. The likely ability of an intervention to continue to deliver benefits for an extended period of time after completion. Projects need to be environmentally as well as financially and socially sustainable
Page 21
Terminal Evaluation GuidelinesRole of Agencies
• Are carried out by Agencies at project completion• Follow GEF IEO Terminal Evaluation Guidelines• Done 6 months (±) of project completion• Reviewed by Agency evaluation office ( WB, UNDP and UNEP)• Submitted to the GEF IEO within 12 months of evaluation
completion
Page 22
Terminal Evaluation Reviews by the GEF IEO
• GEF IEO reviews– Terminal evaluations not reviewed by the Agency Evaluation Offices – Random “spot checking” for the terminal evaluations reviewed by the
Agency Evaluation Offices
• Primary reviewers prepare draft reviews• Peer reviewers provide feedback• If required senior reviewers confirm the findings• Draft reviews shared with Agencies• Agency feedback incorporated
Page 23
Terminal Evaluation Review
• The Annual Performance Report (APR) presents aggregated ratings for the portfolio on:– Outcomes– Sustainability– Supervision– M&E– Progress to Impact
• First presented in OPS 5, focus on GEF contributions to long term processes
Page 24
Management Response and Follow up to Evaluations
• A management response is required for all evaluations and performance reports presented to the GEF Council by the GEF IEO. It is coordinated by GEF CEO with inputs from other GEF stakeholders.
• GEF Council takes into account both the evaluation and the management response when taken a decision.
• GEF IEO reports on implementation of decisions annually trough Management Action Record (MAR).
• IEO is available to provide information on management requests– Examples are: OPS 5 follow-up in RBM, Gender, KM
