The Results Agenda: Where Are We?

Overview and Progress to Date

The Results Agenda: Where Are We?

Overview and Progress to Date

DAC Development Partnership Forum

December 11, 2002

2

Table of Contents

I. Managing for Results : Background

II. Managing for Results : Action Plan

III. Terminology, Abbreviations and Acronyms

3

I. Managing for Results: Background

4

External Context

• Building on the Monterrey/Doha/Johannesburg (“MDJ”) mandates : focus on country outcomes, emphasis on Millennium Development Goals (MDGs)…

• June - International Roundtable on Better Measuring, Monitoring and Managing for Development Results

• July - IDA13 Agreement and focus on country results, including MDGs and related goals

• September - Development Committee endorsed Bank’s overall approach to increasing its results orientation and urged implementation.

5

Internal Context

• Country Business Model: Discussed by Development Committee in Prague; identified four step process—vision (PRSP); diagnosis (CPIA); programming (CAS); and results. Much work hitherto on first three.

• Results-Based Management: Bank has been managing for deliveries and quality. Much work and progress on system development (for tracking lending and knowledge services) and on quality.

• Development Effectiveness: OED and QAG analyses provide basis for accountability and learning; need to strengthen links to strategy formulation and and day-to-day management.

Bottom line: All point to timeliness of enhanced results focus …

6

II. Managing for Results:Action Plan

7

Managing for Results: Implementation

December 18 : CODE Discussion of Implementation Plan

• Support for national strategy development (PRSP…) and capacity building

Knowledge accumulation and dissemination

Bank strategy and instruments

Staff learning and incentives

Corporate reporting

8

PRSP and Capacity Building

Strengthen support for results-focused PRSPs, with clear linkages. Publish PRSP Sourcebook with relevant guidance on measurement and monitoring; and publish guidelines for JSAs on Annual Progress Reports. Focus staff assessments of PRSPs and Annual Progress Reports focused on targets and monitoring of results. FY03 Q1-4

Help countries build statistical systems and analytic capacity to measure and track outcomes and other results. In consultation with partners, design and pilot instrument for enhanced sector-wide support for statistical capacity building. (STATCAP template has already been developed, as simplified financing instrument.) FY03 Q1-4

Help countries build results-based public expenditure management systems, and monitoring and evaluation capacity. Develop/agree Bank approach across units, as basis for reflecting diagnostic tools and possible follow-up support to countries via CAS process. FY03

9

Client-Focused Knowledge Accumulation and Dissemination

Support countries’ achievement of MDGs and other development outcomes through advice and research. Undertake research and prepare policy notes on actions that lead to the achievement of the MDGs, as input into country teams preparing outcome-oriented ESW. (The first deliverables to be based on six country studies—Albania, Argentina, Bangladesh, Uganda, Vietnam, and Yemen—currently underway.) WDR 2004 to present lessons learned on how countries can accelerate progress toward the MDGs by making services work for poor people. FY03

Provide clients with a clear and consistent window for accessing knowledge and best-practice examples across sectors and themes. Review existing knowledge dissemination mechanisms. Adopt harmonized approach to dissemination and knowledge management, making use of Development Gateway. FY03 Q3

10

Bank Strategy and Instruments

Establish results-based CAS to show clear results-chain linking country outcomes to Bank products and services. Launch results-based CASs in five countries. Five pilots : Brazil, Cambodia, Cameroon, Sri Lanka, and Ukraine. FY03

Develop results-based CAS monitoring and evaluation framework. Pilot CAS completion report and associated peer-review mechanism. FY03

Increase feedback from clients on Bank contribution to results. Revise client survey instrument and align timing with CAS cycle. FY03 Q3-4

Enhance results focus of products and services, and linkages to CAS outcomes. As part of the ongoing review of the design of, and guidelines for, Bank lending and nonlending products, strengthen respective monitoring and evaluation frameworks to enhance results focus and alignment with results-based CAS. FY03-04

11

New CAS Results Framework

• Objective is to improve Bank performance by using intended results to define Bank program design ex ante and to assess program success ex post.

• Starts from country objectives (outcome and outputs) to set Bank’s country program (inputs), taking into account what other partners are doing.

• Framework includes CAS matrix and M&E Framework, follow-up in the CAS Progress Report, CAS Completion Report, QAG peer review CAS exit assessment, and OED Country Assistance Evaluation (CAE). This is akin to project cycle M&E framework.

12

CAS—Results Tracking

Next CAS

CAS

CAS Progress Report

CAS Completion

Report

CAS Exit Assessment

(QAG)

Country Assistance Evaluation

(OED)

Lessons Learned

CAS Results Matrix provides links across various reports

13

CAS Results Matrix

CAS matrix to capture three main areas:

• CAS Country Outcomes—selected country goals from the PRSP (or national strategy), such as the MDGs and sector-specific issues that contribute to these goals.

• CAS Intermediate Outcomes—expected country outcomes achievable in the CAS period, which Bank’s interventions support in partnership with other donors.

• Bank Inputs and Outputs—planned Bank interventions supporting these CAS intermediate outcomes (taking account of partners’ activities) with associated targets to monitor Bank performance management.

14

CAS Implementation: Assessing Progress

During CAS implementation, CAS results matrix used to:

• Track priority CAS outcomes

• Assess annual progress towards goals

• Review strategies and actions under way

• Review key indicators

CAS Progress Reports to become more systematic, assessing progress against results framework—and determining need for mid-course correction in CAS strategy and/or programs.

15

CAS Completion and Evaluation of Results

• Self Assessment: At end of CAS cycle, the team prepares the CAS Completion Report (CCR) using results framework to assess extent to which results were achieved. For key areas of Bank involvement, CCR would compare baseline values, expected outputs and outcomes as laid out in CAS, and actuals at time of CAS completion. CCR would be an input into CAS design, annexed to the CAS provided to Executive Directors.

• Peer Assessment: QAG would assess quality/results at exit, drawing on CCR and other information.

• Independent Assessment: As at present, OED would periodically carry out Country Assistance Evaluation, typically spanning more than one CAS period.

16

PRSPM&E

Framework

CAS ResultsFramework

Other Donors’Portfolio’s

M&E

InvestmentLending M&E

AdjustmentLending M&E

Linking M&E Frameworks

ESW Other

Analytic &advisory work

17

M&E Framework for Results-focusedLending Operations

• Existing framework for projects consists of logical framework based on development objectives that are monitored through self-assessment, peer-assessment and independent assessment throughout project cycle.

• Framework is broadly appropriate, but results-focus needs to be sharpened through:

Development objectives that are more outcome-oriented and quantified

More logical links between expected outputs and development objectives

Monitorable indicators of both outputs and outcome-oriented development objectives.

• New priority is coherence with PRSP and CAS M&E frameworks. This is not just good architecture; it also provides a motivational key to systematic implementation.

18

Staff Learning and Incentives

Ensure that staff learning activities, in terms of strategic priorities, timeliness and specific learning content, are aligned with focus on sustained development results. Learning Board to proactively manage staff learning, including in allocation of FY04 resources. FY03-04

Make best practice into standard Bank practice in managing for results. Good Practice Handbook on Managing for Results, with framework and best-practice examples FY03 Q3-4

Provide staff with consistent training on results-based approaches. Assess training needs. Review and rationalize supply of results-based M&E training. FY03 Q2-4

Ensure that internal staff incentives are aligned with focus on sustained development results. Review staff incentives and revise as necessary to reflect enhanced results focus. FY03

19

Corporate Reporting

Report on Results in corporate strategy and budget documents. Review of corporate planning and budget cycle to assess methods for integrating results. Pilot test in FY03 for further integration in corporate reporting in FY04. FY03-FY04

Enhanced results focus of quality assessments of products and services. Increase results focus of quality at entry, supervision, and ESW assessments carried out by Quality Assurance Group. Develop Annual Report on Portfolio Performance and Results. FY03 Q2-4 FY04

Design system to report IDA Deputies on IDA-eligible country results. Develop and refine results measurement system for IDA-eligible countries based on country-specific results defined in PRSPs and supported through CASs and operations. FY03 Q1-4

20

Calendar of Events

• November 4 : IDA13 Technical Meeting

• November 12 : Informal Board Meeting - “Building Statistical

Capacity to Monitor Development Progress”

• December 11-12 : DAC Development Partnership Forum

• December 18 : CODE Discussion of Implementation Plan

February 23-25 : Joint High-Level Forum on Harmonization

April 11, 2003 : IDA Donors Meeting on Results System

(tentative date)

April 13, 2003 : Development Committee

21

III. Results :Terminology, Abbreviations and Acronyms

22

Definitions: Results Terminology

Input Financial, human and material resources used for a development intervention

Output Products, goods and services which result from a development intervention

Outcome Likely or achieved effects or behavior changes resulting from a development intervention

Performance Degree to which a development intervention or partner operates according to specific standards or criteria

Result The output, outcome or impact of a development intervention

Results Chain

The causal sequence for a development intervention, moving from inputs and activities to outputs, outcomes and impacts.

23

Abbreviations and Acronyms

CAS Country Assistance Strategy

CODE Committee of Development Effectiveness

CPIA Country Policy and Institutional Assessment

DAC Development Assistance Committee

ESW Economic and Sector Work

IDA International Development Agency

JSA Joint Staff Assessment

M&E Monitoring and Evaluation

OECD Organization for Economic Co-operation and Development

OED Operations Evaluation Department

PRSP Poverty Reduction Strategy Paper

QAG Quality Assurance Group

WDR World Development Report