1

IndependentEvaluation in IFCPresentation to Staff of Islamic Development Bank May 3, 2009

Marvin Taylor-Dormond

IndependentEvaluation in IFCPresentation to Staff of Islamic Development Bank May 3, 2009

Marvin Taylor-Dormond

2

Constructing an Evaluation Framework

• Why do we evaluate?

3

Why do we evaluate?

• Accountability

• Learning

• Allocation of resources / Better results

4

Director-General, Evaluation (DGE)

IEG: independent reportingIEG: independent reporting

PresidentWorld Bank

Group

Board

Multilateral Inv. Guarantee

AgencyWorld Bank

IEG-MIGAIEG-WB IEG-IFC

5

Micro Evaluation

Macro/Aggregate Evaluation

Quality,Knowledge,

Partnerships, Communication

IEG-IFC structure

6

Micro Evaluation

IEG-IFC structure

7

• Independent project-level evaluation of IFC’s investment and advisory operations

• Accountability for IFC’s operations at the project level

• Provides the foundation for IEG’s macro evaluations

Scope of micro evaluation

8

Self-evaluation – IFC Staff• IFC staff prepare Expanded Project Supervision Reports

(XPSRs)

Evaluation of investment projects (XPSR system)

Independent Evaluation - IEG-IFC • Sample selection• Guidance• Independently validated XPSR ratings • Collect and disseminate lessons• Desk-based or field-based validations• Mini-XPSRs / Project Evaluation Summaries

9

Independent Evaluation - IEG-IFC • Independent evaluation / validation of PCRs• Desk review of adherence to guidelines in PCR ratings• Desk and field reviews for validation of results

Evaluation of advisory services (PCR evaluation)

Self-evaluation – IFC Staff• IFC staff prepare Project Completion Reports (PCRs)

10

Measuring development Measuring development

DevelopmentEffectiveness

DevelopmentOutcome

Investment Operations Advisory Services

ProjectBusinessSuccess

EconomicSustain-ability

Environ-mental &

Social

PrivateSector

Devpmnt

StrategicRelevance

OutputAchievement

OutcomeAchievement

ImpactAchievement

Efficiency

11

Highly unsuccessful

Unsuccessful Mostly unsuccessful

Mostly successful

Successful Highly successful

1. Project’s Development Outcome1. Project’s Development Outcome

Satisfactory ExcellentUnsatisfactory UnsatisfactoryPartly

Project’s impacts on:

Business success

Economic sustainability

Private sector development

Environmental & social effects

Investment project evaluation: Ratings

XPSR example: Real Sector

12

Macro/Aggregate Evaluation

IEG-IFC structure

13

• Focus on and inform IFC’s strategies, policies & procedures

• Provide accountability• Country, sector, & thematic evaluations• Forward-looking

Scope of macro evaluation

14

StudiesStudies

• Country and regional• Thematic• Sectoral• Corporate• Process

15

Outcome

Five drivers explain two-thirds of project outcomes

Five Outcome Drivers

16

Frontier strategy targetshigh-risk countriesFrontier strategy targetshigh-risk countries

Success rates were

HIGHER for projects incountries that

were HIGH riskat approval0%

10%

20%

30%

40%

50%

60%

70%

Developmentoutcome

IFC investmentoutcome

Developmentoutcome andinvestmentoutcome

Su

cce

ss r

ate

Low HighCountry risk at approval:

Development Success rate, 2005-2007 (%)

17

Improvement in business climates key to successImprovement in business climates key to success

Country risk changes from Approval Evaluation

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

Investment Outcome

Su

cc

es

s r

ate

Worsened Got better

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

Development outcome

Su

cce

ss

ra

te

Worsened Got better

Development Success rate, 2005-2007 (%)

18

A low-quality sponsor can jeopardize project successA low-quality sponsor can jeopardize project success

• IEG defines sponsors in terms of experience, financial capacity, commitment, and reputation

• Low sponsor quality was prevalent in about 35-40% of IFC commitments between 2002 and 2008

• Sponsor quality can vary considerably by sector, and even within the same sector

19

Better work quality leadsto better outcomesBetter work quality leadsto better outcomes

Work quality comprises:• Screening, appraisal and structuring• Supervision & administration• Role & contribution throughout investment

cycle

With high work quality:• Development success increased

from 18% to 73% • Investment success increased

from 52% to 74%

20

Tracking RecommendationsTracking Recommendations

ManagementAction Tracking

IFC updatesstatus and

level of adoption (LOA)

Annual Updates

IEG updates its own

Assessmentof the status

and LOA

IEG and IFC agree on appropriate

indicators

Step Two

IEG recommendation and IFC Management response

recorded

Step One

InternalPublication

ExternalPublication

Dissemination

21

Knowledge,Dissemination, Communication

IEG-IFC structure

22

What we do

• Disseminate lessons learned from IFC’s experience among staff for better operational results

• Build knowledge on private sector development issues

• Disseminate IEG evaluations to public-at-large for accountability and awareness

• Develop and foster partnerships with evaluators in other development institutions

23

Products we offerProducts we offer

StudiesStudies

Annual Review

Portfolio presentations

PublicationsIntranetTraining

Quick TakesTop Lessons(country, sector,subject)

Quick TakesTop Lessons(country, sector,subject)

Lessons Retrieval Network (LRN)

Lessons Retrieval Network (LRN)

Learning by doing

Staff use evaluations

Project level evaluations

StudiesStudies

Studies & Publications

Annual Review

Portfolio presentations

PublicationsIntranetTrainingAnnual

Review

Findings &Evaluation

Briefs

Portfolio presentations

Portfolio presentations

PublicationsIntranetTraining

BBLs Intranet Training

Quick TakesTop Lessons(country, sector,subject)

Quick TakesTop Lessons(country, sector,subject)

Quick TakesTop Lessons(country, sector,subject)

Quick TakesTop Lessons(country, sector,subject)

Lessons Retrieval Network (LRN)

Lessons Retrieval Network (LRN)

Lessons Retrieval Network (LRN)

Lessons from operations:

e-LRN

Learning by doingLearning by doing

Staff use evaluationsStaff use

evaluationsProject level evaluationsProject level evaluations

24

Easy search

Featured lessons

Newest lessons

E-LRN Home page

Most commented

lessons

Best

lessons

25

• Independent Evaluation of IFC’s Development Results 2007 & 2008

• Evaluation of IFC’s Private Enterprise Partnership Program – Europe and Central Asia

• Evaluation of IFC’s Assistance to Micro, Small and Medium Enterprises

• Ukraine, Indonesia and Nigeria Country Impact Reviews

• Evaluation of WBG’s Experience with Guarantees

Reports disclosed to date…

26

BROE: Adequacy, coverage, and quality of IFC’s development results measurement systems

Health: An assessment of the WBG’s support for health, nutrition, and population since1997

Two recent reports…

27

Accountability & DisseminationAccountability & Dissemination

28

“I could tell you of projects that failed because lessons were

ignored or missed”

75% of respondents the June 2005 IFC Knowledge Survey

Learn from experience …Learn from experience …

29

Call, e-mail, stop by!

IEG can help get better resultsIEG can help get better results

www.ifc.org/ieg

IEG Help Desk1 (202) 458-2299

AskIEG@ifc.org

30

IndependentEvaluation in IFCPresentation to Staff of Islamic Development BankMay 3, 2009

Marvin Taylor-Dormond

IndependentEvaluation in IFCPresentation to Staff of Islamic Development BankMay 3, 2009

Marvin Taylor-Dormond