worldbank fmis & results - hadden

IFMIS Implementations

a history of success and failures

World Bank study

http://blog-pfm.imf.org/

Context: What we Know and What we Don’t Know • 2003 FMIS report

– disappointing results• 2010 Report

– World Bank projects 1984-2010– 55 closed, 32 active– primarily from World Bank documents &

analysis

55 closed, 32 active FMIS projects

http://maps.google.com/maps/ms?ie=UTF&msa=0&msid=101777630030704959267.000491cccf1c14ff54bda

FMIS Profiles• Client/Server through web-based• Small countries to large• Very low to high capacity• Across all World Bank regions (& regional

variances)• COTS and LDSW• Narrow focused Treasury to large

integrated systems• Some with multiple World Bank loans

Document Structure

• Descriptive Data Analysis• Project Performance• 5 Case Studies• Conclusions• Appendixes

FMIS Benefits

• Predictability• Participation• Transparency• Accountability

1. How many project completed on budget?1. Under 1/32. Between 1/3 and 2/33. Above 2/3

3. 82%

On BudgetOver Budget

What factor most influences FMIS project failure?1. Complex project design/large # of procurement

packages2. Inadequate capacity/training of project teams3. Inadequate ICT infrastructure4. Inappropriate technology5. Ineffective project coordination6. Lack of leadership commitment7. Lack of proper skills in project team8. Organizational structure poorly suited for

integration9. Weak project preparation and planning

3. Inadequate capacity/training of project teams

Unclear delineation of authority to implement

External environment (political unrest, disasters)

Ineffective project coordination

Inappropriate technology

Lack of leadership commitment

Lack of proper skills in project team

Inadequate ICT infrastructure

Organizational structure poorly suited for integration

Weak project preparation and planning

Complex project design/large # of procurement packages

Inadequate capacity/training of project teams

0% 10% 20% 30% 40% 50% 60%

What is the most important FMIS success factor?1. Adequate preparation and clarity of design2. Close World Bank supervision3. External environment (uncontrollable)4. Flexible project management5. Focus on capacity building and training6. Good project management and coordination7. Pre-existing enabling environment (ICT, HR,

accounting)8. Suitable political environment & committed

leadership

5. Focus on capacity building and training

External environment (uncontrollable)

Good project management and coordination

Adequate preparation and clarity of design

Pre-existing enabling environment (ICT, HR, accounting)

Flexible project management

Suitable political environment & committed leadership

Close World Bank supervision

Focus on capacity building and training

0% 10% 20% 30% 40% 50% 60%

Trend Lines COTS & LDSW• Cautions

about making conclusions

Case Studies

• Outcome, sustainability, development impact, bank performance, borrower performance– Implementation Completion Report– Independent Evaluation Group

Blending the evaluation criteria, which country FMIS project was highest rate?

1. Albania2. Guatemala3. Mongolia4. Pakistan5. Turkey

2. GuatemalaAll Average

Outcome

Sustainability

Development Impact

Bank performance

Borrower performance

MongoliaTurkeyAlbaniaGuatemalaPakistan

ICR FMIS ICT rating

ICR Program Development Objectives

ICR Implementation

ICR Project Outcome

ICR Sustainability

IEG Sustainability

IEG Project Outcome

IEG Development Impact

0% 20% 40% 60% 80% 100%

HighSatisfactoryBelow

Checklist

What is the largest barrier to making FMIS software sustainable?1. Civil servant capacity building2. Civil servant retention3. Costs of maintaining technical

infrastructure: computers, networks, data centres, database software, virus protection etc.

4. Costs to adapt software for reform5. Costs to maintain and upgrade FMIS

software

What is the most important pre-requisite for FMIS success?1. Budget classification2. unified chart of accounts, integrated with

budget classification3. commitment control and monitoring

mechanisms4. cash management functions5. secure countrywide communication network6. system/data centers7. core team of ICT specialists within PFM

organizations8. Treasury single account operations

What recommendation do you think has the biggest impact on reliability + cost effectiveness of an FMIS?

1. Using electronic payment systems2. Using digital/electronic signatures for all transactions3. Electronic document management4. Publishing budget execution and performance

monthly5. Interoperability and reusability of the information

system6. FMIS development and project management based

on international standards7. Using Free/Open Source Software (FLOSS) in PFM

applications

Which Success Factor do you think is most important?1. proper attention to capacity building

and training plans,2. close World Bank supervision of the

projects,3. strong leadership and a conducive

political environment, and4. Flexibility in the way the project was

designed and managed.

What transparency initiative is or would be the most beneficial for your country?

1 2 3 4 5

8%

58%

23%

10%

3%

1. Aid transparency2. Budget execution +

financial reports3. Budget preparation +

budget books4. Civil service transparency in

hiring and salaries5. Revenue transparency –

taxes, extractive industries

How many projects were completed on budget?

1 2 3

60%

7%

33%

1. Under 1/32. Between 1/3 and 2/33. Above 2/3

What factor most influences FMIS project failures?

1 2 3 4 5 6 7 8 9

8%

25%

8%

3%

23%

3%

8%

23%

3%

1. Complex project design/large # of procurement packages

2. Inadequate capacity/training of project teams

3. Inadequate ICT infrastructure4. Inappropriate technology5. Ineffective project coordination6. Lack of leadership commitment7. Lack of proper skills in project

team8. Organizational structure poorly

suited for integration9. Weak project preparation and

planning

What is the most important FMIS success factor?

1 2 3 4 5 6 7 8

35%

7%

4%

24%

9%

11%11%

0%

1. Adequate preparation and clarity of design

2. Close World Bank supervision3. External environment

(uncontrollable)4. Flexible project management5. Focus on capacity building and

training6. Good project management and

coordination7. Pre-existing enabling environment

(ICT, HR, accounting)8. Suitable political environment &

committed leadership

What country listed in the case studies do you think had the best blended score (outcomes, sustainability, bank performance, borrower performance)?

1 2 3 4 5

12%

30%

21%

14%

23%

1. Albania2. Guatemala3. Mongolia4. Pakistan5. Turkey

What is the most important pre-requisite for FMIS success?

1 2 3 4 5 6 7 8

15%

40%

10%

5%

15%

3%

10%

3%

1. Budget classification2. Development of a unified chart of

accounts, integrated with budget classification

3. Development of commitment control and monitoring mechanisms

4. Establishment of cash management functions

5. Establishment of secure countrywide communication network

6. Preparation of system/data centers7. Presence of a core team of ICT

specialists within PFM organizations

8. Treasury single account operations

What is the largest barrier to making FMIS software sustainable?

1 2 3 4 5

26%

18%

10%

18%

28%

1. Civil servant capacity building

2. Civil servant retention3. Costs of maintaining

technical infrastructure: computers, networks, data centres, database software, virus protection etc.

4. Costs to adapt software for reform

5. Costs to maintain and upgrade FMIS software

Which recommendation do you think has the biggest impact on reliability + cost effectiveness of a FMIS?

1 2 3 4 5 6 7

34%

5%

12%

7%

22%

5%

15%

1. Using electronic payment systems

2. Using digital/electronic signatures for all transactions

3. Electronic document management

4. Publishing budget execution and performance monthly

5. Interoperability and reusability of the information system

6. FMIS development and project management based on international standards

7. Using Free/Open Source Software (FLOSS) in PFM applications

Which of the following options is the most critical success factor for a successful FMIS?

1 2 3 4

32%

16%

43%

9%

1. Proper attention to capacity building and training plans

2. Close World Bank supervision of the projects

3. Strong leadership and a conducive political environment

4. Flexibility in the way the project was designed and managed