The Evaluation of the Paris Declaration

Julia Betts IOD PARCHigh Level Global Meeting: Gender Responsive Budgeting and

PlanningKigali July 2011

Presentation

• Part 1 – background• Part 2 – findings• Part 3 – possible implications

3

Part 1: BackgroundBackground • The Declaration pledged an independent evaluation - mutual

accountability• Fully joint evaluation over 4 years (Phase 1: 2007-8; Phase 2: 2009-

11) • Governance structure – International Reference Group (52

members)• International Synthesis reportEvidence base• 22 Country-level evaluations led by partner countries (managed in-

country)• 18 Donor/agency HQ studies• 7 supplementary studies on key topicsA main source for Busan

4

Phase 2: Three Questions

1. The PD in Context - “What factors have affected the relevance and implementation of the Paris Declaration?”

2. Aid Effectiveness - “Has the implementation of the Paris Declaration led to improvements in the efficiency of aid delivery, the management and use of aid and better partnerships?”

3. Development Effectiveness - “What contributions can aid effectiveness reforms plausibly be judged to have made to development results?”

555

Country Evaluations & Donor Studies

6

Gender and exclusion in the Evaluation

• Evaluation Matrix - AAA commitments and G / E integrated throughout and peer reviewed

• Specific sub-question 3b – Development Results ‘Did the implementation of the PD / AAA help countries to

improve the prioritisation of the needs of the poorest, including women and girls?’

• Within ToRs for country evaluations• Guidance and support• However…donor evaluations used ToRs from Phase 1

7

Gender / Exclusion Results Logic – Examples

INTENDED RESULTS

Greater prioritisation of the needs of the poorest including women and girls

Reductions in social exclusion

INTERIM RESULTS

Disaggregation of data and analysis Increased recognition within policy and planning?

Improved institutional machinery

Increased resource allocations / expenditure flows

CONTRIBUTION OF AID

What has been the scale of aid?

Intensity and productivity of policy dialogue?

Joint recognition of barriers to achievement

Strategies in place e.g. joint thematic platforms, strategies and reviews?

EFFECTS OF THE PD / AAA ON THE AID RELATIONSHIP

Joint statements / dialogue / analysis?

Joint programmes / activity?

Joint groups / structures?

Institutional commitments?

8

Part 2: Findings: The Central Message

In a context of changing development partnerships…

• The global campaign to make international aid programmes more effective is showing results.

• But the improvements are slow and uneven in most developing countries and even more so among donors – although the changes expected of them less demanding

• There has been better progress among partner countries than among donors, who (with some striking exceptions) have been too uncoordinated and too risk averse to play their full part

9

Implementation of the 5 Paris Principles

• Country ownership has advanced farthest • Alignment and harmonisation have improved

unevenly. • Mutual accountability and managing for results

lagging most

• Action on mutual accountability is now the most important need - backed by transparency and a realistic acceptance & management of risks

10

Findings on gender and exclusion

Evidence base mainly against Q3b)

Core findings1. Little progress in most countries studied in delivering on

these commitments – some exceptions2. But evidence of some positive contributions by aid and

some value-added by Declaration reforms. – building partnership-based frameworks for dialogue and

implementation– programming / targeted and non-targeted– monitoring and reporting on results– leveraging in resources

11

Challenges and barriers

Barriers identified• Lack of political will• Absence of tools and mechanisms to implement

policy• Lack of capacity• Lack of resource• Lack of data• Donor–govt tensions

12

Afghanistan: a conflict setting

‘Finally, the evaluation in Afghanistan is harshly critical of bothgovernment and donors on their minimal responses to the

needs of women and girls, an especially crucial issue in thatcountry. The implication is that donors could and should

have done more to push these priorities, given their prominentrole in the country…This case is another

illustration of the primacy of national ownership and the limits of aid and aid reforms when confronted with powerful

obstacles of ingrained resistance and limited nationalcommitment to profound development change.’

13

Main Recommendations I

A. For decision-makers in both partner and donor countries and agencies (at Busan and beyond):

1. Make the hard political choices and follow through

2. Focus on transparency, mutual accountability and shared risk management

3. Centre and reinforce the aid effectiveness effort in countries (take it home)

4. Work to extend the aid reform gains to all forms of development cooperation

5. Reinforce the improved international partnerships in the next phase of reforms

14

Main Recommendations 2B. For policymakers in partner countries: 1. Take full leadership and responsibility at home for further aid

reforms 2. Set strategies and priorities for strengthening capacities 3. Intensify the political priority and concrete actions to combat

poverty, exclusion and corruption

C. For policymakers in donor countries and agencies:1. Match the crucial global stakes in aid and reform with better

delivery on promises made2. Face up to and manage risks honestly, admit failures 3. Apply peer pressure to ‘free-riders’ for more balanced donor

efforts

15

Recommendation 8 For partner countries.

Intensify the political priority and concrete actions to combat poverty, exclusion and corruption

The Evaluation has confirmed – in assessing the widespread lack of progress for the poorest, and particularly women and girls – that even the best of aid and aid reforms can encourage and reinforce, but not replace, strong and effective national commitment and action.

[AAA commitments and priorities] reflect not only widespread expectations but also shared international commitments…

They must be central to both development strategies and to the continuing dialogue around aid and its implementation.

16

Quote

‘Compared with the aid situation 20 to 25 years ago current practice presents a global picture of far greater transparency and far less donor-driven aid today…. The Declaration has raised expectations for rapid change, perhaps unrealistically, but also strengthened agreed norms and standards of better practice and partnership.

There is no going back – expectations are more likely to keep

rising than to diminish – so that the standard expected has permanently been raised for all engaged in development cooperation’.

17

Part 3: Possible Implications?• ‘There is no going back’• Follow through - from commitment / policy to implementation

5 themes arising:• Political will and partnership are pre-requisites• Capacity

– institutional strengthening / institutionalisation– attitudes and culture – all levels

• Policy coherence• Accountability

– data & analysis– aid compact at country level accountable – shared outcomes / indicators – tools: markers / budget classifications

• Results– Clear objectives –- clear steps along the way / risks– Focus on impact– Interconnections – from budget to results

18

Full reports and supporting materials

Thank you for your attention

All documents from the Evaluation, including the full country evaluations and donor studies, can be found - in English,

French and Spanish - on

www.busanhlf4.org and

www.oecd.org/dac/evaluationnetwork/pde

19

Phase 2

• Evaluation Matrix - 11 outcomes of PD and AAA agreements.

• Governance structure - International Reference Group (52 members) / Management Group

• Integrated QA and peer review• Recognised the limits of aid in development and

applied “contribution analysis.”• Targeted process of guidance & support, recognising

the primacy of country studies.

20

Evaluation components

21

Key limitations

• Evaluating the effects of a political Declaration - traditional ‘linear’ approaches were not relevant

• Limited time elapsed since 2005 • No comprehensive data from country studies on

multilaterals and donors • Different methodology for donors (carried over from

Phase 1) • Self-selection of participating countries / agencies –

but still reasonably representative “sample”

22

Context: Aid and aid reform in the bigger picture

• Diversity is the rule: The Declaration campaign proved relevant to many countries and agencies, but differently. All were engaged in aid reforms before 2005, some were far more advanced than others.

• Limits of aid and aid reform: The Evaluation highlights the other powerful influences at work in development and the realistic limits on the role of aid.

• Key political, economic and bureaucratic influences and events – e.g. food and fuel crises, global recession and natural disasters - have shaped and limited the reform process in partner and donor countries, as well as aid and development.

• The effects of different contexts come out repeatedly. So do questions about the changing nature and the roles of aid alongside other resource flows and relationships. But the basic lessons of decades about aid itself are still valid.

23

Examples of the range of performance against each intended improvement (From Fig. 5)

24

Context: Aid reform in perspective

Aid influenced by Declaration commitments

The Aid Partnership

Overall development processes

Other international

& national influences &

forces

25

Contributions to aid effectiveness• Pulled together and focused global attention on ambitious,

experience-based measures to improve development cooperation and aid for better development results

• Clarified the roles of ‘aid’ and ‘better aid’

• Strengthened global norms of good practice

• Helped progress toward 11 key outcomes set in 2005• Improved the quality of aid partnerships and supported

rising aid volumes and expectations for improved “North-South” relations at a global level

• Better progress among partner countries than among donors, who (with some striking exceptions) have been too uncoordinated and risk averse to play their full part

26

1. Stronger national strategies and operational frameworks2. Increased alignment of aid with country systems3. Meeting defined measures and standards, e.g. in financial

mgt.4. Reduced duplication of donor effort, more cooperation5. Reformed and simplified donor policies and procedures 6. Increased predictability of aid7. Sufficient delegation to donor field staff8. Sufficient integration of global initiatives 9. Increased capacity 10. Enhanced accountability 11. Reduced corruption and increased transparency

The 11 intended improvements for effective aid

27

Contributions to Development Results 1

Assessed in four key areas, through a three-question sequence:– First, were development results achieved? – Second, did aid contribute? – Third, did aid reforms plausibly strengthen the aid

contribution?

28

Contributions to Development Results 3

3. Strengthening institutional capacities and social capitalInsufficient capacity still a central obstacle to development - and aid could help more with this than it does. Modest contributions by aid and reforms to the long-term strengthening of institutional capacities. Clearer evidence for contributions to modest improvements in social capital.

4. Improving the mix of aid modalitiesEvidence that employing a wider range of (especially joint) modalities, has improved contributions to development results in half the countries – especially at sector level. A mix of aid modalities has continued to make sense for all actors.

29

The aid reform campaign

30

Overall Conclusions 1Relevance of the Declaration and its implementation?• Proven relevant to all the diverse countries and agencies

involved• For partner countries - Slow and varied implementation but

overall reforms have now generally taken hold. Reforms serve wider national needs than aid alone, and momentum has held up through political changes and crises.

• For donors – Much more uneven implementation. With striking exceptions, donors have been risk-averse and slow to make the less demanding changes expected of them. Peer pressure and collective action are not yet embedded in donor systems.

• The nature and place of aid itself is changing. Aid actors, forms of co-operation and partnerships not yet covered also need greater transparency and proven good practices.

31

Overall ConclusionsWhat has the aid reform campaign achieved? • Now more focused global attention on relevant problems and

remedies – succeeded as an international “compact” for reform

• Compared with 20 to 25 years ago, aid is now far more transparent and less “donor-driven.” Since 2005 scattered reforms have become widespread norms

• Raised expectations for change, strengthened agreed norms and standards of better practice and partnership. Legitimised demands for norms of good practice to be observed

• Sustainability – Paris reform agenda now seen to serve more important needs than aid management

• A platform for the future – applying and adapting the disciplines of aid reform to forms of development co-operation not yet covered by the Declaration