presentation to dpsg october 20, 2009
DESCRIPTION
OECD DAC Evaluation of Donor Activities in Support of Conflict-Sensitive Development and Conflict Prevention and Peacebuilding in Sri Lanka A Pilot Test of OECD DAC Guidance. Presentation to DPSG October 20, 2009. The Evaluation Study. The Purpose : - PowerPoint PPT PresentationTRANSCRIPT
OECD DAC Evaluation of Donor Activities in OECD DAC Evaluation of Donor Activities in Support of Conflict-Sensitive Development Support of Conflict-Sensitive Development and Conflict Prevention and Peacebuilding and Conflict Prevention and Peacebuilding
in Sri Lankain Sri LankaA Pilot Test of OECD DAC GuidanceA Pilot Test of OECD DAC Guidance
Presentation to DPSG Presentation to DPSG
October 20, 2009October 20, 2009
The Evaluation Study The Evaluation Study
The Purpose : The Purpose : to collect evidence on the applicability of the draft OECD to collect evidence on the applicability of the draft OECD
guidance that would enable its finalization, guidance that would enable its finalization, to provide targeted advice and support to DAC partners at to provide targeted advice and support to DAC partners at
headquarters and in the field to improve their effectiveness headquarters and in the field to improve their effectiveness and impactand impact
Three outputs :Three outputs :(i)(i) a report that presents the results of the pilot a report that presents the results of the pilot
exercise in Sri Lanka in November 2008, exercise in Sri Lanka in November 2008, (ii)(ii) a lessons learned paper documenting the process a lessons learned paper documenting the process
of conducting the pilot evaluation, and of conducting the pilot evaluation, and (iii)(iii) edited comments on the OECD DAC Guidance.edited comments on the OECD DAC Guidance.
Areas of focusAreas of focus Initial TOR ambitious : so narrowed focus and based evaluation on large Initial TOR ambitious : so narrowed focus and based evaluation on large
evidence base of evidence base of published published strategies and evaluations strategies and evaluations 17 strategies from 10 donors 17 strategies from 10 donors 28 evaluations from 13 donors28 evaluations from 13 donors
Excluded track 1, political/ diplomacy, security, humanitarianExcluded track 1, political/ diplomacy, security, humanitarian No independent baseline or conflict analysis: used SCA1+2No independent baseline or conflict analysis: used SCA1+2 Looked at Relevance (Strategies), Results (Evaluations), Process Looked at Relevance (Strategies), Results (Evaluations), Process
(Coordination)(Coordination) Three phases covered: Three phases covered:
Pre-Cease Fire Agreement periodPre-Cease Fire Agreement period 2002-2005 – CFA period2002-2005 – CFA period 2005 on – new govt, war situation2005 on – new govt, war situation
Target groups (national, conflict-affected, and special groups: journalists, Target groups (national, conflict-affected, and special groups: journalists, police etc)police etc)
TimeframeTimeframe Issues Paper Issues Paper January 2008January 2008 Original TOR Original TOR April 2008April 2008 Team recruited Team recruited September 2008September 2008 Inception Inception October 2008October 2008 Main mission Main mission November 2008November 2008 First Draft First Draft February 2009February 2009 Final Draft Final Draft June 2009June 2009
TeamTeamNick Chapman Team Leader, Development Evaluation Specialist
Debi Duncan, Conflict and Peacebuilding Specialist
David Timberman, Governance and Human Rights
Kanaka Abeygunawardana, Local Facilitation
Context Context PovertyPoverty: SL lower-middle income status but poverty reduction uneven. : SL lower-middle income status but poverty reduction uneven.
2004 tsunami worsened poverty levels in the affected areas, and the N & E 2004 tsunami worsened poverty levels in the affected areas, and the N & E much worse than the rest of the countrymuch worse than the rest of the country
ConflictConflict: rooted in : rooted in failure to institutionalise democratic politicsfailure to institutionalise democratic politics not in ethnic not in ethnic differences (SCA2) and also differences (SCA2) and also ‘political culture, the institutional framework of ‘political culture, the institutional framework of policy, uneven development patterns, and competing nationalisms’policy, uneven development patterns, and competing nationalisms’
Development assistance Development assistance ADB, World Bank and Japan account for 60% of aid (2002-07) but have no ADB, World Bank and Japan account for 60% of aid (2002-07) but have no
mandate to work on political / governance issues mandate to work on political / governance issues Bilaterals are either exiting or reducing their programmesBilaterals are either exiting or reducing their programmes Newer partners have emerged some with more pro-government stance ~ China, Newer partners have emerged some with more pro-government stance ~ China,
India, Iran and Pakistan. India, Iran and Pakistan.
Increased emphasis on global security and terrorism, but tackling sensitive Increased emphasis on global security and terrorism, but tackling sensitive issues is difficult with little financial leverage and a strong (now ‘victorious’) issues is difficult with little financial leverage and a strong (now ‘victorious’) governmentgovernment
Fragile state thinking relatively new (DAC principles 2005)Fragile state thinking relatively new (DAC principles 2005)
Strategies included Strategies included Development
PartnerStrategy
1. Ausaid Development Cooperation Regional Framework 2003-07
2. ADB Country Strategy and Program 2002-04Country Strategy and Program 2004-08Country Partnership Strategy 2009-11
3. EC Cooperation Strategy 2002-06 Multi Annual Indicative Programme 2007 –10
4. Japan Country Assistance Program 2004
5. The Netherlands Multi Annual Strategic Plan 2005-08Multi Annual Strategic Plan 2009-11
6. Switzerland Medium Term Plan for Human Security 2007-09
7. Sweden Country Strategy 2003-07Country Strategy 2008-10
8. UN / UNDP Development Assistance Framework 2002-06 UNDP Country Cooperation Framework 2002-06
9. USA Country Strategy Plan 2003-07
10. World Bank Country Assistance Strategy 2003-06Country Assistance Strategy 2009-12
Four other countries (UK, Germany, CIDA, Norway) were unable to share their strategies.
Strategies ~ FindingsStrategies ~ Findings Many strategies promoted “peace,” and some provided support for the peace
process. Only a few explicitly addressed the root causes of the conflict Few strategies were based on in-depth or recurring conflict analysis Liberal use of ‘peacebuilding’ and ‘peace dividend’. But no serious consideration of
whether a “peace dividend” could change the attitudes of hardliners Most focus on ‘costs’ not ‘causes’ of conflict. So less attention paid to power
sharing, the political system and problems of injustice and impunity Little recognition of political risks (such as delivering aid through a party to the
conflict or supporting the agenda of a government that represented only a portion of the political spectrum and was vulnerable to electoral defeat).
Over-emphasis of the extent to which civil society and citizens could bring about transformation and peacebuilding.
Increasing use of scenarios in strategies Whole of government approach an important strategic approach ~ but difficult to
evaluate A weak approach to conflict sensitivity in early strategies, but this aspect was more
explicit in later strategies
Theories of changeTheories of change Theories of change are not explicit in strategies, though several have
implicit causal logic linking proposed actions and the achievement of outcomes
The most common involve:The most common involve: Community reintegration and grassroots mobilisation building a culture of peaceCommunity reintegration and grassroots mobilisation building a culture of peace Meeting basic needs and improving economic conditions leads to poverty Meeting basic needs and improving economic conditions leads to poverty
reduction and a ‘peace dividend’ reduction and a ‘peace dividend’ Reintegration of displaced people to live in relative harmony with their Reintegration of displaced people to live in relative harmony with their
neighboursneighbours, will contribute to security and economic recovery, will contribute to security and economic recovery Peace is secured by establishing stable/reliable institutions that guarantee Peace is secured by establishing stable/reliable institutions that guarantee
democracy, equity, justice, and fair allocation of resourcesdemocracy, equity, justice, and fair allocation of resources Promote peace by Promote peace by mobilisingmobilising grassroots groups to either oppose war or to grassroots groups to either oppose war or to
changechange public attitudes and build greater tolerance in societypublic attitudes and build greater tolerance in society Economic action (trade sanctions) can alter political commitment to peaceEconomic action (trade sanctions) can alter political commitment to peace
Development and governance projects treat conflict as an external factor & Development and governance projects treat conflict as an external factor & in the post-CFA period, adopted a in the post-CFA period, adopted a post-conflict mind-set post-conflict mind-set that saw them that saw them engage in reconstruction work under the assumption that improved socio- engage in reconstruction work under the assumption that improved socio- economic outcomes would support the transition to peace. economic outcomes would support the transition to peace.
From 2005, socio-economic development projects increasingly accepted the From 2005, socio-economic development projects increasingly accepted the need for need for conflict sensitivityconflict sensitivity and “do no harm” principles, and dropped the and “do no harm” principles, and dropped the notion of a “peace dividend” notion of a “peace dividend”
For peacebuilding work, there has been growing concentration on For peacebuilding work, there has been growing concentration on local local initiatives initiatives through development approaches rather than more directly such through development approaches rather than more directly such as on human rights and at the “Track 1” level. Some saw development as on human rights and at the “Track 1” level. Some saw development projects as a way to explore peacebuilding work in a politically sensitive projects as a way to explore peacebuilding work in a politically sensitive environment. environment.
Several projects focus on conflict transformation through inter-ethnic Several projects focus on conflict transformation through inter-ethnic initiatives and initiatives and community peacebuildingcommunity peacebuilding, but little evidence of how they , but little evidence of how they explicitly addressed the driving factors of the conflict. Very few tried to explicitly addressed the driving factors of the conflict. Very few tried to address the “Sinhala south”. address the “Sinhala south”.
Project StrategiesProject Strategies
Results ~ Quality of EvidenceResults ~ Quality of Evidence Many evaluations are premature and impacts are not given time to Many evaluations are premature and impacts are not given time to
emerge. They are more concerned with lessons for future than about emerge. They are more concerned with lessons for future than about impact impact
Many evaluations: Many evaluations: focus on focus on resultsresults rather than outcomes rather than outcomes are based on are based on partialpartial evidence evidence are beset by a are beset by a shifting contextshifting context where project designs are changed as where project designs are changed as
circumstances altercircumstances alter miss miss baselinesbaselines and follow-up surveys and follow-up surveys are affected by both are affected by both natural and political eventsnatural and political events that have disrupted the orderly that have disrupted the orderly
tracking of progress. tracking of progress. contain contain sensitivesensitive findings that limit sharing of findings and subsequent lesson findings that limit sharing of findings and subsequent lesson
learning.learning.
Despite this, important findings emerge around the effective delivery of Despite this, important findings emerge around the effective delivery of benefits especially at the grassroots level; and on how conflict affects benefits especially at the grassroots level; and on how conflict affects project performance. But the centralised nature of politics means local project performance. But the centralised nature of politics means local initiatives rarely have any impact on peace processes.initiatives rarely have any impact on peace processes.
ResultsResults Some peacebuilding evaluations are too Some peacebuilding evaluations are too conceptualconceptual. Some focus more on . Some focus more on
organisational aspects than on the impact of the initiatives. organisational aspects than on the impact of the initiatives. Some peacebuilding programmes have Some peacebuilding programmes have shifted focusshifted focus from conflict from conflict
transformation / co-existence to more classical development work, since transformation / co-existence to more classical development work, since overt peacebuilding activities are not acceptable (and also post-tsunami overt peacebuilding activities are not acceptable (and also post-tsunami needs have stimulated this).needs have stimulated this).
The dilemma of most peacebuilding / conflict transformation work generally The dilemma of most peacebuilding / conflict transformation work generally is the is the relevancerelevance of a peace project when injustice and inequality are not of a peace project when injustice and inequality are not addressed.addressed.
Findings on Findings on gendergender show mixed performance. show mixed performance. Governance and human rightsGovernance and human rights projects generally have been more projects generally have been more
successful at addressing individual and/or highly localized needs than at successful at addressing individual and/or highly localized needs than at promoting broader group-based or systemic changes.promoting broader group-based or systemic changes.
Community-based programmesCommunity-based programmes aimed at building “capacities for peace” aimed at building “capacities for peace” were more successful at community level than in making linkages were more successful at community level than in making linkages nationally. Some evidence that programmes on inter-ethnic issues created nationally. Some evidence that programmes on inter-ethnic issues created space for communities, especially those working with youth.space for communities, especially those working with youth.
Results at grassroots levelResults at grassroots level
Rich evidence baseRich evidence base Many DPs targeted grassroots groups for Many DPs targeted grassroots groups for
either development or PB purposes, with a either development or PB purposes, with a range of results, butrange of results, but Weak linkage to national processesWeak linkage to national processes Weak capacity to do conflict transformationWeak capacity to do conflict transformation Muddled theories of changeMuddled theories of change Small efforts individuallySmall efforts individually
ButBut Positive resultsPositive results
Local capacity built, community relations improved, void filled for civic Local capacity built, community relations improved, void filled for civic participationparticipation
Economic and social assets builtEconomic and social assets built Inter-ethnic trust builtInter-ethnic trust built Maybe collectively donor effort had impact on CPPB, but not yet Maybe collectively donor effort had impact on CPPB, but not yet
evaluatedevaluated
Nevertheless, under conditions where parties to the conflict see the Nevertheless, under conditions where parties to the conflict see the continuation of war as preferable to a negotiated political continuation of war as preferable to a negotiated political settlement….settlement….
explicit peacebuilding measures are not necessarily more effective explicit peacebuilding measures are not necessarily more effective in mitigating conflict than long-term socio-economic investmentsin mitigating conflict than long-term socio-economic investments
Conducting EvaluationsConducting Evaluations Evaluation work in Sri Lanka has Evaluation work in Sri Lanka has limitationslimitations even without conflict issues. even without conflict issues.
Donors do little independent evaluation, mainly using supervision missions, Donors do little independent evaluation, mainly using supervision missions, completion reports or in-house reviewscompletion reports or in-house reviews
Most TORs for Most TORs for socio-economic development evaluationssocio-economic development evaluations don’t call for conflict don’t call for conflict prevention and peacebuilding aspects to be addressed. prevention and peacebuilding aspects to be addressed.
Few evaluations do their own Few evaluations do their own conflict analysis conflict analysis or were able to draw on a or were able to draw on a baselinebaseline against which to gauge impact. against which to gauge impact.
Most evaluations were largely donor-managed exercises with Most evaluations were largely donor-managed exercises with limited limited consultation with Govt. consultation with Govt.
Few examples of Few examples of joint donor evaluationsjoint donor evaluations, and opportunities have been , and opportunities have been overlooked, even where joint-funding occurred.overlooked, even where joint-funding occurred.
A A shortage of consultantsshortage of consultants with the mix of evaluation + conflict skills, and with the mix of evaluation + conflict skills, and shortage of institutional guidance on conflict sensitive evaluationsshortage of institutional guidance on conflict sensitive evaluations
Project Project M&E systemsM&E systems can be biased or affected by conflict setting can be biased or affected by conflict setting Only few examples where there is an explicit use of Only few examples where there is an explicit use of Theories of ChangeTheories of Change The climate of mistrust in Sri Lanka means that information sharing is reduced The climate of mistrust in Sri Lanka means that information sharing is reduced
and the willingness to discuss results and engage in and the willingness to discuss results and engage in lesson learninglesson learning is limited. is limited.
Donor coordinationDonor coordination Coordination hasCoordination has declineddeclined from the relatively strong period around the from the relatively strong period around the
ceasefire. The level of coordination between donors and the GoSL has ceasefire. The level of coordination between donors and the GoSL has become increasingly difficult - and for some pointless.become increasingly difficult - and for some pointless.
For peacebuilding, the Donor Working Group \ Peace Support Group For peacebuilding, the Donor Working Group \ Peace Support Group reduced its scope but set up useful reduced its scope but set up useful sub-committees sub-committees
Mixed reaction: some donors liked the opportunity to pursue themes in sub-Mixed reaction: some donors liked the opportunity to pursue themes in sub-groups, others regard structure as over-elaborate and irrelevant. groups, others regard structure as over-elaborate and irrelevant.
Useful Useful analysisanalysis commissioned that led to better understanding commissioned that led to better understanding Weak Weak policy coherence policy coherence amongst members ~ except in some sub-groupsamongst members ~ except in some sub-groups Limited consideration of Limited consideration of gendergender, of , of viewsviews beyond Colombo, or views of beyond Colombo, or views of other other
partiesparties in conflict beyond the two main ones in conflict beyond the two main ones
As donors have come under increasing criticism, there is a need for As donors have come under increasing criticism, there is a need for stronger coordination, yet DPSG has become weaker. The Trust Fund was stronger coordination, yet DPSG has become weaker. The Trust Fund was not used productively. not used productively.
In donor strategies, coordination has modest importance In donor strategies, coordination has modest importance
RecommendationsRecommendations on Strategies on Strategies1.1. More rigorous use of More rigorous use of conflict and political-economy analysisconflict and political-economy analysis (preferably (preferably
joint) will inform strategic choices joint) will inform strategic choices
2.2. For strategic and programmatic reasons, be clear exactly For strategic and programmatic reasons, be clear exactly which aspects of which aspects of
CPPBCPPB are to be addressed and are to be addressed and what theories underpinwhat theories underpin how interventions how interventions
will make a difference will make a difference
3.3. Look for strategic ways to address the Look for strategic ways to address the root causesroot causes of conflict of conflict
4.4. Careful consideration is needed of what can and cannot be achieved by Careful consideration is needed of what can and cannot be achieved by
offering a ‘offering a ‘peace dividendpeace dividend’. ’.
5.5. More use of More use of scenarios / flexibilityscenarios / flexibility helps strategies to be responsive and to helps strategies to be responsive and to
manage risk manage risk
6.6. Recognise and declare Recognise and declare institutional capacity and comparative advantage to institutional capacity and comparative advantage to
work on CPPBwork on CPPB
7.7. Improve Improve indicators indicators to measure strategic outcomes on conflict, specify how to measure strategic outcomes on conflict, specify how
they will be they will be measuredmeasured and what and what resourcesresources available to collect the data. available to collect the data.
Recommendations on ProjectsRecommendations on Projects1.1. Use Use short-term programmesshort-term programmes on CPPB, provided they have focused, on CPPB, provided they have focused,
specific objectives and a strategy for withdrawal. specific objectives and a strategy for withdrawal.
2.2. Be Be flexibleflexible in choice of partners, in types of peacebuilding support, and in in choice of partners, in types of peacebuilding support, and in funding channel when working on peacebuilding in a volatile conflict funding channel when working on peacebuilding in a volatile conflict settingsetting
3.3. RethinkRethink your programme strategy in response to major shifts in the your programme strategy in response to major shifts in the political environment, don’t carry on as normal or shift a littlepolitical environment, don’t carry on as normal or shift a little
4.4. Better address Better address horizontal inequalitieshorizontal inequalities (between ethnic groups and (between ethnic groups and
geographic regions). geographic regions).
5.5. Build strategic co-ordination across Build strategic co-ordination across different levelsdifferent levels for any future peace for any future peace work (i.e. across Tracks and linking national and local initiatives). work (i.e. across Tracks and linking national and local initiatives).
6.6. Don’t assume that Don’t assume that civil societycivil society can be a major force in support of conflict can be a major force in support of conflict transformationtransformation
7.7. But better to deliver through But better to deliver through CBOsCBOs rather than NGOs in grassroots rather than NGOs in grassroots empowerment and conflict mitigation empowerment and conflict mitigation
8.8. Address Address gendergender aspects better in CPPB work, especially at grassroots aspects better in CPPB work, especially at grassroots
Recommendations on M&ERecommendations on M&E
Require or do a Require or do a conflict analysisconflict analysis
Develop more explicit Develop more explicit theories of changetheories of change
Find good Find good indicatorsindicators at outcome level at outcome level
Use more Use more jointjoint evaluations evaluations
Focus more on Focus more on impactimpact and be prepared to wait and be prepared to wait
Use consultant teams with Use consultant teams with mixed backgrounds mixed backgrounds
Plan in Plan in advanceadvance and be and be flexibleflexible in timing in timing
Allow Allow additional timeadditional time for preparation and expect delays for preparation and expect delays
Recommendations on Recommendations on CoordinationCoordination Address Address leadership gapleadership gap Use the Use the Trust FundTrust Fund more effectively ~ coordinated more effectively ~ coordinated
action and sharing of responsibilities helps donors action and sharing of responsibilities helps donors reach beyond their limitsreach beyond their limits
Do more Do more joint workjoint work for greater buy-in (for example on for greater buy-in (for example on how partners have provided support to NGOs). how partners have provided support to NGOs).
Newer and larger donorsNewer and larger donors must engage more fully so must engage more fully so that coordinated approaches have a real impact on that coordinated approaches have a real impact on the ground. This will require finding areas of mutual the ground. This will require finding areas of mutual interest around do no harm principles, and may interest around do no harm principles, and may preclude wider discussion on more sensitive issues.preclude wider discussion on more sensitive issues.