Accountability ResearchMyanmar case study

Discussion on interim findings with Save the Children in Myanmar

Andy Featherstone, 19 March 2013

Accountability research – why?The purpose of the research is to collect evidence of the

impact of accountability mechanisms on programme qualityThere is an assumption that the introduction of accountability

mechanisms leads to more effective projects but little evidence exists (Ref. synthesis paper developed from evidence submitted by the HAP peer learning group)

We understand effective projects to be those that are relevant, effective, efficient and sustainable in line with the DAC Criteria.

Research question - In what ways does an effective accountability mechanism which provides information to affected communities, facilitates their participation in programme design and implementation and offers a means for communities to feedback and/or complain contribute to the quality of a humanitarian or development programme.

Accountability research – how?

The approachA methodology was developed which mixed

quantitative and qualitative participatory tools and was used consistently throughout the research

Scorecards and opinion ranking exercises were used to describe accountability mechanisms and as entry points to wider discussions about their contribution to project quality

All of the exercises were translated into the Myanmar language and the discussions were facilitated

Each community meeting lasted between 1.5 – 2.5 hours

ItineraryDate Activity

Day 1 Travel from Yangon to Mandalay Travel from Mandalay to Meiktila Orientation and meeting with SCI staff

Day 2 Field Visit to Lat Pan Khar Kough (Meiktila) 3 x group consultations

Day 3 Field Visit to Yae Cho (Meiktila) 3 x group coonsultations

Day 4 Field Visit to Nat Gyi Kone (Meiktila) 3 x group consultations

Day 5 Travel from Meiktila to Kyaukpadaung Field Visit to Zay Kone (Kyaukpadaung) 3 x group consultations

Day 6 Field Visit to Yone (Kyaukpadaung) 2 x group consultation

Day 7 Return to Yangon

Programme profileNon-formal education, child

protection and early childhood care and development programmes

Within the breadth of the programme, a participatory approach exists for CBO problem identification, targeting and selection

Formal complaints response mechanism (CRM) in 3 villages; additional informal mechanisms across all villages

SCI programme for between 2- and 7-years

Participation

Discussions were held in mixed groups of 10-25 people. 3 meetings were held in each village each day (women, men, children). Results of the methods were disaggregated by gender. There was a bias towards the inclusion of children’s group & CBO members. Non-members were omitted from most of the meetings

Total men = 63

Total women = 75

Total boys = 44

Total girls = 47

Total participants = 229

Lessons about the methodologyThe methodology worked well with all groups (esp.

children) and engagement was good across each of the communities

Strong facilitation (as opposed to translation alone) was key to getting good results

The methods used produced quantitative data but qualitative follow-up discussions were of greatest value to the research

The children’s meetings went extremely well and often revealed a level of understanding about the research topic that was equal to if not greater than the adults

Findings – Accountability IAccountability to project participants was

routinely considered strong across all villages (including counterfactuals)

Informal mechanisms (particularly face-to-face) often favoured over formal CRM mechanism for complaints.

“We prefer face-to-face meetings as this avoids misunderstandings and when we can’t use this method we will use the phone”

Feedback from Save the Children generally takes between 1-2 days and 1-2 weeks depending on the mechanism used

FUNCTIONING OF THE ACCOUNTABILITY MECHANISM

Findings – Accountability IIParticipation felt to be strong particularly

amongst active CBO members and children’s groups

The rich (‘not interested’) and extremely poor (‘no time’) were considered to be those who had least access to/involvement in information, participation and feedback. Those who couldn’t attend village meetings (elderly, disabled and ‘outliers’) also tended to miss out

“Those who have the most insecure livelihoods cannot participate in the programme as they do not have time to attend the meetings and get involved”

FUNCTIONING OF THE ACCOUNTABILITY MECHANISM

Accountability in Action

RelevanceRelevance: Fairly consistent feedback about

participation strengthening the relevance of the project (targeting and project selection)

“If we weren’t involved [in the implementation] the project could not be successful as it wouldn’t meet children’s needs”

“This is ‘our’ project because we can decide what our priorities are. SC discusses these with us and we decide”

“We can request what we need as we have a good relationship and are an equal partner. This makes the project more relevant to our needs”

“If children cannot point out mistakes or activities that aren’t relevant then interest would be low as it would feel wrong and children wouldn’t want to waste their time”

CONTRIBUTION OF ACCOUNTABILITY TO PROJECT QUALITY

Effectiveness & SustainabilityEffectiveness & sustainability: Provision

of information promotes knowledge of the programme and participation in it; participation and ability to feedback ensure that community voice is heard and promotes ownership and trust

“The decision-making process is important as it brings ownership of the people which will continue beyond the project lifespan”

“People were initially nervous of sending their children to the training…but we were reassured by the information we received from Save the Children which helped us to trust them”

CONTRIBUTION OF ACCOUNTABILITY TO PROJECT QUALITY

EfficiencyEfficiency: Much less evidence of

contribution than for other DAC criteria. The best example came from the Save the Children area office;

“We formed a training task force after the village complained that they were overloaded as they had received 3 different trainings on the same subject. This led to the creation of a single training programme. This has saved time and increased efficiency”

CONTRIBUTION OF ACCOUNTABILITY TO PROJECT QUALITY

Impact Impact: Some good examples that go beyond

project-based outcomes “[As a community] we used to make decisions in a top-down

way; participation was very new but it has been welcome. We did not know how to decide together so this is a lesson which we have learnt together. We have found that it reduces conflict.”

The village has a children’s health system which we contribute to in case of emergency but we often make poor decisions. After seeing how the SC programme worked we decided to have a formal decision-making process so there was transparency in how we made our decisions. This is a lesson we learnt from Save the Children (paraphrased)

“In this monestary there is an annual ceremony where money is given to the monks. Previously we didn’t know how this was used but starting in 2009 we agreed to keep account of the money and keep a record of the balance [in the same way as for SCI projects]”

CONTRIBUTION OF ACCOUNTABILITY TO PROJECT QUALITY

Issues for the researchDespite the lack of a formal CRM no significant

difference was found in people’s perceptions of SCI’s accountability to them – largely as a result of the strong relationship which exists (face-to-face contact was preferred)

Implications for the use of the HAP benchmark for longer-term programming – a mature relationship based on strong participation may be more important than access to formal feedback mechanisms

Both the case studies (Kenya & Myanmar) were selected in part because of the strong accountability to project participants. This has yielded good results

The counterfactuals have worked less well. There would be value in commissioning a case study where participation is weak and/or where the accountability mechanisms are poor (humanitarian case study?)

Questions & discussion