the most significant change as a part of a project evaluation in india
DESCRIPTION
Presentation from the conference of the Czech Evaluation Society on 28 May 2013.TRANSCRIPT
The Most Significant Change ProLearn Project in India
Inka Píbilová Czech Evaluation Society Conference in Prague, 28 May 2013
A „tsunami relief“ project
2004 first tsunami relief in Andra Pradesh India funded by individual donors of ADRA Czech Republic
2007 needs assessment with Education Office among 20 schools
2008 – 2009 infrastructure development in 8 schools to increase and retain the number of children
– Classrooms construction
– Teacher training
– Learning aids
– Community sensitisation
– Children participation
Evaluation Objectives • Show evidence of the project impact on access to
education, increased education quality and child-friendly learning environment, thus achieving learning outcomes and higher literacy.
• Show evidence of attitudinal changes of community, school staff, children and government representatives in education.
• Assess sustainability of the “Pro Learn” project after ADRA withdrew support.
• Draw lessons learnt and recommendations for improvements in future projects with respect to planning, implementation, monitoring and evaluation.
Evaluation Methodology - Factors
1. Logical matrix did not correspond to reality
2. Audit of the construction done, client interested in impacts and sustainability.
3. Donor did not have experience with evaluations and did not have a specific method in mind.
4. Lack of funding to conduct the evaluation.
5. Lack of baseline data.
6. Field mission restricted to 3 weeks
Term
s o
f R
efe
rence –
Obje
ctives,
scope
, sta
kehold
ers
, questions,
bu
dg
et,
schedule
, o
utp
uts
, use.
Desk study
Interviews
The Most
Significant
Change
Community
conversations
Observations
Pre
limin
ary
fin
din
gs &
conclu
sio
ns
Final
debriefing
of project team
and
community
volunteers
Communication with the Project Partners
Draft evaluation
report Fin
al evalu
ation r
eport
Inception phase
1 month
Field research
1 months
Reporting phase
2 months
Initia
l briefing a
nd inception
Evaluator
Selection
Evaluation Methodology
http://www.mande.co.uk/docs/MSCGuide.pdf
http://www.mande.co.uk/docs/MSCGuide.pdf
http://www.mande.co.uk/docs/MSCGuide.pdf
How was the technique applied
MSC Training of community volunteers
Collection of the MSC
stories by volunteers
Selection of the MSC with
community volunteers and project team
MSC drawings by children
How we selected the „MSC story“
What were the findings
• Renovated or constructed school buildings and games for children were appreciated, but...
What were the findings
The Most Significant Change was the encreased empowerment of children in established Child Clubs, who
were able to secure access to potable water or clean toilets from the local authorities within 3-6 months.
What were the findings
Communication of findings • Fiel mission closed after evaluation finished - key
communication had to be done at the end of the evaluation.
• The local community was to a big extent illiterate - community volunteers were expected to debrief the respective communities.
• Headquarters in India unavailable for debriefing – done by Skype, donor organisation debriefed back in Europe.
• Donor organisation decided to delay the communication and finally did not published the evaluation report.
• Though a short debriefing was held with District Education Office, despite the original plan findings were not officially communicated to school management, teachers and students.
• Stories had already been used in local media.
Lessons Learnt • The method can be used for evaluating impacts of projects
related to behavioral changes as an additional data collection method, even in case of lack of resources.
• Secure and train reliable volunteers / interviewers.
• The atmosphere of trust and openess is key (interpreting)
• Verify the stories in the field.
• Ensure availability of key stakeholders in the selection of the Most Significant Change story.
• The stories help in communicating successes of the project, therefore communication incl. budget needs to be carefully planned (to be a part of the evaluation mission or a subsequent project atcivity)
Links Detailed information on the method, including studies, evaluation reports etc. :
• http://mande.co.uk/special-issues/most-significant-change-msc/
• http://groups.yahoo.com/group/MostSignificantChanges
Guide:
• http://www.mande.co.uk/docs/MSCGuide.pdf
Trainings currently available in Europe:
• www.socialimpactconsulting.com.au
• www.traceydelaney.net/
