impact assessment eda university of rome

8/6/2019 Impact Assessment EDA University of Rome

1/19

1

EMMANUEL DEVELOPMENT ASSOCIATION

IMPACT ASSESSMENT REPORT

THE IMPACT OF EDA PROGRAMMING ON THE LIVES OF WOMEN IN

AKAKI-KALITY, ETHIOPIA

Universit degli studi Roma Tre (Rome, Italy)October 2010


2/19

2

TABLE OF CONTENTS

1. EMMANUEL DEVELOPMENT ASSOCIATION BACKGROUND 3

2. AKAKI-KALITY PROGRAMME 4

3. ALTERNATIVE BASIC EDUCATION 5

4. MONITORING AND EVALUATION 6

5. EMMANUEL DEVELOPMENT ASSOCIATION SUCCESSES 7

6. RESEARCH METHOD 8

7. IMPACT ASSESSMENT RESULTS 10

8. A NOTE ON FOOD SECURITY 16

9. OPPORTUNITIES 18

10. EDA SUCCESS STORY: CHILD SPONSORSHIP 19


3/19


4/19

4

2. Akaki-Kality ProgrammeAkaki-Kality is a sub-city of the bustling urban Ethiopian capital, Addis Ababa.

It is located approximately 25kms southwest of Addis in the industrial outskirts of thecapital, where approximately 60% of all of the industries of Addis are found. Akaki-

Kality covers an area of approximately 6,143.4 hectares, and has a population ofapproximately 220,740 (approximately 114,095 female and 106,645 male).5 Poorinfrastructures such as schools, health institutions along with a high prevalence ofpoverty-induced socio-economic problems such as HIV/AIDS, prostitution, begging arecommon throughout Akaki-Kalitys 8 kebeles (neighbourhoods), and Akaki-Kality ischaracterized by both deeply impoverished urban slums, as well as extensive ruralfarming areas. Further, Akaki-Kality has an HIV prevalence rate of 9.1%, compared tothe approximate national prevalence rate of 3.5%, which estimates suggest range from0.7% in rural areas to 5.5% in urban centres. As a result of this, the ratio of HIV/AIDSorphans is also higher in Akaki-Kality than in other communities throughout Ethiopia,and these children represent a population particularly vulnerable to extreme poverty, food

insecurity, and a lack of a social safety net.

It was therefore within this context that EDA began working in Akaki-Kalityfourteen years ago. The Akaki-Kality programs have expanded to include programs and projects in Alternative Basic Education, improving Formal (government) Education,Youth Training and Livelihoods, Community Health and HIV/AIDS, Early ChildhoodDevelopment (Child Sponsorship), Water Sanitation and Hygiene (WASH) initiatives,Womens Empowerment and Livelihoods, Capacity-building of Community-BasedOrganizations, Urban Agriculture (Food Security), and Smallholders SustainableDevelopment. All programs maintain the target groups of vulnerable women, youth and

children, and function within the framework of promoting self-initiated communitydevelopment initiatives and enabling self-sufficiency in order to achieve sustainability indevelopment programs.

5 FSCE Ethiopia 2006.


5/19

5

3. Alternative Basic EducationThe Alternative Basic Education (ABE) program was EDAs first

major project undertaken, and Akaki-Kality was the first project location.

In line with the second United Nations Millennium Development Goal,to achieve Education for All by 2015, the ABE program focuses onproviding quality basic primary school education for marginalized and outof school children. The class schedules of ABE schools are explicitlydesigned to work in harmony with the community in regards to schoollocations and class scheduling, particularly allowing for flexibleschedules to meet the needs of children who must participate in theincome-generating or asset-guarding [such as watching the goats andsheep] activities of their families. The ABE school system acknowledgesthe reality that many of these children must work during normal schoolhours, and thus attempts to embrace and work within this reality by

offering flexible scheduling, cost-free

6

.

The pedagogical methodology of ABE schools is participatory, andstudents are encouraged to ask questions and to take ownership of theirlearning. This methodology is especially empowering for Ethiopian girls,who might otherwise not be encouraged to develop their own sense ofself, voice and confidence. As Kalkidan Abu, a 10-year-old girlattending ABE school in Akaki-Kality, states, now I feel more

confident, at the ABE schools I am less scared to respond in class.7

It is this qualitative sense of self and confidence that will have a long-

lasting impact on the community of Akaki-Kality, as more girls are ableto participate in schooling and gain confidence in themselves. Theflexible scheduling and participatory methodology of ABE schoolsprepares students to be graduated after three years of ABE schooling intothe fifth year of the public schooling system.

Over the fourteen years of its existence and programming in this area,EDA has supported more than 33,000 students (60% girl-students)through the ABE program and graduated them on to secondary school.8Currently, EDA has 11 ABE schools and 40 facilitators. 8 of theseschools are located in Akaki-Kality.

6 Although all public schools in Ethiopia are tuition free, the cost of purchasing notebooks, textbooks,writing utensils and uniforms, let alone the loss of the labour-hours children in poor families provide

towards their families livelihoods, makes attending school too costly and therefore unaccessible for too

many children throughout Ethiopia.7 EDA, (2010),Emmanuel Development Association: Success Stories, p. 58 EDA, (2009),Annual Report.


6/19

6

4. Monitoring and EvaluationEDA believes that development is a process, which begins with identifying a community problem and then doing something about it. Monitoring and Evaluation is one of themanagement tools that can help to improve this process. M&E focuses on: process of

EDAs operation; performance of those involved in the development process;progress/results achieved due to EDAs program/projects; resources used; overall impactof EDAs program on the lives of children and poor communities.

On top of the above, qualitative social aspects including degree of communityparticipation, question of equity and distribution of benefits, and gender differentials andchanging attitudes of communities will be monitored during the strategic planning period.

It is within this process of evaluation of the qualitative social aspects and impacts of

EDA programs that this Participatory Impact Assessment is based. The goal of thisreport is to understand how EDA programmes have IMPACTED the lives and well-

being of participants (and particularly, the target groups of vulnerable women and

children)

EDA staff at head office and project level in cooperation with communities and otherstakeholders will continuously engage in monitoring of the progress of programinterventions, and further oversee periodic review of each activity, at every level ofimplementation.


7/19

7

5. Emmanuel Development Association Successes:Over the fourteen years of its existence as an organization, EDA has contributed tonumerous tangible impacts. Some of these include:

Direct support for 80,000 people and indirect support for 400,000 40 first cycle primary schools currently operating Over 33,000 marginalized children have graduated primary school and joined

secondary schools (60% girls) (*from attendance records)

1 medium health clinic serving over 25,000 people (2008 Report) 8 junior secondary schools constructed to build the capacity of formal schools 2,387 youth trained through Business Skills workshops 8,250 orphans and vulnerable children (OVC) supported in health, education and

nutrition Multipurpose youth training centre established, operational and is a learning

centre and demonstration centre for 8,923 community members 877 smallholder farmers supported with agricultural imputs Constructed one Biogas plant for sanitation improvement and alternative energy

sources in the community (Akaki-Kality)

Organizational Awards:

World Bank Award for local initiative (2003) UNESCO Award for MDG in Quality Education best practice (2005) USAID Certificate of Achievement (2005)

Millennium Award from Regional Governments in Ethiopia (2007/08)o Tarmaber woreda administration (region): Among many NGOs working

in the district, EDA is nominated for the award not only by goodwill butfor its best performance and tangible achievements, we thank EDA againand again.

o This Impact Assessment will examine and highlight not only the tangibleachievements of EDA in Akaki-Kality, so widely recognized by theinternational donor community and the National and Regionalgovernments, but also investigate some of the less tangible but equallyvaluable impacts such as the increasing wellbeing, confidence andcapabilities of individuals particularly women and children involved in

EDA programming. National Award for Addressing Vulnerable Communities (OVCS) National Award for Best Practice from H.E. President Girma, FDRE (March

2010)


8/19

8

6. Research MethodThe main component of this impact assessment was a qualitative evaluation. Thequalitative component of the methodology was designed to employ a combination of

participatory tools including focus group discussions which sought depictions of typesEDA programming, level of involvement of individuals and households, and the effectsparticipation in programmes had on the participants conception of their own well-beingin terms of their changed levels of functioning and the (potentially) resulting expansionof their self-identified capabilities.

The process of this impact assessment was designed within the overall framework ofAlkires 2002 Valuing Freedoms impact assessment, with adapted tools from standardimpact assessments as suggested by Roche (Oxfam) and Bird (ODI), and, during thefocus group discussions and workshops, using some participatory approaches and tools asdefined by SEAGA (Socio-Economic And Gender Analysis tools from FAO). In an

attempt to understand the impacts of EDA programming on the individual well-being ofwomen and children and their own self-identified changes in functioning and capabilities,it seemed necessary to gain individualparticipant responses, information and views, asopposed to simply proxy indicators of increased household incomes, asset andeducational levels.

We hope that this small initial and exploratory study will provide a baseline of qualitativeimpacts for future programming and evaluation tools, and will demonstrate that EDA hashad a significant impact on the well-being of beneficiaries participating in their

programming. The voices of women and children were chosen to be the subjects of thisproject, as marginalised women and children are the target groups of EDA programming.

Ethical Considerations

To ensure accessibility to all participants in EDA programmes, we decided to pursuemultiple avenues of data collection, including focus group discussions (short, concise,over the lunch time period), semi-structured interviews questionnaires (also short,administered during focus group discussions or at the school or workplace), as well asholding two workshops: one, for women on the overall theme of leadership andcapabilities and one for children and youth on child rights and capabilities. as well asensuring that all participants were explained the purpose of the survey (to understand theimpacts of EDA programming in order to improve and expand future programming), andexplicitly stating that there was no obligation to participate, nor was there any threat tothe individuals future participation in any EDA program through any of their anonymousresponses.


9/19

9

Summary of Sample Subjects and Techniques

No. Subjects / Samples SampleSize

Sampling

Techniques

Methods of Data

Collection

Quantitative Data Sources1. Youth Participants /

Beneficiaries19 Random Sampling Semi-structured

InterviewQuestionnaire

2. Adult Participants /Beneficiaries

40 Random Sampling Semi-StructuredInterviewQuestionnaire

Qualitative Data Sources

3. Key Informant Interview(Govt)

1 Purposive sampling(one interview)

Semi-structuredInterview

4. Key Informant Interview

(ED)

1 Purposive sampling

(one interview)

Semi-Structured

Interview5. Youth Participants /

Beneficiaries1(group)

Focus GroupDiscussion

6. Adult Participants /Beneficiaries

1(group)

Focus GroupDiscussion

7. Secondary Sources Review Review of EDADocuments; otherNGOs; UNorganizations


10/19

10

7. Impact Assessment Results:Figure 1: Respondent Ages (Adult)

The 40 respondents ages ranged from 27 to 70, with an average age (mean) of 39.9 yearsold. Although basic, this sort of demographic information helps to contextualise how therespondents might value the impacts of various qualitative aspects of their lives, and alsohow they might define or assess their change in well-being. Further, this informationhelps EDA to determine which variety of participatory activities, projects or programmesthey might pursue, according to the need in the community, as there are changing needsand values at different stages of life.


11/19

11

Figure 2: Respondent Level of Education

This graph demonstrates the respondents current level of education. The number

assigned to each level of education is valued as follows:

1 = No Formal Education2 = Some Primary School3 = Primary School Completed4 = Some Secondary School9 = Other

This education graph demonstrates that the majority of respondents in this project arerepresentative of one of the main target groups (marginalised women) of EDA. This

information is useful in determining the type of future programmes offered by EDA, andalso as a potential baseline for more standard impact assessment evaluations on tangiblechanging levels of education. After describing the qualitative aspects impact graphs, wewill examine a comparison between education and overall qualitative impact which is amuch more illustrative comparison.


12/19

12

Qualitative Aspect Impact Rankings

Development projects have not only benefits as seen through cost-benefit analyses, butalso a number of non-quantifiable effects, including the acquisition of usefulknowledge, the cultivation of friendships amongst each other etc. Sabina Alkire

further suggests that the ethical reason for studying these components is central: just ashealth and knowledge may be valued as ends as well as means to increasing productivity,so the abilities to enjoy other functionings (pertaining perhaps to participation, religion,art, human relationships) can be considered intrinsically valuable whether or not they arealso instrumentally valuable.9 Alkires study in India was the direct inspiration for thisproject in Ethiopia; in particular, her methodology, which included a ranking evaluationon specific aspects of an individuals life was the most dominant inspiration of this shortimpact assessment. It is the methodology of this research, which this dissertation reportnow turns.

The following information was gathered during the semi-structured interview when each

participant was asked to give a ranking of 1 (no impact) to 4 (large impact) of theimpact of participating in a particular EDA programme on the following qualitativeaspects of their lives:

Overall Life

Personal Health

Knowledge

Friendships

Family

Role in the Community

Happiness

Confidence

Although the question was supposed to elicit a simple single number answer, during the process of the interview each respondent started speaking in detail about his or herexperienced change in functioning of each aspect. It very quickly became evident that itwas an important as part of the participatory process (potentially contributing toexpanding capabilities even through the impact assessment process itself) that weacknowledge, listen, and record each of the participants comments. During the dataanalysis, a value of 5 was given to any aspect in which a respondent had valued 4 andhad spoken in great length or detail about (usually only 1 or 2 aspects per respondent).This resulted in a maximum impact level of 40 (8 aspects, each with a possible maximum

impact value of 5). The following chart summarizes each respondents maximum impactlevel.

9 Alkire, 2002, p. 204.


13/19

13

Figure 3: Qualitative Aspect Rankings

Figure 4: Respondent Total Impact Value


14/19

14

As is evident on the chart above, the total impact per respondent ranges from a level of 14to a level of 35 (out of 40). What is particularly interesting in this context is thecomparison between the average (mean) total impact per respondent, and their personallevel of education.

The following pivot table demonstrates this connection:

Table 1: Respondent Level of Education and (Avg.) Total Impact

Average of Total:

Level of Education Total

1 30

2 28

3 28

4 25

9 22

Grand Total 27

What we observe on the table above is that there is a correlation between a lower

level of education and a highertotal impact value.

Stated another way, this table directly suggests that participants who have a lower levelof education, ranked their participation in an EDA programme as having a higher impacton the various qualitative aspects of their lives than participants who had a higher level ofeducation.

This information suggests that EDA has more impact on the lives of

those who are their direct target group: marginalised and vulnerable

(women).

Further, this information provides a key benchmark in terms of an ex-post, improvingimpact assessment, as it suggests that what we can learn from this sample, is that

EDA is doing excellent work targeting their programmes and

should continue in this direction.


15/19

15

Moving away from a respondent-centric analysis of impacts towards a qualitative aspectanalysis, the following graph shows each aspectwith its corresponding total value (fromall respondents):

Figure 5: Qualitative Aspect Total Impacts

What we observe here is that the most impacted aspect of the respondents lives were their Role in the Community (with 152 points), and their Friendships (with 151 points).These total values are further corroborated by the mode of each aspect, in which Role inthe Community received an average ranking of 5, andFriendships an average ranking of4.

Further, the comments that many of the women made during the semi-structuredinterview process were particularly illustrative. As mentioned above, almost all of the

respondents (93% women) wanted to discuss howthese various qualitative aspects oftheir lives had been affected by participation in EDA programs (such as Micro-

credit group, Livelihoods support, Child sponsorship, Alternative Basic Education

etc.), rather than simply assign each aspect a number value.


16/19

16

8. A Note on Food SecurityFood Security exists when all people at all times have physical andeconomic [and social] access to sufficient, safe and nutritious food to

meet their dietary needs and food preferences for an active and healthylife. (World Food Summit, Rome, 1996).

The four main dimensions of food security are:

Physical Availability of food: Food availability addresses the supply side offood security and is determined by the level of food production, stock levels andnet trade.

Economic and physical Access to food: An adequate supply of food at thenational or international level does not in itself guarantee household level food

security. Concerns about insufficient food access have resulted in a greater policyfocus on incomes, expenditure, markets and prices in achieving food securityobjectives.

Food Utilization: Utilization is commonly understood as the way the body makesthe most of various nutrients in the food. Sufficient energy and nutrient intake byindividuals is the result of good care and feeding practices, food preparation,diversity of the diet and intra-household distribution of food. Combined withgood biological utilization of food consumed, this determines the nutritionalstatus of individuals.

Stability of the other three dimensions over time: Even if your food intake isadequate today, you are still considered to be food insecure if you haveinadequate access to food on a periodic basis, risking a deterioration of yournutritional status. Adverse weather conditions, political instability, or economicfactors (unemployment, rising food prices) may have an impact on your foodsecurity status.

Source: FAO, Food Security Information for Action, Practical Guides

To put into socio-economic context, the average amount earned each month (in thewomens households) was approximately 200 399 birr per month (or 10 to 20 Euros, bycurrent exchange rates). The average amount spent on food each month was also

between 200 and 399 birr (10 to 20 Euros) per month. In terms of the womens reactionsto the question, most of the women (32 out of 40 respondents) stated something to theequivalent that however much we earn [in a given month] our household spends onfood.10 The more a household earned in a given month, they more they spent on food.Further, through our discussions with the participants in EDA programs (mostly ruralwomen who earned their livings through either selling injera and participating in a co-


17/19

17

operative that EDA started), food issues, and being able to provide enough nutritiousfood for their families, was a constant concern, and participating in EDA programmingon womens empowerment projects had a very large impact on the quality of their lives.In this regard, EDA programming is contributing to the access dimension of foodsecurity.


18/19

18

9. OpportunitiesBased on the findings and analysis of this initial study of a capability approach to impactassessment, there are many potentialities for further research and opportunities in this

area. Some of the main opportunities are:

Identifying a minimally accepted Impact Value (both individually as well as peraspect) in order to pursue future capability approach impact assessments and / ormonitoring and evaluation of EDA programming.

Including space for participant definitions of well-being (and how that well-beingis impacted by any particular programming) during the process of an impactassessment

Pursuing further research on the connection between Level of Education andIndividual Total Impact

Triangulating more research information by including information on thecounterfactual and / or an outside control group (one within a similar socio-economic reality as the organization participants but notorganization participants)

EDA should continue their successes with targeting ofmarginalised and vulnerable women and children, as both their

qualitative and quantitative impacts were significantly higher


19/19

19

10.EDA Child Sponsorship Program Success Story:Lohme Medeksa (Mother)Quiet and reserved yet beaming with life, Lohme Medeksa is an inspiration. Living with

two of her four children in a one room thatched roof hut, she proudly shows her twoprized possessions: a cow and its calf both animals also living with Lohme and her twodaughters inside their hut.

Her youngest child, Abiti Medeksa, 12 years old, is part of the Child Sponsorship program with Emmanuel Development Association. As a participant in this programAbiti receives educational materials, clothes, hygiene items and school fees covered.Through Child Sponsorship, Abitis family also has access to health care and medicinesthrough the local EDA clinic. Perhaps the biggest change in the lives of Abiti and Lohmeand their family, however, has been the cow Lohme received through a lottery of ChildSponsorship participants.

Before receiving the cow, Lohme was a daily labourer, and she describes life as beingvery difficult. Her daily wages were barely enough to feed her children. After herhusband passed, two of Lohmes children went to live with other relatives because shecould no longer afford to support them. After receiving the cow through a ChildSponsorship lottery programme, Lohme says her life changed: She used the money shemade from selling the milk not for building a bigger house or purchasing householdgoods, but instead, saved the money to ensure the health of her cow in order for her cowto produce a health calf.

Now, Lohme hopes to transfer a legacy for her children. She no longer worries about her

income but rather about the future, and the future for her children. Her goal is tocontinue growing her calf and save enough income from selling milk to have all of herchildren living together with her again. Lohmes far-sighted determination to keep hercow and not sell it or its calf will definitely help her reach her goal.

Meeting Lohme Medeksa and listening to her story was an incredibly humbling andinspiring experience. Her wisdom, inner strength and capacity to reach her goals are allshining, and she praises God, EDA and CCFC for having a role in this incredible changein her life.