impact assessment report - danish refugee … assessment report nusaf2 - northern uganda social...
TRANSCRIPT
IMPACT ASSESSMENT REPORT NUSAF2 - Northern Uganda Social Action Fund
2012-2013 Project in Moroto Municipality and Nadunget Sub-County
Karamoja, Uganda
2
Summary of main findings
There is a reduction from 40% to 27% of households that live on one meal pr. day
74.5% of households have not experienced starvation in the past 12 months, while 7.4% have
Staple food and green vegetables are the main types of food consumed in households
47.3% of respondents rate their income situation as ‘good’ or ‘very good’ at the time of impact
assessment
41.4% says that their ability to meet basic needs has improved in the past 12 months
30.1% of beneficiaries allocate less than half of their monthly expenditures on food
The proportion of households that make savings has increased from 41% to 51%
50% of households would be able to cope with an economic shock
64.1% find that their access to a market has been highly improved
76.9% of beneficiaries are satisfied with the available choices between types of Household
Income Support Projects
Both NUSAF2 beneficiaries and community members are positive about the quality of the public
works projects carried out
93.9% of beneficiaries express that they were not given a clear idea about what to expect from
the project before the activities started
62.5% of beneficiaries says that DRC-DDG has lived up to commitments to a high extent or
completely
3
Table of contents
Executive summary
1. Introduction
a. DRC-DDG Program in Uganda
b. Description of project and working context
2. Methodology
a. Data collection methods
b. Sampling and characteristics of respondents
c. Data handling and analysis
d. MitHISPtion of methodological challenges
3. Findings
a. Indicator I: Food Security
b. Indicator II: Income and economic resilience
c. Indictator III: Market access
4. Quality assessment: Project relevance and accountability in delivery
5. Conclusion
4
1. Introduction
Description of project and work context
DRC-DDG has been an implementing partner to the WFP-funded Northern Uganda Social Action Fund 2
program in Karamoja from March 2012. DDG has been responsible for Public Works projects and
Household Income Support Projects (HISP) in Moroto and Nadunget, and has administered the
distribution of cash and food for work.
The overall objective for DRC-DDG’s work has been: To support livelihoods of poverty-stricken
households in Nadunget Subcounty and Moroto Municipality.
DRC-DDG has sought to contribute to this overall objective, by pursuing the immediate objective:
Improved access to income earning opportunities and basic social services of food insecure households
in Nadunget Subcounty and Moroto Municipality
The project has targeted a total of 6,492 households with an estimated 38,900 direct and indirect
individual beneficiaries.1 The beneficiaries have been organized into 135 clustered, each carrying out one
Public Works project, and one HISP. In total, the following activities have been carried out under NUSAF
2 in the period March 2012 to March 2013.
135 Public works projects:
Community Access Road 48
Cattle Crush 7
Tree Planting 80
135 Household Income Support Projects:
Vegetable garden 45
Grinding mill 16
Cereal banking 4
Field crop cultivation 30
Bee keeping 10
Mushroom growing 8
Orchards 5
Tree nursery 10
Groundnuts grinding mill 6
Poultry 1
In addition, beneficiaries have been encouraged to establish Village Savings and Loans Associations
(VSLA), and the clusters who took interest have been given training in VSLA and saving-boxes with
record books.
1 A direct beneficiary is defined as a household member who actively takes part in public works and HISP. An indirect
beneficiary is a household member of a direct beneficiary. Number of indirect beneficiaries is estimated based on an average household size of 6 persons.
5
2. Methodology
Data collection methods This impact assessment aims to explore impact of the NUSAF2 implementation phase that started in
March 2012. It is based on a mixed methodology comprising a quantitative household survey and
qualitative focus group discussions. The household survey has been carried out in August 2013, six
months after the completion of the implementation phase, and the qualitative data in October 2013 with
the purpose of leaving enough time for project outcomes and impact to materialize. A second phase has
continued after March 2013, whereby this assessment also serves as a mid-term assessment for the two-
year program phase. Data has been collected by a team of external enumerators under the management
of DRC-DDGs M&E Unit. Enumerators have undergone a one-day training prior to data collection, and
the tools have been pilot-tested prior to usage. All data collection tools have been translated into
Ngakarimojong, and respondents have been interviewed in their mother tongue.
Sampling and characteristics of respondents
The household survey is based on responses from 599 randomly sampled NUSAF2 beneficiaries in all
locations and clusters of the project. Sampling has been made proportional to number of cluster members
to ensure that all clusters have relatively equal representation in the sample, and gender and age has
been made to reflect the distribution among the beneficiary population. In consequence, 62.5% of
respondents are female and 37.5% are male, while the age distribution is as follows:
Mitigation of methodological challenges
Prior to data collection in 2013, the data collection tools have been revised and specified with a further
level of detail than the baseline data collection in 2012. The end-line data thus introduces additional
measurements that were not included in the baseline, and to enable tracking of change in the
implementation period, such new questions have been accompanied by a question assessing
respondent’s perception of change in comparison with one year earlier. Therefore, this impact
assessment comprise a combination of direct comparisons between respondents answers to questions in
2012 and 2013 respectively, and perception-based assessments of change. This will be clearly indicated
with a label of when data was collected (2012 or 2013) in all tables in the report.
Age distribution of survey respondents
25 years andbelow
26-45 years
46 years andabove
6
3. Findings
Indicator 1: Food Security
Description of the indicator:
This indicator seeks to map food beneficiaries’ food security by assessing number of meals pr. day, level
of food sufficiency and diversity of food intake.
Impact assessment:
In 2012, 40% of beneficiary households stated to rely on one meal pr. day. In 2013, this proportion is
reduced to 27%, while 55.6% of respondents in 2013 live on two meals pr. day and 16.2% on three meals
pr. day. In the end-line survey, respondents have in addition been asked whether they in comparison to
the time before the project have experienced any change in the amount of food available in the
household. To this, 45.3% have indicated that there has been an increase in amount of food available in
the household, while 30.7% says the amount has remained the same and 9.6% that the amount has
decreased.
When this is coupled whether beneficiary households have experienced starvation in the past 12 months,
it is found that a fairly small fraction of 7.4% says this has been the case, and 74.5% says that their
household has not suffered from starvation in the past 12 months. In the qualitative data, it is a frequent
statement that they are “now able to feed the family”, and respondents have stated that their main usage
of additional income from NUSAF 2- related activities goes into food, medical treatment and school fees.
Respondents to the household questionnaire have, however, indicated that there are issues of food
insufficiency in relation to having food for all household members in the meals served. Less than 10% of
respondents says that there ‘always’ or ‘most often’ are food for all household members, while 48.5%
indicate this to be the case ‘sometimes’. As much as 25.5% says that there is ‘rarely’ food for all
household members in the meals in the home.
40 48.2
11
27
55.6
16.2
0
10
20
30
40
50
60
70
80
90
100
1 2 3
Pe
rce
nt
Number of meals pr. day
Number of meals pr. day
2012
2013
7
In a focus group discussion in Motoro town, a lady of a group that had a grinding mill as their HISP,
responded to a question on food diversity. She stated “those days [before joining the NUSAF project] the
only thing me and my children were eating was the residuals from brewing. Now, we eat posho”. In the
household questionnaire, it has been assessed whether this statement reflects a general trend. 34.8% of
respondents indicate that they today consume more varied types of food than one year ago, which
indicates a positive trend for food diversity. At the same time, 31% of beneficiaries says that the diversity
of their food consumption is at the same level as one year ago, and 21.2% says that they eat less varied
food. It thereby appears, that there are different trends of food security among different groups of
beneficiaries. Overall, the distribution of frequency of consuming various food items indicates that the
main sources of food today comprises carbohydrate-based staple foods (referred to by beneficiaries as
“bread”) and green vegetables.
7.4
74.5
17.4
0
20
40
60
80
100
Yes No No answer
Pe
rce
nt
Have you in the past 12 months experienced starvation in the household?
2013
18
1.3 3 0.7
7.59 5
0.89
21
0
5
10
15
20
25
30
Nu
mb
er
of
day
s in
mo
nth
Food item
Frequency of consuming various food items in a month
2013
8
28.4 29.5
41.4
0
10
20
30
40
50
Worsened Same Improved
Pe
rce
nt
Change in ability to meet basic needs
Endline: Do you think there in the past 12 months has been any change in your ability to meet basic needs?
2013
Indicator 2: Income and economic resilience
Description of the indicator:
In this section, beneficiaries’ level of income, savings and assets in the form of livestock is assessed to
explore trends in self-reliance and ability to cope with economic shocks.
Impact assessment:
Respondents in the household survey have made a subjective rating of their income situation. In the
baseline survey, the rating is made numerically on a scale from 1 to 5, and in the end-line the scale has
had five categories from ‘very bad’ to ‘very good’. Due to difference in the scales across the two surveys,
they cannot be directly compared, but the findings presented in the table below do indicate that there has
been a positive trend in the period from 2012 to 2013. In the end-line assessment, 47.3% of respondents
have rated their income situation as ‘good’ or ‘very good’.
Due to the difference in the scales, this measurement is in the end-line survey triangulated with a
question regarding beneficiaries’ perception of change in their ability to meet basic needs. In these
answers there is a similarity with the self-assessments of change in food security above as answers are
spread across the answer categories in a manner that indicates that there has been varied impact among
different groups of beneficiaries. The most significant trend among the answers is, however, that 41.4%
find that their ability to meet basic needs has been improved in the past 12 months.
5.3
15.2 18.4
23.6 23.7
0.7
26.8
46.6
18.5
8.1
0 0.8 0
10
20
30
40
50
Very bad Bad Fair Good Very good Don't know
Pe
rce
nt
How do you perceive the income situation for your household?
2013
2012
9
Additionally, respondents have answered an open-ended question about whether they think that being
part of the project has been of any benefit to their daily life, and if so in which way. The responses have
been mapped into a visual overview, with the size of the text indicating the frequency by which something
is mentioned:
The responses point at that the main benefits experienced by beneficiaries relate to improved access to
food and ability to feed the family, better possibilities of paying school fees and providing scholastic
materials to children, improved ability to buy subsidiary food, better and more balanced diet and improved
medical care and health. During focus group discussions, a number of beneficiaries highlighted that they
had found the project to motivate them to work, and that they had learnt how to engage in productive
activities that had otherwise been outside reach. This relates both to practical skills, but also to the
attitude to work. A man in Nadunget expressed the latter in the following way: “Let me say this like we
pastoralists do. If you want a cow to give milk, you first have to make the effort and take her grazing.
What we have learnt is that it is the same for earning money, you first have to work”.
In the household survey in 2013, household’s economic situation has been explored. There does not exist
equivalently detailed information in the baseline study, but these figures can be used for future reference,
and can be compared to other livelihoods studies among the target group in Karamoja.The figures will be
reported firstly in averages for the entire group of respondents followed by a detailed break-down that
describes the variance among beneficiaries. In 2013, NUSAF2 beneficiaries in Moroto and Nadunget
reportedly had the following average economic details:
Household monthly income 49,000 UGX
Household monthly expenditures 81,300 UGX
Monthly spending on food 47,900 UGX
Estimation of value of household savings 15,700 UGX
10
This indicates that beneficiaries on average spend almost all earnings on food, and that they are faced
with expenditures that exceed their income. When the numbers are broken further down, it is found that
there is a significant degree of variance across beneficiaries behind these average findings.
In relation to the distribution of the proportion of expenditure allocated to food, it is found that 30.1%
spend less than half of their monthly earnings on food, and that the remaining 69.9% of beneficiaries are
spread between spending between the ranges of 50% to 100% of their expenditures.
As a measurement of economic resilience, respondents have been asked about their saving habits and
ability to cope with an economic shock. Regarding savings, there has from 2012 to 2013 been an
increase from 41% to 51% of beneficiaries who state to make savings in the household. In 2013,
respondents have additionally been asked whether they make savings in a group, which 57.6% state to
do.
8.5 21.6
33.3 36.6
0
20
40
60
80
100
1-25% 26-50% 51-75% 76-100%
Pe
rce
nt
ho
use
ho
lds
Percentage of expenditure spent on food
Endline: Distribution of proportion of expenditure spent on food
2013
41 51
57.6
0
20
40
60
80
100
2012 2013 - inhousehold
2013 - in a group
Pe
rce
nt
Proportion of households making savings
11
Among the beneficiaries who make savings, the value of household savings has been broken down in
categories:
Here it is found that a greater percentage of 39.6% of respondents have less than 5,000 UGX saved, and
thus a quite limited liquidity available for crisis situations. 50.2% of beneficiaries have between 5,000 and
40,000 UGX, and 10.3 % have more than 40,000 UGX. Respondents have in addition been asked directly
whether they, in the event that member of the household urgently needed 20.000 UGX for medical care,
would be able to raise the money. To this, 50% of respondents have said yes – divided between 29.2%
who claimed to be able to get the money from household funds and 20.8% who claims they would be able
to borrow the money. It thus appears that there is a social economic resilience among the beneficiaries
that enables a larger fraction than the ones who have liquidity at hand to cope with an economic shock.
During focus group discussions, it has frequently been mentioned that beneficiaries seek to multiply
income that has been earned under NUSAF2, and that many beneficiaries succeed in setting up small
businesses. When inquiring about the types of business vested in, a common answer is brewing of local
brew. While this is a rational choice in a setting where there is a market for it, it may be considered if this
is the most constructive outcome of NUSAF activities – taken the challenges with alcohol addiction and
violence related to alcohol-intake into account.
One more measurement regarding the household’s economic situation has been made in relation to
number of livestock lost and gained in the previous 12 months. These findings indicate that 10% of
households have increased the number of livestock with 1 head, and 4% with 2 heads or more. 14.9%
state to have the same number as earlier, and 11% that they have lost livestock in the period. These
figures are at a background of a significant number of beneficiaries indicating that they neither at the time
of baseline nor end-line were in possession of livestock.
39.6
20.8 15.8 13.6
7.2 3.1
05
101520253035404550
Pe
rce
nt
Amount saved
Endline: Distribution of value of savings among households who make savings
2013
12
Indicator 3: Market Access
Description of indicator:
The measurements under this indicator are made with the purpose of exploring whether there is any
change in beneficiaries’ access to market places and thereby improved access to economic activities.
Impact assessment:
In order to establish information about changes in beneficiaries’ access to markets, a direct and an
indirect question has been included in the survey. In 2013, respondents have directly been asked,
whether their access to a market place has improved in the past 12 months. In this matter, there is a very
clear trend that beneficiaries do experience to have improved market access, as indicated in the following
table.
Also, respondents have been asked where they buy items to cover their daily basic needs, e.g. food and
other essentials. In both baseline and end-line, it is by far the majority of respondents who indicate that
they get this from a town or trading centre. The table above should thus be interpreted in the way that the
access to these trading centers is perceived to be improved in the period. At the same time, there is a
change in the proportion of respondents who indicate to purchase essential items in the village itself. At
the time of baseline it was as few as 1.7% of respondents that stated this as the main avenue for
household purchases, while it by the time of the end-line is 5.3%. This could reflect an emerging level of
economic activities in the villages, while still at a very modest level.
8.4 7.8 6.9
26
38.1
12.8
0
10
20
30
40
50
Pe
rce
nt
Level of change in access to market place
Has your access to a market place been improved in the past 12 months?
2013
13
4. Quality Assessment: Project relevance and accountability in delivery
As a result of DRC-DDG’s commitment to Humanitarian Accountability Partnership (HAP), an assessment
of beneficiaries’ rating of project relevance, delivery on commitments and rating of how accessible it is for
beneficiaries to raise complaints. In relation to project relevance, beneficiaries have rated the choices
available to them for different types of HISPs. A significantly positive trend is found here, with 76.7%
saying ‘satisfied’ or ‘very satisfied, and less than 6% expressing dissatisfaction with the available types of
HISPs.
Subsequently, beneficiaries and a sample of other community members have been asked how satisfied
they are with the quality of the public works project that has been carried out in their area. There appears
to be a high level of satisfaction with the results of public works, and community members who has not
taken part in the public works process are slightly more positive than sampled NUSAF2 beneficiaries.
0.4 5.5 4.6
13.9
62.8
12.8
0102030405060708090
100
Verydissatisfied
Dissatisfied Neitherdissatisfied
norsatisfied
Satisfied Verysatisfied
No answer
Pe
rce
nt
How satisfied have you been with the choices available to your group between different types of IGAs?
2013
2.8 3.7 5.2
15.9
59.4
12.8
1.8 3.6 7.1
23.3
64.3
0
10
20
30
40
50
60
70
Not at all Low extent Someextent
High extent Comletely No answer
Pe
rce
nt
Level of satisfaction with quality
How satisfied are you with the quality of the public works project that has been implemented under NUSAF2 in your parish?
NUSAF2Beneficiaries
Othercommunitymembers
14
On the matter of level of information to beneficiaries and clarity about project details prior to
implementation, the findings are much less positive. In fact, 93.9% of respondents state that they did not
get a clear idea about what to expect from the project before activities started in 2011. This could point at
that there have been weaknesses in the community liaison and community entry process during project
inception.2
In the end-line assessment, beneficiaries have been asked to rate the quality of DRC-DDG’s service
delivery, by answering to what extent DRC-DDG has lived up to commitments. Findings are generally
positive with 69.7% of respondents stating either to a ‘high extent’ or ‘completely’, while 6% say to a ‘low
extent’ or ‘not at all’.
2 DRC-DDG became an implementing partner for NUSAF2 in 2012
93.9
6.1
0
20
40
60
80
100
No Yes
Pe
rce
nt
Did NUSAF staff give you a clear idea of what to expect from the project before activities started?
2013
0.4
5.6
11.2
34.2 35.5
13
0
5
10
15
20
25
30
35
40
45
50
Not at all Lowextent
Someextent
Highextent
Comletely Noanswer
Pe
rce
nt
To what extent has DRC-DDG lived up to commitments?
2013
15
Lastly, as assessment of the access to DRC-DDG’s beneficiary complaints mechanisms has been made,
by asking beneficiaries to rate implementing staff’s approachability in case a beneficiary wants to express
dissatisfaction. Here it is found that 62.5% of beneficiaries find that implementing staff are approachable
either to a ‘high extent’ or ‘completely, and that 17.1% finds that this is the case ‘to a low extent’ or not at
all.
11.1 6 7.1
19.5
43.4
13
0
10
20
30
40
50
Not at all Lowextent
Someextent
Highextent
Comletely Noanswer
Pe
rce
nt
To what extent do you think DRC-DDG staff are approachable if you want to express dissatisfaction?
2013
16
5. Conclusion
In relation to food security, a reduction has been found in the number of people who live on one meal pr.
day. Only a very limited number of households have been faced with starvation in the past 12 months,
and the amount of food available in the households has increased. There has been found increased
dietary diversity and beneficiaries have indicated to experience improved nutrition and health among their
family members. There are, however, still a significant portion of beneficiaries who point at challenges in
ensuring food for all members of the household in the number of meals prepared pr. day.
In relation to household income and economic resilience, there has been found a positive trend in
beneficiaries’ self-assessment of their income situation. In 2013, 47.3% of beneficiaries rate their income
situation as ‘good’ or ‘very good’ There is also a trend towards improved ability to meet basic needs, as it
has been found that 41.4% of beneficiaries state a positive development in this regard. In focus group
discussions, beneficiaries have emphasized that the households are able to access medical care, school
enrollment and purchase of subsidiary food which had been out of reach before.
There has been an increase in the number of persons making savings, and 50% of respondents indicate
to be able to cope with an economic shock for the household.
There has been a significant improvement in market access, and beneficiaries are found to utilize
earnings from Household Income Support Projects (HISP) for investments in small scale business.
Beneficiaries express a high level of satisfaction with the available choices of HISPs, and the quality of
the public works projects has been rated highly by NUSAF2 beneficiaries as well as by other community
members.
There have been identified weaknesses in the information provided to beneficiaries prior to launching of
the project, which DRC-DDG will take into account in the event of leading a community entry process in
the future. Lastly, the majority of beneficiaries find that DRC-DDG has lived out to its commitments in
service delivery in the implementation period.