understand rural livelihoods- comparing disaggregated vs. aggregated approaches to household...

World Development Vol. 40, No. 9, pp. 1810–1823, 2012� 2012 Elsevier Ltd. All rights reserved.

0305-750X/$ - see front matter

www.elsevier.com/locate/worlddevhttp://dx.doi.org/10.1016/j.worlddev.2012.04.030

Asking Questions to Understand Rural Livelihoods: Comparing

Disaggregated vs. Aggregated Approaches to Household

Livelihood Questionnaires

PAMELA JAGGERUniversity of North Carolina at Chapel Hill, USA

Center for International Forestry Research, Bogor, Indonesia

MARTY K. LUCKERTUniversity of Alberta, Edmonton, Canada

Center for International Forestry Research, Bogor, Indonesia

ABWOLI BANANA a
nd JOSEPH BAHATI *
Makerere University, Kampala, Uganda

Summary. — This study tests the hypothesis that both disaggregated and aggregated data collection methods produce similar estimatesof the relative importance of livelihood portfolio activities and expenditures. The results show that different methods of data collectionyield substantively different estimates of livelihood strategies for two indicators: income and expenditure. We also find evidence of a sea-sonal bias in responses to household livelihood questions asked at higher levels of aggregation. Our findings highlight the challenge ofdesigning household surveys to elicit accurate and precise information, and demonstrate that different methods of data collection influ-ence our understanding of rural livelihoods.� 2012 Elsevier Ltd. All rights reserved.

Key words — Africa, livelihoods, forestry, household surveys, methods, Uganda

* We are grateful to the following organizations for funding this research:

the Center for International Forestry Research (CIFOR); the Collective

Action and Property Rights Initiative (CAPRi) of the Consultative Group

on International Agricultural Research; the National Science Foundation

(NSF, Grant No. DDIG 0622392); the Social Sciences and Humanities

Research Council of Canada (SSHRC); the Social Science Research Co-

uncil (SSRC); and the Sustainable Agriculture and Natural Resource

Management Collaborative Research Support Program (SANREM CR-

SP, Grant No. EPP-A-00-04-00013-00). Roughly 270 households partici-

pated in this study. Their willingness to share information and to host us

for numerous hours in their homes was critical to the success of this

project. We are grateful to colleagues at CIFOR, in particular those who

are part of the Poverty Environment Network (PEN) for comments on

early drafts of this paper. Colleagues with the International Forestry R-

esources and Institutions (IFRI) research group also provided valuable

comments. We are particularly grateful to Arild Angelsen for substantive

feedback on this research. Any errors or omissions are our own. Final

1. INTRODUCTION

Academic researchers, development practitioners, donors,and others increasingly recognize the importance of under-standing micro-level socioeconomic dimensions of the ruraleconomy. Our interest in livelihood portfolios, often charac-terized by income, consumption, expenditure, or time use pat-terns, is motivated by the desire to understand the lives of thepoor, how to best design development interventions, and toprovide information about how development policies and pro-jects affect livelihood outcomes. Demand for high qualityinformation on rural livelihoods has resulted in the develop-ment of a diversity of methods for eliciting information fromlocal people (e.g., Basic Necessities Survey (Davies, 1997);Stages of Progress (Krishna, 2005); Sustainable LivelihoodsFramework (Carney, 1998; Ellis & Freeman, 2004); House-hold Livelihood Security Assessments (CARE, 2002); theLiving Standards Measurement Study Surveys (LSMS) etc.).Data collection methods vary greatly with respect to: the useof qualitative and quantitative data collection methods;whether questions are put to individuals or groups of people;and the degree to which information is collected by an inter-viewer vs. generated through joint learning with representativegroups. Discussions regarding the strengths and weaknesses ofalternatives approaches have accompanied the emergence ofthese various methods (Campbell, 2002; Davis & Whittington,1998; Fisher, Reimer, & Carr, 2010; Menton, Lawrence,Merry, & Brown, 2010; Scoones, 1995; Schrekenberg et al.,2010).

1810

Collecting data that accurately characterize rural livelihoodportfolios has proven to be a daunting task. The breadth oflivelihood activities presents particular challenges to research-ers, as many aspects, for example the role of forest or environ-mental goods and services that have thin or missing markets,may be omitted from standardized data collection efforts(Campbell & Luckert, 2002; Sjaastad, Angelsen, Vedeld, &Bojo, 2005; Vedeld, Angelsen, Sjaastad, & Berg, 2004).Characterization of rural livelihoods is difficult even in theabsence of such problems. To be able to fully describe rural

revision accepted: December 12, 2011.

Figure 1. Study area (highlighted in box), Hoima and Kibaale Districts, western Uganda.

ASKING QUESTIONS TO UNDERSTAND RURAL LIVELIHOODS: COMPARING DISAGGREGATED 1811

livelihoods, one would have to follow and document the pro-duction, consumption, expenditure, and/or time allocationdecisions of all household members for an extended periodof time. This approach is both intrusive to the respondent,and has high financial and time costs for the person collectingthe data. Further, there is no consensus regarding what set ofvariables provides the most accurate information about house-hold welfare. Development economists often favor detailedconsumption and expenditure data (e.g., LSMS, the RePEATsurvey (Research on Poverty, Environment, and Agriculturein Ethiopia, Kenya, and Uganda)) whereas other social scien-tists may consider subjective assessments of the relative impor-tance of income sources to more accurately representhousehold livelihood portfolios (Chambers, 1994). Theseinquiries are informed by a diversity of social science researchmethods drawing on a wide range of disciplines including eco-nomics, sociology, anthropology, and geography.

Virtually every researcher engaged in social science fieldwork has faced decisions regarding the appropriate level ofaggregation to collect data. But these decisions are made lar-gely based on individual experiences and disciplinary training,as there is little empirical literature on this topic. Our goal inthis paper is to discuss issues regarding the accuracy and pre-cision of data associated with aggregated vs. disaggregated ap-proaches to household livelihood surveys. We hope to informfurther research that addresses the biases inherent in variousapproaches to collecting data on livelihood portfolios in devel-oping country settings.

One means of differentiating between data collection ap-proaches is to consider how specific vs. general the questionsthat researchers ask to respondents are. In the extreme, the

research question under investigation could be put directlyto local respondents. But this approach is rarely taken. Forexample, researchers would not generally ask respondentsthe question: “How diversified is your livelihood portfolio?”Instead, researchers tend to approach this more general re-search question by asking more specific questions abouthousehold activities. But these questions may still vary signif-icantly regarding the specific vs. general approach taken. Forexample, specific questions may be asked with income andexpenditure surveys that collect information which is highlydisaggregated, requiring the researcher to aggregate individualquantities and prices to calculate representations of portfolios(Campbell et al., 2002; Cavendish, 2000). In contrast, a typicalvillage level participatory rural appraisal exercise (PRA) mayask people to rank the relative importance of livelihoodactivities for a subset or all households in a village (Krishna,Kristjansen, Radeny, & Nindo, 2004). Such an approach isrequesting general information from respondents that isreported in a more aggregated form.

This study asks the question: How does the level of aggrega-tion in data collection affect our understanding of rural liveli-hoods? We test the hypothesis that both disaggregated andaggregated data collection methods produce similar estimatesof the relative importance of livelihood portfolio activities. Inpractice, whether data are collected in an aggregated ordisaggregated fashion is a matter of degree. In the context ofour study, we define disaggregated data collection as informa-tion that is collected with detailed structured household-levelsocioeconomic surveys designed to elicit fine-scale data thatcan be aggregated over variables such as income and expendi-ture, to provide summary information about livelihoods. The

1812 WORLD DEVELOPMENT

aggregated approach, in the context of our study, is a partic-ipatory rural appraisal exercise that asks household memberto think in an aggregate way about categories of livelihoodactivities. Both methods rely on participation from the diver-sity of household members in responding to questions, includ-ing household heads, spouses, children, and other individualsthat contribute to the economic welfare of the household. 1

We use two indicators in our analysis of livelihood portfo-lios: income and household expenditures. Our aim is to betterunderstand how to collect accurate and precise livelihoodsdata, to consider the potential biases arising from the use ofvarious data collection methods, and to contribute to a discus-sion about methodological choices and their implications forour ability to design development projects, formulate policiesto improve rural livelihoods, and assess the impact of develop-ment programs and policies.

We begin in the next section by identifying potentialstrengths and weaknesses associated with disaggregated vs.aggregated approaches to data collection. We then presentan experiment where we collect data on indicators of rurallivelihood portfolios for two sub-samples of the same popula-tion of households in western Uganda using different surveyinstruments: a highly disaggregated income and expendituresurvey, and a participatory rural appraisal (PRA) surveyinstrument that collects more aggregated information. Wecompare the results of these two approaches to see whetherand why they are different. We conclude with a summary ofour assessment about the appropriateness of the methods forvarious research and data collection efforts.

2. ACCURACY AND PRECISION OFDISAGGREGATED AND AGGREGATEDAPPROACHES TO COLLECTING DATA

(a) Data accuracy and precision

In this paper we focus our efforts on issues of data qualityassociated with disaggregated vs. aggregated approaches tocollecting household level data that may be used to character-ize livelihood portfolios. 2 We focus on two attributes thatcontribute to the quality of data: accuracy and precision.Accuracy implies that the data are representative of the truth,although the data may be widely distributed around the truevalue. For example, a grouping of arrows closer to thebull’s-eye of a target is more accurate than a similarly dis-persed grouping that is farther away. 3 The concept of accu-racy is parallel to the concept of bias in statistics. Moreaccurate estimates are less biased. Precision refers to the tight-ness of the distribution of the data collected, whether or notthe distribution is around the true value. For example, youcould have a tight grouping of arrows that is precise, but itcould be inaccurate if not grouped around the bulls-eye. Theconcept of precision is parallel to the concept of variance instatistics. More precise estimates have smaller variances.

The precision of data can be conceptualized at various lev-els. At one level, the precision of a single piece of informationcollected from a household may be assessed by repeated mea-sures to see if there is measurement error. This type of preci-sion can be difficult to measure in household livelihoodsurveys. If one were to visit the same household repeatedly,to see if the same answers were forthcoming, the conditionssurrounding a household’s livelihood, such as seasonality,would likely change between visits, causing answers to change.Alternatively, if one were to repeatedly ask the same questionswithin a short time frame, respondents may get fatigued or

respond differently after having time to contemplate questions.Instead, for the purposes of our study, we consider the preci-sion of a measure of a central tendency (i.e., means). Even ifevery single data point is precise, a mean value may not be,if there is a large variance in the data.

Data on rural livelihood portfolios that are more accurateare desirable as they are closer to being representative of thetrue values of local people. Accurate data that also yield a pre-cise measure of a central tendency may also be desirable asthey may better support conclusions related to research ques-tions. For example, accurate data that yield precise measuresof central tendencies could cause a researcher to state with ahigh level of confidence that “forests contribute 20% to house-hold incomes”. Data that give us precise measures of centraltendencies, but inaccurate responses, are dangerous, as confi-dence intervals built around mean estimates do not capturethe fact that there is an underlying bias that has caused thedata to vary from the truth. While accurate data are alwaysdesired, the precision of central tendencies may or may notbe a desirable attribute. Though precision in a central ten-dency can increase the confidence in our statement regardingthe contribution of forests to total income, variation in datacan be useful to identify causal relationships. For example, ifwe wish to investigate impacts of labor on forest income, thenvariation in the data on forest income, labor, and otherexplanatory variables are desirable.

Disaggregated approaches may be viewed as sampling exer-cises, where each bit of information is collected and then com-bined by the researcher to construct a larger picture.Disaggregated approaches frequently involve collecting datarepresenting short time intervals, repetitively and in detail.Though more aggregated approaches also tend to collectand combine information to construct a large picture, theydo so with less detailed and frequently fewer questions, there-by counting on the respondent to supply more aggregatedinformation and leaving the researcher with less informationto aggregate. Therefore in the discussion that follows, a keyunderlying question is: “Who is in a better position to aggre-gate information; the researcher or the respondent?”

(b) Advantages of disaggregated approaches to collectinghousehold livelihood data

Disaggregated approaches to data collection may be justi-fied by considering that all respondents are limited, to somedegree, in their cognitive abilities to respond to questions.Respondents’ cognitive ability may be limited with respect tomemory, the ability to synthesize or add up activities orevents, and their scope of knowledge. Our ability to recallevents in the medium to long run may make it difficult forrespondents to characterize livelihood activities that occur sea-sonally or over irregular intervals during the course of a year.Under such conditions, responses may be biased toward recentactivities that may not be representative of the entire year (e.g.,Campbell et al., 2002). With repeated seasonal sampling,respondents are tasked with recalling events in recent memory.

Limitations regarding respondent’s ability to synthesizeinformation may similarly be addressed by collecting disag-gregated data. For example, asking a respondent the impor-tance of forest products to their household livelihoodrequires synthesizing the contribution of many types ofgoods and services that forests provide (e.g., fuel wood,poles, mushrooms, ropes, bamboo, etc.). In contrast, disag-gregated information about consumption and expendituresassociated with specific products do not require high levelsof synthesis. Indeed, some of the information collected with


disaggregated approaches, such as gendered time use data incollecting forest products, may be a direct measure of thebehavior that the researcher is trying to understand. Finally,the respondent’s scope of knowledge may be better under-stood with disaggregated rather than aggregated approaches.For example, if questions are asked about the importance offorest products in general, then the researcher has little infor-mation about the breadth and depth of the respondent’sknowledge underlying the response. In contrast, in disaggre-gated questionnaires, respondents answer detailed questionsabout specific forest products such as fuel wood, poles,mushrooms, and bushmeat. The level of confidence and de-tail of their responses is generally a good indicator of overallknowledge of the livelihood activity.

Another potential advantage of the disaggregated ap-proach to data collection is that it may help limit strategicbehavior by respondents. When household members respondto questions, they may answer strategically for a number ofreasons. For example, respondents may purposively portrayan overly negative picture of their situation because they be-lieve it will increase the probability of receiving aid. Alterna-tively, respondents may purposively overemphasize aspects oftheir livelihoods because researchers have told them ahead oftime the objectives of the research (e.g., to value forest prod-ucts). Whatever the source of bias, strategic answers may bedifficult to formulate with disaggregated questions. Withquestions directed toward collecting many discrete pieces ofinformation, it can be challenging for respondents to consis-tently answer numerous questions such that a strategic biasemerges from the data. In contrast, more aggregated ap-proaches may make it easier for respondents to express astrategic bias. An additional advantage to more disaggregat-ed approaches comes from the greater number of observa-tions that such an approach implies. With repeated visitsand more data comes greater power in the use of statisticaltools for assessing the accuracy and precision of information.Finally, we note that disaggregated approaches allow for theestimation of absolute values of income and expenditures(i.e., vs. relative values) which may be important to donors,policy makers, and others engaged in understanding not onlythe relative importance of rural livelihoods but the magni-tude of different activities.

The cognitive limitations of respondents and the potentialfor strategic behavior suggest that a disaggregated approachto data collection could increase the accuracy of the informa-tion collected. If disaggregated approaches are successful atcollecting information within the bounds of respondent’s cog-nition and successful in limiting strategic answers, then, allother things being equal, the information may be thought ofas being closer to the true values of the respondent. But it isnot clear whether such an approach will lead to greater preci-sion of estimates of central tendencies. Even if disaggregatedquestions are posed within the cognitive bounds of respon-dents, these bounds may be highly variable causing disperseddistributions of information.

(c) Advantages of aggregated approaches to collecting householdlivelihood data

Aggregated approaches may also have advantages in uncov-ering the truth. Policy makers and development practitionersare often interested in understanding the livelihood behaviorof local households. If we are collecting information to tryto understand what local people value, and why they do whatthey do, then local perceptions are a primary concern. The day

to day livelihood decisions of respondents are made in the con-text of “bounded rationality” (Jones, 2001) that recognizes thelimits in cognition discussed above. Therefore, informationwhich reflects the more general perceptions of respondentsmay be more relevant to understanding values and behaviorthan the disaggregated information compiled by researchers.Moreover, these general perceptions may, in some ways, bemore holistic indications of respondents’ values than disaggre-gated approaches. For example, in disaggregated approachesthat collect data on prices and quantities of sold and con-sumed forest products, there may be questions regarding therelevance of prices in reflecting local values. Many forest prod-ucts are harvested for home consumption and not marketed.Responses about products that have market prices may notinternalize negative environmental effects of activities such asforest clearing. The market-based information that the re-searcher is aggregating may be less reflective of social valuesthan the values that respondents are aggregating in their per-ceptions.

The advantages regarding the relevance and holistic quali-ties of respondent’s perceptions suggest that an aggregated ap-proach could increase the accuracy of the informationcollected. If aggregated approaches are successful in capturingthese perceptions then, all other things being equal, the infor-mation may be thought of as being closer to the true values ofthe respondent. But, similar to the situation described above,it is not clear whether such an approach will lead to greaterprecision of estimates of central tendencies. Perceptionsregarding the importance of livelihood activities can vary sig-nificantly within a population even if they are accurately col-lected.

A final advantage of aggregated information is the cost ofdata collection. The repeated and intensive sampling associ-ated with more disaggregated approaches generally involveshigher costs. Data collection costs frequently involve expensesto hire, train, and monitor teams of enumerators and supervi-sors. Once the data have been collected, coding, entering, pro-cessing, and analysis of disaggregated data can be costly. Inaddition to saving researchers time, aggregated methods alsosave respondents time. Disaggregated household surveys cantake a very long time to administer, particularly during theearly rounds of data collection when respondents are askedto recall very detailed information in a structured format.Aggregated surveys generally demand far less respondenttime.

(d) Levels of aggregation and data quality

The above discussion points out that, conceptually, thereare potential data quality problems associated with bothresearchers and respondents aggregating information. Withlittle guidance regarding how severe the problems are, orwhich problems are likely to be worse, we turn to an empir-ical approach to attempt to shed light on these issues. Withpotential biases influencing both aggregated and disaggregat-ed approaches, we cannot interpret either approach as beingmore accurate than the other. Therefore, in the empiricalanalysis that follows, we are restricted to assessing whetherdifferent levels of respondent and researcher aggregationyield different results, and then conjecturing about why theresults may be different. Moreover, with no a priori expecta-tions about the relative precision of central tendencies ofdata collected with aggregated vs. disaggregated approaches,we can only report results with the hope that they providefurther insights.


3. STUDY CONTEXT AND DATA

(a) Study area and overview of data collection

Data are from two separate households surveys conductedin western Uganda. We investigate differences in responsesfrom a survey which takes a disaggregated approach to datacollection (the PEN questionnaire), 4 and one which takes anaggregated approach to collecting data on household liveli-hoods (the UFRIC questionnaire) 5. We designed our data col-lection effort to avoid differences in our samples of householdsthat could confound our comparisons. We administered sepa-rate questionnaires (i.e., disaggregated and aggregated) to sub-sets of the same population of rural households. The question-naires were administered contemporaneously, and designed tocapture data on livelihood portfolios for a years’ worth ofactivities.

The study villages fall within Hoima and Kibaale districts(Figure 1) in Uganda’s western banana, coffee, cattle agro-eco-logical zone. 6 Rainfall is moderate and altitude ranges from1000 to 1500 m above sea level. Primary crops planted includebanana, coffee, maize, sweet potato, and cassava. Smallholderskeep cattle, small ruminants, and poultry in extensively man-aged pasture systems (MAAIF, 1995). Landholdings in thearea are relatively small averaging 2.65 hectares per house-hold. Two land tenure systems are common: freehold andmailo. 7 The area has undergone rapid settlement over the past10 years largely by Bakiga migrants from land scarce KabaleDistrict in southwestern Uganda. Tropical high forest is thedominant land use, though forests are being rapidly clearedby recent migrants. Degraded forest mosaics are common,particularly in areas with relatively good market access. Live-lihood strategies in the study area fall within five main catego-ries: agriculture; livestock husbandry, collection of forest andwild products, wage labor, and self employment (i.e., smallbusiness). The labor force is relatively stationary suggestingfew opportunities for households to receive remittance in-come.

(b) The disaggregated approach to data collection (PEN)

The disaggregated survey was conducted in six villagesdrawn from a stratified random sample of villages throughoutHoima and Kibaale Districts. Thirty households were ran-domly selected from each village (N = 170). A list of house-holds residing in each village was compiled, drawing uponinformation from village registers, lists provided by villageleaders, and information from key informants. Polygamoushouseholds were listed according to the wife’s name; each wifewas considered a separate household unless key informantsindicated that wives undertook activities such as cookingand cultivating jointly. Each household was visited quarterly.During each visit, data on the household’s income portfolioand expenditures were collected. When applicable, respon-dents were asked to differentiate between subsistence incomeand cash income. 8 Income and expenditure data were col-lected according to various recall periods thought to matchthe cognitive abilities of respondents (Cavendish, 2002). Addi-tional data were collected on various socioeconomic indicatorsincluding endowments of land, labor, capital, assets, and re-source use. The field work was undertaken between October2006 and August 2007.

A significant amount of time is required to administer thistype of survey, both due to the nature and number of questions,and also due to the need to undertake repeated visits requiredto capture seasonal changes in income and expenditures.

Making repeated visits to households is both time and resourceintensive, particularly if the research is undertaken in a remotearea, or includes households scattered across a large geo-graphic area. Detailed data collection of this nature also re-quires significant time, patience, and recall abilities on thepart of respondents. The average time to complete the incomeand expenditure components of the questionnaire was 45–60 min per quarter. Ideally, both an adult male and adult fe-male from each household participated in each interview, andas many household members as possible were asked to partic-ipate. Analysis of our data reveal that depending on the quar-ter, between 65% and 79% of households with both an adultmale and female had both present for the duration of the inter-view. We estimate that each household devoted roughly 4 h(1 h � 4 quarters) to the study, and in many cases, given theparticipation of multiple household members, the data collec-tion took eight or more hours of household time over thecourse of the year. Gifts of sugar, match boxes, and soap,approximately equivalent to the value of two adult days of dai-ly wage labor, were given to each household after each quar-terly survey to compensate respondents for time away fromproductive activities.

(c) The aggregated approach to data collection (UFRIC)

The aggregated survey was undertaken in one village, purpo-sively selected from among the six villages included in the PENstudy. The criterion for village selection was the presence of acommunity forest, which was important to the broader objec-tives of the UFRIC survey. The sample (N = 89) was the pop-ulation of households in the village, omitting the 30 householdsthat responded to the PEN questionnaire. We purposefullyavoided interviewing the same households as participation inthe PEN study may have biased responses to the UFRIC ques-tionnaire. Income and expenditure data were collected usingparticipatory rural appraisal (PRA) methods involving a twostage process (Chambers, 1994). First, respondents were giventwo sets of cards with pictures depicting 10 categories of activ-ities related to income and expenditure, along with verbalexplanations given by enumerators. Respondents were askedto rank the cards according to their relative importance tothe household. Providing pictures allowed respondents to viewand adjust their ranking decisions; the use of pictures isthought to enhance cross-cultural communication (Bradley,1995). Second, after placing the cards in rank order, householdmembers were asked to distribute 50 beans according to the rel-ative importance or weight of each category of income andexpenditure to the household. The UFRIC questionnaire wasundertaken during the final 2 weeks of March 2007.

The administration of household level PRA exercises is timeconsuming. The household survey took between 40 and60 min per household. Roughly half of the time was devotedto the ranking and weighting exercise for the income andexpenditure data collection. But, by visiting each householdonly once to collect data retrospectively for a year’s worthof activities, the aggregated method significantly reduces theburden on respondents. The household level PRA exercisewas one component of a larger household survey focused onforest management. Data on other socioeconomic variables,household demographics, household welfare, land, and forestuse were also collected. The UFRIC questionnaire was con-ducted in tandem with village level focus groups and the col-lection of plot level biophysical data on forest compositionand condition. Table 1 summarizes the types of questions thathouseholds were asked to respond to regarding activities overa one year period.

Table 1. Summary of survey questions, recall periods and respondents for disaggregated (PEN) and aggregated (UFRIC) questionnaires

Recall period Respondents Questions

Disaggregated Aggregated Disaggregated Aggregated Disaggregated (e.g.) Aggregated

Income from:� Unprocessed forest products� Processed forest products� Fishing/aquaculture� Wild products� Wage labor� Businesses

30 days 1 year At least 1 adult;Preferably 1 adult maleand 1adult female andany other householdmembers

At least 1 adult;Preferably 1 adult maleand 1 adult female andany other householdmembers

During the past 30 days,how much fuel wood didyour household collect (forsubsistence use and sale)?

Step 1: Rank 10 major sourcesof net annual income usingcards according to their relativeimportance to your household

Income from:� Agriculture� Livestock� Livestock� Products� Other

3 months 1 year At least 1 adult;Preferably 1 adult maleand 1adult female andany other householdmembers


During the past3 months, how muchfresh cassava has yourhousehold harvested (forsubsistence use and sale)?

Step 2: Using 50 beans,assign relative weights toeach income categoryaccording to the relativeimportance of each category

Expenditures 1 week 1 year 1 adult male and 1adult female


What cash purchaseshave you made over thepast 7 days?

Step 1: Rank 10 majorsources of cash expendituresusing cards according to theirrelative importance to yourhouseholdStep 2: Using 50 beans,assign relative weights to eachexpenditure category accordingto the relative importance of eachcategory

AS

KIN

GQ

UE

ST

ION

ST

OU

ND

ER

ST

AN

DR

UR

AL

LIV

EL

IHO

OD

S:

CO

MP

AR

ING

DIS

AG

GR

EG

AT

ED

1815


(d) Comparing disaggregated and aggregated data

In our analysis, we begin by testing whether the two samplesare considered part of one population. We do this by comparingdifferences of means for a number of demographic and socio-economic variables. Our logic is that if we can demonstrate thatthe samples are not significantly different with respect to basicsocioeconomic and demographic characteristics, then differ-ences in measures of income and expenditures can be attributedto the different data collection approaches. Our empirical ap-proach is to compare the mean shares and rankings of incomeand expenditures for the various livelihood activities for bothdata collection approaches. In cases where differences betweenthe two approaches are found, we do not conclude which ismost accurate. However, we do discuss potential reasonsfor differences that could be biasing the results. We presentstandard deviations to investigate whether one method yieldsmore precise measures of mean values than the other.

The two approaches produce different types of data. Thedisaggregated approach collects data on absolute levels of in-come and expenditures in units of Ugandan Shillings(UgShs.). The aggregated approach does not involve collectingdata that would allow us to calculate absolute values with spe-cific units, but seeks to measure the relative importance ofalternative activities. Therefore, in order to compare the re-sults of the two approaches we make the data comparable.We do this by calculating relative measures (i.e., shares andranks for each livelihood category) for each data set. We com-pare the results of the two data collection approaches usingtwo general methods. One method is to use pair-wise tests ofsignificant differences between means of shares for each typeof activity. However, because shares of each activity withineach data collection approach sum to one, if there is a largedifference in means for one activity, there is an increasedchance that the remaining activities will also differ significantlybetween the two approaches. 9 Because of this problem, wealso rank the activity shares for each approach, and then com-pare the rank orders.

Another consideration is that for expenditures, the disaggre-gated data are collected for one adult male and one adult fe-male in each household, whereas the aggregated approach isbased on expenditure estimates for all household members. 10

Because we do not have gender specific data for the aggregatedapproach, we summed the data from the disaggregated ap-proach across the two adults. This implies that expendituresby children, the elderly, and other adults in the householdare not sampled in the disaggregated data set, though presum-

Table 2. Descriptive statistics for disaggregated

Variable Disaggregated (PENMe

Forest owned by household, hectares 0.47 (Arable land owned by household, hectares 1.69 (Female headed household (0/1) 0.17 (Household size, number 5.6 (Dependency ratiob 145.3 (Education of household head, years 4.3 (Household head is migrant (0/1) 0.23 (Household owns bike (0/1) 0.55Household owns mobile phone (0/1) 0.13 (Distance to nearest forest, minutes 11.3*

a Values in parentheses are standard deviations.b The dependency ratio is the number of household members under 15 years plof members between 15 and 65 years of age. The ratio is then multiplied by 1* Means for the disaggregated approach are significantly different than those f

ably are part of the household responses in the aggregated ap-proach. Given the nature of cash expenditures in thisparticular setting, we feel that expenditure data for one adultfemale and one adult male are fairly representative of totalexpenditures for a household (i.e., most children and elderlyare not making significant cash expenditures).

4. EMPIRICAL ANALYSIS

(a) How comparable are the disaggregated (PEN) and aggre-gated (UFRIC) samples?

Before making comparisons between our variables of interest(i.e., income and expenditure portfolios) we investigate whetherthe samples we drew for the disaggregated and aggregated sur-veys are comprised of comparable households (Table 2). Weconducted t-tests (i.e., two-group mean comparison tests) tocompare means of common demographic and socioeconomiccharacteristics of the two groups. 11 The test assumes the vari-ances for the two populations are the same. If two tailed p valueis <0.05 we conclude that the difference of means between malesand females is different from 0. The data demonstrate that thereare few differences between the samples. The variable minutes tonearest forest is the only variable where we observe mean valuesthat are significantly different between the two samples at the5% level. Though the lack of significant differences in meanscould also be due to large variances, we note, subjectively, thatneither the variances nor the differences in the absolute values ofthe means appear to be large. From these data we conclude thatwe have administered the two separate questionnaires to rela-tively comparable sub-samples of households, and that if differ-ent methods yield different characterizations of livelihoodportfolios, these differences are likely to be due to differencesin survey methods.

(b) Household income portfolios

Estimates of the proportion of annual net household incomefrom various activities are presented in Table 3. Householdswere asked to consider sources of subsistence and cash in-come, less cash payments for variable inputs and hired labor(i.e., net income), for both the disaggregated and aggregatedquestionnaires. The results indicate the disaggregated andaggregated methods yield quantitatively different pictures ofthe relative importance of various sources of income. Themean values or share of total net income from various

(PEN) and aggregated (UFRIC) samplesa

) sample (N = 170) Aggregated (UFRIC) sample (N = 89)an Mean

0.78) 0.68 (1.45)1.18) 1.88 (3.14)0.37) 0.12 (0.32)2.6) 5.2 (2.6)109.8) 134.3 (115.9)3.3) 4.5 (3.2)0.42) 0.31 (0.46)(0.5) 0.56 (0.54)0.33) 0.07 (0.26)(13.2) 7.4 (7.8)

us the number of household members over 65 years divided by the number00.rom the aggregated approach at the 5% level.

Table 3. Measures of income, shares and ranks for disaggregated (PEN) and aggregated (UFRIC) samplesa

Sources of income Mean net income (subsistence + cash) Mean net cash income

Disaggregated (N = 170) Aggregated (N = 88) Disaggregated (N = 170) Aggregated (N = 88)

% Share of total Rank % Share of total Rank % Share of total Rank % Share of total Rank

Agriculture 47.0* 1 34.7 1 27.5* 1 39.5 1(17.2) (17.6) (23.6) (21.3)

Business 12.7* 2 6.1 6 27.5* 1 7.2 5(17.2) (11.0) (28.7) (13.8)

Unprocessed forest products 10.9* 3 14.4 2 1.1* 8 4.8 9(6.7) (6.6) (5.0) (5.6)

Wild products 10.2* 4 4.5 8 0.3* 10 4.9 8(5.9) (4.7) (1.3) (5.3)

Livestock 5.7* 5 9.5 3 14.6* 3 10.6 3(7.4) (8.3) (17.8) (10.0)

Wages 5.1* 6 9.5 3 12.5 4 10.9 2(9.9) (13.0) (20.4) (15.2)

Other 3.4 7 4.1 9 10.8* 5 5.2 7(4.5) (8.6) (14.9) (11.6)

Livestock products 2.4 8 2.7 10 1.1* 8 3.0 10(4.8) (4.4) (4.1) (5.2)

Remittances 1.3* 9 6.7 5 3.0* 6 8.4 4(5.6) (13.8) (11.3) (16.6)

Processed forest products 1.2* 10 4.6 7 1.4* 7 5.5 6(3.0) (7.1) (6.7) (9.3)

a Values in parentheses are standard deviation.* Means for the disaggregated approach are significantly different than those from the aggregated approach at the 5% level.


livelihood activities are significantly different for the two sur-vey methods for all categories, with the exception of the rela-tive shares of income from livestock products and otherincome. 12 The largest differences are observed for agriculture(+12.3% for disaggregated approach), business (+6.6% fordisaggregated approach), and wild products (+5.5% for disag-gregated approach).

Another way of comparing data on income portfolios is byranking the categories used in both surveys and observingwhether there is general consensus between the two methodson the order of importance of each income category. InTable 3, agriculture is ranked first using both survey meth-ods, for both net total and cash income, but beyond this firstcategory there is considerable variation. For example, if youwanted to design a policy intervention around the three mostimportant sources of net income for rural households in thestudy area, the disaggregated method points to focusingattention on agriculture, business, and the harvesting ofunprocessed forest products. Analysis of the data using theaggregated method points to agriculture, unprocessed forestproducts, and wage income. If we look at the top five cate-gories, three sources of net income are common to bothsurvey methods.

To test whether respondents are better at recalling cash (vs.subsistence) income we present relative shares and rankingsfor sources of household cash income. A limitation of collect-ing income data in rural settings where a considerable shareproduction is for home use is that it may be cognitively diffi-cult for respondents to aggregate items they have not mar-keted (e.g., number of cobs of corn harvested and consumedby all household members, or number of head loads of fuelwood collected by women and children). With the exceptionof wage income all estimates of sources of income are signifi-cantly different between the two survey methods. There arelarge differences in the relative importance of agriculture(+12% for aggregated approach), and business income

(+20.3% for disaggregated approach). Rankings of cash in-come suggest that there is considerable synergy between thetwo methods. Agriculture, business, livestock, and wages areamong the top five sources of cash income using both meth-ods.

Further to the discussion above, we hypothesize that someof these differences may arise because the aggregated approachis more directed toward subjective measures of importance,while the disaggregated approach is more directed towardobjective quantitative measures of income. The subjectivemeasures may include considerations such as the importanceof income sources as safety nets during hard times from eventssuch as droughts or a household member falling ill. If thisinterpretation is correct, we would expect unprocessed forestproducts, livestock, wage income, and remittances to be moreimportant in the subjective, aggregated approach becausethese activities play larger safety net functions than agricultureand business income which are more regularized activities.This explanation seems to be supported by the results, as localreturns to agriculture and business are likely to be disruptedby droughts and other shocks, while forests products (pro-cessed and unprocessed), remittances, wages, and livestockmay act as safety nets for livelihoods (Dercon, 2002; Pattana-yak & Sills, 2001).

Considering measures of net cash income also seems tosupport this line of reasoning. For those sectors that havecash flows that persist during droughts and other shocksto the household, we would expect subjective measures oftheir importance to be higher than the more quantitativemeasures. Accordingly, as per above results, we see higherrankings for unprocessed and processed forest products,and remittances for aggregated approaches. But the same lo-gic does not hold for agriculture, wild products, or livestockwhich have reversed patterns compared to net total incomemeasures. For agriculture and wild products, higher aggre-gated values suggest that the cash part of income may be

Table 4. Proportion of expenditures for various categories of goods and servicesa

Categories of expenditures Mean expenditures with “other” categorya Mean expenditures without “other” categorya

Disaggregated(N = 173)

Aggregated (N = 80) Disaggregated(N = 173)

Aggregated (N = 80)

Share of total Rank Share of total Rank Share of total Rank Share of total Rank

Other 46.3* 1 9.2 4 NA NA NA NA(27.7) (11.9)

Agricultural production and food 19.6 2 23.9 1 32.9 1 25.8 2(26.3) (12.0) (33.0) (12.2)

Livestock 11.0* 3 5.8 7 23.4* 2 6.1 7(14.1) (7.4) (26.3) (7.7)

Medical 10.3* 4 23.2 2 18.3* 3 26.5 1(17.2) (14.7) (26.6) (16.9)

Entertainment 5.4* 5 11.2 3 10.9 4 12.5 3(11.0) (7.5) (20.1) (8.3)

Transportation 3.6 6 3.4 9 6.1 5 3.6 8(11.6) (4.5) (0.36) (4.8)

School fees and other education 2.5 7 8.9 5 4.1* 6 9.7 4(9.3) (9.4) (14.1) (10.4)

Formal social occasions 1.2* 8 5.4 8 4.0 7 6.3 6(3.8) (5.0) (14.4) (6.3)

Forest products 0.1* 9 8.9 5 0.2* 8 9.3 5(0.7) (9.4) (1.8) (10.4)

Fish 0 10 0.1 10 0 9 0.2 9(1.1) (1.5)

a Values in parentheses are standard deviation.* Means for the disaggregated approach are significantly different than those from the aggregated approach at the 5% level.


important during periods of low net income flows, while netcash income from livestock may be less important duringlow income flows.

Comparisons of the standard deviations between the twoapproaches suggest that the precision of estimates of meansare similar for most sources of income. Standard deviationsfor all values of income shares are generally two times meanvalues or smaller, with exceptions largely occurring in caseswhere there are small mean values. In sum, neither methodseems to distinguish itself regarding the precision of mean in-come estimates.

(c) Household expenditure portfolios

The mean share of annual household expenditures attrib-uted to various activities is summarized in Table 4. As withthe income portfolio data, there are considerable differencesin the expenditure shares observed using the disaggregatedand aggregated data collection methods; six of the 10 catego-ries are significantly different. The most striking finding is theimportance of the “other expenditure” category. The collec-tion of detailed data on weekly expenditures illustrates the rel-ative importance of many “small” purchases to overallexpenditures. Data from the disaggregated questionnaire dem-onstrate that that there is a diverse set of expenditures thatwere not accounted for in the categories that were formulatedfor the PRA exercise that produced expenditure shares for theaggregated method. For the disaggregated data we coded over55 different types of expenditures reported by adult males andadult females.

Because of the overwhelming impact of the “other” categoryon shares, we also present results without the “other” categoryincluded. For these results, there are significant differences infour of the nine activities; livestock, medical, school fees,and forest products. For livestock expenses, we suspect the

same bias is occurring as may have been the case for the“other” category. Livestock includes frequent and numeroustypes of expenditures that may be underestimated if not delin-eated with disaggregated methods. On the other hand, thedisaggregated approach, which only collected data for expen-ditures seven days prior to each quarterly visit, may havemissed some irregular, yet relatively large household expendi-tures such as medical and school fees. The high standard devi-ations associated with these categories for the disaggregatedapproach supports this hypothesis. The rank order of house-hold expenditure categories is also somewhat inconsistent be-tween the two approaches. For the disaggregated approachwithout the other category, the three most important expendi-ture categories are agricultural production and food, livestock,and medical expenses. The aggregated method tells us thatmedical expenses, agricultural production and food, and enter-tainment are the three most important expenditure categories.Considering the top five categories for each method, three cat-egories are common. Comparisons of the standard deviationsbetween the two approaches suggest that the precision of esti-mates of means are somewhat greater for the aggregated ap-proach, which generally has standard deviations similar to,or lower than, mean values. In contrast, the disaggregated ap-proach tends to have standard deviations larger than themeans.

As noted in the discussion of sampling and questionnaire de-sign, the data we present for the disaggregated method reflectexpenditures over a seven day time period (for each of fourvisits) for one adult male and one adult female from eachhousehold. Given the limited amount of cash income thathouseholds in these communities have, and limited opportuni-ties for spending cash, we feel that our analysis of the disaggre-gated data are a reasonable proxy of household expenditurepatterns for the entire household. That is, we do not feel thatwe missed a large share of expenditures by failing to collect

Table 5. Impact of seasonal timing on responses for net income.

Source of income Mean shares (%) of total net income (subsistence + cash)a

Disaggregated all quarters Aggregated Disaggregated third quarter(N = 170) (N = 88) (N = 162)

Agriculture 47.0* 34.7 37.3(17.2) (17.6) (29.5)

Business income 12.7* 6.1 12.9**

(17.2) (11.0) (21.6)Unprocessed forest products 10.9* 14.4 13.4

(6.7) (6.6) (10.5)Wild products 10.2* 4.5 15.2**

(5.9) (4.7) (14.1)Livestock 5.7* 9.5 6.4**

(7.4) (8.3) (12.7)Wage income 5.1* 9.5 4.3**

(9.9) (13.0) (10.7)Other income 3.4 4.1 3.6

(4.5) (8.6) (6.1)Livestock products 2.4 2.7 3.2

(4.8) (4.4) (7.9)Remittances 1.3* 6.7 1.5**

(5.6) (13.8) (8.0)Processed forest products 1.2* 4.6 1.9**

(3.0) (7.1) (5.1)a Values in parentheses are standard deviation.* Means for the disaggregated approach are significantly different than those from the aggregated approach at the 5% level.** Means for the disaggregated third quarter data are significantly different than those from the aggregated approach at the 5% level.


expenditure data for children, elderly, or other householdmembers. Research efforts that involve expenditures shouldexperiment with other categories, acknowledging that rankingand weighting more than 10 categories may be cognitivelychallenging for respondents.

(d) Does seasonal timing of questionnaires affect responses?

Recall that the disaggregated data were collected over fourquarters in order to capture data through a full calendar year.The aggregated data were collected once during a 2 week per-iod that overlapped with the third quarter of data collectionfor the disaggregated survey. Ideally, and consistent with thequestions asked, we would like the aggregated responses to re-flect the entire previous year. However, as discussed above,limited ability to recall information could cause the responsesto be biased by what has happened recently, in this case duringthe third quarter. If this is the case we would expect responseswith the aggregated method to be more similar to the thirdquarter results of the aggregated method than to the resultsfor the full year. Table 5 presents the results of this compari-son for net income. The results of the comparison betweenthe disaggregated third quarter data and the aggregated datasuggest that the aggregated results are influenced by recentlivelihood activities for some categories. Whereas almost allsources of income from the disaggregated all quarters ap-proach are significantly different to the aggregated results,we find that there are no significant differences between re-sponses for third quarter income in the disaggregated data col-lection and the aggregated approach for four categories:agriculture; unprocessed forest products; livestock products;and other sources of income. 13 The findings suggest that theaggregate approach is biased toward the specific season duringwhich the data are collected for two very important categoriesof income, agriculture and unprocessed forest products. The

finding highlights the importance of recall periods, and dem-onstrates the potential limitations of collecting household in-come data using a one year recall period.

5. DIFFERENT METHODS, DIFFERENT STORIES

Household livelihood data are widely used by developmentpractitioners, donors, researchers, and other stakeholders for adiversity of purposes including: characterization of rural live-lihoods; targeting development interventions; evaluating thesuccess of a policy or program; etc. For any of these typesof activities, monitoring or research initiatives having datathat accurately portray rural livelihoods are essential. How-ever, what we have learned through our experiment in usingdifferent data collection methods to elicit the same informationfrom comparable samples is that different methods yield differ-ent stories. We consider the question of the relatively impor-tance of forest products to rural households using both thedisaggregated and aggregated datasets.

(a) How important are forests to rural livelihoods in westernUganda? (disaggregated)

Rural households in western Uganda derive 10.9% of theirtotal annual net income from unprocessed forest productssuch as fuel wood, bamboo, mushrooms, and other forestfoods, and only 1.2% of income from processed forest prod-ucts including charcoal, sawn wood, woven products, etc. In-come from agriculture and business are both more importantthan forest income (47.0% and 12.7% respectively). The major-ity of forest products are harvested for subsistence use byhouseholds, with only a very small share (approximately2.5%) of total cash income coming from the sale of forestproducts. Households in western Uganda seldom purchase

Table 6. Determinants of share of forest incomea,b

Disaggregated Aggregated

Forest owned by household, hectares 2.355***R 0.992(0.844) (1.044)

Arable land owned by household, hectares �1.772***R �0.304(0.597) (0.481)

Female headed household (0/1) 0.959 �0.623(1.788) (3.644)

Dependency ratioc �0.006 0.031***

(0.006) (0.009)Household head is migrant (0/1) 1.900 �6.321***

(1.461) (2.306)Education of household head, years �0.218 0.409

(0.202) (0.339)Owns bike (0/1) 1.287 �1.362

(1.318) (2.097)Owns mobile phone (0/1) �0.030**R+ �10.752**

(0.047) (4.161)Time to nearest forest, minutes �0.029 0.116

(0.048) (0.134)N 167 81R—squared 0.1663 0.2815

a Values in parentheses are standard errors.b R denotes robust finding for sign of the coefficient when comparing the two models; R+ denotes robust finding both in sign and level of significancebetween the two models.c The dependency ratio is the number of household members under 15 years plus the number of household members over 65 years divided by the numberof members between 15 and 65 years of age. The ratio is then multiplied by 100.

* Indicate statistically significantly at 10% respectively.** Indicate statistically significantly at 5% respectively.

*** Indicate statistically significantly at 1% respectively.


forest products. Data on household expenditures by one adultmale and one adult female from each household suggest thatrural households are on average spending 0.1% of their cashincome on forest products. The largest shares of cash expendi-tures are for agriculture, livestock, and medical expenses(19.6%, 11.0% and 10.3% respectively). Rural households inthis region have surprisingly diverse expenditure patterns witha large number of items falling outside of standard expendi-ture categories (46.3%).

(b) How important are forests to rural livelihoods in westernUganda? (aggregated)

Rural households in western Uganda derive 14.4% of theirtotal annual net income from unprocessed forest productssuch as fuel wood, bamboo, mushrooms and other forestfoods, and 4.6% of income from processed forest productsincluding charcoal, sawn wood, woven products, etc. Incomefrom agriculture is the only category that is more importantthan forest income (34.7%). Livestock and agricultural wagelabor are the third and fourth most important sources of in-come for rural households. Approximately 10% of annualcash income for households in the study area comes fromthe sale of forest products. Our analysis suggests that house-holds are selling almost half of all harvested forest productsfor cash income. In addition to selling forest products,households in western Uganda purchase forest products.Nine percent of total cash expenditures are on unprocessedor processed forest products. After agricultural productionand food, medical expenses, entertainment, and miscella-neous expenditures, forest products are the 5th most impor-tant expenditure.

(c) Determinants of the relative importance of forest income

We estimate a simple OLS regression model to explain thedeterminants of the share of household income from forestproducts. We consider endowments of forested and agricul-tural land, basic household demographics including indicatorsof human capital, ownership of assets, and the distance to thenearest forest according to the following specification:

Y i ¼ b0 þ b1land þ b2labor þ b3capitalþ b4minforest þ ei

ð1ÞAccess to forests is expected to be an important determinant ofthe share of income a household derives from forests. We expecta positive relationship between forest owned and household in-come derived from forests, and a negative relationship betweenthe distance from the households to the nearest forest and theshare of income from forests. Households that have to travelfurther to collect forest products will gather fuel wood, wildfoods, and other common forest products from other land typesincluding fallows and bush lands. In many settings in sub-Sah-aran Africa we find a correlation between forest dependenceand poverty suggesting that female headed households, house-holds with high dependence ratios, and households with limitedassets should be more dependent on forests. We do not have astrong hypothesis about the relationship between migrationand forest dependence in the study area. Migrant householdsmay be more dependent on forests if they have limited landholdings and portfolios dominated by relatively few incomesources. However, migrant households may have trouble gain-ing access to forested areas, which would limit the share of in-come from forests. Regression results are presented in Table 6.


Relatively few of the findings are robust across datasets. Weconclude that forest ownership is a very important determi-nant of forest income, and that distance to nearest forest isnegatively, but not significantly correlated with forest depen-dence. We also draw the conclusion that having a large endow-ment of arable land reduces forest dependence. Using thedisaggregated data none of the demographic or human capitalvariables are significant. We find that ownership of a mobilephone, an important indicator of wealth in the study area, isnegatively associated with forest dependence.

Regression results for the aggregated data highlight a differ-ent set of relationships. We find that forest ownership is pos-itively correlated with forest dependence, and that ownershipof arable land is negatively correlated with forest dependence,however, neither of these findings is statistically significant. Ahigh dependency ratio, which is an indicator of large numbersof children and elderly in a household relative to the numberof productive adults in the household, is a statistically signifi-cant determinant of forest dependence. In the model using theaggregated data there is a significant and negative relationshipbetween migration and forest dependence, suggesting thathouseholds that do not have longstanding ties to the commu-nity are less forest dependent. This could be interpreted as alimitation on access to forest resources for households thatare relatively new to a community. As with the regressionmodel using the disaggregated data, mobile phone ownershipis negatively correlated with forest dependence, suggestingthat wealthier households are less reliant on forests.

The major point is that the stories derived from these tworegression models are slightly different. In the disaggregatedmodel we emphasize the importance of ownership of forests,access to forests, and relative wealth as evidenced in mobilephone ownership as important determinants of forest depen-dence. We summarize the story from the aggregated datasetas one of wealth determining forest dependence, large house-holds with fewer productive adults are dependent on forests,and households with mobile phones are less dependent on for-ests. If our interpretation of migrant households having lim-ited access to forests is accurate then we also conclude thataccess to forests is important, but we cannot confirm this giventhe lack of significant coefficient on the forest ownership var-iable. Further the positive relationship between time to thenearest forest and forest dependence muddies the story; wehypothesize a negative relationship between these variablesthat is supported by the analysis of the disaggregated data.

6. DISCUSSION AND CONCLUSIONS

Our goal in designing this study is to understand how two ofthe methods commonly used to capture data on rural liveli-hoods influence the calibration of indicators frequently usedfor understanding the lives of the poor, targeting developmentinterventions, and monitoring and evaluating the effectivenessof policies, programs, and projects. Our results demonstratethat we obtain different views of the relative importance of rur-al livelihood strategies across indicators of income and expen-

diture, depending on the method used to collect the data.Ranking data are also different using disaggregated vs. aggre-gated methods, though there is considerable overlap in the cat-egories that show up as most important for estimates of theimportance of income sources. We find evidence of seasonalbias in the reporting of the relative importance of agricultureand unprocessed forest products using more aggregated datacollection methods, suggesting that disaggregated methods fo-cused on capturing seasonal variation may be more accurate.

Despite evidence of seasonal bias, the puzzle for us as we seekto draw lessons from this research is that we cannot say withconfidence which approach, disaggregated or aggregated, pro-vides the most accurate representation of the truth. One obvioussolution to the problem of differing findings on the relativeimportance of livelihood strategies is to triangulate data collec-tion efforts. However, the challenge for the use of mixed meth-ods is that disaggregated data need to be processed before wecan begin to identify trends and patterns at the aggregate level.For accuracy to be verified through triangulation, researcherswould need to be able to process disaggregated data on the spotand highlight conflicting observations immediately. 14 Further,we are cognizant of the additional time and resources involvedin collecting data at multiple levels of aggregation.

Another important question is what the data are to be usedfor. Generating a broad understanding of livelihood strategiesin rural settings is a different proposition than developingbehavior models. What we do know is that researchers andpractitioners are employing both methods in field settingsand that their findings are frequently drawn upon to informpolicy. Researchers and development practitioners shouldstrive to collect both accurate and precise data, taking careto understand and qualify findings in light of the apparentstrengths and weaknesses of various data collection methods.All too frequently we are quick to judge the method supportedby our disciplinary training as the most accurate.

There are several opportunities for future research on the to-pic of the appropriate level of aggregation for collecting dataon livelihood indicators. Our experiment could be tested rela-tively easily in other settings at the household level to see if theresults are generalizable. In particular, looking across settingswith higher and lower levels of market integration would helpus better understand the importance of level of aggregation indesigning data collection instruments. It may be that house-holds more fully integrated into the market economy providedisaggregated and aggregated data that are more similar. Fi-nally, another direction for future research is to use innovativemethods for collecting data on “true” income and expendi-tures of respondents. As noted in the introduction to this pa-per, direct observation of a large sample of households for theduration of a year is not generally feasible. However, technol-ogy is rapidly evolving, and the use of devices such as GPSunits, PDAs, and cellular phones may be used to collect andtransmit data in real time. Such approaches may present animportant opportunity to collect data that would allow for tri-angulation with conventional survey methods to enhance ourunderstanding of rural livelihoods.

NOTES

1. For a discussion on issues associated with asking questions of differenthousehold members, see Fisher et al. (2010).

2. As mentioned above, there are numerous issues in addition to dataquality, such as the role of local participation in generating information,which have been discussed elsewhere and are beyond the scope of thispaper.


3. Explanations that relate the target analogy to concepts such asaccuracy, precision, bias, and variance are widespread (e.g. Wonnacott &Wonnacott, 1977).

4. The PEN data were collected as part of a Poverty EnvironmentNetwork (PEN) study undertaken in western Uganda. PEN is a networkof roughly 35 Ph.D. scholars contributing to a global comparativedatabase on the relationship between forest use and poverty in the lowincome tropics. PEN is administered by the Center for InternationalForestry Research (CIFOR). See http://www.cifor.cgiar.org/pen/_ref/home/index.htm. Modules on household expenditures and time use wereadded to the core PEN questionnaire.

5. The UFRIC data were collected as part of the Sustainable Agricultureand Natural Resource Management Collaborative Research SupportProgram (SANREM-CRSP) Long-Term Research Grant Decentralization

Reforms and Property Rights: Potentials and Puzzles for Forest Sustain-

ability and Livelihoods. The project is a collaboration between: Workshopin Political Theory and Policy Analysis, Indiana University; CGIARSystem Wide Program on Collective Action and Property Rights(CAPRi); and CIFOR. The Uganda Forestry Institutions and ResourceCenter (UFRIC) at the Faculty of Forestry and Nature Conservation,Makerere University is the local collaborator on the project.

6. The total population of the four sub-counties which the study takesplace in (i.e. Kiziranfumbi (Hoima District)), Kakindo, Kasambya, andKiryanga (all in Kibaale District) was estimated to be 84,587 in 2002(UBOS 2006). We estimate that our sample includes approximately 1,200individuals. The participants in our study represent approximately1.4% ofthe total population of the four sub-counties in which the study tookplace.

7. Under the freehold system landowner property rights are thought tobe secure; landowners hold registered land indefinitely. The mailo tenuresystem was established by the British colonial government in 1900. Legalland titles were given to the Buganda royal family. The land was measuredin square miles which is where the term mailo comes from. Land waspartitioned into smaller units and rented out to tenants. Under the 1998

Land Act tenants were granted freehold status for mailo parcels held since1986. While the 1998 Land Act clearly outlines the provisions of variousland tenure systems and land rights, the act is weakly enforced (Nkonya et

al., 2004).

8. The categories of wage income, business income, and remittances areassumed to be entirely cash income.

9. Note that the problem of interdependent observations is an inherentproblem in using data that rely on relative rather than absolute measures.

10. In the study area the average number of adults per household is 2.2(i.e. those over the age of 15).

11. We calculated means and standard deviations for the sub-populationof the disaggregated dataset (N = 28) for the villages where theaggregated survey was conducted. These households are not statisticallysignificantly different from the other households in the disaggregatedsample for variables important to forest-based livelihoods includinghectares of forest owned by the household, distance to forest etc. Therewere differences with respect to the number of migrant households (40%),and asset ownership (bicycle and mobile phone ownership was 64% and21% respectively).

12. Other income includes: support from the government or NGOs; giftsincluding condolences; pension income; payments for forest services;payments for renting out land/houses; sale of land/standing trees; etc.

13. Recall that we hypothesize above that results for forest productsfrom the aggregated approach may be biased by the strategic behavior ofhouseholds.

14. Recent innovations in tracking household activities (cf. Shirima et al.

(2007) who use PDAs to collect and aggregate data in real time) could helpus understand for a set of representative households what the mostaccurate representation of income, expenditures, and time use.

REFERENCES

Bradley, S. (1995). How people use pictures: An annotated bibliography.London, UK: International Institute for Environment and Develop-ment.

CARE. (2002). Household livelihood security assessments: a toolkit forpractitioners. Atlanta, GA: CARE, USA.

Campbell, J. (2002). A critical appraisal of participatory methods indevelopment research. International Journal of Social Research Meth-odology, 5(1), 19–29.

Campbell, B. M., Jeffrey, S., Kozanayi, W., Luckert, M. K., Mutamba,M., & Zindi, C. (2002). Household livelihoods in semi-arid regions:Options and constraints. Bogor, Indonesia: Center for InternationalForestry Research.

Campbell, B. M., & Luckert, M. K. (Eds.) (2002). Uncovering the hiddenharvest: Valuation methods for woodland and forest resources. Peopleand Plants Conservation Series. London, UK: Earthscan PublicationsInc.

Carney, D. (Ed.) (1998). Sustainable rural livelihoods: What contributioncan we make?. London, UK: DFID.

Cavendish, W. (2000). Empirical regularities in the poverty-environmentrelationship of rural households: Evidence from Zimbabwe. WorldDevelopment, 28(11), 1979–2003.

Cavendish, W. (2002). Quantitative methods for estimating the economicvalue of resource use to rural households. In B. M. Campbell, & M. K.Luckert (Eds.), Uncovering the hidden harvest: Valuation methods forwoodland and forest resources. London, UK: Earthscan PublicationsLtd.

Chambers, R. (1994). The origins and practice of participatory ruralappraisal. World Development, 22(7), 953–969.

Davies, R. (1997). Beyond wealth ranking: The democratic definition andmeasurement of poverty. Available from: <www.mande.co.uk/docs.democrat.htm>.

Davis, J., & Whittington, D. (1998). “Participatory” research fordevelopment projects: A comparison of the community meeting andhousehold survey techniques. Economic Development and CulturalChange, 47(1), 73–94.

Dercon, S. (2002). Income risk, coping strategies, and safety nets. WorldBank Research Observer, 17(2), 141–166.

Ellis, F., & Freeman, H. A. (2004). Rural livelihoods and povertyreduction strategies in four African countries. Journal of DevelopmentStudies, 40(4), 1–30.

Fisher, M., Reimer, J. J., & Carr, E. R. (2010). Who should be interviewedin surveys of household income?. World Development, 38(7), 966–973.

Jones, B. D. (2001). Politics and the architecture of choice: Boundedrationality and governance. Chicago, IL: University of Chicago Press.

Krishna, A. (2005). Stages of progress field manual: a community basedmethodology for defining and understanding poverty. Version 2.0.Available from: <http://sanford.duke.edu/krishna/SoP.pdf>.

Krishna, A., Kristjansen, P., Radeny, M., & Nindo, W. (2004). Escapingpoverty and becoming poor in 20 Kenyan villages. Human Organiza-tion, 5(2), 211–226.

MAAIF. (1995). Basic facts on agricultural activities in Uganda.Kampala, Uganda: Ministry of Agriculture, Animal Industry andFisheries.

Menton, M., Lawrence, A., Merry, F., & Brown, N. D. (2010). Estimatingnatural resource harvests: Conjectures?. Ecological Economics, 69,1330–1335.


Nkonya, E., Pender, J., Jagger, P., Sserunkuuma, D., Kaizzi, C. K., &Ssali, H. (2004). Strategies for sustainable land management and povertyreduction in Uganda. IFPRI Research Report No. 133. Washington,DC: International Food Policy Research Institute.

Pattanayak, S. K., & Sills, E. O. (2001). Do tropical forests providenatural insurance? The microeconomics of non-timber forestproduct collection in the Brazilian Amazon. Land Economics,77(4), 595–612.

Schrekenberg, K., Camargo, I., Withnall, K., Corrigan, C., Franks, P.,Roe, D., et al. (2010). Social assessment of protected areas: A review ofrapid methodologies. A report for the Social Assessment of ProtectedAreas (SAPA) Intiative. London: International Institute for Environ-ment and Development.

Scoones, I. (1995). Investigating difference: Applications of wealth rankingand household survey approaches among farming households insouthern Zimbabwe. Development and Change, 26(1), 67–88.

Shirima, K., Mukasa, O., Armstrong Schellenberg, J., Manzi, F., John,D., Mushi, A., et al. (2007). The use of personal digital assistants fordata entry at the point of collection in a large household survey insouthern Tanzania. Emerging Themes in Epidemiology, 4(5), 1–8.

Sjaastad, E., Angelsen, A., Vedeld, P., & Bojo, J. (2005). What isenvironmental income?. Ecological Economics, 55, 37–46.

UBOS. (2006). 2002 Uganda population and housing census. Kampala,Uganda: Uganda Bureau of Statistics.

Vedeld, P., Angelsen, A., Sjaastad, E., & Berg, G. K. (2004). Counting onthe environment: Forest incomes and the rural poor. In EnvironmentalEconomics Series, No. 98. Washington, DC: World Bank.

Wonnacott, T., & Wonnacott, R. (1977). Introductory statistics forbusiness and economics (2nd ed.). New York: Wiley.

understand rural livelihoods- comparing disaggregated vs. aggregated approaches to household...

Documents

usatry research

toprovide information

precise information

diversity of methods

household livelihood

livelihood outcomes

development policies

substantiveworld development