measuring nutrition, health and poverty in small areas – how low can you go?

8/12/2019 Measuring nutrition, health and poverty in small areas how low can you go?

1/61

Measuring nutrition, healthand poverty in small areas how low can you go?

Prof Stephen Haslett

Institute of Fundamental Sciences

- StatisticsMassey UniversityPalmerston North

New Zealand


2/61

Abstract

Efficient targeting of aid relies on detailed information. Thecomplication is that even large scale national sample surveys areusually not accurate enough to provide this detail. Small area

estimation is a statistical technique developed over the last twentyyears or so which can improve accuracy of surveys by usingstatistical modelling. A particular type of small area estimationtechnique, which is generically called poverty mapping and links asurvey with a census, is now often used for estimating and mappingpoverty estimates at a fine level. This method has now been applied

in more than 70 countries. Where there is no recent census,alternative methods exist. Small area estimation can also beextended, for example to estimating malnutrition including stunting,underweight and wasting in children under five years of age. Thisgeneral talk covers context, what small area estimation is and thelevel at which it can work, and why it is useful.


3/61


4/61

Goals, Targets and IndicatorsGoal 1. Eradicate extreme poverty and hunger

Target 1. Halve, between 1990 and 2015, the proportion of people whose income is less thanone dollar a day

Indicators 1. Proportion of population below $1 (1993 PPP) per day (World Bank) a 2. Poverty gap ratio [incidence x depth of poverty] (World Bank)3. Share of poorest quintile in national consumption (World Bank)

Target 2. Halve, between 1990 and 2015, the proportion of people who suffer from hunger

Indicators 4. Prevalence of underweight children under five years of age (UNICEF-WHO)5. Proportion of population below minimum level of dietary energy consumption (FAO)

Footnotes: a For monitoring country poverty trends, indicators based on national poverty lines shouldbe used, where available.
http://localhost/var/www/apps/conversion/BU/LaptopToshiba%20-%20KEEP/LaptopToshiba061009/E/PovertyGeneralSeminars/Seminar/mi_indicator_xrxx.asp%3find_code=5http://localhost/var/www/apps/conversion/BU/LaptopToshiba%20-%20KEEP/LaptopToshiba061009/E/PovertyGeneralSeminars/Seminar/mi_indicator_xrxx.asp%3find_code=5


5/61

Progress


6/61

Food requirements (kcal/day)

Basket of typical food items

Kcals from basket

Cost of suitably-scaled basket

Cash required for a person to be able to eat enough

Food Poverty Line

3.60 kcal/g

$1.25


7/61

Household Income and Expenditure Surveys

Income or Expenditure?

Per capita? Per adult equivalent?

Income and Expenditure Data


8/61

Poverty Line = Food Poverty Line + Non-food Expenses

Look at non-food expenditure of those near food poverty

Economic Poverty


9/61

Poverty Incidence proportion below the poverty line

Poverty Gap average amount below the poverty line

Poverty Severity gives more weight to the very poor

FGT = Fos t e r-Greer-Tho rb ecke

FGT Measures

ijR ij

ij R R

Z - Y1P = Ind(Y < Z)

N Z


10/61

Measure kCal intake directly- measurement error ; equivalence scale

Malnutrition: stunting, wasting, underweight- reference population; wasting vs underweight

Aggregation- weighted index; multivariate statistics

using Principal Component Analysis (PCAs)

Other Measures


11/61

What is small area estimation?SAE is a statistical modelling technique for extracting more

information out of existing data.

For modelling poverty measures, it usually involves usingboth survey and census data.

Small area estimates are often presented as poverty maps,which lose detail but are easier to look at than tables ofnumbers.


12/61

Where?

Brazil

Ecuador

South Africa

Cambodia

Thailand

MozambiqueMadagascar

Bangladesh

Philippines

Vietnam

Indonesia

NepalLao PDR


13/61

Why bother?

Small area estimation (SAE) and mapping have importantroles in understanding the dimensions and finer details ofpoverty

They have also become increasingly important as requiredbackground information for aid allocation nationally andinternationally, and remain central in their monitoring rolefor MDGs


14/61

SAE - Bangladesh


15/61

SAE - Philippines

Provincial Poverty Incidence0.066 - 0.200

0.201 - 0.400

0.401 - 0.500

0.501 - 0.600

0.601 - 0.675

World BankNATIONAL STATISTICAL COORDINATION BOARDNATIONAL STATISTICAL COORDINATION BOARD

Municipal Poverty Incidence0.03 - 0.25

0.26 - 0.50

0.51 - 0.60

0.61 - 0.70

0.71 - 0.90

Region


16/61

SAE - Nepal


17/61

SAE - Cambodia


18/61

Data Framework

Small-area EstimationMicrosimulationMass Imputation

All three techniques can be viewed as solutions to amissing data problem

SAE and Microsimulation The unsurveyedobservations for the variable of interest are missing.

Mass Imputation - random observations and in somecases complete records are missing


19/61

Models

Small area estimation works by borrowing strength fromthe relationship between observations via a statisticalmodel.

For example, in the World Bank (Elbers, Lanjouw &Lanjouw, 2003) method, a regression model is fitted to

survey data, and then the model used as a predictor usingcensus data.

Other methods (eg Rao, 2003) use only survey data.


20/61

Combining Survey and Census

Regression

Census

X

ij ij i ijY X h e

Expenditure ppKcal paeHeight-for-age

Weight-for-age

Survey

Y, X


21/61

What is X ?- X is a design or model matrix- Matrix manipulations can be used for any X- In practice we need to decide which variables to include in X- In the World Bank small area estimation methodology, these

variables need to be in both the survey and the census- Examples include:

household size, percentage of children in various age ranges, percentage adult men, ethnic group, religion, whether family membersabroad, household head female, education level of household, rooftype, wall type, floor type, rural / urban, size of land ownership if rural,house ownership, cooking fuel type, source of water, toilet type if any,availability of electricity, mortality of children.


22/61

But measuring X is difficultRoof type in Cambodia

Bamboo / thatch / grassTilesWood / plywoodConcrete / brick / stoneGalvanised iron / aluminium / other

metal sheets Asbestos cement sheetsPlastic / synthetic material sheetsOther (specify)

Wall type in Cambodia

Bamboo / thatch / grass / reedsEarthWood / plywoodConcrete / brick / stoneGalvanised iron / aluminium /other metal sheets

Asbestos cement sheetsSalvaged / improvised materialsOther (specify)


23/61


24/61


25/61


26/61


27/61


28/61


29/61


30/61

Education, Literacy & Occupation


31/61

Cambodia Model for Stunting(Standardised height-for-age from CAS2008)


32/61

Pakistan Feasibility Phase Available data sources:

SURVEY Abbrev YEAR Coverage

National Nutrition Sur vey of Pakist an NNS 2011 Pakistan

Demograp hic Health Survey DHS -Pakistan 2006-7 Pakistan

Pakistan Demographic Survey 2007 Pakistan excluding FA TA& Militaryrestricted areas ofNWFP.

Pakistan Social & Living Sta ndards Measure ment Sur vey PSLM 2007-8 Pakistan

2010-11 Pakistan

Household Income & Expendit ure Survey HIES 2007-8 Pakistan

Note: HIES is part of PSLM HIES 2010-11 Pakistan

Benazir Income Support Progr amme BISP 2008-201 2 Pakistan

Multiple Indicato r Cluster Survey MICS 2010 Balochistan

2009 FATA2007-8 Punjab2007-8 Azad Jammu & Kashmir2008 Khyber Pakht unkhwa

Pakistan Economic Survey PES 2011-12 Pakistan


33/61

Pakistan Feasibility PhaseSuitable recent surveys:

Expenditure povertyHousehold Income and Expenditure Survey 2010-11Food security; Stunting, underweight & wasting in children

National Nutrition Survey of Pakistan 2011

Census equivalentBenazir Income Support Programme

27,000,000 households; all households in all except three districts


34/61

Nepal Feasibility Study

Surveys: 2010/11 Nepal Living Standards Survey (NLSS-III) 2011 Nepal Demographics and Health Survey (DHS2011)

Census: 2011 Nepal National Population and Housing Census (NPHC2011)

Variables of interest: Food poverty (prevalence, gap and severity, based on food expenditure) Malnutrition indicators in children under five years of age:

Stunting, Underweight Wasting. Undernourishment (kilocalorie deficiency in daily food intake) Prevalence of diarrhoea


35/61

Nepal NDHS Matching with Census


36/61

SAE - Benefits and Disadvantages

Using small area estimation is much less expensive thanincreasing the sample size in sample surveys. Forexample doubling the size of an important nationalsurvey may cost several million dollars, but the sameaccuracy can sometimes be achieved by analysing thedata using statistical models for much smaller cost.

However, the level of statistical modelling expertise

required to produce robust sound small area models isvery high.


37/61

Benefits

Cost: It is possible to produce acceptably accurate estimates (for example ofpoverty) without having to spend anywhere near so much

Such estimates can be produced even after the survey has been designed andrun.

It is often possible to produce sound small area estimates (via the small areamodel) even where there is no survey data in the small area.

Measures of accuracy for the small area estimates are possible (as well as theestimates of poverty).

Maps: It is possible to map the results, which often avoids having to look atpages of numbers


38/61

Caveats

Solution can seem an arcane art.

Very careful statistical modelling is required, and this requires experienced

mathematical statisticians rather than people who may know how to runsoftware.

The accuracy of small area estimates is dependent on the statistical model beingcorrect (and is otherwise underestimated).

The apparently most accurate estimates also use census data (as well as surveydata), are only currently possible when census and survey are run at thetime, and when both contain the same other variables which can be usedfor modelling (for the survey) and prediction (from the census)


39/61

Essential Requirements for

SAE of Poverty

Local knowledge

FundingComputer equipment and softwareData access / Official sanctionCleaned (and matched) dataStatistical modelling and programming

expertiseGIS mapping expertise


40/61

Validation

Always worth visiting places to check any small area resultsthat appear unusual or anomolous

Getting local experts to rank small areas in terms of poverty

and then asking then to discuss any differences betweenthese and the small area estimates can be a usefultechnique.


41/61

Supplementary material


42/61

Modelling

Regression

Survey

Y, X

Census

X

ij ij i ijY X h e

Clusters:psu, ea, village


43/61

ModellingUse survey data to fit (using survey regression)

ijijij u X Y

Decompose residual variationinto two (or three) levels

ijiij ehu

(selecting the X variables is difficult)

Diagnostics: R 2 and22 /uh


44/61

Nepal Feasibility Study


45/61

ModellingModel heteroscedasticity (if any)

ijijij

ij r Z e A

e

2

2

ln

From regressions save: ]

V[, *ij,, ehi

2]

r ,V[,,

Use to estimate e, ij and e *ij = e ij / e, ij (formula too horrible to include)


46/61

Models for Variance

Variance model for ln(kilocalories)

p = 6, R 2 = 0.011

Variable Coef. Std. Err. t P>t Labelnlvst4 0.2356 0.1026 2.3 0.022 Rural and 1-2 livestockhethn5 0.2294 0.0889 2.58 0.010 HH head Hill Janajatisnagar7 -0.2100 0.1034 -2.03 0.043 Rural and agri area 0.5-1.0Hasamen -0.5032 0.2204 -2.28 0.023 %adult menpnnepcv 5.3950 2.6568 2.03 0.043 % with non-Nepali citizenship, vdcstdht 0.6519 0.2332 2.79 0.006 SD height of vdc in km

_cons -4.7187 0.0842 -56.06 0.000 Constant


47/61

PredictionThe predicted Y for X ij in the census data is:

ijij X Y

To allow for statistical uncertainty we produce a lot of possiblepredicted values:

b

ij

b

i

b

ij

b

ij eh X Y

for b = 1, , B (maybe 100)


48/61

Prediction

]

[

V b ,Normalfrom

distnempirical-from }

,,

{ 1 nbi hhh

][ V b ,Normalfrom

}

,

{ **1 iinib

ij eee from*


49/61

Small-Area EstimatesFood poverty measure P R for region R :

)(1

Z Y Z

Y Z

N P ij

Rij

ij

R

R

Ind

Calculate P R b for each b

Calculate the mean and sd of the P R b

)se(,,Output R R P P R


50/61

Is 5% good enough?

Ranking of areasProbability of highneed: Z=(P 0-P*)/se 0

PrecisionPlot of standard errors against municipal size

Poverty estimates and their standard errors

Incidence

P0 se0

Mean 0.4549 0.0428

SD 0.1689 0.0143

Min 0.0274 0.0050 Max 0.8968 0.1973


51/61

CorrelationsP0 K0 S2 U2 W2

P0 1

K0 0.4659 1

S2 0.5792 0.4609 1

U2

0.3954

0.0324

0.5468

1

W2 -0.1235 -0.3590 -0.1123 0.6658 1

. 2

. 4

. 6

. 8

K 0

0 .05 .1 .15 .2 .25W2

Mts Hills Terai


52/61


53/61

Institute of Fundamental Sciences - Statistics, Massey University, New Zealand

For height-for-age, weight-for-age, weight-for-height it is possibleto get small standard errors at small area level, even thoughpredictive models at child level have comparatively low R-square .

Low R-square values are acceptable, if unexplained variation is randomfor households or individuals, with little or no cluster-level variation.

However some of this variation represents missing variables in themodel which would give better prediction if they were available. Ifimportant factors are missing then the small-area estimates obtainedwill not reflect the true variability in these malnutrition indicators.

Technical Conclusions


54/61


In choosing appropriate regression models,caution is needed.Separate survey based models for subgroups and selecting variables

automatically from a very large pool of possibilities is not recommended.Model-fitting criteria such as adjusted R-square or AIC penalize for fitting too

many variables, but do not account for the number of variables that arebeing selected from. Cross-validation can be useful.We have in general fitted a single model for the whole population, by looking

carefully at whether the model makes sense. This is a time-consumingprocedure but can lead to more stable parameter estimation and morereliable prediction.

Even a single model can produce very different area estimates at area level.Subgroup or area effects (eg urban/rural) are included in models when

required, and this adjusts for local features in the data without fittingseparate models.

If there is prior knowledge on which factors are likely to affect the targetvariable, this can be built into the model selection. Formal Bayesiananalysis may be useful.



55/61


Using specialized survey regression routines (eg Stata, Sudaan,WesVar) in the initial survey model fitting has distinct advantages,since it incorporates the entire survey design and gives a consistentestimate of the covariance matrix.

These routines use robust estimation, collapsing the covariance matrix overclusters.

Such methods are more stable than estimating a covariance within eachcluster. Robust methods may give poor estimates for small subpopulationswith few clusters, but this is a real effect, not an artefact of the fittingprocedure.

These routines require all survey data to be included in any analysis (even of asubpopulation) to give unbiased standard errors. Hence analysis of sub-setsof survey data is not recommended, even if different models are being fitted

to different subgroups.The weighting of the survey observations is complicated not only because ofthe survey design but also because the target variable is often a per capitaaverage.

When individual data are used, robust variance estimates are still validbecause they only assume independence between clusters, not ofobservations within clusters.



56/61


To allow for non-independence between children inthe same household at the prediction stage, wehave extended the ELL approach to incorporatethree levels of variation.

While the estimation of variance components in such a hierarchicalmodel is now well-understood, the use of estimated random effects ina non-parametric bootstrap raises some theoretical issues, such asadjustment for degrees of freedom, which might provide fruitful areasfor further research.



57/61


ELL methodology is useful when there is interest in severalnonlinear functions of the same target variable, as in thecase here of six poverty measures defined on householdper capita expenditure, or in distributional properties.

For only one measure we could use direct modelling, eg small-areaestimates of poverty incidence could be derived using logisticregression.

If instead there are several highly correlated target variables, it mightincrease efficiency to use a multivariate model.

However the multivariate graphs raise interesting issues about the utilityof such multivariate models. Such techniques shrink estimates ofeach component toward one another, and it is sometimes the contrastrather than the combination of variables such as height-for age,weight-for-age, and weight-for-height that is important.



58/61


Technical ConclusionsThe provision of standard errors with the small-areaestimates is important because it gives the user anindication of how much accuracy is being claimed,conditional on the model being correct.Ultimately decisions need to be made on which areas should receive the mostdevelopment assistance, so it is important that this information be given tousers in a way that is most useful for this purpose.It is not clear exactly how the standard error information is best incorporated,but this is partly because the answer will depend on the parameters of thedecision problem.We have explored a possible way of incorporating the standard errors into apoverty map, first calculating standardized departures from a pre-specifiedincidence level, say 40%, as z=(estimate-0.4) / standard error, and thencalculating the probability based on a normal distribution. This can be mappedand interpreted as the probability that the corresponding area has a povertyincidence at least as high as the pre-chosen level. Thus when targetingassistance we could focus on those areas which we believe have the greatestchance of exceeding a threshold poverty incidence. (Some caution is requiredif the population sizes in the areas differ markedly.)



59/61


From a technical perspective, although sound, the statisticalmethods used would benefit from further theoreticaldevelopment and justification. We are currentlyundertaking some such research.

The range of models that use small-area estimation is very broad.ELL methodology has a number of theoretical and practical advantages,

but sensitivity of estimates to different small-area estimation modelsremains an only partially explored issue. This question relates both tothe choice of the ELL method, vis--vis others, and to the choice ofexplanatory variables within models (eg sub-models for differentareas, cross-validation of variables selected from a large poolincluding higher level interactions, consistency of sign and magnitudeof parameter estimates with likely influence on poverty in thepresence of correlated variables). These questions need theoreticalwork and extend beyond the present study.



60/61


Validation of small-area estimates by visits to selected smallareas after small-area estimates have been derived fromfitted models can be a useful exercise, but caution iswarranted.

Small area estimation works best in aggregate - not every small-area estimatewill have precise information, so that choosing areas to visit on the basis ofpossible anomalies can give a biased picture of the utility of the estimates asa whole.

It is difficult to ask validation participants to differentiate types of poverty or not toinclude aspects (such as health or water quality) which because they are notincluded in the census variables cannot be part of the small area estimatesthemselves.

Validation exercises are usually limited by funds, so formal testing of accuracy ofsmall area estimates is not possible by this method.

Nevertheless, validation can provide useful insights and even more importantly aforum for discussion of results of poverty mapping with local communities.



61/61

measuring nutrition, health and poverty in small areas – how low can you go?

Documents