welcome to fantasyland: comparing approaches to land area measurement in household surveys sydney...
TRANSCRIPT
Welcome to Fantasyland: Comparing Approaches to Land Area Measurement in Household SurveysSydney GourlaySurvey Specialist
Living Standards Measurement Study TeamDevelopment Research Group, The World Bank
2015 World Bank Conference on Land and Poverty
Motivation & Key Questions• Land area is critical in:
– Measuring yields– Land distribution– Titling schemes– Anything agriculture…
• Gaps in literature:– How do measurement
methodologies stack-up?• Is GPS as accurate as the gold-
standard?– What characteristics influence
measurement accuracy?– How does the measurement
methodology employed influence analytical outcomes?
Measuring Land Area: Methodological OptionsFarmer self-reported
estimate
PROS- Inexpensive
- Less missingness
CONS- Subjective
- Complicated by traditional units
-Potential ulterior motives
Compass and rope (aka traversing)
PROS-Traditional gold
standard for accuracy- Eliminates subjectivity
CONS- Time/labor intensive
(leading to higher costs)
- Requires travel to plot
GPS
PROS- Significantly quicker than traversing with
advantages of objective
measurement
CONS- Questions of
accuracy on small plots (?)
- Requires travel to plot
Remote Sensing (?)
PROS - Potential to
eliminate plot visits
CONS- Resolution limitations
- Feasibility of boundary
identification
LSMS Methodological Validation Program (MVP)• UK Aid-funded project: Improving Measurement of Agricultural
Productivity through Methodological Validation and Research
• To date, 3 highly-supervised methodological studies on land area measurement:
• Ethiopia• Tanzania• Nigeria
Methodological Validation ExperimentsLand and Soil Experimental
Research (LASER)
Improving the Measurement of
Cassava Productivity
Area Measurement
Validation
Location Ethiopia Zanzibar, Tanzania Nigeria
Components:Land Area;
Soil Testing; Crop Cutting
Land Area; Production Diaries;
Crop CuttingLand Area
Partnerships:
World Agroforestry
Centre (ICRAF), Central
Statistical Agency
Ministry of Agriculture and
Natural Resources, Office of the Chief
Government Statistician
National Bureau of Statistics
Fieldwork Complete
March 2014 July 2014 May 2013
1798Ethiopia
1945Tanzania
494Nigeria
4237Total
Plots with GPS and Compass & Rope
These studies included farmer self-reported estimate, GPS, and compass & rope measurement
Scope of (Preliminary) Analysis• Comparison of competing measurements:
– Bias: Difference between two measurements (in acres)• For example: GPS - CR
– Absolute Value of Bias• For example: |GPS – CR|
– Relative Bias: Difference between two measurements (in terms of %)• For example: (GPS- CR)/CR * 100%
– Absolute Value of Relative Bias• Analysis of systematic measurement error between measurements via OLS regression• Determinants of “High Bias” plots
Comparison of Measurements: Subjective vs Objective (1)SR estimates are sensitive to respondent characteristics, including the tendency to round off numbers:
0
10
20
30
Perc
en
t
0 1 2 3Acres
SR GPS
*Limited to plots <=3 acres
Plot Area Distribution
LSMS Malawi 2010/11
Comparison of Measurements: Subjective vs Objective (2)SR vs CRAcres
Level (CR)Ethiopia Tanzania Nigeria
N SR CR BiasMean Bias /
Mean CR N SR CR BiasMean Bias /
Mean CR N SR CR BiasMean Bias /
Mean CR1 (< 0.05 acres) 352 0.09 0.02 0.07 307% 44 0.32 0.04 0.28 661% - - - - -2 (< 0.15 acres) 392 0.27 0.09 0.18 188% 622 0.41 0.11 0.31 288% 21 0.15 0.11 0.03 30%3 (< 0.35 acres) 351 0.40 0.23 0.17 72% 816 0.62 0.23 0.39 173% 73 0.39 0.25 0.14 55%4 (< 0.75 acres) 316 0.66 0.51 0.15 29% 323 0.98 0.49 0.49 100% 129 0.79 0.53 0.26 50%5 (< 1.25 acres) 179 0.95 0.97 -0.02 -2% 63 1.53 0.92 0.61 66% 108 1.31 0.99 0.32 33%6 (>= 1.25 acres) 99 1.42 1.90 -0.47 -25% 20 2.05 1.81 0.24 13% 154 2.57 2.87 -0.30 -10%
Total 1689 0.47 0.38 0.09 23% 1888 0.65 0.27 0.38 143% 486 1.38 1.31 0.07 5%
Pooled data reveals self-reported estimates are over-estimated by 51% on average (over CR measurement)
Comparison of Measurements: Subjective vs Objective (3)0
2
4
6
0 2 4 6GPS
Ethiopia
0
2
4
6
CR
0 2 4 6SR
0
1
2
3
0 1 2 3 4GPS
Tanzania
0
1
2
3
CR
0 2 4 6 8SR
05101520
0 5 10 15 20GPS
Nigeria
05101520
CR
0 5 10 15 20SR
Ethiopia
Tanzania
Nigeria
Comparison of Measurements: Objective vs Objective (1)GPS vs CRMeans; Acres
Level (CR)
Ethiopia Tanzania Nigeria
GPS CR Bias
Mean Bias /
Mean CR GPS CR Bias
Mean Bias /
Mean CR GPS CR Bias
Mean Bias /
Mean CR1 (< 0.05 acres) 0.02 0.02 0.00 0% 0.04 0.04 0.00 -3% - - - -2 (< 0.15 acres) 0.10 0.09 0.00 2% 0.11 0.11 0.00 2% 0.11 0.11 -0.01 -7%3 (< 0.35 acres) 0.24 0.24 0.01 3% 0.23 0.23 0.01 2% 0.24 0.25 -0.01 -4%4 (< 0.75 acres) 0.52 0.51 0.01 2% 0.51 0.49 0.02 4% 0.52 0.53 -0.01 -2%5 (< 1.25 acres) 0.98 0.96 0.02 2% 0.94 0.92 0.02 2% 0.97 0.99 -0.02 -2%6 (>= 1.25 acres) 1.91 1.89 0.02 1% 1.91 1.81 0.09 5% 2.87 2.87 0.00 0%
Total 0.38 0.38 0.01 2% 0.28 0.27 0.01 3% 1.30 1.31 -0.01 -1%
Pooled data reveals GPS measurements are over-stated by 1% on average (over CR measurement)
Comparison of Measurements: Objective vs Objective (2)0
2
4
6
0 2 4 6GPS
Ethiopia
0
2
4
6
CR
0 2 4 6SR
0
1
2
3
0 1 2 3 4GPS
Tanzania
0
1
2
3
CR
0 2 4 6 8SR
05101520
0 5 10 15 20GPS
Nigeria
05101520
CR
0 5 10 15 20SR
Correlation between GPS and CR measurements: 0.997 (pooled data)
Comparison of Measurements: Objective vs Objective (3)Despite high correlation of GPS and CR measurements, evidence of “high bias” or problem plots exists:
High bias plots are generally:• smaller than average, • have a higher closing error (suggesting bias is at least partially attributable to CR), and • have a more complex shape (proxied by perimeter:area ratio).
Comparison of Measurements: Objective vs Objective (4)We attempt to explain the discrepancy between GPS and CR by:
Yi=Li+Ci+Si+SATi+Ti+Wi+ei
- Y is one of the four measures of bias, - L is the measure of the plot taken using CR, - C is the closing error of the CR measure, - S is a vector of proxies for the shape of the plot, - SAT is the number of satellites the GPS device had acquired, - T is a vector of dummy variables related to tree canopy cover, - W is a vector of dummy variables related to weather conditions, and - e is a random error.
Comparison of Measurements: Objective vs Objective (5)The small difference is difficult to capture, but:• Heavy tree cover & cloudy conditions slightly increase absolute value of percent deviation between measurements (in Ethiopia and Tanzania)• Perimeter : Area ratio is statistically significant in all countries, suggesting that in plots with more complex shapes the absolute value of percent bias is higher.• Plot area is the most significant factor…
• Plot area is the most significant factor…-.0
3-.
02
-.0
10
.01
GP
S-C
R (
ac
res
)
0 1 2 3P lot Size (acres, CR)
0.0
2.0
4.0
6.0
8.1
|GP
S-C
R|
(ac
res
)
0 1 2 3Plot Size (acres, CR)
-20
24
Re
lati
ve B
ias
(%)
0 1 2 3Plot S ize (acres, CR)
45
67
89
Ab
solu
te V
alu
e o
f R
ela
tive
Bia
s (
%)
0 1 2 3Plot Size (acres, C R)
Bias (acres)Ethiopia
| Bias | (acres) Relative Bias (%) | Relative Bias | (%)
0.0
5.1
.15
GP
S-C
R (
ac
res
)
0 1 2 3P lot Size (acres, CR)
0.0
5.1
.15
.2|G
PS
-CR
| (a
cre
s)
0 1 2 3Plot Size (acres, CR)
24
68
10
Re
lati
ve B
ias
(%)
0 1 2 3Plot S ize (acres, CR)
02
46
8A
bso
lute
Va
lue
of
Re
lativ
e B
ias
(%
)
0 1 2 3Plot Size (acres, C R)
Tanzania
-.06
-.05
-.04
-.03
-.02
-.01
GP
S-C
R (a
cres
)
0 2 4 6 8 10P lot Size (acres, CR)
0.1
.2.3
|GP
S-C
R| (
acr
es)
0 2 4 6 8 10Plot Size (acres, CR)
-10
12
3R
ela
tive
Bia
s (%
)
0 2 4 6 8 10Plot S ize (acres, CR)
-.1
-.05
0.0
5A
bso
lute
Va
lue
of R
elat
ive
Bias
(%)
0 2 4 6 8 10Plot Size (acres, C R)
Nigeria
Bias (acres) | Bias (acres)| Relative Bias (%) | Relative Bias (%) |
Comparison of Measurements: Objective vs Objective (7)
0
20
40
60
80
100
Ethiopia Tanzania Total
GPS CRPlot Size Level (CR) & MinutesAverage Measurement Duration
• Ethiopia:– GPS = 13.7 minutes– CR = 56.8 minutes
• Tanzania:– GPS = 7.4 minutes– CR = 29.3 minutes
Final Thoughts• Lower limit to GPS use not yet clearly defined(But lower than most literature alludes to)• GPS + SR
– When GPS measurements are missing, impute them using the self-reported area estimates a la Kilic et al:• Kilic, Zezza, Carletto, and Savastano. “Missing(ness) in Action: Selectivity Bias in GPS-Based Land Area Measurements.” World Bank Policy Research Working Paper No. 6490.
Welcome to Fantasyland: Comparing Approaches to Land Area Measurement in Household SurveysSydney GourlaySurvey Specialist
Living Standards Measurement Study TeamDevelopment Research Group, The World Bank
2015 World Bank Conference on Land and Poverty