measurement 17.871 spring 2003. topics in measurement from abstraction to measure sources of error...
Post on 20-Dec-2015
222 views
TRANSCRIPT
![Page 1: Measurement 17.871 Spring 2003. Topics in Measurement From abstraction to measure Sources of error What to do about error Practical ways to improve measurement](https://reader030.vdocuments.site/reader030/viewer/2022020111/56649d4c5503460f94a2b14c/html5/thumbnails/1.jpg)
Measurement
17.871
Spring 2003
![Page 2: Measurement 17.871 Spring 2003. Topics in Measurement From abstraction to measure Sources of error What to do about error Practical ways to improve measurement](https://reader030.vdocuments.site/reader030/viewer/2022020111/56649d4c5503460f94a2b14c/html5/thumbnails/2.jpg)
Topics in Measurement
• From abstraction to measure
• Sources of error
• What to do about error
• Practical ways to improve measurement
![Page 3: Measurement 17.871 Spring 2003. Topics in Measurement From abstraction to measure Sources of error What to do about error Practical ways to improve measurement](https://reader030.vdocuments.site/reader030/viewer/2022020111/56649d4c5503460f94a2b14c/html5/thumbnails/3.jpg)
The Mapping
X Y
x y
ex ey
Theory
Observation
![Page 4: Measurement 17.871 Spring 2003. Topics in Measurement From abstraction to measure Sources of error What to do about error Practical ways to improve measurement](https://reader030.vdocuments.site/reader030/viewer/2022020111/56649d4c5503460f94a2b14c/html5/thumbnails/4.jpg)
Mapping from the Abstract to the Measurement
• Some abstract things we try to measure– Alienation– Moral decay– Democracy– Party identification– Fear of defeat– Terrorism– Ideology
![Page 5: Measurement 17.871 Spring 2003. Topics in Measurement From abstraction to measure Sources of error What to do about error Practical ways to improve measurement](https://reader030.vdocuments.site/reader030/viewer/2022020111/56649d4c5503460f94a2b14c/html5/thumbnails/5.jpg)
Sources of Error in Measurement(Mosteller)
• Conceptual or design error
• Bad breaks in random sampling
• Survey question wording
• Non-random out-selection
• Transcription errors
• Calculation & mechanization errors
![Page 6: Measurement 17.871 Spring 2003. Topics in Measurement From abstraction to measure Sources of error What to do about error Practical ways to improve measurement](https://reader030.vdocuments.site/reader030/viewer/2022020111/56649d4c5503460f94a2b14c/html5/thumbnails/6.jpg)
What to Do About Error
• Practice safe data– Know where your data come from– Watch for anomalies– Use multiple measurement techniques– Collect as much data as possible and
disaggregate
![Page 7: Measurement 17.871 Spring 2003. Topics in Measurement From abstraction to measure Sources of error What to do about error Practical ways to improve measurement](https://reader030.vdocuments.site/reader030/viewer/2022020111/56649d4c5503460f94a2b14c/html5/thumbnails/7.jpg)
Practical things to do about measurement
• Distinction between a measure and an indicator– Measure: straightforward quantification of a
variable of interest– Indicator: quantification of a variable that is
believed (or known) to be highly correlated with the “real” variable of interest
![Page 8: Measurement 17.871 Spring 2003. Topics in Measurement From abstraction to measure Sources of error What to do about error Practical ways to improve measurement](https://reader030.vdocuments.site/reader030/viewer/2022020111/56649d4c5503460f94a2b14c/html5/thumbnails/8.jpg)
Examples of Measures
• Income in $$• Age in years• Votes• Number of wars (hmmmmmm…..)• Campaign contributions
• Measurement issues tend to focus on the quality of the data-gathering method, especially sampling
![Page 9: Measurement 17.871 Spring 2003. Topics in Measurement From abstraction to measure Sources of error What to do about error Practical ways to improve measurement](https://reader030.vdocuments.site/reader030/viewer/2022020111/56649d4c5503460f94a2b14c/html5/thumbnails/9.jpg)
Examples of Indicators
• Public opinion– Party identification– Trust in government– Ideology
• Economics– Gross domestic (national) product
• Characterizations of political systems– Freedom– Transparency– Democracy
![Page 10: Measurement 17.871 Spring 2003. Topics in Measurement From abstraction to measure Sources of error What to do about error Practical ways to improve measurement](https://reader030.vdocuments.site/reader030/viewer/2022020111/56649d4c5503460f94a2b14c/html5/thumbnails/10.jpg)
How do we generate indicators?
• Trust others and hope for the best– GDP, etc.– Presidential approval
• Use a single measure and hope for the best (or convince ourselves it’s OK)– 7-point ideology scale– 7-point party identification scale– Codings of “wars” or “rally events”– Bulletin of the Atomic Scientist “nuclear clock”
• Use multiple measures creatively
![Page 11: Measurement 17.871 Spring 2003. Topics in Measurement From abstraction to measure Sources of error What to do about error Practical ways to improve measurement](https://reader030.vdocuments.site/reader030/viewer/2022020111/56649d4c5503460f94a2b14c/html5/thumbnails/11.jpg)
Multiple-measure indicators• “Trust in government” battery
– How much of the time do you think you can trust the government in Washington to do what is right?
– Would you say the government is pretty much run by a few big interests looking out for themselves or that it is run for the benefit of all the people?
– Do you think that people in the government waste a lot of money we pay in taxes, waste some of it, or don’t waste very much of it?
– Do you think that quite a few of the people running the government crooked, not very many are, or do you think hardly any of them are crooked?
• External political efficacy battery– Sometimes politics and government seem so complicated that a
person like me can’t really understand what’s going on– People like me don’t have any say about what the government
does– Public officials don’t care much what people like me think.
![Page 12: Measurement 17.871 Spring 2003. Topics in Measurement From abstraction to measure Sources of error What to do about error Practical ways to improve measurement](https://reader030.vdocuments.site/reader030/viewer/2022020111/56649d4c5503460f94a2b14c/html5/thumbnails/12.jpg)
Trust and Efficacy over Time
![Page 13: Measurement 17.871 Spring 2003. Topics in Measurement From abstraction to measure Sources of error What to do about error Practical ways to improve measurement](https://reader030.vdocuments.site/reader030/viewer/2022020111/56649d4c5503460f94a2b14c/html5/thumbnails/13.jpg)
Other Multiple-Variable Indicators
• Americans for Democratic Action Support Scores
• Freedom House freedom assessment
• MIT Teaching Quality
• Transparency International’s “Corruption Perceptions Index”
• DNominate Scores
![Page 14: Measurement 17.871 Spring 2003. Topics in Measurement From abstraction to measure Sources of error What to do about error Practical ways to improve measurement](https://reader030.vdocuments.site/reader030/viewer/2022020111/56649d4c5503460f94a2b14c/html5/thumbnails/14.jpg)
ScalingOr, How to Create Multiple Indicators More Generally
![Page 15: Measurement 17.871 Spring 2003. Topics in Measurement From abstraction to measure Sources of error What to do about error Practical ways to improve measurement](https://reader030.vdocuments.site/reader030/viewer/2022020111/56649d4c5503460f94a2b14c/html5/thumbnails/15.jpg)
Imagine an unobserved factor that gives rise to observable factors
• Democracy• Healthiness• Racism• Conservatism• Religiosity
Unobservedfactor
Measure 1
Measure 2
Measure n
. . .
b1
b2
bn
e1
e2
e3
Polyarchy
Fairness ofelections
Meaningfulnessof elections
Freedom of expression
b1
b2
b4
e1
e2
e3
Unbiased reporting ofofficial pronouncements
e4
b3
![Page 16: Measurement 17.871 Spring 2003. Topics in Measurement From abstraction to measure Sources of error What to do about error Practical ways to improve measurement](https://reader030.vdocuments.site/reader030/viewer/2022020111/56649d4c5503460f94a2b14c/html5/thumbnails/16.jpg)
Two-factor (pure) polyarchy example
Polyarchy Polyarchy
Fai
rnes
s of
ele
ctio
ns
Fre
edom
of
expr
essi
onA B DC FE G H I J
![Page 17: Measurement 17.871 Spring 2003. Topics in Measurement From abstraction to measure Sources of error What to do about error Practical ways to improve measurement](https://reader030.vdocuments.site/reader030/viewer/2022020111/56649d4c5503460f94a2b14c/html5/thumbnails/17.jpg)
What We Observe in the Pure Case
Fai
rnes
s of
ele
ctio
ns
Freedom of expression
![Page 18: Measurement 17.871 Spring 2003. Topics in Measurement From abstraction to measure Sources of error What to do about error Practical ways to improve measurement](https://reader030.vdocuments.site/reader030/viewer/2022020111/56649d4c5503460f94a2b14c/html5/thumbnails/18.jpg)
Two-factor (impure) polyarchy example
Polyarchy Polyarchy
Fai
rnes
s of
ele
ctio
ns
Fre
edom
of
expr
essi
onA B DC FE G H I J
![Page 19: Measurement 17.871 Spring 2003. Topics in Measurement From abstraction to measure Sources of error What to do about error Practical ways to improve measurement](https://reader030.vdocuments.site/reader030/viewer/2022020111/56649d4c5503460f94a2b14c/html5/thumbnails/19.jpg)
What We Observe in the Impure Case
Polyarchy
Fai
rnes
s of
ele
ctio
ns
![Page 20: Measurement 17.871 Spring 2003. Topics in Measurement From abstraction to measure Sources of error What to do about error Practical ways to improve measurement](https://reader030.vdocuments.site/reader030/viewer/2022020111/56649d4c5503460f94a2b14c/html5/thumbnails/20.jpg)
fair
elt
frexp.5 3.5
.5
3.5
USSR,Libya, Iran,China
Congo, LebanonHungary
Bhutan
Egypt, Sri Lanka,Mexico
Jamaica,Greece
Peru, India,Israel
Argentina,U.S., Finland
Source: http://www.nd.edu/~mcoppedg/crd/polydat.htm
![Page 21: Measurement 17.871 Spring 2003. Topics in Measurement From abstraction to measure Sources of error What to do about error Practical ways to improve measurement](https://reader030.vdocuments.site/reader030/viewer/2022020111/56649d4c5503460f94a2b14c/html5/thumbnails/21.jpg)
Simple way to create multiple-indicator scale with just 1 factor
SX X
sii
Ni t t
tt
T
1 1
( ),