multidirectional survey measurement errors: the latent class mtmm model
TRANSCRIPT
Multidirectional errors Latent class MTMM Conclusions
Multidirectional survey measurement errors:the latent class MTMM model
Daniel Oberski / [email protected]
Department of methodology & statisticsAAPOR 2015
Latent class MTMM Daniel Oberski / [email protected]
Multidirectional errors Latent class MTMM Conclusions
..1 Multidirectional survey measurement errors
..2 Latent class multitrait-multimethod model
..3 Conclusions
Latent class MTMM Daniel Oberski / [email protected]
Multidirectional errors Latent class MTMM Conclusions
Stochastic errorsTwo respondents who:
• Went to the doctor the same number oftimes
• Have the same opinion about the role ofwomen in society
give different answers to these questions.
This will bias estimates of relationshipsbetween the variables.
Latent class MTMM Daniel Oberski / [email protected]
Multidirectional errors Latent class MTMM Conclusions
Stochastic errors• Latent variables can never be recovered
• But latent variable models can recover theamount of influence they exert
• This is useful to remove bias
• Multitrait-multimethod(MTMM)approach to estimating this influence(Andrews 1984; Saris & Andrews 1991; Saris &Gallhofer 2007);
• Quasi-simplex approach (Wiley & Wiley 1970;Alwin 2007).
↑ Linearmodels
Latent class MTMM Daniel Oberski / [email protected]
Multidirectional errors Latent class MTMM Conclusions
Stochastic errors• Latent variables can never be recovered• But latent variable models can recover the
amount of influence they exert
• This is useful to remove bias
• Multitrait-multimethod(MTMM)approach to estimating this influence(Andrews 1984; Saris & Andrews 1991; Saris &Gallhofer 2007);
• Quasi-simplex approach (Wiley & Wiley 1970;Alwin 2007).
↑ Linearmodels
Latent class MTMM Daniel Oberski / [email protected]
Multidirectional errors Latent class MTMM Conclusions
Stochastic errors• Latent variables can never be recovered• But latent variable models can recover the
amount of influence they exert• This is useful to remove bias
• Multitrait-multimethod(MTMM)approach to estimating this influence(Andrews 1984; Saris & Andrews 1991; Saris &Gallhofer 2007);
• Quasi-simplex approach (Wiley & Wiley 1970;Alwin 2007).
↑ Linearmodels
Latent class MTMM Daniel Oberski / [email protected]
Multidirectional errors Latent class MTMM Conclusions
Stochastic errors• Latent variables can never be recovered• But latent variable models can recover the
amount of influence they exert• This is useful to remove bias
• Multitrait-multimethod(MTMM)approach to estimating this influence(Andrews 1984; Saris & Andrews 1991; Saris &Gallhofer 2007);
• Quasi-simplex approach (Wiley & Wiley 1970;Alwin 2007).
↑ Linearmodels
Latent class MTMM Daniel Oberski / [email protected]
Multidirectional errors Latent class MTMM Conclusions
Stochastic errors• Latent variables can never be recovered• But latent variable models can recover the
amount of influence they exert• This is useful to remove bias
• Multitrait-multimethod(MTMM)approach to estimating this influence(Andrews 1984; Saris & Andrews 1991; Saris &Gallhofer 2007);
• Quasi-simplex approach (Wiley & Wiley 1970;Alwin 2007).
↑ Linearmodels
Latent class MTMM Daniel Oberski / [email protected]
Multidirectional errors Latent class MTMM Conclusions
Stochastic errors• Latent variables can never be recovered• But latent variable models can recover the
amount of influence they exert• This is useful to remove bias
• Multitrait-multimethod(MTMM)approach to estimating this influence(Andrews 1984; Saris & Andrews 1991; Saris &Gallhofer 2007);
• Quasi-simplex approach (Wiley & Wiley 1970;Alwin 2007).
↑ Linearmodels
Latent class MTMM Daniel Oberski / [email protected]
Multidirectional errors Latent class MTMM Conclusions
Stochastic errors• Latent variables can never be recovered• But latent variable models can recover the
amount of influence they exert• This is useful to remove bias
• Multitrait-multimethod(MTMM)approach to estimating this influence(Andrews 1984; Saris & Andrews 1991; Saris &Gallhofer 2007);
• Quasi-simplex approach (Wiley & Wiley 1970;Alwin 2007).
↑ Linearmodels
Latent class MTMM Daniel Oberski / [email protected]
Multidirectional errors Latent class MTMM Conclusions
Stochastic errors• Latent variables can never be recovered• But latent variable models can recover the
amount of influence they exert• This is useful to remove bias
• Multitrait-multimethod(MTMM)approach to estimating this influence(Andrews 1984; Saris & Andrews 1991; Saris &Gallhofer 2007);
• Quasi-simplex approach (Wiley & Wiley 1970;Alwin 2007).
↑ Linearmodels
Latent class MTMM Daniel Oberski / [email protected]
Multidirectional errors Latent class MTMM Conclusions
Stochastic errors in the literature
• Random mistakes• Acquiescence• Answering in the socially desirable direction• Tending to choose the first/last of several categories• Extreme response (outer categories)• Avoiding some particular category for whatever reason• Preferring the midpoint• Heaping (rounding)• ...
Problem: Not all of these are linearStochastic errors are multidirectional
Latent class MTMM Daniel Oberski / [email protected]
Multidirectional errors Latent class MTMM Conclusions
Example of the effect of a nonlinear error on estimate ofrelationships
True relationship1 2 3 4
1 136 112 91 752 112 75 50 343 91 50 28 154 75 34 15 7
Polychoric correlation = -0.25
Relationship with ERS1 2 3 4
1 374 41 34 2052 41 4 3 123 34 3 1 64 205 12 6 19
Polychoric correlation = -0.43
Latent class MTMM Daniel Oberski / [email protected]
Multidirectional errors Latent class MTMM Conclusions
Problem: Stochastic errors can strongly bias relationshipestimates;
Problem: Linear latent variable models assume errors are allone-way, so bias is not appropriately removed.
Solution: Latent class models to allow for multidirectional errors.Here MTMM but could also be quasi-simplex
Latent class MTMM Daniel Oberski / [email protected]
Multidirectional errors Latent class MTMM Conclusions
The latent class MTMM modelOberski, Hagenaars & Saris, to appear in Psychological Methods.
Latent class MTMM Daniel Oberski / [email protected]
Multidirectional errors Latent class MTMM Conclusions
M1 M2 M3
T1 T2 T3
y11 y21 y31 y12 y22 y32 y13 y23 y33
• Latent variables are discrete (categorical)• Observed may be continuous or discrete• Relationships (can be) nonparametric
Latent class MTMM Daniel Oberski / [email protected]
Multidirectional errors Latent class MTMM Conclusions
Experimental design: split-ballot MTMM
Method 1 Method 2 Method 3Random group 1 . .Random group 2 . .
Latent class MTMM Daniel Oberski / [email protected]
Multidirectional errors Latent class MTMM Conclusions
Opinion about the role of women experiment: Mainquestionnaire (first method)
Latent class MTMM Daniel Oberski / [email protected]
Multidirectional errors Latent class MTMM Conclusions
Opinion about the role of women experiment:Supplementary group 1 (second method)
Latent class MTMM Daniel Oberski / [email protected]
Multidirectional errors Latent class MTMM Conclusions
Opinion about the role of women experiment:Supplementary group 2 (third method)
Latent class MTMM Daniel Oberski / [email protected]
Multidirectional errors Latent class MTMM Conclusions
Effect of trait on item distribution, Greece
Men more right, positive agree-disgree
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
Latent class MTMM Daniel Oberski / [email protected]
Multidirectional errors Latent class MTMM Conclusions
Effect of trait on item distribution, Slovenia
Men more right, negative agree-disgree
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
Latent class MTMM Daniel Oberski / [email protected]
Multidirectional errors Latent class MTMM Conclusions
Method factors have a nonlinear influence on the items
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
Cut down, method 1
Trait
Me
tho
d
1
234
5
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
Cut down, method 2
Trait
Me
tho
d
1
2 34
5
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
Cut down, method 3
Trait
Me
tho
d
1
23
4
5
Latent class MTMM Daniel Oberski / [email protected]
Multidirectional errors Latent class MTMM Conclusions
Prevalence of method behavior
Latent class MTMM Daniel Oberski / [email protected]
Multidirectional errors Latent class MTMM Conclusions
Conclusion
Latent class MTMM Daniel Oberski / [email protected]
Multidirectional errors Latent class MTMM Conclusions
Conclusion
• Stochastic errors are important when you are interested inrelationships;
• Need latent variable models to estimate their extent so theireffects can be removed statistically;
• In traditional MTMM and quasi-simplex models, these effectsare all one-way;
• Latent class models allow for multidirectional errors.• Example: the latentclassMTMM model
(Oberski et al., to appear - see http://daob.nl/publications)
Latent class MTMM Daniel Oberski / [email protected]
Multidirectional errors Latent class MTMM Conclusions
Thank you for your attention!
[email protected]@DanielOberski
See http://daob.nl/publications for preprints
Supported by Veni grant number 451-14-017 from the NetherlandsOrganization for Scientific Research (NWO).
Latent class MTMM Daniel Oberski / [email protected]