Lord’s Lord’s ParadoxParadox

Analysis of Covariance vs. Analysis of Analysis of Covariance vs. Analysis of Difference ScoresDifference Scores

Lord, F. M. (1967). A paradox in the interpretation of group comparisons. PsychologicalBulletin, 68 (5). 304-305.

Inconsistency in significance testingInconsistency in significance testing

• In analyzing how subjects change, analysis of In analyzing how subjects change, analysis of covariance, and analysis of difference scores yield covariance, and analysis of difference scores yield different resultsdifferent results

• The same data sets, analyzed for the same The same data sets, analyzed for the same effects, using different techniques yielded effects, using different techniques yielded qualitatively different resultsqualitatively different results

Lord’s example A test of sex differences in the “Freshman

15”

Analysis of difference scores for males and females shows no difference

Analysis of covariance shows a significant difference

So, what is going on?

-Distributions on top = initial weight distributions-Distributions on side = final weight distributions-Ovals = scatter plots of male and female weights

““with the data usually available for with the data usually available for such studies, there simply is no such studies, there simply is no

logical or statistical procedure that logical or statistical procedure that can be counted on to make proper can be counted on to make proper allowances for uncontrolled pre-allowances for uncontrolled pre-

existing conditions between existing conditions between groups”groups”

-Frederic Lord-Frederic Lord

Different questions = Different answers

ANCOVA “asks” how can one variable predict variation in another (in our example how can weight at the beginning be used to predict weight at the end)

Analysis of difference scores “asks” how do initial scores differ from final scores

Pike, G. R. (2004). Lord’s paradox and the assessment of change during college. Journal of College Student Development, 45 (3). 348-353.

Take Home Message

When evaluating differences within individuals, there may not be a consistently best way

Think of the question that you want to ask

Think of what question the statistical tools can ask

Be careful in interpreting your results

Thanks!