KNR 295Measureme

ntSlide 1

Measurement Theory &

Construct Validity

Chapter 3

KNR 295Measureme

ntSlide 2

Measurement validity = construct validity

KNR 295Measureme

ntSlide 3

Construct Validity

KNR 295Measureme

ntSlide 4

Construct Validity

Determined by Operationalization

KNR 295Measureme

ntSlide 5

Construct Validity

So construct validity assesses how well your procedures/measures match your ideas/theories General = construct Specific = operationalization

KNR 295Measureme

ntSlide 6

Construct Validity

Two views – Definitionalist

“The construct,the whole construct, and nothing but the construct”

Impossible! Relativist

Define your construct Explain how and why what you’re doing

measures the construct in question Produce some evidence (whether we do this

depends on the kind of research in question)

US

KNR 295Measureme

ntSlide 7

Construct Validity

Translation validity Face validity Content validity

Criterion-related validity Predictive validity Concurrent validity Convergent validity Discriminant validity

KNR 295Measureme

ntSlide 8

Construct Validity

Translation validity vs. Criterion-related validity Translation validity assesses whether

the operationalization matches what you know of the construct

Criterion validity actually measures this assessment (uses other measures to assess the construct validity)

What we do is translation validity – arguing about the construct validity but not measuring it directly

KNR 295Measureme

ntSlide 9

Construct Validity

Translation validity Face validity:

Does it look like you got it right? Ask others…more objective

Content validity Good definition of the construct Good match between your measure and

the definition E.G. Fitness program – does it abide by

ACSM guidelines?(e.g. construct…Body fat. Measure…sum of

skinfolds)

KNR 295Measureme

ntSlide 10

Construct Validity

Criterion-related validity – all involve some direct test of CV Predictive validity

Does it predict what it ought to? E.G. Does sum of skinfolds predict

cardiovascular disease? Concurrent validity

Can your measure discriminate between 2 similar groups?

E.G. Measure sum of skinfolds of males and females – females should be higher than males(?)

KNR 295Measureme

ntSlide 11

Construct Validity

Criterion-related validity Convergent validity

Correlation between this operationalization and other similar ones

E.G. Sum of skinfolds and…BMI, underwater weighing, cadaver dissection…

Discriminant validity This operationalization is different from other stuff

that is not supposed to measure the same thing E.G. Sum of skinfolds vs. age, vs. weight, vs.

gender, and so on. Note – this is why these items are included in

popular equations converting sum of skinfolds to BF%

KNR 295Measureme

ntSlide 12

Construct validity

Convergent vs. Discriminant Validity

KNR 295Measureme

ntSlide 13

Construct validity

Convergent vs. Discriminant Validity Note – once you’ve done this, you still

need translation validity to establish that the measures are what you purport them to be

KNR 295Measureme

ntSlide 14

Threats to Construct Validity

The laundry list… Use this to “ask the right questions” about the studies you critique

& so to begin…

KNR 295Measureme

ntSlide 15

Threats to Construct Validity

Inadequate preoperational explication of constructs Construct not defined carefully

enough

KNR 295Measureme

ntSlide 16

Threats to Construct Validity

Mono-operation bias (independent variable) Only one example of the construct E.G. only one training program…but

there are many out there Mono-method bias (dependent

variable) Only one example of the construct E.G. only one strength measure for a

program that trained the whole body

KNR 295Measureme

ntSlide 17

Threats to Construct Validity

Interaction of different treatments Use one or more control groups to

isolate cause E.G. You want to show that strength

training improves self-esteem…but it could have been as a result of meeting you 3 times per week, not strength training…so use a control group to compare results

KNR 295Measureme

ntSlide 18

Threats to Construct Validity

Interaction of testing and treatment Imagine I was interested in whether

research methods improved reasoning skills… If I tested you every week on some IQ

tests, these become part of the treatment, and impair construct validity (you might be getting better because of the test, not because of being in research methods)

KNR 295Measureme

ntSlide 19

Threats to Construct Validity

Restricted generalizability across constructs “Unintended consequences” E.G. Finnish epidemiological study –

divide people into groups according to level of smoking & cholesterol

Take half of each group and assign to fitness & nutrition program, half untreated

All in fitness and nutrition program reduced smoking and improved fitness measures

BUT they also had increased all cause mortality!!! (may be an apocryphal story)

KNR 295Measureme

ntSlide 20

Threats to Construct Validity

Confounding constructs and levels of constructs “I’m finding out whether aerobics or

strength training has the greater impact on muscle tone” Ok, but what amount of strength training?

How often are they doing aerobics? The labels are not in sufficient detail Could be that only the particular versions

of these programs that you used will produce the results you found

KNR 295Measureme

ntSlide 21

Threats to Construct Validity

Social threats to construct validity Hypothesis guessing

I never collect data using people that have completed my motor learning class…

Evaluation apprehension Experimental booth in Bangor…

Experimenter expectancies I love my research…

KNR 295Measureme

ntSlide 22

Reliability and Levels of Measurement

We’ll leave those till we deal with conclusion validity

They are really the concern of a good stats course