Research: Conceptualization and Measurement• Conceptualization• Steps in measuring a variable• Operational definitions• Confounding• Criteria for measurement quality• Techniques of dealing with problems in

measurement• Ensuring reliability• Validity

• Face validity • Content validity• Criterion related validity• Construct validity 1

Conceptualization• We want to speak of abstract things:

• “Intelligence”

• “Ability to cope with stress”

• “Life satisfaction”

• “Happiness”

• We cannot research these things until we know exactly what they are.

• Everyday language often vague and unspecified meanings.

2

Conceptualization • Specify exactly what we mean and don’t mean

by the terms we use in our research.• No “true” (final) definitions of “the stuff of life”

• Conceptualization:• The process of identifying and clarifying concepts• We specify what we mean by using certain terms

• Indicators=the presence or absence of the concept we are studying. • These are often multi-dimensional • More than one specifiable aspect or facet

• What do we mean by happiness?3

Steps in reaching a measurement of a variable

• We all have conceptions of what we understand by compassion, prejudice, poverty, etc.

• People do not always agree about the meanings

• Begin by asking people to describe what they mean when they use certain terms such as “intelligence”

• Consult the EXPERTS• Literature review • Even the experts do not agree

• Coming to an agreement on what we understand is called conceptualization.• Result of this process is a concept • e.g., “prejudice”

4

Operational definitions

• Specifying exactly what we are going to observe, and how we will do it.

• Make the variable directly measurable

• Describe of the “operations” used to measure a concept

5

Examples• “Socio-economic Status” (SES):

• What was your total family income during the past 12 months?

• What is the highest level of school you completed?

• How would you operationalize “success at university?”

• If you operationalize badly, you end up not studying what you want (invalid operational definitions)• E.g., operationalizing “success in career” by looking only

at pay check 6

Confounding

• Intelligence tests require knowledge of the language in which they are given• Also measuring acquired language skills

• Juvenile delinquency can be defined in terms of convictions in court• But convictions are more frequent when they are

not legally represented - thus also measuring economic status

• Confounding = When operational definitions measure more than one thing

7

Criteria for measurement quality

Reliability• Does it yield the same result every time?

Stability over time.

• If I measure you now and again in half-an-hour, do I get the same reading?

• Maximum reliability depends on the construct – some constructs are unstable, e.g., heart rate.

• Single observers or raters8

Techniques of dealing with problems in measurement reliability

• Test-retest method – Make the same measurement more than once (external)

• Split-half method – Divide the instrument in two halves - Cronbach's alpha (internal)

9

Ensuring reliability

• Reliability suffers when respondents or researchers have to interpret

• Objective scales are always more reliable• Allow for little interpretation

• Using a fixed-response format helps• e.g. Multiple choice, Likert type response

formats • Researcher does not have to interpret what the

respondent meant 10

***Validity

• The extent to which an empirical measure adequately reflects the meaning of the concept under investigation

• For a scale: The degree to which it measures what it is supposed to measure

• Validity is divided into many types:• Content validity• Criterion-related validity• Construct validity• Face validity 11

Face validity

• How a measure conforms to our common agreements

• Examine the wording of the items • Submit items to expert judges

12

Content validity

• How much a measure covers every element of a concept.• Example: Measuring only the affective

aspect of love, but not the behavioral.

• Experts in a given domain generally judge content validity. • For example, the content of the SAT Subject

Tests™ is evaluated by committees made up of experts who ensure that each test covers content that matches all relevant subject matter in its academic discipline. 13

Criterion-related validity

• Sometimes called predictive validity• How a measure predicts performance on an

external criterion

• e.g., How ACT or SAT results predict academic success at university, as a way of saying they have validity

14

Construct validity• Closely tied to into the theoretical underpinnings of

the concept. • Variables ought to be related, theoretically, to other

variables. • This kind of validity is based on logical relationships

between variables• So: Does the instrument actually measure the concept

(or construct)?• e.g., Measure cranial circumference or brain weight

to measure intelligence

• Most difficult to achieve, most important – measures lacking in construct validity are almost useless 15

How to check for construct validity

• How can you show that a measurement truly measures what it claims to?

• How would you show that your depression scale has construct validity?

16

How to check

• 1. See how it relates to similar and dissimilar concepts:• Show that your depression scale relates

positively to similar concepts• e.g., People who score high on your

depression scale will have many sad thoughts

17

How to check

• 2. Show that your depression scale relates negatively to opposite concepts. Examples:

• People who score high on it will have very low energy levels

• Husbands who score high on a measure of marital satisfaction have fewer extra-marital affairs

18

Conclusion

• The more a scale or instrument has the qualities of reliability and validity, the better it is.

• Reliability and validity need to be sorted out before you run the study

19

Conclusion• Tension between validity and reliability

• Richness of meaning of concepts

• Operational definitions and measurements seem to rob concepts of their “richness of meaning”

• But the more conceptual variation and richness we allow in our study, the more opportunity for disagreement on how it applies in this situation

• Related to the tension between quantitative, structured techniques such as surveys, and qualitative, semi-structured methods such as interviews

20