Chapter 7 & 8

Experimental Design&

Measurement of variables

Adnan Khurshid

Experiments

An experiment is defined as manipulating (changing values/situations) one or more independent variables to see how the dependent variable's) is/are affected, while also controlling the affects of additional extraneous variables.

Experimentation

Experimental design fall into to categories

1. Lab Experiments2. Field Experiments

Types of Experiments

Two broad classes:Laboratory experiments: those in which the

independent variable is manipulated and measures of the dependent variable are taken in a contrived, artificial setting for the purpose of controlling the many possible extraneous variables that may affect the dependent variable

Field experiments: those in which the independent variables are manipulated and measurements of the dependent variable are made on test units in their natural setting

Experimentation

Controlling the Nuisance Variables

1. Matching Group2. Randomization

Experimentation

Validity:The correctness or truthfulness of an inferenceInternal Validity:Refers to the accuracy of the inference that the IV caused the effect observed in the DVThe observed change in the dependent variable is, in fact, due to the independent variable (internal validity)

Experimentation

Field ExperimentsAn experiment done in the natural environment in which the

work goes on as usual, but the treatment is given to one or more group.

In the field experiments it may not possible to control all the nuisance variables because members cannot be either randomly assigned to the group.

Any cause and effect relationship found in these conditions would have wider generalizability to the other similar production settings.

Experimentation

Threats to Internal ValidityThe Eight Classic Threats

• History• Maturation• Testing• Instrumentation• Statistical Regression• Selection• Subject Mortality• Selection Interactions

Experimentation

History

Factors occurring external to the research situation that may appreciably influence the dependent variable,

For example, if researchers measure hyperactivity in the morning for the control group and in the afternoon for the experimental group, the time of day could produce group differences that are unrelated to the independent variable (medication).

Sales Promotion ---------------------- Sales Dairy Farmers Advertisement

Experimentation

MaturationFactors occurring within research subjects over time that

account for changes in the dependent variable, e.g., ill health, fatigue, depression.

Independent Variable Dependent variableEnhanced Technology --Efficiency Increases Getting Experience and doing Job faster

Experimentation

Testing

Testing threat is a threat to internal validity produced by a previous administration of the same test or other measure.

For example, if rats practiced navigating a milk maze in a pretest condition and then performed the same task in the posttest condition after receiving a drug, learning could reduce the time required to complete the maze, independent of drug effects.

Experimentation

InstrumentationInstrumentation threat is a threat to internal

validity produced by changes in the measurement instrument itself.

For example,if a scale did not return to zero after each subject

was weighed, group differences in weight could be due to changes in the scale that are unrelated to the independent variable (exercise program).

Experimentation

Statistical RegressionStatistical regression threat is a threat to internal validity that

can occur when subjects are assigned to conditions on the basis of extreme scores on a test. During retest, the scores of extreme scorers tend to regress toward the mean even without treatment.

For example, students with extremely high scores in one basketball game may

obtain less extreme scores during a second game without any real change in ability Conceptually, the initial extremely high basket ball score was raised by measurement error (representing the variability across games). When this changed randomly during the next game, high scores were no longer boosted as much as before. This resulted in a regression to the mean.

Experimentation

SelectionSelection threat is a threat to internal validity that can

occur when nonrandom procedures are used to assign subjects to conditions or when random assignment fails to balance out differences among subjects across the different conditions of the experiment.

For example, if a social psychologist assigned the first 20 student volunteers to an experimental condition and the next 20 to a control condition, these groups could differ on subject variables that could affect the dependent variable

Experiments

Subject MortalitySubject mortality threat is a threat to internal

validity produced by differences in dropoutrates across the conditions of the experiment.For example, if dropout rates are different across

three different psychotherapy conditions, this could create group differences on subject variables that could affect clinical outcome, independent of the independent variable.

Thus the morality can also lower the internal validity of the experiment.

Experimental Design

• An experimental design is a procedure for devising an experimental setting such that a change in the dependent variable may be solely attributed to a change in an independent variable.

• Symbols of an experimental design:• O = measurement of a dependent variable• X = manipulation, or change, of an independent

variable• R = random assignment of subjects to

experimental and control groups• E = experimental effect

Experimental Design

• After-Only Design: X O1

• One-Group, Before-After Design: O1 X O2

• Before-After with Control Group:• Experimental group: O1 X O2

• Control group: O3 O4

• Where E = (O2 – O1) – (O4 – O3)

How Valid Are Experiments?

• An experiment is valid if:• the observed change in the dependent variable

is, in fact, due to the independent variable (internal validity)

• if the results of the experiment apply to the “real world” outside the experimental setting (external validity)

From Concepts to Observations

A concept is a mental image that summarizes a set of similar observations, feelings, or ideas.

Operationalization is the process of connecting concepts to observations.

• The goal is to devise operations that measure the concepts we intend to measure—in other words to achieve measurement validity.

• Researchers develop an operational definition including:– what is measured– how the indicators are measured– the rules used to assign value to what is observed

Types of Indicators

• Self-report (surveys, interviews, etc.)– Report on the past (retrospective)– Report on the present– Predict the future

• Direct observation• Scales/Indices

From Operationalization to Levels of Measurement

When we know a variable’s level of measurement, we can better understand how cases vary on that variable and so understand more fully what we have measured.

Levels of Measurement

The nominal (or categorical) level of measurement, which is qualitative, has no mathematical interpretation;

• The nominal level of measurement identifies variables whose values have no mathematical interpretation; they vary in kind or quality but not in amount.

In terms of the variable “Occupation”, you can say that a lawyer is not equal to a nurse, but you cannot say that the “lawyer” is “more occupational” or “less occupational” than the nurse.

Ordinal Measures

At this level, you specify only the order of the cases, in “greater than” and “less than” distinctions. A common ordinal measure used in social service agencies is client satisfaction. You may ask clients to indicate whether they are “very satisfied,” “satisfied,” dissatisfied,” or “very dissatisfied” with a particular service. A client who responds “very satisfied” is clearly more satisfied than a client who responds “dissatisfied” – but not twice as satisfied or 2 units more satisfied.

Interval Measures

• At the interval level of measurement, numbers represent fixed measurement units but have no absolute zero point.

4 °F -12 °F

Your text uses the example of temperatures measured with the Fahrenheit scale. The temperature can definitely go below zero, as indicated in this weather forecast

Ratio Measures

A ratio level of measurement represents fixed measuring units with an absolute zero point. Zero, in this situation, means absolutely no amount of whatever the variable indicates. On a ratio scale, 10 is two points higher than 8 and is also two times greater than 5. Ratio numbers can be added and subtracted, and because the numbers begin at an absolute zero point, they can also be multiplied and divided (so ratios can be formed between the numbers).

Primary Scales of Measurement

• Scale Numbers• Nominal Assigned• to Runners

• Ordinal Rank Order• of Winners

• Interval Performance• Rating on a • 0 to 10 Scale

• Ratio Time to • Finish, in• Seconds

7 8 3

Thirdplace

Secondplace

Firstplace

13.314.115.2

9.69.18.2

Primary Scales of Measurement

Scale Basic Characteristics

Common Examples

Marketing Examples

Nominal Numbers identify & classify objects

Social Security nos., numbering of football players

Brand nos., store types

Percentages, mode

Chi-square, binomial test

Ordinal Nos. indicate the relative positions of objects but not the magnitude of differences between them

Quality rankings, rankings of teams in a tournament

Preference rankings, market position, social class

Percentile, median

Rank-order correlation, Friedman ANOVA

Ratio Zero point is fixed, ratios of scale values can be compared

Length, weight Age, sales, income, costs

Geometric mean, harmonic mean

Coefficient of variation

Permissible Statistics Descriptive Inferential

Interval Differences between objects

Temperature (Fahrenheit)

Attitudes, opinions, index

Range, mean, standard

Product-moment