Types of Variables

Fall 2014

Dr. Pedro L. Martinez

Educational Research and Assessment

Exploring Research Language

Construct - a label for an abstract trait or ability (e.g, intelligence, creativity) that is presumed to exist, it cannot be measured directly

Indicator- overt manisfestation(s) of a presumed trait and the operationalized definition

Operationalized definition- an empirical measure of the presumed indicator of a construct or state

Construct - a label for an abstract trait or ability (e.g, intelligence, creativity) that is presumed to exist, it cannot be measured directly

Indicator- overt manisfestation(s) of a presumed trait and the operationalized definition

Operationalized definition- an empirical measure of the presumed indicator of a construct or state

Construct

Operationalize these constructs:Operationalize these constructs:

School achievement Motivation Get along Self-concept

School achievement Motivation Get along Self-concept

When dealing with definitions of constructs…When dealing with definitions of constructs…

Constructs have to be empirically proven

What is the meaning of empirical? Is it an objective definition and

verifiable? Putting numbers next to the name. Operational definitions will

determine the validity of the study.  

Constructs have to be empirically proven

What is the meaning of empirical? Is it an objective definition and

verifiable? Putting numbers next to the name. Operational definitions will

determine the validity of the study.  

Defining VariablesDefining Variables

Variable-any dimension that has two or more values. Dimensions with changing values (gender)

Constant has only one value (female) What is the value of a constant?

-to control for extraneous factors. Variables are used to study relationships. Can a constant become a variable for another

researcher?

Variable-any dimension that has two or more values. Dimensions with changing values (gender)

Constant has only one value (female) What is the value of a constant?

-to control for extraneous factors. Variables are used to study relationships. Can a constant become a variable for another

researcher?

Types of VariablesTypes of Variables

Independent variable-the variable presumed to have a direct cause

If A then B hypothesis True Independent-randomized Quasi Independent-not randomized Dependent-(outcomes, criterion measures) is the

measured outcome the presumed effect of some cause.

Independent variable-the variable presumed to have a direct cause

If A then B hypothesis True Independent-randomized Quasi Independent-not randomized Dependent-(outcomes, criterion measures) is the

measured outcome the presumed effect of some cause.

Attribute-pre-existing variable that cannot be manipulated and yielded results that are masked by independent variables. They defy manipulation but are important because they may provide or shed a light on differences that were not caused by the manipulation of the independent variable.

  Extraneous- any variable external to the study that

impacts the results of the research and needs to be controlled to neutralize its effect ( the third variable).

 

Attribute-pre-existing variable that cannot be manipulated and yielded results that are masked by independent variables. They defy manipulation but are important because they may provide or shed a light on differences that were not caused by the manipulation of the independent variable.

  Extraneous- any variable external to the study that

impacts the results of the research and needs to be controlled to neutralize its effect ( the third variable).

 

Attribute

How do we control for them?How do we control for them?

Random Assignment Avoid confounding Consider a research study that

intends to use a new teaching method in math called Chisenbop

(Finger counting) deem superior than the use of a calculator?

Random Assignment Avoid confounding Consider a research study that

intends to use a new teaching method in math called Chisenbop

(Finger counting) deem superior than the use of a calculator?

Quasi-Experimental and Single-Case Designs Quasi-Experimental and Single-Case Designs

The Quasi-Experimental research designs are used when it is impossible to randomly assign participants to comparison groups (quasi-experimental designs) and when a researcher is faced with a situation where only one or two participants can participate in the research study (single case designs).

Like experimental research, quasi-experimental and single-case designs do have manipulation of the independent variable (otherwise they would not be “experimental research” designs).

The Quasi-Experimental research designs are used when it is impossible to randomly assign participants to comparison groups (quasi-experimental designs) and when a researcher is faced with a situation where only one or two participants can participate in the research study (single case designs).

Like experimental research, quasi-experimental and single-case designs do have manipulation of the independent variable (otherwise they would not be “experimental research” designs).

Quasi DesignsQuasi Designs

These are designs that are used when it is not possible to control for all potentially confounding variables; in most cases this is because the participants cannot be randomly assigned to the groups.

Causal explanations can be made when using quasi-experimental designs but only when you collect data that demonstrate that plausible rival explanations are unlikely, and the evidence will still not be as strong as with one of the strong designs discussed in the experimental research section.

These are designs that are used when it is not possible to control for all potentially confounding variables; in most cases this is because the participants cannot be randomly assigned to the groups.

Causal explanations can be made when using quasi-experimental designs but only when you collect data that demonstrate that plausible rival explanations are unlikely, and the evidence will still not be as strong as with one of the strong designs discussed in the experimental research section.

Where do Quasi Designs Fall?Where do Quasi Designs Fall?

You can view quasi-experiments as falling in the center of a continuum with weak experimental designs on the far left side and strong experimental designs on the far right side.  (In other words, quasi designs are not the worst and they are not the best. They are in-between or moderately strong designs.)

/------------------------------------/------------------------------------/ Weak                            Quasi                              Strong Designs                          Designs                          Designs

You can view quasi-experiments as falling in the center of a continuum with weak experimental designs on the far left side and strong experimental designs on the far right side.  (In other words, quasi designs are not the worst and they are not the best. They are in-between or moderately strong designs.)

/------------------------------------/------------------------------------/ Weak                            Quasi                              Strong Designs                          Designs                          Designs

Types of Quasi DesignsTypes of Quasi Designs

Three quasi-experimental research designs include the following:

1)the nonequivalent comparison-group design,

2)the interrupted time-series design, and

3)the regression discontinuity design

Three quasi-experimental research designs include the following:

1)the nonequivalent comparison-group design,

2)the interrupted time-series design, and

3)the regression discontinuity design

Nonequivalent Comparison-Group Design

Nonequivalent Comparison-Group Design This is a design that contains a treatment group and

a nonequivalent untreated comparison group about of which are administered pretest and posttest measures. The groups are “nonequivalent” because you lack random assignment (although there are some control techniques that can help make the groups similar such as matching and statistical control). Because of the lack of random assignment, there is no assurance that the groups are highly are similar at the outset of the study.

This is a design that contains a treatment group and a nonequivalent untreated comparison group about of which are administered pretest and posttest measures. The groups are “nonequivalent” because you lack random assignment (although there are some control techniques that can help make the groups similar such as matching and statistical control). Because of the lack of random assignment, there is no assurance that the groups are highly are similar at the outset of the study.

Here is a depiction of the nonequivalent comparison-group design:

Here is a depiction of the nonequivalent comparison-group design:

The Bivariate Distribution The Bivariate Distribution

The Bivariate Distribution

Another Visual ExplanationAnother Visual Explanation

C O X O C O O

C indicates that groups are assigned by means of a cutoff score,

an O stands for the administration of a measure to a group,

an X depicts the implementation of a program, and each group is described on a single line (i.e., program

group on top, control group on the bottom).

C O X O C O O

C indicates that groups are assigned by means of a cutoff score,

an O stands for the administration of a measure to a group,

an X depicts the implementation of a program, and each group is described on a single line (i.e., program

group on top, control group on the bottom).

Possible explanationPossible explanation The design is not particularly strong at controlling for threats to internal validity:

1) History:  did some other current event affect the change in the dependent variable?  Researcher must gather qualitative data on possible events that could have affected the fatality rate.

2) Maturation:  were changes in the dependent variable due to normal developmental processes? 

3) Statistical Regression:  did subjects come from low or high performing groups?  Statistical analysis is used to determine whether changes are due to statistical regression or the independent variable.

4) Selection:  were the subjects self-selected into experimental and control groups, which could affect the dependent variable?  Researcher must determine whether there were any major changes in the population between the before and after measures.

5) Experimental Mortality:  did some subjects drop out?  did this affect the results?  Researcher must check whether some of the population dropped out after the implementation of the treatment

6) Testing:  Did the pre-test affect the scores on the post-test? 

7) Instrumentation:  Did the measurement method change during the research?  Researcher must ensure that the same test or versions of the test were measured in the same way in all the years considered.

8) Design contamination:  did the control group find out about the experimental treatment?  did either group have a reason to want to make the research succeed or fail? 

The design is not particularly strong at controlling for threats to internal validity:

1) History:  did some other current event affect the change in the dependent variable?  Researcher must gather qualitative data on possible events that could have affected the fatality rate.

2) Maturation:  were changes in the dependent variable due to normal developmental processes? 

3) Statistical Regression:  did subjects come from low or high performing groups?  Statistical analysis is used to determine whether changes are due to statistical regression or the independent variable.

4) Selection:  were the subjects self-selected into experimental and control groups, which could affect the dependent variable?  Researcher must determine whether there were any major changes in the population between the before and after measures.

5) Experimental Mortality:  did some subjects drop out?  did this affect the results?  Researcher must check whether some of the population dropped out after the implementation of the treatment

6) Testing:  Did the pre-test affect the scores on the post-test? 

7) Instrumentation:  Did the measurement method change during the research?  Researcher must ensure that the same test or versions of the test were measured in the same way in all the years considered.

8) Design contamination:  did the control group find out about the experimental treatment?  did either group have a reason to want to make the research succeed or fail? 

Because there is no random assignment to groups, confounding variables (rather than the independent variable) may explain any difference observed between the experimental and control groups.

The most common threat to the internal validity of this type of design is differential selection. The problem is that the groups may be different on many variables that are also related to the dependent variable (e.g., age, gender, IQ, reading ability, attitude, etc.).

Because there is no random assignment to groups, confounding variables (rather than the independent variable) may explain any difference observed between the experimental and control groups.

The most common threat to the internal validity of this type of design is differential selection. The problem is that the groups may be different on many variables that are also related to the dependent variable (e.g., age, gender, IQ, reading ability, attitude, etc.).

Here is a list of all of the primary threats to this design.Here is a list of all of the primary threats to this design.

Consider the following:Consider the following: It is a good idea to collect data

that can be used to demonstrate that key confounding variables are not the cause of the obtained results. Hence, you will need to think about potential rival explanations during the planning phase of your research study so that you can collect the necessary data to control for these factors.

It is a good idea to collect data that can be used to demonstrate that key confounding variables are not the cause of the obtained results. Hence, you will need to think about potential rival explanations during the planning phase of your research study so that you can collect the necessary data to control for these factors.

You can eliminate the influence of many confounding variables by using the various control techniques, especially statistical control (where you measure the confounding variables at the pretest and control for them using statistical procedures after the study has been completed) and matching (where you select people to be in the groups so that the members in the different groups are similar on the matching variables).

Only when you can rule out the effects of confounding variables can you confidently attribute the observed group difference at the posttest to the independent variable.

You can eliminate the influence of many confounding variables by using the various control techniques, especially statistical control (where you measure the confounding variables at the pretest and control for them using statistical procedures after the study has been completed) and matching (where you select people to be in the groups so that the members in the different groups are similar on the matching variables).

Only when you can rule out the effects of confounding variables can you confidently attribute the observed group difference at the posttest to the independent variable.

Confounding Variables

Interrupted Time-Series DesignInterrupted Time-Series Design

This is a design in which a treatment condition is accessed by comparing the pattern of pretest responses with the pattern of posttest responses obtained from a single group of participants. In other words, the participants are pretested a number of times and then posttested a number of times after or during exposure to the treatment condition.

This is a design in which a treatment condition is accessed by comparing the pattern of pretest responses with the pattern of posttest responses obtained from a single group of participants. In other words, the participants are pretested a number of times and then posttested a number of times after or during exposure to the treatment condition.

Here is a depiction of the interrupted time-series designHere is a depiction of the interrupted time-series design

The pretesting phase is called the baseline which refers to the observation of a behavior prior to the presentation of any treatment designed to alter the behavior of interest.

A treatment effect is demonstrated only if the pattern of post-treatment responses differs from the pattern of pretreatment responses. That is, the treatment effect is demonstrated by a discontinuity in the pattern of pretreatment and post-treatment responses.

For example, an effect is demonstrated when there is a change in the level and/or slope of the post-treatment responses as compared to the pretreatment responses.

The pretesting phase is called the baseline which refers to the observation of a behavior prior to the presentation of any treatment designed to alter the behavior of interest.

A treatment effect is demonstrated only if the pattern of post-treatment responses differs from the pattern of pretreatment responses. That is, the treatment effect is demonstrated by a discontinuity in the pattern of pretreatment and post-treatment responses.

For example, an effect is demonstrated when there is a change in the level and/or slope of the post-treatment responses as compared to the pretreatment responses.

Baseline

Many confounding variables are ruled out in the interrupted time-series design because they are present in both the pretreatment and post-treatment responses (i.e., the pretreatment and post-treatment responses will not differ on most confounding variables).

However, the main potentially confounding variable that cannot be ruled out is a history effect. The history threat is a plausible rival explanation if some event other than the treatment co-occurs with the onset of the treatment.

Many confounding variables are ruled out in the interrupted time-series design because they are present in both the pretreatment and post-treatment responses (i.e., the pretreatment and post-treatment responses will not differ on most confounding variables).

However, the main potentially confounding variable that cannot be ruled out is a history effect. The history threat is a plausible rival explanation if some event other than the treatment co-occurs with the onset of the treatment.

Compounding Variables

Think of many reasons why a Pre& Post-Test might not be effectiveThink of many reasons why a Pre& Post-Test might not be effective

At its easiest, time-series experiments require little more than good graphing skills, a skeptical attitude toward one's pet hypotheses and the capacity to subdivide the data to locate hidden effects. At its most complex, it involves complicated statistical analyses to separate the unaccountable variation of indices across time from the determinant effects of planned interventions

At its easiest, time-series experiments require little more than good graphing skills, a skeptical attitude toward one's pet hypotheses and the capacity to subdivide the data to locate hidden effects. At its most complex, it involves complicated statistical analyses to separate the unaccountable variation of indices across time from the determinant effects of planned interventions

Regression Discontinuity DesignRegression Discontinuity Design

This is a design that is used to access the effect of a treatment condition by looking for a discontinuity in regression lines between individuals who score lower and higher than some predetermined cutoff score on an assignment variable. 

This is a design that is used to access the effect of a treatment condition by looking for a discontinuity in regression lines between individuals who score lower and higher than some predetermined cutoff score on an assignment variable. 

Here is the depiction of the design:Here is the depiction of the design:

For example you might use a standardized test as your assignment variable, set the cut off at 50, and administer the treatment to those falling at 50 or higher and use those with scores lower that 50 as your control group.

This is actually quite a strong design, and methodologists have, for a number of years, been trying to get researcher to use this design more frequently.

One uses statistical techniques to control for differences on the assignment variable and then checks to see whether the groups significantly differ.

For example you might use a standardized test as your assignment variable, set the cut off at 50, and administer the treatment to those falling at 50 or higher and use those with scores lower that 50 as your control group.

This is actually quite a strong design, and methodologists have, for a number of years, been trying to get researcher to use this design more frequently.

One uses statistical techniques to control for differences on the assignment variable and then checks to see whether the groups significantly differ.

Discontinuity Design

Single-Case Experimental DesignsSingle-Case Experimental Designs

These are designs where the researcher attempts to demonstrate an experimental treatment effect using single participants, one at a time.

These may be described as single-case designs: A-B-A design, A-B-A-B design, multiple-baseline design, and the changing-criterion design.

These are designs where the researcher attempts to demonstrate an experimental treatment effect using single participants, one at a time.

These may be described as single-case designs: A-B-A design, A-B-A-B design, multiple-baseline design, and the changing-criterion design.

A-B-A and A-B-A-B DesignsA-B-A and A-B-A-B Designs

The A-B-A design is a design in which the participant is repeatedly pretested (the first A phase or baseline condition), then the experimental treatment condition is administered and the participant is repeatedly post-tested (the B phase or treatment phase).  Following the post-testing stage, the pretreatment conditions are reinstated and the participant is again repeatedly tested on the dependent variable (the second A phase or the return to baseline condition).

The A-B-A design is a design in which the participant is repeatedly pretested (the first A phase or baseline condition), then the experimental treatment condition is administered and the participant is repeatedly post-tested (the B phase or treatment phase).  Following the post-testing stage, the pretreatment conditions are reinstated and the participant is again repeatedly tested on the dependent variable (the second A phase or the return to baseline condition).

Here is a depiction of the A-B-A design:Here is a depiction of the A-B-A design:

Multiple Base DesignsMultiple Base Designs

This is a design that investigates two or more people, behaviors, or settings to identify the effect of an experimental treatment. The key is that the treatment condition is successively administered to the different people, behaviors, or settings

This is a design that investigates two or more people, behaviors, or settings to identify the effect of an experimental treatment. The key is that the treatment condition is successively administered to the different people, behaviors, or settings

Here is a depiction of the design

Changing-Criterion DesignChanging-Criterion Design

This is a single-case design that is used when a behavior needs to be shaped over time or when it is necessary to gradually change a behavior through successive treatment periods to reach a desired criterion.

This design involves collecting baseline data on the target behavior and then administering the experimental treatment condition across a series of intervention phases where each intervention phase uses a different criterion of successful performance until the desired criterion is reached.

 The criterion used in each successive intervention phase should be large enough to detect a change in behavior but small enough so that it can be achieved.

This is a single-case design that is used when a behavior needs to be shaped over time or when it is necessary to gradually change a behavior through successive treatment periods to reach a desired criterion.

This design involves collecting baseline data on the target behavior and then administering the experimental treatment condition across a series of intervention phases where each intervention phase uses a different criterion of successful performance until the desired criterion is reached.

 The criterion used in each successive intervention phase should be large enough to detect a change in behavior but small enough so that it can be achieved.

Chisanbop is a method of doing basic arithmetic using your fingers. Is it more effective than the calculator?

Chisanbop is a method of doing basic arithmetic using your fingers. Is it more effective than the calculator?

Select a hypothesis

Research hypothesis(if and then)

Think of an alternative or rival hypothesis

Select a hypothesis

Research hypothesis(if and then)

Think of an alternative or rival hypothesis

A null hypothesis

Select a type of design.

Describe how you would conduct the research