experimental design by julie-anne spatz and adam macleod
TRANSCRIPT
Experimental DesignExperimental Design
By Julie-Anne Spatz and Adam By Julie-Anne Spatz and Adam MacLeodMacLeod
Experiment vs. Observational Experiment vs. Observational StudyStudy
An experiment deliberately imposes a An experiment deliberately imposes a treatment on a group of units or subjects treatment on a group of units or subjects in the interest of observing the response. in the interest of observing the response.
VS.VS.
An observational study involves collecting An observational study involves collecting and analyzing data without changing and analyzing data without changing existing conditions or imposing a existing conditions or imposing a treatment.treatment.
The ExperimentThe Experiment
““This study was designed to examine the This study was designed to examine the effects of carbohydrate-electrolyte effects of carbohydrate-electrolyte ingestion on physical and mental function ingestion on physical and mental function associated with the performance of associated with the performance of intermittent high-intensity (IHI) exercise intermittent high-intensity (IHI) exercise similar to many common competitive similar to many common competitive sporting events.” sporting events.”
http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=11932585&dopt=Abstractcmd=Retrieve&db=PubMed&list_uids=11932585&dopt=Abstract
In Other Words…In Other Words…
Researchers, RS Welsh, JM Davis, JR Researchers, RS Welsh, JM Davis, JR Burke, and HG Williams, examined Burke, and HG Williams, examined athletes physical performance while athletes physical performance while drinking either Gatorade or water. drinking either Gatorade or water. We will use this experiment to We will use this experiment to explain general terms about explain general terms about experimental design.experimental design.
Units or SubjectsUnits or Subjects
Experimental units: the individuals Experimental units: the individuals on which the experiment is doneon which the experiment is done
Subjects: when the units are human Subjects: when the units are human beingsbeings
The subjects used were 60 male The subjects used were 60 male soccer players.soccer players.
TreatmentsTreatments
Treatment: a specific experimental condition applied to Treatment: a specific experimental condition applied to the unitsthe units
Placebo: a dummy treatment, that can have no physical Placebo: a dummy treatment, that can have no physical effecteffect
Factor: a controlled independent variable (called the Factor: a controlled independent variable (called the explanatory variable), whose levels are set by the explanatory variable), whose levels are set by the experimenterexperimenter
Level: each treatments specific valueLevel: each treatments specific value
The factors in this experiment include the amounts of The factors in this experiment include the amounts of Gatorade and water given to each soccer player every Gatorade and water given to each soccer player every fifteen minutes throughout a soccer game.fifteen minutes throughout a soccer game.
Example of levels:Example of levels:Gatorade: 6 ounces, 8 ounces, 10 ouncesGatorade: 6 ounces, 8 ounces, 10 ouncesWater: 6 ounces, 8 ounces, 10 ouncesWater: 6 ounces, 8 ounces, 10 ounces
VariablesVariables
Control Group: the subjects that are not Control Group: the subjects that are not subjected to the treatmentsubjected to the treatment
Experimental Group: the subjects that are Experimental Group: the subjects that are subjected to the treatmentsubjected to the treatment
Control Group: people received only water Control Group: people received only water (30 people)(30 people)
Experimental Group: people received only Experimental Group: people received only Gatorade (30 people)Gatorade (30 people)
RandomizationRandomization
Randomization: the use of chance to Randomization: the use of chance to divide experimental units into groupsdivide experimental units into groups
To conduct a completely randomized To conduct a completely randomized experiment:experiment:
1. use a Table of Random Digits1. use a Table of Random Digits
2. use the calculator to generate 2. use the calculator to generate random numbers random numbers
Table of Random DigitsTable of Random Digits
Calculator TechniqueCalculator Technique
1.1. Stat Edit highlight L1Stat Edit highlight L1
2.2. Math PRB 5:randInt(Math PRB 5:randInt(
3.3. Type in randInt(lower bound, upper Type in randInt(lower bound, upper bound, amount of integers needed)bound, amount of integers needed)
Stratified Random Sampling vs. Stratified Random Sampling vs. Multistage SamplingMultistage Sampling
Stratified Random Sample: an experimenter would first Stratified Random Sample: an experimenter would first divide the population into groups of similar individuals divide the population into groups of similar individuals called a strata, then choose a separate SRS in each stratum called a strata, then choose a separate SRS in each stratum and combine these SRSs to form the full sampleand combine these SRSs to form the full sample
Multistage Sampling: is a sample where the elements are Multistage Sampling: is a sample where the elements are chosen in more than one stagechosen in more than one stage
Ex. 1: If the experimenters wanted to do a stratified random Ex. 1: If the experimenters wanted to do a stratified random sample, they could do this by putting soccer players in to sample, they could do this by putting soccer players in to different stratas by region of the United States and then different stratas by region of the United States and then conduct the experiment.conduct the experiment.
Ex. 2: If the experimenters wanted to do a multistage Ex. 2: If the experimenters wanted to do a multistage sample, they could do this by choosing the soccer players sample, they could do this by choosing the soccer players at the state level, then the city level, then the county level, at the state level, then the city level, then the county level, and then by league. and then by league.
Randomization Technique UsedRandomization Technique Used
The experimenter used a random The experimenter used a random number table to assign each soccer number table to assign each soccer player a number 1 thru 60. Then, player a number 1 thru 60. Then, the people with odd numbers were the people with odd numbers were put into the control group and the put into the control group and the people with even numbers were put people with even numbers were put into the experimental group.into the experimental group.
Single Blind Experiment vs. Double Single Blind Experiment vs. Double Blind ExperimentBlind Experiment
Single Blind Experiment: the experimenters, but not the Single Blind Experiment: the experimenters, but not the subjects know which treatment a subject receivedsubjects know which treatment a subject received
Double Blind Experiment: the experimenters nor the subjects Double Blind Experiment: the experimenters nor the subjects know which treatment a subject receivedknow which treatment a subject received
This experiment was a single blind experiment because the This experiment was a single blind experiment because the experimenter knew what groups the subjects were put in, but experimenter knew what groups the subjects were put in, but the subjects were not aware. To ensure that the subjects the subjects were not aware. To ensure that the subjects could not tell the difference between the Gatorade and water, could not tell the difference between the Gatorade and water, the experimenters put a type of sweetener into the Gatorade. the experimenters put a type of sweetener into the Gatorade. The sweetener only affected the flavor, but not the amount The sweetener only affected the flavor, but not the amount of electrolytes and carbohydrates.of electrolytes and carbohydrates.
If the experiment were to be a double blind experiment, the If the experiment were to be a double blind experiment, the experimenter would have had to of had a separate person experimenter would have had to of had a separate person assign which groups would receive the Gatorade and water, assign which groups would receive the Gatorade and water, without the experimenters knowledge. Then the researcher without the experimenters knowledge. Then the researcher could take over conducting the experiment.could take over conducting the experiment.
Matched PairsMatched Pairs Matched Pairs: experiment that compares two Matched Pairs: experiment that compares two
treatmentstreatments
Ex. 1: If this experimenter were to use matched Ex. 1: If this experimenter were to use matched pairs, every soccer player would drink water pairs, every soccer player would drink water throughout one game, then Gatorade throughout throughout one game, then Gatorade throughout another, and the experimenter would note which another, and the experimenter would note which one yielded higher performance.one yielded higher performance.
Ex. 2: An experiment is conducted to compare the Ex. 2: An experiment is conducted to compare the taste of a new spaghetti sauce with the taste of a taste of a new spaghetti sauce with the taste of a successful sauce. Each of a number of subjects successful sauce. Each of a number of subjects tastes both sauces in random order and notes tastes both sauces in random order and notes which one tastes better.which one tastes better.
Block DesignBlock Design Block: a group of experimental units or subjects that are similar in Block: a group of experimental units or subjects that are similar in
ways that are expected to affect the response to the treatmentsways that are expected to affect the response to the treatments Block Design: the random assignment of units to treatments is Block Design: the random assignment of units to treatments is
carried out separately within each block.carried out separately within each block.
Ex. 1: If the experimenter thought gender was an issue, the Ex. 1: If the experimenter thought gender was an issue, the experimenter would separate the men from the women, and test experimenter would separate the men from the women, and test the effects of both the Gatorade and the water on the two groups. the effects of both the Gatorade and the water on the two groups.
Ex. 2: If the experimenter thought that age was an issue, the Ex. 2: If the experimenter thought that age was an issue, the experimenter would separate the people into four age groups; one experimenter would separate the people into four age groups; one of people aged 1-15, one of people aged 16-30, one of people of people aged 1-15, one of people aged 16-30, one of people aged 31-45, and one of people aged 46-60 and test each group aged 31-45, and one of people aged 46-60 and test each group with the Gatorade and water. with the Gatorade and water.
Ex. 3: If the experimenter was interested in physical condition, the Ex. 3: If the experimenter was interested in physical condition, the experimenter would separate the subjects into two groups of experimenter would separate the subjects into two groups of strong physical condition and weak physical condition, and then strong physical condition and weak physical condition, and then test the Gatorade and water on each group.test the Gatorade and water on each group.
Compare Results
Group 230 Women
Treatment 1Gatorade
Treatment 2Water
Treatment 1Gatorade
Treatment 2Water
Random Allocation
Group 130 Men
Block Design By Gender
Cautions of Experimental DesignCautions of Experimental Design Hidden Bias: if the units or subjects are dealt Hidden Bias: if the units or subjects are dealt
with any different ways, hidden bias can arise with any different ways, hidden bias can arise (unequal conditions introduce bias)(unequal conditions introduce bias)
Ex. If the researcher were to hand a Ex. If the researcher were to hand a soccer player Gatorade and smile and soccer player Gatorade and smile and hand the other soccer player water with a hand the other soccer player water with a frownfrown
Lack of Realism: units or subjects may not Lack of Realism: units or subjects may not believe in the experiment being conducted, believe in the experiment being conducted, making the results of the experiment less making the results of the experiment less reliablereliable
Ex. If the soccer players already Ex. If the soccer players already believed believed that Gatorade produced that Gatorade produced stronger athletic stronger athletic performance than performance than waterwater
Cautions of Experimental Design Cautions of Experimental Design ContinuedContinued
Undercoverage: not including the whole Undercoverage: not including the whole populationpopulation
Ex. If the researcher only included Ex. If the researcher only included soccer players from the East coast.soccer players from the East coast.
Non-response: Some people may choose Non-response: Some people may choose not to respond to a questionnaire or phone not to respond to a questionnaire or phone survey, decreasing the number of survey, decreasing the number of responsesresponses
Ex. If some soccer players turned Ex. If some soccer players turned down down the opportunity to participate in the opportunity to participate in the the experimentexperiment
Confounding VariablesConfounding Variables
Confounding Variable: is a "hidden" Confounding Variable: is a "hidden" variable that affects the variables in variable that affects the variables in question but is not known, and question but is not known, and distorts the resulting data distorts the resulting data
In this experiment, some In this experiment, some confounding variables include diet, confounding variables include diet, athletic ability, age, weather athletic ability, age, weather conditions, gender, etc.conditions, gender, etc.
BibliographyBibliography
http://helios.bto.ed.ac.uk/bto/statistics/tress2.http://helios.bto.ed.ac.uk/bto/statistics/tress2.html#DESIGN%20OF%20EXPERIMENTShtml#DESIGN%20OF%20EXPERIMENTS
http://www.stats.gla.ac.uk/steps/glossary/http://www.stats.gla.ac.uk/steps/glossary/anova.html#treatmentanova.html#treatment
http://www.sytsma.com/phad530/http://www.sytsma.com/phad530/expdesig.htmlexpdesig.html
http://www.stat.yale.edu/Courses/1997-http://www.stat.yale.edu/Courses/1997-98/101/expdes.htm98/101/expdes.htm
http://www.ncbi.nlm.nih.gov/entrez/http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?query.fcgi?cmd=Retrieve&db=PubMed&list_uids=11932cmd=Retrieve&db=PubMed&list_uids=11932585&dopt=Abstract585&dopt=Abstract