Chapter 13 Experiments & observational studies

Chapter 13 Experiments & observational studiesObservational studiesresearchers dont assign choices, they observe them

a study in which no manipulation of factors has been employed

helpful for discovering trends and possible relationships

although an observational study may identify important variables related to an outcome. there is no guarantee that we have found the right or most important related variable

Observational studiesRetrospective StudyProspective Studysubjects are selected then previous conditions or behaviors are determinednot based on random samples, they focus on estimating differences between groups or associations between variableshave a restricted view of the world because they are usually restricted to a small part of the populationbecause they are based on historical data there could be errors identifying subjects in advance and collecting data as the events unfoldObservational studies that try to discover variables related to rare outcomes are often retrospectiveExample: specific diseaseidentify people with the diseaselook at their history and heritagesomething that could be related to their condition

Who gets good grades? And why?In 1981 a study was conducted at a high school in California. Researchers compared the scholastic performance of music students with that of non-music students.The music students had a higher GPA of 3.59 compared to 2.91 that non-music students had16% of music students earned all As where only 5% of non-music students earned all As

Does this study tell us that music will improve students GPAs?students that study music might still differ from the others in some important way that we failed to observe. Can we prove cause and effect?!?Experiment: studies the relationships between two or more variables manipulates factor levels to create treatments, randomly assigns subjects to these treatment levels, and then compares the responses of the subject groups across treatment levels.Random assignment: an experiment must assign experimental units to treatment groups at random for the experiment to be valid.

Experiment Vocabulary Factor: a variable whose levels are controlled by the experimenter. Experiments attempt to discover the effects that differences in factor levels may have on the responses of the experimental unitsResponse variable: a variable whose values are compared across different treatments. In a randomized experiment, large response differences can be attributed to the effect of differences in treatment levels. Subject/experimental unit: individuals on whom an experiment is performed. (participants for humans too)Level of the factor: the specific values that the experimenter chooses for a factor Treatment: the process, intervention, or other controlled circumstance applied to randomly assigned experimental units. Treatments are the different levels of a single factor or are made up of combinations of levels of two or more factorsExperiment exampleDesign an experiment to see whether the amount of sleep and exercise you get affects your performance Subjects: the people in the sleep studyFactors: sleep and exerciseFactor levels:sleep: 4, 6, or 8 hoursexercise: 0 min or 30 mins on a treadmillTreatment: 6 totalOnce all this is set up you need to RANDOMLY assign each subject to a treatment Music & GPA exampleonly an experiment can justify a claim like music lessons cause higher grades

Music and GPA experimenttake a group of 3rd graders (subjects) study music and never take music classes (factor and levels)collect data about their GPA (response variable)4 Principles Of Experimental Design1) Controlmake conditions as similar as possible for all treatments groups. reduces the variability of the responses, making it easier to detect differences among the treatment groupsRisky!! we are testing laundry detergents. we control the water temperature at 1800. this would reduce variation in our results due to water temperature. BUT now we cant say anything about the detergent in cold waterwe control a factor by assigning subjects to different levels because we want to see how the response will change at different levelswe control other sources of variation to prevent them changing and affecting the response variable2) Randomizeallows us to equalize the effects of unknown or uncontrollable sources of variationit does not eliminate the effects of these sources, it spreads them out over all the treatments so we can see past themprotects us from things we didnt even know aboutcontrol what you can, and randomize the rest3) Replicatewe should repeat the experiment, applying the treatments to a number of subjectsan experiment is only complete once you have assessed the variationthe outcome of an experiment on a single subject is an anecdote, not datawhen the subjects are not a representative sample of the population of interest; repeat the experiment with people from different ages and different time of year

Replication of an entire experiment with controlled sources of variation at different levels is an essential step in science(Not Required)4) Blockto reduce the effects of identifiable attributes of the subjects that cannot be controlled. example: 10 people 2 math teachers and 8 math studentswe want to break them up into two groups for a math competition. if we pick people randomly the math teachers could be on the same team. (unfair) so we put 1 math teacher on each team then we randomly assign 4 students to each team. we are blocking the occupation variable allowing us to remove the variability due to the differences among the blocks. Diagramsan experiment is carried out over time with specific actions occurring in a specified ordera diagram of the procedure can help think about experiments

Designing An Experiment:Step-by-stepAn ad for OptiGro plan fertilizer claims that with this product you will grow juicer, tastier tomatoes. You would like to test this claim, and wonder whether you might be able to get by with half the specified dose.

Basically we need to buy some tomato plants and use OptiGro on some of them.

We will set up a completely randomized experiment in one factorThinkPlan: state what you want to knowI want to now whether tomato plants grown with OptiGro yield juicer, tastier tomatoes than plants raised in otherwise similar circumstances but without the fertilizer. Response variableIll evaluate the juiciness and taste of the tomatoes by asking a panel of judges to rate them on a scale from 1 to 7 in juiciness and in taste. Treatments: specify the factor levels and the treatmentsThe factor is fertilizer, specifically OptiGro. Ill grow tomatoes at three different factor levels; some with no fertilizer, some with half the specified amount, and some with the full dose of OptiGro. These are the three treatments.Experimental Units: Ill obtain 24 tomato plants of the same variety from a local garden store. Experimental Design: observe the principles of designControl: any source of variability you know of and can controlRandomly assign: experimental units to treatments, to equalize the effects of unknown or uncontrollable sources of variationReplicate: results by placing more than one plant in each treatment groupIll locate the farm plots near each other so that the plants get similar amounts of sun and rain and experience similar temperatures. I will weed the plots equally and otherwise treat the plants alike. Ill randomly divide the plants into three groups. I will use random numbers from a table to determine the assignment. There will be 8 plants in each treatment group. Draw a pictureSpecify any other experiment details. You must give enough details so that another experimenter could exactly replicate your experiment. Its generally better to include details that might seem irrelevant than to leave out matter that could turn out to make a difference. Specify how to measure the response. I will grow the plants until the tomatoes are mature, as judged by reaching a standard color. I will then harvest the tomatoes when ripe and store then for evaluation. I will set up a numerical scale of juiciness and one for tastiness for the taste testers. Several people will taste slices of tomato and rate them. ShowOnce you collect the data, youll need to display them and compare the results for the three treatment groups. I will display the results with side-by-side box-plots to compare the three treatment groups. I will compare the means of the groups. TellTo answer the initial question, we ask whether the difference we observe in the means of the three groups are meaningful. Control Treatmentanother level of the factor in order to compare the treatment results to a situation in which nothing happensexample: the group of tomatoes that received no fertilized

Different from control (one of the 4 principles of experiments). controlling extraneous sources of variations by keeping them constantexample: buying the plants from the same nursery, weeding all the plots the same way

BlindingTo avoid bias, we disguise the levels of the factors

There are two main classes of individuals who can affect the outcome of the experiment:those who could influence the resultsthe subjects, treatment administrators, or techniciansthose who evaluate the results (judges, treating physicians, etc)Single Blind: when one of the groups is blindedDouble Blind: when both of the groups are blinded

Tomato Experiment: we dont want our tasters to be bias so we will not tell them plants received OptiGroPlaceboa control treatment that mimics the real treatmentthe best way to blind subjects

placebo effect: it is not unusual for 20% or more of subjects given a placebo to report reduction or pain, improved movement, or greater alertnesshighlights both the importance of effective blinding and the importance of comparing treatments with a control

The best experiments are usuallyrandomizedcomparativedouble-blindplacebo-controlledBlockingTomato plants: we want 18 plantswe get 12 from store A and 6 from store B

The store the plants came from could effect our resultsSo we break the 18 plants into two BLOCKSone for each storeThen we randomly assign the plants in each block to one of the three treatments

This is called randomized block designblocking is the same idea for experiments as stratifying for sampleswe use blocks to reduce variability so we can see the effects of the factors, we are nut usually interested in studying the effects of the blocks themselvesDiagramMatchingMatching: when subjects are paired because they are similar in ways NOT being studied in retrospective and prospective studiesMusic and grades: we might match each student who studies an instrument with someone of the same sex and similar family income who doesnt play an instrumentExample ProblemsDescribe a strategy to randomly split the 24 tomato plants into the three groups for the 3 treatments in the OptiGro experiment. A running shoe manufacturer wants to test the speed of its new sprinting shoe on the 100-meter dash times. The company sponsors 5 athletes who are running the 100-meter dash in the 2004 Summer Olympic games. To test the show, they have all 5 runners run the 100-meter dash with a competitor's shoe and then again with their new shoe. They use the difference in times as the response variable. Suggest some improvements to the design.

Why might the shoe manufacturer not be able to generalize the results they find to all runners?Athletes who had suffered hamstring injuries were randomly assigned to one of two exercise programs. Those who engages in static stretching returned to sports activity in a mean of 37.4 days (SD = 27.6 days). Those assigned to a program of agility and trunk stabilization exercises returned to sports in a mean of 22.2 days (SD = 8.3 days). Explain why it was important to assign the athletes to the two different treatments randomly.

There was no control group of athletes who did not participate in a special exercise program. Explain the advantage of including such a group in this experiment.

How might blinding have been used in this experiment?

Do you think the difference in times is statistically significant? Adding More FactorsOptiGro ExperimentWe want to add a factor about wateringLevels: only natural watering (rain) and watering by hand daily

Treatments: 6

Completely randomized two-factor experiment

No fertilizer fertilizerfull fertilizerNo watering123Daily watering456Confounding When the levels of one factor are associated with the levels of another factor, the factors are said to be confounding

Example: A credit card bank wanted to test the sensitivity of the market by two factors: the annual fee charged for a card and the annual percentage rate charged. The bank selected 100,000 people at random from a mailing list. It sent out 50,000 offers with a low rate and no fee and 50,000 offers with a higher rate and a $50 annual fee. They found out people signed up (more than twice the rate) for the card with low rate and no annual fee. Problem: The question that the bank was trying to answer was how much of the change is due to the rate and how much was due to the fee? How could they have avoided this?