If your experiment needs statistics, you ought to have done a better experiment. Ernest Rutherford (1871-1937)The Nature of Statistics: Experimental Design

Observation vs. Experimentation

Observational Studies: observe and reveal associations.

Designed Experiments: impose treatments and controls; can help establish cause-effect relationships.

Experimentation Deliberately imposes some treatment

on the experimental units or subjects in order to observe a response.

Intent of most experiments is to study the effect of changes in some variables by other variables.

Experiment A treatment is applied to part of a population and

responses are observed.

Example: An experiment was performed in which diabetics took cinnamon extract daily while a control group took none. After 40 days, the diabetics who had the cinnamon reduced their risk of heart disease while the control group experienced no change. (Source: Diabetes Care)

5Larson/Farber 4th ed.

Terms used in Experimentation

Experimental Units: the individuals or items on which the experiment is performed (subjects).

Block: a group of subjects that are similar. Response Variable: the variable of interest in the

experiment. The variable that describes the outcome. That which will be measured. (Also called the dependent variable.)

Factor(s): a variable or variables whose effect on the response variable is of interest in the experiment. (Also called the explanatory or independent variable.)

Terminology (cont’d) Level(s): the different categories of the

factor(s), or possible values of the factor(s). Treatments: each experimental condition.

Combinations of the factor(s). Treatment Group: the group receiving the

specified treatment. Control Group: the group receiving the

placebo, or no treatment.

3 Key Elements (or Basic Principles) of Experimental

Design

Control: attempt to control factors not of interest to the study, but known to affect variation.

Randomization: allow each unit in the experiment an equal chance of receiving any particular treatment.

Replication: sufficient number of subjects to ensure that randomization creates groups that resemble each other and allow for repetition.

Key Elements of Experimental Design: Control Control for effects other than the one being

measured. Confounding variables

Occurs when an experimenter cannot tell the difference between the effects of different factors on a variable. Example: A coffee shop owner remodels her shop at the

same time a nearby mall has its grand opening. If business at the coffee shop increases, it cannot be determined whether it is because of the remodeling or the new mall.

9Larson/Farber 4th ed.

Key Elements of Experimental Design: Control Placebo effect

A subject reacts favorably to a placebo when in fact he or she has been given no medical treatment at all.

Blinding is a technique where the subject does not know whether he or she is receiving a treatment or a placebo.

Double-blind experiment neither the subject nor the experimenter knows if the subject is receiving a treatment or a placebo.

10Larson/Farber 4th ed.

Key Elements of Experimental Design: Randomization Randomization is a process of randomly

assigning subjects to different treatment groups.

Completely randomized design Subjects are assigned to different treatment groups

through random selection. Randomized block design

Divide subjects with similar characteristics into blocks, and then within each block, randomly assign subjects to treatment groups.

11Larson/Farber 4th ed.

Experimental Design Examples

Completely Randomized Design: all experimental units are assigned randomly among all treatments. Golf ball brands and driving distance: Which

brand of golf ball goes the furthest? A group of 50 randomly selected golfers are randomly assigned to test five brands of golf balls.

Completely Randomized Design for golf-ball experiment

Randomized Block Design: experimental units are assigned randomly among all treatments within blocks. If the experimenter wishes to control for gender of

the golfer, here perhaps assuming that the distances a golf ball is hit is affected by gender, how might this difference be controlled for?

Randomized Block Design for golf-ball experiment

Key Elements of Experimental Design: RandomizationRandomized block design An experimenter testing the effects of a new

weight loss drink may first divide the subjects into age categories. Then within each age group, randomly assign subjects to either the treatment group or control group.

16Larson/Farber 4th ed.

Key Elements of Experimental Design: Replication

Replication is the repetition of an experiment using a large group of subjects.

To test a vaccine against a strain of influenza, 10,000 people are given the vaccine and another 10,000 people are given a placebo. Because of the sample size, the effectiveness of the vaccine would most likely be observed.

17Larson/Farber 4th ed.

Example: Experimental Design

A company wants to test the effectiveness of a new gum developed to help people quit smoking. Identify a potential problem with the given experimental design and suggest a way to improve it.

The company identifies one thousand adults who are heavy smokers. The subjects are divided into blocks according to gender. After two months, the female group has a significant number of subjects who have quit smoking.

18Larson/Farber 4th ed.

Solution: Experimental Design

Problem:

The groups are not similar. The new gum may have a greater effect on women than men, or vice versa.

Correction:

The subjects can be divided into blocks according to gender, but then within each block, they must be randomly assigned to be in the treatment group or the control group.

19Larson/Farber 4th ed.

Fabric Durability:A fabric researcher is studying the durability of a fabric under repeated washings. Because the durability may depend on the water temperature and the type of cleansing agent used, the researcher decides to investigate the effect of these factors on durability. The water temperature used in the experiment will be: hot (145 degrees F), warm (100 degrees F), and cold (50 degrees F). The cleansing agents used will be regular Tide, regular Cheer, and Ivory Liquid. Each piece of fabric used, will be washed 50 times in a home automatic washer with a specific combination of water and cleansing agent. The strength of the fabric will then be tested by a machine which forces a steel ball through the fabric and records the fabric’s resistance to breaking.

Fabric Durability:Identify the components of the experiment

Identify what is being studied. What are the units? What is the Response Variable? What is/are the Factor(s)? How many levels per Factor? How many Treatments? Describe one Treatment Group. How have Control, Randomization and Replication

been addressed? If not stated, how could they be addressed?

 

Temperature Detergent type

Ivory Liq.CheerTide

145 deg.

100 deg.

50 deg.

Trtmt. 1Tide/145

Trtmt. 2Tide/100

Trtmt. 3Tide/50

Trtmt. 4Cheer/145

Trtmt. 5Cheer/100

Trtmt. 6Cheer/50

Trtmt. 7Ivory/145

Trtmt. 8Ivory/100

Trtmt 9Ivory/50

Fabric Durability:

TREATMENTA TREATMENT is comprised of Levels from one or more Factors (Independent Variables). If there is only one Factor, such as Temperature, then the TREATMENTS are the Levels of the single Factor, Temperature. If there is more than one Factor, then the TREATMENTS represent the combination of Levels from the Factors (here a combination of Temperature and Time).

Anatomy of an Experiment: Terms and their Relationships

FACTOR (aka INDEPENDENT VARIABLE or EXPLANATORY VARIABLE):A FACTOR is a condition applied to the Experimental Units, such as cooking time or cooking temperature.

RESPONSE VARIABLE (aka DEPENDENT VARIABLEor OUTCOME VARIABLE): A RESPONSE VARIABLE is the characteristic of the experiment to be measured or observed, such as the consistency of a cookie.

LEVEL:LEVELS represent the different values of a Factor that are applied to the Experimental Units, such as cooking at 350 and 450 degrees. There is always a minimum of two Levels - one which may represent the existing conditions and thereby represent a Control Level (Control Group) for a Factor and another which represents an Experimental Level.

EXPERIMENTAL UNITS:Experimental Units are the items (here cookies) or individuals (also called Subjects) upon which the experiment is performed. Experimental Units are assigned to Control and Experimental Conditions at random.

The Experiment: The Munchy Cookie Company is famous for its Chewy Mango cookies. In order to increase production the company wants to determine if it can reduce cooking time and increase temperature and still get the same quality cookie (traditionally baked for eight minutes at 350 degrees). The conditions under consideration involve temperatures of 350 and 450 degrees and cooking times of six and eight minutes. Chewy Mango cookies can be bent to a 300 degree outside angle before breaking. If the change in cooking conditions replicates the original conditions, the company will consider this new cooking process.

350 Degrees8 minutes

1000 Chewy Mango cookie dough balls

350 Degrees6 minutes

450 Degrees8 minutes

450 Degrees6 minutes

Random Sample of 250 dough balls to each condition

Six minutes represents a Level of the Time Factor

A Treatment representing one Level from each of two Factors

Compare Cooking Time/Temp. results and state Conclusions

Take Measurements

HELP! We’re getting

a bit crispy

Control Group:One of the treatment conditionsmay represent a Control Group to which others are compared.

Factors applied to samples Control Group: Here, the

current baking process (8 min; 350o)

End of Slides