Observational Studies

Why the caution?

“Dr Rao found that patients who had had a transfusion because of a low red blood-cell count had an 8% chance of dying within 30 days. Without a transfusion, only 3% died. Those numbers need to be treated with caution.”

The Economist

Magazine, 2007

Chance, or is something else going on?

Dr. Rao found a difference of

8% - 3% = 5%more deaths within 30 days with blood transfusion.

Is this a real difference, or just chance? If not chance, what else might be going on?

Dr. Rao found an association with blood transfusion, but he cautioned that the effect of severity of illness might have gotten mixed up with the effect of the exposure.

Confounding variables

Dr. Rao’s study of patients’ survival after blood transfusion is observational, and the severity of illness is a confounding factor…

DefinitionListen, for the sound of two feet dropping…

One: severity of illness correlates with the exposure, blood transfusion

Two: severity of illness correlates with the outcome, death

True or false, and explain…

Some studies find an association between liver cancer and smoking, but alcohol consumption is a confounding variable.

Two strategies…I. Matching

II. Randomized clinical trial

Low Medium

High

“Exposed”

“Control”

Eligible Pop’n or the “pop’n at risk…”

1/2

1/2

Treatment

Control

Three levels of severity

The clofibrate trial for heart disease prevention…

a randomized, controlled, double-blind experiment

708 15% 1,813 15%

357 25% 882 28%

1,103 20% 2,789 21%

Number Deaths

Number

Deaths

Clofibrate

Placebo

Adherers

Non-adherers

Total

There are two conclusions here.

Experiments and Observational Studies

Method of comparison T versus C or Exposed versus Unexposed

Experiments Key variables are held constant Control over group assignment, where…

…impartial chance, called randomization, works best Treatments are interventions, where…

… the goal is a description of the effect.

Observational studies No control over group assignment, often…

…self selection or exposure by association… Observations, not interventions Key variables may vary, so…

…confounding is always a risk Matching, strategic but not always effective

Why not always experiment?

Ethics and human experimentationsmoking and lung cancer

Prohibitive and costlysocial policy (e.g. welfare) and economics

(e.g. monetary policy)natural experiments; $ and happiness in

Switzerland

Modes of scientific discoverynotebooks, laboratories, and theories

Smoking and health

The Public Health Service studied the effects of smoking on health in a large sample of representative households. For men and for women in each age group, those who had never smoked were on average somewhat healthier than current smokers, but the current smokers were on average much healthier than those who had recently stopped smoking.

(a) Why study men and women and the different age groups separately?

(b) The lesson seems to be that you shouldn’t start smoking, but once you’ve started, don’t stop. Comment.

Cervical cancer and circumcision…

Cervical cancer was once quite common. Epidemiologists trying to identify the cause had noticed that in several different countries it was rare among Jews and Moslems. In the 1950s, some investigators concluded that circumcision of males was protective. Was this conclusion justified?

Discussion

*What is the response or outcome? the “exposure” or treatment?

* Do these communities differ from members of other communities in ways besides circumcision?

Cervical cancer…1970’s

According to a study done at Kaiser Permanente in Walnut Creek, California, users of oral contraceptives have a higher rate of cervical cancer than non-users, even after adjusting for age, education, and marital status. Investigators concluded that the pill causes cervical cancer.*

● Experiment? Or, observational study?● Why the adjustments for…?● Users were likely to differ from non-users on another factor affecting the risk….what other factor?● Were the conclusions justified? Yes or no, explain.

*American Journal of Epidemiology, Vol. 106, 1977, pp. 462-69….adjustments were made for religion, smoking, number of PAP smears before entry, and “selected infections.”

What explains the observations?

What is the human papilloma virus? And how is it spread?

Associations establish links, but they are not causal. They can even be spurious.

Vitamins…associations and effects…

People who get lots of vitamins by eating five or more servings of fresh fruits and vegetables (especially “cruciferous” vegetables like broccoli) have much lower death rates from colon cancer and lung cancer, according to many observational studies. These studies were so encouraging that two randomized controlled experiments were done….

Findings were…1. No difference in death rate for colon cancer between T and C;2. Beta carotene (as a dietary supplement) increased death rates due to lung cancer.

True or false, and explain1. Experiments confirm the results of the observational studies;2. People who eat lots of fruits and vegetables have lifestyles that are different in many other ways too—so due to confounding the

a. observational studies could easily have reached the wrong conclusion; or

b. experiments could easily have reached the wrong conclusion.

In what way might the observational studies have gotten it right?

Associations establish links…

Here are two examples…Dr. Rao had established an association or link between blood transfusion and survival. It did point to something causal: if transfusion was a cause of death, then you’d expect….but this alone does not prove causation.

Nurses Health Study found a link between hormones and cardiovascular disease. It did point to something causal: if hormones are protective, then you’d expect…but the Women’s Health Initiative did not confirm the protective effects in an randomize controlled experiment. WHI did, however, corroborate the risks…

Experiments measure effects…

Here are two examplesSalk Vaccine Field Trial held certain key variables constant—protocol, blinding, etc.—while introducing the vaccine to the treatment group but not the controls. Randomization made the controls like the treatment group in important respects, except for the intervention, so that the difference in response is likely to be due to the effect of the vaccine itself.

HIP Screening Trials held certain key variables constant, while introducing screening to treatment and not the controls. Randomization made….so the difference in death rates is likely to be due to the lives saved by screening and early detection.

International Rice Research Institute, PhilippinesI. Subjects and the treatment

Subjects: 20 experimental plots planted with IR 8Treatment: 5 amounts of Nitrogen fertilizer

0 oz, 4 oz, 8 oz, 12 oz, 16 oz light, moisture, grade, and key nutrients held

constant

II. Randomization and the response Amount Nitrogen assigned randomly to plots Ounces of rice, or yield, is the response

III. Response schedule states the effect predicted yield = (20 oz rice per oz nitrogen ) × nitrogen +

240 oz

Summary

1. In observational studies, the investigators do not assign subjects to treatment or control…as in the case of smoker versus non-smokers.2. Observational studies can establish associations, which may point to causation: if exposure causes disease, then the exposed ought to be sicker than the unexposed. Why the caution? Confounding factors…3. With observational studies, and nonrandomized controlled experiments, try to find out how the subjects came to be in treatment or in control. Are the groups comparable? different? Were there confounding factors? adjustments? etc...4. Study design is a central issue in applied statistics, as shown here by contrasting controlled experiments with observational studies. The great weakness of observational studies is confounding; randomized controlled experiments minimize this weakness; when things go as planned, experiments measures the effects of interventions.

Supplements

The following slide provides supplements to our study of experiments and observational studies.

Design is central in applied statistics…Which studies include an intervention? self

selection?Which are observational? experimental?

Salk Vaccine Field Trial

Nurses Health Study of the “effects” of long term oral contraceptive use

Women Health Initiative of HRT

IRRI, Philippines, study of IR8

The next two go ahead…

Predicting a student’s 1st-year GPA from her Math SAT, again using linear regression

Predicting a man’s height from his weight, using linear regressionIf a man puts on 20 pounds, will he grow taller by 20 ×

0.047 inches per lb ≈ 0.9 inches? How does this case differ from a study of Hooke’s Law? (next slide)

Robert Hooke (England, 1653-1703)

Hooke’s Law…Hang a weight on a length of piano wire and it will stretch. In one experiment at Berkeley, it turned out that

length ≈ (0.05 cm per kg) × load + 439.1 cm

Increase the load by 2 kilograms, and the wire lengthens by about 2 × 0.05 = 0.10 centimeter.