from question to hypothesis vincent bonagura m.d. jack hausman professor of pediatrics professor of...

From Question to Hypothesis

Vincent Bonagura M.D.Jack Hausman Professor of Pediatrics

Professor of Molecular MedicineHofstra-NS-LIJ School of MedicineAssociate Chair Dept. Of Pediatrics

Chief, Division of Allergy and Immunology

Course Specific Objectives:

• By the end of this course, you should be able to do the following:– Design a research project– Evaluate the results of this project and

present them– Evaluate a research report:

• determine relevance • strengths, weaknesses of design and analysis• general significance

Sequence of Epidemiologic

and Clinical Research

Anecdotal Observation

Case-Control Study

and/or Cohort Study

Animal Study Clinical Trial

Copyright 2000-2002

Hypothesis

Generating an Hypothesis:

• Hypothesis: very important word in a researcher's vocabulary. Use it with respect!

• An hypothesis has the following characteristics:– It explains or accounts for a set of facts– It can be tested by further investigation – It should predict results

Generating an Hypothesis:

• Other terms that are used in place of hypothesis: – Model– Theory

• In both cases, the key elements are the power to explain and the ability to be tested.

Some Logic re: Hypothesis:

• Consider hypothesis “h”, test “t”, and result “r”

• They are related in the following way:• Hypothesis “h” predicts the result “r”

when a test “t” is performed• If the hypothesis “h” is true, then the

result “r” will occur:

t : h → r

What does Result “r” say about the truth of Hypothesis “h”?

• If t : h → r, can we conclude that t : r → h? – if r is true , then the hypothesis is true?

NO!• Many different hypotheses can predict the

same outcome of a particular test.• Thus, a test cannot prove a hypothesis:• The most that t: h → r allows us to say is that

result “r” supports the hypothesis “

Falsification (not a dirty word!)

• Consider what happens if result “r” does not occur.

• The negative result, “-r” disproves or falsifies the hypothesis:

t : -h → -r • Thus, a positive test result cannot prove

or verify an hypothesis, it can only support it.

• However, a negative test result can disprove an hypothesis.

Distinguishing Alternative Hypotheses

• Experimental tests can distinguish between alternative hypotheses.

• Consider hypotheses “h1” and “h2” that predict opposite results, “r” and “-r” for the same test “t”.

t : h1 → r vs. t : h2 → -r • Result “r” and “–r” are mutually

exclusive, i.e. only one can occur.• Thus, “h1” or “h2” will be falsified

Quote from Medawar:

• If an experiment does not hold out the possibility of causing one to revise one’s views, it is hard to see why it should be done at all.

• Another way of saying this is:– A good hypothesis suggests ways of

disproving itself– A good experiment can yield at least 2

outcomes, one of which will be falsify the underlying hypothesis.

But life is not so simple…

• Preceding view of hypothesis testing works very well for small experiments where most of the variables can be controlled or at least accounted for.

• Many experiments are not capable of falsifying the underlying hypothesis; they can only support them.

• Thus, one can pose a contrasting view about falsification and hypothesis testing.

For example:• Hypothesis: “all swans are white”. • Observing any number of white swans will only serve

to confirm the hypothesis.• But observing only 1 back swan is enough to

disprove the hypothesis.• Well….• Suppose you look closely at the swan and see that it

is black because it is covered in oil.• Does this swan disprove the hypothesis? • Suppose you find a swan that is almost all white but

has a small black spot• Is this enough to disprove the hypothesis?

The “null” hypothesis: • The # of possible hypotheses that can

explain a given set of facts is infinite.• One hypothesis has a special status; it

is the null hypothesis “nh” • The “nh” predicts that a test will show

no difference: “t: nh → no difference”• What if test “t” reveals a difference • (t = -nh)?

– Then the nh is falsified. This concept will be addressed when we discuss statistics later on in the course.

Testing an Hypothesis

• Question put to: Possible Answers:

• Reality

• Experiment

Yes No

Yes No

Testing an Hypothesis

• Experimentation explicitly assumes the existence of a reality or truth that is “out there” and independent of our experimental manipulations.

• Assume the question has been formulated so that the possible answers are “yes” and “no.

• Posing the question to reality and to experimentation yields the following possibilities:

Testing an Hypothesis

• Question put to: Reality:

• Experiment Yes

No

OK Error: Type 1

(false +)

Error: Type 2

(false -)

OK

There are 2 ways that our experimental result can “match reality, and 2 ways that we can make an error.

This is called a 2X2 table or “contingency table”

Distinguishing the Errors:

• Chicken Little, i.e. “which came first the chicken or the egg?” made the first type of error when he said: “The sky is falling.”

• Captain Smith (of the Titanic) made the second type of error when he said: “There aren’t any icebergs out there”

Sequence of Epidemiologic

and Clinical Research

Anecdotal Observation

Case-Control Study

and/or Cohort Study

Animal Study Clinical Trial

Copyright 2000-2002

Hypothesis

What about research that is not-hypothesis-driven?

• Is it ever necessary or appropriate to carry out research that is not hypothesis-driven?

• This is referred to as a “fishing expedition”.

• In most circumstances, a fishing expedition is the “kiss of death” in a grant proposal.

What about research that is not-hypothesis driven?

• In some instances, a narrowly defined hypothesis-driven proposal can be less advantageous than a less focused or “modified” fishing expedition.

• Is there really any non-hypothesis driven question?

What about research that is not-hypothesis driven?

• The case for “Broad” vs. “Explicit” Hypotheses– A “Broad Hypothesis is one in which a general

proposition is put forth : We hypothesize that we will be able to identify a set of light chain genes whose expression is differentially regulated in the different B-cell subsets.”

– An “Explicit” hypothesis might focus on a particular light chain of interest: Light chain 12-46*01, is differentially regulated between the transitional, follicular, and marginal zone subsets”

What about research that is not-hypothesis driven?

• The case for “Broad” vs. “Explicit” Hypotheses– A “Broad Hypothesis: the emphasis is on

understanding data structure and patterns, rather than on p-values and confidence intervals. Observations made first, and then explicit hypotheses are generated and tested, based on observed data.

– Moreover, issues of statistical power or precision is not particularly relevant in studies involving broad hypotheses.

What about research that is not-hypothesis driven?

• The case for “Broad” vs. “Explicit” Hypotheses– A “Explicit Hypothesis: generated y selecting

differences/associations that appear “interesting” or “significant” and then applying standard statistical tests to the data observed.

– Statistical analyses carried out in this way creates a bias that inflates the Type 1 error rate of the test, i.e. a test that was supposed to be carried out at say, 5% significance level is actually being carried out at a higher level.

What about research that is “not-hypothesis” driven?

• Consider a RNA expression analysis test (gene chip) based on the question: if I treat cells with mediator X, something will happen to gene expression. – Is this an hypothesis?– This question is not narrowly focused re: what

genes of interest, and 2) whether they will be activated or inhibited, but it is in a broad way an hypothesis.

• (General Hypothesis)

• However, once you know the “lay of the land” and have the universe of all genes affected in this test, you can now develop a more “explicit” hypothesis based on these results.

Can a “good” hypothesis be too good ?

• The more the hypothesis explains and predicts, the more difficult it is for the researcher to design critical experiments that can disprove the hypothesis.

• So what is the problem?

Can a “good” hypothesis be too good ?

• The more the hypothesis explains and predicts, the more difficult it is for the researcher to design critical experiments that can disprove the hypothesis.

• So what is the problem?

Homework• Propose an hypothesis from the

observations/data you are collecting on your clinical/bench research.

• What will your hypothesis explain?• How can you test your hypothesis?• What will you conclude if the test you are

performing falsifies your hypothesis?• What would you do next if this happens?• Discuss with your mentor, have him/her sign

your homework and then submit to my secretary (Mrs. Pat Bittner (pbittner@nshs.edu) and your Program Director for your file.

Homework• Reading:

• National Academy of Sciences. On being a scientist. “The nature of scientific research”.

• PW Medawar. Advice to a Young Scientist, Chapter 9. Experiment and Discovery” and Chapter 11, “The Scientific Process”