understanding science 2. bayes’ theorem © colin frayn, 2012

Understanding Science

2. Bayes’ Theorem

© Colin Frayn, 2012www.frayn.net

Recap• Assumptions of science

a) Underlying lawsb) Accurate sensesc) Occam’s Razor

• Absolute proof– Can be achieved with mathematical claims– Difficult or impossible for scientific laws

• Spectrum of certainty– Science moves theories on the spectrum

• Scientific Theories– Empirical models– Well tested, predictive, falsifiable


Clarifications• “False” does not imply “completely wrong”

– E.g. Newtonian Physics vs. Relativity– E.g. the Flat Earth theory, the Spherical Earth theory

• Carl Sagan’s Dragon– Can we show it doesn’t exist?– Should we bother?

• Predictive laws versus specific statements– “There are no dragons”


Introduction• New evidence arrives

– What does that do?– Moving around the spectrum of certainty

• Prior knowledge– Did you see Elvis?

• When could we call something a “fact”?– A scientific fact is “near enough”!


Examples• On trial for murder

– DNA testing– Very accurate– …but a very large population

• A rare disease– Rare disease or rare misdiagnosis– Intuition doesn’t help


Organic Gravity – An ExampleOrganic gravity

”Gravity only acts on organic things”

Vs.

Newtonian gravity“Gravity acts identically on every type of object”

• Test 1 – drop an apple– Both theories are equal

• Test 2 – drop a stone– Newtonian gravity wins


In More Detail• Let’s look at what we just did

• Test 1 didn’t really help– It didn’t differentiate– It provided equal support to each

• Test 2 solved the issue– Distinguished between the proposals– Provided support to Newtonian theory


Equal Support• What do we do when we cannot

distinguish between two possibilities?

• Look at the prior probability of each

• Example: Diagnosing a rare disease1. The patient has a rare disease2. The test was wrong


Putting it all together...


Probability of a Hypothesisgiven the Evidence

Probability of a Hypothesisgiven the Evidence

P ( H | E )

Depends on...

1.The support that E gives to H2.The prior probability of H

Finally, Bayes’ Theorem


P (H | E) = P (E | H) * P (H)

P (E)Posterior

Support

Prior

Evidential Support• “How much does evidence E support

hypothesis H?”– P(E|H)/P(E)

• Eating garlic scares away vampires– Given that I don’t see any vampires

• P(E) = 1– Vampires don’t exist!

• P(E|H) is also 1– So test is useless– That is, it has no differentiating power


Non-discriminating Evidence


P ( | ) = P ( | ) * P ( )

P ( )1

Posterior probability is equal to the priori.e. We’ve learned nothing whatsoever

Priors• “What is the chance that our hypothesis might be

true ignoring the new evidence?”– P(H)

• A “flat prior” means “no preference”– P(H) is the same for all hypotheses

• The “status quo”– E.g. “Elvis is alive”– … or any other conspiracy theory– … or and pseudoscientific claim


Organic Gravity Revisited• Dropping an apple gave no preference

– P(H) = 0.5 for both


P(Newtonian | Stone Falls) =

P(Stone Falls | Newtonian) * P(Newtonian)

P(Stone Falls)

P(Organic | Stone Falls) =

P(Stone Falls | Organic) * P(Organic)

P(Stone Falls)

1 0.5

0.5

1

0

0.5

0.5

0

Assumptions• Assumption of completeness

– Don’t have to make this assumption– Though we do need some way to calculate P(E)

• Assumption that the evidence was accurate– Can factor this into P(E|H)

• Assumption that you understand your models– Do you really know P(E|H)?


Summary• Bayes theorem allows us to update

hypotheses in response to evidence

• It evaluates the support that evidence gives for a hypothesis

• It underlies all of science


understanding science 2. bayes’ theorem © colin frayn, 2012

Documents