bayesian statistics without tears: prelude eric-jan wagenmakers

59
Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmake rs

Upload: brian-mahoney

Post on 26-Mar-2015

226 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

Bayesian Statistics Without Tears: Prelude

Eric-Jan

Wagenmakers

Page 2: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

Three Schools of Statistical Inference

Neyman-Pearson: α-level, power calculations, two hypotheses, guide for action (i.e., what to do).

Fisher: p-values, one hypothesis (i.e., H0), quantifies evidence against H0.

Bayes: prior and posterior distributions, attaches probabilities to parameters and hypotheses.

Page 3: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

A Freudian Analogy

Neyman-Pearson: The Superego. Fisher: The Ego. Bayes: The Id.

Claim: What Id really wants is to attach probabilities to hypotheses and parameters. This wish is suppressed by the Superego and the Ego. The result is unconscious internal conflict.

Page 4: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

Internal Conflict Causes Misinterpretations

p < .05 means that H0 is unlikely to be true, and can be rejected.

p > .10 means that H0 is likely to be true.

For a given parameter μ, a 95% confidence interval from, say, a to b means that there is a 95% chance that μ lies in between a and b.

Page 5: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

Two Ways to Resolve the Internal Conflict

1. Strengthen Superego and Ego by teaching the standard statistical methodology more rigorously. Suppress Id even more!

2. Give Id what it wants.

Page 6: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

What is Bayesian Inference?Why be Bayesian?

Eric-Jan

Wagenmakers

Page 7: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

What is Bayesian Inference?

Page 8: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

What is Bayesian Inference?

“Common sense expressed in numbers”

Page 9: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

What is Bayesian Inference?

“The means by which rational agents draw optimal conclusions in an uncertain environment”

Page 10: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

What is Bayesian Inference?

“The only statistical procedure that is coherent, meaning that it avoids statements

that are internally inconsistent.”

Page 11: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

What is Bayesian Inference?

“A method for rational updating of beliefs about the world”

Page 12: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

What is Bayesian Inference?

“The only good statistics”

Page 13: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

Outline

Bayes in a Nutshell The Inevitability of Probability Bayesian Revolutions This Course

Page 14: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

Bayesian Inferencein a Nutshell

In Bayesian inference, uncertainty or degree of belief is quantified by probability.

Prior beliefs are updated by means of the data to yield posterior beliefs.

Page 15: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

Bayesian Parameter Estimation: Example

We prepare for you a series of 10 factual true/false questions of equal difficulty.

You answer 9 out of 10 questions correctly. What is your latent probability θ of

answering any one question correctly?

Page 16: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

Bayesian Parameter Estimation: Example

We start with a prior distribution for θ. This reflect all we know about θ prior to the experiment. Here we make a standard choice and assume that all values of θ are equally likely a priori.

Page 17: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

Bayesian Parameter Estimation: Example

We then update the prior distribution by means of the data (technically, the likelihood) to arrive at a posterior distribution.

The posterior distribution is a compromise between what we knew before the experiment and what we have learned from the experiment. The posterior distribution reflects all that we know about θ.

Page 18: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

Mode = 0.9

95% confidence interval: (0.59, 0.98)

NB. We do not have to use the uniform prior!

Page 19: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

Outline

Bayes in a Nutshell The Inevitability of Probability Bayesian Revolutions This Course

Page 20: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

The Inevitability of Probability

Why would one measure “degree of belief” by means of probability? Couldn’t we choose something else that makes sense?

Yes, perhaps we can, but the choice of probability is anything but ad-hoc.

Page 21: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

The Inevitability of Probability

Assume “degree of belief” can be measured by a single number.

Assume you are rational, that is, not self-contradictory or “obviously silly”.

Then degree of belief can be shown to follow the same rules as the probability calculus.

Page 22: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

The Inevitability of Probability

For instance, a rational agent would not hold intransitive beliefs, such as:

Bel A Bel B

Bel B Bel C

Bel C Bel A

Page 23: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

The Inevitability of Probability

When you use a single number to measure uncertainty or quantify evidence, and these numbers do not follow the rules of probability calculus, you can (almost certainly?) be shown to be silly or incoherent.

One of the theoretical attractions of the Bayesian paradigm is that it ensures coherence right from the start.

Page 24: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

Coherence I

Coherence is also key in de Finetti’s conceptualization of probability.

Page 25: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

Coherence II

One aspect of coherence is that “today’s posterior is tomorrow’s prior”.

Suppose we have exchangeable (iid) data x = {x1, x2}. Now we can update our prior using x, using first x1 and then x2, or using first x2

and then x1. All the procedures will result in exactly the

same posterior distribution.

Page 26: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

Coherence III

Assume we have three models: M1, M2, M3. After seeing the data, suppose that M1 is 3

times more plausible than M2, and M2 is 4 times more plausible than M3.

By transitivity, M1 is 3x4=12 times more plausible than M3.

Page 27: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

Outline

Bayes in a Nutshell The Inevitability of Probability Bayesian Revolutions This Course

Page 28: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

The Bayesian Revolution

Until about 1990, Bayesian statistics could only be applied to a select subset of very simple models.

Only recently, Bayesian statistics has undergone a transformation; With current numerical techniques, Bayesian models are “limited only by the user’s imagination.”

Page 29: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

The Bayesian Revolutionin Statistics

Page 30: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

The Bayesian Revolutionin Statistics

Page 31: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

The Bayesian Revolutionin Psychology?

Page 32: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

Are Psychologists Inconsistent?

The content of Psych Review shows that Psychologists are happy to develop Bayesian

models for human cognition and human behavior based on the assumption that agents or people process noisy information in a rational or optimal way;

But psychologist do not use Bayesian models to analyze their own data statistically!

Page 33: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

Why Bayes is Now Popular

Markov chain Monte Carlo!

Page 34: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

Markov Chain Monte Carlo

Instead of calculating the posterior analytically, numerical techniques such as MCMC approximate the posterior by drawing samples from it.

Consider again our earlier example…

Page 35: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers
Page 36: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

Mode = 0.89

95% confidence interval: (0.59, 0.98)

With 9000 samples, almost identical toanalytical result.

Page 37: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers
Page 38: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

Want to Know MoreAbout MCMC?

Page 39: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

MCMC

With MCMC, the models you can build and estimate are said to be “limited only by the user’s imagination”.

But how do you get MCMC to work? Option 1: write the code it yourself. Option 2: use WinBUGS!

Page 40: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

Outline

Bayes in a Nutshell The Inevitability of Probability Bayesian Revolutions This Course

Page 41: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

A Workshop in Bayesian Modeling for Cognitive Science

Eric-Jan Wagenmakers

Page 42: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

The Bayesian Book

…is a course book used at UvA and UCI. …is still regularly updated. ….is freely available at my homepage, at

http://www.ejwagenmakers.com/BayesCourse/BayesBook.html

…greatly benefits from your suggestions for improvement! [e.g., typos, awkward sentences, etc.]

Page 43: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

Contributors

Michael Leehttp://www.socsci.uci.edu/~mdlee/

Page 44: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

Contributors

Dora Matzke

Page 45: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

Contributors

Ruud Wetzelshttp://www.ruudwetzels.com/

Page 46: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

Why We Like Graphical Bayesian Modeling

It is fun. It is cool. It is easy. It is principled. It is superior. It is useful. It is flexible.

Page 47: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

Our Goals These Weeks Are…

For you to experience some of the possibilities that WinBUGS has to offer.

For you to get some hands-on training by trying out some programs.

For you to work at your own pace. For you to get answers to questions when

you get stuck.

Page 48: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

Our Goals These Weeks Are NOT…

For you become a Bayesian graphical modeling expert in one week.

For you to gain deep insight in the statistical foundations of Bayesian inference.

For you to get frustrated when the programs do not work or you do not understand the materials (please ask questions).

Page 49: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

Want to Know MoreAbout Bayes?

Page 50: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

Want to Know MoreAbout Bayes?

Page 51: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

WinBUGS

Bayesian inference Using

Gibbs Sampling

You want to have thisinstalled (plus the registration key)

Page 52: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

WinBUGS

Knows many probability distributions (likelihoods);

Allows you to specify a model; Allows you to specify priors; Will then automatically run the MCMC

sampling routines and produce output.

Page 53: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

Want to Know MoreAbout WinBUGS?

Page 54: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

WinBUGS & R

WinBUGS produces MCMC samples. We want to analyze the output in a nice

program, such as R. This can be accomplished using the R

package “R2WinBUGS”

Page 55: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

R: “Here’s the data and abunch of commands”

WinBUGS: “OK, I did what you wanted, here’s the samples you asked for”

Page 56: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

Getting Started

Work through some of the exercises of the book.

Most of you will want to get started with the chapter “getting started”.

Page 57: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

Running the R programs

The R scripts have extension .R. You can use “File” -> “Open Script” to read these.

You can run these scripts by copying-and-pasting the scripts in the R console.

Page 58: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

Course Webpage

Check out

http://www.ejwagenmakers.com/BayesCourse/BayesCourse.html

for lectures and a pdf file with answers to the exercises!

Page 59: Bayesian Statistics Without Tears: Prelude Eric-Jan Wagenmakers

Questions?

Feel free to ask questions when you are really stuck.