Probability

The Birthday Paradox

The Birthday ParadoxHow many people would you need in a room to have at

least two of them share the same birthday.

The answer is less a paradox and more a surprise, and points up the trouble people have with probability – they think linearly when they should be thinking exponentially.

So here’s the answer, then I’ll discuss why:

Number of people to be 100% sure: 367*.(*If we account for leap years)

Number of people to be 99.99997% sure: 100.Number of people to be 97% sure: 50.

Number of people to be 50.7% sure: 23.

Surprised? You should be.

The Power of Powers in ProbabilityOnly 23 people to be over 50% sure? And only 50 to be

97%? Come on! So how does that work.

Let me start with another question with a surprising answer: in a coin toss, what are the odds of getting 10

heads in a row?

You might reason that since the odds are 50% of getting one head then it must be 50%/10=5% and you'd be wrong.

You’re wrong because you thought linearly – in probability you don’t divide – you multiply and that means exponents.

So the answer is .510 = 0.00097%

As long as you remember the power of powers then you can understand why probability works.

The Birthday Paradox – By The Numbers

The mistake people make (and why they never believe you about the birthday paradox) is that they always think of the question in the wrong

way.

It’s NOT the chance of someone sharing YOUR birthday.

It’s the chance of ANY TWO PEOPLE sharing A birthday.

But let’s start with you.

The Birthday Paradox – By The NumbersThe probability of someone sharing your birthday is

1/365th or 0.27% - very small. The arithmetic is:

1-(364/365)n

Where ‘n’ would be the number of people for a given probability of having one of them share.

Thus to have 50% odds of someone sharing YOUR birthday would require 253 people:

1-(364/365)253 = 0.50047 or 50.05%

Even in a room of 365 people the odds are only 63% someone shares YOUR birthday, and you’d need 1,000

people to get a 93% chance.

If the 58 people who are still in this course were all here, the odds would be about 15% that two of you share a birthday.

[1-(364/365)58 = 0.15009 or 15%]

The Birthday Paradox – By The ArithmeticLet’s start the arithmetic by still asking the wrong

question: if there were 23 people in a room including you, what’s the chance one would share your birthdate?

Easy answer: compare your birthday with the other 22 people. That’s 22 comparisons.

Since you don’t care if two other people share, the arithmetic is easy: 1-(364/365)22 or @ 6%

Now ask the correct question:Do any two in the room share a birthday?

That’s a little more complicated.

There are 253 pairs of people to

compare: (23*22)/2=253.

We divide by 2 to eliminate the

duplicate pairs (you on her and her on you) and

use 22 to eliminate non-

pairs (you on you, etc).

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 231 x2 x3 x4 x5 x6 x7 x8 x9 x

10 x11 x12 x13 x14 x15 x16 x17 x18 x19 x20 x21 x22 x23 x

Duplicate Pairs.

Person on Person

Countable Pairs.

Number of Countable Pairs Among 23 people

The chance of any 2 people having the same birthday:

1/365 = 0.00274 = 0.274%

And thus having different birthdays:

1-1/365 = 1 - 0.00274 = 0.99726 = 99.726%

But that’s just for one pair of people and one day.

So, to finish the calculation…

Chance of Any 2 People Sharing Birthday

The chance of any 2 people out of 23 having the same birthday now has to be calculated through for the 23

people and 253 other possible pairs – that is, it has to be raised to the power of 253.

So to finish this off in the language of probability:

p(n) = 1-(364/365) = 50.04%

Or the probability of 2 people out of 23 sharing a birthday is about 50%.

Chance of Any 2 People Out of 23 Sharing A Same Birthday

Some other factoids deriving from the birthday surprise:

The sqrt(n) is roughly the number you need to have a 50% chance of a match with n items. Sqrt(365) is about 20. This

comes into play in cryptography for the birthday attack.

ExampleYou only need @ 5 people picking letters from the

alphabet [sqrt(26)=5.1] to have a 50% chance of a match.

MoralExponentiation (multiplying) rapidly decreases the chance of picking unique items (that is, it increases the chances of

a match). Remember: exponents (powers) are non-intuitive and are the heart of probability!

The Birthday Paradox

Bayesian Versus Frequentist Statistics

Bayesian statistics (named after Thomas Bayes) has been around since the 1700s and recently has seen a

surge in interest because it offers a different perspective to doing statistics than the so called

frequentist approach.

In fact, Bayesians would say that the results of using their approach can be opposite to the results gained

from a frequentist approach.

But it is more intuitively difficult to understand than the frequentist approach normally used.

Bayesian Versus Frequentist Statistics

Bayesian stats is based on Bayes' Law and is all about relating prior knowledge or evidence to current (or

posteriori) knowledge or evidence about a situation. That is:

That the probability of something is related to the probability of something else. This is called the prior

condition.

Once you know the probability of that “something else”, it changes the way you judge the probability of the something

. This is called the current (or posteriori) condition.

Bayesian: probability measures degree of belief.Frequentist: probability measures proportion of outcomes.

Bayes’ Law (or Theorem, or Rule)

Another way to look at the two types of stats is to look at the way they treat uncertainty because statistics is

all about quantifying uncertainty.

Frequentists use ‘can’t know’ uncertainty, which results from random processes: e.g. you can’t know which number will come up when you throw a die.

But if you can quantify the system (the die – it has six faces) you can estimate the chances of something

happening (each face has 1/6th chance of coming up).

You still can’t know which number will come up but you know the chance of it coming up.

“Can’t Know” and “Don’t Know” Uncertainty

Bayesians use ‘don’t know’ uncertainty. That is, they are happy to change prior probabilities if they

receive subsequent information that might lead them to do so.

That is, they believe the same things about throwing a die – until it is thrown, and then their

beliefs can change if the die looks unfair.

In fact, they call the terms in their equations a priori and a posteriori knowledge.

Before we go there here’s an example:

“Can’t Know” and “Don’t Know” Uncertainty

A horse wins 5 out of 12 races, and therefore its chance of winning its next race is 5/12 or 41.6%.

For frequentists, this is where the analysis stops. They can’t know what’s behind the curtain.

But 3 of its 5 wins are when its raining, so now 3/5 = 60%.Yet when its raining it wins 3 out of 4, so 3/4 = 75%.

(Note that P of horse winning given that it is raining is not the same as the P of it raining when horse wins!)

For Bayesians it depends on the “don’t know (yet) probabilities under the curtain.

“Can’t Know” and “Don’t Know” UncertaintyRaining Not raining TOTAL

Horse #1 wins 3 2 5Horse #1 loses 1 6 7

TOTAL 4 8 12

The don’t know/can’t know

curtain.

If research question is about how often the horse wins, or even how often it wins in the rain, Bayesians

would also want to account for:

Wins overallLoses overallWins in rains

Wins not rain Loses in rain

Loses in not rain

Bayes is as much about how many times the horse loses as wins.

Winning, Losing, Not Winning, Not Losing

Bayes’ Law Formula PWhere:P(A) is what you know about A at first (called the prior probability).P(A│B) is what you know about A after learning something about B (called the posteriori probability).P(B│A)/P(B) is the support B provides for making decisions about A.

It usually involves something called the Bayes’ Factor (or ratio), expressed in terms such as:A:B

You visit Arizona and you want to go hiking tomorrow. In recent years, it has rained only 5

days each year.

Unfortunately, the weatherperson has predicted rain for tomorrow., and…

When it actually rains, the she correctly forecasts rain 90% of the time, but…

When it doesn't rain, she incorrectly forecasts rain 10% of the time.

What is the probability that it will rain tomorrow?

Example of Bayes’ Law

Event A1: It will rain tomorrow.P( A1 ) = 5/365 =0.0137

[It rains 5 days out of the year.]

Event A2: It will not rain tomorrow.P( A2 ) = 360/365 = 0.986

[It does not rain 360 days out of the year.]

Event B. The weatherperson predicts rain, but…P( B | A1 ) = 0.9

[When it rains, she is right 90% of the time.]

P( B | A2 ) = 0.1[When it does not rain, she is wrong 10% of the time.]

Example of Bayes’ Law

P( A1 | B ) =

P( A1 ) P( B | A1 )_____________________________P( A1 ) P( B | A1 ) + P( A2 ) P( B | A2 )

P( A1 | B ) = [(0.014)(0.9) / [ (0.014)(0.9) + (0.986)(0.1)]

P( A1 | B ) = 0.111

Plug in the numbers and calculate through:

And the real chance of rain tomorrow?Once again counterintuitive - 11% and not the 90% the weatherperson (and frequentist stats) led you

to believe.

Example of Bayes’ Law

Here’s another puzzle, a famous one in conditional Bayesian probabilities, completely non-intuitive and with

a very surprising answer.

It’s a game show and you’re offered three doors.Behind one is a million dollars.

Behind the other two are goats.You choose door #1 but DO NOT open it.

The host of the show opens one of the other doors and there is a goat.

He now offers you the chance to change your mind about your choice of door.

The question is, should you change your choice and switch to the other unopened door?

The Monty Hall Puzzle

Once again the answer is surprising – you should switch because now your odds have doubled that the money is

behind the other door!

How can that be?

Most people calculate that with two doors left, the odds are 50/50 so why switch?

But the real odds are now 33/66 that the other door (the one you didn’t choose) is the correct one.

It works using Bayes’ Law.

The Monty Hall Puzzle

The Monty Hall PuzzleDoor

#1Odds 1/3

Door #2

Odds 1/3

Door #3

Odds 1/3

Cumulative Odds 2/3

Door #2

Odds 2/3

Cumulative Odds 2/3

Using Bayes’ TheoremProbabilities before door 3 is opened (prior condition):

1:1:1

Probabilities after door 3 is opened (current condition):1:2:0

Therefore your probability of winning doubles if you switch.

The Monty Hall Puzzle - ConditionalityYour choice of door is not conditional on Monty’s choice of door (you

chose first without prior knowledge of his choice).But your decision to switch or not is now conditional on your current

knowledge of his choice.

Monty’s choice of door is conditional on your choice of door (he chose after, with the current knowledge of your choice).

XDoor

#1Odds 1/3

Door #2

Odds 1/3

Door #3

Odds 1/3

Choose a Door

Door 1Goat

Door 2Goat

Door 3Car

Stick Stick StickChange Change Change

Goat GoatGoatCar Car Car

This diagram clearly shows that if you change your selection, 2 times out of 3 you will win the car. If you don’t change your

selection, you will only win the car 1 time in 3. You are therefore twice as likely to win if you change your selection

If you choose this door and

you …

If you choose this door and

you …

If you choose this door and

you …

Choose one red ace and two black aces from a deck.Lay them face down, making note of where the red ace is.

Do not show the cards.Ask someone to point to a card.

Now flip one of the remaining cards that has the black ace.Ask them to bet you whether their chosen card is the red one

The probability is 66% that it is not, but they will think it is 50/50.

Here’s another simulator where you can test Bayes’ Law:

SIMULATOR:http://www.curiouser.co.uk/monty/montygame.htm

Bayes’ Law Simulation(aka how to win money at cards – no, seriously)

Statistics and Probability

Statistics and Probability

The formal study of the laws of chance is called probability theory.

Quantifying uncertainty is an essential part of statistics and this requires an understanding of

probability.

There are three areas where uncertainty enters into statistics.

Statistics and Probability First

Inferential statistics deals with samples that only represent a population and thus there will always be

uncertainty about how closely your sample does represent the population.

SecondSampling theory is based on the idea that sample

selection is regulated by chance.

ThirdYou are dealing with a limited number of variables, data from those variables, and information from those data. Thus there are many places where uncertainty exists.

Some Definitions

A random experiment is whatever it is you are doing to answer a research question – for

example, surveying a sample of the population or throwing a pair of dice.

Within a random experiment you will create a sample space comprised of a limited set of

elementary outcomes.

These in turn combine into an unlimited set of events.

Some DefinitionsExamples of sample space:

Sample of a population.(E.G. The numbers 1 to 6 on each of the die in a pair of

dice.)

Examples of elementary outcomes:Results of your questions or observations.

(E.G. The probability of each of the 36 combinations of pairs of numbers.)

Examples of events:A particular response to a question.

(E.G. A particular number from the dice – say a 7 or a 3.)

Number on the die. 1 2 3 4 5 6

Chance of getting it.

1/6th or

16.6% or

.166

1/6th

or16.6%or .16

6

1/6th

or 16.6%

or.166

1/6th or

16.6% or .16

6

1/6th or

16.6% or .16

6

1/6th or

16.6% or .16

6

Definitions Example: throwing one dieSAMPLE SPACE

ELEMENTARY OUTCOMES

Probabilities are expressed as a decimal between 0 and 1.

In this example the probability of any number would be expressed as P(Oi) = 0.166 or in words, the probability (P) of an

outcome (O) for a number (i) is equal to .166 or about 16.6%.

RANDOM EXPERIMENT

Expressing ProbabilitiesThe formal language used to express probabilities is defined by (1) the sample space with which you are working, and (2) the

finite range of probabilties.

It all takes place within a finite range between 0 and 1.0.0 --------------------------0.5--------------------------- 1.0

No chance 50% chance Certainty (0%) (100%)

The reasons for this are:All random experiments, by definition, end up with a result or

they wouldn’t be experiments.

The chance of a result has to be somewhere between 0 and 1.That result will be between one end of your sample space and

the other….

Expressing ProbabilitiesIn the single die random experiment you know that the

probability of getting a number is one sixth or 0.166, which in percentage terms is about 16.6%. That is

written as:P(Oi) = .166 where Oi is a number between and including 1 to 6.

Since there cannot be any other number, the sample space can be written as:P(Oi) ≥ 1 ≤ 6: that is Oi is a number between and

including 1 to 6

Since probability is as much about what’s not going to happen, then the P of a number not coming up is 1-

the probability of it coming up, or 1- P(Oi).

Number of the die.

1 2 3 4 5 6

Chance of getting it.

P(Oi)

1/6th or

16.6% or .16

6

1/6th or

16.6% or .16

6

1/6th or

16.6% or .16

6

1/6th or

16.6% or .16

6

1/6th or

16.6% or .16

6

1/6th or

16.6% or .16

6

Chance of not getting it.

1-P(Oi)

5/6th or

83.4% or .83

0

5/6th or

83.4% or .83

0

5/6th or

83.4% or .83

0

5/6th or

83.4% or .83

0

5/6th or

83.4% or .83

0

5/6th or

83.4% or .83

0

All probabilities associated with throwing one die

SAMPLE SPACE

ELEMENTARY OUTCOMESNote that these also include the probability

of not getting a specific elementary outcome, expressed as 1-P(Oi)

Probabilities when using a pair of dice

Principles are the same, but the table has to have dimensions of 6X6 or 36 cells each comprised of a pair

of numbers in combination.

The probability of any given combination is 1/36th or about 0.028 (1 divided by 36) or 2.8%.

The odds of not getting a given combination are 1-0.028 or 0.972 or 97.2%.

The following table illustrates all of the combinations of numbers you can get with a throw of a pair of dice, with

the shaded columns representing the second die.

6 6 6 5 6 4 6 3 6 2 6 1 5 6 5 5 5 4 5 3 5 2 5 1 4 6 4 5 4 4 4 3 4 2 4 1 3 6 3 5 3 4 3 3 3 2 3 1 2 6 2 5 2 4 2 3 2 2 2 1 1 6 1 5 1 4 1 3 1 2 1 1

Shaded numbers are the second die

There is 1 chance in 36 that any given combination will come up

This is the sample space

These are examples of elementary outcomes:

5+6=115+2=73+4=72+2=4

Expressing Probabilities – Two DiceIn the two dice random experiment the probability of getting a number is 1/36th or about 0.028 (1 divided

by 36) or 2.8%. That is written as:P(Oi) = .028 where Oi is a number between and including 2 to 12.

Since there cannot be any other numbers, the sample space can be written as:P(Oi) ≥ 2 ≤ 12: that is Oi is a number between and

including 2 to 12.

Since probability is as much about what’s not going to happen, then the P of a number not coming up is 1-

the probability of it coming up, or 1- P(Oi).

Summary Rules of ProbabilityFirst, you cannot have a negative probability.

Second, every elementary outcome (e.g. pair of numbers) has the same probability of occurring.

Third, the chance of an event not happening will be 1 minus the chance it will happen.

Fourth, (the corollary of the third point), the sum of all the probabilities of a sample set has to equal one.

These rules are usually written as the following probability expressions:

SOME PROBABILITY RULES

The Rule The Expression The Meaning

The Non-negative Rule P(Oi) ≥ 0Probabilities cannot be less

than zero. That is, they cannot be negative.

The Equality Rule P(O1) = P(O2) =….. P(On)

Each elementary outcome has an equal chance of occurring, as in throwing a die or pair of dice. The probability of events

(actual numbers) may certainly differ.

The Non-occurrence Rule (or NOT rule) P(NOT Oi) = 1- P(Oi)

The probability of an elementary outcome not occurring is 1 minus the

probability that it will occur.

The Unity Rule P(O1) + P(O2) +…+ P(On)=1

All the probabilities of all the elementary outcomes in an experiment must sum to 1

(unity is another word for 1 (or whole) in mathematics).

EventsSo far we have dealt only with the elementary

outcomes (Oi) of an experiment – for example, all possible pairs of numbers.

But you really want to know what’s the probability of getting a specific number with the dice – say a 7?

You might also want to ask other questions, such as getting specific combinations, or adding conditions such as “or” and “and”, or even not getting a number at all.

To do this you use events:

An event is a set of elementary outcomes where the probability of the event will be the sum of the

probabilities of its elementary outcomes.

But there are only 11 different numbers (2-

12), each with a different probability

of occurring.

There are 36 different pairs of

numbers that could come up, each with

the same probability of occurring.

6 6 6 5 6 4 6 3 6 2 6 1 5 6 5 5 5 4 5 3 5 2 5 1 4 6 4 5 4 4 4 3 4 2 4 1 3 6 3 5 3 4 3 3 3 2 3 1 2 6 2 5 2 4 2 3 2 2 2 1 1 6

1 5

1 4

1 3

1 2

1 1

For example, there are 6 pairs on the diagonal each with 1/36th chance of occurring, but they all add up to the number 7, which has 6*1/36th

or 1/6th or 0.16 or 16% chance of occurring.

Probabilities of Different Events Occurring With Two DiceEvent (Ei)

DescriptionEvent’s Elementary

OutcomesProbability of Event

Occurring

Throwing a 7 [1,6],[2,5],[3,4],[4,3],[5,2],[6,1]

[.028]+[.028]+[.028]+[.028]+[.028]+

[.028]=0.167 or 16.7% Throwing a 3 [1,2],[2,1] [.028]+[.028]=.056 or

5.6%White die shows

1 [1,1],[1,2],[1,3],[1,4],

[1,5],[1,6][.028]+[.028]+[.028]+

[.028]+[.028]+ [.028]=0.167 or 16.7%

Black die shows 4

[1,4],[2,4],[3,4],[4,4],[5,4],[6,4]

[.028]+[.028]+[.028]+[.028]+[.028]+

[.028]=0.167 or 16.7%

Getting a 7, black high [1,6],[2,5],[3,4] [.028]+[.028]+[.028]=.083

or 8.3%

Logical Operators OR, AND, NOTSome other questions are:

What is the probability that the black die will equal 1 OR the white die will equal 1?

What is the probability that the black die will equal 1 AND the white die will equal 1?

What is the probability that the black die will NOT equal 1?

Using the dice combinations table we can answer the questions above.

6 6 6 5 6 4 6 3 6 2 6 1 5 6 5 5 5 4 5 3 5 2 5 1 4 6 4 5 4 4 4 3 4 2 4 1 3 6 3 5 3 4 3 3 3 2 3 1 2 6 2 5 2 4 2 3 2 2 2 1 1 6

1 5

1 4

1 3

1 2

1 1

(Bottom row (blue numbers plus the green number) = white die showing 1).(Right column (red numbers plus green number) = black die showing 1).(Green combination has been double counted so must be subtracted once).

Probability of the black die OR the white die

showing a 1.

Event table for OR condition

Event Description Event’s Elementary Outcome

Probability of Event Occurring

White die shows 1OR

black die shows 1

[1,1],[1,2],[1,3],[1,4],[1,5],[1,6]

OR[6,1],[5,1],[4,1],[3,1],[2,1],[1,1]

[.028]+[.028]+[.028]+[.028]+[.028]+[.028]

+[.028]+[.028]+[.028]+[.028]+[.028]+[.028]-[.028] = 0.31 or 31%

Subtract one of the double counted 1,1s

Mutually Exclusive OR StatementsFormally the OR function for this event would be

written:

P(E1 OR E2) = P(E1) + P(E2) – P(E1 AND E2)If there are no overlapping events (i.e. the double counting of 1,1) the term P(E1 AND E2) equals zero so that with such mutually exclusive events as

they are called, the expression is simply:

P(E1 OR E2) = P(E1) + P(E2)An example of this would be E1 = the dice add to 6, E2 = the dice add to 3 - there are no combinations of

elementary outcomes that overlap for these two events.

Event Table for Mutually Exclusive OR condition

Event Description Event’s Elementary Outcome

Probability of Event OccurringE1

Two dice add to 6

[5,1],[4,2],[3,3],[4,2],[5,1]

[.028]+[.028]+[.028]+[.028]+[.028]=0.14 or 14.0%E2

Two dice add to 3

[2,1], [1,2] [.028]+[.028]=.056 or 5.6%

No double counted numbers to be subtracted.

6 6 6 5 6 4 6 3 6 2 6 1 5 6 5 5 5 4 5 3 5 2 5 1 4 6 4 5 4 4 4 3 4 2 4 1 3 6 3 5 3 4 3 3 3 2 3 1 2 6 2 5 2 4 2 3 2 2 2 1 1 6

1 5

1 4

1 3

1 2

1 1

What is the probability of throwing a 1 AND a 1?

Event table for AND condition

Event Description Event’s Elementary Outcome

Probability of Event Occurring

E2

White die shows 1 AND

black die shows 1

[1,1] [.028]=.028 or 2.8%

There is only one elementary outcome that can give a double one (or a double of any number

between 2 and 6).

Note that OR functions always produce larger probabilities than AND functions.

AND, the Special Multiplication Rule, and Joint Probability

The AND function can be used to calculate the probability of a specific series of events occurring (such

as 3,4,5 or 7,7,7).

The expression for the AND function follows what is called the special multiplication rule, stated as:

The probability of ‘n’ specific events occurring is the product of the probabilities of each event ‘n’ .

That is, the individual probabilities are multiplied to get the joint probability thus:

P(E1 AND E2) = P(E1)P(E2)

Example of AND and Joint ProbabilityWhat would be the probability of throwing three 7’s in a

row?

The probability of throwing one 7 (called event 1 or E1) is 0.167 or about 16.7%, and the probability of throwing

three of them in a row is:

P(E1 AND E1 AND E1) = P(E1)*P(E1)*P(E1)0.167*0.167*0.167 = 0.0047

or about 0.47%

POINT: The chance of a random event repeating itself decreases dramatically even in the case of individually

fairly common events such as throwing a 7.

The NOT Logical OperatorThe NOT logical operator is a handy tool for when you

want to know the probability of something not happening.

Assume that E1 is that a double-1 will not be thrown. Now the expression becomes:

P(E1) =1- P(NOT E1)Probability of E1 (a double 1) = 1/36th

Probability of NOT E1 = 1-1/36th =35/36ths or 0.972(35 divided by 36) or about 97.2%

And for not throwing three 7s in a row? 1-0.0047 or 99.53%.

When To Bet NOT

The thing about the NOT operator is you can rarely lose with it, if someone is careless enough to take a

NOT bet.

Even with throwing a 7, you have an 83% chance of winning if you bet someone they will not throw one.

Do not bet?

Do bet NOT.

The Power of Powers in ProbabilityWhat is the probability of guessing the correct answer to a

multiple choice question with 5 possible responses?

1 in 5 or 0.2 or 20%(Hmmm. Not bad. Maybe I’ll try that.)

Now, what is the probability of guessing them all correctly in a 40 question test?

0.0000000000000000000000000001099

Of not getting any?0.00013292279957849187

Half correct?0.00001666462278347505

LESSON FOR THE DAY

Do not bet unless you can bet NOT

(but not in multiple choice quizzes)

What is the probability that you will win Lotto

649 this week?1 in 13,983,816

(or about 0.000000007%.

What is the probability that someone will win Lotto 649 this week?

1 in 1.75(or about 57%)