hudm4122 probability and statistical inference - upenn.edu · flipping a fair coin flipping same...

HUDM4122Probability and Statistical Inference

February 16, 2015

In the last class

• We started Ch. 4.4 in Mendenhall, Beaver, &Beaver

Today

• Ch. 4.4-4.6 in Mendenhall, Beaver, & Beaver

Today

• Sampling without Replacement• Permutations• Combinations• Independence• Conditional Probability

We ended last class with this problem

• I call a radio station, where they make me pick anumber between 1 and 10

• If I get today’s winning number, I get free ticketsto hear Justin Bieber (oh lucky me)

• Let’s say I call 5 days in a row

• What is the probability I get tickets on exactly oneday? (my daughter will still be excited)

Solution A (professor)110 + 110 910 + 110 910 910+ 110 910 910 910+ 110 910 910 910 910= 0.40951

Solution B (class)110 ( 910) ∗ 5= 0.32805

These represent two differentproblems

• One (my solution) is sampling withoutreplacement

• The other is the sum of five independentevents

• So which one is right?

Let’s first compute the sample space

• I call a radio station, where they make me picka number between 1 and 10

• I call 5 days in a row

• 10*10*10*10*10 = 100,000

Let’s first compute the sample space

• I call a radio station, where they make me picka number between 1 and 10

• I call 5 days in a row

• 10*10*10*10*10 = 100,000• A.K.A. too many to list out

Let’s first compute the sample space

• I call a radio station, where they make me picka number between 1 and 10

• I call 5 days in a row

• 10*10*10*10*10 = 100,000• A.K.A. too many to list out

– Or is it?

bieber-tix-example.xlsx

bieber-tix-example.xlsx

• 0.32805

So, the professor was wrong

So, the professor was wrong

• It turns out sleep deprivation is bad forcognition

So, the professor was wrong

• It turns out sleep deprivation is bad forcognition

• Don’t try this on your midterm

So why was this correct?110 ( 910) ∗ 5

5 days to win 1 ticket110 ( 910) ∗ 5• CXXXX• XCXXX• XXCXX• XXXCX• XXXXC

5 days

In each of 5 days,1 answer where right110 ( 910) ∗ 5

5 daysCorrect answer

And 4 more days where wrong;and there are 9 wrong answers110 ( 910) ∗ 5

5 casesCorrect answer

Wrong answers

Wrong days

Each day is independent from other days –that’s why this is the correct math110 ( 910) ∗ 5

5 casesCorrect answer

Wrong answers

Wrong days

Questions? Comments?

The sample space for multi-stage datacollection can be calculated using

• The Extendedmn rule

The Extendedmn rule

• Let’s say you have k stages of your datacollection

The Extendedmn rule

• Let’s say you have k stages of data collection– Unlike the book, I don’t call it an experiment,

because that term is usually given a more specificmeaning by researchers

The Extendedmn rule

• Let’s say you have k stages of data collection• And there are n1 ways to accomplish the first

stage• And n2 ways to accomplish the 2nd stage• And n3 ways to accomplish the 3rd stage• And nk ways to accomplish the kth stage

• Then the sample space = n1 * n2 * n3 … * nk

Any questions about theExtendedmn rule?

• Let’s say you have k stages of data collection• And there are n1 ways to accomplish the first

stage• And n2 ways to accomplish the 2nd stage• And n3 ways to accomplish the 3rd stage• And nk ways to accomplish the kth stage

• Then the sample space = n1 * n2 * n3 … * nk

Note that there doesn’t have to be thesame probability in each stage!

This can come in useful in cases thatare not truly independent

• Unlike the Justin Bieber example

Independence

• Two events A and B are independent if A doesnot affect B and B does not affect A

Which of these are independent?A B

Flipping a fair coin Flipping same fair coin again

Which of these are independent?A B

Flipping a fair coin Flipping same fair coin againFlipping a biased coin Flipping same biased coin again

Which of these are independent?A B

Flipping a fair coin Flipping same fair coin againFlipping a biased coin Flipping same biased coin again

Bob parties late night beforemidterm

Bob falls asleep duringmidterm

Which of these are independent?A B

Flipping a fair coin Flipping same fair coin againFlipping a biased coin Flipping same biased coin again

Bob parties late night beforemidterm

Bob falls asleep duringmidterm

Bob’s grade on midterm Bob’s grade on final

Which of these are independent?A B

Flipping a fair coin Flipping same fair coin againFlipping a biased coin Flipping same biased coin again

Bob parties late night beforemidterm

Bob falls asleep duringmidterm

Bob’s grade on midterm Bob’s grade on finalBob disrupts class Bob gets expelled

Which of these are independent?A B

Flipping a fair coin Flipping same fair coin againFlipping a biased coin Flipping same biased coin again

Bob parties late night beforemidterm

Bob falls asleep duringmidterm

Bob’s grade on midterm Bob’s grade on finalBob disrupts class Bob gets expelledBob gets expelled Bob takes job at McDonald’s

Which of these are independent?A B

Flipping a fair coin Flipping same fair coin againFlipping a biased coin Flipping same biased coin again

Bob parties late night beforemidterm

Bob falls asleep duringmidterm

Bob’s grade on midterm Bob’s grade on finalBob disrupts class Bob gets expelledBob gets expelled Bob takes job at McDonald’s

Bob takes job at McDonald’s Bob wins lottery

Which of these are independent?A B

Flipping a fair coin Flipping same fair coin againFlipping a biased coin Flipping same biased coin again

Bob parties late night beforemidterm

Bob falls asleep duringmidterm

Bob’s grade on midterm Bob’s grade on finalBob disrupts class Bob gets expelledBob gets expelled Bob takes job at McDonald’s

Bob takes job at McDonald’s Bob wins lotteryBob wins lottery Bob becomes ill due to

congenital heart problem

Example of probability calculation withnon-independent events

• Let’s say that I invite 6 friends over to playBeer Hunter

This can come in useful in cases thatare not truly independent

• Let’s say that 6 friends decide to play BeerHunter

• Rules are– 6 cans of beer– 1 violently shaken before playing and then

shuffled– Each person chooses a can and opens it

Initial Sample Space = 6

• 1 Bad outcome• 5 Perfectly fine outcomes

– If you don’t like beer, imagine it’s root beer

Probability of bad outcome

• The probability of a bad outcome for friend 1is 1/6

Probability of bad outcome

• The probability of a bad outcome for friend 1is 1/6

• But if friend 1 comes out OK• The probability of a bad outcome for friend 2

is 1/5, not 1/6!

Probability of bad outcome

• The probability of a bad outcome for friend 1is 1/6

• But if friend 1 comes out OK• The probability of a bad outcome for friend 2

is 1/5, not 1/6!

• Does everyone see why?

Probability of bad outcome

• The probability of a bad outcome for friend 1is 1/6

• But if friend 1 comes out OK• The probability of a bad outcome for friend 2

is 1/5, not 1/6!

• Does everyone see why?– Friend 1 already opened a beer can, and it went

ok– This only leaves 5 closed beer cans

Probability of bad outcome

• If friend 2 comes out OK• The probability of a bad outcome for friend 3

is 1/4

Probability of bad outcome

• If friend 2 comes out OK• The probability of a bad outcome for friend 3

is 1/4

• If friend 3 comes out OK• The probability of a bad outcome for friend 4

is 1/3

Probability of bad outcome

• If friend 4 comes out OK• The probability of a bad outcome for friend 5

is 1/2

Probability of bad outcome

• If friend 4 comes out OK• The probability of a bad outcome for friend 5

is 1/2

• If friend 5 comes out OK

Probability of bad outcome

• If friend 4 comes out OK• The probability of a bad outcome for friend 5

is 1/2

• If friend 5 comes out OK• Friend 6 will need to get a clean shirt

True sample space

• 6 *5 * 4 * 3 * 2 * 1

• This is called sampling without replacement

Any questions?

Now you do an example

• In pairs

Now you do an example

• Let’s say I’m a roadie for the band Van Halen

Now you do an example

• Let’s say I’m a roadie for the band Van Halen– Professors need to moonlight to make ends meet

in this city

Now you do an example

• Let’s say I’m a roadie for the band Van Halen• They have a “no brown M&M’s” clause in their

contract, and if any of the four members get abrown M&M, I’m fired

Now you do an example

• Let’s say I’m a roadie for the band Van Halen• They have a “no brown M&M’s” clause in their

contract, and if any of the four members get abrown M&M, I’m fired

• I’ve just handed them a bowl with 20 M&Ms,including one brown M&M

• Each band member takes 1 M&M withoutlooking

Now you do an example• Let’s say I’m a roadie for the band Van Halen• They have a “no brown M&M’s” clause in their

contract, and if any of the four members get abrown M&M, I’m fired

• I’ve just handed them a bowl with 20 M&Ms,including one brown M&M

• Each band member takes 1 M&M without looking

• What is the sample space?• What is the probability I get fired?

Questions? Comments?

Another example: permutations

• An application of sampling withoutreplacement

• How many orderings can you have between acertain number of objects?

Example• Let’s say that I’m redecorating my office in

preparation for a visit from a funder from the USarmy (“Bob”), a funder from the National ScienceFoundation (“Janet”), and a funder from the USDepartment of Education (“Ed”)

• I want to place Bob’s book, Janet’s book, and Ed’sbook in a place of honor next to my desk

• How many different orders can I put their booksin?

Example

• The first book could be Bob’s, Janet’s, or Ed’s• If the first book is Bob’s, the second book can

only be Janet’s or Ed’s• If the first book is Bob’s, and the second book

is Janet’s, then the third book can only be Ed’s

• We’re back to the same math of 3*2*1

Any questions?

Formal equations

• The sample space for n stages, samplingwithout replacement, is

• n * (n-1) * (n-2) * (n-3) * (n-4) until (n-k)=1

• This is written n!– Pronounced “n factorial”

Formal equations

• The number of permutations for n objects,taking all of them together, is

• Still n!

• n * (n-1) * (n-2) * (n-3) * (n-4) until (n-k)=1

Do it yourself

• What is the number of permutations for 4objects?

• What is the number of permutations for 6objects?

• What is the number of permutations for 10objects?

Do it yourself• What is the number of permutations for 4

objects?– 4*3*2*1=24

• What is the number of permutations for 6objects?– 6*5*4*3*2*1=720

• What is the number of permutations for 10objects?– 10*9*8*7*6*5*4*3*2*1= 3,628,800

Ryan’s daughter suggests

• “Daddy, why don’t we try organizing all thebooks on your bookshelves in every possibleway?”

• I own approximately 600 books

• Is this a good idea?

Questions? Comments?

More general case

• If we only want to pick r objects out of the ntotal objects, the equation becomes

• ! !

Using general case equation

• If I want to find out how many orderings of 2books I can get from 6 total books– n!/(n-r)!– 6!/(6-2)!– 6!/4!– 720/24– 30 possible orderings

Using general case equation

• If I want to find out how many orderings of 4books I can get from 6 total books– n!/(n-r)!– 6!/(6-4)!– 6!/2!– 720/2– 360 possible orderings

Any questions?

Related problem: Combinations

• If we don’t care about order, but only want toknow how many combinations of items wecan get

Combination formula

• The number of combinations of r objects outof n total objects is

• !! !

Example

• I have five friends, and three tickets to seeFerrari Truck

• How many combinations of friends could Ipotentially bring?

Example

• I have five friends, and three tickets to seeFerrari Truck

• How many combinations of friends could Ipotentially bring?

• !! ! = !! ! = !!( !) = ( ) = = 10

Example

• I have 600 books, and want to take 3 books ona ridiculously long flight to the FirstUzbekistani Conference on Educational DataMining

• How many combinations of books could Ipotentially bring?

Example

• I have 600 books, and want to take 3 books on aridiculously long flight to the First UzbekistaniConference on Educational Data Mining

• How many combinations of books could Ipotentially bring?

• !! ! = !! ! = !! ! = ∗ ∗! =

• 35.8 million

Your turn

• Peter’s Pizzeria has 6 toppings, and a 2-topping special

• How many combinations of toppings couldyou get, and have the special?

Questions?

An application

• In my son’s play group, he has 6 playmates– 5 friends and 1 frenemy

• What is the probability that on a specificplaydate with 2 friends, it will involve thefrenemy?

Can be written as

• Number of playdates that involve frenemy = 5– Frenemy plus each of 5 friends

• Total number of combinations (2 of 6)

• !! ! = !! ! = !!( !) = ( ) = = 15

Can be written as

• Number of playdates that involve frenemy = 5– Frenemy plus each of 5 friends

• Total number of combinations (2 of 6) = 15

• 5/15 = 1/3

Any questions?

Conditional Probability

• Let’s take two non-independent events, A andB

• P(A | B) =• Probability of A,• Given that we know that B occurred

Conditional Probability

• Let’s take two non-independent events, A and B

• P(A | B) =• Probability of A,• Given that we know that B occurred

• Note that this tells us nothing aboutP(A | ~B)P(A) overall

General Multiplication Rule

• Probability of A and B equals• Probability of A• Multiplied by• Probability of B, given A

General Multiplication Rule

• Probability of A and B equals• Probability of A• Multiplied by• Probability of B, given A

• Formally• ∩ = ( | )

Example

• P(Q) = 0.2• P(R|Q) = 0.7

• P(Q ∩ R) = ?

Example

• P(Q) = 0.2• P(R|Q) = 0.7

• P(Q ∩ R) = 0.14

You try it

• P(F) = 0.7• P(G|F) = 0.5

• P(F ∩ G) = ?

You try it

• P(F) = 0.7• P(G|F) = 0.5

• P(F ∩ G) = 0.35

You try it

• P(X) = 0.3• P(X|Y) = 0.9

• P(X ∩ Y) = ?

You try it

• P(X) = 0.3• P(X|Y) = 0.9

• P(X ∩ Y) = ?

• Impossible to calculate from information given

A Concrete Example

• P(Bob parties late, night before exam) = 0.5• P(Bob does badly on exam | parties late) = 0.7• P(Bob does badly on exam | ~parties late) = 0.2

• What is the probability that Bob parties late anddoes badly?

A Concrete Example

• P(Bob parties late, night before exam) = 0.5• P(Bob does badly on exam | parties late) = 0.7• P(Bob does badly on exam | ~parties late) = 0.2

• What is the probability that Bob parties late anddoes badly?

• 0.35

Conditional Probability Formula

• A mathematical transformation of the GeneralMultiplication Rule

• That rule was

• ∩ = ( | )

Conditional Probability Formula

• If you divide both sides by P(A)

• ( ∩ )( ) = ( | )( )• Which resolves to

• ( ∩ )( ) = ( | )

Conditional Probability Formula

• P(B|A) = ( ∩ )( )

Conditional Probability Formula

• P(B|A) = ( ∩ )( )• Note that P(A) can’t equal 0, or you’re dividing

by 0…

Example

• P(B|A) = ( ∩ )( )• P(Bob parties late, night before exam) = 0.5• P(Bob does badly on exam AND parties late) =

0.35

• What is the probability that does badly, giventhat he parties late?

Example

• P(B|A) = ( ∩ )( )• P(Bob parties late, night before exam) = 0.5• P(Bob does badly on exam AND parties late) =

0.35

• What is the probability that does badly, given thathe parties late?– 0.7

You try it

• P(B|A) = ( ∩ )( )• P(Student is from a wealthy family) = 0.25• P(Student goes to college AND comes from

wealthy family) = 0.21

• What is the probability that a student goes tocollege, given that he/she comes from a wealthyfamily?

You try it

• P(B|A) = ( ∩ )( )• P(Student is from a wealthy family) = 0.25• P(Student goes to college AND comes from

wealthy family) = 0.21

• What is the probability that a student goes tocollege, given that he/she comes from a wealthyfamily?– 0.84

What if we want to determine

• P(B|A) = ( ∩ )( )• P(Student is from a wealthy family) = 0.25• P(Student goes to college AND comes from

wealthy family) = 0.21

• What is the probability that a student comesfrom a wealthy family, given that he/she went tocollege?

What if we want to determine

• P(B|A) = ( ∩ )( )• P(Student is from a wealthy family) = 0.25• P(Student goes to college AND comes from

wealthy family) = 0.21

• What is the probability that a student comesfrom a wealthy family, given that he/she went tocollege?– We can’t tell, using this formula…

Upcoming Classes

• 2/18 Bayes Theorem– Ch. 4-7– HW3 due

• 2/23 Discrete Random Variables and TheirProbability Distributions– Ch. 4-8

Homework 3

• Due in 2 days• In the ASSISTments system

Questions? Comments?