lecture 04: conditional probability - stanford university · 2018. 7. 3. · conditional...
TRANSCRIPT
Lecture04:ConditionalProbabilityLisaYanJuly2,2018
Announcements
ProblemSet#1dueonFriday◦Gradescope submissionportalup
UsePiazza
NoclassorOHonWednesdayJuly4th
2
Summaryfromlasttime
Samplespace,S:Thesetofallpossibleoutcomesofanexperiment.Eventspace,E:AsubsetofS.Ifwehaveequallylikelyoutcomes,thenP(E)=|E|/|S|.
Twokeytacticstocountingprobabilities:• DeMorgan’s lawandcalculatingP(Ec)helpsusavoidtrickycounting
situations.• Wecantreatindistinctobjectsasdistinctifithelpscreateequallylikely
outcomes.
3
Countsofballsandurns
Supposewehaven=2 balls,labeledAandB,areplacedinm=2 urns,Urns1and2,whereeachballisequallylikelytobeplacedinanyurn.
4
Urn 1 Urn 2
A,B -A BB A- A,B
Possibilities
Equallylikely
Urn 1 Urn 2 Prob
2 0 1/41 1 2/40 2 1/4
Counts
Notequallylikely
Goalsfortoday
Conditionalprobability◦Definitionofconditionalprobability◦ChainRule◦ LawofTotalProbability◦Bayes’Theorem◦ TheMontyHallproblem
5
Dice,ourmisunderstoodfriends
Rolltwo6-sideddice,yieldingvaluesD1 andD2.LetEbeevent:D1 + D2=4.
WhatisP(E)?
6
|S| =36
E ={(1,3),(2,2),(3,1)}
P(E) =3/36
Problem:
Solution:
=1/12
🤔
Dice,ourmisunderstoodfriendsRolltwo6-sideddice,yieldingvaluesD1 andD2.LetEbeevent:D1 + D2=4.LetFbeevent:D1 =2.WhatisP(E,givenFalreadyobserved)?
7
S ={(2,1),(2,2),(2,3),(2,4),(2,5),(2,6)}
E ={(2,2)}
P(E,givenFalreadyobserved)
Solution:
=1/6
ConditionalProbability
ConditionalprobabilityistheprobabilitythatEoccursgiventhatFhasalreadyoccurred.Thisisknownas“ConditioningonF.”
Writtenas: P(E|F)
Means: “P(E,givenFalreadyobserved)”
Samplespace,Sà allpossibleoutcomesconsistentwithF(i.e.SÇ F)
Eventspace,Eà alloutcomesinEconsistentwithF(i.e.EÇ F)
8
ConditionalProbability
ConditionalprobabilityistheprobabilitythatEoccursgiventhatFhasalreadyoccurred.Thisisknownas“ConditioningonF.”
Withequallylikelyoutcomes:
P(E|F)=
9
#ofoutcomesinEconsistentwithF#ofoutcomesinSconsistentwithF
|EF||SF|
|EF||F|
P (E|F ) =3
14⇡ 0.21
P (E) =8
50⇡ 0.16
= =
🙆
ConditionalProbability,ingeneralGeneraldefinition:
P(E|F)=(evenwhenoutcomesarenotequallylikely)
Implies: P(EF)=P(E|F)P(F)
WhatifP(F)=0?
à P(E|F)undefined,ofcourse!
Youhaveobservedtheimpossible!10
P(EF)P(F)
TheChainRule
P(EF)=P(FE)=P(F|E)P(E)Bycommutativityofsetintersection,
🤔
Slicingupthespam24emailsaresent,6eachto4users.• 10ofthe24emailsarespam.• Allpossibleoutcomesareequallylikely.E=user1receives3spamemails.WhatisP(E)?F=user2receives6spamemails.WhatisP(E|F)?G=user3receives5spamemails.WhatisP(G|F)?
11
Solution:103
143
246
≈ 0.3245
P(E)=
43
143
186
≈0.0784
P(E|F)=
141
186
= 0
Nowaytochoose5spamfrom4remainingspamemails!
45P(G|F)=
P(EF)P(F)P(E|F)=
|EF||F|
Forequallylikelyoutcomes: =
BitstringsAbitstringwithm0’sandn1’ssentonanetwork.Alldistinctarrangementsofbitsareequallylikely.
E=firstbitreceivedisa1F=k offirstr bitsreceivedare1’s
WhatisP(E|F)?
12
Solution: P(EF)P(F)
P(E|F) = = P(F|E)P(E)P(F)
P(F|E)= ,P(E)= nm+n ,P(F)=
𝑛 − 1𝑘 − 1
𝑚𝑟 − 𝑘
𝑚 + 𝑛 − 1𝑟 − 1
𝑛𝑘
𝑚𝑟 − 𝑘
𝑚 + 𝑛𝑟
Bydefinitionofconditionalprobability
Bitstrings
13
Solution:
P(E|F)=
𝑛 − 1𝑘 − 1
𝑚𝑟 − 𝑘
𝑚 + 𝑛 − 1𝑟 − 1
𝑛𝑘
𝑚𝑟 − 𝑘
𝑚 + 𝑛𝑟
Abitstringwithm0’sandn1’ssentonanetwork.Alldistinctarrangementsofbitsareequallylikely.
E=firstbitreceivedisa1F=k offirstr bitsreceivedare1’s
WhatisP(E|F)?P(EF)P(F)
P(E|F) = = P(F|E)P(E)P(F)
Bydefinitionofconditionalprobability
nm+n =
kr
Bitstrings
14
Solution2:
P(E|F)=P(pickingoneofk 1’soutofr bits)
Abitstringwithm0’sandn1’ssentonanetwork.Alldistinctarrangementsofbitsareequallylikely.
E=firstbitreceivedisa1F=k offirstr bitsreceivedare1’s
WhatisP(E|F)? Byinsight
=kr Rockon!!!
🙆
GeneralizedChainRule
P(E1E2E3…En)=P(E1)P(E2|E1)P(E3|E1E2)…P(En|E1E2…En-1)
15
Proof:P(E1E2E3…En)=
=P(E1)xP(E1|E2)xP(E3|E1E2)x… xP(En|E1E2…En-1)
P(E1)xP(E2E3…En|E1)
= P(E1)xP(E2|E1)xP(E3…En|E1E2)= P(E1)xP(E2|E1)xP(E3|E1E2)xP(E4…En|E1E2E3)
ChainRule:P(EF)=P(E|F)P(F)
🤔
CardpilesAdeckof52cardsisrandomlydividedinto4piles(13cardsperpile).
WhatisP(eachpilecontainsexactlyoneace)?
16
E1 ={AceSpades(AS)inanyonepile}
E2 ={ASandAceHearts(AH)indifferentpiles}
E3 ={AS,AH,AceDiamonds(AD)indifferentpiles}
E4 ={All4acesindifferentpiles}
ComputeP(E1 E2 E3 E4)
=P(E1)P(E2|E1)P(E3|E1 E2)P(E4 |E1 E2 E3)
Solution:
Cardpiles
17
E1 ={AceSpades(AS)inanyonepile}
E2 ={ASandAceHearts(AH)indifferentpiles}
E3 ={AS,AH,AceDiamonds(AD)indifferentpiles}
E4 ={All4acesindifferentpiles}
P(E1)
P(E2|E1)
P(E3|E1 E2)
P(E4 |E1 E2 E3)
P(E1 E2 E3 E4)
≈ 0.105
39x26x13
51x50x49=
=1
=39/51 (39cardsnotinASpile)
=26/50 (26notinASorAHpiles)=13/49 (13notinAS,AH,ADpiles)
CardpilesAdeckof52cardsisrandomlydividedinto4piles(13cardsperpile).
WhatisP(eachpilecontainsexactlyoneace)?
18
E:multipartexperiment:
1. Deal4acesintodifferentpiles.
2. Distributerestofcardssuchthateachpilehas13cards.
S:Distributeallcardssuchthateachpilehas13cards.
Solution2:
4812,12,12,12
4!
5213,13,13,13
P(E)= |E||S| ≈ 0.105
🙆
LawofTotalProbability
P(E) =P(EF)+P(EFc)=P(F)P(E|F)+P(Fc)P(E|Fc)
19
IfyoucannotcalculateP(E)directly,ithelpstointroduceaneweventFandremoveit.
E=EFÈ EFc
Note:EFÇ EFc =ÆEF +
S
EFc
S
E
S
F E=
GeneralLawofTotalProbability
IfF1,F2,…,Fn areasetofmutuallyexclusiveandexhaustiveevents,then
20
𝑃 𝐸 ==𝑃>
?
𝐹? 𝑃 𝐸 𝐹?
BreakAttendance: t inyurl.com/cs109summer2018
21
🤔
MajoringinCSWhatistheprobabilitythatarandomlychosenstudentoncampusisaCSmajorif:
• 25%ofstudentsoncampusarejuniors
• Ifastudentisajunior,thentheyhave30%likelihoodofbeingaCSmajor
• Ifastudentisnotajunior,thentheyhave20%likelihoodofbeingaCSmajor
22
M=astudentisaCSmajor, J=astudentisajuniorWeknow: P(J)=0.25, P(M|J)=0.30, P(M|Jc)=0.20
P(M)= P(J)P(M|J) + P(Jc)P(M|Jc)
0.25x0.3 + 0.75x0.2
=0.225
LawofTotalprobability
Solution:
𝑃 𝐸 ==𝑃>
?
𝐹? 𝑃 𝐸 𝐹?
GeneralLawofTotalProbability
ThomasBayes
23
Rev.ThomasBayes(~1701-1761):BritishmathematicianandPresbyterianminister
Amomentofsilence…
24
🙆
Bayes’Theorem
25
𝑃 𝐹 𝐸 =𝑃 𝐸 𝐹 𝑃 𝐹
𝑃 𝐸
𝑃 𝐹 𝐸 =𝑃 𝐸 𝐹 𝑃 𝐹
𝑃 𝐸|𝐹 𝑃 𝐹 + 𝑃 𝐸 𝐹B 𝑃(𝐹B) Expandedform:
Mostcommonform:
BayesianInterpretation#1
26
𝑃 𝐹 𝐸 =𝑃 𝐸 𝐹 𝑃 𝐹
𝑃 𝐸
Wanttofind(WTF):P(F|E)
Know:P(E|F)
Bayes’isawayof“flipping”thecondition.
🤔
HIVtestingAtestis98%effectiveatdetectingHIV.• However,thetesthasa“falsepositive”rateof1%.• 0.5%oftheUSpopulationhasHIV.E=youtestpositiveforHIVwiththistestF=youactuallyhaveHIVWhatisP(F|E)?
27
Solution:
P(E|F)=0.98 “truepositive”P(E|Fc)=0.01 “falsepositive”P(F)=0.005 prior
𝑃 𝐹 𝐸 =𝑃 𝐸 𝐹 𝑃 𝐹
𝑃 𝐸|𝐹 𝑃 𝐹 + 𝑃 𝐸 𝐹B 𝑃(𝐹B) (0.98)(0.005)
(0.98)(0.005)+(0.01)(1–0.005)=
≈ 0.330
𝑃 𝐹 𝐸 =𝑃 𝐸 𝐹 𝑃 𝐹
𝑃 𝐸|𝐹 𝑃 𝐹 + 𝑃 𝐸 𝐹B 𝑃(𝐹B)
Bayes’Theorem
AllPeople,S
Bayes’TheoremIntuition
28
Peoplewhotestpositive,E
PeoplewithHIV,F
Say we have 1000 people:
5 have HIVand test positive
985 do not have HIVand test negative.
10 do not have HIVand test positive.
» 0.333
Whyit’sstillgoodtogettested
Ec =youtestnegative forHIV
F =youactuallyhaveHIV
P(F|Ec)?
30
HIV+ HIV–Test+ 0.98=P(E|F) 0.01=P(E|Fc)Test– 0.02=P(Ec |F) 0.99=P(Ec |Fc)
P(Ec |F)P(F)+P(Ec |Fc)P(Fc)P(F|Ec)=
P(Ec |F)P(F)
(0.02)(0.005)+(0.99)(1- 0.005)P(F|Ec)=
(0.02)(0.005)» 0.0001
YourconfidenceofnothavingHIVgoesup!
BayesianInterpretation#2
31
Beforethetest:prior: youmayhaveHIV,butyou
are95.5%confident.(0.5%populationhasHIV)
Thetest:likelihood: thetestreturnsusefulinformationbasedon
whetheryouhaveHIV(98%yesifHIV,1%yesifnotHIV)
Afterthetest:posterior: younowknowhavemoreinformationabout
whetheryouhaveHIV,basedontestresults.
𝑃 𝐹 𝐸 =𝑃 𝐸 𝐹 𝑃 𝐹
𝑃 𝐸 posterior
likelihood prior
Normalizingconstant
BayesianInterpretations
32
𝑃 𝐹 𝐸 =𝑃 𝐸 𝐹 𝑃 𝐹
𝑃 𝐸 posterior
likelihood prior
Normalizingconstant
“Youprobablythinkthatyouarebetternow,betternow,Youonlysaythat'cause I'mnotaround,notaround”– PostMalone,2018Posterior
MontyHallProblem
33
WayneBrady+
🤔
MontyHallProblemakaLet’sMakeaDealBehindonedoorisaprize(equallylikelytobeanydoor).
Behindtheothertwodoorsisnothing
1. Wechooseadoor
2. Hostopens1ofother2doors,revealingnothing
3. Wearegivenanoptiontochangetotheotherdoor.
Shouldweswitch?
34
DoorsA,B,C
Note:Ifwedon’tswitch,P(win)=1/3 (random)
WearecomparingP(win)andP(win|switch).
🤔
Ifweswitch
P(Aprize)=1/3
1.HostopensBorC
2.Weswitch
3.Wealwayslose
P(win|Aprize,pickedA,switched)=0
35
Solution:Withoutlossofgenerality,saywepickA(outofDoorsA,B,C).
P(Bprize)=1/3
1.HostmustopenC
2.WeswitchtoB
3.Wealwayswin
P(win|Bprize,pickedA,switched)=1
P(Cprize)=1/3
1.HostmustopenB
2.WeswitchtoC
3.Wealwayswin
P(win|Cprize,pickedA,switched)=1
P(win|pickedA,switched)=1/3*0+1/3*1+1/3*1=2/3
MontyHall,1000envelopeversion
Startwith1000envelopes(ofwhich1istheprize).
1. Youchoose1envelope.
2. Iopen998ofremaining999(showingtheyareempty).
3. Shouldyouswitch?
36
P(envelopeisprize)=1/1000
P(youwinwithoutswitching)=
P(youwinwithswitching)=
999/1000=P(998emptyenvelopeshadprize)+P(lastotherenvelopehasprize)
=P(remainingotherenvelopehasprize)
P(other999envelopeshaveprize)=999/1000
1original#envelopes
original#envelopes- 1original#envelopes
Summary
37
P(EF)=P(E|F)P(F)
P(E) =P(F)P(E|F)+P(Fc)P(E|Fc)
P(F|E)
P(E|F)P(F)P(E|F)P(F)+P(E|Fc)P(Fc)
DefinitionofConditionalProbability
Bayes’Theorem
ChainRule
LawofTotalProbability
P(E|F)P(F)P(E)
P(EF)P(E)
=
MontyHallexample:conditionalprobabilityhijinks.