mixed - cs.sfu.caoschulte/teaching/882-10/mixed.pdf · 2) = Σ j eu 1 (a j,s 2) x s 1 (a j) = Σ j...
TRANSCRIPT
MixedStrategies
CMPT882ComputationalGameTheory
SimonFraserUniversitySpring2010
Instructor:OliverSchulte
Motivation
• Some games like Rock, Paper, Scissor don’t have a Nash equilibrium as defined so far.
• Intuitively, the reason is that there is no steady state where players have perfect knowledge of each other’s actions: knowing exactly what the other player will do allows me to achieve an optimal payoff at their expense.
• Von Neumann and Morgenstern observed that this changes if we allow players to be unpredictable by choosing randomized strategies.
Definition
• Consider a 2-player game (A1,A2,u1,u2). • The members of Ai are the pure strategies for
player i. • A mixed strategy is a probability distribution
over pure strategies. • Or, if we have k pure strategies, a mixed strategy
is a k-dimensional vector whose nonnegative entries sum to 1.
• Note: this is the notation from the text. More usual are Greek letters for mixed strategies, e.g. !.
Example:MatchingPennies
Heads: 1/2 Tails: 1/2
Heads: 1/3 1,-1 -1,1
Tails: 2/3 -1,1 1,-1
MixedStrategiesctd.
• Supposethateachplayerchoosesamixedstrategysi.• Theexpectedutilityofpurestrategya1isgivenbyEU1(a1,s2)=Σku1(a1,ak)xs2(ak).whereajrangesoverthestrategiesofplayer2.
• Theexpectedutilityofmixedstrategys1istheexpectationoverthepurestrategies:EU1(s1,s2)=ΣjEU1(aj,s2)xs1(aj)=ΣjΣku1(aj,ak)xs1(aj)xs2(ak).
• Wealsowriteu1(s1,s2)forEU1(s1,s2).Thenwehaveineffectanewgamewhosestrategysetsarethesetsofmixedstrategies,andwhoseutilityfunctionsaretheexpectedpayoffs.
Nashequilibriuminmixedstrategies
• Thedefinitionofbestresponseisasbefore:amixedstrategys1isabestresponsetos2ifandonlyifthereisnoothermixedstrategys’1withu1(s’1,s2)>u1(s1,s2).
• Similarly,twomixedstrategies(s1,s2)areaNashequilibriumifeachisabestresponsetotheother.
ExamplesandExercises
• FindamixedNashequilibriumforthefollowinggames.– MatchingPennies.– CoordinationGame.– Prisoner’sDilemma.
– BattleoftheSexesorChicken.• CanyoufindalltheNashequilibria?
VisualizationofEquilibria
• Ina2x2game,wecangraphplayer2’sutilityasafunctionofp,theprobabilitythat1choosesstrategy1.Similarlyforplayer1’sutility.
StrategiesandTopology
• Considermixedstrategyprofilesask‐dimensionalvectors,wherekisthetotalnumberofpurestrategiesforeachplayer.Thissetisconvexandcompact(why?).
VisualizationofMatchingPennies
VisualizationofM.P.equilibrium
ComputationofEqulibria
• DefinitionThesupportofaprobabilitymeasurepisthesetofallpointsxs.t.p(x)>0.
• Proposition.Amixedstrategypair(s1,s2)isanN.E.ifandonlyifforallpurestrategiesaiinthesupportofsi,thestrategysiisabestreplysos‐i.
• Corollary.If(s1,s2)isanN.E.anda1,b1areinthesupportofs1,thenu1(a1,s2)=u1(b1,s2).Thatis,player1isindifferentbetweena1andb1.Dittofora2,b2inthesupportofs2.
AgeneralNPprocedureforfindingaNashEquilibrium
1. Choosesupportforplayer1,supportforplayer2.
2. CheckifthereisaNashequilibriumwiththosesupports.
ExistenceofNashEquilibrium
• Theorem(Nash1950).Inanyfinitegame(anynumberofplayers)thereexistsaNashequilibrium.
• ShortproofbyNash.
ExistenceProof(1)
• Transformagivenpairofmixedstrategies(s1,s2)asfollows.Foreachpurestrategya1ofplayer1,setc(a1):=max(0,[u1(a1,s2)–u1(s1,s2)]).s’1(a1):=[s1(a1)+c(a1)]/[1+Σbc(b)].Dittoforplayer2.
• Thisdefinesacontinuousoperatorf(s1,s2)=(s’1,s’2)onthespaceofmixedstrategypairs.
• Astrategyaisabestreplyifandonlyifc(a)=0.• Soif(s1,s2)isanN.E.,thenc(a)=0foralla,sof
(s1,s2)=(s1,s2).
ExistenceProof(2)
• ThegeneralizedBrouwerfixedpointtheoremstatesthatifKisconvexandcompact,andf:KKiscontinuous,thenfhasafixedpointf(x)=x.
• If(s1,s2)isafixedpointofthemappingonthepreviousslide,then(s1,s2)isanN.E.
• Proofoutline:Someactiona1withs1(a)>0mustbeabestreplyagainstanys2.Thereforec(a1)=0.Sincewehaveafixedpoint,1+Σbc(b)=1.Thisimpliesthatc(b)=0foreachb.
IllustrationinExcel
• IllustrateconstructioninExcel.• Notethattheupdateoperationcanbeviewedasalocalcomputationmethod,andevenasalearningmethod!
MaxminandMinmax
Considera2‐playergame.• Amaxminstrategyforplayer1solvesmaxs1mins2u1(s1,s2).Dittoforplayer2.
• Interpretation:Conservativelychoosestrategyagainstworst‐caseadversary.
• Thevaluemaxs1mins2u1(s1,s2)iscalledthesecuritylevelofplayer1.
• Aminmaxstrategyforplayer1solvesmins1maxs2u2(s1,s2).Dittoforplayer2.
• Interpreation:Punishtheotherplayerbyminimizingthebestpayoffshecanget.
N.E.inZero‐SumGames:TheMinimaxTheorem(vonNeumann1928).
.Considera2‐playerzero‐sumgame.1. siisamaxminstrategyifandonlyifsiisa
minmaxstrategyfori=1,2.2. Forbothplayers,themaxminvalue=minmax
value.3. Ifs1,s2areeachmaxmin(minmax)strategies,
then(s1,s2)isaNashequilibrium.4. If(s1,s2)isanN.E.,thens1ands2aremaxmin
(minmax)strategies.
InterpretationofmixedN.E.
N.E.inPopulationGames