The Computational Complexityof Finding a Nash Equilibrium
Edith Elkind, U. of Warwick
Based On…• Reducibility Among Equilibrium Problems
(Goldberg, Papadimitriou): Aug 2005 • The Complexity of Computing a Nash Equilibrium
(Goldberg, Daskalakis, Papadimitriou): Sep 2005
• 3-NASH is PPAD-Complete (Chen, Deng): Nov 2005
• Three-Player Games Are Hard (Daskalakis, Papadimitriou): Nov 2005
• Settling the Complexity of 2-Player Nash-Equilibrium (Chen, Deng): Dec 2005
Normal Form Games
2 0
0 1
1 0
0 3
Row player:
Column player:
0
1
0 1 0 1
0
1
• finite set of players {1, …, n}
• each player has k actions
(pure strategies): 1, …, k
• payoffs of the ith player: Pi: {1, …, k}n → R
Nash Equilibrium
2 0
0 1
1 0
0 3
Row player:
Column player:
0
1
0 1 0 1
0
1
• Nash equilibrium: a strategy profile such that
noone wants to deviate given other players’ strategies, i.e., each player’s strategy is a best response to others’ strategies:– (0, 0) and (1, 1) are both NE
Pure vs. Mixed Strategies
1 -1
-1 1
-1 1
1 -1
Row player:
Column player:
H
T
H T H T
H
T
• NE in pure strategies may not exist!– “matching pennies”
• Mixed strategy: a probability distribution over actions– 50% tail, 50% head
Existence of NE
• Theorem (Nash 1951): any game in normal form has an equilibrium in mixed strategies
$1 000 000 question:
how to find one?
Finding mixed NE in 2 x 2 Games
2 0
0 1
1 0
0 3
Row player:
Column player:
0
1
0 1 0 1
0
1
Suppose R plays 1 w.p. r
EP(C) from playing 0: (1-r)*1 EP(C) from playing 1: r*3
1-r = 3r iff r = ¼
Suppose C plays 1 w.p. c
EP(R) from playing 0: (1-c)*2EP(R) from playing 1: c*1
(1-c)*2 = c iff c = 2/3
NE: r=1/4, c=2/3
2 players, k actions
• Representation: two k x k matrices• Checking for pure NE: easy
– at most k2 of them
• Checking for mixed NE:– all straightforward methods are exptime– Lemke-Howson algorithm is exptime, too
(previous talk)
• For 2 players all NE are rational– but not for 3 and more players…
n players, 2 actions
• Representation: payoffs to each player for every action profile (vector in {0, 1}n): n2n numbers
• graphical games:– players are associated with the vertices of a graph;– each player’s payoff depends on his own action and
actions of his neighbors– n players, max degree d => n2d+1 numbers
TU
V
W t=0, u=0, v=0, w=0: 12t=1, u=0, v=0, w=0: 31 ….t=1, u=1, v=1, w=1: -6
W’s payoffs(16 cases):
Algorithms for NE in Graphical Games
• Bounded-degree trees:– Exp-time algorithm/poly-time approximation
algorithm to find all NE (Kearns, Littmann, Singh, UAI 2001)
– ??? poly-time algorithm to find a single NE (Kearns, Littmann, Singh, NIPS’2001):
• shown to be incorrect in E., Goldberg, Goldberg, ACM EC’06
• Graphs of max degree 2: – poly-time algorithm (EGG’06)
Is Finding NE NP-hard?
• Reminder: a problem P is NP-hard if you can reduce 3-SAT to it:– “yes”-instance 3-SAT → “yes”-instance of P– “no”-instance 3-SAT → “no”-instance of P
• Problem: each instance of NASH is a “yes”-instance!– every game has a NE– need complexity theory for search problems
• Side note: pure Nash for n players, NE of total value > K are NP-hard
Reducibility Among Search Problems
• S associates x in X with a solution set S(x)• Total search problem: for any x, S(x) is not empty
S: X Y
T: X’ Y’
If T is easy, so is S• S is reducible to T if:
– f, g easy to compute– g(T(f(x))) is in S(x)
f g
Equivalences: GP’05r-player game G NE of G
deg 3 graphical game G’ NE of G’
f g
d2-player game G’ NE of G’
deg d graphical game G NE of G
f g
• Color the graph of GG: d(u,v) ≤ 2 color(u) ≠ color(v)
• Each color is a player of G• RED chooses a red vertex in GG
and an action for that vertex in GG
• payoff=payoff1+payoff2
– payoff1: BLUE tries to guess which vertex RED chose; RED pays a penalty if BLUE guesses correctly
– payoff2: if all neighbors of a chosen vertex are also chosen, it gets same payoff as in GG, else 0
d-Graphical Game GG → d2-Player Game G
r-Player Game G → 3-Graphical Game GG
• Si: space of pure strategies of player i
• S- i = S1 * … Si-1*Si+1 *.. * Sr
• xij: the probability that ith player uses jth strategy
• xs: x1s1 * x2
s2 … * xrsr (for s in S-i)
• uijs: utility of the ith player when he plays j and others play
according to s
-p
NE: 0 ≤ xij ≤ 1
j xij =1
s in S uijsxs > s in S ui
j’sxs implies xij’ = 0-p
r-Player Game G → 3-Graphical Game GG
• Vertex Vij for any pair (player=i, action=j)
• Want: Pr[Vij plays 1] = Pr [i plays j in G]=xij
• Idea: graphical games can do math!– Enforce constraints from the previous slide…
v1 v2 v3
u
Set payoffs to u, v3 so that p[v3]=p[v1] * p[v2]
Need gadgets for +, *, c, =, min, max, …
Equivalences: GP’05r-player game G NE of G
deg 3 graphical game G’ NE of G’
f g
d2-player game G’ NE of G’
deg d graphical game G NE of G
f g
Combining Reductions: GP’05
r-player game G NE of G
9-player game G’ NE of G’
f g
Finding NE in a 4-player game is as hard as
finding NE in a r-player game for any constant r
X4
Completeness Results?
• Can we prove that any total search problem is reducible to r-NASH?
• Not really: the class T of all total search problems is a semantic class– not known how to find complete problems for these
• Want to pick a large subclass S of T s.t.– S includes some natural problems– there are problems that are complete for S– in particular, r-NASH is complete for S
• Input: Boolean circuits S (Successor), P (Predecessor):– n inputs, n outputs– S(0n) ≠ 0n, P(0n) = 0n
• Output: x ≠ 0n s.t. – S(P(x)) ≠ x or P(S(x)) ≠ x
Intuition: G=(V, E): – V = n; – E = {(x,y) | y=S(x), x=P(y)}
END OF THE LINE
00000
01011
11001
01011
PPAD
• PPAD: Polynomial Parity Argument, Directed version
• PPAD is the class of all search problems that are reducible to END OF THE LINE
search problem solution
circuits S, T “end of the line”f g
r-NASH is in PPAD
• Proof on Nash’s theorem:– existence of NE reduces to Brouwer’s fixpoint
theorem– Brouwer’s fixpoint theorem reduces to
Sperner’s lemma– Sperner’s lemma is proven by a parity
argument (similar to END OF THE LINE)
• Reduction of r-NASH to END OF THE LINE can be extracted from these proofs (Papadimitriou 94)
Brouwer’s Fixpoint Theorem
• Brouwer’s Theorem: Any continuous mapping from the simplex to itself has a fixpoint.
• Nash Brouwer proof sketch:– set of all strategy profiles → simplex– mapping: (s1, …, sn) → (s1+1, …, sn+n),
where i is a shift in the direction of best response to (s1, …, si-1, si+1, …, sn)
– NE is a point where noone wants to deviate, i.e., a fixpoint
• Proper coloring:– vertices on BC are not blue– vertices on AC are not green– vertices on AB are not yellow
• Sperner’s Lemma: there exists a trichromatic triangle
• Brouwer’s theorem Sperner’s Lemma:– x is blue if the grad(F) at x points away from A, etc.– trichromatic triangle “has no direction” – repeat at increased resolution…
Sperner’s Lemma
A
B
C
Opposite Direction: 3D-BROUWER
• Input:– 3D unit cube divided into 23n cubelets– cijk is the center of Kijk
– (cijk)=cijk+ijk, ijk is in {0, 1, 2, 3}, where• 1=(, 0, 0), 2=(0, , 0)• 3=(0, 0, ), 0=(-, -, -)
– circuit C: {0, 1} 3n → {0, 1, 2, 3} selects ijk
• Output: – a panchromatic cubelet, i.e., one that has all
of 0, 1, 2, 3 among its 8 neighbors
3D-BROUWER is PPAD-complete
• Papadimitriou (1994) shows that a more complicated version of 3D-BROUWER is PPAD-complete
• This version was proven hard in DGP’05
• Reduction from END OF THE LINE– embed the line L into 3d cube– “protect” L from color 0 using three other colors– color the rest of inner cubelets with 0
r-NASH vs 3D BROUWER
• Existence of NE follows from Brouwer’s fixpoint theorem
• NE are special cases of Brouwer’s fixpoints– just how special?
• Can any fixpoint be represented as a NE of a game?
• DGP’05: YES! 4-NASH is PPAD complete
• Proof: – 4-NASH deg 3 Graphical Nash– graphical games can compute fixpoints
4-NASH to 3-NASH
• Daskalakis, Papadimitriou: modify arithmetic gadgets so that the graph is 3-colorable
• Chen, Deng: same gadgets, but allow for small error
2-NASH
• Chen, Deng: – avoid graphical games– reduce directly from 3D-BROUWER to
2-NASH using arithmetic gadgets similar to graphical game gadgets
• Game over?
Graphical Games: Open Problems
• Degree:– deg 3 PPAD-complete (DGP’05b)– deg 2 polynomial time solvable (EGG’06)
• Pathwidth:– paths: poly-time– pathwidth 1: maybe algorithm from EGG’06 still works– pathwidth 2: any KLS-style algo is exptime (EGG’06)– pathwidth > r, r constant: PPAD-complete (EGG’06)
• Finding NE on trees?