the computational complexity of finding nash equilibria
DESCRIPTION
The Computational Complexity of Finding Nash Equilibria. Edith Elkind Intelligence, Agents, Multimedia group (IAM) School of Electronics and CS U. of Southampton. Games and Strategies. Games: strategic interactions between rational entities Solution concepts: what’s going to happen? - PowerPoint PPT PresentationTRANSCRIPT
The Computational Complexityof Finding Nash Equilibria
Edith ElkindIntelligence, Agents, Multimedia group (IAM)
School of Electronics and CS
U. of Southampton
Games and Strategies
• Games: strategic interactions between rational entities
• Solution concepts: what’s going to happen?– dominant strategies– Nash equilibrium– ….
• Can it be computed?– if your computer cannot find it, the market
probably cannot either
Matrix (Normal Form) Games
2 0
0 1
1 0
0 3
Row player:
Column player:
0
1
0 1 0 1
0
1
• finite set of players {1, …, n}
• each player has k actions
(pure strategies): 1, …, k
• payoffs of the ith player: Pi: {1, …, k}n → R
Nash Equilibrium
2 0
0 1
1 0
0 3
Row player:
Column player:
0
1
0 1 0 1
0
1
• Nash equilibrium: a strategy profile such that
noone wants to deviate given other players’ strategies, i.e., each player’s strategy is a best response to others’ strategies:– (0, 0) and (1, 1) are both NE
Pure vs. Mixed Strategies
1 -1
-1 1
-1 1
1 -1
Row player:
Column player:
H
T
H T H T
H
T
• NE in pure strategies may not exist!– “matching pennies”
• Mixed strategy: a probability distribution over actions– 50% tail, 50% head
Existence of NE
• Theorem (Nash 1951): any n-player k-action game in normal form has an equilibrium in mixed strategies
can we find one in poly-time?
Plan of the Talk
2 players, k actions
n players, 2 actions
other cool stuff
2 (r=const) players, k actions
• Input representation:– 2 players: two k x k matrices– r players: r k x k x … x k matrices
• poly-size for constant r
• Output representation: – for 2 players all NE are in Q– but not for 3 and more players…
• Checking for pure NE: easy – at most k2 strategy profiles
2 players, k actions: mixed NE
• Naïve approaches: exp(k)
• Simplex-like approach (Lemke-Howson algorithm):– works well in practice– exp(k) in the worst case (2004)
• Is it time to give up?– maybe the problem is NP-hard?
Is Finding NE NP-hard?
• Reminder: a problem P is NP-hard if you can reduce 3-SAT to it:– “yes”-instance 3-SAT → “yes”-instance of P– “no”-instance 3-SAT → “no”-instance of P
• Problem: each instance of NASH is a “yes”-instance!– every game has a NE
• Formally: if NASH is NP-hard then NP = coNP• Need: complexity theory for
total search problems
Reducibility Among Search Problems
• S associates x in X with a solution set S(x)• Total search problem: for any x, S(x) is not empty
S: X Y
T: X’ Y’
If T is easy, so is S• S is reducible to T if:
– f, g easy to compute– g(T(f(x))) is in S(x)
f g
Completeness Results?
• Can we prove that any total search problem is reducible to r-NASH?
• Not really: the class T of all total search problems is a semantic class– not known how to find complete problems for these
• Want to pick a large subclass S of T s.t.– S includes some natural problems– there are problems that are complete for S– in particular, r-NASH is complete for S
• Input: Boolean circuits S (Successor), P (Predecessor):– n inputs, n outputs– S(0n) ≠ 0n, P(0n) = 0n
• Output: x ≠ 0n s.t. – S(P(x)) ≠ x or P(S(x)) ≠ x
Intuition: G=(V, E): – V = n; – E = {(x,y) | y=S(x), x=P(y)}
END OF THE LINE
00000
01011
11001
01011
PPAD
• PPAD: Polynomial Parity Argument, Directed version
• PPAD is the class of all search problems that are reducible to END OF THE LINE
search problem solution
circuits S, T “end of the line”f g
r-NASH is in PPAD
• Proof on Nash’s theorem:– existence of NE reduces to Brouwer’s fixpoint
theorem– Brouwer’s fixpoint theorem reduces to
Sperner’s lemma– Sperner’s lemma is proven by a parity
argument (similar to END OF THE LINE)
• Reduction of r-NASH to END OF THE LINE can be extracted from these proofs (Papadimitriou 94)
Brouwer’s Fixpoint Theorem
• Brouwer’s Theorem: Any continuous mapping from the simplex to itself has a fixpoint.
• Nash Brouwer proof sketch:– set of all strategy profiles → simplex– mapping: (s1, …, sn) → (s1+1, …, sn+n),
where i is a shift in the direction of best response to (s1, …, si-1, si+1, …, sn)
– NE is a point where noone wants to deviate, i.e., a fixpoint
• Proper coloring:– vertices on BC are not blue– vertices on AC are not green– vertices on AB are not yellow
• Sperner’s Lemma: there exists a trichromatic triangle
• Brouwer’s theorem Sperner’s Lemma:– x is blue if the grad(F) at x points away from A, etc.– trichromatic triangle “has no direction” – repeat at increased resolution…
Sperner’s Lemma
A
B
C
Reductions (Papadimitriou 1994)
END OF THE LINE is PPAD-complete
TRICHROMATIC TRIANGLE is PPAD-complete
3D-BROUWER is PPAD-complete
r-NASH is in PPAD
r-NASH vs 3D BROUWER
• Existence of NE follows from Brouwer’s fixpoint theorem
• NE are special cases of Brouwer’s fixpoints– just how special?
• Can any fixpoint be represented as a NE of a game?– Is there a reduction
from 3D BROUWER to r-NASH?
Hardness Reductions: the Timeline
• 3D-BROUWER is PPAD-complete (Papadimitriou, 1994)
• 4-NASH is PPAD-complete (Daskalakis,Goldberg, Papadimitriou, Sep 2005)
• 3-NASH is PPAD-complete (Daskalakis, Papadimitriou, Oct 2005, Chen, Deng, Nov 2005)
• 2-NASH is PPAD-complete !!! (Chen, Deng, Dec 2005)
n players, 2 actions
• representation: payoffs to each player for every action profile (vector in {0, 1}n): n2n numbers
• graphical games:– players are vertices of a graph– V’s payoff depends on actions of W in N(V) U V– n players, max degree d => n2d+1 numbers
TU
V
W t=0, u=0, v=0, w=0: 12t=1, u=0, v=0, w=0: 31 ….t=1, u=1, v=1, w=1: -6
W’s payoffs(16 cases):
Algorithms: What Was Known
• Bounded-degree trees:– Exp-time algorithm/poly-time approximation
algorithm to find all NE (Kearns, Littman, Singh, UAI 2001)
– ??? poly-time algorithm to find a single NE (Kearns, Littman, Singh, NIPS’2001)
• Heuristics for graphs with cycles
Our Results (E., Goldberg, Goldberg’06)
• Algorithm in NIPS’01 paper is incorrect (does not always output a NE)
• We fix the NIPS’01 algorithm, but…– our algorithm runs in poly-time on paths– with a trick, also on cycles
• There is a graph of pathwidth 2 on which our algorithm runs in exp time– true for all algorithms that use the basic
approach of the UAI’01 paper
Warm-up: 2-player 2-action games
2 0
0 1
1 0
0 3
Row player:
Column player:
0
1
0 1 0 1
0
1
Suppose R plays 1 w.p. r
EP(C) from playing 0: (1-r)*1
EP(C) from playing 1: r*3
1-r > 3r iff r < ¼
Suppose C plays 1 w.p. c
EP(R) from playing 0: (1-c)*2
EP(R) from playing 1: c*1
(1-c)*2 > c iff c < 2/3
1/4 1
r
BR(C)c1
2/3BR(R)
mixed NE: r=1/4, c=2/3
• Potential best response: v is a PBR to w iff when W plays w, there is a NE for TV in which V plays v.
• upstream pass: construct PBRV(w) from PBRU1(v), PBRU2(v) and PBRU3(v)
• downstream pass: root selects its strategy based on the children’s PBR’s; propagates to leaves
Algorithm for Trees (KLS’01)
TV
W
V
U1U2
U3
v
w
Computing PBR on a Path
• E0 = EP(V) from playing 0: (1-u)(1-w)*v000+(1-u)w*v001+u(1-w)*v100+uw*v101 = auw+bu+cw+d
• E1 = EP(V) from playing 1: (1-u)(1-w)*v010+(1-u)w*v011+u(1-w)*v110+uw*v111 = a’uw+b’u+c’w+d’
• E0 = E1 iff w = (Au+B)/(Cu+D) = f(u)
U V W
.5 1
1u
v
v
1
1
.5
.1 .9 w
(v, u) → (f(u), v)
PBRU(v) PBRV(w)
Trees: too many segments
v v w
ut v
v1 v2 v1 v2
v1
v2
KLS (NIPS’01): can “trim” PBR
Incorrect!
W
V
T U
(v,t), (v,u) → (f(u,t), v)
u2
u1
t2
t1
Solutions?
• Solution 1 (for paths): algorithm of UAI’01 paper, careful analysis– the number of segments/rectangles in each
PBR is O(n2)– running time O(n3)
• Solution 2 (for paths): can pick a subset of each PBR consisting of O(n) segments– O(n2) running time
Extension to trees? V0 V1 V2 Vn-1 Vn
U1
T1
Un-1
T2
U2
Tn-1 Tn
Un
Graphical games: hardness results
• NP-hard?– no: total search problem
• PPAD-hard?– yes!– in fact, this is how the hardness result for
4-player games was obtained (Goldberg, Papadimitriou, Aug 2005)
Equivalences: GP’05r-player game G NE of G
deg 3 graphical game G’ NE of G’
f g
d2-player game G’ NE of G’
deg d graphical game G NE of G
f g
Combining Reductions: GP’05
r-player game G NE of G
9-player game G’ NE of G’
f g
Finding NE in a 4-player game is as hard as
finding NE in a r-player game for any constant r
X4
PPAD-hardness: missing details
• 3D-Brouwer is PPAD-complete (Papadimitriou, 1994)
• 4-NASH is as hard as deg 3-GG (Goldberg, Papadimitriou, Aug 2005)
• deg 3-GG is PPAD-complete and hence 4-NASH is PPAD-complete
(Daskalakis,Goldberg, Papadimitriou, Sep 2005) • 3-NASH is PPAD-complete
(Daskalakis, Papadimitriou, Oct 2005, Chen, Deng, Nov 2005)
• 2-NASH is PPAD-complete !!! (Chen, Deng, Dec 2005)
NE with special properties
Pure NE:
• easy for constant number of players
• NP-hard for general graphical games – even if max degree = 3– NP vs. PPAD: pure NE may not exist!
• poly-time on trees (KLS algorithm)– also on graphs with bounded treewidth
Welfare-Maximizing NE
2 0
0 1
1 0
0 3
Row player:
Column player:
0
1
0 1 0 1
0
1
Nash equilibria: • (0, 0): total payoff is 3• (1, 1): total payoff is 4• (1/4, 2/3): total payoff is 17/12
not all NE are created equal…
Algorithms for Good NE
• 2-player games: checking for NE with total payoff > T is NP-hard (Gilboa Zemel 89, Conitzer, Sandholm 03)
• Graphical games: for any algebraic , deg() = n, there is a GG with int payoffs on a path of length O(n) in which in the best NE player 1 plays approximation algorithms for any (E., Goldberg, Goldberg 07)
Approximate NE
• -Nash equilibrium: a strategy profile such that noone can gain > by deviating
• Graphical games on trees: poly-time algorithms for any (KLS’01)
• 2-player games ( utilities in [0, 1] ): – PPAD-complete for =O(1/n)– Approximation for constant :
• 0.5 WINE’06 (Dec 2006)• 0.382 ( =1-1/) ACM EC’07 (June 2007)• 0.364 WINE’07 (Dec 2007)• 0.339 WINE’07 (Dec 2007)
Conclusions
• Computational aspects of game-theoretic questions are crucial
• Lots of cool open problems– computing NE in graphical games on trees– finding -Nash in 2-player games for small
• A rich set of techniques
• Talk to me if you want to know more…
Mixed strategies and payoffs
• Payoff matrices:
• the row player plays a = (a1, …, an)• the column player plays b = (b1, …, bn)• expected payoff of R when playing i: (Ri, *, b)• expected payoff of C when playing i: (C*, j, a)
R11 R12 … R1n
R21 R22 … R2n
…Rn1 Rn2 … Rnn
C11 C12 … C1n
C21 C22 … C2n
…Cn1 Cn2 … Cnn
R: C:
• if 1st player’s strategy a supported on I N ai ≠ 0 iff i I
2nd player’s strategy b supported on J N bj ≠ 0 iff j J
• then I BR(b): (b, Ri, *) ≥ (b, Rk, *) for all i I, k N
J BR(a): (a, C*, j) ≥ (a, C*, k) for all j J, k N– LP on variables a1, …, an, b1, …, bn
– solutions to LP ↔ Nash equilibria
• running time: 22kpoly(k)
2 players, k actions: support guessing
linear inequalities!
Reminder: 2-player 2-action games
2 0
0 1
1 0
0 3
Row player:
Column player:
0
1
0 1 0 1
0
1
Suppose R plays 1 w.p. r
EP(C) from playing 0: (1-r)*1
EP(C) from playing 1: r*3
1-r > 3r iff r < ¼
Suppose C plays 1 w.p. c
EP(R) from playing 0: (1-c)*2
EP(R) from playing 1: c*1
(1-c)*2 > c iff c < 2/3
1/4 1
r
BR(C)c1
2/3BR(R)
mixed NE: r=1/4, c=2/3
Computing PBR on a path
• f(u) = (au+b)/(cu+d)• a, b, c, d are determined by V’s payoffs
U V W
.5 1
1u
v
v
1
1
.5
.1 .9 w
(v, u) → (f(u), v) + “tails”
PBRU(v) PBRV(w)