COLOR TESTCOLOR TESTCOLOR TESTCOLOR TEST

Dueling Algorithms

NICOLE IMMORLICA, NORTHWESTERN UNIVERSITY

WITH A. TAUMAN KALAI, B. LUCIER, A. MOITRA, A. POSTLEWAITE, AND M. TENNENHOLTZ

Social Contexts

Normal-form games:Players choose strategies to maximize expected von Neumann-Morgenstern utility.

Social context games [AKT’08]:Players choose strategies to achieve particular social status among peers.

Social Contexts

Ranking games [BFHS’08]:Players choose strategies to achieve particular payoff rank among peers.

Two-Player Ranking Games

GAlice

Bob

RG payoff of Alice:

1 Alice beats Bob in G

Alice and Bob play game:

½ Alice ties Bob in G

0 Alice loses to Bob in G

Implicit Representations

Succinct games [FIKU’08]:Payoff matrix represented by boolean circuit. NE hard to solve or approximate.

Blotto games [B’21, GW’50, R’06, H’08]:Distribute armies to battlefields.

Implicit Representations

Optimization duels [this work]:Underlying game is optimization problem. Goal is to optimize better than opponent.

Ranking Duel

A search engine is an algorithm that inputs• set Ω = {1, 2, …, n} of items• probabilities p1 + … + pn = 1 of eachand outputs a permutation π of Ω.

Monopolist objective: minimize Ei~p[π(i)].

Ranking Duel

Competitive objective: Let the expected score of a ranking π versus a ranking π’ be

Pri~p[ π(i) < π’(i) ] + (½) Pri~p[ π(i) = π’(i) ].

Then objective is to output a π that maximizes expected score given algorithm of opponent.

Optimizing a Search Engine

?

User searches for object drawn according to known probability dist.

0.19 0.16 0.27 0.07 0.22 0.09

Search: pretty shape

1. (27%)

2. (22%)

3. (19%)

4. (16%)

5. (09%)

6. (07%)

Greedy is optimal.

Choosing a Search Engine

1. Search for “pretty shape”.2. See which search engine ranks

my favorite shape higher.3. Thereafter, use that one.

0.19 0.16 0.27 0.07 0.22 0.09

Search: pretty shape

1. (27%)

2. (22%)

3. (19%)

4. (16%)

5. (09%)

6. (07%)

Search: pretty shape

6. (27%)

1. (22%)

2. (19%)

3. (16%)

4. (09%)

5. (07%)

Questions

Can we efficiently compute an equilibrium of a ranking duel?

How poorly does greedy perform in a competitive setting?

What consequences does the duel have for the searcher?

Optimization Problems as DuelsRanking Binary Search Routing

ParkingCompressionHiring

Start

Finish

?

??

??

?

?

Duel Framework

Finite feasible set X of strategies.Prob. distribution p over states of nature Ω.Objective cost c: Ω × X R.

Monopolist: choose x to minimize Eω~p[cω(x)].

Duel Framework

1. Players select strategies x, x’ from X.2. Nature selects state ω from Ω according to p.3. Payoffs v(x,x’), (1-v(x,x’)) are realized.

1 if cω(x) < cω(x’)

0 if cω(x) > cω(x’)

½ if cω(x) = cω(x’)

v(x,x’) = Eω~p

Results: Computation

An LP-based technique to compute exact equilibria, A low-regret learning technique to compute approximate equilibria,

… and a demonstration of these techniques in our sample settings

Computing Exact Equilibria

Formulate game as bilinear duel:1. Efficiently map strategies to points X in Rn.2. Define constraints describing K=convex-hull(X).3. Define payoff matrix M that computes values.4. Maps points in K back to strategies in original

setting.

Bilinear Duels

If feasible strategies X are points in Rn, and payoff v(x, x’) is xtMx’ for some M in Rnxn, then

maxv,x v s.t. xtMx’ ≥ v for all x’ in X x is in K (=convex-hull(X))

Exponential, but equivalent poly-sized LP.

Ranking Duel

Formulate game as bilinear duel:1. Efficiently map strategies to points X in Rn.

X = set of permutation matrices(entry xij indicates item i placed in position j)

2. Define constraints describing K=convex-hull(X).K = set of doubly stochastic matrices(entry yij = prob. item i placed in position j)

Ranking Duel

Formulate game as bilinear duel:4. Design “rounding alg.” that maps points in K back

to strategies in original setting.Birkhoff–von Neumann Theorem: Can efficiently construct permutation basis for doubly stochastic matrix (e.g., via matching).

Ranking Duel

Formulate game as bilinear duel:3. Define payoff matrix M that computes values.

Ep,y,y’[v(x,x’)] = ∑i p(i) ( ½ Pry,y’ [x i = x’i ] + Pry,y’ [x i > x’i ])= ∑i p(i) (∑i yij ( ½ y’ij + ∑k>j y’ik ))

which is bilinear in y,y’ and so can be written ytMy’.

Ranking Duel

Result: Can reduce computation time to poly(n) versus poly(n!) with standard LP approach.

Technique also applies to hiring duel and binary search duel.

Compression Duel

data

Goal: smaller compression (i.e., lower depth in tree).

(each with prob. p(.))

Classical Algorithm

Huffman coding:Repeatedly pair nodes with

lowest probability.

Compression Duel

Formulate game as bilinear duel:1. Efficiently map strategies to points X in Rn.

X = subset of zero-one matrices*(entry xij indicates item i placed at depth j)

2. Define constraints describing K=convex-hull(X).K = subset of row-stochastic matrices*(entry yij = prob. item i placed at depth j)

* Must correspond to depth profile of some binary tree!

Compression Duel

Formulate game as bilinear duel:3. Define payoff matrix M that computes values.

Ep,y,y’[v(x,x’)] = ∑i p(i) (∑i yij ( ½ y’ij + ∑k>j y’ik ))

which is bilinear in y,y’ and so can be written ytMy’.

Compression Duel

Bilinear Form:maxv,x v s.t. xtMx’ ≥ v for all x’ in X x is in K (=convex-hull(X))

Problems:1. How to round points in K back to a random binary tree with right depth profile?2. How to succinctly express constraints describing K?

Approximate Minimax

Defn. For any ε > 0, an approximate minimax strategy guarantees payoff not worse than best possible value minus ε.

Defn. For any ε > 0, an approximate best response has payoff not worse than payoff of best response minus ε.

Best-Response Oracle

Idea. Use approximate best-response oracle to get approximate minimax strategies.

1. Low-regret learning: if x1,…,xT and x’1,…,x’T have low regret, then ave. is approx minimax.2. Follow expected leader: on round t+1, play best-response to x1,…,xt to get low-regret.

Compression Best-Response

Given lists of items with values and weights, pick one from each list with max total value and total weight at most one.

Multiple-choice Knapsack:

Compression Best-Response

Depth: 1 2 3 4

Compression Best-Response

(each with prob. p(.)) x’ in K

For j from 1..n, list of depth j:

v( ) = Pr[win at depth j | x’ ]w( ) = 2-j

… Kraft inequality

Other Duels

1. Hiring duel: constraints defining Euclidean subspace correspond to hiring probabilities.

2. Binary search duel: similar to hiring duel, but constraints defining Euclidean subspace more complex (must correspond to search trees).

3. Racing duel: seems computationally hard, even though single-player problem easy.

Conclusion

• Every optimization problem has a duel.• Classic solutions (and all deterministic

algorithms) can usually be badly beaten.• Duel can be easier or harder to solve, and can

lead to inefficiencies.

OPEN QUESTION: effect of duel on the solution to the optimization problem?