subexponential algorithms for unique games and related problems barriers ii workshop, princeton,...

Subexponential Algorithms for Unique Games and Related Problems

Barriers II Workshop, Princeton, August 2010

David SteurerMSR New England

Sanjeev Arora Princeton University & CCI

Boaz BarakPrinceton & MSR New England

U

Introduction

Small-Set Expansion

Unique Games

UNIQUE GAMESInput: list of constraints of form xi – xj = cij mod k

Goal: satisfy as many constraints as possible

Input: UNIQUE GAMES instance with k << log n (say)

Goal: Distinguish two cases

YES: more than 1 - ² of constraints satisfiableNO: less than ² of constraints satisfiable

Khot’s Unique Games Conjecture (UGC)For every ² > 0, the following is NP-hard:

UG(²)

Implications of UGCFor many basic optimization problems, it is NP-hard to beat current algorithms (based on simple LP or SDP relaxations)

Examples:

VERTEX COVER [Khot Regev’03], MAX CUT [KhotKindlerMosselO’Donnell’04,

MosselO’DonnellOleszkiewicz’05],any MAX CSP [Raghavendra’08], …

Implications of UGCFor many basic optimization problems, it is NP-hard to beat current algorithms (based on simple LP or SDP relaxations)

Examples:

VERTEX COVER [Khot Regev’03], MAX CUT [KhotKindlerMosselO’Donnell’04,

MosselO’DonnellOleszkiewicz’05],any MAX CSP [Raghavendra’08], …

Unique Games Barrier

Example: (®GW + ²)-approximation for MAX CUTat least as hard as UG(²’)

UNIQUE GAMES is common barrier for improving current algorithms of

many basic problems

Reductions show that beating current algorithms for these problems is harder than UNIQUE GAMES

®GW = 0.878…Goemans–Williamson

bound for Max Cut

Implications of UGCFor many basic optimization problems, it is NP-hard to beat current algorithms (based on simple LP or SDP relaxations)Examples:

VERTEX COVER [KR’03], MAX CUT [KKMO,’04 MOO’05],any MAX CSP [Raghavendra’08], …

Consequences for UGC (*)

Analog of UGC with subconstant ² (say ² = 1/log log n) is false(contrast: subconstant hardness for LABEL COVER [Moshkovitz-Raz’08])

Subexponential Algorithm for Unique Games

In particular: UG(²3) has exp(n²)-time algorithm

Given a UNIQUE GAMES instance with alphabet size ksuch that 1 - ² of constraints are satisfiable,can satisfy 1 - √²/¯

3 of constraints in time exp(k n¯)

NP-hardness reduction for UG(²) requires blow-up npoly(1/²)

rules out certain classes of reductions for proving UGC

(*) assuming 3 SAT does not have subexponential algorithms

poly(n) exp(n)

Concrete Complexity Landscape

2-SAT

MAX 3-SAT(7/8)MAX CUT (®GW)

* assuming Exponential Time Hypothesis [Impagliazzo-Paturi-Zane’01]( 3-SAT has no exp(o(n)) algorithm )

3-SAT (*)

FactoringGraph Isomorphism

exp(n1/2)exp(n1/3)exp(n²)

UG(²3)MAX 3-SAT(7/8+²)LABEL COVER(²)

[Moshkovitz-Raz’08+ Håstad’97]

If UGC true, UNIQUE GAMES is first CSP with intermediate complexity

MAX CUT(®GW + ²)?

UGC-based hardness does not rule out subexponential algorithms, Possibility: exp(n²)-time algorithm for MAX CUT(®GW + ²)

UNIQUE GAMES much easier than LABEL COVER

Implications of UNIQUE GAMES algorithm (*)

Introduction

Small-Set Expansion

Unique Games

d-regular graph Gd

vertex set S

Graph Expansion

expansion(S) = # edges leaving S

d |S|

volume(S) = |S||V|

d-regular graph Gd

vertex set S

Graph Expansion

expansion(S) = # edges leaving S

d |S|

volume(S) = |S||V|

S

expansion(S) = # edges leaving S

d |S|

Graph Expansion

volume(S) = |S||V|

SMALL-SET EXPANSION

Goal: find S with volume(S) < ± so as tominimize expansion(S)

Input: d-regular graph G, parameter ± > 0

Important concept in many contexts:derandomization, network routing, coding theory,Markov chains, differential geometry, group theory

close connection to UNIQUE GAMES [Raghavendra-S.’10]

S

expansion(S) = # edges leaving S

d |S|

Graph Expansion

volume(S) = |S||V|

Important concept in many contexts:derandomization, network routing, coding theory,Markov chains, differential geometry, group theory

Subexponential Algorithm for SMALL-SET EXPANSIONIf there exists S with volume(S) < ± and expansion(S) < ²,we can find S’ with volume(S’) < 2± and expansion(S’) < √²/¯

in time exp(n¯/±)

Subexponential Algorithm for SMALL-SET EXPANSIONIf there exists S with volume(S) < ± and expansion(S) < ²,we can find S’ with volume(S’) < 2± and expansion(S’) < √²/¯

in time exp(n¯/±)

1.few large eigenvalues 2.many large eigenvalues

Distinguish two cases:

large eigenvalues¸i > 1 - ´

Eigenvalues of the random walk matrix G:1 = ¸1 ¸ … ¸ ¸m ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1

(pseudorandom graph) (structured graph?)

Subexponential Algorithm for SMALL-SET EXPANSIONIf there exists S with volume(S) < ± and expansion(S) < ²,we can find S’ with volume(S’) < 2± and expansion(S’) < √²/¯

in time exp(n¯/±)

´ À ²(best: ´ = 100²,

simpler: ´ = ²0.75 )

1 - ´

Eigenvalues of the random walk matrix G:1 = ¸1 ¸ … ¸ ¸m ¸

Subexponential Algorithm for SMALL-SET EXPANSIONIf there exists S with volume(S) < ± and expansion(S) < ²,we can find S’ with volume(S’) < 2± and expansion(S’) < √²/¯

in time exp(n¯/±)

¸ ¸m+1 ¸ … ¸ ¸n ¸ -1

large eigenvalues¸i > 1 - ´

Case 1: Few large eigenvalues: (inspired by [Kolla–Tulsiani ’07] and [Kolla’10])

1 = ¸1 ¸ … ¸ ¸m > 1 - ´ ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1

Expander Mixing Lemma: indicator vector of S livesalmost completely in span of top m eigenvectors

Eigenvalues of the random walk matrix G:

can find set S’ close to S in time exp(m)

Enumerate this space in time exp(m)

Case 2: Many large eigenvalues (m > n¯/±)

can find small non-expanding set S’ around that vertexWill show: 9 vertex whose neighborhoods grow very slowly

Subexponential Algorithm for SMALL-SET EXPANSIONIf there exists S with volume(S) < ± and expansion(S) < ²,we can find S’ with volume(S’) < 2± and expansion(S’) < √²/¯

in time exp(n¯/±)

Case 1: Few large eigenvalues:

1 = ¸1 ¸ … ¸ ¸m > 1 - ´ ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1Eigenvalues of the random walk matrix G:

Case 1: Few large eigenvalues

Subspace enumeration (inspired by [Kolla–Tulsiani ’07] and [Kolla’10])

Eigenvalues of the random walk matrix G:

Suffices to show: indicator vector of S is ²/´-close to U

|S ¢ S’| < ²/´ |S [ S’|

For every set S with expansion(S) < ², can find S’ that is ²/´-close to S in time exp(m)

Algorithm: Let U be span of top-m eigenvectorsFor every vector u in ²-net of unit ball of U,

output all level sets of u

U

1 = ¸1 ¸ … ¸ ¸m > 1 - ´ ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1

Case 1: Few large eigenvalues

Subspace enumeration (inspired by [Kolla–Tulsiani ’07] and [Kolla’10])

Eigenvalues of the random walk matrix G:

Suffices to show: indicator vector of S is ²/´-close to U

(generalization of “easy direction” of Cheeger’s inequality)

hx, G xi > 1 - ² because expansion(S) < ²

x = normalized indicator vector of S

1 = ¸1 ¸ … ¸ ¸m > 1 - ´ ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1

For every set S with expansion(S) < ², can find S’ that is ²/´-close to S in time exp(m)

Case 1: Few large eigenvalues

Subspace enumeration (inspired by [Kolla–Tulsiani ’07] and [Kolla’10])

Eigenvalues of the random walk matrix G:

(generalization of “easy direction” of Cheeger’s inequality)

hx, G xi = hu, G ui + hw, G wi < hu,ui + (1 - ´)hw,wi = 1 - ´hw,wi

Suppose x = u + w for u in U and w orthogonal to U

hx, G xi > 1 - ² because expansion(S) < ²

x = normalized indicator vector of S

1 = ¸1 ¸ … ¸ ¸m > 1 - ´ ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1

Suffices to show: indicator vector of S is ²/´-close to U

For every set S with expansion(S) < ², can find S’ that is ²/´-close to S in time exp(m)

Eigenvalues of the random walk matrix G:1 = ¸1 ¸ … ¸ ¸m > 1 - ´ ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1

Case 2: Many large eigenvalues (m > n¯/±)

Eigenvalues of the random walk matrix G:1 = ¸1 ¸ … ¸ ¸m > 1 - ´ ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1

If m > 1, then 9 S with volume(S) < ½ and expansion(S) < √´and we can find S in poly(n)-time.

Compare: “hard direction” of Cheeger’s inequality

Case 2: Many large eigenvalues (m > n¯/±)

1 = ¸1 ¸ …………………… ¸ ¸m > 1 - ´ ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1Eigenvalues of the random walk matrix G:

If m > n¯/±, then 9 S with volume(S) < ± and expansion(S) < √´/¯

and we can find S in poly(n)-time

Number of large eigenvalues vs. small-set expansion

“higher eigenvalue Cheeger bound”

Heuristic: balls tend to be the least expanding sets in graphs

How can we find small non-expanding sets?

Case 2: Many large eigenvalues (m > n¯/±)

1 = ¸1 ¸ …………………… ¸ ¸m > 1 - ´ ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1Eigenvalues of the random walk matrix G:

If m > n¯/±, then 9 S with volume(S) < ± and expansion(S) < √´/¯

and we can find S in poly(n)-time

Number of large eigenvalues vs. small-set expansion

Suffices to show:9 vertex i such that volume( Ball(i, t) ) < ± for t = (¯/´) log(n)

Volume growth vs. small-set expansion

volume growth < 1+(´/¯) in intermediate step expansion( Ball(i,s) ) < (´/¯) for some s < t

Suppose volume( Ball(i, t) ) < ± for t = (¯/´) log(n)

Case 2: Many large eigenvalues (m > n¯/±)

1 = ¸1 ¸ …………………… ¸ ¸m > 1 - ´ ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1Eigenvalues of the random walk matrix G:

If m > n¯/±, then 9 S with volume(S) < ± and expansion(S) < √´/¯

and we can find S in poly(n)-time

Number of large eigenvalues vs. small-set expansion

Volume growth vs. small-set expansion

How can we relate eigenvalues and volume growth?

Case 2: Many large eigenvalues (m > n¯/±)

1 = ¸1 ¸ …………………… ¸ ¸m > 1 - ´ ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1Eigenvalues of the random walk matrix G:

If m > n¯/±, then 9 S with volume(S) < ± and expansion(S) < √´/¯

and we can find S in poly(n)-time

Number of large eigenvalues vs. small-set expansion

Suffices to show:9 vertex i such that volume( Ball(i, t) ) < ± for t = (¯/´) log(n)

Suffices to show:9 vertex i such that volume( Ball(i, t) ) < ± for t = (¯/´) log(n)

Heuristic:collision probability ¼ 1/|support|

Collision probability decay

|| Gt ei ||2 > 1/(±n)

collision probability of t-step random walk from i

Proof follows from local variant of Cheeger’s inequality(e.g. Dimitriou–Impagliazzo’98)

Case 2: Many large eigenvalues (m > n¯/±)

1 = ¸1 ¸ …………………… ¸ ¸m > 1 - ´ ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1Eigenvalues of the random walk matrix G:

If m > n¯/±, then 9 S with volume(S) < ± and expansion(S) < √´/¯

and we can find S in poly(n)-time

Number of large eigenvalues vs. small-set expansion

Volume growth vs. small-set expansion

Number of large eigenvalues vs. collision probability decay

Collision probability decay vs. small-set expansionSuffices to show:

9 vertex i such that || Gt ei ||2 > 1/(±n) for t = (¯/´) log(n)

= ihei, G2t eii = Trace(G2t) > m¢(1 - ´)2t> 1/±i||Gt ei||2

Case 2: Many large eigenvalues (m > n¯/±)

1 = ¸1 ¸ …………………… ¸ ¸m > 1 - ´ ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1Eigenvalues of the random walk matrix G:

If m > n¯/±, then 9 S with volume(S) < ± and expansion(S) < √´/¯

and we can find S in poly(n)-time

Number of large eigenvalues vs. small-set expansion

Case 1: Few large eigenvalues:

1 = ¸1 ¸ … ¸ ¸m > 1 - ´ ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1Eigenvalues of the random walk matrix G:

can find set S’ ²/´-close to S in time exp(m)

Subspace enumeration: discretize span of top m eigenvectors

Case 2: Many large eigenvalues (m > n¯/±)

can find small non-expanding set S’ around that vertex

Trace bound: 9 vertex where local random walk mixes slowlyU

Subexponential Algorithm for SMALL-SET EXPANSIONIf there exists S with volume(S) < ± and expansion(S) < ²,we can find S’ with volume(S’) < 2± and expansion(S’) < √²/¯

in time exp(n¯/±)

Introduction

Small-Set Expansion

Unique Games

UNIQUE GAMESInput: list of constraints of form xi – xj = cij mod k

Goal: satisfy as many constraints as possible

Constraint Graph Gvariable vertexconstraint edge i

j

xi – xj = cij

mod k

Subexponential Algorithm for Unique Games

In particular: UG(²3) has exp(n²)-time algorithm

Given a UNIQUE GAMES instance with alphabet size ksuch that 1 - ² of constraints are satisfiable,can satisfy 1 - √²/¯

3 of constraints in time exp(k n¯)

can solve UG(²3) in time exp(m)

Few large eigenvalues + strong L1 bounds on top-m eigenvectors

large eigenvalues

Eigenvalues of the random walk matrix of constraint graph G:1 = ¸1 ¸ … ¸ ¸m > 1 - ² ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1

[Arora–Khot–Kolla– S.–Tulsiani–Vishnoi’08]

Expanding constraint graph (m=1)

can solve UG(²) in time poly(n)

[Kolla’10]

UNIQUE GAMES on pseudorandom instances is easy

Algorithms for special instances of Unique Games

Strategypartition general instance into pseudorandom instances by changing only a small fraction of edges [Trevisan’05]

Algorithms for general instances of Unique Games

general instance

pseudorandom instances

decomposition

few constraints between parts

[Arora–Impagliazzo– –Matthews–S.’10]

here: Leighton–Rao decomposition tree, constant depth, using higher

eigenvalue Cheeger boundhere: at most n¯ eigenvalues > 1-²

here: at most√²/¯

3 fraction

Subexponential Algorithm for Unique Games

few large eigenvalues: at most n¯ eigenvalues >1-²

Few-large-eigenvalues decomposition

Every regular graph G can be partitioned into components with few large eigenvalues by removing √²/¯

3 fraction of edges

Unique Games with few large eigenvalues

If every component of G has few large eigenvalues, can solve UG(²2) in time exp(n¯)

few large eigenvalues: at most n¯ eigenvalues >1-²

Unique Games with few large eigenvalues

If every component of G has few large eigenvalues, can solve UG(²2) in time exp(n¯)

few large eigenvalues: at most n¯ eigenvalues >1-²

Unique Games with few large eigenvalues

If every component of G has few large eigenvalues, can solve UG(²2) in time exp(n¯)

label-extended graph G*

ij

xi – xj = cij

mod k

constraint graph G

cloud jcloud i

(i,a) » (j,b) if a-b = cij mod k

assignment satisfying 1- ²2 of constraints

set with volume = 1/kand expansion < ²2

Subspace enumeration: can enumerate all nonexpanding sets if G* has few large eigenvalues

When does G* have herefew large eigenvalues?

Unique Games with few large eigenvalues

label-extended graph G*

ij

xi – xj = c ij

mod k

constraint graph G

cloud jcloud i

(i,a) » (j,b) if a-b = cij mod k

When does G* have herefew large eigenvalues?

•if G is an expander [Kolla–Tulsiani’07]•if G has few large eigenvalues and eigenvectors are well-spread [Kolla’10]•if G has few’ large’ eigenvalues (this work, by comparing collision probabilities)

If every component of G has few large eigenvalues, can solve UG(²2) in time exp(n¯)

few large eigenvalues: at most n¯ eigenvalues >1-²

Recall: (with slightly different parameters)

Analysis: For every subdivision S, charge its expansion to vertices in S

KIf component K has many large eigenvalues, then 9 S in K with |S| < |K|1-¯ and expansion(S) < √²/¯

and we can find S in poly(n)-time

Number of large eigenvalues vs. small-set expansion

Algorithm: Subdivide components until all components have few large eigenvalues

no vertex is charged more than log(1/¯)/¯ timesIf vertex is charged t times, its component has size < n(1-¯)t

few large eigenvalues: at most n¯ eigenvalues larger 1-²

Few-large-eigenvalues decomposition

Every regular graph G can be partitioned into components with few large eigenvalues by removing √²/¯

3 fraction of edges

Open Questions

Example: C-approximation for SPARSEST CUT in time exp(n1/C )

How many large eigenvalues can a small-set expander have?

Is Boolean noise graph the worst case? (polylog(n) large eigenvalues)

Thank you! Questions?

More Subexponential Algorithms

Similar approximation for MULTI CUT and d-TO-1 2-PROVER GAMES

Better approximations for MAX CUT and VERTEX COVER on small-set expanders

What else can be done in subexponential time?

Towards better-than-subexponential algorithms for UNIQUE GAMES

Better approximations for MAX CUT or VERTEX COVER on general instances?