subexponential algorithms for unique games and related problems school on approximability,...

Subexponential Algorithms for Unique Games and Related Problems

School on Approximability, Bangalore, January 2011

David SteurerMSR New England

Sanjeev Arora Princeton University & CCI

Boaz BarakMSR New England & Princeton

U

UNIQUE GAMESInput: list of constraints of form

Goal: satisfy as many constraints as possible

UNIQUE GAMESInput: list of constraints of form

Goal: satisfy as many constraints as possible

Input: UNIQUE GAMES instance with (say)

Goal: Distinguish two cases

YES: more than of constraints satisfiableNO: less than of constraints

satisfiable

Unique Games Conjecture (UGC) [Khot’02]

For every , the following is NP-hard:

UG (𝜀)

Implications of UGCFor many basic optimization problems, it is NP-hard to beat current algorithms (based on simple LP or SDP relaxations)

Examples:

VERTEX COVER [Khot-Regev’03], MAX CUT [Khot-Kindler-Mossel-O’Donnell’04,

Mossel-O’Donnell-Oleszkiewicz’05],every MAX CSP [Raghavendra’08], …

Implications of UGCFor many basic optimization problems, it is NP-hard to beat current algorithms (based on simple LP or SDP relaxations)

Unique Games Barrier

Example: -approximation for MAX CUTat least as hard as

UNIQUE GAMES is common barrier for improving current algorithms of

many basic problems

Goemans–Williamson

bound for MAX CUT

Subexponential Algorithm for Unique Games

Input: UNIQUE GAMES instance with alphabet size ksuch that of constraints are satisfiable,

Output:assignment satisfying of constraints Time:

in time

Time vs Approximation Trade-off

Analog of UGC with subconstant (say ) is false (*)(contrast: subconstant hardness for LABEL COVER [Moshkovitz-Raz’08])

NP-hardness reduction for must have blow-up (*)

rules out certain classes of reductions for proving UGC

(*) assuming 3-SAT does not have subexponential algorithms, exp( no(1) )

UGC-based hardness does not rule out subexponential algorithms, Possibility: -time algorithm for MAX CUT() ?

Subexponential Algorithm for Unique Games

in time

Consequences

poly (𝑛) exp (𝑛)

2-SAT

MAX 3-SAT()MAX CUT()

3-SAT (*)

FACTORING

exp (𝑛1 /2)exp (𝑛1 /3)exp (𝑛𝜀 1/3 )

UG (𝜀 )MAX 3-SAT()

LABEL COVER()

[Moshkovitz-Raz’08+ Håstad’97]MAX CUT()?

(*) assuming Exponential Time Hypothesis [Impagliazzo-Paturi-Zane’01]( 3-SAT has no algorithm )

Subexponential Algorithm for Unique Games

in time

GRAPH ISOMORPHISM

Subexponential Algorithm for Unique Games in time

Interlude: Graph Expansion

𝑉

𝑆

expansion() = # edges leaving

𝑑∨𝑆∨¿

normalized indicator vector

-regular graph

normalized adjacency matrix (stochastic)

expansion()

Subexponential Algorithm for Unique Games in time

Easy graphs for UNIQUE GAMES

Constraint Graph variable vertexconstraint edge 𝑖 𝑗

Expanding constraint graph

in time if eigenvalue gap

[Arora-Khot-Kolla-S.-Tulsiani-Vishnoi’08]

Expanding constraint graph

Constraint graph with few large eigenvalues

Easy graphs for UNIQUE GAMES

in time if at most eigenvalues

in time if eigenvalue gap

[Arora-Khot-Kolla-S.-Tulsiani-Vishnoi’08]

Subexponential Algorithm for Unique Games in time

Graph Eigenvalues

𝜆𝑚+1

1−100𝜀

𝜆1𝜆2

10

normalized adjacency matrix …

quasi-expander

[Kolla-Tulsiani’07, Kolla’10, here, Barak-Raghavendra-S.’11]

Expanding constraint graph

Constraint graph with few large eigenvalues

Easy graphs for UNIQUE GAMES

in time if at most eigenvalues

in time if eigenvalue gap

[Arora-Khot-Kolla-S.-Tulsiani-Vishnoi’08]

[Kolla-Tulsiani’07, Kolla’10, here, Barak-Raghavendra-S.’11]

Subspace Enumeration [Kolla-Tulsiani’07]

exhaustive search over certain subspace

Subexponential Algorithm for Unique Games in time

Question: Dimension of this subspace?

quasi-expander

[Kolla-Tulsiani’07, Kolla’10, here, Barak-Raghavendra-S.’11]

quasi-expander

in time if at most eigenvalues

Expanding constraint graph

Constraint graph with few large eigenvalues

Easy graphs for UNIQUE GAMES

in time if eigenvalue gap

[Arora-Khot-Kolla-S.-Tulsiani-Vishnoi’08]

Subexponential Algorithm for Unique Games in time

Graph Decomposition

By removing 1% of edges, decompose constraint graphinto components for which is “easy”

Approach [Trevisan’05, Arora-Impagliazzo-Matthews-S.’10]

hardconstraint graph

remove 1% of edges

easy

easyeasy

easy easy

easy

easy

By removing 1% of edges, decompose constraint graphinto components for which is “easy”

Approach [Trevisan’05, Arora-Impagliazzo-Matthews-S.’10]

quasi-expander

in time if at most eigenvalues

Expanding constraint graph

Constraint graph with few large eigenvalues

Easy graphs for UNIQUE GAMES

in time if eigenvalue gap

[Arora-Khot-Kolla-S.-Tulsiani-Vishnoi’08]

[Kolla-Tulsiani’07, Kolla’10, here]

Subexponential Algorithm for Unique Games in time

Graph Decomposition: Given UG instance, distinguishYES: NO:

YES: opt>1-

opt>1- opt>1-

opt>1-

opt>1-

By removing 1% of edges, decompose constraint graphinto components for which is “easy”

Approach [Trevisan’05, Arora-Impagliazzo-Matthews-S.’10]

in time if at most eigenvalues

Expanding constraint graph

Constraint graph with few large eigenvalues

Easy graphs for UNIQUE GAMES

in time if eigenvalue gap

[Arora-Khot-Kolla-S.-Tulsiani-Vishnoi’08]

[Kolla-Tulsiani’07, Kolla’10, here]

Subexponential Algorithm for Unique Games in time

Graph Decomposition

YES:

?

YES?

YES YES

YES

YES

: Given UG instance, distinguishYES: NO:

By removing 1% of edges, decompose constraint graphinto components for which is “easy”

Approach [Trevisan’05, Arora-Impagliazzo-Matthews-S.’10]

in time if at most eigenvalues

Expanding constraint graph

Constraint graph with few large eigenvalues

Easy graphs for UNIQUE GAMES

in time if eigenvalue gap

[Arora-Khot-Kolla-S.-Tulsiani-Vishnoi’08]

[Kolla-Tulsiani’07, Kolla’10, here]

Subexponential Algorithm for Unique Games in time

Graph Decomposition: Given UG instance, distinguishYES: NO:

NO: opt<

opt< opt<

opt<

opt<

By removing 1% of edges, decompose constraint graphinto components for which is “easy”

Approach [Trevisan’05, Arora-Impagliazzo-Matthews-S.’10]

in time if at most eigenvalues

Expanding constraint graph

Constraint graph with few large eigenvalues

Easy graphs for UNIQUE GAMES

in time if eigenvalue gap

[Arora-Khot-Kolla-S.-Tulsiani-Vishnoi’08]

[Kolla-Tulsiani’07, Kolla’10, here]

Subexponential Algorithm for Unique Games in time

Graph Decomposition

NO:

NO

NO?

NO NO

NO

?

: Given UG instance, distinguishYES: NO:

[Kolla-Tulsiani’07, Kolla’10, here, Barak-Raghavendra-S.’11]

quasi-expander

Expanding constraint graph

Constraint graph with few large eigenvalues

Easy graphs for UNIQUE GAMES

in time if at most eigenvalues

in time if eigenvalue gap

[Arora-Khot-Kolla-S.-Tulsiani-Vishnoi’08]

Subexponential Algorithm for Unique Games in time

Graph Decomposition

By removing 1% of edges, every graph can be decomposed into components with

Classical

at most eigenvalues

Here[Leighton-Rao’88, Goldreich-Ron’98, Spielman-Teng’04, Trevisan’05]

eigenvalue gap

Idea: quasi-expander small-set expander

Idea: quasi-expander small-set expander

Expanding constraint graph

Constraint graph with few large eigenvalues

Easy graphs for UNIQUE GAMES

in time if at most eigenvalues

in time if eigenvalue gap

[Arora-Khot-Kolla-S.-Tulsiani-Vishnoi’08]

Subexponential Algorithm for Unique Games in time

Graph Decomposition

By removing 1% of edges, every graph can be decomposed into components with

Classical

at most eigenvalues

Here

eigenvalue gap

quasi-expander

[Kolla-Tulsiani’07, Kolla’10, here, Barak-Raghavendra-S.’11]

Constraint graph with few large eigenvalues

in time if at most eigenvalues

[Kolla-Tulsiani’07, Kolla’10, here, Barak-Raghavendra-S.’11]

label-extended graph constraint graph

cloud cloud

if a-b=c mod k

assignment satisfying of constraints

vertex set of size and expansion

𝑖 𝑗

Assume: label-extended graph has at most eigenvalues

2

Constraint graph with few large eigenvalues

in time if at most eigenvalues

[Kolla-Tulsiani’07, Kolla’10, here, Barak-Raghavendra-S.’11]

label-extended graph constraint graph

assignment satisfying 90%of

constraints

99% of indicator vector lies inspan of top m eigenvectors

Assume: label-extended graph has at most eigenvalues

enumerate subspace

2

vertex set of size and expansion

assignment satisfying of constraints

Constraint graph with few large eigenvalues

in time if at most eigenvalues

[Kolla-Tulsiani’07, Kolla’10, here, Barak-Raghavendra-S.’11]

label-extended graph

assignment satisfying 90%of

constraints

99% of indicator vector lies inspan of top m eigenvectors

Assume: label-extended graph has at most eigenvalues

vertex set of size and expansion

enumerate subspace

assignment satisfying of constraints

Compare: Fourier-based learning

2

constraint graph normalized indicator vector

Suppose: > 1% of is orthogonal to span

⟨ 𝑥 , �̂� 𝑥 ⟩<0.99𝜆1+0.01 𝜆𝑚+1≤1−𝜀

expansion

Constraint graph with few large eigenvalues

in time if at most eigenvalues

[Kolla-Tulsiani’07, Kolla’10, here, Barak-Raghavendra-S.’11]

Idea: quasi-expander small-set expander

Easy graphs for UNIQUE GAMES

Constraint graph with few large eigenvalues

in time if at most eigenvalues

[Kolla-Tulsiani’07, Kolla’10, here, Barak-Raghavendra-S.’11]

Graph Decomposition

By removing 1% of edges, every graph can be decomposed into components with

Classical

at most eigenvalues

Here

eigenvalue gap

Subexponential Algorithm for Unique Games in time

Constraint graph with few large eigenvalues

in time if at most eigenvalues

[Kolla-Tulsiani’07, Kolla’10, here, Barak-Raghavendra-S.’11]

Easy graphs for UNIQUE GAMES

Graph Decomposition

By removing 1% of edges, every graph can be decomposed into components with

Classical

at most eigenvalues

Here

eigenvalue gap

Subexponential Algorithm for Unique Games in time

Idea: quasi-expander small-set expander

non-expanding set

Graph DecompositionBy removing 1% of edges, every graph can be decomposed into components with at most eigenvalues

For ,

follows from Cheeger bound [Cheeger’70, Alon-Milman’85, Alon’86]

Veigenvalue gap

S

Assume graph is d-regular

expansion()

|𝑆|<|𝑉|/2

⇒

at most eigenvalues

For ,

Graph DecompositionBy removing 1% of edges, every graph can be decomposed into components with

Assume graph is d-regular

follows from Cheeger bound

V⇒S

“higher-order Cheeger bound” [here]

general

eigenvalues

small non-expanding set

¿𝑉∨¿𝑛

|𝑆|<|𝑉|/𝒏𝜷

expansion()

“higher-order Cheeger bound”

eigenvalues

¿𝑉∨¿𝑛

𝑛𝑂 ( 𝛽)

⇒

|𝑆|<|𝑉|/𝑛𝛽

expansion()

“higher-order Cheeger bound”

eigenvalues ¿𝑉∨¿𝑛

𝑛𝑂 ( 𝛽) ⇒|𝑆|<|𝑉|/𝑛𝛽

expansion()

“higher-order Cheeger bound”

eigenvalues ¿𝑉∨¿𝑛

𝑛𝑂 ( 𝛽)

|𝑆|<|𝑉|/𝑛𝛽

expansion()

Idea

⇒

“higher-order Cheeger bound”

eigenvalues ¿𝑉∨¿𝑛

𝑛𝑂 ( 𝛽)

|𝑆|<|𝑉|/𝑛𝛽

expansion()

volume growth in intermediate step

has expansion

( for degree )It would suffice to show:

vertex such that for

⇒

“higher-order Cheeger bound”

eigenvalues ¿𝑉∨¿𝑛

𝑛𝑂 ( 𝛽)

|𝑆|<|𝑉|/𝑛𝛽

expansion()

It would suffice to show:

vertex such that for

collision probability decay in intermediate step s < t

level set of has expansion and size

‖𝐺𝑡𝑒𝑖‖2>𝑛𝛽 /|𝑉|

Heuristic:collision probability

⇒collision probability of

t-step random walk from i

Suffices

local Cheeg

er bound

Heuristic:collision probability

collision probability decay in intermediate step s < t

“higher-order Cheeger bound”

eigenvalues ¿𝑉∨¿𝑛

𝑛𝑂 ( 𝛽)

|𝑆|<|𝑉|/𝑛𝛽

expansion()

It would suffice to show:

vertex such that for

level set of has expansion and size

‖𝐺𝑡𝑒𝑖‖2>𝑛𝛽 /|𝑉|

¿𝑛𝑂 (𝛽) (1−𝜀 )2 𝑡¿𝑛𝛽

“A Markov chain with many large eigenvalues cannot mix locally everywhere”

⇒collision probability of

t-step random walk from i

Suffices

Improved approximations for d-TO-1 GAMES ( Khot’s d-to-1 Conjecture) [S.’10]

Better approximations for MAX CUT and VERTEX COVER on small-set expanders

Open Questions

Example: -approximation for SPARSEST CUT in time

How many large eigenvalues can a small-set expander have?

Is Boolean noise graph the worst case? (large eigenvalues)

Thank you!Questions?

More Subexponential Algorithms

Similar approximation for MULTI CUT and SMALL SET EXPANSION

What else can be done in subexponential time?

Towards refuting the Unique Games Conjecture

Better approximations for MAX CUT or VERTEX COVER on general instances?