recent progress in approximability. administrivia most agreeable times: monday 2:30-4:00 wednesday...
TRANSCRIPT
Recent Progress in Approximability
Administrivia
Most agreeable times:
Monday 2:30-4:00Wednesday 4:00-5:30Thursday 4:00-5:30Friday 1:00-2:30
Please Fill Up Survey: http://www.surveymonkey.com/s/9TSVQM7
Evaluation: 6-8 short homeworks and class participation.
Max Cut
10
15
3
7
11
Max CUTInput: A weighted graph G
Find:A Cut with maximum number/weight of crossing edges
Fraction of crossing edges
MaxCut is NP-complete
(Karp’s original list of 21 NP-complete problems (1971)
An algorithm A is an α-approximation for a problem if for every instance I,
A(I) ≥ α OPT(I)∙
--Vast Literature--
Approximation Algorithms
Max Cut
10
15
3
7
11
Max CUTInput: A weighted graph G
Find:A Cut with maximum number/weight of crossing edges
Trivial ½ Approximation
Assign each vertex randomly to left or right side of the cutAnalysis
For every edge e,
Probability[edge is cut] = ½
Fraction of edges cut = ½
Optimum MaxCut < 1
So,
Solution returned = ½ > ½ *Optimum MaxCut
Till 1994, this was the state of the art.
Many linear programming techniques were known to NOT get any better approximation.
The ToolsTill 1994,A majority of approximation algorithms directly or indirectly relied on Linear Programming.
In 1994,Semidefinite Programming based algorithm for Max Cut
[Goemans-Williamson]
Semidefinite Programming - A generalization of Linear Programming.
Semidefinite Programming is the one of the most powerful tools in approximation algorithms.
Eji
jiij vvw),(
2||4
1
Semidefinite Program
Variables : v1 , v2 … vn
| vi |2 = 1
Maximize
Max Cut SDP
Quadratic Program
Variables : x1 , x2 … xn
xi = 1 or -1
Maximize
10
15
3
7
11
1
1
1
-1
-1
-1
-1-1
-1
Eji
jiij xxw),(
2)(4
1
Relax all the xi to be unit vectors instead of {1,-1}. All products are replaced by inner products of vectors
1 -1
Semidefinite Program:[Goemans-Williamson 94]
Embedd the graph on the N - dimensional unit ball, Maximizing
¼ (Average Squared Length
of the edges)
Eji
jiij vvw),(
2||4
1
Semidefinite Program[Goemans-Williamson 94]
Variables : v1 , v2 … vn
|vi|2 = 1
Maximize
MaxCut
10
15
3
7
11
1
1
1
-1
-1
-1
-1-1
-11 -1
Max Cut ProblemGiven a graph G,Find a cut that maximizes the number of crossing edges
v1
v2
v3
v4
v5
MaxCut Rounding
v1
v2
v3
v4
v5
Cut the sphere by a random hyperplane, and output the induced graph cut.
-A 0.878 approximation for the problem.
[Goemans-Williamson]
Analysisv1
v2v3
v4
v5
SDP Optiumum
1015
3
711
OptimalMaxCut
v1
v2v3
v4
v5
Algorithm’sOutput
0 1
Rounding Ratio > 0.878
Integrality Gap
Algorithm Output > 0.878 X SDP Optimum > 0.878 X Optimum MaxCut
minimumover all instances
=
value of rounded solution
value of SDP solution
rounding – ratioA
(approximation ratio)≤ integrality gap
=
value of optimal solution
value of SDP solution
minimumover all instances
For any rounding algorithm A, and a SDP relaxation ¦
v1v2 v3
v4v5
SDP Optiumum
10153
711OptimalMaxCut
v1v2 v3
v4v5
Algorithm’sOutput
0 1
Rounding Ratio > 0.878
Integrality Gap
=“algorithm achieves the gap’’
InapproximabilityIs 0.878 the best possible approximation ratio for MaxCut?
Satisfiable
Unsatisfiable
MaxCut value = K
MaxCut value < K
10
15
3
7
11
1
1
1
-1
-1
-1-1
3-SAT InstancePolynomial time
reduction
What we need..
(Completeness)
Satisfiable
(Soundness)
Unsatisfiable
MaxCut value = K
MaxCut value < 0.9K
10
15
3
7
11
1
1
1
-1
-1
-1-1
3-SAT InstancePolynomial time
reduction
If we had a polytime 0.95 approximation algorithm for MaxCut
A polytime algorithm for 3-SAT
A probabilistically checkable proof (PCP)
Goal: Alice wants to prove to Bob that 3-SAT instance A is satisfiable
3-SAT Instance A
Alex Bob (polytime machine)Satisfying assignment
A probabilistically checkable proof (PCP)
Goal: Alice wants to prove to Bob that 3-SAT instance A is satisfiable
3-SAT Instance A
Alex Bob (polytime machine)
10
15
3
7
11
1
1
1
-1
-1
-1-1
Polynomial
time
reduction
3-SAT Instance A
10
15
3
7
11
1
1
1
-1
-1
-1-1
Polynomial
time
reduction
Probabilistically Checkable ProofA cut of value > 0.9
Verifier (Bob):
Sample a random edge in graph,
Accept if edge is cut.
Prob[Bob Accepts] =
Value of the Cut
Suppose,
(Completeness)
Satisfiable(Soundness)
Unsatisfiable
MaxCut value = 0.99
MaxCut value < 0.9
10
15
3
7
11
1
1
1
-1
-1
-1-1
3-SAT InstancePolynomial time
reduction
Completeness: There exists a ``proof” that Bob accepts with probability 0.99Soundness: No matter what Alex does, Bob accepts with probability < 0.9
Bob reads only 2 bits of the proof!!
Analogy to Math Proofs
Could you check the proof of a theorem with any reasonable confidence by reading only 3 bits of the proof???
Guess: Probably Not..
Max-SNP complexity class was defined, because it was believable that
we will never be able to get a Gap Reduction aka Probabilistically Checkable Proof for NP.
PCP Theorem: [Arora-Lund-Motwani-Sudan-Szegedy 1991]
Max-3-SAT is NP-hard to approximate better than 1- 10^{-100}.
Corollary:Max-Cut is NP-hard to approximate better than 1- 10^{-200}.
Long and very difficult proof, simplified over the years..(*Check out History of PCP Theorem:
http://www.cs.washington.edu/education/courses/cse533/05au/pcp-history.pdf)
Completely new proof by Irit Dinur in 2005.
Hastad’s 3-Query PCP [Håstad STOC97]
For any ε > 0, NP has a 3-query probabilistically checkable proof system such that:
• Completeness = (1 – ε) • Soundness = 1/2 + ε
Verifier reads only 3-bits, and checks a linear equation on them!
Xi + Xj = Xk + c (mod p)
Alternately,
Hastad’s 3-Query PCP [1997]
For any ε > 0, given a set of linear equations modulo 2 , it is NP-hard to distinguish between:
• (1 – ε) – fraction of the equations can be satisfied.• 1/2 + ε – fraction of the equations can be satisfied.
All equations are of the form Xi + Xj = Xk + c (mod p)
By Very Clever Gadget reductions, [Sudan-Sorkin-Trevisan-Williamson]
MaxCut is NP-hard to approximate beyond 0.94.
ALGORITHMS[Charikar-Makarychev-Makarychev 06]
[Goemans-Williamson][Charikar-Wirth]
[Lewin-Livnat-Zwick][Charikar-Makarychev-Makarychev 07]
[Hast] [Charikar-Makarychev-Makarychev 07]
[Frieze-Jerrum][Karloff-Zwick]
[Zwick SODA 98][Zwick STOC 98]
[Zwick 99][Halperin-Zwick 01]
[Goemans-Williamson 01][Goemans 01]
[Feige-Goemans][Matuura-Matsui]
[Trevisan-Sudan-Sorkin-Williamson]
Approximability of CSPsGap for MaxCUTAlgorithm = 0.878Hardness = 0.941
MAX CUT
MAX 2-SAT
MAX 3-SAT
MAX 4-SAT
MAX DI CUT
MAX k-CUT
Unique GamesMAX k-CSP
MAX Horn SAT
MAX 3 DI-CUTMAX E2 LIN3
MAX 3-MAJ
MAX 3-CSPMAX 3-AND
0 1
NP HARD
Given linear equations of the form:
Xi – Xk = cik mod p
Satisfy maximum number of equations.
x-y = 11 (mod 17)x-z = 13 (mod 17)
…….
z-w = 15(mod 17)
Unique Games Conjecture [Khot 02] [KKMO]
For every ε> 0, for large enough p,Given : 1-ε (99%) satisfiable system,
NP-hard to satisfyε (1%) fraction of equations.
Towards bridging this gap, In 2002, Subhash Khot introduced the
Unique Games Conjecture
Unique Games Conjecture
A notorious open problem.
Hardness Results: No constant factor approximation for unique games. [Feige-Reichman]
Algorithm On (1-Є) satisfiable instances
[Khot 02]
[Trevisan]
[Gupta-Talwar] 1 – O(ε logn)
[Charikar-Makarychev-Makarychev]
[Chlamtac-Makarychev-Makarychev]
[Arora-Khot-Kolla-Steurer-Tulsiani-Vishnoi]
)2/( p)loglog(1 pnO
)log(1 3 nO
))/1log((1 5/12 pO
1log1
Assuming UGCUGC Hardness
Results[Khot-Kindler-Mossel-O’donnell]
[Austrin 06][Austrin 07]
[Khot-Odonnell][Odonnell-Wu]
[Samorodnitsky-Trevisan]
NP HARDUGC HARD
0 1
MAX CUT
MAX 2-SAT
MAX 3-SAT
MAX 4-SAT
MAX DI CUT
MAX k-CUT
Unique GamesMAX k-CSP
MAX Horn SAT
MAX 3 DI-CUTMAX E2 LIN3
MAX 3-MAJ
MAX 3-CSPMAX 3-ANDFor MaxCut, Max-2-SAT,
Unique Games based hardness=
approximation obtained by Semidefinite programming!
The Connection
MAX CUT
MAX 2-SAT
MAX 3-SAT
MAX 4-SAT
MAX DI CUT
MAX k-CUT
Unique GamesMAX k-CSP
MAX Horn SAT
MAX 3 DI-CUTMAX E2 LIN3
MAX 3-MAJ
MAX 3-CSPMAX 3-AND
UGC Hard
GENERICALGORITHM
Theorem:Assuming Unique Games Conjecture, For every CSP, “the simplest semidefinite programs give the best approximation computable efficiently.”
Constraint Satisfaction Problems [Raghavendra`08][Austrin-Mossel]
MAX CUT [Khot-Kindler-Mossel-ODonnell][Odonnell-Wu]
MAX 2SAT [Austrin07][Austrin08]
Ordering CSPs [Charikar-Guruswami-Manokaran-Raghavendra-Hastad`08]
MAX ACYCLIC SUBGRAPH, BETWEENESS
Grothendieck Problems [Khot-Naor, Raghavendra-Steurer]
Metric Labeling Problems [Manokaran-Naor-Raghavendra-Schwartz`08]
MULTIWAY CUT, 0-EXTENSION
Kernel Clustering Problems [Khot-Naor`08,10]
Strict Monotone CSPs [Kumar-Manokaran-Tulsiani-Vishnoi`10]
VERTEX COVER [Khot-Regev], HYPERGRAPH VERTEX COVER
Assuming the Unique Games Conjecture,
A simple semidefinite program (Basic-SDP) yields the optimal approximation ratio for
Is the conjecture true?
Many many ways to disprove the conjecture! Find a better algorithm for any one of these problems.
The UG Barrier
Constraint Satisfaction Problems
Graph Labelling Problems
Ordering CSPs
Kernel Clustering Problems
Monotone Min-One CSPs
UGC HARD
If UGC is true,
Then Simplest SDPs give the best approximation possible.
If UGC is false,
Hopefully, a new algorithmic technique will arise.
What if UGC is false?
Could existing techniques ( LPs/SDPs) disprove the UGC?
What if UGC is false?
UGC is false New algorithms?
Unique Games
Constraint Satisfaction Problems [Raghavendra`08]MAX CUT, MAX 2SAT
Ordering CSPs [GMR`08]MAX ACYCLIC SUBGRAPH, BETWEENESS
Grothendieck Problems [KNS`08, RS`09]
Metric Labeling Problems [MNRS`08]MULTIWAY CUT, 0-EXTENSION
Kernel Clustering Problems [KN`08,10]
Strict Monotone CSPs [KMTV`10]VERTEX COVER, HYPERGRAPH VERTEX COVER
…
Problem X
UGC is false New algorithm for Problem X
Despite considerable efforts,No such reverse reduction known for any of the above problems
[Feige-Kindler-Odonnell,Raz’08, BHHRRS’08]
Graph Expansion
d-regular graph G
d
expansion(S) = # edges leaving S
d |S|
vertex set S
A random neighbor of a random vertex in S is outside of S with probability expansion(S)
ФG = expansion(S)minimum|S| ≤ n/2
Conductance of Graph G
Uniform Sparsest Cut Problem Given a graph G compute ФG and find the set S achieving it.
Approximation Algorithms:
•Cheeger’s Inequality [Alon][Alon-Milman] Given a graph G, if the normalized adjacency matrix has second eigen value λ2 then,
•A log n approximation algorithm [Leighton-Rao 98-99?].•A sqrt(log n) approximation algorithm using semidefinite programming [Arora-Rao-Vazirani 2004].
)1(22
)1(2
2
G
Extremely well-studied, many different contexts
pseudo-randomness, group theory, online routing,
Markov chains, metric embeddings, …
A Reverse Reduction
Graph (Social Network)
Close-knitcommunity
Finding Small Non Expanding Sets
Suppose there exists is a small community say
(0.1% of the population)
99% of whose friends are within the community..
Find one such close-knit community.
Theorem [R-Steurer 10]UGC is false New algorithms to approximate expansion of small sets in graphs
STILL OPEN:
Reverse reduction from Max Cut or Vertex Cover to Unique Games.
What if UGC is false?
Could existing algorithmic techniques (LPs/SDPs) disprove the UGC?
Could LPs/SDPs disprove the UGC?
Question I:
Could some small LINEAR PROGRAM give a better approximation for MaxCut or Vertex Cover
thereby disproving the UGC?
Probably Not!
[Charikar-Makarychev-Makarychev][Schoenebeck-Tulsiani]
For MaxCut, for several classes of linear programs,
exponential sized linear programs are necessary to even beat the trivial ½ approximation!
Question II:
Could some small SEMIDEFINITE PROGRAM give a better approximation for MaxCut or Vertex Cover
thereby disproving the UGC?
We don’t know.
v1
v2
v3
v4
v5
Max Cut SDP:
Embedd the graph on the N - dimensional unit ball,
Maximizing
¼ (Average squared length of
the edges)
In the integral solution, all the vectors vi are 1,-1. Thus they satisfy additional constraintsFor example : (vi – vj)2 + (vj – vk)2 ≥ (vi – vk)2
(the triangle inequality)
The Simplest Relaxation for
MaxCut
Does adding triangle inequalities improve approximation ratio?(and thereby disprove UGC!)
[Arora-Rao-Vazirani 2002]
For SPARSEST CUT, SDP with triangle inequalities gives approximation.
An -approximation would disprove the UGC!
[Goemans-Linial Conjecture 1997] SDP with triangle inequalities would yield -approximation for SPARSEST CUT.
[Khot-Vishnoi 2005]
SDP with triangle inequalities DOES NOT give approximation for SPARSEST CUT
SDP with triangle inequalities DOES NOT beat the Goemans-Williamson 0.878 approximation for MAX CUT
Until 2009:
Adding a simple constraint on every 5 vectorscould yield a better approximation for MaxCut, and disproves UGC!
Building on the work of [Khot-Vishnoi],
[Khot-Saket 2009][Raghavendra-Steurer 2009]
Adding all valid local constraints on at most vectors to the simple SDP DOES NOT improve the approximation ratio for MaxCut
[Barak-Gopalan-Hastad-Meka-Raghavendra-Steurer 2009]
Change to in the above result.As of Now:
A natural SDP of size (the round of Lasserre hierarchy) could disprove the UGC.
[Barak-Brandao-Harrow-Kelner-Steurer-Zhou 2012] round of Laserre hierarchy solves all known instances of Unique Games.
Constraint Satisfaction Problems
Max 3 SAT
Find an assignment that satisfies the maximum number of clauses.
))()()(( 145532532321 xxxxxxxxxxxx
VariablesFinite Domain Constraints
{x1 ,x2 , x3 , x4 , x5}{0,1}Clauses
Kind of constraints permitted Different CSPs
Deeper understanding of the UGC – why it should be true if it is.
Why play this game?
Connections between SDP hierarchies, Spectral Graph Theory and Graph Expansion.
New algorithms based on SDP hierarchies.
[Raghavendra-Tan] Improved approximation for MaxBisection using SDP hierarchies
[Barak-Raghavendra-Steurer]
Algorithms for 2-CSPs on low-rank graphs.
New Gadgets for Hardness Reductions:[Barak-Gopalan-Hastad-Meka-Raghavendra-Steurer]
A more efficient long code gadget.