2 gamarnik
TRANSCRIPT
Power and limits of local algorithms for sparse
random graphs
David Gamarnik
MIT
Joint work with
David Goldberg, Theophane Weber and Madhu Sudan
Workshop on Random Graphs and Their Applications
Яндекс
October, 2013
• Real life natural, technological communication networks are large
(WWW 109, neurons in a human brain 1011)
• This makes computational models traditionally deemed scalable
(polynomial time), too impractical
• Local algorithms – promising framework
• Correlation Decay – property which validates accuracy of local
algorithms
Some high level thoughts…
CD arises in statistical phyisics -- uniqueness of Gibbs measures on infinite
graphs, Dobrushin [60-70s]
Correlation Decay Property
Correlation Decay Property
Decisions based on local neighborhood ¼ decisions based on the entire
network:
Electrical Engineering: Belief Propagation (BP) algorithm
Wainwright & Jordan [2008], Mezard & Montanari [2009]
Theoretical Computer Science:
Nguyen & Onak [2008], Rubinfeld, Tamir, Vardi & Xie [2011], Suomella
[2011]
Mathematics:
Hatami, Lovasz & Szegedy [2012], Elek & Lippner [2010], Lyons &
Nazarov [2011], Czoka & Lippner [2012], Aldous [2012].
Context – graph limits.
Local Algorithms
Local algorithms and CD for combinatorial optimization.
Maximum Independent Set (MIS) problem
Local algorithms and CD for combinatorial optimization.
Maximum Independent Set (MIS) problem
Berman and Karpinski [1999]. NP-hard to approximate for
• Extensions to arbitrary constant degree graphs with weights which are
mixtures of exponentials.
• Proof technique – local algorithm (Cavity Expansion) and correlation decay
Maximum Weight Independent Set (MWIS) problem
Computational results.
G & Yedidia [2013]
Instance I
UB: 569
LB: 568
GREEDY: 420
Instance II
UB: 542
LB: 540
GREEDY: 397
Instance III
UP: 514
LB: 513
GREEDY 392
Instance IV
UB: 499
LB: 498
GREEDY: 384
Cavity Expansion Algorithm for general (non-tree) graphs
Cavity Expansion
Cavity Expansion
Problem: node cavity ! set cavity
Solution inspired by Weitz [2006] – expand set cavity as a sum of node cavities
Cavity Expansion
Theorem. Operator
is contracting – correlation decay takes place.
Cavity Expansion Algorithm
1. Compute using depth-m computation tree
2. Include iff
- Computation tree
Correlation decay property:
Probability of error
Computation time PTAS
Generalization:Optimization on networks with random rewards Economic theory of teams
Theorem. [G, Goldberg and Weber 2009]. Suppose
Then the model exhibits the correlation decay property. As a result the
Cavity Expansion algorithm provides an asymptotically optimal solution.
Generalization: Optimization on networks with random rewards
Random n-node d-regular graph
Local algorithms as i.i.d. factors. Mathematicians
perspective
Local algorithms – i.i.d. factors
1. Fix r>0 and
2. Generate U1,…,Un uniformly at random and apply at every node
i=1,…,n of the graph
3. Output
Local algorithms – i.i.d. factors
Conjecture. Hatami, Lovasz & Szegedy [2012] Max-Independent-Set
on random regular graph problem can be solved by means of local
algorithms. Namely there exists a sequence of functions such that
Conjecture holds for Max-Matching:
Lyons & Nazarov [2011]
Abert, Csoka, Lippner & Terpai [2012]
Best known bound 0.4361 for 3-regular graph is obtained by i.i.d. factors
Csoka, Gerencsér, Harangi, & Virag [2013]
Asymptotics of optimal solutions
Theorem. Bayati,G,Tetali 2010 problem exists w.h.p.
Theorem. Achlioptas & Peres [2003] K-SAT problem. Satisfiability
threshold
Theorem. Frieze & Luczak 1992
Best known algorithms
Greedy algorithms for Max Ind Set problem
Greedy algorithms for K-SAT problem. Finds satisfying assignment when
the clauses to variables ratio is at most
Theorem. [G & Sudan 2012] HLS conjecture is not valid. No local algorithm
can produce an independent set larger than factor of the
optimal for large enough d.
Proof relies on the Spin Glass Theory - clustering (shattering) phenomena
Main Result
Mezard & Parisi [1980s]
Achlioptas & Ricci-Tersenghi [2004]
Mezard, Mora & Zecchina [2005]
Krzakala, Montanari, Ricci-Tersenghi, Semerjian, Zdeborova [2007]
Clustering Phenomena. K-SAT
Clustering Phenomena. Independent Sets
Theorem. [Coja-Oghlan & Efthymiou 2010, G & Sudan 2012]
For every large enough d and every there exist
such that no two independent sets of size
have intersection size between and
Proof of non-existence of local algorithms: if an algorithm exists then
one can construct two independent sets with intersection in the non-
existent region
First direct link between clustering and algorithmic hardness
Proof of non-existence of local algorithms: if an algorithm exists then
one can construct two independent sets with intersection in the non-
existent region
Suppose exists such that
Construct two independent sets I and J using independent sources
U1,…,Un and V1,…,Vn
Then
Proof of non-existence of local algorithms: if an algorithm exists then
one can construct two independent sets with intersection in the non-
existent region
Continuously interpolate between U1,…,Un and V1,…,Vn : let
For each vertex , let with probability p and with
probability 1-p.
Consider independent set K obtained from
K=I when p=1 and K=J when p=0.
Fact: is continuous in p. Then we obtain intersection sizes for
all points in
- contradiction.
Theory is good not when it is correct but when it is interesting…
Ongoing work
Local algorithms (Belief Propagation and Survey Propagation) to solve
random instance of NAE-K-SAT problem
This would refute a conjecture put forward by physicists regarding the
power of these algorithms
Thank you