Yuan Zhou, Ryan O’DonnellCarnegie Mellon University

Constraint Satisfaction Problems• Given:

– a set of variables: V– a set of values: Ω– a set of "local constraints": E

• Goal: find an assignment σ : V -> Ω to maximize #satisfied constraints in E

• α-approximation algorithm: always outputs a solution of value at least α*OPT

Example 1: Max-Cut• Vertex set: V = {1, 2, 3, ..., n}• Value set: Ω = {0, 1}• Typical local constraint: (i, j) in E wants σ(i) ≠

σ(j)

• Alternative description:– Given G = (V, E), divide V into two parts,– to maximize #edges across the cut

• Best approx. alg.: 0.878-approx. [GW'95]• Best NP-hardness: 0.941 [Has'01, TSSW'00]

Example 2: Balanced Separator• Vertex set: V = {1, 2, 3, ..., n}• Value set: Ω = {0, 1}• Minimize #satisfied local constraints: (i, j) in E : σ(i) ≠ σ(j)• Global constraint: n/3 ≤ |{i : σ(i) = 0}| ≤

2n/3

• Alternative description:– given G = (V, E)– divide V into two "balanced" parts,– to minimize #edges across the cut

Example 2: Balanced Saperator (cont'd)

• Vertex set: V = {1, 2, 3, ..., n}• Value set: Ω = {0, 1}• Minimize #satisfied local constraints: (i, j) in E : σ(i) ≠ σ(j)• Global constraint: n/3 ≤ |{i : σ(i) = 0}| ≤

2n/3

• Best approx. alg.: sqrt{log n}-approx. [ARV'04]

• Only (1+ε)-approx. alg. is ruled out assuming 3-SAT does not have subexp time alg. [AMS'07]

Example 3: Unique Games• Vertex set: V = {1, 2, 3, ..., n}• Value set: Ω = {0, 1, 2, ..., q - 1}• Maximize #satisfied local constraints: {(i, j), c} in E : σ(i) - σ(j) = c (mod q)

• Unique Games Conjecture (UGC) [Kho'02, KKMO'07]

No poly-time algorithm, given an instance where optimal solution satisfies (1-ε) constraints, finds a solution satisfying ε constraints

• Stronger than (implies) "no constant approx. alg."

Open questionsIs UGC true?

Is Max-Cut hard to approximate better than 0.878?

Is Balanced Separator hard to approximate with in constant factor?

Easier questionsDo the known powerful optimization algorithms

solve UG/Max-Cut/Balanced Separator?

SDP Relaxation hierarchies• A systematic way to write tighter and tighter

SDP relaxations

• Examples– Sherali-Adams+SDP [SA'90]– Lasserre hierarchy [Par'00, Las'01]

…?

UG(ε)

r -round SDP relaxation in roughly time)(rOn

BASIC-SDP

GW SDP for Maxcut (0.878-approx.)ARV SDP for Balanced Separator

How many rounds of tighening suffice?• Upperbounds

– rounds of SA+SDP suffice for UG [ABS'10,

BRS'11]• Lowerbounds [KV'05, DKSV'06, RS'09, BGHMRS

'12] (also known as constructing integrality gap

instances)– rounds of SA+SDP needed

for UG– rounds of SA+SDP needed

for better-than-0.878 approx for Max-Cut– rounds for SA+SDP needed for

constant approx. for Balanced Separator

)1(n

))logexp((log )1(n

)1()log(log n

))logexp((log )1(n

From SA+SDP to Lasserre SDP• Are the integrality gap instances for SA+SDP

also hard for Lasserre SDP?

• Previous result [BBHKSZ'12]– No for UG– 8-round Lasserre solves the Unique Games

lowerbound instances

From SA+SDP to Lasserre SDP (cont’d)

• Are the integrality gap instances for SA+SDP also hard for Lasserre SDP?

• This paper– No for Max-Cut and Balanced Separator– Constant-round Lasserre gives better-than-

0.878 approximation for Max-Cut lowerbound instances

– 4-round Lasserre gives constant approximation for the the Balanced Separator lowerbound instances

Proof overview • Integrality gap instance

– SDP completeness: good vector solution– Integral soundness: no good integral

solution

• Show the instance is not integrality gap instance for Lasserre SDP – no good vector solution– we bound the value of the dual of the SDP– interpret the dual as a proof system (”SOSd/sum-of-squares proof system")– lift the soundness proof to the proof

system

What is the SOSd proof system?

Polynomial optimization• Maximize/Minimize• Subject to

all functions are low-degree n-variate polynomials

• Max-Cut example: Maximize s.t.

)(xp

0)(,0)(,0)( 21 xqxqxq m0)(,0)(,0)( '21 xrxrxr m

2)(E jiE(i,j)xx

ixx ii ,0)1(

Polynomial optimization (cont'd)• Maximize/Minimize• Subject to

all functions are low-degree n-variate polynomials

• Balanced Separator example: Minimize s.t.

)(xp

0)(,0)(,0)( 21 xqxqxq m0)(,0)(,0)( '21 xrxrxr m

32

31 ][E,][E

,0)1(

iiii

ii

xxixx

2)(E jiE(i,j)xx

Certifying no good solution• Maximize• Subject to

• To certify that there is no solution better than , simply say that the following equalities & inequalities are infeasible

)(xp

0)(,0)(,0)( 21 xqxqxq m0)(,0)(,0)( '21 xrxrxr m

)(xp0)(,0)(,0)( 21 xqxqxq m0)(,0)(,0)( '21 xrxrxr m

The Sum-of-Squares proof system• To show the following equalities &

inequalities are infeasible,

• Show that

• where is a sum of squared polynomials, including 's

• A degree-d "Sum-of-Squares" refutation, where

0)(,0)(,0)( 21 xqxqxq m0)(,0)(,0)( '21 xrxrxr m

)}deg(),deg(){deg(max hqfd iii

)()()(1...1

xhxqxfmi

ii

)(xh)(xri

PositivstellensatzSubject to some mild technical conditions,

every infeasible system has such a refutation

Caveat: fi’s and h might need to have high degree.

Lasserre SDP and SOSd proof systemA degree-d SOS refutation

O(d)-round Lasserre SDP is infeasible

Summary• Defined the degree-d SOS proof system

• Remaining task Integral soundness proof low-degree refutation in the SOS proof system

Example 1• To refute

• We simply write

• A degree-2 SOS refutation

2)1()2()1(1 xxxx

0)1(2

xx

x

One-slide How-to

Thm: Min-Balanced-Separator in this graph is ≥ blahProof: … hypercontractivity…“Check out these polynomials.”

Thm: Max-Cut of this graph is ≤ blahProof: … Invariance Principle … … Majority-Is-Stablest… “Check out these polynomials.”

Example 2: Max-Cut on triangle graph• To refute

• We "simply" write ... ...

0)1(,0)1(,0)1( 332211 xxxxxx

2)()()( 213

232

221 xxxxxx

)12)(1()3222)(1(

)12)(1(

)1()1()(

2)()()(

212133

313123122

3223

2211

232

221

22313221

213

232

221

xxxxxxxxxxxxxx

xxxxxx

xxxxxxxxxxx

xxxxxx

Example 2: Max-Cut on triangle graph (cont'd)

• A degree-4 SoS refutation

Latest results• Our theorem on Max-Cut is improved by

[DMN’12]– Constant-round Lasserre SDP almost exactly

solves the known instances

• Constant-round Lasserre SDP solves the hard instances for Vertex-Cover [KOTZ’12]Open question

• Does constant-round Lasserre SDP solve the known instances for all the CSPs?– I.e. SOS-ize Raghavendra’s theorem.

Thank you!