1

Paul Beame University of Washington

Satisfiability and Unsatisfiability:Proof Complexity and Algorithms

2

Outline

Intro to proof complexityProof complexity and complete SAT algs Review Resolution/DPLL connection Application to random k-SAT Proof complexity and DPLL extensions

Clause learningCaching

Proof system survey

3

Proof Complexity

Study of the number of symbols required for proofs of unsatisfiability (or tautology) in propositional logic

Does not address algorithmic issue How would you find short proofs if they

existed?

Existence of short proofs for every unsatisfiable formula is equivalent to NP = co-NP (and is implied by P=NP) Generally believed that such proofs don’t exist

4

Propositional proof systems

A propositional proof system is a polynomial time algorithm V s.t. for all formulas F

F is unsatisfiable there exists a string P s.t. V accepts input (P,F)Note:

direction is usually called soundness direction is usually called completeness

Proof complexity is size of P as a function of the size of F

5

Sample propositional proof systems

Truth tables proof is a fully filled out truth table

easy to verify that it is filled out correctly and all truth assignments yield T

Frege systems inference rules: e.g. modus ponens

A, (A B) | B axioms: e.g. excluded middle | (A A) axioms & inference rules are schemas

can make consistent substitution of arbitrary formulas for variables in schema

e.g. excluded middle yields ((xy) (xy))

6

The graph of a proof

Axioms/inputs are sources

F7F1

F11

F3

F4

F8

F9 F12

F2

F10

F5 F6

F13

Inference ruleassociated with each node

Sink labelled by tautology (or for refutation)

7

Resolution

Start with clauses of CNF formula F

Resolution rule Given (A x), (B x) derive

(A B)

The empty clause is derivable F is unsatisfiable

Proof size = # of clauses used

8

Restricted forms of Resolution

Resolution In general, graph of inferences is a directed

acyclic graph

Tree Resolution Graph of inferences forms a binary tree

i.e., if you want to use a clause more than once you need to re-derive it

Regular Resolution Once a variable is resolved out of a clause

you can’t add it back later More general than Tree Resolution

9

Satisfiability Algorithms

Incomplete Algorithms Local search

GSAT [Selman,Levesque,Mitchell 92] Walksat [Kautz,Selman 96]

Belief PropagationSP [Braunstein, Mezard, Zecchina 02]

Complete Algorithms Backtracking search

DPLL [Davis,Putnam 60]

[Davis,Logeman,Loveland 62]DPLL + clause learning GRASP, SATO, zchaff, berkmin

etc

10

Proof Complexity and SAT Algorithms

Proof Complexity Unsatisfiable formulas Nondeterministic algorithms

SAT Algorithms Satisfiable formulas Deterministic or randomized

algorithms

11

Proof complexity and complete satisfiability

algorithmsIn general The transcript of the run of a

complete satisfiability algorithm on an unsatisfiable formula is a proof of its unsatisfiability

For backtracking style algorithms Even runs on satisfiable formulas

yield proofs of unsatisfiability of related formulas

12

Backtracking search/DPLL

DPLL(F) while F contains a 1-clause x

F F|x=1

if F is empty return 1 if F has an empty clause return 0

else select a literal x to branch on return DPLL(F|x=1) DPLL(F|x=0)

Residual formula

13

Resolution and DPLL

DPLL tree with any variable selection rule on an unsatisfiable formula F generates a Resolution refutation of F # of clauses running time

Note: Regular resolution corresponds to a similar DAG (read-once branching program) rather than a tree but general resolution has no such analogue

14

DPLL Refutation

Clauses1. a b c2. ab3. c4. a d5. d b

a

b b

c d3 3

21 4 5

a a

b

d

bb

d

b

a b c ac

b b

a d d b

c c

15

DPLL Refutation= Tree-like Resolution Proof

Clauses1. a b c2. ac3. b4. a d5. a b

a:

b: a b: a

d:a b3 3

21 4 5

a b c ac

b b

a d d b

c: a b

16

Random k-CNF formulas

Randomly choose m clauses over n variables independently each of size k Each size k clause is equally likely

Key parameter r=m/n, the ratio of clauses to variables:

Threshold value rk* of rBelow, almost certainly satisfiableAbove, almost certainly unsatisfiableHardest problems near threshold

17

DPLL on random 3-CNF*

0

1

probability

satisfiable

4.267

ratio of clauses to variables

# of DPLLbacktracks

* n = 50 variables

Proof complexity

shows 2(n/r) time is required for unsatisfiable

formulas for r r3*[B,Karp,Saks,Pitassi 98]

[Ben-Sasson 02]

What about satisfiableformulas below threshold?

r[Mitchell,Selman,Levesque 92]

18

Some simple select choices for DPLL algorithms

UC: Unit Clause/Ordered DLL Choose variables in a fixed order Always set True first

UCwm: Unit Clause with majority Choose variables in a fixed order Apply a majority vote among 3-clauses for

assigning each value

GUC: Generalized Unit Clause Choose a variable v in a shortest clause C Set v to satisfy C

19

Parameters of residual formulas follow trajectories

1

2/3

[Chao,Franco 88]

[Frieze,Suen 95]

[Achlioptas 00]

[Achlioptas,Sorkin 00]UCGUC

3.523-clause ratio

2-c

laus

e r

atio

8/3

[Kaporis et al 03]

0

20

Satisfiability for mixed random formulas: proven

properties

1

4.501

SAT

UNSAT

3.522/3

?

??

?

?

?

?

?

?

?

?

?

2.28

3-clause ratio

2-c

lau

se r

ati

o

[Achlioptas et al 96]

[Kaporis et al 03]

[Dubois 01]

21

Resolution proof complexity of mixed random formulas

Theorem: [Achlioptas, B, Molloy 01] For any constants r21 and r3 0 random formulas with r2n 2-clauses and r3n 3-clauses need exponential-size resolution proofs almost surely

Extends [Chvatal-Szemeredi 88]

22

Long-running DPLL Executions and Proof Complexity

Residual formula at is unsatisfiable

Algorithm’sproof of unsatisfiability is exponentially long

Every resolution2n

Residual formula at each node is a mix of2- and 3-clauses

23

Trajectory on 3-CNF

1UC Algorithm Trajectory

2-c

lau

se r

ati

o

4.51

Provably UNSAT& Hard

3.52 4.267

ProvablySAT & Easy

3-clause ratio

3.81

0

Exponential lower bounds for 3-CNF formulas below ratio 4.267

Corollary For almost all 3-CNF formulas, above ratio 3.81 UC takes exponential time 3.83 UCwm takes exponential time 4.01 GUC takes exponential time

25

Exponential lower bound below the proven k-CNF threshold

[Achlioptas, Peres 02] rk* 2k ln 2 – (k+4)/2

[Achlioptas, B, Molloy 04] For k 4, for almost all k-CNF formulas UC takes exponential time above ratio c 2k/k

Note These formulas have huge numbers of satisfying assignments (more than 2 (1-) n out of a possible 2n) but still are hard

26

Open Problem

Closing gap for unsatisfiability of mixed formulas would yield sharp threshold behavior for each algorithm A Below rA algorithm runs in linear time

Above rA algorithm requires exponential time

Conjecture: no polynomial time selection heuristic will work up to the threshold rA r3* for all algorithms A

Backtracking algorithms for other random problems with phase transitions? e.g. k-colorability on random graphs G(n,r/n)

Unsatisfiable phase exp(cn/rk) [B, Culberson, Mitchell, Moore 03]

27

Outline

Intro to proof complexityProof complexity and complete SAT algs Review Resolution/DPLL connection Application to random k-SAT Proof Complexity and DPLL

extensionsClause learningCaching

Proof system survey

28

Clause Learning

At each backtrack point in DPLL search add new clauses to the input formula that correspond to known causes of failure of the search Critical to the good behavior of the best

current algorithms

What is the power of clause learning? Intuitively: it makes DPLL trees DAG-like regular resolution? general

resolution?

29

Conflict Graph

Decision scheme(p q b)

1-UIP scheme(t)

p

q

b

a

x1

x2

x3

y

yfalset

Known Clauses(p q a)

( a b t)(t x1)(t x2)(t x3)

(x1 x2 x3 y)(x2 y)

Current decisionsp falseq falseb true

30

Proof complexity and clause learning

Clause learning is no stronger than Resolution Implication graph yields resolution

derivation of each learned clause

There are examples where clause learning works in polynomial time but any Regular Resolution proof is exponential [Sabharwal, B, Kautz 04] Clause learning can sometimes be much

stronger than Regular Resolution

31

Proof complexity and clause learning

Separation is true for any proper subsystem R of Resolution for which refuting F|x is no harder than F Clause learning does not have this property

unless it is as powerful as resolution

Idea: Take formula F easy for Resolution but hard for R and create new formula F’ that is F plus clauses for (CpC) for each C in Resolution proof of F where each pC is a new propositional variable Setting each pC to true yields original F but

branching on sequence of pC allows sequence of clauses C to be learned

32

Still Open

Can clause learning do everything regular resolution can?*

Is clause learning as powerful as general resolution?

* Note: Alan Van Gelder has pointed out that if one returns learned clauses up the tree and resolves them while backtracking then clause learning does efficiently simulate regular resolution

33

Proof Complexity and Formula Caching

Formula Caching: Memoization

Cache residual formulas that have been already found to be unsatisfiable

Do not make recursive call if the current residual formula is already cached

34

DPLL(F)if F is empty report satisfiable and haltif F contains the empty clause

backtrackelse select a literal x

DPLL(F|x)

DPLL(F|x)

DPLL

35

FC(F,L)if F is empty report satisfiable and haltif F contains the empty clause or FL

returnelse choose a literal x

FC(F|x,L)

FC(F|x,L)

add F to L

Basic Formula Caching

Start with FC(F,)

36

Proof Complexity of Formula Caching [B, Impagliazzo, Pitassi, Segerlind

03]

Basic Formula Caching can exponentially improve DPLL

Basic Formula Caching is not as powerful as regular resolution but some of its natural extensions are

These natural extensions can be exponentially more powerful than general resolution or even Res(k) for any k ! Res(k) is like resolution but uses k-DNFs

instead of clauses

37

Extensions of Formula Caching

Don’t just check membership in the cache F is trivially unsatisfiable given G

iff every clause of G contains some clause of F

Record the reasons why residual formulas are known to be unsatisfiable and check the reasons rather than the formulas

38

FC*(F,L)if F is empty report satisfiable and haltif F is trivially unsatisfiable given L

add F to Lreturn

else choose a literal xFC*(F|x,L)

FC*(F|x,L)

add F to L

More General Formula Caching

39

FCreason(F,L)if F is empty report satisfiable and haltif F is trivially unsatisfiable based on GL

add F to Lreturn(G)

else choose a literal xGFCreason(F|x,L)

HFCreason(F|x,L)J (xG) (xH) add F,J to Lreturn(J)

Adding Reasons

40

Combining Clause Learning and Formula Caching

Cachet system used to count # of satisfying assignments

[Sang, Bacchus, B., Kautz, Pitassi 04]

Built on zchaff’s clause learning Uses decomposition of residual

formula into components Caches components rather than the

whole formula Subtle interactions

41

Outline

Intro to proof complexityProof complexity and complete SAT algs Resolution/DPLL review Application to random k-SAT DPLL extensions

Clause learningCaching

Proof system survey

42

Proof Systems Hierarchy

Truth Tables

DPLL Nullstellensatz

Polynomial CalculusResolution

Cutting Planes

Frege

AC0-Frege

ZFC

Ext-Frege

PCR

LS

Res(k)

43

AC0-Frege

Constant-depth Frege (AC0-Frege) Like Frege but formulas in proof only

have constant # of alternations between and

Resolution is depth-1 Frege

44

Some Known Hard Problems for Resolution

Exact/Modular Counting Pigeonhole principle: n+1 pigeons can’t fit

into n pigeonholes without getting cosy with each other

Parity Principle: e.g. can’t pair up an odd number of people

Tseitin Tautologies These problems are also hard for AC0-Frege

Random Unsatisfiable k-CNF Graph k-colorability Existence of k-cliques in graphs

45

Cutting Planes/PseudoBoolean

Introduced to relate integer and linear programming [Gomory 59, Chvatal 73]: Objects are linear integer inequalities Clause (x1 x2 x3) becomes

inequality x1+(1-x2)+x3 1

Add inequalities xi 0 and 1-xi 0

Goal: derive 0 1

46

Cutting Planes rules

addition:

multiplication by positive integer:

Division by positive integer:

a1x1 + ... + anxn A

b1x1 + ... + bnxn B

(a1+b1)x1+...+(an+bn)xn A+B

a1x1 + ... + anxn A

ca1x1 + ... + canxn cA

ca1x + ... + canxn B

a1x1 + ... + anxn B/c

47

Limitations of Cutting Planes

At least as good as resolution

Can also handle pigeonhole principle parity principle

But can’t handle counting when it is a derived property Clique-Coloring

A graph can’t have a k-clique and be k-1-colorable

Tseitin tautologies still open

48

Proof Systems Hierarchy

Truth Tables

DPLL Nullstellensatz

Polynomial CalculusResolution

Cutting Planes

Frege

AC0-Frege

ZFC

Ext-Frege

PCR

LS

Res(k)

49

Hilbert’s Nullstellensatz

System of polynomials Q1(x1,…,xn)=0,…,Qm(x1,…,xn)=0

over field K has no solution in any extension field of K there exist polynomials P1(x1,…,xn),…,Pm(x1,…,xn) in K[x1,…,xn] s.t.

m

i ii=1

1QP

50

Nullstellensatz proof system

Clause (x1 x2 x3) becomes equation (1-x1)x2(1-x3)=0

Add equations xi2-xi =0 for each

variable Guarantees only 0-1 solutions

A proof is polynomials P1,…, Pm+n

proving unsatisfiability: i.e. such that j

m n2

j m+iCj=1 i=1

+ (x - x) 1QP P

C

QC

51

Polynomial CalculusSimilar to Nullstellensatz except: Begin with Q1,…,Qm+n as before Given polynomials R and S can infer

aR + bS for any a, b in KxiR

Derive constant polynomial 1 Degree = maximum degree of polynomial

appearing in the proof Can find proof of degree d in time nO(d) using

Groebner basis-like algorithm (linear algebra)

52

PCR = PC + ResolutionTwo variables x and x’ for each proposition x x’ stands for x include equations x+x’-1=0, x2-x=0, and

(x’)2-x’=0

Translate (x1 x2 x3) as x’1 x2 x’3=0

Same proof rules and proof search as polynomial calculus Efficiently simulates resolution: clausemonomial Degree-size relationship for PCR is same as width-

size relationship of resolution

53

Lovasz-Schrijver (LS) Proofs

Linear inequalitiesSteps: Lift to get degree 2 inequalities

E.g. multiply inequalities Project

Combine degree 2 inequalities plus x2=x to cancel out degree 2 terms

Captures properties of semi-definite programming

54

Proof Systems Hierarchy

Truth Tables

DPLL Nullstellensatz

Polynomial CalculusResolution

Cutting Planes

Frege

AC0-Frege

ZFC

Ext-Frege

PCR

LS

Res(k)

55

Conclusions

Exchange of ideas between proof complexity and SAT algorithms can be fruitful Better understanding of existing algorithms Source of ideas for new algorithms and

proof systems

Potential to branch out beyond DPLL derivatives Wide range of proof systems to explore

56

Thank You

Thank you to

Toby Walsh Fahiem Bacchus Ian Miguel