boolean satisfiability the most fundamental np-complete problem, and now a powerful technology for...
TRANSCRIPT
Boolean Satisfiability
The most fundamental NP-complete problem, and now a powerful technology
for solving many real world problems
Overview
• CNF, SAT, 3SAT and 2SAT• Resolution and Unit Propagation• DPLL search• Conflict-driven Nogood Learning• Activity-based Search• Modelling for SAT
Conjunctive Normal Form
• SAT solvers solve problems in
–Conjunctive Normal Form (CNF)• Boolean variable: b (true/false) or (0/1)• Literal: l variable b or its negation –b
– negating a literal: -l = -b if l = b, and
-l = b if l = -b
• Clause: C disjunction or set of literals• CNF: theory T set of clauses
• Assignment: set of literals A with {b,-b} not subset: e.g. {-b1,b2} Assign b1=false, b2=true
Boolean Satisfiability (SAT)
• This is the most basic NP-complete problem.
• SAT: Given a set of clauses T, find an assignment A such that
for each C in T
• Each clause C in T is satisfied by A
€
A ∩ C ≠ ∅
3SAT
• 3SAT: SAT with the restriction that each clause has length at most 3.
• SAT -> 3SAT– {l1, l2, l3, l4, l5, l6} l1 ∨ l2 ∨ l3 ∨ l4 ∨ l5 ∨ l6
becomes the set of clauses– {l1, l2, b1}, {-b1, l3, b2}, {-b2, l4, b3}, {-b3, l5, l6}
where b1, b2, b3 are new Boolean variables
• 3SAT is NP-complete (by the above reduction)
2SAT
• 2SAT: SAT with the restriction that each clause has length at most 2.
• 2SAT is decidable in polynomial time (n3)
• 2SAT is NL-complete– which is a crazy complexity class– Nondeterministic Turing machine with log writeable
memory!
Resolution
• The fundamental inference for CNF– {l1, l2, l3, b} {-b, l4, l5, l6} implies– {l1, l2, l3, l4, l5, l6}
• Or– (l1 ∨ l2 ∨ l3 ∨ b) ∧ (-b ∨ l4 ∨ l5 ∨ l6) ->
l1 ∨ l2 ∨ l3 ∨ l4 ∨ l5 ∨ l6
• One can prove unsatisfiability of a CNF formula by repeatedly applying all possible resolutions– if this generates an empty clause then UNSAT– otherwise SAT
• Exponential process!
Unit Propagation
• Resolution = too expensive• Unit Propagation (=unit resolution)
– Restricted form of resolution = finds unit clauses
• {l1, l2, …, ln, l} where –li in A forall 1 ≤ i ≤ n• Add l to assignment A
• {-b1, b2, b} A = {b1, -b2}– {-b1, b2, b} A := {b1, -b2, b}
• (-b1 ∨ b2 ∨ b) ∧ b1 ∧ -b2 -> b
Unit Propagation
• Repeatedly apply unit propagation, – until no new unit consequences can be found– or failure detected
• A = {b1, -b2, b3, -b4} C = {-b1, -b3, b4}• {-b1, -b3, b4}
• Failure detected
Unit Propagation Example• T = {{-e11,-e21}, {-e11,-e31},{-e11,-e41},
{e21,-b21}, {e31,-b31},{e41,-b41},
{b21,-b51},{b51,-b52,e52},{b41,-e52},• A = {e11,b52}
{-e11,-e21}A ={e11,b52,-e21}
{-e11,-e31}A ={e11,b52,-e21,-e31}
{-e11,-e41}A ={e11,b52,-e21,-e31,-e41}
{-e21,-b21}A ={e11,b52,-e21,-e31,-e41,-b21}
{-e31,-b31}A ={e11,b52,-e21,-e31,-e41,-b21,-b31}
{-e41,-b41}A ={e11,b52,-e21,-e31,-e41,-b21,-b31,-b41}
{b21,-b51}A ={e11,b52,-e21,-e31,-e41,-b21,-b31,-b41,-b51}
{b51,-b52,e52}A ={e11,b52,-e21,-e31,-e41,-b21,-b31,-b41,-b51,e52}
e11
-e21
-e31
-e41
-b21
-b31
-b41
-b51
b52
e52
fail
{b41,-e52}A = false
Implication Graph
• Records the reason why each Boolean literal became true– Decision– Propagation(and why)
• Used for nogood reasoning!
e11
-e21
-e31
-e41
-b21
-b31
-b41
-b51
b52
e52
fail
Unit Propagation Exercise• T = {{-b1,-b2,b3},{-b4,b5},{b2,-b5},
{b2,b7}, {b5,-b11,b9},{b4,-b9},{-b1,-b8,b10},
{-b5,-b10},{b5,b6,-b10} ,{-b8,b11}}• A = {b1,-b3,b8}• Perform unit propagation and draw the implication
graph
DPLL search
• Davis-Putnam-Logemann-Loveland algorithm– interleave decisions + unit propagation
• dpll(A)A' = unitprop(A)
if A' == false return false
else if exists Boolean variable b not appearing in A
if dpll(A' union {b}) return true
else if dpll(A' union {-b}) return true
else return false
else return true
DPLL search with nogood learning
• dpll(A)A' = unitprop(A)
if A' == false
Add a clause C explaining the failure to T
Backjump to the first place C can give new information
else if exists Boolean variable b not appearing in A
if dpll(A' union {b}) return true
else if dpll(A' union {-b}) return true
else return false
else return true
Nogood Learning
• The implication graph shows a reason for failure• Any cut separating the fail node from decisions
– explains the failure
e11
-e21
-e31
-e41
-b21
-b31
-b41
-b51
b52
e52
fail
e52 ∧ -b41 false {b41,-e52}b52 ∧ -b51 ∧-b41 false {b41,b51,-b52}b52 ∧ -b51 ∧-e41 false {e41,b51,-b52}b52 ∧ e11 false {-e11,-b52}
Which Nogood?
• SAT solvers almost universally use the– First Unique Implication Point (1UIP) Nogood
• Closest nogood to failure– only one literal from the last decision level
• Asserting: on backjump it will unit propagate• General: more general than many other choices• Fast: doesn’t require too much computation
– replace last literal in nogood until only one at last level
1UIP Nogood Creation
e11
-e21
-e31
-e41
-b21
-b31
-b41
-b51 b52 e52
b22 e22
-e32
-e42
-b32
-b42
b33
b43
e33
e43
fail
e33 ∧ e43 false {-e33,-e43}
e43
e33
-b42 ∧ b43 ∧ e33 false {b42,-b43,-e33}
b43
{-b32,-b42,b33,b43} false {b32,b42,-b33,-b43}
b33
{-b21,-b32,-b42,b33} false {b21,b32,b42,-b33}
-b32
-b21
-b42
{-b21,-b32,-b42} false {b21,b32,b42} {-b21,-b41,-e42,-b32} false {b21,b41,e42,b32}
-e42-b41
{-b21,-b31,-b41,-e32,-e42} false {b21,b31,b41,e32,e42}
-e32-b31
{-b21,-b31,-b41,e22,-e32} false {b21,b31,b41,-e22,e32}
e22
{-b21,-b31,-b41,e22} false {b21,b31,b41,-e22}
1 UIP Nogood
Backjumping
• Backtrack to second last level in nogood
• Nogood will propagate• e.g. {b21,b31,b41,-e22}
e11
-e21
-e31
-e41
-b21
-b31
-b41
-b51
-e22
{b21,b31,b41,-e22}
Continue unit propagationthen make next choice
-b22
-b52
Why Add Nogoods
• We will not make the same choices again– {e11,b52} leads to failure– After choosing e11 we infer –b52– Better yet, any choice that leads to b21,b31,b41
prevents the choice b52
• Drastic reduction in search space• Faster solving
1UIP Exercise• T = {{-b1,-b2,b3},{-b4,b5},{b2,-b5},
{b2,b7}, {b5,-b11,b9},{b4,-b9},{-b1,-b8,b10},
{-b5,-b10},{b5,b6,-b10},{-b8,b11}}• A = {b1,-b3,b8}• Assume the order of the decisions was
b8, -b3, b1
Determine the 1UIP nogood
Where does execution backjump to?
Activity
• Each time a Boolean variable is seen during nogood creation, i.e.– appears in final nogood, or– is eliminated in the reverse propagation
• Increase activity by one• These variables are helping cause failure• Periodically divide all activities by some amount
– activity reflects helping cause recent failure
Activity-based Search
• Select the unfixed variable with highest activity• MiniSat: set to false• RSAT: set to the last value it took
– works with backjumping to recreate the same path– e.g. -b1, b3, -b11, b4, -b5, b7, fail– backjump to -b1, b3, -b11 – If b4 is now highest activity variable set it true [b4]– If b5 is next highest activity variable set it false [-b5]
• Activity-based search– concentrates on the variables causing failure– learns shorter nogoods by failing earlier
Activity-based Search
• Works well with restart• On restart we concentrate on the now most active
variables– a new part of the search space– learn new nogoods about this
Modern SAT Solvers
• Modern SAT solvers can handled problems with– (low) millions of clauses– millions of variables
assuming the input has structure• Random CNF is much harder (but uninteresting)• Before the advent of nogood learning
– (low) thousands of clauses– hundreds of variables
Modelling for SAT
• Boolean constructs can all be converted to clauses• b1 = b2∨b3
– {-b2,b1}, {-b3,b1}, {b2,b3,-b1}
• b1 = b2∧b3– {-b1,b2}, {-b1,b3}, {b1,-b2,-b3}
• b1 = (b2b3)– {-b2,b3,b1}, {b2,-b3,b1}, {b2,b3,-b1}, {-b2,-b3,-b1}
• etc.
Modelling Integers in SAT
• Two main representations: i in 0..m (m = 2k-1 – 1)• Binary: i = 2k-1ik-1 + 2k-2ik-2 + … + 2i1 + i0
• Unary: i = im + im-1 + … + i1
– where ik ik-1
• Example representations
Integer Binary Unary
11 1011 0000011111111111
0 0000 0000000000000000
6 0110 0000000000111111
Pseudo-Boolean Constraints
• Psuedo-Boolean constraints: bi Boolean
• Cardinality constraints:
• Note that is b1 ∨ … ∨ bm
• Straight clausal definitions are exponential• Encode: Binary Decision Diagram, Binary or
Unary arithmetic• Examine cardinality constraint encodings
€
bii=1
m
∑ ≤ k
€
bii=1
m
∑ = k
€
bii=1
m
∑ ≥ k
€
bii=1
m
∑ ≥ 1
€
aibii=1
m
∑ ≤ k
€
aibii=1
m
∑ = k
€
aibii=1
m
∑ ≥ k
Cardinality Constraints (BDD)
5
1
3i
ix
5
2
2i
ix
5
2
3i
ix1x
a if b then c else d
{-a,-b,c}, {-a,b,d}, {a,-b,-c}, {a,b,-d}
if then else
Cardinality Constraints (BDD)
5
1
3i
ix if then else
5
2
2i
ix
5
2
3i
ix1x
5
2
3i
ix if then else
5
3
2i
ix
5
3
3i
ix2x
5
2
2i
ix if then else
5
3
1i
ix
5
3
2i
ix2x
5
3
1i
ix if then else
5
4
0i
ix
5
4
2i
ix3x∨
∨
∨
∨
Cardinality Constraints (BDD)
5
1
3i
ix
5
3
2i
ix
5
2
2i
ix
true
x1 = 0 x1 = 1
€
xi ≤ 1i=3
5
∑
x2
x3
5
2
3i
ix
€
x4 + x5 ≤ 0
€
x5 ≤ 0
€
x4 + x5 ≤ 1
• Dashed arc: xi = 0
• Full arc: xi = 1• Number of intermediatestates is (n – k) ×k
x4
x5
x1
BDD Exercise
• Draw the BDD for the cardinality constraint
€
xi = 2i=1
4
∑
Cardinality Constraints (binary)
bi
1 2 3 bi =(sk-1,…,s0)
s2
s1
s0
Sum(n)
Cardinality Constraints (binary)
Sum(n)
Sum(n/2)
1
1 2 3 bi =(sk-1,…,s0)
Sum(n/2)
2
s2
s1
s0B
inary adder 3B
inary adder 3bi
Binary Adder
FAFA
a2 b2 a1 b1
C2 C1 Cin
y3 y2 y1
{y2,-a2,-b2,-c1}, {y2,-a2,b2,c1},{y2,a2,-b2,c1}, {y2,a2,b2,-c1}, {-y2,a2,b2,c1}, {-y2,a2,-b2,-c1}, {-y2,-a2,b2,-c1}, {-y2,-a2,-b2,c1},{y3,-a2,-b2}, {y3,-a2,-c1}, {y3,-b2,-c1}, {-y3,a2,b2}, {-y3,a2,c1}, {-y3,b2,c1}
Cardinality Constraints (binary)
bi
s2
s1
s0
Sum(n)
bi ≤4
s2 s1 ∧s0
(s2,s1,s0) ≤4
{-s2,-s1}, {-s2,-s0}How do you encode bi ≤5
Sorting Networks
A Comparator is a device with two inputs, a and b, and two outputs, c and d
comparatorab
c=max(a,b)
d=min(a,b)
A Sorting network is a composition of wires and comparators that sorts its inputs
b1a1
a2 b2
b3a3
a4 b4
CNF of comparator{-a,c}, {-b,c}, {-a,-b,d}, {a,-d}, {b,-d}, {a,b,-c}
Sorting Networks
There are “handcrafted” sorting networks
And there are sorting networks based on your favorite sorting algorithm
x1x2x3x4
x8
y1y2y3
x5x6x7
y5y6y7y8
x9 y9
Custom sorting network, n=9
x1x2x3x4
y1y2y3
x5x6
y5y6
y4
y4
“Bubble sort” network
163245
613245
631245
632145
632415
654321
Odd-Even (OE) Sorting Networks
x1
x2
x3
x4
x5
x6
x7
x8
y1
y2
y3
y4
y5
y6
y7
y8
Sorter(n)
Sorter(n/2)
Sorter(n/2) OE
-Mer
ger(
n)
Sorter(n/4)
OE
-Mer
ger(
n/2)}
),...,1(
),...,12
(
)2
,...,1(
{ )(
nmerge
nn
sort
nsort
nsort
OE
-Mer
ger(
n/2)
Sorter(n/4)
Sorter(n/4)
Sorter(n/4)
Odd-Even Merger
OE
-Mer
ger(
4)
x1
x2
x3
x4
x5
x6
x7
x8
y1
y2
y3
y5
y6
y7
y8
y4
Given two sorted input sequences
sorted
sorted
Merge the odd input sequences
Merge the even input sequences
OEMerger(8)
OE
-Mer
ger(
4)
Odd-Even Merger
x1
x2
x3
x4
x5
x6
x7
x8
y1
y2
y3
y5
y6
y7
y8
y4
Given two sorted input sequences
sorted
sorted
Merge the odd input sequences
Merge the even input sequences
Combine the outputs into a sorted output
OE
-Mer
ger(
4)
OE
-Mer
ger(
4)
Sorting Networks
• Draw the best sorting network you can for 5 inputs! Minimize levels and/or comparators
Cardinality Constraints (unary)
• Sorting Network encodes unary sum
• Cardinality by setting output bits
0110
1100
1010
1010
1010
x1
x2
x3
x4
x5
x6
x7
x8
y1
y2
y3
y4 Sorter(8)0 0 0 0
€
y ii=1
8
∑ ≤ 4
€
x ii=1
8
∑ ≤ 4
Cardinality Constraints
• BDD– quadratic size construction– Unit propagation on CNF enforces domain consistency
• Binary– Linear size construction– Unit Propagation on CNF does not enforce domain
consistency
• Unary– n log2 n size construction– unit propagation enforces domain consistency
Psuedo-Boolean Constraints
• Similar to Cardinality Constraints• BDD
– exponential in size, strong propagation
• Binary– O(n log n) in size, weak propagation
• Unary– O(n log2 n) in size, medium propagation– interesting problem to find base for representation
Integer Constraints
• BDD– can handle arbitrary formula but exponential
• Unary– addition, subtraction, multiplication by constant– multiplication is impractical
• Binary– addition, subtraction– multiplication is quadratic
Boolean Solving in MiniZinc
• mzn –b satinvokes a SAT solver
• all constraints and variables must be Boolean• Without recursion it’s a bit hard to model
interesting problems with just MiniZinc
Example: Latin Squaresint: n; % sizearray[1..n,1..n,1..n] of var bool: b;
predicate atmostone(array[int] of var bool:x) = forall(i,j in index_set(x) where i < j)( (not x[i]) \/ (not x[j]));
predicate exactlyone(array[int] of var bool:x) = atmostone(x) /\ exists(x);
constraint forall(i,j in 1..n)( exactlyone([ b[i,j,k] | k in 1..n]) /\ exactlyone([ b[i,k,j] | k in 1..n]) /\ exactlyone([ b[k,i,j] | k in 1..n]));
solve satisfy;
output [ if fix(b[i,j,k]) then show(k) ++ if j == n then "\n" else " " endif else "" endif | i,j,k in 1..n];
5 7 4 6 8 2 1 3 9 12 10 11 138 2 1 7 6 4 11 10 3 9 5 13 126 8 7 1 2 3 10 9 4 5 13 12 117 6 5 3 4 1 9 11 2 13 12 10 83 5 8 4 7 13 2 1 12 6 11 9 101 3 2 8 5 12 13 4 11 10 6 7 92 1 6 5 3 10 4 12 13 11 9 8 74 9 3 2 1 11 12 13 10 7 8 5 612 4 10 11 13 9 3 2 1 8 7 6 510 11 9 13 12 5 6 7 8 1 2 3 49 10 13 12 11 6 5 8 7 2 1 4 311 13 12 9 10 7 8 5 6 3 4 1 213 12 11 10 9 8 7 6 5 4 3 2 1
size 13
Summary
• Modern SAT solvers are powerful solvers– nogood learning– activity based search– restart
• State of the art solutions to Boolean problems• Many CP problems which focus on alldifferent
and ≠ can be mapped to Boolean problems– quasi-group completion (latin squares)– graph coloring
Exercise 1: atmostone
• There is a linear encoding of atmostone(x)– introduce a new array of sum variables s
encoding s[i] = exists(j in 1..i)(x[i])– s[1] = x[1] – if s[i] is true then x[i+1] must be false– s[i+1] can be defined in terms of s[i] and x[i+1]– its actually the sorting network circuit simplified
• Write a new predicate defn of atmostone(x)• Compare the execution speed on latin squares