CS151Complexity Theory

Lecture 5

April 13, 2015

April 13, 2015

Introduction

Power from an unexpected source?

• we know P ≠ EXP, which implies no poly-time algorithm for Succinct CVAL

• poly-size Boolean circuits for Succinct CVAL ??

Does NP have linear-size, log-depth Boolean circuits ??

2

April 13, 2015

Outline

• Boolean circuits and formulas

• uniformity and advice

• the NC hierarchy and parallel computation

• the quest for circuit lower bounds

• a lower bound for formulas

3

April 13, 2015

Boolean circuits

• C computes function f:{0,1}n {0,1} in natural way – identify C with function f it computes

• circuit C– directed acyclic graph

– nodes: AND (); OR (); NOT (); variables xi

x1 x2

x3 … xn

4

April 13, 2015

Boolean circuits

• size = # gates

• depth = longest path from input to output

• formula (or expression): graph is a tree

• every function f:{0,1}n {0,1} computable by a circuit of size at most O(n2n)

– AND of n literals for each x such that f(x) = 1– OR of up to 2n such terms

5

April 13, 2015

Circuit families

• circuit family : a circuit for each input length C1, C2, C3, … = “{Cn}”

• “{Cn} computes f” iff for all x

C|x|(x) = f(x)

• Definition: circuit family {Cn} is logspace uniform iff TM M outputs Cn on input 1n and runs in O(log n) space

6

April 13, 2015

TMs that take advice

• family {Cn} without uniformity constraint is called “non-uniform”

• regard “non-uniformity” as a limited resource just like time, space, as follows:– add read-only “advice” tape to TM M– M “decides L with advice A(n)” iff

M(x, A(|x|)) accepts x L– note: A(n) depends only on |x|

7

April 13, 2015

TMs that take advice

• Definition: TIME(t(n))/f(n) = the set of those languages L for which:

– there exists A(n) s.t. |A(n)| ≤ f(n)

– TM M decides L with advice A(n) in time t(n)

• most important such class:

P/poly = k TIME(nk)/nk

8

April 13, 2015

Uniformity

Theorem: P = languages decidable by logspace uniform, polynomial-size circuit families {Cn}.

• Proof:– already saw ()

– () on input x, generate C|x|, evaluate it and accept iff output = 1

9

April 13, 2015

TMs that take advice

Theorem: L P/poly iff L decided by family of (non-uniform) polynomial size circuits.

• Proof:– () Cn from CVAL construction; hardwire

advice A(n)

– () define A(n) = description of Cn; on input x, TM simulates C|x|(x)

10

April 13, 2015

Approach to P/NP

• Believe NP P– equivalent: “NP does not have uniform,

polynomial-size circuits”

• Even believe NP P/poly– equivalent: “NP (or, e.g. SAT) does not have

polynomial-size circuits”– implies P ≠ NP– many believe: best hope for P ≠ NP

11

April 13, 2015

Parallelism

• uniform circuits allow refinement of polynomial time:

circuit C

depth parallel time

size parallel work

12

April 13, 2015

Parallelism

• the NC (“Nick’s Class”) Hierarchy (of logspace uniform circuits):

NCk = O(logk n) depth, poly(n) size

NC = k NCk

• captures “efficiently parallelizable problems”

• not realistic? overly generous

• OK for proving non-parallelizable13

April 13, 2015

Matrix Multiplication

• what is the parallel complexity of this problem?– work = poly(n) – time = logk(n)? (which k?)

n x n matrix A

n x n matrix B =

n x n matrix AB

14

April 13, 2015

Matrix Multiplication

• two details– arithmetic matrix multiplication…

A = (ai, k) B = (bk, j) (AB)i,j = Σk (ai,k x bk, j)

… vs. Boolean matrix multiplication:

A = (ai, k) B = (bk, j) (AB)i,j = k (ai,k bk, j)

– single output bit: to make matrix multiplication a language: on input A, B, (i, j) output (AB)i,j

15

April 13, 2015

Matrix Multiplication

• Boolean Matrix Multiplication is in NC1

– level 1: compute n ANDS: ai,k bk, j

– next log n levels: tree of ORS

– n2 subtrees for all pairs (i, j)– select correct one and output

16

April 13, 2015

Boolean formulas and NC1

• Previous circuit is actually a formula. This is no accident:

Theorem: L NC1 iff decidable by polynomial-size uniform family of Boolean formulas.

Note: we measure formula size by leaf-size.17

April 13, 2015

Boolean formulas and NC1

• Proof: – () convert NC1 circuit into formula

• recursively:

• note: logspace transformation (stack depth log n, stack record 1 bit – “left” or “right”)

18

April 13, 2015

Boolean formulas and NC1

– () convert formula of size n into formula of depth O(log n) • note: size ≤ 2depth, so new formula has

poly(n) size

D

C C1

1

C0

0

D

D

key transformation

19

April 13, 2015

Boolean formulas and NC1

– D any minimal subtree with size at least n/3 • implies size(D) ≤ 2n/3

– define T(n) = maximum depth required for any size n formula

– C1, C0, D all size ≤ 2n/3

T(n) ≤ T(2n/3) + 3

implies T(n) ≤ O(log n)

20

April 13, 2015

Relation to other classes

• Clearly NCµ P– recall P uniform poly-size circuits

• NC1 µ L

– on input x, compose logspace algorithms for:• generating C|x|

• converting to formula• FVAL

21

April 13, 2015

Relation to other classes

• NL µ NC2: S-T-CONN NC2

– given G = (V, E), vertices s, t– A = adjacency matrix (with self-loops)– (A2)i, j = 1 iff path of length ≤ 2 from node i to

node j– (An)i, j = 1 iff path of length ≤ n from node i to

node j– compute with depth log n tree of Boolean

matrix multiplications, output entry s, t – log2 n depth total

22

April 13, 2015

NC vs. P

• can every efficient algorithm be efficiently parallelized?

NC = P• P-complete problems least-likely to be

parallelizable– if P-complete problem is in NC, then P = NC – Why?

we use logspace reductions to show problem P-complete; L in NC

?

23

April 13, 2015

NC vs. P

• can every uniform, poly-size Boolean circuit family be converted into a uniform, poly-size Boolean formula family?

NC1 = P?

24

April 13, 2015

NC Hierarchy Collapse

NC1 µ NC2 µ NC3 µ NC4 µ … µ NC

Exercise

if NCi = NCi+1, then NC = NCi

(prove for non-uniform versions of classes)

25

April 13, 2015

Lower bounds

• Recall: “NP does not have polynomial-size circuits” (NP P/poly) implies P ≠ NP

• major goal: prove lower bounds on (non-uniform) circuit size for problems in NP– believe exponential – super-polynomial enough for P ≠ NP – best bound known: 4.5n– don’t even have super-polynomial bounds for

problems in NEXP26

April 13, 2015

Lower bounds

• lots of work on lower bounds for restricted classes of circuits

– we’ll see two such lower bounds: • formulas • monotone circuits

27

April 13, 2015

Shannon’s counting argument

• frustrating fact: almost all functions require huge circuits

Theorem (Shannon): With probability at least 1 – o(1), a random function

f:{0,1}n {0,1}

requires a circuit of size Ω(2n/n).

28

April 13, 2015

Shannon’s counting argument

• Proof (counting):– B(n) = 22n

= # functions f:{0,1}n {0,1} – # circuits with n inputs + size s, is at most

C(n, s) ≤ ((n+3)s2)s

s gates

n+3 gate types 2 inputs per gate

29

April 13, 2015

Shannon’s counting argument

– C(n, c2n/n) < ((2n)c222n/n2)(c2n/n)

< o(1)22c2n

< o(1)22n (if c ≤ ½)

– probability a random function has a circuit of size s = (½)2n/n is at most

C(n, s)/B(n) < o(1)

C(n, s) ≤ ((n+3)s2)s

30

April 13, 2015

Shannon’s counting argument

• frustrating fact: almost all functions require huge formulas

Theorem (Shannon): With probability at least 1 – o(1), a random function

f:{0,1}n {0,1}

requires a formula of size Ω(2n/log n).

31

April 13, 2015

Shannon’s counting argument

• Proof (counting):– B(n) = 22n

= # functions f:{0,1}n {0,1} – # formulas with n inputs + size s, is at most

F(n, s) ≤ 4s2s(2n)s

4s binary trees with s internal nodes 2 gate choices

per internal node

2n choices per leaf

32

April 13, 2015

Shannon’s counting argument

– F(n, c2n/log n) < (16n)(c2n/log n)

< 16(c2n/log n)2(c2n) = (1 + o(1))2(c2n)

< o(1)22n (if c ≤ ½)

– probability a random function has a formula of size s = (½)2n/log n is at most F(n, s)/B(n) < o(1)

F(n, s) ≤ 4s2s(2n)s

33

April 13, 2015

Andreev function

• best formula lower bound for language in NP:

Theorem (Andreev, Hastad ‘93): the Andreev function requires (,,)-formulas of size at least

Ω(n3-o(1)).

34

April 13, 2015

Andreev function

selector

yi

n-bit string yXOR XOR

. . .

log n copies; n/log n bits each

the Andreev function A(x,y) A:{0,1}2n

{0,1}35

April 13, 2015

Random restrictions

• key idea: given function f:{0,1}n {0,1}

restrict by ρ to get fρ

– ρ sets some variables to 0/1, others remain free

• R(n, єn) = set of restrictions that leave єn variables free

• Definition: L(f) = smallest (,,) formula computing f (measured as leaf-size)

36

April 13, 2015

Random restrictions

• observation:

EρR(n, єn)[L(fρ)] ≤ єL(f)– each leaf survives with probability є

• may shrink more…– propogate constants

Lemma (Hastad 93): for all f

EρR(n, єn)[L(fρ)] ≤ O(є2-o(1)L(f))

37

April 13, 2015

Hastad’s shrinkage result

• Proof of theorem:– Recall: there exists a function

h:{0,1}log n {0,1} for which L(h) > n/2loglog n.

– hardwire truth table of that function into y to get A*(x)

– apply random restriction from R(n, m = 2(log n)(ln log n))

to A*(x).

38

April 13, 2015

The lower bound

• Proof of theorem (continued):– probability given XOR is killed by restriction is

probability that we “miss it” m times:

(1 – (n/log n)/n)m ≤ (1 – 1/log n)m

≤ (1/e)2ln log n ≤ 1/log2n– probability even one of XORs is killed by

restriction is at most:

log n(1/log2n) = 1/log n < ½.

39

April 13, 2015

The lower bound

– (1): probability even one of XORs is killed by restriction is at most:

log n(1/log2n) = 1/log n < ½.– (2): by Markov:

Pr[ L(A*ρ) > 2 EρR(n, m)[L(A*

ρ)] ] < ½.

– Conclude: for some restriction ρ • all XORs survive, and

• L(A*ρ) ≤ 2 EρR(n, m) [L(A*

ρ)]

40

April 13, 2015

The lower bound

• Proof of theorem (continued):– if all XORs survive, can restrict formula further

to compute hard function h • may need to add ’s

L(h) = n/2loglogn ≤ L(A*ρ)

≤ 2EρR(n, m)[L(A*ρ)] ≤ O((m/n)2-o(1)L(A*))

≤ O( ((log n)(ln log n)/n)2-o(1) L(A*) )

– implies Ω(n3-o(1)) ≤ L(A*) ≤ L(A).

41

April 13, 2015 42

Clique

CLIQUE = { (G, k) | G is a graph with a clique of size ≥ k }

(clique = set of vertices every pair of which are connected by an edge)

• CLIQUE is NP-complete.

April 13, 2015 43

Circuit lower bounds

• We think that NP requires exponential-size circuits.

• Where should we look for a problem to attempt to prove this?

• Intuition: “hardest problems” – i.e., NP-complete problems

April 13, 2015 44

Circuit lower bounds

• Formally: – if any problem in NP requires super-

polynomial size circuits– then every NP-complete problem requires

super-polynomial size circuits

– Proof idea: poly-time reductions can be performed by poly-size circuits using a variant of CVAL construction

April 13, 2015 45

Monotone problems

• Definition: monotone language = language

L {0,1}*

such that x L implies x’ L for all x ¹ x’.

– flipping a bit of the input from 0 to 1 can only change the output from “no” to “yes” (or not at all)

April 13, 2015 46

Monotone problems

• some NP-complete languages are monotone– e.g. CLIQUE (given as adjacency matrix):

– others: HAMILTON CYCLE, SET COVER…– but not SAT, KNAPSACK…

April 13, 2015 47

Monotone circuits

A restricted class of circuits:

• Definition: monotone circuit = circuit whose gates are ANDs (), ORs (), but no NOTs

• can compute exactly the monotone fns.– monotone functions closed under AND, OR

April 13, 2015 48

Monotone circuits

• A question:

Do all

poly-time computable monotone functions

have

poly-size monotone circuits?

– recall: true in non-monotone case

April 13, 2015 49

Monotone circuits

A monotone circuit for CLIQUEn,k

• Input: graph G = (V,E) as adj. matrix, |V|=n– variable xi,j for each possible edge (i,j)

• ISCLIQUE(S) = monotone circuit that = 1

iff S V is a clique: i,j S xi,j

• CLIQUEn,k computed by monotone circuit:

S V, |S| = k ISCLIQUE(S)

April 13, 2015 50

Monotone circuits

• Size of this monotone circuit for CLIQUEn,k:

• when k = n1/4, size is approximately:

n k

k 2

1/ 41/ 4

n 1/ 4n

4

2

1/

nn

2n

n

April 13, 2015 51

Monotone circuits

• Theorem (Razborov 85): monotone circuits for CLIQUEn,k with k = n1/4 must have size at least

2Ω(n1/8).

• Proof: – next lecture