computational complexity

Computational Complexity

Lecture 3in which we come across

Diagonalization and Time-hierarchies(But first some more of NP-completeness)

1

NP-Complete Languages

2


A language L1 is NP-complete if L1 is in NP and any NP language L can be reduced to L1 (Karp reduction: polynomial time many-one reduction)

2



Trivial example: L1 = TMSAT

2




First interesting example: L1 = CKT-SAT

2





Convert x into a circuit C s.t. C is satisfiable iff x in L

2






More examples, bootstrapping from CKT-SAT

2






More examples, bootstrapping from CKT-SAT

If L ≤p L1 and L1 ≤p L2, then L ≤p L2

2

CKT-SAT ≤p SAT

3

CKT-SAT ≤p SATSAT: Are all given “clauses” simultaneously satisfiable? (Conjunctive Normal Form)

3


Converting a circuit to a collection of clauses:

3



For each wire (connected component), add a variable

3




Add output variable as a clause. And for each gate, add a clause involving variables for wires connected to the gate:

3





e.g. : (z⇒x), (z⇒y), (¬z ⇒ ¬x ∨ ¬y). i.e., (¬z ∨ x), (¬z ∨ y), (z ∨ ¬x ∨ ¬y).

AND

xy z

3





e.g. : (z⇒x), (z⇒y), (¬z ⇒ ¬x ∨ ¬y). i.e., (¬z ∨ x), (¬z ∨ y), (z ∨ ¬x ∨ ¬y).

and : (z⇒x ∨ y), (¬z⇒¬x), (¬z⇒¬y).

AND

xy z

OR

xy z

3

SAT ≤p 3SAT

4

SAT ≤p 3SATPrevious reduction was to 3SAT, so 3SAT is NP-complete. And SAT is in NP. So SAT ≤p 3SAT.

4


More directly:

4


More directly:

(a ∨ b ∨ c ∨ d ∨ e) → (a ∨ b ∨ x), (¬x ∨ c ∨ d ∨ e) → (a ∨ b ∨ x), (¬x ∨ c ∨ y), (¬y ∨ d ∨ e)

4


More directly:


Reduction needs 3SAT

4


More directly:



2SAT is in fact in P! [Exercise]

4


More directly:



2SAT is in fact in P! [Exercise]

Reduction not parsimonious (can you make it? [Exercise])

4

3SAT ≤p CLIQUE

5

3SAT ≤p CLIQUECLIQUE: Does graph G have a clique of size m?

5


(x ∨ ¬y ∨ ¬z)

(w ∨ x ∨ ¬z)

(w ∨ y)

5


Clauses → Graph

(x ∨ ¬y ∨ ¬z)

(w ∨ x ∨ ¬z)

(w ∨ y)

5


Clauses → Graph

(x ∨ ¬y ∨ ¬z)

(w ∨ x ∨ ¬z)

0*1*1*1*

1*0*

(w ∨ y)

5


Clauses → Graph

(x ∨ ¬y ∨ ¬z) *000

*001*010

*100

*101*110

*111

(w ∨ x ∨ ¬z)

0*1*1*1*

1*0*

(w ∨ y)

5


Clauses → Graph

(x ∨ ¬y ∨ ¬z) *000

*001*010

*100

*101*110

*111

(w ∨ x ∨ ¬z)00*0

01*101*0

10*0

10*111*0

11*1

0*1*1*1*

1*0*

(w ∨ y)

5


Clauses → Graph

vertices: each clause’s satisfying assignments (for its variables)

(x ∨ ¬y ∨ ¬z) *000

*001*010

*100

*101*110

*111

(w ∨ x ∨ ¬z)00*0

01*101*0

10*0

10*111*0

11*1

0*1*1*1*

1*0*

(w ∨ y)

5


Clauses → Graph


edges between consistent assignments

(x ∨ ¬y ∨ ¬z) *000

*001*010

*100

*101*110

*111

(w ∨ x ∨ ¬z)00*0

01*101*0

10*0

10*111*0

11*1

0*1*1*1*

1*0*

(w ∨ y)

5


Clauses → Graph



m-clique iff all m clauses satisfiable

(x ∨ ¬y ∨ ¬z) *000

*001*010

*100

*101*110

*111

(w ∨ x ∨ ¬z)00*0

01*101*0

10*0

10*111*0

11*1

0*1*1*1*

1*0*

(w ∨ y)

5


Clauses → Graph



m-clique iff all m clauses satisfiable

(x ∨ ¬y ∨ ¬z) *000

*001*010

*100

*101*110

*111

(w ∨ x ∨ ¬z)00*0

01*101*0

10*0

10*111*0

11*1

0*1*1*1*

1*0*

(w ∨ y)

1*1**11011*0----1110sat assignment

3-Clique

5

INDEP-SET and VERTEX-COVER

6


CLIQUE ≤p INDEP-SET

6



G has an m-clique iff Gc has an m-independent-set

6




INDEP-SET ≤p VERTEX-COVER

6




INDEP-SET ≤p VERTEX-COVER

G has an m-indep-set iff G has an (n-m)-vertex-cover

6

NP, P, co-NP and NPC

P

NP coNP

NPC coNPC

7

NP, P, co-NP and NPCWe say class X is “closed under polynomial reductions” if (L1 ≤p L2 and L2 in class X) implies L1 in X

P

NP coNP

NPC coNPC

7


e.g. P, NP are closed under polynomial reductions

P

NP coNP

NPC coNPC

7



So is co-NP (If X is closed, so is co-X. Why?)

P

NP coNP

NPC coNPC

7




If any NPC language is in P, then NP = P

P

NP coNP

NPC coNPC

7





If any NPC language is in co-NP, then NP = co-NPP

NP coNP

NPC coNPC

7





If any NPC language is in co-NP, then NP = co-NP

Note: X ⊆ co-X ⇒ X = co-X (Why?)P

NP coNP

NPC coNPC

7






Note: X ⊆ co-X ⇒ X = co-X (Why?)

L is NP-complete iff Lc is co-NP-complete (Why?)

P

NP coNP

NPC coNPC

7






Note: X ⊆ co-X ⇒ X = co-X (Why?)

L is NP-complete iff Lc is co-NP-complete (Why?)

co-NP complete = co-(NP-complete)

P

NP coNP

NPC coNPC

7

Separating Classes

8

Separating Classes

How to prove a set X strictly bigger than Y

8

Separating Classes


Show an element not in Y, but in X? For us, not in Y may often be difficult to prove for (familiar) elements

8

Separating Classes



Count? What if both infinite?!

8

Separating Classes



Count? What if both infinite?!

Comparing infinite sets: diagonalization!

8

Cantor’s Diagonal Slash

9

Cantor’s Diagonal SlashAre real numbers (say in the range [0,1) ) countable?

9


Suppose they were: consider enumerating them along with their binary representations in a table

9



Ri

R1 =

R2 =

R3 =

R4 =

R5 =

R6 =

R7 =

1 0 0 1 0 0 0 0 1

0 0 1 0 1 0 0 1 1

1 1 1 1 1 1 1 0 0

1 1 0 1 0 1 0 1 1

1 1 0 0 0 0 1 0 0

0 0 0 0 0 0 1 1 0

0 1 0 1 0 1 0 1 19



Consider the real number corresponding to the “flipped diagonal”

Ri

R1 =

R2 =

R3 =

R4 =

R5 =

R6 =

R7 =

1 0 0 1 0 0 0 0 1

0 0 1 0 1 0 0 1 1

1 1 1 1 1 1 1 0 0

1 1 0 1 0 1 0 1 1

1 1 0 0 0 0 1 0 0

0 0 0 0 0 0 1 1 0

0 1 0 1 0 1 0 1 19




Doesn’t appear in this table!

Ri

R1 =

R2 =

R3 =

R4 =

R5 =

R6 =

R7 =

1 0 0 1 0 0 0 0 1

0 0 1 0 1 0 0 1 1

1 1 1 1 1 1 1 0 0

1 1 0 1 0 1 0 1 1

1 1 0 0 0 0 1 0 0

0 0 0 0 0 0 1 1 0

0 1 0 1 0 1 0 1 19




Doesn’t appear in this table!

Ri

R1 =

R2 =

R3 =

R4 =

R5 =

R6 =

R7 =

1 0 0 1 0 0 0 0 1

0 0 1 0 1 0 0 1 1

1 1 1 1 1 1 1 0 0

1 1 0 1 0 1 0 1 1

1 1 0 0 0 0 1 0 0

0 0 0 0 0 0 1 1 0

0 1 0 1 0 1 0 1 1

This table

can’t

have all re

als

9

Undecidable Languages

LM1 =

LM2 =

LM3 =

LM4 =

LM5 =

LM6 =

LM7 =

1 0 0 1 0 0 0 0 1

0 0 1 0 1 0 0 1 1

1 1 1 1 1 1 1 0 0

1 1 0 1 0 1 0 1 1

1 1 0 0 0 0 1 0 0

0 0 0 0 0 0 1 1 0

0 1 0 1 0 1 0 1 110

Undecidable LanguagesLanguages, like real numbers, can be represented as infinite bit-vectors LM1 =

LM2 =

LM3 =

LM4 =

LM5 =

LM6 =

LM7 =

1 0 0 1 0 0 0 0 1

0 0 1 0 1 0 0 1 1

1 1 1 1 1 1 1 0 0

1 1 0 1 0 1 0 1 1

1 1 0 0 0 0 1 0 0

0 0 0 0 0 0 1 1 0

0 1 0 1 0 1 0 1 110

Undecidable LanguagesLanguages, like real numbers, can be represented as infinite bit-vectors

TMs can be enumerated!

LM1 =

LM2 =

LM3 =

LM4 =

LM5 =

LM6 =

LM7 =

1 0 0 1 0 0 0 0 1

0 0 1 0 1 0 0 1 1

1 1 1 1 1 1 1 0 0

1 1 0 1 0 1 0 1 1

1 1 0 0 0 0 1 0 0

0 0 0 0 0 0 1 1 0

0 1 0 1 0 1 0 1 110



Table of languages recognized by the TMs

LM1 =

LM2 =

LM3 =

LM4 =

LM5 =

LM6 =

LM7 =

1 0 0 1 0 0 0 0 1

0 0 1 0 1 0 0 1 1

1 1 1 1 1 1 1 0 0

1 1 0 1 0 1 0 1 1

1 1 0 0 0 0 1 0 0

0 0 0 0 0 0 1 1 0

0 1 0 1 0 1 0 1 110




L = ”diagonal language”

LM1 =

LM2 =

LM3 =

LM4 =

LM5 =

LM6 =

LM7 =

1 0 0 1 0 0 0 0 1

0 0 1 0 1 0 0 1 1

1 1 1 1 1 1 1 0 0

1 1 0 1 0 1 0 1 1

1 1 0 0 0 0 1 0 0

0 0 0 0 0 0 1 1 0

0 1 0 1 0 1 0 1 110





Lc does not appear as a row in this table. Hence not recognizable!

LM1 =

LM2 =

LM3 =

LM4 =

LM5 =

LM6 =

LM7 =

1 0 0 1 0 0 0 0 1

0 0 1 0 1 0 0 1 1

1 1 1 1 1 1 1 0 0

1 1 0 1 0 1 0 1 1

1 1 0 0 0 0 1 0 0

0 0 0 0 0 0 1 1 0

0 1 0 1 0 1 0 1 110





Lc does not appear as a row in this table. Hence not recognizable!

LM1 =

LM2 =

LM3 =

LM4 =

LM5 =

LM6 =

LM7 =

1 0 0 1 0 0 0 0 1

0 0 1 0 1 0 0 1 1

1 1 1 1 1 1 1 0 0

1 1 0 1 0 1 0 1 1

1 1 0 0 0 0 1 0 0

0 0 0 0 0 0 1 1 0

0 1 0 1 0 1 0 1 1

This table ca

n’t have

all language

s

10

Diagonalization to Separate Classes

Diagonalization can separate the class of decidable languages (from the class of all languages)

Plan: Use similar techniques to separate complexity classes

11

DTIME Hierarchy

12

DTIME HierarchyFix a TM model (one-tape, binary alphabet)

12


DTIME(T) = class of languages that can be decided in O(T(n)) time, by such a TM

12



Theorem: DTIME(nc) ⊊ DTIME(nc+1) for all c ≥ 1

12




More generally DTIME(T) ⊊ DTIME(T’) if T, T’ “nice” (and ≥ n)

and T(n)log(T(n)) = o(T’(n))

12






Consequences, for e.g., P ⊊ EXP

12






Consequences, for e.g., P ⊊ EXP

P ⊆ DTIME(2n) ⊊ DTIME(22n) ⊆ EXP

12

DTIME Hierarchy: Proof

13

Mi be an enumeration of TMs, each TM appearing infinitely often


13


Consider table(i,j) = UTM|T’ (Mi,j), where T log T = o(T’)


Mi

1 0 0 1 0 0 0 0 1

0 0 1 0 1 0 0 1 1

1 1 1 1 1 1 1 0 0

1 1 0 1 0 1 0 1 1

1 1 0 0 0 0 1 0 0

0 0 0 0 0 0 1 1 0

0 1 0 1 0 1 0 1 1

j

13




Mi

1 0 0 1 0 0 0 0 1

0 0 1 0 1 0 0 1 1

1 1 1 1 1 1 1 0 0

1 1 0 1 0 1 0 1 1

1 1 0 0 0 0 1 0 0

0 0 0 0 0 0 1 1 0

0 1 0 1 0 1 0 1 1

j

T’ large and nice enough to allow

simulation

13




Mi

1 0 0 1 0 0 0 0 1

0 0 1 0 1 0 0 1 1

1 1 1 1 1 1 1 0 0

1 1 0 1 0 1 0 1 1

1 1 0 0 0 0 1 0 0

0 0 0 0 0 0 1 1 0

0 1 0 1 0 1 0 1 1

j

Think DTIME(T) ⊆ rows


simulation

13



Let L’ = inverted diagonal.


Mi

1 0 0 1 0 0 0 0 1

0 0 1 0 1 0 0 1 1

1 1 1 1 1 1 1 0 0

1 1 0 1 0 1 0 1 1

1 1 0 0 0 0 1 0 0

0 0 0 0 0 0 1 1 0

0 1 0 1 0 1 0 1 1

j



simulation

13

L’ in DTIME(T’)





Mi

1 0 0 1 0 0 0 0 1

0 0 1 0 1 0 0 1 1

1 1 1 1 1 1 1 0 0

1 1 0 1 0 1 0 1 1

1 1 0 0 0 0 1 0 0

0 0 0 0 0 0 1 1 0

0 1 0 1 0 1 0 1 1

j



simulation

13

L’ in DTIME(T’)

On input i, run UTM|T’ (Mi,i), modified to invert output





Mi

1 0 0 1 0 0 0 0 1

0 0 1 0 1 0 0 1 1

1 1 1 1 1 1 1 0 0

1 1 0 1 0 1 0 1 1

1 1 0 0 0 0 1 0 0

0 0 0 0 0 0 1 1 0

0 1 0 1 0 1 0 1 1

j



simulation

13

L’ in DTIME(T’)


L’ not in DTIME(T)





Mi

1 0 0 1 0 0 0 0 1

0 0 1 0 1 0 0 1 1

1 1 1 1 1 1 1 0 0

1 1 0 1 0 1 0 1 1

1 1 0 0 0 0 1 0 0

0 0 0 0 0 0 1 1 0

0 1 0 1 0 1 0 1 1

j



simulation

13

L’ in DTIME(T’)


L’ not in DTIME(T)

If M accepts L’ in time T, then for sufficiently large i s.t. Mi=M, UTM can finish simulating Mi(i). Then table(i,i)=L’(i)!





Mi

1 0 0 1 0 0 0 0 1

0 0 1 0 1 0 0 1 1

1 1 1 1 1 1 1 0 0

1 1 0 1 0 1 0 1 1

1 1 0 0 0 0 1 0 0

0 0 0 0 0 0 1 1 0

0 1 0 1 0 1 0 1 1

j



simulation

13

NTIME Hierarchy

14

NTIME Hierarchy

Finer hierarchy

14

NTIME Hierarchy

Finer hierarchy

NTIME(T) ⊊ NTIME(T’) if T(n)=o(T’(n)), and T, T’ nice

14

NTIME Hierarchy

Finer hierarchy


Because a more sophisticated Universal NTM has less overhead

14

NTIME Hierarchy

Finer hierarchy



Diagonalization is more complicated

14

NTIME Hierarchy

Finer hierarchy




Issue: NTIME(T’) enough to simulate NTIME(T), but not to simulate co-NTIME(T)!

14

NTIME Hierarchy

Finer hierarchy




Issue: NTIME(T’) enough to simulate NTIME(T), but not to simulate co-NTIME(T)!

In fact,

T(n+1) = o(

T’(n))

14

NTIME Hierarchy

15

NTIME Hierarchy“Delayed flip” on a “rapidly thickening diagonal”

15


Think NTIME(T) ⊆ rows15


f(i+1)=exp(f(i))



f(i+1)=exp(f(i))

f(1) f(2) f(3) f(4) f(5)



f(i+1)=exp(f(i))

Let L be the “diagonal” language

f(1) f(2) f(3) f(4) f(5)



f(i+1)=exp(f(i))


f(1) f(2) f(3) f(4) f(5)

L



f(i+1)=exp(f(i))


f(1) f(2) f(3) f(4) f(5)

L

L’



f(i+1)=exp(f(i))


L’(j)=L(j+1)f(1) f(2) f(3) f(4) f(5)

L

L’



f(i+1)=exp(f(i))


L’(j)=L(j+1)

except if j=f(i), then L’(j) = 1 - L(f(i-1)+1)

f(1) f(2) f(3) f(4) f(5)

L

L’



f(i+1)=exp(f(i))


L’(j)=L(j+1)


L’ not in NTIME(T), but is in NTIME(T’)

f(1) f(2) f(3) f(4) f(5)

L

L’



f(i+1)=exp(f(i))


L’(j)=L(j+1)



f(1) f(2) f(3) f(4) f(5)

L

L’

Think NTIME(T) ⊆ rows

Flip, Diagonal

15

Delay, R

apid

thicken

ing


f(i+1)=exp(f(i))


L’(j)=L(j+1)



f(1) f(2) f(3) f(4) f(5)

L

L’

Think NTIME(T) ⊆ rows

Flip, Diagonal

15

Time Hierarchy

16

Time Hierarchy

Within DTIME and NTIME fine gradation

16

Time Hierarchy


In particular P ⊊ EXP, NP ⊊ NEXP

16

Time Hierarchy



Tells nothing across DTIME and NTIME

16

Time Hierarchy




P and NP?

16

Time Hierarchy




P and NP?

Just diagonalization won’t help (next lecture)

16

Today

DTIME Hierarchy

DTIME(T) ⊊ DTIME(T’) if T log T = o(T’)

NTIME Hierarchy

NTIME(T) ⊊ NTIME(T’) if T = o(T’)

Using diagonalization

17

Next Lecture

Another application of diagonalization

Ladner’s Theorem: If P≠NP, NP language which is neither in P nor NP-complete

Limits of Diagonalization

Starting Space Complexity

18

computational complexity

Documents