problem analysis and complexity theory ss16 · birgit vogtenhuber problem analysis and complexity...

Graz University of TechnologyInstitute for Software Technology

Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, summer term 2016 1

Welcome to ...

Problem Analysis and Complexity Theory716.054, 3 VU

Birgit VogtenhuberInstitute for Software Technology

Graz University of Technology

email: [email protected]: Inffeldgasse 16B/II, room IC02044

office hour: Wednesday 10:30-11:30

slides: http://www.ist.tugraz.at/pact16.html



Last Time• Two computational problems:

◦ the seating problem: HAMILTONIAN CYCLE

◦ the tour problem HAMILTONIAN PATH

• For both:

◦ efficient verification of a proposed solution

◦ no real statement about their computationalcomplexity

• Showed that their computational complexity is related:Tour ≤P Seating, Seating ≤P Tour

• Claimed that they are part of a complexity class calledNPC (NP-complete problems)


Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, summer term 2016 3-1

Last Time• Two classes of computational models:

1. Deterministic: DTM, multi-tape DTM, RAM

◦ all polynomially equivalent: Simulation of

- O(f(n))-time multi-tape DTM withO(f(n)2)-time DTM

- O(f(n))-time RAM with O(f(n)3)-time7-tape DTM

2. Nondeterministic: NTM

◦ simulation of NTM by DTM with anexponential blowup in time



Last Time• Church-Turing (hypo)thesis:

Conjecture: Deterministic Turing machines areas powerfull as the intuitive notion of algorithms.

• Definition of complexity classes via bounds on theamount of needed ressources of Turing machines.

• Number of different configurations of a Turing machinethat uses n cells of the tape (roughly):

|Γ|N × N × |Q|



Time ComplexityDefinition: Let t : N→ N be a complexity function.

• Deterministic Time:TIME(t(n)) := {L | L is a language decidable by an

O(t(n))-time det. TM }• Nondeterministic Time:

NTIME(t(n)) := {L | L is a language decidable by anO(t(n))-time nondet. TM }

• Det. Polynomial Time: P :=⋃

k≥1 TIME(nk)

• Nondet. Polynomial Time: NP :=⋃

k≥1 NTIME(nk)

• Det. Exponential Time: EXP :=⋃

k≥1 TIME(enk

)



Space ComplexityDefinition: Let t : N→ N be a complexity function.

• Deterministic Space:SPACE(t(n)) := {L | L is a language decidable by an

O(t(n))-space det. TM }• Nondeterministic Space:

NSPACE(t(n)) := {L | L is a language decidable by anO(t(n))-space nondet. TM }

• Det. Logarithmic Space: L := SPACE(log n)

• Nondet. Logarithmic Space: NL := NSPACE(log n)

• Det. Polynomial Space: PSPACE :=⋃

k≥1 SPACE(nk)



Space vs. TimeEXP

Theorem: P ⊆ PSPACEProof:• A TM that runs in O(t(n)) steps

uses at most O(t(n)) space.

Theorem: PSPACE ⊆ EXPProof:• A run of a determinstic TM M that halts must avoid

repeating a configuration.

• The running time is bounded from above by the numberc of configurations that M has.

• For M ∈ PSPACE, c is at most exponential.

PSPACE

P



Exercise: Name the Class

EXP

PSPACENPP

NL

L

• This picture presents a sequence of containmentbetween complexity classes.◦ Are you sure they are all true?◦ Which ones can you prove?◦ Are any containments strict?

• Consider the following problems:◦ anbncn◦ Minimum spanning tree◦ Hamiltonian cycle◦ Towers of Hanoi◦ Halting Problem

Can you name the smallest class for each of them?



Containment Relations

EXP

PSPACENPP

NL

L

L ⊆ NL ⊆ P ⊆ NP ⊆ PSPACE ⊆ EXP

• L ⊆ NL, P ⊆ NP:DTM = special case of NTM

• PSPACE ⊆ EXP, L ⊆ P:# configurations of a DTM

• P ⊆ PSPACE:O(f(n))-time ⇒ O(f(n))-space

• NP ⊆ PSPACE:simulate O(f(n))-time NTM with O(f(n))-space DTM

• NL ⊆ P: later in this course



Next Topics

EXP

PSPACENPP

NL

L

• The halting problem

• Separating complexityclasses

• Reductions

• NP-completeness



The Halting ProblemThe Problem: You are given some arbitrary DTM M

and some input x. ⇒ Does M halt on x?

Question: What is the language for the halting problem?

L = {(M,x) | M is the representation of a DTM andM halts on input x}

Theorem: The halting problem is Turing-recognizable.

Proof: Construct DTM H such that• On input (M,x), H simulates M on x.

• H accepts (M,x) if M halts on x.



The Halting ProblemTheorem: The halting problem is undecidable.

Proof: • Assume there exists a DTM H such that• On input (M,x), H simulates M on x.• H accepts (M,x) if M halts on x.• H rejects (M,x) if M does not halt on x.

• Construct DTM D such that• On input M , D runs H on input (M,M).• D accepts M if H rejects (M,M).• D loops infinitely on M if H accepts (M,M).

• Run D on input D• D accepts D if H rejects (D,D) if D loops on D• D loops on D if H accepts (D,D) if D accepts D



P vs. EXPTheorem: P 6= EXP.

Proof Idea: We construct a language L ∈ EXP,which is not accepted by any TM in polynomial time:

L = {x | x = M#1c#1e#1d, d ≥ 0,M does not halt on x within c|x|esteps}

Lemma: L ∈ EXP.

Proof: L can be decided in O(|x| · |x||x|) time:Run M for c|x|e < |x| · |x||x| steps.






Lemma: L 6∈ P.

Proof: Assume a DTM MP accepts every x ∈ Lwithin polynomial time: O(nb) time for some b.

⇒ ∃a, b, n0: MP accepts x in a|x|b steps if |x| ≥ n0

Def. O-notation






Lemma: L 6∈ P.

Proof: Assume a DTM MP accepts every x ∈ Lwithin polynomial time: O(nb) time for some b.

⇒ ∃a, b, n0: MP accepts x in a|x|b steps if |x| ≥ n0

⇒ Run MP on x = MP #1a#1b#1n0

⇒ MP accepts x if MP needs > a|x|b steps.



Reductions• Motivation:

◦ Reductions are a main tool for comparing problems.

• Objective:

◦ Formalize the notion of “reductions”.

• Overview:

◦ Define Karp reductions (notation: ≤P )

◦ Example: show HAMPATH ≤P HAMCYCLE

◦ Closeness under reductions

◦ Define Cook reductions

◦ Discuss Completeness



ReductionsConcept: Consider two problems / languages A and B.Assume we can make

an efficient procedure for problem A

using an efficient procedure for problem B

Then it follows that

• A cannot be radically harder than B,

or, in other words,

• B is at least as hard as A.

Notation: A ≤x B



Karp Reductions: DefinitionDefinition: A language A is polynomial time (Karp)

reducible to a language B (denoted by A ≤P B), if

• there exists a polynomial time computable function

f : Σ∗A → Σ∗B (called reduction-function), such that

• w ∈ A ⇔ f(w) ∈ B.

there exists a DTM which computesf(w) from w in polynomial time







• w ∈ A ⇔ f(w) ∈ B.

AB

Σ∗A\A Σ∗B\B

f

Questions: Σ of DTM for f? Bounds on |f(w)|?







• w ∈ A ⇔ f(w) ∈ B.

Teminology: This type of reductions is also called . . .

. . . polynomial time Karp reduction

. . . polynomial time many-one reduction

. . . polynomial time mapping reduction

We will mostly use reductions of this type.



Reductions and EfficiencyConcept: Consider two problems / languages A and B.Assume we can make



polynomial-timealgorithm for B

Realization:

x ∈ B?x







polynomial-timealgorithm for f

w f(w)∈B?f(w)

Realization:







polynomial-timealgorithm for f

w f(w)∈B?f(w)

Realization:

w ∈ A?

a polynomial time algorithm for A

w ∈ A?



How to Reduce?Steps to show A ≤P B:

1. Come up with a reduction function f .

2. Show that f is polynomial time computable.

3. Prove that f is a reduction (of A to B), i.e., show:

• w ∈ A ⇒ f(w) ∈ B• f(w) ∈ B ⇒ w ∈ A



HAMPATH ≤P HAMCYCLEHAMPATH: Hamiltonian path

• L = {(G, s, t) | G is a directed graph with aHamiltonian path from s to t }

• Instance: A directed graph G = (V,E)and two vertices s 6= t ∈ V .

• Problem: Decide if G contains a Hamiltonian pathfrom s to t, i.e., a path that goes througheach node of G exactly once.



HAMPATH ≤P HAMCYCLEHAMCYCLE: Hamiltonian cycle

• L = {G | G is a directed graphwith a Hamiltonian cycle }

• Instance: A directed graph G = (V,E)

• Problem: Decide if G contains a Hamiltonian cycle,i.e., a cycle that goes througheach node of G exactly once.



HAMPATH ≤P HAMCYCLESteps to show HAMPATH ≤P HAMCYCLE:

1. Come up with a reduction function f :f(G=(V,E), s, t) := G′=(V ∪ u,E ∪ {(u, s), (t, u)})


X

X



HAMPATH ≤P HAMCYCLE

X

X

Steps to show HAMPATH ≤P HAMCYCLE:



3. Prove that f is a reduction:

• w ∈ HAMPATH ⇒ f(w) ∈ HAMCY CLE:Given a Hamiltonian path (s, . . . , t) in G,(s, . . . , t, u) is a Hamiltonian cycle in G′.

X



HAMPATH ≤P HAMCYCLE

X

X

X

Steps to show HAMPATH ≤P HAMCYCLE:



3. Prove that f is a reduction:

• w ∈ HAMPATH ⇒ f(w) ∈ HAMCY CLE:

• f(w) ∈ HAMCY CLE ⇒ w ∈ HAMPATH:In a Ham. cycle in G′, u must be preceeded by t andfollowed by s ⇒ A Ham. cycle in G′ is of the form(s, . . . , t, u). ⇒ Removing u yields a Ham. path(s, . . . , t) in G.

X

X

X



Closeness: DefinitionDefinition: A complexity class C is closed under

(some type x of) reductions, if

• L is x-reducible to L′ and L′ ∈ C⇒ L ∈ C as well.

Theorem:• The complexity classes P, NP, PSPACE, and EXP are

closed under polynomial time (Karp) reductions.

Proof:• Exercise

Question: What could be a different type of reduction?



Log-Space ReductionsDefinition: A language A is log-space (Karp) reducible

to a language B (denoted by A ≤L B), if

• there exists a log-space computable function


• w ∈ A ⇔ f(w) ∈ B.

AB

Σ∗A\A Σ∗B\B

f



Log-Space ReductionsDefinition: A language A is log-space (Karp) reducible

to a language B (denoted by A ≤L B), if

• there exists a log-space computable function


• w ∈ A ⇔ f(w) ∈ B.

Theorem:• The complexity classes L, NL, P, NP, PSPACE, and

EXP are closed under log-space (Karp) reductions.

Question: What could be a different type of reduction?



Cook ReductionsDefinition: A language A is Cook reducible

to a language B (denoted by A ≤PT B), if

• there exists an algorithm that solves A using

• a polynomial number of calls to a subroutine for B,

• and polynomial time outside of these subroutine calls.

algorithm for B

algorithm for AO(p(n)) calls

Teminology: Cook reductions are also calledpolynomial time Turing reductions



Cook ReductionsDefinition: A language A is Cook reducible

to a language B (denoted by A ≤PT B), if

• there exists an algorithm that solves A using

• a polynomial number of calls to a subroutine for B,

• and polynomial time outside of these subroutine calls.

Exercise: Make Cook reductions for

HAMPATH ≤PT HAMCYCLE,

HAMCYCLE ≤PT HAMPATH.

Teminology: Cook reductions are also calledpolynomial time Turing reductions



Cook vs. Karp Reductions

From now on: mostly Karp reductions

Question: Which complexity classes are closed underCook reductions?

Question: How do Cook reductions and polynomial timeKarp reductions differ?

• Cook reductions allow us to call the proceduremany times (instead of only once)

• When Karp reducing we must output the answerof the procedure (no post-processing)

• polynomial time Karp reductions are a special caseof Cook reductions



CompletenessDefinition: Let C be a complexity class and let L be alanguage. We say that L is C-complete (w.r.t. x-red.), if

1. L is in C, and

2. for every language L′ in C, L′ is x-reducible to L.

In other words:L is at least as hard as any language in C.

some type x of reductions



CompletenessDefinition: Let C be a complexity class and let L be alanguage. We say that L is C-complete (w.r.t. x-red.), if

1. L is in C, and

2. for every language L′ in C, L′ is x-reducible to L.

Theorem: If two complexity classes C and C′are bothclosed under x-reductions and there is a language L whichis complete (w.r.t. x-red.) for both, C and C′, thenC = C′.

Proof: We only show C ⊆ C′:All languages in C are x-reducible to L. Since C′ is closedunder x-reductions, it follows that C ⊆ C′.



P-CompletenessDefinition: A language L is P-complete (w.r.t. ≤P ), if

1. L is in P (membership), and

2. for every language L′ in P: L′ ≤P L (hardness).

Question: Which languages in P are P-complete(w.r.t. ≤P )? Which are not?

Question: What about P-completeness with respect toCook reductions?



NP-Completeness

Theorem: If there exists any language that isNP-complete and is in P, then it follows that P = NP.

Definition: A language L is NP-complete (w.r.t. ≤P ), if

1. L is in NP (membership), and

2. for every language L′ in NP: L′ ≤P L (hardness).

Proof: Exercise



DiscussionWhat we achieved:

• We’ve introduced different types of reductions:

◦ Cook / Karp reductions

◦ log-space / poly-time reductions

• We’ve defined the notions ofcloseness and completeness

• We’ve seen how this can help us to prove equality ofcomplexity classes









What’s missing:

We do not yet know any NP-complete languages.









What’s next:

• Present the first NP-complete problem: SAT

• Prove its NP-completenes: The Cook-Levin theorem



SAT: DefinitionSAT: Satisfiability of Boolean formulas

• SAT = {φ | φ is a satisfiable Boolean formula}Boolean formula: Boolean variables (values 0/F, 1/T)

Boolean operations (∧,∨,¬)

• Instance: A Boolean formula φExamples: ((x1 ∨ x2 ∨ ¬x3) ∧ ¬x1) ∨ ¬(x3 ∧ x2)

x1 ∧ ¬x1

• Problem: Decide if there exists atruth assignment for the variables of φsuch that φ evaluates to TRUE.



SAT is NP-complete: Overview

To show that SAT is NP-complete, we need to show ...

• membership: SAT ∈ NP.

• hardness: L ≤P SAT ∀ L ∈ NP.



SAT is a member of NP

Proof:

• Given a boolean formula φ and a truth assignment Tfor the variables of φ

• We can verify efficiently if φ(T ) evaluates to true.

Theorem:• SAT is in NP.

Proof: Exercise



SAT is NP-hardTo show that SAT is NP-hard, we need to show thatfor every problem L ∈ NP, L ≤P SAT.

Recall:The necessay steps to show A ≤P B are:




• w ∈ A ⇒ f(w) ∈ B• f(w) ∈ B ⇒ w ∈ A




Proof Idea: Given a p(n)-time NTM Nand an input x for N ,

• construct a Boolean formula φN,x such that

• φN,x satisfiable ⇔ N accepts x.

Observation:If L ∈ NP, then there must exist an NTM N thatdecides L in at most p(n) = O(nc) steps.



SAT is NP-hard1a. The NTM N = (Q,Σ,Γ,∆,q0,qaccept,qreject):

• States: Q = {q0, . . . qk}• Alphabet: Γ = {a1, . . . al}• Input: x = {x1, . . . xn}• additional “Input”:

x0 := .xj := t ∀j ∈ {n+ 1, . . . , p(n)}

· · ·x1 x2 x3 x4. t t t



SAT is NP-hard1b. The variables for our formula φN,x:

variable indices meaning

statet,q t ∈ {0, . . . , p(n)} statet,q ≡ T⇔ N is in

q ∈ Q state q after t steps

post,i t, i ∈ {0, . . . , p(n)} post,i ≡ T⇔ N ’s head

is on cell i after t steps

tapet,i,a t, i ∈ {0, . . . , p(n)} tapet,i,a ≡ T⇔ cell i

a ∈ Γ contains symbol a

after t steps



SAT is NP-hard1c. Overview of the formula φN,x:

The formula φN,x consists of several parts:

• the boundary conditions B,

• the start conditions S,

• the transition conditions T1, T2, and

• the end condition E

Altogether, φN,x is of the form

φN,x := B ∧ S ∧ T1 ∧ T2 ∧ E



SAT is NP-hard1d. The helping formula G:

We will repeatedly use the following formula:

G(v1, . . . , vm) :=

(m∨i=1

vi

)∧

m−1∧j=1

m∧i=j+1

¬(vi ∧ vj)

Question: What is the size of G with respect to m?

Question: When is G(v1, . . . , vm) true?

G(v1, . . . , vm) ≡ T ⇔ exactly one vi ≡ T.

G has size O(m2).



SAT is NP-hard1e. The boundary conditions B for φN,x:

ensure that at any point t of time ...

• the NTM N is in exactly one state,

• the head of N is at exactly one position, and

• on the tape of N there is exactly one symbol.

Altogether, we obtain

B :=∧p(n)

t=0 G(statet,q0 , . . . , statet,qk)

∧∧p(n)

t=0 G(post,0, . . . , post,p(n))

∧∧p(n)

t=0

∧p(n)i=0 G(tapet,i,a1

, . . . , tapet,i,al)

Observation: The size of B is O(p(n)3).



SAT is NP-hard1f. The start conditions S for φN,x:

guarantee that for t = 0:

• the NTM N is in its start state q0,

• the head of N is at position zero, and

• on the tape of N there is the initial input.


S := state0,q0 ∧ pos0,0 ∧p(n)∧j=0

tape0,j,xj

Observation: The size of S is O(p(n)).



SAT is NP-hard1g. The transition conditions T1 for φN,x:

describe the changes during a transition from t to t+ 1:

• the local situation before the transition:Tb(t, i, q, a) := (statet,q ∧ post,i ∧ tapet,i,a)

• the local situation after the transition:Ta(t, i, q, a) :=

∨(q′,a′,r)∈∆(q,a)

(statet+1,q′∧post+1,i+D(r)∧tapet+1,i,a′)

with D(left) := −1, D(right) := 1


T1 :=

p(n)∧t,i=0

∧q∈Q,a∈Γ

(Tb(t, i, q, a)→ Ta(t, i, q, a))

A→B := ¬A ∨B

Size of T1 = O(p2(n))



SAT is NP-hard1h. The transition conditions T2 for φN,x:

ensures that all band positions but the one where the headis currently are not altered by a transition:

T2 :=

p(n)∧t,i=0

∧a∈Γ

((¬post,i ∧ tapet,i,a)→ tapet+1,i,a

)1i. The end condition E for φN,x:

checks whether the accepting state qa is reached.Assuming that once reached, the accepting state is neverleft again, it suffices to check for t = p(n).

E := statep(n),qa

Size of T2 = O(p2(n))

Size of E = O(1)



SAT is NP-hard1c. Overview of the formula φN,x:

The formula φN,x consists of several parts:

• the boundary conditions B,

• the start conditions S,

• the transition conditions T1, T2, and

• the end condition E

Altogether, φN,x is of the form

φN,x := B ∧ S ∧ T1 ∧ T2 ∧ E

2.All of sizepolynomialin p(n)




Recall:The necessay steps to show A ≤P B are:




• w ∈ A ⇒ f(w) ∈ B• f(w) ∈ B ⇒ w ∈ A

XX

problem analysis and complexity theory ss16 · birgit vogtenhuber problem analysis and complexity...

Documents