Complexity 15-1

ComplexityAndrei Bulatov

Hierarchy Theorem

Complexity 15-2

NL and coNL

For a language L over an alphabet , we denote the complementof L, the language * - L

L

Definition The class of languages L such that can be solved by a non-deterministic log-space Turing machine verifier is called coNL

Definition The class of languages L such that can be solved by a non-deterministic log-space Turing machine verifier is called coNL

L

Theorem NL = coNL

Theorem NL = coNL

Complexity 15-3

Proof

Therefore it is enough to show that No-Reachability is in NL.

The properties of NL and coNL are similar to those of NP and coNP:

• if a coNL-complete problem belongs to NL then NL = coNL

• if L is NL-complete then is coNL-completeL

Reachability is NL-complete.

In order to do this, we have to find a non-deterministic algorithm that proves in log-space that there is a path between two specified vertices in a graph

Complexity 15-4

Counting the number of reachable vertices

Given a graph G and two its vertices s and t; let n be the number of vertices of G

First, we count the number c of vertices reachable from s

Let be the set of all vertices connected to s with a path of length at most i, and

iA|| ii Ac

Clearly, , and nAAAs 10}{ ccn

We compute the numbers inductivelynccc ,,, 10

Complexity 15-5

Suppose is known. The following algorithm non-deterministically either compute or reject

ic

1ic

• set 01 ic

• for every vertex v from G do

• set icm

• for every vertex w from G non-deterministically do or not do

- check whether or not using random walk iAw

- if not then reject

- if yes then set m = m – 1

- if there is the edge (w,v) then set 111 ii cc

• if m 0 reject

• output 1ic

Complexity 6-16

Checking Reachability

Given G, s, t and c

• for every vertex v from G non-deterministically do or not do

- check whether or not v is reachable from s using random walk

- if not then reject

- if yes then set m = m – 1

- if v = t then reject

• set cm

• if m 0 reject

• accept

Complexity 15-7

Complexity Classes

We know a number of complexity classes and we how they relate each other

L NL P NP , coNP PSPACE

However, we do not know if any of them are different

Questions P NP and L NL concern the (possible) difference between determinism and nondeterminism and known to be extremely difficultComplexity classes can be distinguished using another parameter:

The amount of time/space available

Complexity 15-8

Space Constructable Functions

Definition A function f: N N, where f(n) log n, is called space constructable, if the function that maps to the binary representation of f(n) is computable in space O(f(n)).

Definition A function f: N N, where f(n) log n, is called space constructable, if the function that maps to the binary representation of f(n) is computable in space O(f(n)).

n1

Examples

• polynomials

• n log n

•

•

,3 ,2 nn

Complexity 15-9

Hierarchy Theorem

Theorem For any space constructable function f: N N, there exists a language L that is decidable in space O(f(n)), but not in space o(f(n)).

Theorem For any space constructable function f: N N, there exists a language L that is decidable in space O(f(n)), but not in space o(f(n)).

Corollary If f(n) and g(n) are space constructable functions, and f(n) = o(g(n)), then SPACE[f] SPACE[g]

Corollary If f(n) and g(n) are space constructable functions, and f(n) = o(g(n)), then SPACE[f] SPACE[g]

Corollary L PSPACE

Corollary L PSPACE

Corollary NL PSPACE

Corollary NL PSPACE

Complexity 15-10

Proof Idea

Diagonalization Method:

• apply a Turing Machine to its own description

• revert the answer

• get a contradiction

In our case, a contradiction can be with the claim that something is computable within o(f(n))

Let L = {“M” | M does not accept “M” in f(n) space}

Computability and Complexity 15-11

If M decides L in space f(n) then what can we say about M(“M”)?

• if M(“M”) accepts then “M does not accept M in space f(n)”

• if M(“M”) rejects then “M accepts M in space f(n)”

There are problems

• we showed that L cannot be decided in space f(n), while what need is to show that it is not decidable in space o(f(n))

• what we can assume about a decider for L is that it works in O(f(n)); but this means the decider uses fewer than cf(n) cells for inputs longer than some . What if “M” is shorter than that?0n

Complexity 15-12

ProofIn order to kick in asymptotics, change the language L = {“M ” | simulation of M on a UTM does not accept “M ” in f(n) space}

k10 k10

The following algorithm decides L in O(f(n))

On input x

• Let n be the length of x

• Compute f(n) and mark off this much tape. If later stages ever attempt to use more space, reject

• If x is not of the form M for some M, rejectk10

• Simulate M on x while counting the number of steps used in the simulation. If the count ever exceeds , accept)(2 nf

• If M accepts, reject. If M rejects, accept

Complexity 15-13

The key stage is the simulation of M

Our algorithm simulates M with some loss of efficiency, because the alphabet of M can be arbitrary.

If M works in g(n) space then our algorithm simulates M using bg(n) space, where b is a constant factor depending on M

Thus, bg(n) f(n)

Clearly, this algorithm works in O(f(n)) space

Complexity 15-14

Suppose that there exists a TM M deciding L in space g(n) = o(f(n))

We can simulate M using bg(n) space

There is such that for all inputs x with we have0n 0|| nx |)(||)(| xfxbg

Consider )10( 0nMM

Since , the simulation of M either accepts or rejects on this input in space f(n)

0|10| 0 nM n

• If the simulation accepts then does not accept )10( 0nMM

• If the simulation rejects then accepts )10( 0nMM

Complexity 15-15

Time Hierarchy Theorem

Theorem For any time constructable function f: N N, there exists a language L that is decidable in time O(f(n)), but not in

time .

Theorem For any time constructable function f: N N, there exists a language L that is decidable in time O(f(n)), but not in

time .

)(log)(nf

nfo

Corollary If f(n) and g(n) are space constructable functions, and

, then TIME[f] TIME[g]

Corollary If f(n) and g(n) are space constructable functions, and

, then TIME[f] TIME[g]

)(log)(

)(nf

nfong

Complexity 16-16

Definition Definition

0

]2TIME[

k

nk

EXPTIME

The Class EXPTIME

Corollary P EXPTIME

Corollary P EXPTIME

Complexity 15-17

ComplexityAndrei Bulatov

Search and Optimization

Complexity 15-18

Search Problems

Often we need to find a solution to some problem, rather than to show that a solution exists

In this case the problem is said to be a search problem

Instance: A finite graph G.

Objective: Find a Hamilton Circuit in G, or report it does not exist

Hamilton Circuit(S)

Complexity 15-19

More Examples

Instance: A finite graph G and an integer K.

Objective: Find a colouring of G with K colours or report it does not exist

Colouring(S)

Instance: A conjunctive normal form .

Objective: Find a satisfying assignment for or report it does not exist

Satisfiability(S)

Complexity 15-20

Reduction to Decision Versions

Search problems look much more practical than decision problems, but we have studied mostly decision problems because:

• there always is the decision version of a search problem

• complexity classes, reducibility, etc. for decision problems are simpler than those for search problems

• normally, if we know how to solve a decision problem, we can also solve its search version

Complexity 15-21

Reduction for Satisfiability

We use an algorithm for the decision version as an oracle

Given a formula with the set of variables V

• For every X V do

• If has a satisfying assignment set X = 00| X

- otherwise, if has a satisfying assignment set X = 1 1| X

report “there is no solution”

• output the obtained assignment

Complexity 15-22

Reduction for HamCircuit

Observation:

A graph G has a Hamilton Path from u to v if and only if the following graph has a Hamilton Circuit

G:

u

v

u

v

Complexity 15-23

Given a graph G with vertex set V

• take a vertex Vv 0

• for i= 1 to |V| –1 do

• for every neighbour v of do1iv

- if there is a Hamiltonian path in H from to v then0v

• set H := G

set vvi :

remove and all adjacent edges from Hiv

• until is definediv

• if is not defined then output “there is no Hamiltonian Circuit”iv

• output the Hamiltonian Circuit 1||10 ,,, Vvvv

Complexity 15-24

Optimization Problems

In some problems we seek to find the best solution among a collection of possible solutions

Problems of this type are called optimization problems

Definition An optimization problem is a 4-tuple (I,S,m,opt), where

• I is a set of possible instances (encoded by strings over some alphabet

• S is a function mapping each instance to a set of feasible solutions

• m is a function mapping pairs of instances and solutions to a positive measure of goodness

• opt is either max or min, and indicates whether we seek maximum or minimum goodness

Complexity 15-25

To solve an optimization problem we must find for any given x I, a solution y S(x) such that

)}(|),({),( xSzzxmoptyxm

The optimal value will be denoted OPT(x)

Complexity 15-26

Example

Instance: A finite set of cities , and a positive integer distance , between each pair .

Objective: Find a permutation of that minimizes

Travelling Salesperson(O)

},,,{ 21 nccc ),( jid

},,1{ n

},{ ji cc

))1(),(())1(),((1

1

ndiidn

i

• I contains sets of cities and distances

• S gives the set of all possible orderings for each instance

• m gives the length of the tour for each instance and choice of ordering

• opt is min, because we seek for a shortest tour

Complexity 15-27

Instance: A graph G = (V,E).

Objective: Find a smallest set M N such that for each edge (u,v) E we have u,v M

Minimal Vertex Cover¹

• I contains all graphs

• S gives the set of all vertex covers for each graph

• m gives the size of the vertex cover (the number of vertices)

• opt is min, because we seek for a smallest vertex cover

Example

¹Sometimes called Min Node Cover or just Minimum Cover

Complexity 15-28

NP-Hard

By extending the notion of reduction, we can show that many optimization problems are hard to solve

Algorithms for the examples above would allow us to solve the corresponding decision problems, which are all NP-complete

We will call a problem NP-hard if all problems in NP are reducible to it in polynomial time

Complexity 15-29

Reduction for TSP

Given a set of cities and distances},,,{ 21 nccc ),( jid

First, we find the length of the shortest tour

• set L := 0 and )},(max{: jidnU

• do

- set

2:

ULM

- if there is a tour of length M then

set U := M

- otherwise set L := M

• until L = U

• return L

Complexity 15-30

Now we can find a shortest tour. Let its length be denoted L and let B = 2L

• for i = 1 to n –1 do

• for every do},,{},,{ 101 in vvccv

- if there is a tour for and of length then

• set

set vvi :

set and • until is definediv

• output the shortest tour 1||10 ,,, Vvvv

10 : cv

- let kcv

- set and otherwise0),(' 1 ki cvd ),(),(' ljdljd

),(' 1 ki cvdL ),(' ljd},,,{ 21 nccc

0:),(' 1 ki cvd ),(': 1 ki cvdLL

Complexity 15-31

Both, search and optimization problems can be reduced to decision problems

Why optimization problems are more interesting?

Because we can approximate…