computability and computational complexity · space complexity n we say that a language l is in the...

May 21, 2018 1

Computability and computational complexity

Fall 2015http://users.abo.fi/ipetre/computability/

Lecture 3: Linear speed-up; Space complexity; Nondeterministic complexity

Ion PetreComputer Science, Åbo Akademi University

http://users.abo.fi/ipetre/computability/

http://users.abo.fi/ipetre/computability/

Content

n Linear speed-up

n Space complexity

n Nondeterministic TM

n Nondeterministic time and space complexity

May 21, 2018 http://users.abo.fi/ipetre/computability/ 2

LINEAR SPEED-UP


Linear speed-up

n Next result: the rate of growth of the time/space requirements is important, but the multiplicative constants are not

n Theorem. Let LÎTIME(f(n)). Then for any e>0, LÎTIME(f’(n)), where f’(n)=ef(n)+n+2

n Proof sketch.q Let M=(K,S,d,s) be a k-tape machine deciding L in time f(n). Construct a k’-tape

machine M’ =(K’,S’,d’,s’) deciding L in time f’(n)q We need at least 2 tapes. If k=1, then k’=2, otherwise k’=kq Main trick: increase the “word length”, i.e., each symbol of M’ encodes several

symbols of M and each move of M’ several moves of Mq Let m be a constant (that we define later)q Setup: read the input left to right and compress it on the second tape: first m

letters encoded by a single character of M’, the following m letters encoded by the second character, etc. Consider padding with #s if the input length is not a multiple of m


Linear speed-up

n Proof (continued)q A stage: M’ will simulate m steps of M in at most 6 steps of its ownq In each stage M’ reads the characters to the left and to the right of the cursor (one

move to the left, two to the right, one to the left) and remembers the content into its staten it can now predict the next m moves of M because within m moves M cannot affect more

than m symbols either to the left or to the right of its cursorsn implement all m moves in two steps of M’ (changes in the current tuple and possibly in

one of its neighbors)q Once M accepts/rejects, M’ does the sameq Total time spent on input x in M’ is at most |x|+2+6éf(|x|)/mù

n enough to take m=é6/eù


Consequences of the linear speedup theorem

n TMs operating in less than n steps are anyway not interestingq they should read their whole input, so f(n)³n

n If f(n) is linear, say f(n)=an+b, then, according to the theorem, the constant a can be made arbitrarily close to 1

n If f(n) is super-linear, say f(n)=14n2+31n, then the constant 14 can be made arbitrarily small

n Conclusion: If L is a polynomially decidable language, then L is in TIME(nk), for some k>0

n Definition: The set P of all languages decided by Turing machines in

polynomial time is P=Èk³1TIME(nk)


SPACE COMPLEXITY


Space complexity

n Goal: define a measure of complexity in terms of space

n Note: tapes cannot become shorter during computationq also, an “interesting” machine should read all its input

n We can define the space complexity either through the sum of the space used by the end of the computation on all tapes, or through the max space on any one of the tapesq the two measures only differ through a constant

n Space complexity should capture the amount of the extra “work space” and not count the space needed to get the input and the space used to write the output

n Idea: how about treating the input tape as a read-only tape and the output tape as a write-only tape and ignoring their length from the estimate of the space complexity


Turing machines with input and output

n Definition. A k-tape Turing machine with input and output is a k-tape TM with the following restriction on the transition function: if d(p,a1,…,ak)=(q,b1,D1,…,bk,Dk), then q b1=a1 (read-only input tape)q if a1=#, then D1=¬ (end of input, do not create more space on the first tape)q Dk¹¬ (write-only output tape)

n Question: Do we lose computing power by restricting to TMs with input/output?

n Proposition. For any k-tape Turing machine M operating within time bound f(n) there is a (k+2)-tape TM M’ with input and output operating within time bound O(f(n))Proof sketchq copy the input to the second tapeq simulate M on tapes 2 through k+1q when M halts, copy the output to tape k+2


Space consumption

n Let M be a k-tape TM and x an input with (s,>,x,>e,…,>,e)→(H,w1,u1,…,wk,uk), where H is a halting state.

q The space required by M on input x is ∑ki=1|wiui|

n If M is a TM with input and output, then the space required by M on x is ∑k-1i=2|wiui|

n For f:N→N, the TM M is said to operate within space bound f(n) if for any input x it requires space at most f(|x|)


Space complexity

n We say that a language L is in the space complexity class SPACE(f(n)) if there is a TM with input and output that decides L and operates within

space bound f(n)

q Notation: L=SPACE(log(n))

n There is a linear speed-up theorem also for space complexity:

n Theorem. Let L be a language in SPACE(f(n)). Then for any e>0,

LÎSPACE(2+ef(n)).

q In other words, constants can be ignored in space complexity too


NONDETERMINISTIC TURING MACHINES


Nondeterministic Turing machines

n An unrealistic model of computationn Show that they can be simulated by deterministic machines

q exponential loss of efficiencyq can we really not do better?

n Definition: A nondeterministic Turing machine is a quadruple M=(K,S, d,s), where K, ∑, s are like before andq d Í (KxS) ´ [(K È {h,’yes’, ’no’}) x S x {¬,®,-}] is the transition relation

n in other words, for each state-symbol combination, there may be more than one legal move

n compare with the deterministic TM: d : KxS → (K È {h,’yes’, ’no’}) x S x {¬,®,-} is the transition function


Nondeterministic TM

n Definition: A configuration is (q,w,u) where: q qÎK, q w is the string to the left of the cursor, including the symbol scanned by the cursor, q u is the string to the right of the cursor

n Definition: Configuration (q,w,u) yields configuration (q’,w’,u’) in one step, denoted (q,w,u)→(q’,w’,u’) if:q w=xa, u=by, w’=xa’b, u’=y and [(q,a),(q’,a’,®)]Îdq w=xa, u=1, w’=xa’*, u’=1 and [(q,a),(q’,a’,®)} Îdq w=xba, w’=xb, u’=a’u and [(q,a),(q’,a’,¬)] Îdq w=xa, w’=xa’, u’=u and [(q,a),(q’,a’,-)] Îdq Note: compare with the correspondent notion for deterministic TM

n Define “yields” as a transitive closure of “yields in one step”: q configuration (q,w,u) yields (q’,w’,u’), denoted (q,w,u)®*(q’,w’,u’) if there is n³0 and

configurations (qi,wi,ui), 1£i £n such thatn (q,w,u)=(q0,w0,u0), (q’,w’,u’)=(qn,wn,un)n (qi,wi,ui)→(qi+1,wi+1,ui+1), for all 1£i £n-1


Nondeterministic computation


Time

“no”

“no”

“no”

“no” “no”“yes” “yes”

“yes”

Computing with a nondeterministic TM

Deterministic TM

n Let LÌ(S-{#})*n A Turing machine decides L iff for

every xÎ (S-{#})*, q if xÎL, then M(x)=‘yes’q if xÏL, then M(x)=‘no’

n A Turing machine accepts L iff for every xÎ (S-{#})*, xÎL iff M(x)=‘yes’q if xÏL, M might not terminate on x

n we could ask that M(x)= for all xÏL

n A Turing machine computes a partial (string) function f : (S-{#})* → S* iff for every xÎ (S-{#})*, M(x)=f(x)q In this case we call f a recursive function

Nondeterministic TM

n Let LÌ(S-{#})*n A nondeterministic Turing machine

decides L iff for every xÎ (S-{#})*, q if xÎL, then there is a ‘yes’-computation

starting from xq if xÏL, then all computations starting

from x end in “no”

n A nondeterministic Turing machine accepts L iff for every xÎ (S-{#})*, xÎL if and only if (s,>,x) ®*(“yes”,w,u)q if xÎL, then there is a ‘yes’-computation

starting from x

q if xÏL, some of the computations starting from x might not halt; those that halt end in “no”


Discussion

n An input is accepted if there is some sequence of nondeterministic choices that lead to “yes”q Many other computations on the same input might exist, some of them

maybe even non-haltingn An input is rejected if there are no accepting computations

n Note the asymmetry in the definitions of accepting/rejectingn Similar asymmetry in defining the accepting for a deterministic TM


Nondeterministic time complexity

n Definition: A nondeterministic TM decides language L in time f(n) where f:N→N, if q N decides Lq for any xÎ∑*, if (s,>,x)®k(q,u,v), then k£f(|x|)

n Discussionq any computation must halt in at most f(n) steps, where n is the length of the

inputq there may be exponentially many computations to explore

n to say “yes”, one single “witness” is enoughn to say “no”, all witnesses must be checked

n Definition: The set of languages decided by nondeterministic TM within time f is denoted as NTIME(f(n))

n In particular, NP=Èk³1NTIME(nk)q Note: PÍNP


Example: TSP(D): is there a tour of budget at most B

n Not known whether it is in P

n It is clearly in NP: can be decided by a nondeterministic TM in O(n2)

n Algorithm:

q consider a 2-tape nondeterministic TM

q for a given input with n cities, write an arbitrary sequence of letters of length

n on the second tape

q check (deterministically!) whether the string is a permutation of the cities and

if so, whether it is a tour of cost at most B.

q Answer “yes” if this is the case, “no” otherwise

q Needs at most O(n2) steps


Example: TSP(D): is there a tour of budget at most B - continued

n The algorithm on the previous slide decides TSP(D)q It accepts the input iff its associated TSP(D) problem has answer “yes”:

q if xÎTSP(D), then there is a tour of cost at most B and on the branch that generates that tour, the nondeterministic TM will halt in the accepting staten many other tours and even non-sense strings might be checked as welln not more than O(n2) steps on each branch

q if xÏTSP(D), then all branches will end in the rejecting state


From nondeterministic to deterministic Turing machines


From non-deterministic to deterministic

n Question: Can nondeterministic Turing machines be simulated by deterministic Turing machines?q Answer: YES!!q Catch: but with an exponential slow-down

n The core of the P=NP? problem: can it be done with just a polynomial slow-down?


From nondeterministic to deterministic TM

n Theorem. Suppose that a language L is decided by a nondeterministic Turing machine N in time f(n). Then it is decided by a 3-tape deterministic Turing machine M in time O(cf(n)), where c>1 is some constant depending on N. In other words:

NTIME(f(n)ÍÈc>1TIME(cf(n))


Proof

n Let N=(K,∑,d,s) be a NTM

n Let d be the degree of nondeterminism of N – the maximum number of nondeterministic choices in any state-symbol pair in dq in some configurations there is only one choice

n Any computation of N is a sequence of nondeterministic choices, i.e., a sequence of integers in the range 0…d-1

n Idea: simulate first all computation branches of length 1, then all those of length 2, etc.


Proof – continued

n Simulating all computation branches of N of length at most t with a deterministic TM M:q consider all sequences of integers 0…d-1 in increasing order of their length and

simulate N on each such sequence of nondeterministic choicesq write the current sequence on a second tape; the next one is the successor of the

current sequence when seen as a number in base dq if N answers “yes” on any such sequence of choices, then M halts with “yes”

n When do we reject?q whenever there is no continuing computation of the current length

n Time complexity for Mq number of sequences to examine: ∑t=1

f(n) dt = O(df(n)+1)q generating and simulating each sequence: clearly less than O(2f(n))


Nondeterministic space complexity


Nondeterministic space complexity

n The definition of NTM with input/output is similar to the corresponding notion for TMq first tape read-onlyq last tape write-only

n Definition. Given a k-tape nondeterministic TM with input and output N=(K,∑,d,s), we say that N decides L within space f(n) ifq N decides L andq for any xÎ(∑-{#})*, if (s,>,x,>,1,…,>,1)®*(q,w1,u1,…,wk,uk), then

∑k-1i=2|wiui|£f(n)

n this is similar as for deterministic TMn it applies even to non-accepting computations n N may not even halt on all computations


Example: REACHABILITY

n REACHABILITYq Instance: a finite directed graph G=(V,E) and two nodes 1,n ∈ Vq Question: Is there a path in G from 1 to n?

n The deterministic algorithm in Lecture 1 required O(n) spacen REACHABILITY is nondeterministically decidable within space O(log n)

q use two more tapes – in one of them write the current node i in binary (initially node 1), in the other “guess” an integer 1<j£n (intuitively the next node on the alleged path from 1 to n)

q check whether there is an edge from i to jq if not, halt with answer “no”q if yes, check if j=n. If yes, halt with answer “yes”. If no, replace i with j and repeat

n Analysisq clearly the algorithm solves the problemq the space taken by the two tapes is O(log n)q note that there are infinite computations within this machine (can be eliminated)

n Noteq the best known deterministic algorithm in space: O(log2n) – discuss it later on in the course


Discussion

n NTM is not a realistic model of computationq not introduced as a reaction to developments in technology

n It is a central concept in complexity theory because of its affinity to applications of computingq Proofs in logic: a fundamentally non-deterministic concept; one may try

on a statement all possible proofs, it becomes a theorem if one of them succeeds

q Artificial intelligence, combinatorial optimization: seek a solution among exponentially many that has the lowest cost

q The idea of verifying an alleged proof


Learning objectives

n Why constants do not matter in calculating time complexity

n The notion of space complexityq TMs with input and output

n Nondeterministic TM

n Nondeterministic complexity

n Definition of NP

n Equivalence of nondeterministic and deterministic TM


computability and computational complexity · space complexity n we say that a language l is in the...

Documents