applied computer science ii chapter 7: time complexity prof. dr. luc de raedt institut für...
Post on 21-Dec-2015
213 views
TRANSCRIPT
Applied Computer Science IIChapter 7: Time Complexity
Prof. Dr. Luc De Raedt
Institut für InformatikAlbert-Ludwigs Universität Freiburg
Germany
Overview
• Measuring complexity• The class P• The class NP• NP-completeness• Additional NP-complete problems
Measuring complexity
• Worst-case versus average-case
• We typically use big-O and small-o notation (cf. ACS I); asymptotic analysis
The time complexity of a TM is the function
: , where ( ) is the max number of steps
that uses on any input of length .
M
f f n
M n
Big-O
0
0
Let , : be two functions. We say
that if positive integers and exist
so that for every integer holds that
( )
(
is ( ( ))
( ( )) is an upper bound for ( )
) . ( )
f g R
c n
n n f n
f n O g n
c g n
O g n f n
3 2
30
4 2
( ) 5 2 22 5
= ( ) choose 6 and 10
is also ( ) but not ( )
f n n n n
f O n c n
f O n O n
2 2 2( ) 3 .log 5 .log log 2
( ) ( .log )
f n n n n n
f n O n n
pb
b b b
b b b
p p q (pq)b b
b a b
log x = p if and only if b = x (definition)
log x*y = log x + log y
log x/y = log x - log y
log x = p log x which implies that (x ) = x
log x = log x * log a
Small-o
0
Let , : be two functions. We say
that if
( )lim 0, i.e. for any real number 0,
( )
there is a number where ( ) . ( ) for
( ) is o( ( ))
all
n
o
f g R
f nc
g n
n
f n g
f n c g n n n
n
Time Complexity
k 2
Let : be a function. Define the
( ( )) ={ | is a language decided by an ( ( )) time Turing machine}
So, {0 1 | 0} ( )
Can we find a machine that decides as
time complexity c
symptotic
lass
k
t
TIME t n L L O t n
k TIME n
A
2
ally more quickly ?
i.e. is ( ( )) for ( ) ( ) ?A TIME t n t n o n
Complexity relationship among models
2
7.8
Let ( ) be a function where ( ) .
Then every ( ) time multitape Turing machine has
an equivalent ( ( ) ) time single-tape Turing machine.
t n t n n
t n
O t n
Theorem
2
elements
Let be a tape TM that runs in ( )
Construct (single-tape TM) that runs in ( ( ) ) time
Analyze simulation
To simulate each step of ,
makes two scans over active input tape
M k t n
S O t n
M
S
Proof
each of the active parts of the tapes are of length at most ( )
i.e. . ( ) ( ( ))
and possibly shifts (for inserting a new cell)
shifts take ( ( )) time
First phase
k t n
k t n O t n
k
O t n
2
2 2
: copying inputs from to
takes ( ) steps
Second phase : simulate the ( ) steps of
takes ( ). ( ( )) ( ( ) ) steps
( ) ( ( ) ) ( ( ) ) when ( )
M S
O n
t n M
t n O t n O t n
O n O t n O t n t n n
• Definition 7.9 including figure 7.1
Let be an NTM that is a decider. The running
time of is the function : , where ( ) is the
maximum number of steps that uses on any branch of its
computation of any input of length as
N
N f f n
N
n
Definition
shown below
( ( ))
7.10
Let ( ) be a function where ( ) .
Then every ( ) time nondeterministic Turing machine has
an equivalent 2 time deterministic single-tape Turing machine.O t n
t n t n n
t n
Theorem
2 ( )
elements
Every branch of has length at most ( )for input of length
Every node in tree has at most children (determined by )
Exploration tree is breadth first :
1+ ... nodes and let n
N t n n
b
b b b
Proof
( )
( ) ( ( ))
aves
= ( )
Time for processing one node (or leaf) starting from root
( ( ))
Running time of deterministic 3-tape TM
( ( )). ( ) 2
Use transformation to deterministic single tape
t n
t n O t n
O b
O t n
O t n O b
( ( )) 2 ( ( )) gives (2 ) 2O t n O t n
Differences in models
• Polynomial difference between single- and multiple tape deterministic TM
• Exponential difference between deterministic and non-deterministic TM
• Compare exponential versus polynomial behaviour !
Towards the class P
Distinction between :• Exponential algorithms
– Brute force search– Impractical
• Polynomial– Practical – Robust for reasonable models of computation
(I.e. polynomially equivalent)
Here, we ignore differences between,e.g. , level of abstraction5( ) and ( )O n O n
• P is invariant for all models of computation that are polynomially equivalent to a single tape deterministic TM
• P roughly corresponds to the class of problems that are realistically solvable on a computer
The class P
P is the class of languages that are decidable in polynomial time
on a deterministic single -tape TM, i.e.
P= ( )k
k
TIME n
Definition
Conventions
• High-level descriptions of algorithms• Abstraction of specific model used (as
long as polynomially equivalent)• Algorithms proceed in numbered stages• Analysis in two steps :
– A polynomial bound on the number of stages– Each stage can be implemented in
polynomial time on deterministic single tape TM (or poly. Equivalent model)
Conventions
• Encoding method for problems• We use <.> notation as before• Reasonable methods require
encoding and decoding of objects into other (internal ?) representation to require polynomial time.
• E.g. 111111111 (unreasonable) instead of 9 (or other base k-notation)
Encoding Graphs
• Option 1 : – List of nodes– List of edges
• Option 2 :– Adjacency matrix
1 iff there is an edge from node to node Entry ( , )
0 otherwise
i ji j
• Insert algorithm
{ , | and are relative prime}
i.e. the greatest common divisor of and is 1
P
RELPRIME x y x y
x y
RELPRIME
Theorem
Every context-free language is a member of P
Theorem
Assume language in Chomsky Normal Form
Then each derivation of a string has 2 1steps
(Generating all
Dynamic programming
derivations is polynomial!)
accumulate information about smaller problem
:
s
w n
not
keep track of table of size
( , ) the set of variables generating ...
with
i j
n n
table i j w w
i j
Towards the class NP{ , , | is a directed graph with a hamiltonian path from to }
i.e. it is directed and contains all nodes in
HAMPATH G s t G s g
G
HAMPATH
• Easy to obtain exponential time algorithm :– Generate all paths of length m (no. of nodes)
and check whether they are hamiltonian
• Polynomial verifiability :– Checking whether a given path is hamiltonian
can be done in polynomial time
• Verifying existence of hamiltonian path is easier than determining the existence
• Consider also • Similar for COMPOSITES
{ | . for integers , 1}COMPOSITES x x p q p q
HAMPATH
The class NP
A verifier for a language is an algorithm s.t.
{ | accepts , for some string }
: certificate, proof
Time of verifiers is measured in terms of size of
(implies that is of size polyn
A V
A w V w c c
c
w
c
Definition
omial in size )
Polynomial time verifiers
is the class of languages that have
NP comes from Nondeterministic polyno
polynomial time
mial
verifiers
time
w
Definition
NP
• Proof
A language is in NP iff it is decided by some nondeterministic
polynomial time algorithm
Theorem
idea
NTIME( ( )) { | is a language decided by an ( ( )) time NTM}
NP= ( )k
k
t n L L O t n
NTIME n
Definition
Corollary
{ , | is an undirected graph with a -clique}CLIQUE G k G k
1 1 1{ , | { ,..., } and for some { ,..., } { ,..., }we have }
The and are multi-sets.
is in NP
k l k ii
SUBSET SUM S t S x x y y x x y t
y x
SUBSET SUM
Theorem
P versus NP
• P= the class of languages where membership can be decided quickly
• NP=the class of languages where membership can be verified quickly
• NP-complete problems : if a polynomial time (determ.) algorithm exists for one NP-complete problem, then P=NP
NP EXPTIME= (2 )kn
k
TIME
true is denoted by 1
false is denoted by 0
and
or
not
0 0 0 0 0 0 0 0 1
0 1 0 0 1 1 1 1 0
1 0 0 1 0 1
1 1 1 1 1 1
A Boolean formula is an expression involving Boolean variables and operations
E.g.
Boolean Logic
=( ) ( )
Assignment of values (0 and 1) to variables
For each assignment one can evalue a formula
If the formula evaluates to 1, we say that the assignment satisfies the formula.
T satisfiabilie yh t
x y x z
problem is to test whether a formula is satisfiable
(i.e. whether there exists an assignment that satisfies the formula)
{ | is a satisfiable boolean formula}
(Cook-Levin)
P iff P=NP
SAT
SAT
Theorem
Polynomial time reducibility
• Fig 7.6
* * A function : is a
if some polynomial time TM exists that halts with
just
polyno
( ) on its tape, when started on any inp
mial time computable func
ut
n
tiof
M
f w w
* *
Language is to language
written , if a polynomial time computable function :
exists where for every
polynomial time (mapping) reducible
polynomial time red
: iff ( )
is a uction
P
A B
A B f
w w A f w B
f
3 { | is a satisfiable 3cnf formula}
cnf : conjunctive normal form, i.e. a conjunction of disjunctions
(disjunctions are clauses, elements of disjunctions are literals)
3cnf : each clause/di
SAT
1 1 1
sjunction has exactly three literals
e.g. ( ) ( )
3 is polynomial time reducible to
Let be of the form ( ) ... ( )
( ) , where
all nodes in are org
k k k
x y z y x w
SAT CLIQUE
a b c a b c
f G k
G
Theorem
Proof
1anised in triples ,...,
each triple corresponds to a cluase and has 3 nodes
The edges of connect all nodes except for :
1. edges between the nodes of the same triple
2. nodes with contradict
kt t
G
ory label, i.e. and
Now prove that is a polynomial time reduction
x x
f
NP-completeness
A language is NP-complete if
1. is in NP, and
2. every in NP is polynomial time reducible to
If is NP-complete and , then P=NP
If is NP-complete and for in NP, P
B
B
A B
B B P
B B C C
Definition
Theorem
Theorem
then is NP-completeC
Cook-Levin
is NP-complete
1. is in NP (easy)
2. any language in NP is polynomial time reducible to
Let be a NTM that decides in time
Define an (accepting) tableau for on
k
SAT
SAT
A SAT
N A n
N w
Theorem
Proof
, ,
define reduction from to
on input , define ( )
let us first define the boolean variables.
Let {#}
for 1 , , for :
define ( , ) contains symbol
Now define
k
i j s
cell start mov
f A SAT
w f w
C Q
i j n s C
x cell i j s
0 1
, , , , , ,, ,1 ,
1,1,# 1,2, 1,3, 1, 2, 1, 3, 1, 1, 1, ,#
( )
"each cell contains exactly one symbol"
... ...
" the first row correspo
k
k kn
e accept
cell i j s i j s i j ts C s t C s ti j n
start q w n w n n n
x x x
x x x x x x x
nds to the starting situation"
, ,1 ,
1 1 1 2 2
"at least one of the configurations is an accept state"
concept of legal windows
e.g. ( , ) {( , , )} and ( , ) {( , , ), ( , , )}
if the top row of the table is t
k acceptaccept i j qi j n
x
q a q b R q b q c L q a R
Claim he start configuration
and if every window (2x3) is legal then each row legally follows the preceding one.
1 2 3 4 5 61 6
1 ,1
, 1, , , , 1, 1, 1, 1, , 1, 1,,..., is a legal window
2
"the , window is legal"
for each window :
( )
3.Analyze the complexity
tableau has ce
k kmovei n j n
i j a i j a i j a i j a i j a i j aa a
k k k
i j
x x x x x x
n n n
2
lls
each cell has variables (for symbols in )
thus number of variables is ( )
Now investigate different components of
(All ( ) or ( ); indices can be encoded in (log
k
cell start move accept
k k
l C
O n
O n O n O n
)
( ) is of polynomial size and can be computed in polynomial timef w
3 is NP-complete
modify construction in proof
first rewrite in cnf form.
second rewrite clauses such that they have 3 literals
1. if less than 3 literals e.g. rewrite to e.g.
SAT
SAT
a b a b
Theorem
Proof
1
1 2 1
1 3 2
3 1
2. if more than 3 literals ( ... )
rewrite into
( )
( )
...
( )
l
l l l
b
a a
a a z
z a z
z a a
is NP-completeCLIQUE
Corollary
{ , | is an undirected graph that has a -node vertex cover}
Let be an undirected graph ( , ) and a number
A is a set of nodes ' such ve thrtex at
for all edges
cover
( , ) holds that
VERTEX COVER G k G k
G V E k
V V
u v E
'or '
is NP-complete
produce clause gadget and variable gadget
set 2 where has variables and clauses
then prove satisfiability if and only if graph has a vertex cover
u V v V
VERTEX COVER
k m l m l
Theorem
Proof
with nodesk
1 1 1
is NP-complete
we show that 3
Let be of the form ( ) ... ( )P
k k k
HAMPATH
SAT HAMPATH
a b c a b c
Theorem
Proof
{ | is an undirected graph that contains a hamiltonian path}
is NP-complete
( ) '
for all nodes in , introduce nodes , , in '
for and in
P
in mid out
UHAMPATH G G
UHAMPATH
HAMPATH UHAMPATH
f G G
u G u u u G
s t
Theorem
Proof
1
1 1
, introduce only ,
introduce edges between and and between and
connect also to if there is an edge from to
each path , ,..., , in has a counterpart
, ,
out in
in mid mid out
out in
k
in miout
G s t
u u u u
u v u v
s u u t G
s u u 1, ,..., and vice versa
d out inu t
1 1 1{ , | { ,..., } and for some { ,..., } { ,..., }we have }
The and are multi-sets.
is NP-complete.
1. NP
2.3
Let be in 3cnf and h
k l k ii
P
SUBSET SUM S t S x x y y x x y t
y x
SUBSET SUM
SUBSET SUM
SAT SUBSET SUM
Theorem
Proof
1 1ave vars ,..., and clauses ,...,l kx x c c
Set Covering
1
1
j
SetCover={< ,..., , , | (finite set) and there exist
sets ,..., such that }
Example
<{a,b,c},{b,c,d},{c.d},{a,b,c,d},2> SetCover
<{a,b,c},{b,c,d},{c.d},{a,b,c,d},1> SetCover
n j
k i
i i i
T T M n T M n
T T T M
BinPacking
1BinPacking={< ,..., , , | (all in ) and the objects
be divided over the bins of size such that no bin overflows}
Example
<4,5,7,2,2,9> BinPacking
because 4+5 and 7+2 are both smaller than
n i ia a k b a b a
k b
9
2 bins of size 9
<4,5,7,2,2,8> BinPacking
Traveling Salesman Problem
TSP={< , | contains distance between city and
and there exists a path through all cities of size less than }
0 5 5
5 0 5
5 5 0
10
1,2,3 is a solution
for 9 instead of 10, no solution
ijM k M i j
k
path
Coloring
Coloring={< , | is a graph and there
exists a coloring with colors so
that no two adjacent have the same color}
G k G
k
Key Reductions
SAT
3SAT
SetCover Clique SubsetSum Hampath Coloring
VertexCover Partition
BinPacking
UHampath
TSP
Conclusions
• The class P• The class NP• NP-completeness(e.g. 3SAT)• Polynomial time reductions• Two further classes :
– coNP : – NP-hard:
A language is NP-hard if
every in NP is polynomial time reducible to
B
A B
coNP={ | NP}A A