Tomado del libro de CormenTomado del libro de Cormen
nalysis ofnalysis of
lgorithmslgorithms
OUTLINEOUTLINE
IntroductionWhat is an algorithm?
Semantic analysis
Correctness
Analysis of complexity
What is an algorithm?What is an algorithm?
• Well-defined computational procedure that takes Well-defined computational procedure that takes some value, or a set of values, as some value, or a set of values, as inputinput and produces and produces some value, or a set of values, as some value, or a set of values, as outputoutput. An . An algorithm is thus a algorithm is thus a sequence of computational of sequence of computational of stepssteps that transforms the input into the output. that transforms the input into the output. We can also view an algorithms as a tool for We can also view an algorithms as a tool for solving solving a well-specified computational problema well-specified computational problem[Cormen, 91].[Cormen, 91].
•A A set of instructionsset of instructions, clearly specified, that are , clearly specified, that are performed in order to performed in order to solve a problemsolve a problem [Baase, 91]. [Baase, 91].
•Unambiguous specification of a Unambiguous specification of a sequence of stepssequence of steps performed to performed to solve a problemsolve a problem [Aho, 95]. [Aho, 95].
AlgorithmAlgorithm
A sequence of computational of stepssequence of computational of steps for solving a well-specified computational problemwell-specified computational problem
The statement of the problem specifies in general terms the desired input/output relationship
The algorithm describes a specific computational procedure for achieving that input/ouput
relationship
AlgorithmAlgorithm
I: input / O:output
I Ostepssteps
INPUT: A sequence of numbers
<a1,a2,a3,...,an>
OUTPUT: A permutation
<a’1,a’2,a’3,...a’n> such that
a’1 ai2 ai3 ... ain
Sorting problemSorting problem
A possible input sequence is a instance of the sorting problem
<31,41,59,26,41,58>
A sorting algorithms must return: <26,31,41,41,58,59>
Instance of the sorting problemInstance of the sorting problem
Analysis of algorithmsAnalysis of algorithms
ComplexityComplexity
Time complexityTime complexity
Space complexitySpace complexityAnalysisAnalysis
Semantic Semantic AnalysisAnalysis Correctness Correctness
Semantic AnalysisSemantic Analysis What does an algorithm do? What does an algorithm do? (Discovering its function?)(Discovering its function?)
Is the study of the function associated to an algorithm:
• I I input space : input space : the space of values of the input variables defined in the algorithm.
• O O output space : output space : the space of the values of the output variables defined in the algorithm.
• X X state space : state space : the space of the values of the variables defined in the algorithm.
f :f : II O O { {}} function associated to the algorithmfunction associated to the algorithm
function mistery( n )function mistery( n ) i i 1 1 j j 0 0 for k for k 1 to n do1 to n do
j j i + j i + j i i j - i j - iend-for end-for **return jreturn j
Value in the variables at the point ** (n) (n) XX0 0 XX1 1 XX2 2 XX3 3 XX4 4 XX5 5 XX6 6 XX77 .… .… XXn n (j)(j) k k 0, 1, 2, 3, 4, 5, 6, 7,.…, 0, 1, 2, 3, 4, 5, 6, 7,.…, j j 0, 1, 1, 2, 3, 5, 8, 13,.…, 0, 1, 1, 2, 3, 5, 8, 13,.…, i i 1, 0, 1, 1, 2, 3, 5, 8,.…, 1, 0, 1, 1, 2, 3, 5, 8,.…,
ExamplExamplee
mistery( 0) = mistery( 0) = 00
mistery( 1) = mistery( 1) = 11
mistery( n ) =mistery( n ) = mistery( n - 1 ) + mistery( n - 1 ) + mistery( n - 2), for n > 2 .mistery( n - 2), for n > 2 .
““mistery( n ) computess the n-th Fibonacci’s mistery( n ) computess the n-th Fibonacci’s
number”number”
mistery( n ) =(mistery( n ) =(n - - n)) //5 with5 with
=(1 + =(1 + 5)/2 5)/2 [Golden ratio]
=(1 - =(1 - 5)/25)/2
following an algorithm.following an algorithm.
( ( I ) ) X X00 (O (O00) ) X X1 1 (O(O11) ) ...... X Xm m (O(Omm) ) ......
• If it stops in m steps the output is {OIf it stops in m steps the output is {O0,0, O O11 ... ... OOmm } }
• If it does not stop {OIf it does not stop {O0,0, O O11 ... ... OOmm … … }}
f :f : II O O { {}} with with I II , , O O OO , , XXii XX
I Of(X)
CorrectnessCorrectnessIs an algorithm correct for a problem?Is an algorithm correct for a problem?(Does it do a given function?)(Does it do a given function?)
Given a computational problem
f :f : II O O { {}}
and a algorithm
A .
The algorithm A is correct for the problem ff if the function that A calculates is ff.
Problem: Finding the minumun Problem: Finding the minumun element in an array of n realselement in an array of n reals
Input : T[1..n] : array of realInput : T[1..n] : array of real
Output : x Output : x min{T[i] | 1 min{T[i] | 1 i i n n}}
min : N x Rmin : N x RN R R
(n,T[1..n] T[1..n] ) ) min{T[i] | 1 min{T[i] | 1 i i n} n}
“In order to prove that a program or algorithm is correct, we need to show
that the program gives the correct answer for every possible input.”
function min( n , T[1..n] ) : function min( n , T[1..n] ) : realreal
x x T[1] T[1] for i for i 2 to n do2 to n do
if T[i] < x thenif T[i] < x then x x T[i] T[i]
end_forend_forreturn xreturn x
endend
function min( n , T[1..n] ) : realfunction min( n , T[1..n] ) : real if n =1 then if n =1 then return T[1]return T[1] else else return MIN(min(n-1, T[1..n-1] ),T[n])return MIN(min(n-1, T[1..n-1] ),T[n]) endend
function MIN(a,b ) : realfunction MIN(a,b ) : real if a < b then if a < b then return areturn a else else return breturn bendend
function min( n , T[1..n] ) : realfunction min( n , T[1..n] ) : real if n =1 then if n =1 then return T[1]return T[1] else if n=2 then else if n=2 then if T[1] < T[2] then if T[1] < T[2] then return T[1]return T[1] else else return T[2]return T[2] else else return min(2,[min(n-1, T[1..n-return min(2,[min(n-1, T[1..n-
1] ),T[n]])1] ),T[n]]) endend
Analysis of complexityAnalysis of complexity
How much resources are needed to run an algorithm as a function of the size of the problem it solves?
Is the study of the amount of resources:
• timetime, [ time complexity] t(n) t(n) ,
• memormemory, [ space complexity] s(n) s(n) ,
• …… [ other resources],
that an algorithm will require to process an input of size nn.
ComplexityComplexity
•Time complexity Time complexity t(n)t(n)
computational stepscomputational steps
•Space complexity Space complexity s(n)s(n)
storage - memorystorage - memory
AnalysisAnalysis
Size of a problem n Size of a problem n
A measure of the amount of data that an A measure of the amount of data that an
algorithm takes as input algorithm takes as input
Examples:
•Sorting
nn = number of elements in the input sequence
•Inverting a square matrix A
nn = number of elements in the matrix
•Finding the root of a function f(x)
nn = number of precise digits in the root
•Multiplying two integers
nn = number of bits needed to represent the input
•Finding the shortest path in graph, in this case the size of the input is described by two numbers rather than one.
nn = number of vertex
mm = number of edges
Best case Best case ttbb(n) (n) Minimum execution time for any input of size n
Worst case Worst case ttww(n) (n) Maximum execution time over all the inputs of size n
Average case Average case ttaa(n) (n) Average execution time over all the inputs of size n
Time Complexity Time Complexity t(n) t(n) Number of computational stepsNumber of computational steps
SortingSorting average time taken on the n! different average time taken on the n! different
ways of ways of initially ordering n elements (assuming initially ordering n elements (assuming
they arethey are all different)all different)
• Worst-case analysis is appropriate for an algorithm whose response time is critical
•In a situation where an algorithm is to be used many times on many different instances, it may be more important to know the average execution time on instances of time n
An operation whose execution time can be bounded above by a constant depending only on the particular implementation used (machine, programming language etc). In general we use RAM elementary instructions as the model.
Elementary OperationElementary Operation
Data types Data types Integer Integer
Up to values n uses c log n bits.
Floating point Floating point Bounded rage of values.
Elementary (primitive) instructions Elementary (primitive) instructions ArithmeticArithmetic
Add +, substrac -, multiply *, divide /, remainder %, floor , ceiling .
Data movement Data movement load, store, copy.
RAM ModelRAM ModelRandom Access ModelRandom Access Model
ControlControlConditional and unconditional branch, subroutine call, return.
Special exponentiation Special exponentiation 2k for k small k-shift left.
Each instructions takes a Each instructions takes a constant amount of timeconstant amount of time
Memory-hierarchy effects Memory-hierarchy effects main – cache – virtual main – cache – virtual
[demand paging] [demand paging] are considered only in special are considered only in special casescases
ExamplExamplee
function mistery( n function mistery( n )) i i 1 1 j j 0 0 for k for k 1 to n do1 to n do
j j i + j i + j i i j - i j - iend-for end-for return jreturn j
If the operations are elementary the algorithm takes a time equal to a+bn, for appropriate constants a and b, i.e.,
time in O(n).
We must attribute to the additions a cost proportional to the length of the operands
• For n=47 the addition “j i + j” causes arithmetic overflow on a 32-bit machine
• 45.496 bits are needed to hold the result corresponding to n=65.536
time in O(n2)
!!!But the operations are not elementary !!!
““Two different implementations of Two different implementations of the same algorithm will not differ in the same algorithm will not differ in
efficiency by more than some efficiency by more than some multiplicative constant”multiplicative constant”
Principle of InvariancePrinciple of Invariance
El orden de complejidad de un algoritmo
Es una expresion matematica que indica como los recursos
necesitados para correr un algoritmo se incrementan como
se incrementa el tamaño del problema (que el resuelve).
Orden de complejidad de un algoritmoOrden de complejidad de un algoritmo
Complejidad LinealComplejidad Lineal
La complejidad de un algoritmo se dice que es lineal si:
t(n)=a * n + bt(n)=a * n + b
Con a y b son constantes
function min( T[1..n] ) : realfunction min( T[1..n] ) : real x x T[1] T[1] /* b *//* b */
for i for i 2 to n do 2 to n do /* n times *//* n times */
if T[i] < x then if T[i] < x then /* a *//* a */ x x T[i] T[i]
end_forend_for
return xreturn x
ExamplExamplee
By the invariance principle we if we have two different implementations
(computer + language + compiler +code optimizations)
with worst case time complexity tw1(n) and tw2(n) then
tw1(n) = c tw2(n)
Worst case time complexity
tw(n) = a*n + b(n) = a*n + b