cs 221

CS 221

Analysis of AlgorithmsInstructor: Don McLaughlin

Algorithms

What are they and Why do we care?

Algorithms

“A well defined computational procedure that takes some value or set of values, as input and produces some value or set of values as output”

“A sequence of computational steps that transforms input to the output”

From Cormen, Leiserson, Rivest and Stein

Algorithms

“Algorithms + Data Structures = Programs”

Niklaus Wirth, Swiss Computer Scientist

Or, perhaps- Programs = Programming_language(Algoritms +

Data Structures)

Algorithms

So you might think of an algorithm as the equation behind a function

y=f(x)

Just as an equation is a computational procedure for creating a certain output from a certain input

Algorithms “A sequence of unambiguous instructions for

solving a problem, i.e. for obtaining a required output for any legitimate input in a finite amount of time” From Levitin, 2007

Algorithms

So, why do we want to study them? Because there may be many algorithms

intended to solve a problem… …, some are correct and some are not

correct …and of the correct ones some are

better than others (as you will see)

Algorithms

We are largely going to be concerned about the efficiency of algorithms What do we mean by efficiency?

Memory

Time

Algorithms

Mostly we are going to explore the efficiency of algorithms in terms of run time (T)

In particular we are going to study the growth of algorithms in relation to the size of the problem

Algorithms

But, how? Theoretically, mathematically Empirically Visually (algorithm visualization)

Theoretical Analysis of Algorithms

To do this we need a few things- A computational model (what is the

computer) A language for expressing the algorithm A metric and methodology for measuring

the performance and efficiency of an algorithm


RAM – Random Access Machine Model essentially Von Neumann architecture A CPU… …with a fixed and finite set of primitive

instructions …which run in a fixed amount of time …connected to a bank of memory… And the CPU can read or write any

memory location in one primitive operation


RAM – Random Access Machine Model Primitive operations might be

Assign a value to variable (memory location)

Call a method or subroutine Arithmetic operation Compare two values Access elements of an array via index Return from method Return from algorithm


Language for expressing an algorithm Programming Language

C, VB, Java,… Flowcharts


Language for expressing an algorithm Pseudocode

Natural language expression of algorithm Really mix of natural language and

language from programming language constructs

Intended for human consumption, not computers

Theoretical Analysis of Algorithms Pseudo-code – has some rules

Algorithm declaration – must name algorithm and its parameters

Must define range of input, and domain of output Expressions – use <- for assignment, = for logical

comparison Scope denoted by indentation Decision structures

If – Then – Else While – Do loops Repeat – until loops For loops

Method calls Method returns (including return from algorithm)

Theoretical Analysis of Algorithms Pseudo-code – an example arrayMax – find the largest value in an array of

values

Algorithm arrayMax(A, n)Input:An array A storing n >= 1 integers)Output: the maximum value in the array AcurrentMax <- A[0]for i <-1 to n – 1 do

if currentMax < A[i] thencurrentMax <- A[i]

return currentMax


Insertion-Sort

Algorithm Insertion-sort(A, n)Input:An array A storing n >= 1 integers)Output: the array A sorted such that A0 < A1 < A2 < …An

for j <- 2 to ndo key <- A[j]#insert A[j] into the sorted sequencei <- j – 1while i > 0 and A[j] > key do

A[j +1] <- A[j]i <- i -1

A[i + 1] <- keyreturn A


So now the question is - how do we determine the run-time T of these algorithms?

Keep in mind that we need to understand the run-time of algorithms in the context of the size of the problem …which we will express as n

So we are looking for T(n) … for any given algorithm


So how are we going to do this?

Counting

Theoretical Analysis of AlgorithmsPrimative Cost

Algorithm arrayMax(A, n) 0

Input:An array A storing n >= 1 integers) 0

Output: the maximum value in the array A 0

currentMax <- A[0] 2

for i <- 1 to n – 1 do (init loop counter) (check loop boundary) (increment loop counter)

112

if currentMax < A[i] then 2

currentMax <- A[i] 2

return currentMax 1

Theoretical Analysis of AlgorithmsPrimitive Cost




currentMax <- A[0] 2

for i <- 1 to n – 1 do (init loop counter) (check loop boundary) (increment loop counter)

112

1nn-1

if currentMax < A[i] then 2 n-1

currentMax <- A[i] 2 0(n-1)

return currentMax 1 1


So T(n) for arrayMax is

T(n) = 2 + 1 + n + 4(n-1) + 1 = 5n

T(n) = 2 + 1 + n + 6(n-1) + 1 = 7n-2


So T(10) for arrayMax is

At least 5(10) = 50

Or

No worse than 7(10)-2 = 68


Primitive Cost

Algorithm arrayMax(A, n) 0 Input:An array A storing n >= 1 integers) 0

Output: the maximum value in the array A 0 currentMax <- A[0] C1

for i <- 1 to n – 1 do C2

if currentMax < A[i] then C3

currentMax <- A[i] C4

return currentMax C5


Primitive Cost




currentMax <- A[0] C1

for i <- 1 to n – 1 do C2

if currentMax < A[i] then C3

currentMax <- A[i] C4

return currentMax C5


Primitive Cost




currentMax <- A[0] C1 1

for i <- 1 to n – 1 do C2 n-1

if currentMax < A[i] then C3 n-1

currentMax <- A[i] C4 n-1

return arrayMax C5 1





currentMax <- A[0] C1

for i <- 1 to n – 1 do C2 n-1



return arrayMax C5 1

So, what is the run-time T(n) for the algorithm arrayMax?

T(n)

T(n) = c1 + c2(n-1) + c3(n-1) + c4(n-1) + 1

T(n) = c1 + (n-1)(c2 + c3 +c4) + 1


Was that every-case, best-case, worst-case?

Why?

Theoretical Analysis of AlgorithmsPrimitive Cost




currentMax <- A[0] C1 1

for i <- 1 to n – 1 do C2 n-1



return A C5 1

So, what would T(n) be for best-case?

T(n) = c1 + c2(n-1) + c3(n-1) + 1c5

T(n) = c1 + (n-1)(c2 + c3) + 1c5


Insertion-Sort

Algorithm Insertion-sort(A, n)Input:An array A storing n >= 1 integers)Output: the array A sorted such that A0 < A1 < A2 < …An

for j <- 2 to ndo key <- A[j]#insert A[j] into the sorted sequencei <- j – 1while i > 0 and A[j] > key do

A[j +1] <- A[j]i <- i -1

A[I + 1] <- keyreturn A


Algorithm Insertion-Sort(A, n) 0


Output: array A sorted such that a0<a1<…an 0

for j <- 2 to n C1

do key <- A[j] C2

#insert A[j] into the sorted sequence 0

i <- j – 1 C4

while i > 0 and A[j] > key do C5

A[j +1] <- A[j]C6

i <- i -1C7

A[I + 1] <- keyC8

return A 1





for j <- 2 to n C1 n

do key <- A[j] C2 n-1

#insert A[j] into the sorted sequence 0 n-1

i <- j – 1 C4 n-1

while i > 0 and A[j] > key do C5 Σ(tj)

A[i +1] <- A[i]c6 Σ(tj-1)

i <- i -1C7 Σ(tj-1)

A[i + 1] <- keyC8 n-1

return A


T(n) for Insertion-Sort

T(n) = c1n + c2(n-1) + c4(n-1) + C5Σ(tj) + c6Σ(tj-1) + c7Σ(tj-1) + c8(n-1)

So… what is the best-case scenario (shortest run-time)?



Ok, that would mean what in terms of the execution of the algorithm?

… that tj = 1 for all j

…therefore…… Σ(tj) = n – 1

…and c6 and c7 drop out



So, T(n) best-case is

T(n) = c1n + c2(n-1) + c4(n-1) + c5(n-1) + c8(n-1)

= c1n + (c2 + c4 + c5 + c8) (n-1)

= (c1+c2+ c4+c5+c8)n-(c2+c4+c5+c8)





for j <- 2 to n C1 n

do key <- A[j] C2 n-1

#insert A[j] into the sorted sequence 0 n-1

i <- j – 1 C4 n-1

while i > 0 and A[j] > key do C5 Σ(tj)

A[i +1] <- A[i]c6 Σ(tj-1)

i <- i -1C7 Σ(tj-1)

A[i + 1] <- keyC8 n-1

return A



T(n) = c1n + c2(n-1) + c4(n-1) + C5Σ(tj) + c6Σ(tj-1) + c7Σ(tj-1) + c8(n-1)

So… what is the worst-case scenario (longest run-time)?


By the way - rules of thumb (for now)

Σnj=2j = n(n+1)/2 - 1

Σnj=2(j-1) = n(n-1)/2


So, the worst-case

T(n) = c1n + c2(n-1) + c4(n-1) + C5(n(n+1)/2-1) + c6(n(n-1)/2) + c7(n(n-1)/2) + c8(n-1)

cs 221

Documents

computera language

maximum value

largest value

computational model

range of input

legitimate input

efficiency of algorithmswhat

certain output