cs 615: design & analysis of algorithms chapter 2: efficiency of algorithms

CS 615: Design & Analysis of

Algorithms

Chapter 2: Efficiency of Algorithms

April 21, 2023 CS 615 Design & Analysis of Algorithms 2

Course Content

1. Introduction, Algorithmic Notation and Flowcharts (Brassard & Bratley, Chap. Chapter 3)

2.2. Efficiency of Algorithms (Brassard & Efficiency of Algorithms (Brassard & BratleyBratley,, Chap. 2) Chap. 2)

3. Basic Data Structures (Brassard & Bratley, Chap. 5)

4. Sorting (Weiss, Chap. 7)5. Searching (Brassard & Bratley Chap.: 9)6. Graph Algorithms (Weiss, Chap.: 9)7. Randomized Algorithms (Weiss, Chap.: 10)8. String Searching (Sedgewick, Chap. 19)9. NP Completeness (Sedgewick, Chap. 40)


DefinitionsProblem:

A situation to be solved by an algorithmExample:

Multiply two integers

InstanceA special case of the problemExample:

Multiply(981, 1234)

An algorithm must work correctly On every instance of the problem it claims to solve

To prove an algorithm is not correctFind an instance which the algorithm cannot solve correctly


Efficiency of Algorithms

To decide which algorithm to choose:

Empirical ApproachProgram the competing algorithmsTry them on different instances

with the help of the computer(s)

Resources:Computing timeStorage spaceNumber of processes (for parallel algorithms)



To decide which algorithm to choose:

Theoritical ApproachUsing formal methods to anlyze the efficiencyDoes not depend on the computerNo need to make “programming”



To decide which algorithm to choose

Hybrid ApproachDescribe algorithm’s efficiency function theoritically

Empirically determine numerical parameters for a particular machine

Predict the time an actual implementation will take to solve an instance


Principle of Invariance

Two different implementations of an algorithm

Will not differ in efficiency by more than some multiplicative constant

ExampleIf the constant is 5:if the first implementation

takes 1 second to solve an instance

then a second implementation (possible on a different machine)

Will not take more than 5 seconds


Principle of Invariance

For an instance of size nImplementation 1:

Takes time of t1(n)

Implementation 2:Takes time of t2(n)

There always exist positive constants c and d such that

t1(n)c*t2(n) and

t2(n)d *t1(n)

where n is sufficiently large


Results of Principle of Invarience1. A change on the implementation of the

same algorithm 1. Can only cause a constant of change in

efficiency

2. The principle does not depend onThe computer we useThe compiler we implementThe abilities of the person making the coding

3. If we want a radical change in efficiencyWe need to change the algorithm itself


Theoritical EfficiencyFor a given function t

an algorithm for some problem takes a time in the order of t(n),

if there exist a positive constant c the algorithm is capable of

solving every instance of size

thenn is not more than

c*t(n) seconds/hours/years.

For numerical problems n may sometimes be the value rather than the size of the instance


Algorithm TypesTime takes to solve an instance of a

Linear Algorithm isNever greater than c*n

Quadratic Algorithm isNever greater than c*n2

Cubic Algorithm isNever greater than c*n3

Polynomial Algorithm isNever greater than nk

Exponential Algorithm isNever greater than cn

where c & k are appropriate constants


Worst Case AnalysisWhen to analyse an algorithm

Considering the cases only take maximum amount of time

If the algorithm is capable of solving cases in t(n)

then the worst case should not be greater thanc*t(n)

Useful if the algorithm is to be applied to casesthe upper bound of an algorithm must be known

Example: Response time for a nuclear power plant.


Average Time Analysis

If the algorithm is going to be used many times

it is useful to know the average execution time

on instances of size n

It is harder to analyse the average casethe distribution of data should be known

Insertion sorting average time isin the order of n2


Elementary OperationIs one whose execution time is

bounded above by a constantThe constant does not depend on

The sizeor other parameters of the instance considered

Examplex=y+w*z is it elementary operation?Suppose

ta : time to execute an addition (constant)tm : time to execute a multiplication (constant)ts : time to execute an assignment (constant)

t: time required to execute an addition, multiplication, & asignment:

ta ta + m tm + ts s where a,m,s are constantst max(ta , tm , ts ) x (a+m+s)


Elementary Operation

A single line of of programmay correspond to a variable number of elemenary operations

x=min{T[i] | 1 i n}Time required to compute min

increases with n

min() is not an elementary operation !


Elementary Operation

addition, multiplication:Normally

Time required to computeaddition, multiplication

Depends on the length of the operands

Butit is reasonable to assume

addition and multiplication are elementary operations

when the operands are in fixed length


Some Algorithm Examples

Calculating DeterminantsSortingMultiplication of Large IntegersCalculating the Greatest Common DivisorCalculating Fibonacci SequencesFourier Transforms


Calculating DeterminantsRecursive definition of Algorithm

To compute a determinant of n x n matrixTakes time proportional to n!Worse than taking exponential timeExperiments:

5 x 5 matrix 20 sec.10 x 10 matrix 10 min.

Estimation20 x 20 matrix 10 million years.

Gauss-Jordan EliminationTo compute a determinant of n x n matrixTakes time proportional to n3

Experiments:10 x 10 matrix 0.01 sec.20 x 20 matrix 0.05 sec.100 x 100 matrix 5.5 sec.


SortingArranging n objects based on

the “ordering function” defined for these objects

No sorting algorithm is faster than order of nlogn

Insertion sortingTakes time proportional to n2

Experiment: Sorting 1000 elements takes 3 sec.

Estimation: Sorting 100 000 elements would take 9.5 hrs.

Selection sortingTakes time proportional to n2


SortingHeapsort

Takes time proportional to nlogneven in worst cases

MergesortTakes time proportional to nlogn

even in worst cases

QuicksortTakes time proportional to nlognExperiment:

Sorting 1000 elements takes 0.2 sec.Sorting 100 000 elements takes 30 sec.


Multiplication of Large IntegersWhen multiplying large integers

Operands might become too large to hold in a single word

Assume two large integers of sizes m and n are to be multipliedMultiply each digit of one integer

by the digit of the second digit

Takes time proportional to m x n

More efficient algorithms:Divide-and-conquer:

Takes time proportional to n x mlg(3/2)

=n x m0.59

m is the size of the smaller integer


Calculating the Greatest Common Divisor

Denoted by gcd(m,n)Finding the largest integer

divides both m and n exactly

gcd(6,15)=3gcd(10,21)=1

gcd algorithm takes time of order nEuclid’s algorithm takes order of logn

function gcd(m,n)i=min(m,n)+1repeat

i=i-1until i divides both m and n exactlyreturn i

function Euclid(m,n)while m>0 do

t=mm=n mod mn=t

return n


Calculating Fibonacci Sequences

Fibonacci Sequence:f0=0;

f1=1;

fn= fn-1 + fn-2

Order of Fibrec is fn

Order of Fibiter is n

function Fibrec(n)if n<2 then return nelse return Fibrec(n-1)+Fibrec(n-2)

function Fibiter(n)i=1j=0for k=1 to n do

j=i+ji=j-i

return j

n 10 20 30 50 100

Fibrec 8ms 1sec 2min 21days

109years

Fibiter 0.17ms

0.33ms 0.5ms 0.75ms

1.5ms


Fourier TransformsOne of the most useful algorithm in historyUsed in

OpticsAcousticsQuantum physicsTelecommunicationsSystem theorySignal processingSpeech processing

ExampleUsed to analyze data from

earthquake in Alaska 1964Classic algorithm takes 26 minutes of computationA new algorithm need less than 2.5 seconds


End of Chapter 3Efficiency of Algorithms

cs 615: design & analysis of algorithms chapter 2: efficiency of algorithms

Documents