algorithms : introduction and analysis

Analysis and Design of Algorithms

2

Part 1 Basics of Algorithm

1. Introduction2. Characteristics3. Use of Algorithms


3

1. IntroductionI. The word Algorithm come from the name of 9th century Persian mathematician

Al-Khwārizmī, Whose work built upon the work of 7th century Indian mathematician Brahmagupta.

II. Ada Lovelace invented 1st Algorithm !

III. An Algorithm is any well-defined computational procedure that takes some value or set of values, as input and produces some value or set of values as output.

IV. The Internet enables people all around the world to quickly access and retrieve large amount of information. In order to do so, clever algorithms are employed to manage and manipulate this large volume of data. Example of that is, finding good routes on which the data will travel.


Analysis and Design of Algorithms 4

o If all Algorithms are performing same procedure, all of them are equal and if not then what differentiates them?

1. Correctness of Algorithm

I. Produce correct answer always.

II. If it is taking lot of time and resources, We build our algorithms as to give the best possible answer.

2. Efficiency of Algorithm

I. Time it takesII. ComplexityIII. Space requirement

5

2. Characteristics

• Modularity• Correctness• Maintainability• Functionality• Robustness

• User-friendliness• Simplicity• Programmer time• Extensibility• Reliability


6

3. Use of AlgorithmsI. Algorithms help us to understand scalability.

II. Public-key cryptography and digital signatures are among the core technologies used and are based on numerical algorithms.

III. Algorithmic mathematics provides a language for talking about program behaviour.

IV. Speed is fun!V. Local-area and wide-area networking.


7

Part 2 Analysis of Algorithms

1. Overview2. Time Base Analysis3. Space Base Analysis4. Three Cases


8

Figure : Types of analysis


Analysis

Time Complexity

Space Complexity

9

1. Time Complexity

I. In computer science, the time complexity of an algorithm quantifies the amount of time taken by an algorithm to run as a function of the length of the string representing the input.

II. The time complexity of an algorithm is commonly expressed using big O notation, which excludes coefficients and lower order terms.


10

2. Space Complexity

I. Space complexity is a measure of the amount of working storage an algorithm needs. That means how much memory, in the worst case, is needed at any point in the algorithm.

II. As with time complexity, we're mostly concerned with how the space needs grow, in big-Oh terms, as the size N of the input problem grows.



To find Time Complexity of a program Cost sum = 0; c1

for(i=0; i<N; i++) c2

for(j=0; j<N; j++) c2 sum += arr[i][j]; c3 ------------c1 + c2 x (N+1) + c2 x N x (N+1) + c3 x N2

12

Three Cases

1. Best Case2. Average Case3. Worst Case


13

1. Best case

I. Cheat with a slow algorithm that works fast on some input

II. Provides a lower bound on running timeIII. Input is the one for which the algorithm runs the

fastest


Lower Bound Running Time Upper Bound

14

2. Average caseI. T(n) = expected time of algorithm over all inputs of

size n.II. Need assumption of statistical distribution of inputs.III. Provides a prediction about the running time.IV. Assumes that the input is random.


15

3. Worst caseI. T(n) = maximum time of algorithm on any input of

size n.II. Provides an upper bound on running time.III. An absolute guarantee that the algorithm would not

run longer, no matter what the inputs are.



Analysis of INSERTION sort

1]1[8

)1(17

)1(][]1[6

][05

11410]11[ sequence

sorted theinto][Insert 31][2

][21timescost SORT(A)-INSERTION

8

27

26

25

4

2

1

nckeyiA

tcii

tciAiA

tckeyiAandi

ncjinjA

jAncjAkey

ncAlengthj

nj j

nj j

nj j

do

while

do

tofor



I. Total Running Time

II. Best Case: The array is already sorted (tj =1 for j=2,3, ...,n)

)1()1()1()(2

62

5421

n

jj

n

jj tctcncnccnT ).1()1( 8

27

nctc

n

jj

)1()1()1()1()( 85421 ncncncncncnT

).()( 854285421 ccccnccccc



• Worst Case: The array is reverse sorted (tj =j for j=2,3, ...,n)

)12/)1(()1()( 521 nncncncnT)1()2/)1(()2/)1(( 876 ncnncnnc

ncccccccnccc )2/2/2/()2/2/2/( 87654212

765

2)1(

1

nnjn

j

cbnannT 2)(

19

Part 3 Asymptotic Notation

1. Big-O Notation - ⃝2. Theta Notation - Ө3. Omega Notation - Ω4. Example


20Analysis and Design of Algorithms

1. Big-O Notation

For a given function g(n), we denote by O(g(n)) the set of functions.

The above statement implies that if, f(n)=O(g(n)) then F(n) is always

We use the O-notation to give an asymptotic upper bound of a function, to within a constant factor.

)(ncg

0

0

allfor )()(0s.t.and constants positiveexist there:)(

))((nnncgnf

ncnfngO

22

2. Ω-Omega NotationI. For a given function , ,we denote by the

set of functions.

II. The above statement implies that, if f(n)=O(g(n)) then F(n) is always

III. We use Ω-notation to give an asymptotic lower bound on a function, to within a constant factor.


)(ng ))(( ng

)(ncg

0

0

allfor )()(0s.t.and constants positiveexist there:)(

))((nnnfncg

ncnfng

24

3. Ө-Theta Notation• For a given function , we denote by the set of

functions

• A function belongs to the set if there exist positive constants c1 and c2 such that it can be “sand- wiched” between and • The above statement implies that, if f(n)=O(g(n)) then F(n) is

always Analysis and Design of Algorithms

)(ng ))(( ng

021

021

allfor )()()(c0s.t.and,, constants positiveexist there:)(

))((nnngcnfng

nccnfng

)(nf ))(( ng)(1 ngc)(2 ngc

)()()( 21 ngcnfngc


Examples (Big-O)•30n+8=O(n).•30n+8 cn, n>n0 .

• Let c=31, n0=8. Assume n>n0=8. Thencn = 31n = 30n + n > 30n+8, so 30n+8 < cn.


• Note 30n+8 isn’tless than nanywhere (n>0).• But it is less than

31n everywhere tothe right of n=8.

n>n0=8

Big-O example, graphically

Increasing n Va

lue

of fu

nctio

n

n

30n+8cn =31n

30n+8O(n)


• Subset relations between order-of-growth sets.

Relations Between Different Sets

RR( f )O( f )

( f )• f

29

Thank you.


algorithms : introduction and analysis

Engineering