8/10/2019 Daaa 1 Introduction

1/33

Dr. Md. Rafiqul Islam INTRODUCTION TO ALGORITHMS Slide #

Introduction

Prof. Dr. Md. Rafiqul Islam

Chapter 1

8/10/2019 Daaa 1 Introduction

2/33

DATA STRUCTURE FUNDAMENTALS D RAFIQUL ISLAM 2

Basic goals for an algorithm

always correctalways terminates

very good performance

8/10/2019 Daaa 1 Introduction

3/33

Slides# 3

What will we do?

Design and Analysis of Algorithms

Des ign : Develop or write algorithm which gives correctoutput and minimize the cost.

Analys i s : show or predict the cost of an algorithm interms of resources and performance.

8/10/2019 Daaa 1 Introduction

4/33

DATA STRUCTURE FUNDAMENTALS D RAFIQUL ISLAM 4

Definition of Algorithm

It is a sequence of steps (instructions) that canbe followed to solve a problem (to perform atask).An algorithm takes some values as input andproduces some values as output .An algorithm is thus a sequence ofcomputational steps that transform the input

into the output.

8/10/2019 Daaa 1 Introduction

5/33

DATA STRUCTURE FUNDAMENTALS D RAFIQUL ISLAM 5

General Section of Algorithm

Usually each and every algorithm has threesections .Input section

where we show which data elements to be given.Operational or processing section

This is main section and h ere we have to do all necessaryoperations, such as computation, taking decision, callingother procedure (algorithm) etc.

Output sectionwhere we display the result found from the secondsection.

To write an algorithm we do not strictly follow the grammar ofany particular programming language. However its language maybe near to a programming language.

8/10/2019 Daaa 1 Introduction

6/33

DATA STRUCTURE FUNDAMENTALS D RAFIQUL ISLAM 6

Study areas of algorithm

The study of algorithm includes the following areas:

Devise algorithmThere are several techniques to devise algorithms, such asdivide and conquer, dynamic programming, greedy and back

tracking methods and so on.By the acquiring knowledge about the above mentionedmethods we will be able to devise algorithm.

Validation of algorithmAfter devising the algorithm it is necessary to show that it gives

the correct output for all possible correct input. This process isreferred as algorithm validation.If the algorithm is simple one, the validation can be verifiedusing paper pencil.Otherwise a program should be written to check the validation.

8/10/2019 Daaa 1 Introduction

7/33

DATA STRUCTURE FUNDAMENTALS D RAFIQUL ISLAM 7

Study area [contunued]

Analysis of algorithmWhen an algorithm is executed using a program ituses the Central Processing Unit (CPU) to performnecessary operations and memory to hold theprogram and data.

Analysis of algorithm or performance analysis refersto the task determining how much computationaltime and storage (memory) are required for analgorithm.We can compare performance of several algorithmsfor the same problem by analysis of the algorithms.This is a challenging area and required mathematicalskill.

8/10/2019 Daaa 1 Introduction

8/33

DATA STRUCTURE FUNDAMENTALS D RAFIQUL ISLAM 8

Complexity of algorithm

Time complexityThis complexity is related to execution(computational) time of the algorithm.It depends on the number of element (item)comparisons, number of element movements(movement of data from one place to another) andsome other computational operations such asaddition, subtraction, multiplication, division etc.However, the complexity of an algorithm is generallydominated by a single operation such as, the numberof element comparisons , the number ofmultiplications, the number of additions andsubtractions etc.

8/10/2019 Daaa 1 Introduction

9/33

DATA STRUCTURE FUNDAMENTALS D RAFIQUL ISLAM 9

Complexity [continued]

Space complexityThis complexity is related to space (memory) needsfor the data (data structure) used in the algorithm.If there n data items used in an algorithm, the space

complexity of the algorithm will be proportional ton .

8/10/2019 Daaa 1 Introduction

10/33

DATA STRUCTURE FUNDAMENTALS D RAFIQUL ISLAM 10

Cases for analysis of algorithm

For analysis of algorithm three cases may beconsidered. They are:

Best case

Average case

Worst case

8/10/2019 Daaa 1 Introduction

11/33

DATA STRUCTURE FUNDAMENTALS D RAFIQUL ISLAM 11

Cases of algorithm analysis

Best caseIn this case for a given input the algorithm executesminimum of steps to produce the desired output.Execution time is minimum in best case .

As for example, in a sorting problem if the datawas already stored in the target order (ascending ordescending order), then this is best case for thesorting problem.

8/10/2019 Daaa 1 Introduction

12/33

DATA STRUCTURE FUNDAMENTALS D RAFIQUL ISLAM 12

Cases of algorithm analysis

Average caseHere for a given input the algorithm uses theaverage number of steps to produce the desiredoutput.

Execution time is in between minimum andmaximum.For example in case of sorting problem the randominput may be considered as average case.

8/10/2019 Daaa 1 Introduction

13/33

8/10/2019 Daaa 1 Introduction

14/33

DATA STRUCTURE FUNDAMENTALS D RAFIQUL ISLAM 14

Best, Average and Worst Cases

For inputs of all sizes:

1n

2n

3n

4n

5n

6n

Input instance size

R u n n i n g

t i m e

1 2 3 4 5 6 7 8 9 10 11 12 ..

best-case

average-case

worst-case

8/10/2019 Daaa 1 Introduction

15/33

DATA STRUCTURE FUNDAMENTALS D RAFIQUL ISLAM 15

Example of of an Algorithm

Insertion SortIn this method we take the data of second position firstand compare it with the data of first position .If the data in the second position is smaller than thedata in the first position, then we shift the data in firstposition to the right (to the second position) and insertthe data of second position in the first position.Otherwise the data will remain in their own positions.After that we take data in the third position andcompare it with the data in the second position.

8/10/2019 Daaa 1 Introduction

16/33

8/10/2019 Daaa 1 Introduction

17/33

DATA STRUCTURE FUNDAMENTALS D RAFIQUL ISLAM 17

Insertion Sort

Using steps we can describe the process asfollows.1. Given a list of elements (data).2. We have to insert a data into its correct position by

moving all data (before it) to the right (that aregreater than the data which is being considered atthis moment).

3. By repeating Step-2 for all considerable data we

can arrange the whole list in ascending order.

8/10/2019 Daaa 1 Introduction

18/33

DATA STRUCTURE FUNDAMENTALS D RAFIQUL ISLAM 18

Pictorial view of insertion sort

Shift 17 to right.

18

17 12 18 5 7 108

517 1812 7 108

5 7 8 10 12 17

517 1812 7 108

Is 12 < 17? Yes.

Is 18 < {17, 12} ? No.

Is 5 < {18, 17, 12} ? Yes.

1812 175 7 108

Shift {12, 17, 18} to right.

177 125 18 108

Is 7 < {18, 17, 12, 5} ?7 < {18, 17, 12} and 7> 5. Shift {12, 17, 18} to right.

Insert 7 after 5 (at blank space).

Sorted List.

8/10/2019 Daaa 1 Introduction

19/33

19

Example of insertion sort

8 2 4 9 3 6

8/10/2019 Daaa 1 Introduction

20/33

20

Example of insertion sort

8 2 4 9 3 6

8/10/2019 Daaa 1 Introduction

21/33

21

Example of insertion sort

8 2 4 9 3 6

2 8 4 9 3 6

8/10/2019 Daaa 1 Introduction

22/33

22

Example of insertion sort

8 2 4 9 3 6

2 8 4 9 3 6

8/10/2019 Daaa 1 Introduction

23/33

23

Example of insertion sort

8 2 4 9 3 6

2 8 4 9 3 6

2 4 8 9 3 6

8/10/2019 Daaa 1 Introduction

24/33

24

Example of insertion sort

8 2 4 9 3 6

2 8 4 9 3 6

2 4 8 9 3 6

8/10/2019 Daaa 1 Introduction

25/33

8/10/2019 Daaa 1 Introduction

26/33

26

Example of insertion sort

8 2 4 9 3 6

2 8 4 9 3 6

2 4 8 9 3 6

2 4 8 9 3 6

8/10/2019 Daaa 1 Introduction

27/33

27

Example of insertion sort

8 2 4 9 3 6

2 8 4 9 3 6

2 4 8 9 3 6

2 4 8 9 3 6

2 3 4 8 9 6

8/10/2019 Daaa 1 Introduction

28/33

28

Example of insertion sort

8 2 4 9 3 6

2 8 4 9 3 6

2 4 8 9 3 6

2 4 8 9 3 6

2 3 4 8 9 6

8/10/2019 Daaa 1 Introduction

29/33

29

Example of insertion sort

8 2 4 9 3 6

2 8 4 9 3 6

2 4 8 9 3 6

2 4 8 9 3 6

2 3 4 8 9 6

2 3 4 6 8 9 done

8/10/2019 Daaa 1 Introduction

30/33

DATA STRUCTURE FUNDAMENTALS D RAFIQUL ISLAM 30

Algorithm for insertion sorting

Algorithm 9.4: Algorithm for insertion sorting 1. Input Array A[1 n]2. (i). for ( j = 2 to n)

{key-value = A [ j];i = ( j-1);

(ii). while ( i > 0 and A[ i] > key-value ){A[ i + 1] A[ i];i = i -1;} // end of while

A[ i + 1] key-value ;} //end of for

3. Output: sorted list.

8/10/2019 Daaa 1 Introduction

31/33

DATA STRUCTURE FUNDAMENTALS D RAFIQUL ISLAM 31

Insertion sort

Time complexityo. of element comparisons = 1 + 2 + 3 + . + (n -1)

nnnn

21

21

2)1( 2

Therefore, complexity = O(n 2 ). This is upper bound of the complexity.

The space complexity of this sorting problem is O (n). That means, the space for the dateused in the algorithm is proportional to n . Such as, the required space is n bytes or 2n or4n bytes etc.

8/10/2019 Daaa 1 Introduction

32/33

DATA STRUCTURE FUNDAMENTALS D RAFIQUL ISLAM 32

References

1. Introduction to algorithms, Third editionThomas H. Corman, Charles E. Leiserson, Ronald L.

Rivest .

2. Fundamentals of computer algorithmEllis Horowitz, Sartaj Sahni and Rajasekaran.

3. Data Structure Fundamentals, 2 nd EditionMd. Rafiqul Islam and M. A. Mottalib.

8/10/2019 Daaa 1 Introduction

33/33

DATA STRUCTURE FUNDAMENTALSD RAFIQUL ISLAM 33

This is the end of chapter 1

THANK YOU.