1

Description: A detailed discussion about algorithms and their measures, and understanding sorting.

Duration: 90 minutesStarts at: Saturday 11th May 2013, 11:00AM

-by Dharmendra Prasad

Data Structures and Algorithms – Sorting Techniques

2

Table of Contents

1. A detailed talk on algorithms.2. Measuring the effectiveness of an algorithm.3. Sorting Algorithms (In memory)4. Some real world sorting scenarios.

Data Structures and Algorithms – Sorting Techniques

3

Algorithm:

1. A step by step procedure to solve a given problem.2. It takes an input and applies the sequence of steps to

arrive on a desired output.3. Represented as a Pseudo Codes, which are English like

statements.

Data Structures and Algorithms – Sorting Techniques

4

Analysis of Algorithms:

1. The theoretical study of computer program performance and resource usage.

2. Things more important than the performance and resource usage are:

1. Correctness2. Simplicity3. Features4. User friendliness5. Maintainability6. Security7. Algorithms

Data Structures and Algorithms – Sorting Techniques

5

Why study performance if other things are more important than performance?

Performance measures the line between the feasible or in feasible: E.g. : If something takes un limited time, it is not useful to us. If it takes a lot of space which is more than available , it is infeasible. Real time constraints, need the result on time else the result is not useful.

We are the decision makers when have to choose between performance and other factors like user experience, maintainability etc.

Analyzing an algorithm means studying its performance in terms of running time, space required, etc.

Data Structures and Algorithms – Sorting Techniques

6

Sorting: The classic problem

Problem definition: Input: We have a sequence <a1, a2, … , an> of numbers.Output expected: A permutation <a1’, a2; …, an’> such that a1’ <= a2’ <= a3’ … <=an’

Basic definition: Take a bunch of numbers and put them in order. Order necessarily doesn’t mean increasing order. It can be ascending/descending etc.

Data Structures and Algorithms – Sorting Techniques

7

Sorting: The classic problem

Pseudo Code:

Insertion-Sort(A, n) // sorts A[1..n] for j <- 2 to n

do key <- A[j]; i <- j-1 while i > 0 and A[i] > key

do A[i+1] <- A[i] i = i -1

A[i+1] = key

Data Structures and Algorithms – Sorting Techniques

A->

sorted

J

key

8

Sorting: The classic problem

Example:

8 2 4 9 3 6

2 8 4 9 3 6

2 4 8 9 3 6

2 4 8 9 3 6

2 3 4 8 9 6

2 3 4 6 8 9

Data Structures and Algorithms – Sorting Techniques

9

Analyzing the above algorithm:

Running Time:• Depends on the input (e.g. if it is already sorted)• Depends on input size ( e.g. 10 elements vs. 10,000,000)

If the input is already sorted, Insertion Sort has nothing much to do.( This is called the best case)

If the input is reverse sorted, Insertion sort will shuffle in each step and takes the maximum possible time. ( This is called the worst case)

If the input is random that will be called as the average case.

Time taken in the above algorithm = Sum of the work inside the outer loop

T(n) =∑ f( j ) which is proportional to j²You can also say as T(n) = θ(n²)

Can we say that insertion sort is fast?

Data Structures and Algorithms – Sorting Techniques

10

Merge Sort:Merge Sort A[1 .. n] Step 1: if n = 1, done; Step 2: Recursively sort A[1..(n/2)] and A[n/2 +1,

n] Step 3: Merge both the sorted lists.

Merge A, B

Data Structures and Algorithms – Sorting Techniques

2 7 13 20 1 9 11 12A B

1 2 7 9 11 12 13 20

11

How much time merging took ?

It just took n steps or we can say the work done is proportional to n , hence the time taken will be:

T(n) proportional to n where n is the total number of elements in the sorted array.

T(n) = θ(n)

Analyzing merge sort:Step 1: How much this step takes?

Constant time, you just need to find the middle index of the array. This means it is not dependent on the input size.

Also called θ(1)

Step 2: How much this step takes?Time taken in sorting A[1, n/2] + Time taken in sorting A[n/2+1, n]

Lets say that each of the two steps take T[n/2] time.

Data Structures and Algorithms – Sorting Techniques

12

Step 3: The merge step took θ(n).

So the total time taken by the merge sort is a sum of all the three steps. That can be defined as below:

θ (1) , if n =1T(n) = 2T(n/2) + θ (n) if n > 1

Hence, T(n) = 2T(n/2) + c.n, where c > 0

Data Structures and Algorithms – Sorting Techniques

13

Data Structures and Algorithms – Sorting Techniques

cn

T(n/2) T(n/2)

cn

cn/2 cn/2

T(n/4) T(n/4) T(n/4) T(n/4)

14

Height of the above tree is : log nT(n) = cn (log n) + something proportional to nT(n) = cn (log n) + θ (n)

Hence T(n) = θ (n log n)

Comparing Insertion Sort and Merge Sort

Insertion sort time T(n) = θ(n²)Merge sort time T’(n) = θ (n log n)

Which one do you think is more time?

Data Structures and Algorithms – Sorting Techniques

15

Real life usage of sorting algorithm:

1) A cable operators personalized service.2) Alphabetical arrangement of list of items on a shopping

website.3) Offering admissions to applicants with highest grades.

And many more…

Data Structures and Algorithms – Sorting Techniques

16

Question &

Answers

Data Structures and Algorithms – Sorting Techniques