algorithms and their applications cs2004 (2012-2013) professor jasna kuljis (adapted from dr steve...

Algorithms and their ApplicationsCS2004 (2012-2013)

Professor Jasna Kuljis

(adapted from Dr Steve Swift)

6.1 Classic Algorithms - Sorting

Classic Algorithms - Sorting

Previously On CS2004…

So far we have looked at: Concepts of Computation and

Algorithms Comparing algorithms Some mathematical foundation The Big-Oh notation Computational Complexity Last week we also looked at data

structures

Slide 2


A Note on Laboratory Worksheets

￭ You should all have done the mathematics test at least once now

￭ You should by now have worksheet four (4) signed off

￭ Remember not to let these worksheets “pile up”

Do not leave any sheet longer than 4 weeks

Slide 3



￭ Within this lecture we are going to look at three sorting algorithms in detail

Bubble Sort Quick Sort Radix Sort

￭ We will then look briefly at other sorting algorithms

Slide 4


A Note of Growth Rates

Slide 5

A plot of n against a number of possible T(n) The y axis is in logarithmic scale so that all the data can be

viewedWe are using log base 10, but usually log base 2 is used

When n = 100log(n) = 2, n = 100, nlog(n) = 200, n2 = 10,000 and n3 = 1,000,000


Why Bother With Sorting?￭ Sorting applications are one of the most

common algorithms￭ There are implemented in

computer games, network routing software, operating systems, AI algorithms, bin packing algorithms, etc….

￭ The sorting problem itself is easily understood It does not require a large amount of background

knowledge before you can start implementing it

￭ The algorithms are quite small and manageable

They can be implemented in a short period of time The are relatively simple to analyse

Slide 6


Formal Definition of Sorting￭ The sorting problem is a mapping from x to y,

where x and y are both n length real vectors (lists and/or

arrays) x is a permutation of y (different ordering) yi < yi+1 for all i=0,...,n-1

￭ All of the algorithms will apply to whether we want the list sorted

from smallest to largest (ascending order), or from largest to smallest (descending order) or

￭ The algorithms can easily be applied to integers or character vectors

Sorting whole numbers (numerical order) Sorting names and/or addresses (alphabetic order)

Slide 7


Bubble Sort

￭ Imagine that the items to be sorted are kept in an array held vertically.

The items with low values are “light” and bubble up to the top.

We make repeated passes over the array, from bottom to top. If two adjacent elements are out of order (the “lighter”

one is below) we swap them. The effect of this operation is that on the first pass the

“lightest” item rises all the way to the top. On the second pass, the second lowest key rises to the

second position, and so on.

Slide 8

Bubble Sort – Algorithm 1INPUT: List X of n numbers

(1) for i=0 to n-2

(2) for j=n-1 downto i

(3) if xj< xj-1 then

(4) Swap xj and xj-1

(5) End if

(6) End if

OUTPUT: List X sorted in ascending order

Classic Algorithms - Sorting Slide 9

Bubble Sort – Example 115

15

15

15

15

4 4 4 4

4 4 4 4 4 15

15

15

15

9 9 9 6 6 6 6 6 6

11

11

6 9 9 9 9 7 7

7 6 11

11

11

11

7 9 9

6 7 7 7 7 7 11

11

11


4 4 4 4 4 4 4 4

6 6 6 6 6 6 6 6

15

15

15

7 7 7 7 7

7 7 7 15

15

9 9 9

9 9 9 9 9 15

11

11

11

11

11

11

11

11

15

15

s

s

s

s

s

s


Bubble Sort - Performance￭ How many comparisons do we need to make

in the first pass (placing the smallest element to the top)

in the second pass in the last pass

￭ The best case (the list is already sorted) How many passes? How many swaps?

￭ The worst case (the list is sorted in the reverse order)

How many passes? How many swaps?

Slide 11


Bubble Sort: Summary￭ If we compare pairs of adjacent elements and

none are out of order the list is sorted if any are out of order we must have to swap them to get

an ordered list

￭ Bubble sort will make passes though the list swapping any elements that are out of order

￭ After the first pass, we know that the smallest element must be in the correct place

￭ After the second pass, we know that the second smallest element must be in the correct place

￭ Because of this, we can shorten each successive pass of the comparison loop

Slide 12

Bubble SortInput: X - a list of n numbers

(1) Let NoSwaps = False

(2) While NoSwaps = False

(3) Let NoSwaps = True

(4) For i = n-1 downto 0

(5) If xi-1 < xi then

(6) Swap xi-1 and xi

(7) Let NoSwaps = False

(8) End If

(9) End For

(10) End While

Output: X – sorted (ascending)

The algorithm works by running through the list of numbers, swapping pairs that are out of order

The algorithm terminates when no swaps need to be made


Bubble Sort – Example 23 3 1 1

2 1 3 2

1 2 2 3


swap

swap

swap

3 swaps

2 3 1 1

1 1 3 2

3 3 2 3

1 1 1

2 2 2

3 3 3

swap swa

p2 swaps

No swaps


Bubble Sort: Best-Case Analysis

￭ If the elements are in sorted order at the start,

the for loop will compare the adjacent pairs but will make no swaps

￭ This means that the NoSwaps variable will remain true

The While loop is only done once

￭ There are n-1 comparisons in the best case

B(n)O(n)Slide 15


Bubble Sort: Worst –Case Analysis

￭ Worst case the while loop must be done as many

times as possible each pass of the For loop must make at

least one swap of the elements

￭ The number of comparisons will be:

Slide 16

21

1

O 2

)1( )( n

nninW

n

i


Bubble Sort: Average-Case Analysis (Part 1)￭ Much more difficult…

￭ How many passes do we need?

￭ We can potentially stop after any of the (at most) n-1 passes of the For loop

￭ This means that we have n-1 possibilities

￭ Our average case is given by

Slide 17

1

1

)(1

1 )(

n

i

iCn

nA

where C(i) is the work done in the first i passes


Bubble Sort: Average-Case Analysis (Part 2)

￭ On the first pass we do n-1 comparisons

￭ On the second pass we do n-2 comparisons

￭ The number of comparisons in the first i passes, in other words C(i), is given by:

Slide 18

i

j

iinijniC

1 2

)1( )(


Bubble Sort: Average-Case Analysis (Part 3)

￭ Putting the equation for C(i) into the equation for A(n) we get:

Slide 19

)O(312

44

2

)1(

12

)12)(1(

2

)1(

1

1

221

1)(

1

1)(

222

2

1

1

1

1

1

1

21

1

nnnnn

nnnnnnn

n

iini

niC

nnA

n

i

n

i

n

i

n

i


Quick Sort￭ Quicksort is a divide and conquer

algorithm￭ Quicksort picks an element from the list

as the pivot, and partitions the list into two:

Those elements smaller than the pivot value (not necessarily in order)

Those elements larger than the pivot value (not necessarily in order)

￭ Quicksort is then applied recursively on both sub-lists as long as they consist of more than 1 element

Slide 20

Quick Sort: Example 1

4

2

6

5

3

7

1

Pivot:4 2

3

1

4

6

5

7

Pivot:2 1

2

3

4

6

5

7

Pivot:6

1

2

3

4

5

6

7

Slide 21Classic Algorithms - Sorting

Quick Sort: Example 2

26 33 35 29 19 12 22

33 35 2919 12 22

29 3512 22

12 19 22 29 33 35

12 19 22 26 29 33 35

pivot

pivot partition

low high

partition

combine with pivot

combine with pivot


pivot

The Quick Sort Algorithm

Recursive Algorithm

QuickSort(List,First,Last)

Input: List, the elements to be put into order

First, the index of the first element

Last, the index of the last element

(1) If First < Last Then

(2) Let Pivot = PivotList(List,First,Last)

(3) Call QuickSort(List,First,Pivot-1)

(4) Call QuickSort(List,Pivot+1,Last)

(5) End If

Output: List in a sorted order

PivotList is defined laterSlide 23Classic Algorithms - Sorting

The PivotList AlgorithmPivotList(list,first,last)Input: List- the elements to be put into order

First- the index of the first element

Last- the index of the last element1) PivotValue = list[first]2) PivotPoint = first3) For index = first+1 to last4) If list[index] < PivotValue Then5) PivotPoint=PivotPoint+16) Swap(list[PivotPoint],list[index])7) End if8) End For9) Swap(list[first],list[PivotPoint])

Output: PivotPoint

pivot < pivot ≥ pivot unknown

First Pivot point Index LastSlide 24Classic Algorithms - Sorting

Pick the first element as the pivotSet the pivot point as the first location of the listMove through the list comparing the pivot

element to the rest

If an element is smaller increase the pivot point

Swap this element into the new pivot point location

Move pivot value into correct place


Quick Sort: Worst-case Analysis

￭ PivotList will do n-1 comparisons, but creates one partition that has n-1 elements and the other will have no elements

When will this happen?

￭ Because we wind up just reducing the partition by one element each time, the worst case is given by:

Slide 25

)O( 2

)1( 1 )( 2

2

nnn

inWn

i

Quick Sort: Best-case Analysis

￭ PivotList creates two parts that are the same size

￭ And then all subsequent parts are the same size as the algorithm calls itself.

this can be modelled as a binary tree

￭ Summing up over the partitions we get

B(n)=O(nLog2(n))Classic Algorithms - Sorting Slide 26


Quick Sort: Average-Case Analysis Part 1

￭ In the average case, we need to consider all of the possible places where the pivot point winds up

￭ Because for one partition of the list, there are n-1 comparisons done, and there are n ways to partition the list, we have:

Slide 27

0 )0(

0 )1(

)()1(1

)1( )(1

A

A

inAiAn

nnAn

i


Quick Sort: Average-Case Analysis Part 2

￭ Algebra can be used to simplify this recurrence relation to:

￭ This will then solve to:

￭ Therefore, A(n)O(nLog2(n))Slide 28

0 )0(

0 )1(

22)1()1( )(

A

An

nnAnnA

)(Log)1(4.1 )( 2 nnnA

Comparison of Sorting AlgorithmsMethod Best Average Worst

Bubble sort O(n) O(n2) O(n2)

Insertion sort O(n) O(n2) O(n2)

Merge sort O(nlog2(n)) O(nlog2(n)) O(nlog2(n))

Quick sort O(nlog2(n)) O(nlog2(n)) O(n2)

Selection sort O(n2) O(n2) O(n2)



Radix Sort

￭ The Radix sort method works only on binary or integer data

￭ In a computer all numbers are represented as binary

￭ Radix sort works by using a binary bucket sort for each binary digit

￭ Integers in Java are stored using 32 bits

Slide 30

Radix Sort – Example part 1￭ Consider sorting the following list

331, 454, 230, 34, 343, 45, 59, 453, 345, 231, 9

￭ Apply the bucket sort on the last radix position

￭ After being removed from the bucket, the elements are in the following order

230, 331, 231, 343, 453, 454, 34, 45, 345, 59, 9

￭ Apply the bucket sort on the second-to last last radix position, and the elements are put in the buckets as follows


230

331

231

343

453

454

34

4534

5

599

[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

Radix Sort – Example part 2

￭ After being removed from the bucket the elements are in the following order

9, 230, 331, 231, 34, 343, 45, 345, 453, 454, 59

￭ Apply the bucket sort on the third-to last last radix position, and the elements are put in the buckets as follows

￭ After removal from the bucket the elements are in the following order

9, 34, 45, 59, 230, 231, 331, 343, 345, 453, 454Classic Algorithms - Sorting Slide 32

9 230

331

231

34

343

4534

5

453

454

59

[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

9344559

230231

331343345

453454

[0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

Radix Sort – Part 3￭ This algorithm is applied on a list of binary

numbers, a decade equivalent would be similarRadixSort(List,Bits)

Input: List- the elements to be put into order

Bits- the number of bits in each List[i]

(1) For i = 0 to Bits-1

(2) Let b = 2i

(3) Bucket sort list on bit mask b

(4) End If

Output: List in a sorted order



Radix Sort – Part 4

￭ The Radix sort takes O(dn) time complexity

n is the number of items d is the maximum number of radix

positions Best, worse and average case

￭ Can be used for alphabetical sorting, i.e. strings

￭ It is very, very fast!

Slide 34


This Weeks Laboratory

￭ This laboratory does contribute towards your overall grade…

￭ You will be conducting some experiments which test three sorting algorithms

Slide 35


Next Lecture

￭ We will be looking at classic graph based algorithms

￭ This will be in two weeks time￭ Next week is ASK week (effective

learning week /reading week) no lectures and no labs are scheduled

in that week

Slide 36

algorithms and their applications cs2004 (2012-2013) professor jasna kuljis (adapted from dr steve...

Documents