1 data structures and algorithms sorting. 2 sorting is the process of arranging a list of items...

1

Data Structures and Algorithms

Sorting

2

Sorting Sorting is the process of arranging a list of items

into a particular order

There must be some value on which the order is based

There are many algorithms for sorting a list of items

These algorithms vary in efficiency

3

SORTING

We will examine several algorithms:

Selection Sort

Bubble Sort

Insertion Sort

4

Selection SortThe approach of Selection Sort:

select one value and put it in its final place in the sort list

repeat for all other values

5

In more detail:

find the smallest value in the list

switch it with the value in the first position

find the next smallest value in the list

switch it with the value in the second position

repeat until all values are placed

6

Selection SortAn example:

original: 3 9 6 1 2 smallest is 1: 1 9 6 3 2 smallest is 2: 1 2 6 3 9 smallest is 3: 1 2 3 6 9 smallest is 6: 1 2 3 6 9

Pick any item and insert it into its proper place in a sorted sublist

repeat until all items have been inserted

7

In more detail Selection Sort:

consider the first item to be sorted as a sub list (of one item)

insert the second item into the sorted sub list, shifting items as necessary to make room for the new addition

insert the third item into the sorted sublist (of two items), shifting as necessary

repeat until all values are inserted into their proper position

8

Insertion Sort An example of an insertion sort occurs in everyday

life while playing cards.

To sort the cards in your hand you extract a card, shift the remaining cards, and then insert the extracted card in the correct place.

This process is repeated until all the cards are in the correct sequence. Both average and worst-case time is O(n2).

9

Sorting Card players all know how to sort …

First card is already sortedWith all the rest,

Scan back from the end until you find the first card larger than the new one,

Move all the lower ones down one slot

insert it

A

K

10

J

Q

10

Insertion Sort– Another n^2 sortAn example, :

original: 3 9 6 1 2

insert 9: 3 9 6 1 2

insert 6: 3 6 9 1 2

insert 1: 1 3 6 9 2

insert 2: 1 2 3 6 9

Each loop iteration produces a sorted sub list.

11

Insertion Sort

12

INSERTION SORTAssuming there are nn elements . elements .

For each entry, we may need to examine and shift up to nn - 1 other entries, resulting in a - 1 other entries, resulting in a

O(n2) algorithm.

The insertion sort is an in-place sort. That is, we sort the array in-place, no needed helper array.

13

INSERTION SORTNo extra memory is required.

The insertion sort is also a stable sort.

Stable sorts retain the original ordering of keys when identical keys are present in the input data.

14

Comparing Sorts Both Selection and Insertion sorts are similar in

efficiency

The both have outer loops that scan all elements,

and inner loops that compare the value of the outer loop with almost all values in the list

15

Comparing Selection & Insertion Sort

Therefore approximately nn22 number of comparisons number of comparisons are made to sort a list of size nare made to sort a list of size n

We therefore say that these sorts are of order norder n22

Other sorts are more efficient: order n log2 n

16

Sorting - Insertion sort

ComplexityFor each card

Scan for smaller card O(n)Shift cards down O(n)Insert O(1)Total O(n)

First card requires O(1), second O(2), …For n cards operations

O(n2) ii=1

n

17

Sorting - Insertion sort

ComplexityFor each card

Scan O(n) O(O(loglog n) n)Shift up O(n)O(n)Insert O(1)Total O(n)O(n)

First card requires O(1), second O(2), …

For n cards operations O(n2) ii=1

n

Unchanged!Because the

shift up operationstill requires O(n)

time

Use binary search to find location!

18

Sorting - BubbleFrom the first element

Exchange pairs if they’re out of orderExchange pairs if they’re out of order

Last one must now be the largest

Repeat from the first to n-1

Stop when you have only one element to check

19

Bubble Sort

SWAP(a,b) { int t; t=a; a=b; b=t; }

void bubble( int a[], int n ) { int i, j;

for( i=0; i < n; i++) { /* n passes thru the array */

/*From start to the end of unsorted part */ for( j=1; j < (n-i) ; j++) {

/* If adjacent items out of order, swap */ if( a[j-1]>a[j] ) SWAP(a[j-1],a[j]); } }

}

20

Bubble Sort - Analysisvoid SWAP(a,b) { int t; t=a; a=b; b=t; // see below}void SWAP(a,b) { int t; t=a; a=b; b=t; // see below}

void bubble( int a[], int n ) { int i, j; for( i=0; i < n; i++) { /* n passes thru the array */ /* From start to the end of unsorted part */ for(j=1; j < (n-i); j++) { /* if adjacent items out of order, swap */ if( a[j-1] > a[j] ) SWAP ( a[j-1], a[j]);} }

O(1) statement

21

Bubble Sort - Analysisvoid bubble( int a[], int n ) { int i, j; for(i=0;i<n;i++) for(i=0;i<n;i++) { /* n passes thru the arrayn passes thru the array */

/* From start to the end of unsorted part */ for( j=1;j < (n-i); j++) {for( j=1;j < (n-i); j++) { /* If adjacent items out of order, swap */ if( a[j-1]>a[j] ) SWAP(a[j-1],a[j]); } } }

Inner loopn-1, n-2, n-3, … , i iterations

O(1) statement

22

Bubble Sort - Analysis

Overall

ii=n-1

1=

n(n+1)2

= O(n2)

n outer loop iterations inner loop iteration count

BubbleSort

23

Complexity of the Bubble Sort

Thus:Thus:

((N-1) + (N-2) + N-1) + (N-2) + ...... + 2 + 1 = N*(N-2)/2 = O(N^2) + 2 + 1 = N*(N-2)/2 = O(N^2)

Thus Bubble Sort is an O(N^2) algorithmThus Bubble Sort is an O(N^2) algorithm. .

24

Sorting - SimpleBubble sort

O(n2)Very simple!!! code – slow, should be retired.

Insertion sortSlightly better than bubble sort

Fewer comparisons

Also O(n2) Selection sort is similar

But HeapSort is O(n log n) Where would you use selection or

insertion sort?

25

Simple Sorts Selection Sort or Insertion Sort

Use when n is small

Simple code compensates for low efficiency!

n^2 and n log n

0

500

1000

1500

2000

2500

0 10 20 30 40 50 60

n

Tim

e n log n

n 2̂

26

QuicksortEfficient sorting algorithm

Discovered by C.A.R. Hoare

Example of Divide and Conquer algorithm

Two phasesPartition phase

Divides the work into half

Sort phaseConquers the halves!

27

QuicksortPartition

Choose a pivotFind the position for the pivot so that

all elements to the left are lessall elements to the right are greater

< pivot > pivotpivot

28

Quicksort

ConquerApply the same algorithm to each half

< pivot > pivot

pivot< p’ p’ > p’ < p” p” > p”

29

QuickSortquick_sort(left, right) { // Boundary Condition: when left >= right -- Boundary Condition: when left >= right --

do nothing do nothing // Place first item of current sublist in its

correct position, C, in the sublist itself, such that

// // List[i] <= List[C], for all i, First <= i List[i] <= List[C], for all i, First <= i

< C < C

// List[J] > List[C], for all J, C < J <= // List[J] > List[C], for all J, C < J <= Last Last

quick_sort (left, C-1); quick_sort (C+1, right);

}

30

QuickSort codeint partition(int array [], int left, int right)int partition(int array [], int left, int right)

{{ int i = left, j = right;int i = left, j = right;

int tmp;int tmp;

int pivot = array[(left + right) / 2];int pivot = array[(left + right) / 2]; while (i <= j)while (i <= j)

{{

while (array[i] < pivot)while (array[i] < pivot)

i++;i++;

while (array[j] > pivot)while (array[j] > pivot)

j--;j--;

if (i <= j)if (i <= j)

{{

tmp = array[i];tmp = array[i];

array[i] = arrar[j];array[i] = arrar[j];

array[j] = tmp;array[j] = tmp;

i++;i++;

j--;j--;

}}

};};

return i; return i; }

31

Quicksort

Implementationquicksort( int [] a, int low, int high ) { int pivot; /* Termination condition! */ if ( high > low ) { pivot = partition( a, low, high ); quicksort( a, low, pivot-1 ); quicksort( a, pivot+1, high ); } }

Divide

Conquer

32

QuickSortvoid quickSort(int []array, int left, int right)

{

int index = partition(array, left, right);

// sorts the left side of the array

if (left < index - 1)

quickSort(array, left, index - 1);

// sorts the right side of the array

if (index < right)

quickSort(array, index, right);

}

33

Quicksort - Partition

int partition( int []a, int low, int high ) { int left, right; int pivot_item; pivot_item = a[low]; // pivot is first element in the array pivot = left = low; right = high;

while ( left < right ) { /* Move left while item < pivot */ while( a[left] <= pivot_item ) left++;

/* Move right while item > pivot */ while( a[right] >= pivot_item ) right--; if ( left < right ) SWAP(a,left,right); } /* right is final position for the pivot */ a[low] = a[right]; a[right] = pivot_item; return right; }

34


int partition( int []a, int low, int high ) { int left, right; int pivot_item; pivot_item = a[low]; pivot = left = low;pivot = left = low; right = high;right = high; while ( left < right ) {while ( left < right ) { /* Move left while item < pivot *//* Move left while item < pivot */ while( a[left] <= pivot_item ) left++;while( a[left] <= pivot_item ) left++; /* Move right while item > pivot *//* Move right while item > pivot */ while( a[right] >= pivot_item ) right--;while( a[right] >= pivot_item ) right--; if ( left < right ) SWAP(a,left,right);if ( left < right ) SWAP(a,left,right); }} /* right is final position for the pivot *//* right is final position for the pivot */ a[low] = a[right];a[low] = a[right]; a[right] = pivot_item;a[right] = pivot_item; return right;return right; }}

This exampleuses int’s

to keep thingssimple!

23 12 15 38 42 18 36 29 27

low high

Any item will do as the pivot,choose the leftmost one!

35


int partition( int []a, int low, int high ) { int left, right; int pivot_item; pivot_item = a[low];

pivot = left = low; right = high; while ( left < right ) {while ( left < right ) { /* Move left while item < pivot *//* Move left while item < pivot */ while( a[left] <= pivot_item ) left++;while( a[left] <= pivot_item ) left++; /* Move right while item > pivot *//* Move right while item > pivot */ while( a[right] >= pivot_item ) right--;while( a[right] >= pivot_item ) right--; if ( left < right ) SWAP(a,left,right);if ( left < right ) SWAP(a,left,right); }} /* right is final position for the pivot *//* right is final position for the pivot */ a[low] = a[right];a[low] = a[right]; a[right] = pivot_item;a[right] = pivot_item; return right;return right; }}

Set left and right markers

23 12 15 38 42 18 36 29 27

low highpivot: 23

left right

36


int partition( int a[], int low, int high ) { int left, right ,pivot_item; pivot_item = a[low]; pivot = left = low; right = high;

while ( left < right ) { /* Move left while item < pivot */ while( a[left] <= pivot_item ) left++; /* Move right while item > pivot */ while( a[right] >= pivot_item ) right--; if ( left < right ) SWAP(a,left,right);if ( left < right ) SWAP(a,left,right); }} /* right is final position for the pivot *//* right is final position for the pivot */ a[low] = a[right];a[low] = a[right]; a[right] = pivot_item;a[right] = pivot_item; return right;return right; }}

Move the markers until they cross over

23 12 15 38 42 18 36 29 27

low highpivot: 23

left right

37

Quicksort - Partitionint partition( int []a, int low, int high ) { int left, right;int left, right; int pivot_item;int pivot_item; pivot_item = a[low];pivot_item = a[low]; pivot = left = low;pivot = left = low; right = high;right = high;

while ( left < right ) { /* Move left while item < pivot */

while( a[left] <= pivot_item ) left++; while( a[right] >= pivot_item ) right--; if ( left < right ) SWAP(a,left,right);if ( left < right ) SWAP(a,left,right); }} /* right is final position for the pivot *//* right is final position for the pivot */ a[low] = a[right];a[low] = a[right]; a[right] = pivot_item;a[right] = pivot_item; return right;return right; }}

Move the left pointer while items <= pivot

23 12 15 38 42 18 36 29 27

low highpivot: 23

left right Move right Items >=pivot

38


int partition( int []a, int low, int high ) { int left, right;int left, right; int pivot_item;int pivot_item; pivot_item = a[low];pivot_item = a[low]; pivot = left = low;pivot = left = low; right = high;right = high;

while ( left < right ) {while ( left < right ) { /* Move left while item < pivot *//* Move left while item < pivot */

while( a[left] <= pivot_item ) left++;while( a[left] <= pivot_item ) left++; /* Move right while item > pivot */

while( a[right] >= pivot_item ) right--;

if ( left < right ) SWAP(a, left, right); } /* right is final position for the pivot *//* right is final position for the pivot */ a[low] = a[right];a[low] = a[right]; a[right] = pivot_item;a[right] = pivot_item; return right;return right; }

Swap the two itemson the wrong side of the value of pivot

23 12 15 38 42 18 36 29 27

low highpivot: 23

left right

39




while( a[left] <= pivot_item ) left++; /* Move right while item > pivot */

while( a[right] >= pivot_item ) right--;while( a[right] >= pivot_item ) right--; if ( left < right ) SWAP(a,left,right);if ( left < right ) SWAP(a,left,right); }} /* right is final position for the pivot *//* right is final position for the pivot */ a[low] = a[right];a[low] = a[right]; a[right] = pivot_item;a[right] = pivot_item; return right;return right; }}

left and right have swapped over,

so stop

23 12 15 18 42 38 36 29 27

low highpivot: 23

leftright

40





while( a[right] >= pivot_item ) right--; if ( left < right ) SWAP(a,left,right); } /* right is final position for the pivot */

a[low] = a[right]; a[right] = pivot_item; return right; }

Finally, swap the pivotand a[right}

23 12 15 18 42 38 36 29 27

low highpivot: 23

leftright

41





while( a[right] >= pivot_item ) right--; if ( left < right ) SWAP(a,left,right); } /* right is final position for the pivot */ a[low] = a[right]; a[right] = pivot_item;

return right; }

Return the positionof the pivot

18 12 15 23 42 38 36 29 27

low high

pivot: 23right

42

Quicksort - Conquer

pivot

18 12 15 23 42 38 36 29 27pivot: 23

Recursivelysort left half

Recursivelysort right half

43

Quicksort - Analysis

PartitionCheck every item once O(n)

ConquerDivide data in half O(log2n)

TotalProduct O(n log n)

But there’s a catch …………….

44

Quicksort - The truth!

What happens if we use quicksorton data that’s already sorted(or nearly sorted)

We’d certainly expect it to perform well!

45


Sorted data

1 2 3 4 5 6 7 8 9

pivot

< pivot

?

> pivot

46


Sorted dataEach partition

produces

a problem of size 0and one of size n-1!

Number of partitions?

1 2 3 4 5 6 7 8 9

> pivot

2 3 4 5 6 7 8 9

> pivot

pivot

pivot

47


Sorted data

Each partitionproducesa problem of size 0and one of size n-1!

Number of partitions?n each needing time O(n)Total nO(n)

or O(n2)

? Quicksort is as bad as bubble or insertion sort

1 2 3 4 5 6 7 8 9

> pivot

2 3 4 5 6 7 8 9

> pivot

pivot

pivot

48


Quicksort’s O(n log n) behaviour

Depends on the partitions being nearly equalthere are O( log n ) of them

On average, this will nearly be the case

and quicksort is generally O(n log n)

Can we do anything to ensure O(n log n) time? In general, no

But we can improve our chances!!

49

Quicksort - Choice of the pivot

Any pivot will work …Choose a different pivot …

so that the partitions are equal

then we will see O(n log n) time

1 2 3 4 5 6 7 8 9

pivot

< pivot > pivot

50

Quicksort - Median-of-3 pivot Take 3 positions and choose the median

say … First, middle, last

median is 5perfect division of sorted data every time!O(n log n) time

Since sorted (or nearly sorted) data is common,median-of-3 is a good strategy

• especially if you think your data may be sorted!

1 2 3 4 5 6 7 8 9

51

Quicksort - Random pivot Choose a pivot randomly

Different position for every partition

On average, sorted data is divided evenly

O(n log n) time

• Key requirement

• Pivot choice must take O(1) time

52

Quicksort - Guaranteed O(n log n)? Never!!

Any pivot selection strategy could lead to O(n2) time

• Here median-of-3 chooses 2One partition of 1 and

• One partition of 7

• Next it chooses 4One of 1 and

• One of 5

1 4 9 6 2 5 7 8 3

1 2 4 9 6 5 7 8 3

53

Sorting- Key PointsSorting

Bubble, Insertion, selection sortO(n2) sortsSimple codeMay run faster for small n,

n ~10 (system dependent)

Quick SortDivide and conquerO(n log n)

54

Sorting - Key PointsQuick Sort

O(n log n) but ….Can be O(n2)

Depends on pivot selectionMedian-of-3Random pivot Better but not guaranteed

55

Quicksort - Why bother? Use Heapsort instead?

• Quicksort is generally faster• Fewer comparisons and exchanges

• Some empirical data

n Quick Heap InsertComp Exch Comp Exch Comp Exch

100 712 148 2842 581 2595 899200 1682 328 9736 9736 10307 3503500 5102 919 53113 4042 62746 21083

1 data structures and algorithms sorting. 2 sorting is the process of arranging a list of items...

Documents

place sort

sort list repeat

list slide

insertion sort n elements

approach of selection

efficiency slide

list of items

algorithms sorting slide