1

Computer AlgorithmsLecture 8

Sorting Algorithms

Some of these slides are courtesy of D. Plaisted, UNC and M. Nicolescu, UNR

The Sorting Problem

• Input:

– A sequence of n numbers a1, a2, . . . , an

• Output:

– A permutation (reordering) a1’, a2’, . . . , an’

of the input sequence such that a1’ ≤ a2’ ≤ ·

· · ≤ an’

Why Study Sorting Algorithms?

• There are a variety of situations that we can encounter– Do we have randomly ordered keys?– Are all keys distinct?– How large is the set of keys to be ordered?– Need guaranteed performance?

• Various algorithms are better suited to some of these situations

Sorting – Definitions

• Sorting: determine a permutation = (p1, … , pn) of n records that puts the keys in non-decreasing order Kp1 < … < Kpn.

• Permutation: a one-to-one function from {1, …, n} onto itself. There are n! distinct permutations of n items.

• Rank: Given a collection of n keys, the rank of a key is the number of keys that precede it. That is, rank(Kj) = |{Ki| Ki < Kj}|. If the keys are distinct, then the rank of a key gives its position in the output file.

Sorting Terminology• Internal (the file is stored in main memory and can be randomly accessed)

vs. External (the file is stored in secondary memory & can be accessed sequentially only)

• Comparison-based sort: uses only the relation among keys, not any special property of the representation of the keys themselves.

• Stable sort: records with equal keys retain their original relative order; i.e., i < j & Kpi = Kpj pi < pj (next slide)

• Array-based (consecutive keys are stored in consecutive memory locations) vs. List-based sort (may be stored in nonconsecutive locations in a linked manner)

• In-place sort: needs only a constant amount of extra space in addition to that needed to store keys.

Stability

• A STABLE sort preserves relative order of records with equal keys

Sort file on first key:

Sort file on second key:

Records with key value 3 are not in order on first key!!

6

Sorting Categories

• Sorting by Insertion insertion sort

• Sorting by Exchange bubble sort, quicksort

• Sorting by Selection selection sort, heapsort

• Sorting by Merging merge sort

• Sorting by Distribution radix sort

Elementary Sorting Methods

• Easier to understand the basic mechanisms of sorting.

• Good for small files.

• Good for well-structured files that are relatively easy to sort, such as those almost sorted.

• Can be used to improve efficiency of more powerful methods.

Selection Sort

Selection-Sort(A, n)1. for i = n downto 2 do2. max i3. for j = i – 1 downto 1 do4. if A[max] < A[j] then5. max j6. t A[max]7. A[max] A[i]8. A[i] t

Selection-Sort(A, n)1. for i = n downto 2 do2. max i3. for j = i – 1 downto 1 do4. if A[max] < A[j] then5. max j6. t A[max]7. A[max] A[i]8. A[i] t

Insertion Sort

InsertionSort(A, n)1. for j = 2 to n do2. key A[j]3. i j – 14. while i > 0 and key < A[i]5. A[i+1] A[i]6. i i – 17. A[i+1] key

InsertionSort(A, n)1. for j = 2 to n do2. key A[j]3. i j – 14. while i > 0 and key < A[i]5. A[i+1] A[i]6. i i – 17. A[i+1] key

Worst-case Analysis

• The maximum number of comparisons while inserting A[i] is (i-1). • So, the number of comparisons is

Cwc(n) i = 2 to n (i -1)

= j = 1 to n-1 j

= n(n-1)/2

= (n2)

• For which input does insertion sort perform n(n-1)/2 comparisons?

Bubble Sort (another elementary sorting algorithm)

• Idea:– Repeatedly pass through the array– Swaps adjacent elements that are out of order

• Easier to implement, but slower than Insertion sort

1 2 3 n

i

1329648

j

Bubble Sort

Alg.: BUBBLESORT(A)

for i 1 to length[A]

do for j length[A] downto i + 1

do if A[j] < A[j -1]

then exchange A[j] A[j-1]

1329648i = 1 j

i

Example1329648

i = 1 j

3129648i = 1 j

3219648i = 1 j

3291648i = 1 j

3296148i = 1 j

3296418i = 1 j

3296481i = 1 j

3296481i = 2 j

3964821i = 3 j

9648321i = 4 j

9684321i = 5 j

9864321i = 6 j

9864321i = 7

j

14

Bubble Sort

Alg.: BUBBLESORT(A)

for i 1 to length[A]

do for j length[A] downto i + 1

do if A[j] < A[j -1]

then exchange A[j] A[j-1]

1329648i = 1 j

i

Bubble-Sort Running Time

T(n) = (n2)

222

)1()(

1 1

22

1

nnnnninin

n

i

n

i

n

i

Alg.: BUBBLESORT(A)for i 1 to length[A]

do for j length[A] downto i + 1

do if A[j] < A[j -1]

then exchange A[j] A[j-1]

T(n) = c1(n+1) +

n

i

in1

)1(c2 c3

n

i

in1

)( c4

n

i

in1

)(

= (n) + (c2 + c3 + c4)

n

i

in1

)(

Comparisons: n2/2 Exchanges: n2/2

16

Merge Sort

Alg.: MERGE-SORT(A, p, r)

if p < r Check for base case

then q ← (p + r)/2 Divide

MERGE-SORT(A, p, q) Conquer

MERGE-SORT(A, q + 1, r) Conquer

MERGE(A, p, q, r) Combine

• Initial call: MERGE-SORT(A, 1, n)

1 2 3 4 5 6 7 8

62317425

p rq

Sorting• Insertion sort

– Design approach:– Sorts in place:– Best case:– Worst case: – n2 comparisons, n2 exchanges

• Bubble Sort– Design approach:– Sorts in place:– Running time:– n2 comparisons, n2 exchanges

Yes(n)

(n2)

incremental

Yes(n2)

incremental

Sorting• Selection sort

– Design approach:– Sorts in place:– Running time: – n2 comparisons, n exchanges

• Merge Sort– Design approach:– Sorts in place:– Running time:

Yes

(n2)

incremental

Nodivide and conquer

(n lgn)

Sorting Files with Huge Records and Small Keys

• Insertion sort or bubble sort?– NO, too many exchanges

• Selection sort?– YES, it takes linear time for exchanges

• Merge sort?– Probably not: needs extra space

Example application: Reorganize your MP-3 files

Sorting Huge, Randomly - Ordered Files

• Selection sort?– NO, always takes quadratic time

• Bubble sort?– NO, always takes quadratic time

• Insertion sort?– NO, quadratic time for randomly-ordered keys

• Merge sort?– YES, it is designed for this problem

Application: Process transaction record for a phone company

Sorting Files That are Almost in Order

• Selection sort?– NO, always takes quadratic time

• Bubble sort?– NO, always takes quadratic time

• Insertion sort?– YES, takes linear time for most definitions of

“almost in order”

• Merge sort?– Probably not: takes nlgn

Applications:Re-sort a huge database after a few changesDouble-check that someone else sorted a file