lecture 6: analysis of sorting methods

Algorithmics - Lecture 6 1

LECTURE 6:

Analysis of sorting methods


Outline

• The problem of sorting

• Insertion sort – correctness and efficiency analysis

• Selection sort – correctness and efficiency analysis

• Bubble sort – correctness and efficiency analysis


The problem of sortingLet us consider:

• A sequence of items

• Each item has a characteristic called sorting key. The values of the sorting key belong to a set on which a total order relationship exists

• Sorting the sequence = arrange its elements such that the sorting keys are in increasing (or decreasing) order


The problem of sortingExamples:

1. Sequence of numbers: (5,8,3,1,6) The sorting key is the entire entity (an integer value)

The result of the sorting process is (1,3,5,6,8)

2. Sequence of records containing two fields (name and mark): ((Peter,9), (John, 10), (Victor,8), (Adam,9))

There are two possible criteria of sorting here: name and mark

Increasing sorting by name: ((Adam,9),(John,10),(Peter,9),(Victor,8))Decreasing sorting by mark: ((John,10),(Peter,9),(Adam,9),(Victor,8))


The problem of sortingMore formally …

Ordering (increasingly) a sequence (x1,x2,…,xn) is equivalent to finding a permutation (p(1),p(2),…,p(n)) of the indices such that

key(xp(1))<=key(xp(2)) <= … <=key(xp(n))

In the following we shall consider that the sorting key is the element itself, thus the following condition is satisfied:

xp(1)<=xp(2) <= … <=xp(n)


The problem of sortingOther assumptions:

• We shall consider that the sequence is stored in a random access memory (e.g. in an array)

This means that we will discuss about internal sorting

• We shall analyze only sorting methods which are in place (the additional space needed for sorting has at most the size of an element).


Properties of sorting methods:1. Simplicity

The method is simple if it is easy to understand and to implement

2. EfficiencyThe method is efficient if it doesn’t use a large amount of resources

(running time)

3. NaturalnessThe method is natural if the number of operations depends on how

out of order is the initial sequence

4. StabilityThe method is stable if it preserves the initial ordering of any two

elements having identical keys


StabilityExample:

Initial configuration: ((Adam,9), (John, 10), (Peter,9), (Victor,8))

Stable sorting : ((John,10),(Adam,9),(Peter,9),(Victor,8))

Unstable sorting : ((John,10), (Peter,9),(Adam,9), (Victor,8))


Basic sorting methodsAre simple, intuitive but not very efficient methods…

However, they are the starting point for advanced methods.

Examples:• Insertion sort• Selection sort• Bubble sort


Insertion sort

Basic idea: Starting with the second element of the array, each element is

inserted in the subsequence which precedes it such that this subsequence is kept sorted

(x1,x2,…,xi-1,xi,xi+1,…xn)

Already sorted subsequence(destination subsequence) Element to be inserted


Insertion sort - example

8 5 9 8 1 8 8 9 8 1aux=5

5 8 9 8 1

5 8 9 8 1aux=9

5 8 9 8 1

5 8 9 8 1aux=8

5 8 9 9 1 5 8 8 9 1

5 8 8 9 1aux=1

5 8 8 9 9 5 8 8 8 9 5 8 8 8 9 5 5 8 8 9

1 5 8 8 9

i

2

3

4

5

j i-1 i-2 i-3 i-4

WHILE (j>=1) AND (aux<x[j])

FO

R

i:=

2,n


Insertion sort – algorithm

General structure

FOR i:=2,n DO

<insert x[i] in the already sorted subarray x[1..i-1] such that x[1..i] is also sorted>

ENDFOR

AlgorithmInsertion(x[1..n])

FOR i:=2,n DO

aux:=x[i]

j:=i-1

WHILE (j>=1) AND (aux<x[j)) DO

x[j+1]:=x[j]

j:=j-1

ENDWHILE

x[j+1]:=aux

ENDFOR

RETURN x[1..n]


Insertion sort – a variant

Insertion(x[1..n])

FOR i:=2,n DO

aux:=x[i]

j:=i-1

WHILE (j>=1) AND (aux<x[j]) DO

x[j+1]:=x[j]

j:=j-1

ENDWHILE

x[j+1]:=aux

ENDFOR

RETURN x[1..n]

Insertion(x[1..n])

FOR i:=2,n DO

x[0]:=x[i]

j:=i-1

WHILE (x[0]<x[j]) DO

x[j+1]:=x[j]

j:=j-1

ENDWHILE

x[j+1]:=x[0]

ENDFOR

RETURN x[1..n]


Insertion sort – correctness analysis

Insertion(x[1..n])

i:=2 {x[1..i-1] sorted}

WHILE i<=n

aux:=x[i]

j:=i-1 {x[1..j] sorted, aux<=x[j+1]<=..<=x[i]}

WHILE (j>=1) AND (aux<x[j]) DO

x[j+1]:=x[j] {x[1..j-1] sorted, aux<x[j]=x[j+1]<= … <=x[i]}

j:=j-1 {x[1..j] sorted, aux<x[j+1]=x[j+2]<= … <=x[i]}

Either {aux>=x[j] and x[1]<=…x[j]<=aux<x[j+1]=x[j+2]<=…<=x[i]}

or {(j=0) and aux<x[1]=x[2]<=…<=x[i]}

ENDWHILE

x[j+1]:=aux {x[1]<=x[2]<=…x[j+1]<=x[j+2]<=…<=x[i]} (x[1..i] sorted)

i:=i+1 {x[1..i-1] sorted}

ENDWHILE

RETURN x[1..n]


Insertion sort – correctness analysis

Thus we obtained …

An invariant for the outer loop: {x[1..i-1] sorted}

An invariant for the inner loop:

{x[1..j] sorted, aux<=x[j+1]<=x[j+2]<=..<=x[i]}

Both loops are finite:

termination function for the outer loop: t(p)=n+1-ip termination function for the inner loop:

jp if aux <x[jp]

t(p)=

0 if aux>=x[jp]


Insertion sort – efficiency analysis

Insertion(x[1..n])FOR i:=2,n DO x[0]:=x[i] j:=i-1 WHILE x[0]<x[j] DO x[j+1]:=x[j] j:=j-1 ENDWHILE x[j+1]:=x[0]ENDFORRETURN x[1..n]

Input size: nOperations:• Comparisons (TC(n))• Movements (of the elements)

(TM(n))

For each i (2 <=i <= n):

1 <=TC(n) <=i

For all i:

(n-1) <=TC(n) <=(n+2)(n-1)/2



Insertion(x[1..n])FOR i:=2,n DO x[0]:=x[i] j:=i-1 WHILE x[0]<x[j] DO x[j+1]:=x[j] j:=j-1 ENDWHILE x[j+1]:=x[0] ENDFOR RETURN x[1..n]

Input size: nOperations:• Comparisons (TC(n))• Movements (of the elements) (TM(n))


0+2 <=TM(n) <=(i-1)+2=i+1

For all i:

2 (n-1) <=TM(n) <=(n+1)(n+2)/2-3



Comparisons:

(n-1) <=TC(n) <=(n+2)(n-1)/2Movements:

2 (n-1) <=TM(n) <=(n+1)(n+2)/2-3Total: 3(n-1) <=T(n) <=n2+2n-3

Efficiency classes: T(n) Ω(n) T(n) O(n2)


Insertion sort – stability

8 5 9 8’ 1 5 8 9 8’ 1

5 8 9 8’ 1 5 8 9 8’ 1

5 8 9 8’ 1 5 8 8’ 9 1

5 8 8’ 9 1 1 5 8 8’ 9

The insertion method is stable


Outline


• Insertion sort

• Selection sort

• Bubble sort


Basic idea:

For each position, i (starting with the first one) the subsequence x[i..n] is scanned in order to identify the minimum. This minimum is swapped with the i-th element.

(x1,x2,…,xi-1,xi,xi+1,…xn)

Selection sort

Already sorted subsequence

Subsequence searched for the minimumThe position where the minimum will be placed


Selection sort

8 5 9 8 1 8 5 9 8 1 8 5 9 8 1

i

1

2

3

4

j i i+1 … 5

FOR j:=i+1,n

8 5 9 8 1 8 5 9 8 1

1 5 9 8 8 1 5 9 8 8 1 5 9 8 8 1 5 9 8 8

1 5 9 8 8 1 5 9 8 8 1 5 9 8 8

1 5 8 9 8 1 5 8 9 8

1 5 8 8 9

FO

R i:

=1,

n-1


General structure

FOR i:=1,n-1 DO<find the minimum of x[i..n] and swap it with x[i]>

ENDFOR

Selection sort

Algorithm

Selection (x[1..n])

FOR i:=1,n-1 DO

k:=i;

FOR j:=i+1,n DO

IF x[k]>x[j] THEN k:=j ENDIF

ENDFOR

IF k<>i

THEN x[i]<->x[k]

ENDIF

ENDFOR

RETURN x[1..n]


Selection sort – correctness analysis

AlgorithmSelection (x[1..n])

i:=1 {x[1..i-1] sorted and x[i-1]<=x[j], j=i..n}

WHILE i<=n-1 DO

k:=i

j:=i+1 {x[k]<=x[r] for all r=i+1..j-1}

WHILE j<=n DO

IF x[k]>x[j] THEN k:=j ENDIF

j:=j+1

ENDWHILE {x[k]<=x[r] for all r=i+1..n}

IF k<>i THEN x[i]<->x[k] {x[1..i] sorted and x[i]<=x[j], j=i+1..n}

i:=i+1

ENDWHILE {x[1..i-1] sorted and x[i-1]<=x[j], j=i..n}

RETURN x[1..n]


Selection sort – correctness analysisThus we obtained …

An invariant for the outer loop:

{x[1..i-1] sorted and x[i-1]<=x[j], j=i..n}


{x[k]<=x[r] for all r=i+1..j-1}


termination function for the outer loop: t(p)=n-ip termination function for the inner loop: t(p)=n+1-jp



(TM(n))

For each i (1 <=i <= n-1):

TC(n,i) =n-i

For all i:

TC(n) =n(n-1)/2

Selection sort – efficiency analysis

Selection (x[1..n])

FOR i:=1,n-1 DO

k:=i

FOR j:=i+1,n DO

IF x[k]>x[j] THEN k:=j

ENDFOR

IF k<>i THEN x[i]<->x[k]

ENDFOR

RETURN x[1..n]



(TM(n))


0 <= TM(n,i) <=3

For all i:

0 <= TM(n) <= 3(n-1)


Selection (x[1..n])

FOR i:=1,n-1 DO

k:=i

FOR j:=i+1,n DO

IF x[k]>x[j] THEN k:=j

ENDFOR

IF k<>i THEN x[i]<->x[k]

ENDFOR

RETURN x[1..n]


Comparisons:

TC(n) =n(n-1)/2Movements:

0 <= TM(n) <= 3(n-1)Total: n(n-1)/2 <=T(n) <=n(n-1)/2+3(n-1)

Efficiency class: T(n) (n2)



Selection sort - stability

8 5 9 8’1 8 5 9 8’1 8 5 9 8’1 8 5 9 8’1 8 5 9 8’1

1 5 9 8’8 1 5 9 8’8 1 5 9 8’8 1 5 9 8’8

1 5 9 8’8 1 5 9 8’8 1 5 9 8’8

1 5 8’9 8 1 5 8’9 8

1 5 8’8 9

The method is not stable


Outline


• Insertion sort

• Selection sort

• Bubble sort


Bubble sort

Basic idea:

The array is scanned from left to right and the adjacent elements are compared. If they are out of order then they are swapped. The scanning process is repeated until the array is sorted.

(x1,x2,…,xm-1,xm,xm+1,…xn)

Already sorted subsequence


Bubble sort

8 5 9 8 1 5 8 9 8 1 5 8 9 8 1

i

5

4

3

2

j 1 2 … 4

FOR j:=1,i-1

5 8 8 9 1 5 8 8 1 9

5 8 8 1 9 5 8 8 1 9 5 8 8 1 9 5 8 1 8 9

5 8 1 8 9 5 8 1 8 9 5 1 8 8 9

5 1 8 8 9 1 5 8 9 8

FO

R i:

=n,

2,-

1


Bubble sort

General structure

FOR i:=n,2,-1 DO< scan x[1..i-1], compare the

adjacent elements and swap them if they are out of order>

ENDFOR

Algorithm

Bubblesort(x[1..n])

FOR i:=n,2,-1 DO

FOR j:=1,i-1 DO

IF x[j]>x[j+1]

THEN x[j]<->x[j+1]

ENDIF

ENDFOR

ENDFOR

RETURN x[1..n]


Bubble sort – correctness analysisBubble(x[1..n])

i:=n {x[i..n] is sorted and x[i]>=x[j], j=1..i}

WHILE i>=2 DO

j:=1 {x[j]>=x[k], k=1..j-1}

WHILE j<=i-1 DO

IF x[j]>x[j+1] THEN x[j]<->x[j+1] {x[j+1]>=x[k], k=1..j}

j:=j+1 {x[j]>=x[k], k=1..j-1}

ENDWHILE {x[i-1]>=x[j], j=1..i-1}

{x[i-1..n] is sorted and x[i-1]>=x[j], j=1..i-1}

i:=i-1

ENDWHILE {x[i..n] is sorted and x[i]>=x[j], j=1..i}

RETURN x[1..n]


Bubble sort – correctness analysisThus we obtained …

An invariant for the outer loop:

{x[i..n] is sorted and x[i]>=x[j], j=1..i}


{x[j]>=x[k], k=1..j-1}


termination function for the outer loop: t(p)=ip-1

termination function for the inner loop: t(p)=ip-jp


Bubble sort – efficiency analysis

Bubblesort(x[1..n])

FOR i:=n,2,-1 DO

FOR j:=1,i-1 DO

IF x[j]>x[j+1]

THEN x[j]<->x[j+1]

ENDIF

ENDFOR

ENDFOR

RETURN x[1..n]

Input size: n

Operations:

Comparisons (TC(n))

Movements (of the elements) (TM(n))

For each i (1 <=i <= n-1):

TC(n,i) =i-1

For all i:

TC(n) =n(n-1)/2



Bubblesort(x[1..n])

FOR i:=n,2,-1 DO

FOR j:=1,i-1 DO

IF x[j]>x[j+1]

THEN x[j]<->x[j+1]

ENDIF

ENDFOR

ENDFOR

RETURN x[1..n]

Input size: nOperations:

Comparisons (TC(n))

Movements (of the elements) (TM(n))

For each i (2 <= i <= n):

0 <= TM(n,i) <= 3(i-1)

For all i:

0 <= TM(n) <= 3n(n-1)/2



Comparisons:

TC(n) =n(n-1)/2

Movements:

0 <= TM(n) <= 3n(n-1)/2 Total:

n(n-1)/2 <= T(n) <= 2n(n-1)

Efficiency classes: T(n) (n2)


Bubble sort – other variantsIdea: repeat the array scanning

only if at the last scan at least one swap has been made

Bubblesort(x[1..n])sw:=TRUEWHILE sw=True DO sw:=FALSE FOR j:=1,n-1 DO IF x[j]>x[j+1] THEN x[j]<->x[j+1] sw:=TRUE ENDIF ENDFOR ENDWHILERETURN x[1..n]

n-1<=TC(n)<=n(n-1)0<=TM(n)<=3n(n-1)/2

Idea: scan the array only up to the index of the last swap

Bubblesort(x[1..n])

t:=n

WHILE t>1

m:=t; t:=0;

FOR j:=1,m-1 DO

IF x[j]>x[j+1]

THEN x[j]<->x[j+1]; t:=j

ENDIF ENDFOR

ENDWHILE

RETURN x[1..n]

n-1<=TC(n)<=n(n-1)/20<=TM(n)<=3n(n-1)/2


Bubble sort - stability

8 5 9 8’ 1 5 8 9 8’ 1 5 8 9 8’ 1 5 8 8’9 1 5 8 8’ 1 9

5 8 8’1 9 5 8 8’1 9 5 8 8’1 9 5 8 1 8’ 9

5 8 1 8’ 9 5 8 1 8’ 9 5 1 8 8’ 9

5 1 8 8’ 9 1 5 8 9 8’

Bubble sort is stable


Some useful (or just interesting) links…

http://www.sorting-algorithms.com/

http://www.softpanorama.org/Algorithms/sorting.shtml

http://www.brian-borowski.com/Software/Sorting/

http://www.youtube.com/watch?v=INHF_5RIxTE ☺

http://boingboing.net/2010/08/19/what-does-a-bubble-s.html ☺


Next lecture will be on …

… algorithm design techniques

… recursive algorithms

… decrease and conquer

lecture 6: analysis of sorting methods

Documents

insertedalgorithmics

keyxp1algorithmics lecture

orderalgorithmics lecture

auxalgorithmics lecture

decreasing sorting

unstable sorting

sorting process

sorting keys