Insertion Sort

Algorithm : Design & Analysis

[5]

In the last class…

Recurrence Algorithm and Recursion Equations

Solution of the Recurrence Equations Guess and proving Recursion tree Master theorem

Divide-and-conquer

Insertion Sort

Comparison-based sorting Insertion sort Analysis of insertion sorting algorithm Lower bound of local comparison based

sorting algorithm Shell sort

Why Sorting

Practical use Enough different algorithms for comparing Easy to derive the strong lower bounds for

worst case and average behavior

Ordering Is Out of Order

According to an English dictionary, sorting is defined as a process of separating or arranging things according to class or kind.

Here, “sorting” is “ordering”

A sentence from Knuth: Since only two of our tape drives were in

working order, I was ordered to order more tape units in short order, in order to order the data several orders of magnitude faster.

Comparison-Based Algorithm

The class of “algorithms that sort by comparison of keys” comparing (and, perhaps, copying) the key no other operations are allowed

The measure of work used for analysis is the number of comparison.

As Simple as Inserting

Unsorted Sorted

The “vacancy”, to be shifed leftward, by comparisons

Sorted Unsorted (empty)

Initial Status

On Going

Final Status

Shifting Vacancy: the Specification

int shiftVac(Element[ ] E, int vacant, Key x) Precondition: vacant is nonnegative Postconditions: Let xLoc be the value returned

to the caller, then: Elements in E at indexes less than xLoc are in their

original positions and have keys less than or equal to x.

Elements in E at positions (xLoc+1,…, vacant) are greater than x and were shifted up by one position from their positions when shiftVac was invoked.

Shifting Vacancy: Recursion

int shiftVacRec(Element[] E, int vacant, Key x)int xLoc

1. if (vacant==0)2. xLoc=vacant;3. else if (E[vacant-1].keyx)4. xLoc=vacant;5. else6. E[vacant]=E[vacant-1];7. xLoc=shiftVacRec(E,vacant-1,x);8. Return xLoc

The recursive call is working on a smaller range, so terminating;

The second argument is non-negative, so precondition holding

Worst case frame stack size is (n)

Shifting Vacancy: Iteration

int shiftVac(Element[] E, int xindex, Key x)int vacant, xLoc;vacant=xindex;xLoc=0; //Assume failurewhile (vacant>0)

if (E[vacant-1].keyx)xLoc=vacant; //Succeedbreak;

E[vacant]=E[vacant-1];vacant--; //Keep Looking

return xLoc

Insertion Sorting: the Algorithm

Input: E(array), n0(size of E) Output: E, ordered nondecreasingly by keys Procedure:

void insertSort(Element[] E, int n)int xindex; for (xindex=1; xindex<n; xindex++)

Element current=E[xindex];Key x=current.key;int xLoc=shiftVac(E,xindex,x);E[xLoc]=current;

return;

Worst-Case Analysis

At the beginning, there are n-1 entries in the unsorted segment, so:

To find the right position for x in the sorted segment, i comparisons must be done in the worst case.

Sorted (i entries)

x

2

)1()(

1

1

nninW

n

i

The input for which the upper bound is reached does exist, so: W(n)(n2)

Average Behavior

Assumptions: All permutations of the keys are equally likely as input. There are not different entries with the same keys.Note: For the (i+1)th interval(leftmost), only i comparisons are needed.

x may be located in any one of the i+1 intervals(inclusive), assumingly, with the same probability

Sorted (i entries)

x

Average Complexity

The average number of comparisons in shiftVac to find the location for the ith element:

For all n-1 insertions:

1

11

212)(

1

1

1

1

1

i

i

i

iii

ij

i

i

jfor the leftmost interval

)(ln4

3

4

1

4

)1(

11

4

)1(

1

11

2)(

2

1

2

2

1

1

nnnn

jn

nn

jn

nn

i

inA

n

j

n

j

n

i

Inversion and Sorting

An unsorted sequence E:x1, x2, x3, …, xn-1, xn

If there are no same keys, for the purpose of sorting, it is a reasonable assumption that {x1, x2, x3, …, xn-1, xn}={1,2,3,…,n-1,n}

<xi, xj> is an inversion if xi>xj, but i<j All the inversions must be eliminated

during the process of sorting

Eliminating Inverses: Worst Case

Local comparison is done between two adjacent elements.

At most one inversion is removed by a local comparison.

There do exist inputs with n(n-1)/2 inversions, such as (n,n-1,…,3,2,1)

The worst-case behavior of any sorting algorithm that remove at most one inversion per key comparison must in (n2)

Eliminating Inverses: Average

Computing the average number of inversions in inputs of size n (n>1): Transpose: x1, x2, x3, …, xn-1, xn

xn, xn-1, …, x3, x2, x1

For any i, j, (1jin), the inversion (i,j ) is in exactly one sequence in a transpose pair.

The number of inversions (i,j ) is n(n-1)/2. So, the average number of inversions in all possible inputs is n(n-1)/4.

The average behavior of any sorting algorithm that remove at most one inversion per key comparison must in (n2)

Traveling a Long Way

Problem If a1, a2, …an is a random permutation of {1,2,…n},

what is the average value of

| a1-1|+| a2-2|+…+| a1-n| The answer is the average net distance traveled by all

records during a sorting process.

)1(3

1

)(1

))()((1

|)|...|2||1(|1

||),1(

2

1

1

1

1

1 1

ngivesjonsum

iin

jiijn

jnjjn

isjaofvalueaveragethenjjspcificaForjn

i

j

i

j

i

n

ji

j

Shellsort: the Strategy

7 19 24 13 31 8 82 18 44 63 5 29

7 18 24 13 5 8 82 19 44 63 31 29

5 8 24 13 7 18 31 19 44 63 82 29

5 7 18 13 8 24 31 19 29 63 82 44

h=6

h=4

h=3

h=2

Shellsort: the Strategy (cont.)

For each subsequence, insertion sorting is use, since, generally, in most cycle, there are not many element out of the order.

5 7 8 13 18 19 29 24 31 44 82 63h=1

5 7 8 13 18 19 24 29 31 44 63 82

The solutionThe solution

Shellsort: the Algorithm Input: E, an unsorted array of elements, n0, the number of elements, a

sequence of diminishing increments ht, ht-1,…, h1, where h1=1, and t, the number of increments.

Output: E, with the elements in nondecreasing order of their keys.

Procedure:void shellSort(Element [ ] E, int n, int [ ] h, int t); int xindex, s; for (s=t; s1; s--) for (xindex=h[s]; xindex<n; xindex++) element current=E[xindex]; key x= current.key; int xLoc=shiftVacH(E, h[s], xindex,x); E[xLoc]=current; return;

A special version for Shellsort

A special version for Shellsort

Is there any possibility that Shellsort is an improvement on Insertion Sort, since only the last pass has to do the same thing as Insertion Sort, let along the passes done before that?

Is there any possibility that Shellsort is an improvement on Insertion Sort, since only the last pass has to do the same thing as Insertion Sort, let along the passes done before that????

Complexity of Shellsort The efficiency of Shellsort should be higher than

insertSort, since: More than 1 inverse may be removed by one

comparison, when h>1 Sorting with one increment will not undo any of the work

done previously when a different increment was used

The behavior of Shellsort is mostly unknown Function of the sequence of increments used Influences of “pre-processing”(when h>1) is unknown

Invariable Ordering Pairs under Sorting

rmjmmmm xxxxxxx ,...,,...,,,,...,, 2121

rnrrj yyyyyy ,...,,,...,,...,, 121

Unordered

No greater than

At least r x’s no less than r different y’s

rmjmmmm kkkkkkk xxxxxxx

,...,,...,,,,...,,2121

rnrrj tttttt yyyyyy

,...,,,...,,...,,121

Ordered independently

“No greater than” hold

In non-decreasing order

Previous k-Ordering Kept

If a k-ordered file is h-ordered, it remains k-ordered.That is: assume that a k-ordered sequence a1,a2, … ,an has been h-sorted, then for each i=1,2,…n-k, we have to prove that aiai+k

rhvhrvhvhvhvv aaaaaa ,,...,,,, )1(32

hruhvhuhuu aaaaa )1(32 ,...,,,,

Let iu, i+kv (mod h), note:

hku if

if

hku

hkukuv

Non-existing if v+rh>n

hrkuhkuhkuhkuku aaaaa )1(32 ,...,,,, Last r items

First r items No greater than

r is the largest integer making the subscript maximal no greater than n

Some Results about Shellsort If an h-ordered array is k-sorted, the array will still be h-

ordered. For a two-pass case of Shellsort(i.e. the increments are

1,h) The average number of inversions in an h-ordered

permutation of {1,2,…n} is

Running time is approximately proportional to the best choice of h is about ,which gives O(n5/3)

hnrandhnqwhere

qrh

qr

qqh

q

qqhnf

q

mod/

22

1)1(

2)1(

2)!12(

!!2),(

12

hNhN 32 /2 33 72.1/16 NN

Home Assignment

pp.208- 4.6 4.8 4.9 4.11 4.45