chapter 9 sorting and searching arrays

Programming and Problem SolvingWith Java

Copyright 1999, James M. Slack

Chapter 9Sorting and Searching ArraysSorting an ArraySearching an ArrayAnalysis of Algorithms

Programming and Problem Solving With Java 2

Sorting an ArrayRearrange the contents to be in ascending order

Very common activity for computersPhone booksBusiness reportsLists of files

How to do it?

5.29.8

2.51.0

3.1

9.87.4

5.23.1

2.5Sort Process7.4 1.0


Sorting an ArrayMany sorting algorithms, but basically two groups

Algorithms that are simple and slowAlgorithms that are complex and fast

Simple and slow algorithmsSelection sort Insertion sortBubble sort

Complex and fast algorithmsMerge sortHeap sortQuick sort

Will examine selection and merge sort


Sorting an Array: Selection SortSelection sort

Simple and slow algorithmLet's see how it sorts this array

5.29.82.51.03.17.4

0 1 2 3 4 5


Sorting an Array: Selection SortStep 1

Find the element that goes in location 0

Swap that with the element in location 0

5.29.82.51.03.17.4

0 1 2 3 4 5

5.29.82.53.1

0 1 2 3 4 5

5.29.82.57.43.11.0

0 1 2 3 4 5

7.4

1.0

Beforeexchange

Duringexchange

Afterexchange

sorted portionof array


Sorting an Array: Selection SortStep 2

Find the element that goes in location 1

Swap that with the element in location 1

5.29.82.57.43.11.0

0 1 2 3 4 5

5.29.87.41.0

0 1 2 3 4 5

5.29.83.17.42.51.0

0 1 2 3 4 5

3.1

2.5

Beforeexchange

Duringexchange

Afterexchange

sorted portionof array


Sorting an Array: Selection SortRemaining steps similar

Find the element that goes in next location

Swap that with the element in that location

9.87.45.23.12.51.0

0 1 2 3 4 5

5.23.12.51.0

0 1 2 3 4 5

7.4

9.8

9.83.12.51.0

0 1 2 3 4 5

5.2

7.4

5.29.82.51.0

0 1 2 3 4 5

3.1

7.4

array is sorted!


Sorting an Array: Selection SortSelection sort algorithm

Step 0: Search the whole array for the smallest element. Exchange that element with the element in location 0.

Step 1: Search the array from location 1 to the end for the smallest element. Exchange that element with the element in location 1.

Step 2: Search the array from location 2 to the end for the smallest element. Exchange that element with the element in location 2.

Steps 3, ...: Continue in this way for all the other locations in the array.


Sorting an Array: Selection SortSelection sort in Java

// selectionSort: Returns the contents of array sorted in ascending// orderstatic void selectionSort(double[] array){ int locationOfSmallest; double temp;

for (int step = 0; step < array.length; step++) { // Find the location of the smallest element from array[step] // to the end of the array locationOfSmallest = step; for (int loc = step; loc < array.length; loc++) { if (array[loc] < array [locationOfSmallest]) { locationOfSmallest = loc; } }

// Exchange array[step] with array[locationOfSmallest] temp = array[step]; array[step] = array[locationOfSmallest]; array[locationOfSmallest] = temp; }}


Sorting an Array: Merge SortMuch faster sort algorithm than selection sort

0

10000

20000

30000

40000

50000

60000

70000

80000

0 200 400 600 800 200 400 600 800 1000 1200 1400

Number of elements

Timein

milliseconds

SelectionSort

MergeSort


Sorting an Array: Merge SortUses process of merging two sorted sets

1. Compare the topmost index card from each list2. Put the smaller of the two, on the back of the new list3. Continue steps 1 and 2 until one of the lists is empty4. Put the rest of the cards on the back of the new list

Merge Process

9.48.6

7.96.7

5.14.6

2.41.3

9.48.6

4.62.4

7.96.7

5.11.3


Sorting an Array: Merge SortMerge pseudocode

while (both lists still have elements){ if (list1 has the smallest element on top) { remove the topmost element from list1 and put on the output list } else { remove the topmost element from list2 and put on the output list }}

if (list1 still has remaining elements){ move remaining elements from list1 to the output list}

if (list2 still has remaining elements){ move remaining elements from list2 to the output list}


Sorting an Array: Merge SortExample of merging: the mergeArrays() method

// Merges two arrays and displays the merged array.// Demonstrates use of the mergeArrays() method.

public class DemonstrateMergeArrays{ // mergeArrays: Returns the merge of arrays firstSource and // secondSource. Elements of firstSource and // secondSource must be in ascending order static double[] mergeArrays(double[] firstSource, double[] secondSource) { ... }

public static void main(String[] args) { double[] firstArray = {2.4, 4.6, 8.6, 9.4}; double[] secondArray = {1.3, 5.1, 6.7, 7.9}; double[] mergedArray = mergeArrays(firstArray, secondArray);

for (int i = 0; i < mergedArray.length; i++) { System.out.println(mergedArray[i]); } }}

codegoeshere


Sorting an Array: Merge SortMethod mergeArrays()

// mergeArrays: Returns the merge of arrays firstSource and // secondSource. Elements of firstSource and // secondSource must be in ascending order static double[] mergeArrays(double[] firstSource, double[] secondSource) { double[] target = new double[firstSource.length + secondSource.length]; int firstIndex = 0, secondIndex = 0, targetIndex = 0;

// Merge elements from firstSource and secondSource into // target, until all elements from either firstSource or // secondSource are copied while (firstIndex < firstSource.length && secondIndex < secondSource.length) { if (firstSource[firstIndex] < secondSource[secondIndex]) { target[targetIndex] = firstSource[firstIndex]; firstIndex++; } else { target[targetIndex] = secondSource[secondIndex]; secondIndex++; } targetIndex++; }


Sorting an Array: Merge SortMethod mergeArrays() (continued)

// If anything left in firstSource, append it to target while (firstIndex < firstSource.length) { target[targetIndex] = firstSource[firstIndex]; targetIndex++; firstIndex++; }

// If anything left in secondSource, append it to target while (secondIndex < secondSource.length) { target[targetIndex] = secondSource[secondIndex]; targetIndex++; secondIndex++; }

return target; }


Sorting an Array: Merge SortHow mergeArrays() works

9.48.64.62.4

0 1 2 3

9.48.67.96.75.14.62.41.3

0 1 2 3 4 5 6 7

7.96.75.11.3

0 1 2 3

firstSource

secondSource

target


Sorting an Array: Merge SortHow Merge Sort works

Original array to sort

Merge sort also requires an empty array (work)

7.4

0

3.1

1

1.0

2

2.5

3

9.8

4

5.2

5

0 1 2 3 4 5

number

work


Sorting an Array: Merge SortHow Merge Sort works

Merge individual elements into pairs -- in work array

7.4

0

3.1

1

1.0

2

2.5

3

9.8

4

5.2

5

Merge Merge Merge

3.1

0

7.4

1

1.0

2

2.5

3

5.2

4

9.8

5

number

work


Sorting an Array: Merge SortHow Merge Sort works (continued)

Merge pairs from work array into groups of 4 in the original array

1.0

0

2.5

1

3.1

2

7.4

3

5.2

4

9.8

5

Merge Copy

3.1

0

7.4

1

1.0

2

2.5

3

5.2

4

9.8

5

number

work



Merge groups of 4 from original array into groups of 8 (or less) in the work array

1.0

0

2.5

1

3.1

2

7.4

3

5.2

4

9.8

5

Merge

1.0

0

2.5

1

3.1

2

5.2

3

7.4

4

9.8

5

number

work



Result is sorted, but in work arrayMust copy elements back to original array

Copy

1.0

0

2.5

1

3.1

2

5.2

3

7.4

4

9.8

5

number

work

1.0

0

2.5

1

3.1

2

5.2

3

7.4

4

9.8

5


Sorting an Array: Merge SortMerge sort steps

Each element in the array is a single partition. Merge adjacent partitions to a new array, resulting in partitions of size two. Assign the new array to the original.

Each pair of elements in the array is a single partition. Merge adjacent partitions to another new array, resulting in partitions of size four. Assign the new array to the original.

Each group of four elements in the original array is a single partition. Merge adjacent partitions to another new array, resulting in partitions of size eight. Assign the new array to the original.

Continue this process until the partition size isat least as large as the whole array.


Sorting an Array: Merge SortMerge sort in Java

Uses mergePass method -- like mergeArrays(), but input is partitions in a single array, output is a partition in an array

// mergePass: Returns contents of source array in which each// consecutive sequence of elements of length// partitionLen * 2 is sortedstatic double[] mergePass(double[] source, int partitionLen){ int firstPart = 0, secondPart = 0, firstIndex, secondIndex, targetIndex, firstEnd, secondEnd; double[] target = new double[source.length];

// Merge adjacent pairs of partitions while (firstPart < source.length) { secondPart = firstPart + partitionLen; firstIndex = firstPart; secondIndex = secondPart; targetIndex = firstPart; firstEnd = Math.min(firstPart + partitionLen, source.length); secondEnd = Math.min(secondPart + partitionLen, source.length);

"work" array allocatedfor each call to

mergePass()


Sorting an Array: Merge SortMerge sort in Java (continued)

// Merge elements from first partition and second partition into

// target, until all elements from either first partition or // second partition are copied while (firstIndex < firstEnd && secondIndex < secondEnd) { if (source[firstIndex] < source [secondIndex]) { target[targetIndex] = source[firstIndex]; firstIndex++; } else { target[targetIndex] = source[secondIndex]; secondIndex++; } targetIndex++; }

// If anything left in first partition, append it to target. while (firstIndex < firstEnd) { target[targetIndex] = source[firstIndex]; targetIndex++; firstIndex++; }


Sorting an Array: Merge SortMerge sort in Java (continued)

// If anything left in second partition, append it to target while (secondIndex < secondEnd) { target[targetIndex] = source[secondIndex]; targetIndex++; secondIndex++; } firstPart = secondPart + partitionLen; }

return target;}

// mergeSort: Returns the contents of array in ascending orderstatic double[] mergeSort(double[] array){ int partitionLen = 1, index;

// Sort the array by successive merges while (partitionLen < array.length) { // Merge array, creating new array array = mergePass(array, partitionLen); partitionLen = partitionLen * 2; }

return array;}

The mergeSort()method!


Searching an ArraySearching: Basic feature of any data structureTwo kinds of searching

Positional access: what's the value in a particular location?

Associative access: what the location of a particular value?

Arrays work great for positional accessExample: what's the name in location 24?

System.out.println(name[24]);

No predefined associative access for arrays in JavaExample: what's the location of "Smith" in the name

array?


Searching an Array: Linear SearchLinear search with index cards

Start with the first card. If the number you're looking for is on the first card, stop-you've found it.

Stop if the number you're looking for is on the second card.

Continue looking through the remaining cards. Stop if you see the number you're looking for.

If you get to the last card without finding the number, then you know it isn't in the index cards.

5.29.8

2.51.0

3.17.4


Searching an Array: Linear SearchLinear search in Java

Start at one endKeep checking elements until end of array or you find the

element"Quick and dirty" version (can't be used inline)

// linearSearch: Returns the location of itemToFind, or -1 if // not found.static int linearSearch(double[] array, double itemToFind){ for (int itemNum = 0; itemNum < array.length; itemNum++) { if (array[itemNum] == itemToFind) { return itemNum; } { return -1;}

5.29.82.51.03.17.4

0 1 2 3 4 5


Searching an Array: Linear SearchLinear Search in Java

Version that uses while statement (can be inlined)// linearSearch: Returns the location of itemToFind, or -1 if // not found.static int linearSearch(double[] array, double itemToFind){ int itemNum = 0;

while (itemNum < array.length && array[itemNum] != itemToFind) { itemNum++; }

if (itemNum == array.length) { return -1; } else { return itemNum; }}

5.29.82.51.03.17.4

0 1 2 3 4 5

Note order ofsubconditions


Searching An Array: Sets ExampleCan use array as basis for Set class (integers only)Set operations

O p e r a t i o n S y m b o l D e s c r i p t i o n E x a m p l e

U n i o n T h e u n i o n o f t w o s e t s i se v e r y t h i n g t h a t i s i n e i t h e rs e t .

{ , } { , } { , , }1 2 2 3 1 2 3

I n t e r s e c t i o n T h e i n t e r s e c t i o n o f t w o s e t si s e v e r y t h i n g t h a t i s i n b o t hs e t s .

{ , } { , } { }1 2 2 3 2

D i f f e r e n c e T h e d i f f e r e n c e o f t w o s e t s i se v e r y t h i n g t h a t i s t h e f i r s t s e tb u t n o t i n t h e s e c o n d .

{ , } { , } { }1 2 2 3 1


Searching An Array: Sets ExampleVenn diagrams of Set operations

Union

Intersection Difference

A B

A B A B


Searching An Array: Sets ExampleMethods in the IntSet class

add(): Adds an element to a set. If the element is already in the IntSet, it is not added again.

remove(): Removes an element from an IntSet. toString(): Returns a String version of the IntSet contents in

standard format: {1, 3, 2} isEmpty(): Returns true if the IntSet is empty, false otherwise. isFull(): Returns true if the IntSet is full, false otherwise. isMember(): Returns true if item is in the IntSet, false otherwise. union(): Returns the union of the current IntSet and another

IntSet. intersection(): Returns the intersection of the current IntSet and

another IntSet. difference(): Returns the difference of the current IntSet and

another IntSet.


Searching An Array: Sets ExampleExample of using the IntSet class

// Finds the intersection of {1, 2} and {2, 3}

import IntSet;

class FindIntersection{ public static void main(String[] args) { // Define two IntSets; each can store up to 10 values IntSet x = new IntSet(10), y = new IntSet(10);

// Put values in the sets x.add(1); x.add(2); y.add(2); y.add(3);

// Find the intersection IntSet result = x.intersection(y);

// Display the result System.out.println("The intersection of " + x + " and " + y + " is " + result); }}

OutputThe intersection of {1, 2} and {2, 3} is {2}


Searching An Array: Sets ExampleIntSet class in Java

Uses Debug.assert() (from appendix C)import Debug;

public class IntSet{ // Constructor: Initializes a new IntSet with the given // maximum size public IntSet(int maxSize) { numElements = 0; data = new int[maxSize]; }

// add: Adds an element to a set. If the element is already in // the IntSet, it is not added again. Array must not // be full. public void add(int item) { Debug.assert(!isFull(), "IntSet is full"); if (!isMember(item)) { data[numElements] = item; numElements++; } }


Searching An Array: Sets ExampleIntSet class in Java (continued)

// remove: Removes an element from a set. Does nothing if // the element is not in the set. public void remove(int item) { int itemNum = search(item);

if (itemNum != -1) { numElements--; data[itemNum] = data[numElements]; } }

// isMember: Returns true if item is in the set, false otherwise public boolean isMember(int item) { return search(item) != -1; }



// search: Returns location of item or -1 if not there. private int search(int item) { int itemNum = 0;

while (itemNum < numElements && data[itemNum] != item) { itemNum++; }

if (itemNum == numElements) { return -1; } else { return itemNum; } }

Linearsearch!



// toString: Returns a String version of the set's contents // in standard format: {1, 3, 2} public String toString() { String resultString = "{"; for (int i = 0; i < numElements - 1; i++) { resultString = resultString + data[i] + ", "; } return resultString + data[numElements - 1] + "}"; }

// isEmpty: Returns true if the set is empty, false otherwise public boolean isEmpty() { return numElements == 0; }

// isFull: Returns true if the set is full, false otherwise public boolean isFull() { return numElements == data.length; }



// union: Returns the union of the current set and anotherSet public IntSet union(IntSet anotherSet) { // Assume no duplicates (worst case) and make the result set // set large enough to hold everything from both sets IntSet resultSet = new IntSet(numElements + anotherSet.numElements);

// Copy items from this set to the resultSet for (int i = 0; i < numElements; i++) { resultSet.add(data[i]); }

// Copy elements from anotherSet to resultSet (The add() // method will make sure that no element is added twice.) for (int i = 0; i < anotherSet.numElements; i++) { resultSet.add(anotherSet.data[i]); }

return resultSet; }



// intersection: Returns the intersection of the current set // and anotherSet public IntSet intersection(IntSet anotherSet) { // Make the size of the resultSet the same as the smaller of // this set and anotherSet (worst case, if all elements in // smaller set are also in the larger one) IntSet resultSet = new IntSet( Math.min(numElements, anotherSet.numElements));

// Put elements from this set in resultSet only if they are // also in anotherSet for (int i = 0; i < numElements; i++) { if (anotherSet.isMember(data[i])) { resultSet.add(data[i]); } }

return resultSet; }



// difference: Returns the difference of the current set // and anotherSet public IntSet difference(IntSet anotherSet) { // Make the size of the resultSet the same as this set // (worst case, if anotherSet has no elements in common // with this one) IntSet resultSet = new IntSet(numElements);

// Copy items from this set to the resultSet, but only // if they aren't in anotherSet for (int i = 0; i < numElements; i++) { if (!anotherSet.isMember(data[i])) { resultSet.add(data[i]); } } return resultSet; }

// IntSet variables private int data[]; // Set data stored in an array private int numElements; // Number of values in the array}


Searching an Array: Binary SearchLinear search vs. binary

search Linear search fine for many

applications Some applications need

faster searchingBinary search is much

faster Requirement: array must be

sorted

ArraySize

LinearSearch

BinarySearch

10 10 4

50 50 6

100 100 7

500 500 9

1,000 1,000 10

5,000 5,000 13

10,000 10,000 14

50,000 50,000 16

100,000 100,000 17

500,000 500,000 19

1,000,000 1,000,000 20

Elements searched


Searching an Array: Binary SearchBinary search is like looking up a phone number

Start in middle of book If name you're looking for comes before names on page,

look in first halfOtherwise, look in second half


Searching an Array: Binary SearchExample of binary search: find 9.8

Look in middle firstCan use either 3.1 or 5.2 as middle (but should always

pick lower or upper when given a choice)We'll always use the lower value as the middle

9.87.45.23.12.51.0

0 1 2 3 4 5

Note that thearray is sorted


Searching an Array: Binary SearchExample of binary search: find 9.8 (continued)

9.8 greater than 3.1 Therefore 9.8 can't appear in bottom half of arrayNow look at middle of what's left: 7.4

9.87.45.23.12.51.0

0 1 2 3 4 5



9.8 greater than 7.4 Therefore 9.8 can't appear below 7.4Now look at middle of what's left: 9.8

9.87.45.23.12.51.0

0 1 2 3 4 5



9.87.45.23.12.51.0

0 1 2 3 4 5

Found it!


Searching an Array: Binary SearchBinary search in Java - first version

// binarySearch: (First version - incomplete) Returns the location// of itemToFind, or -1 if not found. The array must// be sorted in ascending order.static int binarySearch(double[] array, double itemToFind){ int bottom = 0; int top = array.length - 1; int middle = (bottom + top) / 2;

while (array[middle] != itemToFind) { if (array[middle] > itemToFind) { // Element must be in the lower half of the array } else { // Element must be in the upper half of the array } middle = (bottom + top) / 2; }

// Return position or -1 if not there}

What goeshere?


Searching an Array: Binary SearchBinary search in Java

Value we're looking for is in lower half of arrayExample: Looking for 2.5

9.87.45.23.12.51.0

0 1 2 3 4 5

9.87.45.23.12.51.0

0 1 2 3 4 5

middlebottom

bottom top

Before

After

top



Value we're looking for is in upper half of arrayExample: Looking for 5.2

9.87.45.23.12.51.0

0 1 2 3 4 5

9.87.45.23.12.51.0

0 1 2 3 4 5

middlebottom

bottom top

Before

After

top


Searching an Array: Binary SearchBinary search in Java - second version

// binarySearch: (Second version - incomplete) Returns the location// of itemToFind, or -1 if not found. The array must// be sorted in ascending order.static int binarySearch(double[] array, double itemToFind){ int bottom = 0; int top = array.length - 1; int middle = (bottom + top) / 2;

while (array[middle] != itemToFind) { if (array[middle] > itemToFind) { // Element must be in the lower half of the array top = middle - 1; } else { // Element must be in the upper half of the array bottom = middle + 1; } middle = (bottom + top) / 2; }

// Return position or -1 if not there}

What goeshere?

Doesn't workif itemToFindnot in array



How to detect when value isn't there?When all values "crossed off" -- top < bottom

9.87.45.23.12.51.0

0 1 2 3 4 5

9.87.45.23.12.51.0

0 1 2 3 4 5

bottom

bottomtop

Before

After

top


Searching an Array: Binary SearchBinary search in Java - final version

// binarySearch: (Final version) Returns the location of// itemToFind, or -1 if not found. The array must be // sorted in ascending order.static int binarySearch(double[] array, double itemToFind){ int bottom = 0; int top = array.length - 1; int middle = (bottom + top) / 2;

while (bottom <= top && array[middle] != itemToFind) { if (array[middle] > itemToFind) { // Element must be in the lower half of the array top = middle - 1; } else { // Element must be in the upper half of the array bottom = middle + 1; } middle = (bottom + top) / 2; }

// Return position or -1 if not there if (bottom > top) { return -1; } else { return middle; }}


Algorithm AnalysisHow do we choose between algorithms?

We’ve seen 2 sorting algorithms, 2 searching algorithmsKinds of algorithm analysis

Best case: What is the fastest the algorithm will go -- assuming the data is set up just right?

Average case: What is the average speed of the algorithm

Worst case: What is the slowest the algorithm will go -- assuming the data is set up just right?

Most people usually interested in worst caseThe algorithm will perform no worse that thatThe algorithm may perform better


Analysis of Algorithms: SortingMerge sort runs more quickly than selection sort

But how much faster is merge sort?Program to get this information empirically

// This program times how long a sorting algorithm// takes to complete. The program first asks for the number// of elements to sort. It then displays the number of milliseconds// taken by the sort.

import Keyboard;import java.util.Random;

public class SortTest{

// ---------------------------- // | Put a sort method here | // ----------------------------


Analysis of Algorithms: Sorting // makeRandomArray: Returns an array of size numElements // with random double values static double[] makeRandomArray(int numElements) { Random randomGenerator = new Random(); double[] array = new double[numElements];

// Put random floating-point numbers into the array for (int i = 0; i < array.length; i++) { array[i] = randomGenerator.nextDouble(); }

return array; }

public static void main(String[] args) throws java.io.IOException { long startTime, finishTime;

double[] array = makeRandomArray( Keyboard.readInt("Number of elements to sort: ", 0, Integer.MAX_VALUE));

startTime = System.currentTimeMillis();

// Call sort method here (mergeSort or selectionSort) selectionSort(array);

finishTime = System.currentTimeMillis();

System.out.println("The sort took " + (finishTime - startTime) + " milliseconds"); }}


Analysis of Algorithms: SortingMerge sort vs. selection sort

Number of elements

Timein

milliseconds

0

10000

20000

30000

40000

50000

60000

70000

80000

0 200 400 600 800 200 400 600 800 1000 1200 1400

SelectionSort

MergeSort


Analysis of Algorithms: SortingWhat makes selection sort so slow with large

arrays?Analysis of selection

sort wth 6 elementsFirst step requires

5 comparisons1 exchange

5.29.82.51.03.17.4

0 1 2 3 4 5

5.29.82.53.1

0 1 2 3 4 5

5.29.82.57.43.11.0

0 1 2 3 4 5

7.4

1.0

B eforeex change

Duringex change

A fterex change


Analysis of Algorithms: SortingSecond step requires


5.29.82.57.43.11.0

0 1 2 3 4 5

5.29.87.41.0

0 1 2 3 4 5

5.29.83.17.42.51.0

0 1 2 3 4 5

3.1

2.5

B eforeex change

Duringex change

A fterex change


Analysis of Algorithms: SortingThird step


Fourth step2 comparisons1 exchange

Fifth (last) step1 comparison1 exchange

Sum5+4+3+2+1 = 15 comparisons5 exchanges

9.87.45.23.12.51.0

0 1 2 3 4 5

5.23.12.51.0

0 1 2 3 4 5

7.4

9.8

9.83.12.51.0

0 1 2 3 4 5

5.2

7.4

5.29.82.51.0

0 1 2 3 4 5

3.1

7.4


Analysis of Algorithms: SortingAnalysis of selection sort for 6 elements (again)

5+4+3+2+1 = 15 comparisons5 exchanges

Generalized for n elements (n - 1) + (n - 2) + ... + 3 + 2 + 1 = n(n - 1) / 2 comparisonsn -1 exchanges

Amount of “work” done by selection sortn(n - 1) / 2 + n - 1

Double check for n = 6 (6 x 5 / 2) + 6 - 1 = 15 + 5 = 20 stepsThis matches 15 + 5 (above)


Analysis of Algorithms: SortingAnalysis of merge sortFirst step requires

n comparisons (approx)n copies

7.4

0

3.1

1

1.0

2

2.5

3

9.8

4

5.2

5

M erge M erge M erge

3.1

0

7.4

1

1.0

2

2.5

3

5.2

4

9.8

5

number

work


Analysis of Algorithms: SortingSecond step of merge sort requires


1.0

0

2.5

1

3.1

2

7.4

3

5.2

4

9.8

5

M erge Copy

3.1

0

7.4

1

1.0

2

2.5

3

5.2

4

9.8

5

number

work


Analysis of Algorithms: SortingThird step of merge sort requires


1.0

0

2.5

1

3.1

2

7.4

3

5.2

4

9.8

5

M erge

1.0

0

2.5

1

3.1

2

5.2

3

7.4

4

9.8

5

number

work


Analysis of Algorithms: SortingRemaining steps are the sameMay need to do a copy step, requiring

n copies

Copy

1.0

0

2.5

1

3.1

2

5.2

3

7.4

4

9.8

5

number

work

1.0

0

2.5

1

3.1

2

5.2

3

7.4

4

9.8

5


Analysis of Algorithms: SortingThe amount of work done by merge sort

Number of steps x n x 2 (approximately)Plus n x 2 more if copy step needed

How many steps are there? If n is 8, then 3 steps If n is 32, then 5 steps In general, log2 n steps

Amount of work done by merge sort is then about2n(log2 n) (+ n if copy step needed)


Analysis of Algorithms: Big-OhHow do we compare selection sort and merge sort?

Selection sort: n(n - 1) / 2 + n - 1Merge sort: 2n(log2 n) + n

We can classify algorithms by how their running time increases with data sizeGroup algorithms by complexity classesComplexity class: a group of algorithms whose running

times increase about the same (proportionally) as the data size increases

Write a complexity class using Big-Oh notationFor example, O(n)

Algorithms in different complexity classes have HUGE differences in running times (generally)


Analysis of Algorithms: SortingCommon complexity classes

O rd e r N a m e E x a m p le s

O (1 ) C o n s ta n t A s s ig n m e n t, i f , a n dS y s t e m . o u t . p r i n t l n ( ) . . .

O ( lo g 2 n ) L o g a r i th m ic B in a r y s e a rc h

O (n ) L in e a r l in e a r s e a r c h

O ( n nlo g 2 ) . S o r t M e rg e s o r t

O (n 2 ) S q u a r e S e le c t io n s o r t , b u b b le s o r t , in s e r t io n s o r t

O (n 3 ) C u b ic M a tr ix m u lt ip l ic a t io n , o r th r e e n e s t e d f o r lo o p s

f o r ( i n t i = 1 ; i < n ; i + + )

f o r ( i n t j = 1 ; j < n ; j + + )

f o r ( i n t k = 1 ; k < n ; k + + )

O (2 n ) E x p o n e n tia l T h e " tr a v e l in g s a le s p e rs o n p ro b le m ."


Analysis of Algorithms: SortingHow to find the complexity class of selection sort

Running time: n(n - 1) / 2 + n - 1Multiply out: n2/2 + n/2 - 1

Eliminate all but the most significant term (one with highest exponent)Why?Because the

smaller termscontributerelativelylittle to therunning time

0

1000

2000

3000

4000

5000

6000

0 10 20 30 40 50 60 70 80 90 100

Number of elements

Timein

seconds

n / 22

n / 2


Analysis of Algorithms: SortingResult for selection sort is n2/2

Now eliminate any remaining constantsWhy? Because all algorithms in a complexity class are

proportionally the same -- constants don’t matterLeft with n2 -- this is the O(n2) complexity class


Analysis of Algorithms: SortingHow to find the complexity class of merge sort

Running time: 2n(log2 n) + nEliminate all but the most significant term (one with

highest exponent)Left with 2n(log2 n)

Remove constantsn(log2 n) -- this O(n log2 n)

Merge sort is so much faster than selection sort because it’s in a faster complexity classSelection sort: O(n2)Merge sort: O(n log2 n)

Remember: Algorithms in different complexity classes have HUGE differences in running times (generally)


Analysis of Algorithms: SortingMore formal definition of

Big-Oh An algorithm with running

time g(n) is O(f(n))if g(n) < kf(n) for some constant k and n > n0

Informally, to show that g(n) is O(f(n)) Must show that the graph

for g(n) stays below that of kf(n) after it reaches n0.

Run

ning

Tim

e

Input S ize n

k f(n)

g(n)

n 0


Analysis of Algorithms: SortingExample: show n2/2 + 3n/2 is O(n2)

Must find k and n0 such that n2/2 + 3n/2 < kn2 for all n > n0

Solve for k (n2 + 3n) / 2 < kn2

(n2 + 3n) / 2n < kn (n + 3) / 2n < k3n/2 + n/2 < k

Choose a value of 2 or more for k to satisfy the equationTherefore, n2/2 + 3n/2 is O(n2)


Analysis of Algorithms: SortingHow to use complexity classes to improve speed

Primary requirement of any program: correctnessTherefore, choose simple algorithms at firstSimpler algorithms usually in slower complexity classes If program not fast enough, profile to find slow spotsReplace key algorithms with faster algorithms (ones in

faster complexity classes)Continue until program is fast enough

DON’T “tweak”!!Tweaking doesn’t change complexity classesMax speed improvement from tweaking -- maybe 20%Plus -- tweaking makes a program harder to maintainReplacing an algorithm with one in faster complexity

class can give 10X, 100X (or more) improvement!!


Analysis of Algorithms: EmpiricalSometimes, easier find complexity class with a

spreadsheetAllows analysis from formulas or from timing studiesNot definitive, though -- may get different results on

different computers, compilers, ...


Analysis of Algorithms: EmpiricalExample: find complexity class of n2/2 + 3n/2

Put n in column A

Put calculatedformula (orrunning time)in column B

A B C D E F G1 n Time2 10 653 20 2304 30 4955 40 8606 50 13257 60 18908 70 25559 80 3320

10 90 418511 100 515012 110 621513 120 738014 130 864515 140 1001016 150 1147517 160 1304018 170 1470519 180 1647020 190 1833521 200 20300



Put complexityclass in columnC

(use value incolumn Afor n)

A B C D E F G1 n Time n*n2 10 65 1003 20 230 4004 30 495 9005 40 860 16006 50 1325 25007 60 1890 36008 70 2555 49009 80 3320 6400

10 90 4185 810011 100 5150 1000012 110 6215 1210013 120 7380 1440014 130 8645 1690015 140 10010 1960016 150 11475 2250017 160 13040 2560018 170 14705 2890019 180 16470 3240020 190 18335 3610021 200 20300 40000



In column D,divide columnB by column C

If column Dconverges asn gets large,this isevidencethe algorithmis in the givencomplexityclass

A B C D E F G1 n Time n*n Converge?2 10 65 100 0.65003 20 230 400 0.57504 30 495 900 0.55005 40 860 1600 0.53756 50 1325 2500 0.53007 60 1890 3600 0.52508 70 2555 4900 0.52149 80 3320 6400 0.5188

10 90 4185 8100 0.516711 100 5150 10000 0.515012 110 6215 12100 0.513613 120 7380 14400 0.512514 130 8645 16900 0.511515 140 10010 19600 0.510716 150 11475 22500 0.510017 160 13040 25600 0.509418 170 14705 28900 0.508819 180 16470 32400 0.508320 190 18335 36100 0.507921 200 20300 40000 0.5075

converges as n gets large


Analysis of Algorithms: EmpiricalExample of non-convergence

Shows thatn2 - 25n - 100is not O(n log2n).

A B C D E F G1 n Time n*log2(n) Converge?2 100 7400 664.385619 11.13813 200 34900 1528.771238 22.82884 300 82400 2468.645607 33.37865 400 149900 3457.542476 43.35456 500 237400 4482.892142 52.95697 600 344900 5537.291214 62.28688 700 472400 6615.847778 71.40439 800 619900 7715.084952 80.3491

10 900 787400 8832.403072 89.149011 1000 974900 9965.784285 97.824712 1100 1182400 11113.61659 106.392013 1200 1409900 12274.58243 114.863414 1300 1657400 13447.58468 123.248915 1400 1924900 14631.69556 131.556916 1500 2212400 15826.12018 139.794217 1600 2519900 17030.1699 147.966818 1700 2847400 18243.24235 156.079719 1800 3194900 19464.80614 164.137320 1900 3562400 20694.38904 172.143321 2000 3949900 21931.56857 180.1011

doesn’t converge


Analysis of Algorithms: SearchingLinear search

Worst case: need to examine each value in the arrayTherefore, linear search is O(n)

5.29.82.51.03.17.4

0 1 2 3 4 5


Analysis of Algorithms: SearchingBinary search

First step -- look in middle

Second step -- look in middle of what’s left

Third step -- look in middle of what’s left

9.87.45.23.12.51.0

0 1 2 3 4 5

9.87.45.23.12.51.0

0 1 2 3 4 5

9.87.45.23.12.51.0

0 1 2 3 4 5


Analysis of Algorithms: SearchingBinary search (continued)

Keep looking in the middle what’s left until either•find the element•run out of elements (worst case)

How many times can we divide n by 2? log2n

Therefore, binary search is O(log2n)


Analysis of Algorithms: SearchingComparison of linear

and binary search (worst case)HUGE difference in

running time (because theyare in different complexityclasses!)

However, remember thatbinary search requires asorted array...

ArraySize

LinearSearch

BinarySearch

10 10 4

50 50 6

100 100 7

500 500 9

1,000 1,000 10

5,000 5,000 13

10,000 10,000 14

50,000 50,000 16

100,000 100,000 17

500,000 500,000 19

1,000,000 1,000,000 20

chapter 9 sorting and searching arrays

Documents