measuring the efficiency of algorithms analysis of algorithms provides tools for contrasting the...

47

Upload: joanna-bell

Post on 31-Dec-2015

227 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,
Page 2: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

Measuring the Efficiency of AlgorithmsAnalysis of algorithms

Provides tools for contrasting the efficiency of different methods of solution Time efficiency, space efficiency

A comparison of algorithmsShould focus on significant differences in

efficiencyShould not consider reductions in computing

costs due to clever coding tricks

2

Page 3: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

Measuring the Efficiency of AlgorithmsThree difficulties with comparing programs

instead of algorithmsHow are the algorithms coded?What computer should you use?What data should the programs use?

3

Page 4: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

Measuring the Efficiency of AlgorithmsAlgorithm analysis should be independent of

Specific implementationsComputersData

4

Page 5: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

The Execution Time of AlgorithmsCounting an algorithm's operations is a way

to assess its time efficiencyAn algorithm’s execution time is related to the

number of operations it requiresExample: Traversal of a linked list of n nodes

n + 1 assignments, n + 1 comparisons, n writesExample: The Towers of Hanoi with n disks

2n - 1 moves

5

Page 6: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

Algorithm Growth RatesAn algorithm’s time requirements can be

measured as a function of the problem sizeNumber of nodes in a linked listSize of an arrayNumber of items in a stackNumber of disks in the Towers of Hanoi

problem

Algorithm efficiency is typically a concern for large problems only

6

Page 7: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

Algorithm Growth Rates

7

Figure 9-1 Time requirements as a function of the problem size n

•Algorithm A requires time proportional to n2

•Algorithm B requires time proportional to n

Page 8: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

Algorithm Growth Rates

8

•An algorithm’s growth rate–Enables the comparison of one algorithm with another–Algorithm A requires time proportional to n2

–Algorithm B requires time proportional to n–Algorithm A is faster than Algorithm B –n2 and n are growth-rate functions–Algorithm A is O(n2) - order n2

–Algorithm B is O(n) - order n

•Big O notation

Page 9: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

Order-of-Magnitude Analysis and Big O NotationDefinition of the order of an algorithm

Algorithm A is order f(n) – denoted O (f(n)) – if constants k and n0 exist such that A requires no more than k * f(n) time units to solve a problem of size n ≥ n0

Growth-rate function f(n)A mathematical function used to specify an

algorithm’s order in terms of the size of the problem

9

Page 10: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

Order-of-Magnitude Analysis and Big O Notation

10

Figure 9-3a A comparison of growth-rate functions: (a) in tabular form

Page 11: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

Order-of-Magnitude Analysis and Big O Notation

11

Figure 9-3b A comparison of growth-rate functions: (b) in graphical form

Page 12: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

Order-of-Magnitude Analysis and Big O NotationOrder of growth of some common functions

O(1) < O(log2n) < O(n) < O(n * log2n) < O(n2) < O(n3) < O(2n)

Properties of growth-rate functionsO(n3 + 3n) is O(n3): ignore low-order termsO(5 f(n)) = O(f(n)): ignore multiplicative

constant in the high-order termO(f(n)) + O(g(n)) = O(f(n) + g(n))

12

Page 13: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

Order-of-Magnitude Analysis and Big O NotationWorst-case analysis

A determination of the maximum amount of time that an algorithm requires to solve problems of size n

Average-case analysisA determination of the average amount of time that

an algorithm requires to solve problems of size nBest-case analysis

A determination of the minimum amount of time that an algorithm requires to solve problems of size n

13

Page 14: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

Keeping Your PerspectiveOnly significant differences in efficiency are

interestingFrequency of operations

When choosing an ADT’s implementation, consider how frequently particular ADT operations occur in a given application

However, some seldom-used but critical operations must be efficient

14

Page 15: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

Keeping Your PerspectiveIf the problem size is always small, you

can probably ignore an algorithm’s efficiencyOrder-of-magnitude analysis focuses on large

problems

Weigh the trade-offs between an algorithm’s time requirements and its memory requirements

Compare algorithms for both style and efficiency

15

Page 16: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

The Efficiency of Searching AlgorithmsSequential search

Strategy Look at each item in the data collection in turn Stop when the desired item is found, or the end of

the data is reachedEfficiency

Worst case: O(n) Average case: O(n) Best case: O(1)

16

Page 17: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

The Efficiency of Searching Algorithms

Binary search of a sorted arrayStrategy

Repeatedly divide the array in half Determine which half could contain the item,

and discard the other halfEfficiency

Worst case: O(log2n) For large arrays, the binary search has an

enormous advantage over a sequential search At most 20 comparisons to search an array of

one million items

17

Page 18: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

Sorting Algorithms and Their EfficiencySorting

A process that organizes a collection of data into either ascending or descending order

The sort key The part of a data item that we consider when

sorting a data collection

18

Page 19: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

Sorting Algorithms and Their Efficiency

Categories of sorting algorithmsAn internal sort

Requires that the collection of data fit entirely in the computer’s main memory

An external sort The collection of data will not fit in the

computer’s main memory all at once, but must reside in secondary storage

19

Page 20: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

Selection SortStrategy

Place the largest (or smallest) item in its correct place

Place the next largest (or next smallest) item in its correct place, and so on

Analysis Worst case: O(n2)Average case: O(n2)

Does not depend on the initial arrangement of the data

Only appropriate for small n

20

Page 21: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

Selection Sort

21

Figure 9-4 A selection sort of an array of five integers

Page 22: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

Algorithmvoid selection(int A[]){ for (int l = arraySize -1; l >= 0; --l) { int LI = l; for (int CI = 0; CI < l; ++CI) { if (A[CI] > A[LI]) LI = CI; } int temp = A[LI]; A[LI]=A[l]; A[l] = temp; } // end for} // end selectionSort

22

Sorting.cpp

Page 23: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

Bubble SortStrategy

Compare adjacent elements and exchange them if they are out of order Moves the largest (or smallest) elements to the end

of the array Repeating this process eventually sorts the array

into ascending (or descending) order

AnalysisWorst case: O(n2)Best case: O(n)

23

Page 24: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

Bubble Sort

24

Figure 9-5

The first two passes of a bubble sort of an array of five integers: (a) pass 1; (b) pass 2

Page 25: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

Algorithmvoid bubble(int *arr){for (int i = 0; i< arraySize; i++) for (int j = 0; j <arraySize-i; j++) {if (arr[j]>arr[j+1]) { int temp = arr[i]; arr[i] = arr[j]; arr[j] = temp; } }}

25

Sorting.cpp

Page 26: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

Insertion SortStrategy

Partition the array into two regions: sorted and unsorted Take each item from the unsorted region and

insert it into its correct order in the sorted region

AnalysisWorst case: O(n2)

Appropriate for small arrays due to its simplicity Prohibitively inefficient for large arrays

26

Page 27: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

Insertion Sort

27

Figure 9-7 An insertion sort of an array of five integers.

Page 28: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

Algorithmvoid insertion(int A[]){ for (int i = 1; i < arraySize; ++i) { int nextItem = A[i]; int loc; for (loc = i;loc > 0;--loc) if (A[loc-1]> nextItem) A[loc] = A[loc-1]; else break; A[loc] = nextItem; } // end for} // end insertionSort

28

Sorting.cpp

Page 29: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

MergesortA recursive sorting algorithmPerformance is independent of the initial

order of the array itemsStrategy

Divide an array into halvesSort each halfMerge the sorted halves into one sorted arrayDivide-and-conquer

29

Page 30: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

MergesortAnalysis

Worst case: O(n * log2n)

Average case: O(n * log2n)Advantage

Mergesort is an extremely fast algorithmDisadvantage

Mergesort requires a second array as large as the original array

30

Page 31: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

Mergesort

31

Figure 9-8 A mergesort with an auxiliary temporary array

Page 32: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

Mergesort

32

Figure 9-9 A mergesort of an array of six integers

Page 33: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

The Algorithmvoid mergesort(int theArray[], int first, int last){ if (first < last) { int mid = (first + last)/2; // index of midpoint mergesort(theArray, first, mid); mergesort(theArray, mid+1, last); merge(theArray, first, mid, last); } // end if} // end mergesort

33

Page 34: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

void merge(int theArray[], int first, int mid, int last){ int tempArray[arraySize]; // temporary array int first1 = first; // beginning of first subarray int last1 = mid; // end of first subarray int first2 = mid + 1; // beginning of second subarray int last2 = last; // end of second subarray int index; // next available location in for (index = first1; (first1 <= last1) && (first2 <=

last2); ++index) { if (theArray[first1] < theArray[first2]) { tempArray[index] = theArray[first1]; ++first1; } else { tempArray[index] = theArray[first2]; ++first2; } // end if } // end for 34

Page 35: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

for (; first1 <= last1; ++first1, ++index) // Invariant: tempArray[first1..index-1] is

in order tempArray[index] = theArray[first1];

for (; first2 <= last2; ++first2, ++index) tempArray[index] = theArray[first2];

for (index = first; index <= last; ++index) theArray[index] = tempArray[index];} // end merge

35

Page 36: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

QuicksortA divide-and-conquer algorithmStrategy

Choose a pivotPartition the array about the pivot

items < pivot items >= pivot Pivot is now in correct sorted position

Sort the left sectionSort the right section

36

Page 37: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

37

36 9 23 15 43 56 45 67 97 87

0, 1, p = 0

36 9 23 15 43 56 45 67 97 87

0, 9, p = 4

5, 9, p = 7

15 9 23 36 43 56 45 67 97 87

2, 3, p = 2

0, 3, p = 1

5, 6, p = 5

8, 9, p = 8

9 15 23 36 43 45 56 67 87 97

Page 38: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

Algorithmvoid quicksort (int *a, int lo, int hi){// lo is the lower index, hi is the upper index// of the region of array a that is to be sorted int i=lo, j=hi, h; int x=a[(lo+hi)/2]; do { // partition while (a[i]<x) i++; while (a[j]>x) j--; if (i<=j) { h=a[i]; a[i]=a[j]; a[j]=h; i++; j--; } //exchange i and j } while (i<=j); if (lo<j) quicksort(a, lo, j); if (i<hi) quicksort(a, i, hi);}

38

Page 39: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

QuicksortAnalysis

Average case: O(n * log2n)Worst case: O(n2)

When the array is already sorted and the smallest item is chosen as the pivot

Quicksort is usually extremely fast in practiceEven if the worst case occurs, quicksort’s

performance is acceptable for moderately large arrays

39

Page 40: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

Radix SortStrategy

Treats each data element as a character stringRepeatedly organizes the data into groups

according to the ith character in each elementAnalysis

Radix sort is O(n)

40

Page 41: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

Radix Sort

41

Figure 9-21 A radix sort of eight integers

Sorting.cpp

RadixSort.doc

Page 42: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

A Comparison of Sorting Algorithms

42

Figure 9-22 Approximate growth rates of time required for eight sorting algorithms

Page 43: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

The STL Sorting AlgorithmsSome sort functions in the STL library header <algorithm>sort

Sorts a range of elements in ascending order by default

stable_sort Sorts as above, but preserves original ordering of

equivalent elements

43

Page 44: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

The STL Sorting Algorithmspartial_sort

Sorts a range of elements and places them at the beginning of the range

nth_element Partitions the elements of a range about the nth

element The two subranges are not sorted

partition Partitions the elements of a range according to a

given predicate

44

Page 45: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

SummaryOrder-of-magnitude analysis and Big O

notation measure an algorithm’s time requirement as a function of the problem size by using a growth-rate function

To compare the efficiency of algorithmsExamine growth-rate functions when

problems are largeConsider only significant differences in

growth-rate functions

45

Page 46: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

SummaryWorst-case and average-case analyses

Worst-case analysis considers the maximum amount of work an algorithm will require on a problem of a given size

Average-case analysis considers the expected amount of work that an algorithm will require on a problem of a given size

46

Page 47: Measuring the Efficiency of Algorithms Analysis of algorithms Provides tools for contrasting the efficiency of different methods of solution Time efficiency,

SummaryOrder-of-magnitude analysis can be the basis

of your choice of an ADT implementationSelection sort, bubble sort, and insertion sort

are all O(n2) algorithmsQuicksort and mergesort are two very fast

recursive sorting algorithms

47