chapter 3 the efficiency of algorithms
DESCRIPTION
Chapter 3 The Efficiency of Algorithms. Algorithmic problem solving. Attributes of Algorithms. Are some algorithms better than others? We expect correctness from our algorithms Ease of understanding; Elegance Analysis of Algorithms Efficiency - PowerPoint PPT PresentationTRANSCRIPT
Chapter 3Chapter 3
The Efficiency of AlgorithmsThe Efficiency of Algorithms
Algorithmic problem solving
Attributes of AlgorithmsAttributes of Algorithms Are some algorithms better than others?Are some algorithms better than others? We expect We expect correctness correctness from our algorithmsfrom our algorithms Ease of understanding; EleganceEase of understanding; Elegance
Analysis of AlgorithmsAnalysis of Algorithms EfficiencyEfficiency
Term used to describe an algorithm’s careful use of resourcesTerm used to describe an algorithm’s careful use of resources BenchmarksBenchmarks
Useful for rating one machine against another and for rating Useful for rating one machine against another and for rating how sensitive a particular algorithm is with respect to how sensitive a particular algorithm is with respect to variations in input on one particular machinevariations in input on one particular machine
22
Measuring EfficiencyMeasuring Efficiency
Analysis of algorithmsAnalysis of algorithms The study of the efficiency of algorithmsThe study of the efficiency of algorithms An important part of computer scienceAn important part of computer science
Invitation to Computer Science, 5th EditionInvitation to Computer Science, 5th Edition 33
Sequential SearchSequential Search
Search for Search for NAMENAME among a list of among a list of nn names names
Start at the beginning and compare Start at the beginning and compare NAMENAME to each entry until a match is to each entry until a match is foundfound
Invitation to Computer Science, 5th EditionInvitation to Computer Science, 5th Edition 44
Invitation to Computer Science, Invitation to Computer Science, 5th Edition5th Edition 55
Figure 3.1 Sequential Search Algorithm
Invitation to Computer Science, Invitation to Computer Science, 5th Edition5th Edition 66
Figure 3.2 Number of Comparisons to Find NAME in a List of n Names Using Sequential Search
Order of Magnitude - Order Order of Magnitude - Order nn
Order of magnitude Order of magnitude nn Anything that varies as a constant times Anything that varies as a constant times
n n (and whose graph follows the basic (and whose graph follows the basic shape of shape of n)n)
Sequential searchSequential search An Θ(An Θ(nn) algorithm in both the worst case ) algorithm in both the worst case
and the average caseand the average case
Invitation to Computer Science, 5th EditionInvitation to Computer Science, 5th Edition 77
Invitation to Computer Science, 5th EditionInvitation to Computer Science, 5th Edition 88
Figure 3.3 Work = 2n
Invitation to Computer Science, 5th EditionInvitation to Computer Science, 5th Edition 99
Figure 3.4 Work = cn for Various Values of c
Invitation to Computer Science, 5th EditionInvitation to Computer Science, 5th Edition 1010
Figure 3.5 Growth of Work = cn for Various Values of c
Selection SortSelection Sort Selection sort algorithmSelection sort algorithm
Sorts in ascending orderSorts in ascending order Subtask within selection sortSubtask within selection sort
Task of finding the largest number in a listTask of finding the largest number in a list When selection sort algorithm beginsWhen selection sort algorithm begins
The largest-so-far value must be compared to The largest-so-far value must be compared to all the other numbers in the listall the other numbers in the list
If there are If there are n n numbers in the list, numbers in the list, n n – 1 – 1 comparisons must be donecomparisons must be done
Invitation to Computer Science, 5th EditionInvitation to Computer Science, 5th Edition 1111
Invitation to Computer Science, Invitation to Computer Science, 5th Edition5th Edition 1212
Figure 3.6 Selection Sort Algorithm
Invitation to Computer Science, 5th EditionInvitation to Computer Science, 5th Edition 1313
Figure 3.7 Comparisons Required by Selection Sort
Selection Sort (continued)Selection Sort (continued) Selection sort algorithm Selection sort algorithm
Does Does n n exchanges, one for each position in the exchanges, one for each position in the list to put the correct value in that positionlist to put the correct value in that position
Space efficiency of the selection sortSpace efficiency of the selection sort Original list occupies Original list occupies n n memory locationsmemory locations Storage is needed for: Storage is needed for:
The marker between the unsorted and sorted sectionsThe marker between the unsorted and sorted sections Keeping track of the largest-so-far value and its Keeping track of the largest-so-far value and its
location in the listlocation in the list
Invitation to Computer Science, 5th EditionInvitation to Computer Science, 5th Edition 1414
Invitation to Computer Science, 5th EditionInvitation to Computer Science, 5th Edition 1515
Figure 3.8 An Attempt to Exchange the Values at X and Y
Invitation to Computer Science, 5th EditionInvitation to Computer Science, 5th Edition 1616
Figure 3.9 Exchanging the Values at X and Y
Order of Magnitude - Order Order of Magnitude - Order nn22
Order of magnitude Order of magnitude nn22, or Θ(, or Θ(nn22)) An algorithm that does An algorithm that does cncn22 work for any work for any
constant constant cc Selection sortSelection sort
An Θ(An Θ(nn22) algorithm (in all cases) ) algorithm (in all cases) Sequential search Sequential search
An Θ(An Θ(nn) algorithm (in the worst case)) algorithm (in the worst case)
Invitation to Computer Science, 5th EditionInvitation to Computer Science, 5th Edition 1717
Invitation to Computer Science, 5th EditionInvitation to Computer Science, 5th Edition 1818
Figure 3.10 Work 5 cn2 for Various Values of c
Invitation to Computer Science, 5th EditionInvitation to Computer Science, 5th Edition 1919
Figure 3.11 A Comparison of n and n2
Invitation to Computer Science, 5th EditionInvitation to Computer Science, 5th Edition 2020
Figure 3.12 For Large Enough n, 0.25n2 Has Larger Values Than 10n
Invitation to Computer Science, 5th EditionInvitation to Computer Science, 5th Edition 2121
Figure 3.13 A Comparison of Two Extreme Q(n2) and Q(n) Algorithms
Analysis of AlgorithmsAnalysis of Algorithms
Data cleanup algorithmsData cleanup algorithms The Shuffle-Left AlgorithmThe Shuffle-Left Algorithm The Copy-Over Algorithm The Copy-Over Algorithm The Converging-Pointers AlgorithmThe Converging-Pointers Algorithm
Invitation to Computer Science, 5th EditionInvitation to Computer Science, 5th Edition 2222
The Shuffle-Left AlgorithmThe Shuffle-Left Algorithm
Scans list from left to rightScans list from left to right When a zero is found, copy each When a zero is found, copy each
remaining data item in the list one remaining data item in the list one cell to the leftcell to the left
Value of Value of legitlegit Originally set to the length of the listOriginally set to the length of the list Is reduced by 1 every time a 0 is Is reduced by 1 every time a 0 is
encounteredencountered
Invitation to Computer Science, 5th EditionInvitation to Computer Science, 5th Edition 2323
Invitation to Computer Science, 5th EditionInvitation to Computer Science, 5th Edition 2424
Figure 3.14 The Shuffle-Left Algorithm for Data Cleanup
The Shuffle-Left Algorithm The Shuffle-Left Algorithm (continued)(continued)
To analyze time efficiency:To analyze time efficiency: Identify the fundamental units of work Identify the fundamental units of work
the algorithm performsthe algorithm performs Copying numbersCopying numbers
Best case occurs when the list has no 0 Best case occurs when the list has no 0 values because no copying is requiredvalues because no copying is required
Worst case occurs when the list has all Worst case occurs when the list has all 0 values0 values
Invitation to Computer Science, 5th EditionInvitation to Computer Science, 5th Edition 2525
The Shuffle-Left Algorithm The Shuffle-Left Algorithm (continued)(continued)
Worst case Worst case Occurs when the list has all 0 valuesOccurs when the list has all 0 values An Θ(An Θ(nn22) algorithm ) algorithm
Space-efficient Space-efficient Only requires four memory locations to Only requires four memory locations to
store the quantities store the quantities nn,, legit legit,, left left,, and and rightright
Invitation to Computer Science, 5th EditionInvitation to Computer Science, 5th Edition 2626
The Copy-Over AlgorithmThe Copy-Over Algorithm Scans the list from left to right, copying every Scans the list from left to right, copying every
legitimate (non-zero) value into a new list that it legitimate (non-zero) value into a new list that it createscreates
Every list entry is examined to see whether it is 0 Every list entry is examined to see whether it is 0 Every non-zero list entry is copied onceEvery non-zero list entry is copied once Best case Best case
Occurs if all elements are 0Occurs if all elements are 0 Θ(Θ(nn) in time efficiency) in time efficiency No extra space is usedNo extra space is used
2727
Invitation to Computer Science, 5th EditionInvitation to Computer Science, 5th Edition 2828
Figure 3.15 The Copy-Over Algorithm for Data Cleanup
The Copy-Over Algorithm The Copy-Over Algorithm (continued)(continued)
Worst case Worst case Occurs if there are no 0 values in the listOccurs if there are no 0 values in the list Algorithm copies all Algorithm copies all n n non-zero elements non-zero elements
into the new list and doubles the space into the new list and doubles the space requiredrequired
Θ(Θ(nn) in time efficiency) in time efficiency Time/space tradeoffTime/space tradeoff
You gain something by giving up You gain something by giving up something elsesomething else
Invitation to Computer Science, 5th EditionInvitation to Computer Science, 5th Edition 2929
The Converging-Pointers The Converging-Pointers AlgorithmAlgorithm
Swap zero values from left with values from Swap zero values from left with values from right until pointers converge in the middleright until pointers converge in the middle
Best case Best case A list containing no 0 elementsA list containing no 0 elements
Worst caseWorst case A list of all 0 entriesA list of all 0 entries Θ(Θ(nn) in time efficiency) in time efficiency Is space-efficientIs space-efficient
Invitation to Computer Science, 5th EditionInvitation to Computer Science, 5th Edition 3030
Invitation to Computer Science, 5th EditionInvitation to Computer Science, 5th Edition 3131
Figure 3.16 The Converging-Pointers Algorithm for Data Cleanup
Invitation to Computer Science, 5th EditionInvitation to Computer Science, 5th Edition 3232
Figure 3.17 Analysis of Three Data Cleanup Algorithms
The Converging-Pointers The Converging-Pointers Algorithm (continued)Algorithm (continued)
In an Θ(In an Θ(nn) algorithm:) algorithm: The work is proportional to The work is proportional to nn
In an Θ(In an Θ(nn22) algorithm:) algorithm: The work is proportional to the The work is proportional to the square square
of of nn
Invitation to Computer Science, 5th EditionInvitation to Computer Science, 5th Edition 3333
Binary SearchBinary Search ProcedureProcedure
First looks for First looks for NAME NAME at roughly the halfway point in at roughly the halfway point in listlist
If name equals If name equals NAMENAME,, search is oversearch is over If If NAME NAME comes alphabetically before name at comes alphabetically before name at
halfway point, search is narrowed to the front half halfway point, search is narrowed to the front half of listof list
If If NAME NAME comes alphabetically after name at halfway comes alphabetically after name at halfway point, search is narrowed to the back half of the listpoint, search is narrowed to the back half of the list
Algorithm halts when Algorithm halts when NAME NAME is foundis found
Invitation to Computer Science, 5th EditionInvitation to Computer Science, 5th Edition 3434
Invitation to Computer Science, 5th EditionInvitation to Computer Science, 5th Edition 3535
Figure 3.18 Binary Search Algorithm (list must be sorted)
Binary Search (continued)Binary Search (continued) Logarithm of Logarithm of n n to the base 2to the base 2
Number of times a number Number of times a number n n can be cut in half and can be cut in half and not go below 1not go below 1
As As n n doubles:doubles: lg lg n n increases by only 1, so lg increases by only 1, so lg n n grows much more grows much more
slowly than slowly than nn Worst case and average caseWorst case and average case
Θ(lg Θ(lg nn) comparisons) comparisons Works only on a list that has already been sortedWorks only on a list that has already been sorted
3636
Invitation to Computer Science, 5th EditionInvitation to Computer Science, 5th Edition 3737
Figure 3.19 Binary Search Tree for a 7-Element List
Invitation to Computer Science, 5th EditionInvitation to Computer Science, 5th Edition 3838
Figure 3.20 Values for n and lg n
Invitation to Computer Science, 5th EditionInvitation to Computer Science, 5th Edition 3939
Figure 3.21 A Comparison of n and lg n
Pattern-MatchingPattern-Matching
Usually involves a pattern length that Usually involves a pattern length that is short compared to the text lengthis short compared to the text length That is, when That is, when m m is much less than is much less than nn
Best caseBest case Θ(Θ(nn))
Worst caseWorst case Θ(Θ(m m * * nn))
Invitation to Computer Science, 5th EditionInvitation to Computer Science, 5th Edition 4040
When Things Get Out of When Things Get Out of HandHand
Polynomially bound algorithmsPolynomially bound algorithms Work done is no worse than a constant Work done is no worse than a constant
multiple of multiple of nn22
GraphGraph A collection of nodes and connecting A collection of nodes and connecting
edgesedges Hamiltonian circuitHamiltonian circuit
A path through a graph that begins and A path through a graph that begins and ends at the same node and goes through ends at the same node and goes through all other nodes exactly onceall other nodes exactly once
Invitation to Computer Science, 5th EditionInvitation to Computer Science, 5th Edition 4141
Invitation to Computer Science, 5th EditionInvitation to Computer Science, 5th Edition 4242
Figure 3.22 Order-of-Magnitude Time Efficiency Summary
Invitation to Computer Science, 5th EditionInvitation to Computer Science, 5th Edition 4343
Figure 3.23 Four Connected Cities
Invitation to Computer Science, 5th EditionInvitation to Computer Science, 5th Edition 4444
Figure 3.24 Hamiltonian Circuits among All Paths from A in Figure 3.23 with Four Links
When Things Get Out of When Things Get Out of HandHand
Exponential algorithmExponential algorithm An Θ(2An Θ(2nn) algorithm) algorithm
Brute force algorithmBrute force algorithm One that beats the problem into submission by One that beats the problem into submission by
trying all possibilitiestrying all possibilities Intractable problemIntractable problem
No polynomially bounded algorithm existsNo polynomially bounded algorithm exists Approximation algorithmsApproximation algorithms
Do not solve the problem, but provide a close Do not solve the problem, but provide a close approximation to a solutionapproximation to a solution
4545
Invitation to Computer Science, 5th EditionInvitation to Computer Science, 5th Edition 4646
Figure 3.25 Comparisons of lg n, n, n2, and 2n
Invitation to Computer Science, 5th EditionInvitation to Computer Science, 5th Edition 4747
Figure 3.26 Comparisons of lg n, n, n2, and 2n for Larger Values of n
Invitation to Computer Science, 5th EditionInvitation to Computer Science, 5th Edition 4848
Figure 3.27 A Comparison of Four Orders of Magnitude