chapter 3 the efficiency of algorithms the efficiency of algorithms algorithmic problem solving

Chapter 3Chapter 3

The Efficiency of AlgorithmsThe Efficiency of Algorithms

Algorithmic problem solving

Attributes of AlgorithmsAttributes of Algorithms Are some algorithms better than others?Are some algorithms better than others? We expect We expect correctness correctness from our algorithmsfrom our algorithms Ease of understanding; EleganceEase of understanding; Elegance

Analysis of AlgorithmsAnalysis of Algorithms EfficiencyEfficiency

Term used to describe an algorithm’s careful use of resourcesTerm used to describe an algorithm’s careful use of resources BenchmarksBenchmarks

Useful for rating one machine against another and for rating Useful for rating one machine against another and for rating how sensitive a particular algorithm is with respect to how sensitive a particular algorithm is with respect to variations in input on one particular machinevariations in input on one particular machine

22

Measuring EfficiencyMeasuring Efficiency

Analysis of algorithmsAnalysis of algorithms The study of the efficiency of algorithmsThe study of the efficiency of algorithms An important part of computer scienceAn important part of computer science

Invitation to Computer Science, 5th EditionInvitation to Computer Science, 5th Edition 33

Sequential SearchSequential Search

Search for Search for NAMENAME among a list of among a list of nn names names

Start at the beginning and compare Start at the beginning and compare NAMENAME to each entry until a match is to each entry until a match is foundfound


Invitation to Computer Science, Invitation to Computer Science, 5th Edition5th Edition 55

Figure 3.1 Sequential Search Algorithm


Figure 3.2 Number of Comparisons to Find NAME in a List of n Names Using Sequential Search

Order of Magnitude - Order Order of Magnitude - Order nn

Order of magnitude Order of magnitude nn Anything that varies as a constant times Anything that varies as a constant times

n n (and whose graph follows the basic (and whose graph follows the basic shape of shape of n)n)

Sequential searchSequential search An Θ(An Θ(nn) algorithm in both the worst case ) algorithm in both the worst case

and the average caseand the average case



Figure 3.3 Work = 2n


Figure 3.4 Work = cn for Various Values of c


Figure 3.5 Growth of Work = cn for Various Values of c

Selection SortSelection Sort Selection sort algorithmSelection sort algorithm

Sorts in ascending orderSorts in ascending order Subtask within selection sortSubtask within selection sort

Task of finding the largest number in a listTask of finding the largest number in a list When selection sort algorithm beginsWhen selection sort algorithm begins

The largest-so-far value must be compared to The largest-so-far value must be compared to all the other numbers in the listall the other numbers in the list

If there are If there are n n numbers in the list, numbers in the list, n n – 1 – 1 comparisons must be donecomparisons must be done



Figure 3.6 Selection Sort Algorithm


Figure 3.7 Comparisons Required by Selection Sort

Selection Sort (continued)Selection Sort (continued) Selection sort algorithm Selection sort algorithm

Does Does n n exchanges, one for each position in the exchanges, one for each position in the list to put the correct value in that positionlist to put the correct value in that position

Space efficiency of the selection sortSpace efficiency of the selection sort Original list occupies Original list occupies n n memory locationsmemory locations Storage is needed for: Storage is needed for:

The marker between the unsorted and sorted sectionsThe marker between the unsorted and sorted sections Keeping track of the largest-so-far value and its Keeping track of the largest-so-far value and its

location in the listlocation in the list



Figure 3.8 An Attempt to Exchange the Values at X and Y


Figure 3.9 Exchanging the Values at X and Y

Order of Magnitude - Order Order of Magnitude - Order nn22

Order of magnitude Order of magnitude nn22, or Θ(, or Θ(nn22)) An algorithm that does An algorithm that does cncn22 work for any work for any

constant constant cc Selection sortSelection sort

An Θ(An Θ(nn22) algorithm (in all cases) ) algorithm (in all cases) Sequential search Sequential search

An Θ(An Θ(nn) algorithm (in the worst case)) algorithm (in the worst case)



Figure 3.10 Work 5 cn2 for Various Values of c


Figure 3.11 A Comparison of n and n2


Figure 3.12 For Large Enough n, 0.25n2 Has Larger Values Than 10n


Figure 3.13 A Comparison of Two Extreme Q(n2) and Q(n) Algorithms

Analysis of AlgorithmsAnalysis of Algorithms

Data cleanup algorithmsData cleanup algorithms The Shuffle-Left AlgorithmThe Shuffle-Left Algorithm The Copy-Over Algorithm The Copy-Over Algorithm The Converging-Pointers AlgorithmThe Converging-Pointers Algorithm


The Shuffle-Left AlgorithmThe Shuffle-Left Algorithm

Scans list from left to rightScans list from left to right When a zero is found, copy each When a zero is found, copy each

remaining data item in the list one remaining data item in the list one cell to the leftcell to the left

Value of Value of legitlegit Originally set to the length of the listOriginally set to the length of the list Is reduced by 1 every time a 0 is Is reduced by 1 every time a 0 is

encounteredencountered



Figure 3.14 The Shuffle-Left Algorithm for Data Cleanup

The Shuffle-Left Algorithm The Shuffle-Left Algorithm (continued)(continued)

To analyze time efficiency:To analyze time efficiency: Identify the fundamental units of work Identify the fundamental units of work

the algorithm performsthe algorithm performs Copying numbersCopying numbers

Best case occurs when the list has no 0 Best case occurs when the list has no 0 values because no copying is requiredvalues because no copying is required

Worst case occurs when the list has all Worst case occurs when the list has all 0 values0 values


The Shuffle-Left Algorithm The Shuffle-Left Algorithm (continued)(continued)

Worst case Worst case Occurs when the list has all 0 valuesOccurs when the list has all 0 values An Θ(An Θ(nn22) algorithm ) algorithm

Space-efficient Space-efficient Only requires four memory locations to Only requires four memory locations to

store the quantities store the quantities nn,, legit legit,, left left,, and and rightright


The Copy-Over AlgorithmThe Copy-Over Algorithm Scans the list from left to right, copying every Scans the list from left to right, copying every

legitimate (non-zero) value into a new list that it legitimate (non-zero) value into a new list that it createscreates

Every list entry is examined to see whether it is 0 Every list entry is examined to see whether it is 0 Every non-zero list entry is copied onceEvery non-zero list entry is copied once Best case Best case

Occurs if all elements are 0Occurs if all elements are 0 Θ(Θ(nn) in time efficiency) in time efficiency No extra space is usedNo extra space is used

2727


Figure 3.15 The Copy-Over Algorithm for Data Cleanup

The Copy-Over Algorithm The Copy-Over Algorithm (continued)(continued)

Worst case Worst case Occurs if there are no 0 values in the listOccurs if there are no 0 values in the list Algorithm copies all Algorithm copies all n n non-zero elements non-zero elements

into the new list and doubles the space into the new list and doubles the space requiredrequired

Θ(Θ(nn) in time efficiency) in time efficiency Time/space tradeoffTime/space tradeoff

You gain something by giving up You gain something by giving up something elsesomething else


The Converging-Pointers The Converging-Pointers AlgorithmAlgorithm

Swap zero values from left with values from Swap zero values from left with values from right until pointers converge in the middleright until pointers converge in the middle

Best case Best case A list containing no 0 elementsA list containing no 0 elements

Worst caseWorst case A list of all 0 entriesA list of all 0 entries Θ(Θ(nn) in time efficiency) in time efficiency Is space-efficientIs space-efficient



Figure 3.16 The Converging-Pointers Algorithm for Data Cleanup


Figure 3.17 Analysis of Three Data Cleanup Algorithms

The Converging-Pointers The Converging-Pointers Algorithm (continued)Algorithm (continued)

In an Θ(In an Θ(nn) algorithm:) algorithm: The work is proportional to The work is proportional to nn

In an Θ(In an Θ(nn22) algorithm:) algorithm: The work is proportional to the The work is proportional to the square square

of of nn


Binary SearchBinary Search ProcedureProcedure

First looks for First looks for NAME NAME at roughly the halfway point in at roughly the halfway point in listlist

If name equals If name equals NAMENAME,, search is oversearch is over If If NAME NAME comes alphabetically before name at comes alphabetically before name at

halfway point, search is narrowed to the front half halfway point, search is narrowed to the front half of listof list

If If NAME NAME comes alphabetically after name at halfway comes alphabetically after name at halfway point, search is narrowed to the back half of the listpoint, search is narrowed to the back half of the list

Algorithm halts when Algorithm halts when NAME NAME is foundis found



Figure 3.18 Binary Search Algorithm (list must be sorted)

Binary Search (continued)Binary Search (continued) Logarithm of Logarithm of n n to the base 2to the base 2

Number of times a number Number of times a number n n can be cut in half and can be cut in half and not go below 1not go below 1

As As n n doubles:doubles: lg lg n n increases by only 1, so lg increases by only 1, so lg n n grows much more grows much more

slowly than slowly than nn Worst case and average caseWorst case and average case

Θ(lg Θ(lg nn) comparisons) comparisons Works only on a list that has already been sortedWorks only on a list that has already been sorted

3636


Figure 3.19 Binary Search Tree for a 7-Element List


Figure 3.20 Values for n and lg n


Figure 3.21 A Comparison of n and lg n

Pattern-MatchingPattern-Matching

Usually involves a pattern length that Usually involves a pattern length that is short compared to the text lengthis short compared to the text length That is, when That is, when m m is much less than is much less than nn

Best caseBest case Θ(Θ(nn))

Worst caseWorst case Θ(Θ(m m * * nn))


When Things Get Out of When Things Get Out of HandHand

Polynomially bound algorithmsPolynomially bound algorithms Work done is no worse than a constant Work done is no worse than a constant

multiple of multiple of nn22

GraphGraph A collection of nodes and connecting A collection of nodes and connecting

edgesedges Hamiltonian circuitHamiltonian circuit

A path through a graph that begins and A path through a graph that begins and ends at the same node and goes through ends at the same node and goes through all other nodes exactly onceall other nodes exactly once



Figure 3.22 Order-of-Magnitude Time Efficiency Summary


Figure 3.23 Four Connected Cities


Figure 3.24 Hamiltonian Circuits among All Paths from A in Figure 3.23 with Four Links

When Things Get Out of When Things Get Out of HandHand

Exponential algorithmExponential algorithm An Θ(2An Θ(2nn) algorithm) algorithm

Brute force algorithmBrute force algorithm One that beats the problem into submission by One that beats the problem into submission by

trying all possibilitiestrying all possibilities Intractable problemIntractable problem

No polynomially bounded algorithm existsNo polynomially bounded algorithm exists Approximation algorithmsApproximation algorithms

Do not solve the problem, but provide a close Do not solve the problem, but provide a close approximation to a solutionapproximation to a solution

4545


Figure 3.25 Comparisons of lg n, n, n2, and 2n


Figure 3.26 Comparisons of lg n, n, n2, and 2n for Larger Values of n


Figure 3.27 A Comparison of Four Orders of Magnitude

chapter 3 the efficiency of algorithms the efficiency of algorithms algorithmic problem solving

Documents