cmsc 100 efficiency of algorithms guest lecturers: clay alberty and kellie laflamme professor marie...
TRANSCRIPT
CMSC 100CMSC 100
Efficiency of AlgorithmsEfficiency of Algorithms
Guest Lecturers: Clay Alberty and Kellie LaFlamme
Professor Marie desJardinsTuesday, October 2, 2012
Some material adapted from instructor slides for Schneider & Gerstung
2
OverviewOverview What makes a good algorithm?
Correctness Ease of understanding Elegance Efficiency
Computational efficiency – order of magnitude of running time Polynomial – time increases (reasonably) slowly as problem size
increases tractable (solvable in reasonable time) Exponential (or worse!) – time increases explosively as problem size
increases intractable (can’t be solved in practice for big problems)
(We are also sometimes interested in memory or space efficiency)
Tue 10/2/12Efficiency of Algorithms
3
Introduction Introduction There are many solutions to any given problem How can we judge and compare algorithms? Analogy: Purchasing a car
safety ease of handling style fuel efficiency
Evaluating an algorithm correctness ease of understanding elegance time/space efficiency
Efficiency of Algorithms Tue 10/2/12
4
Attributes of Attributes of AlgorithmsAlgorithms
Attributes of interest: correctness, ease of understanding, elegance, and efficiency
Correctness: Is the problem specified correctly? Does the algorithm produce the correct result?
Example: pattern matching Problem specification: “Given pattern p and text t, determine the
location, if any, of pattern p occurring in text t” Correctness: does the algorithm always work?
If p is in t, will it say so? If the algorithm says p is in t, is it?
Efficiency of Algorithms Tue 10/2/12
5
Attributes of Attributes of Algorithms (continued)Algorithms (continued)
Ease of understanding, useful for: checking correctness program maintenance
Elegance: using a clever or non-obvious approach Example: Gauss’ summing of 1 + 2 + … + 100
Attributes may conflict: Elegance often conflicts with ease of understanding
Attributes may reinforce each other: Ease of understanding supports correctness
Efficiency of Algorithms Tue 10/2/12
6
Attributes of Attributes of Algorithms (continued)Algorithms (continued)
Efficiency: an algorithm’s use of time and space resources We’ll focus on computational efficiency (time) Timing an algorithm with a clock is not always useful Confounding factors: machine speed, size of input
Benchmarking: timing an algorithm on standard data sets Testing hardware and operating system, etc. Testing real-world performance limits
Analysis of algorithms: the study of the efficiency of algorithms
Order of magnitude Θ() or just O() (“big O”): The class of functions that describes how time increases as a function of
problem size (more on which later)
Efficiency of Algorithms Tue 10/2/12
Sequential Search Analysis
Tue 10/2/12Efficiency of Algorithms
8
Measuring EfficiencyMeasuring EfficiencySequential SearchSequential Search
Searching: the task of finding a specific value in a list of values, or deciding it is not there
Solution #1: Sequential search (from Ch. 2):
“Given a target value and a random list of values, find the location of the target in the list, if it occurs, by checking each value in the list in turn”
Efficiency of Algorithms Tue 10/2/12
9
Sequential Search Sequential Search AlgorithmAlgorithm
get (NameList, PhoneList, Name)i = 1N = length(NameList)Found = FALSEwhile ( (not Found) and (i <= N) ) { if ( Name == NameList[i] ) { print (Name, “’s phone number is ”, PhoneList[i]) Found = TRUE } i = i+1}if ( not Found ) { print (Name, “’s phone number not found!”) }
Tue 10/2/12Efficiency of Algorithms
10
Measuring EfficiencyMeasuring EfficiencySequential Search (continued)Sequential Search (continued)
Central unit of work: operations that occur most frequently
Central unit of work in sequential search: Comparison of target Name to each name in the list Also add 1 to i Typical iteration: two steps (one comparison, one addition)
Given a large input list: Best case: smallest amount of work algorithm must do Worst case: greatest amount of work algorithm must do Average case: depends on likelihood of different scenarios occurring
What are the best, worst, and average cases for sequential search?
Efficiency of Algorithms Tue 10/2/12
11
Measuring EfficiencyMeasuring EfficiencySequential Search (continued)Sequential Search (continued)
Best case: target found with the first comparison (1 iteration)
Worst case: target never found or last value (N iterations)
Average case: if each value is equally likely to be searched, work done varies from 1 to N, on average N/2 iterations
Most often we will consider the worst case Best case is too lucky – can’t count on it Average case is much harder to compute for many problems (hard to know the distribution of possible solutions)
Efficiency of Algorithms Tue 10/2/12
12
Measuring EfficiencyMeasuring EfficiencyOrder of Magnitude—Order nOrder of Magnitude—Order n
Sequential search worst case (N) grows linearly in the size of the problem 2N steps (one comparison and one addition per loop) Also some initialization steps... On the last iteration, we may print something... After the loop, we test and maybe print...
To simplify analysis, disregard the “negligible” steps (which don’t happen asoften), and ignore the coefficient in 2N
Just pay attention to the dominant term (N)
Order of magnitude O(N): the class of all linear functions (any algorithm that takes C1N + C2 steps for any constants C1 and C2)Efficiency of Algorithms Tue 10/2/12
13
Binary SearchBinary Search Solution #2: Binary search
Assume list is sorted Split the list in half on each iteration On each iteration:
If we’ve run out of things to look at, quit Is the centerpoint of the list the name we’re looking for? If so, we’re done! If not, check whether the name is alphabetically after or before
the centerpoint of the list If it’s after, take the second half of the list and continue
looping If it’s before, take the first half of the list and continue looping
Tue 10/2/12Efficiency of Algorithms
14
Binary Search: Binary Search: AlgorithmAlgorithm
get (NameList, PhoneList, Name)N = length(NameList)upper = Nlower = 1Found = FALSEwhile ( (not Found) and (lower <= upper) ) { if ( Name == NameList[(lower+upper)/2] ) { // is it in the center? print (Name, “’s phone number is ”, PhoneList[lower+upper/2]) Found = TRUE } else if ( Name < NameList[(lower+upper)/2] ) { upper = ((lower+upper)/2) – 1 // keep looking AFTER center } else if (Name > NameList[(lower+upper)/2] ) { lower = ((lower+upper)/2) + 1 // keep looking BEFORE center}if ( not Found ) { print (Name, “’s phone number not found!”) }
Tue 10/2/12Efficiency of Algorithms
15
Measuring EfficiencyMeasuring EfficiencyBinary SearchBinary Search
Best case: target found with the first comparison (1 iteration)
Worst case: target never found or last value (split the list in half repeatedly until only one item to examine) For a list of length 2, test twice For a list of length 4, test three times (split twice) For a list of length 8, only test four times! (split three times) For a list of length 2k, how many times to test? For a list of length N, how many times to test?
Average case: harder than you would think to analyze... and surprisingly, the average case is only one step better than the worst
case Why? Hint: Try drawing a tree of all of the cases (left side, right side
after each time the list is split in half)
Efficiency of Algorithms Tue 10/2/12
16
Orders of Magnitude: Orders of Magnitude: log Nlog N
Binary search has order of magnitude O(log2 N): grows very slowly Here: log2 (base 2) but other bases behave similarly
Tue 10/2/12Efficiency of Algorithms
Sorting Algorithms and
Analysishttp://www.youtube.com/watch?v=k4RRi_ntQc8http://www.youtube.com/watch?v=2HjspVV0jK4
Tue 10/2/12Efficiency of Algorithms
18
Measuring EfficiencyMeasuring EfficiencySelection SortSelection Sort
Sorting: The task of putting a list of values into numeric or alphabetical order
Selection sort: Pass repeatedly over the unsorted portion of the list On each pass, select the largest remaining value Move that value immediately after the other unsorted values
Accumulate the largest values, in reverse order, at the end of the list
After each iteration, the number of sorted values grows by one and the number of unsorted values shrinks by one
Efficiency of Algorithms Tue 10/2/12
19Efficiency of Algorithms Tue 10/2/12
How long does this step take?(Hint: do we use sequential search or binary search?
20
Measuring EfficiencyMeasuring EfficiencySelection Sort (continued)Selection Sort (continued)
Central unit of work: hidden in “find largest” step
Work done to find largest changes as unsorted portion shrinks
(N-1) + (N-2) + … + 2 + 1 = N (N-1) / 2
Efficiency of Algorithms Tue 10/2/12
21
Measuring EfficiencyMeasuring EfficiencyOrder of Magnitude—Order NOrder of Magnitude—Order N22
Selection sort takes N(N-1)/2 steps = ½ N2 – N/2
Order of magnitude:ignore all but thedominant (highest-order)term; ignore thecoefficient
Order O(N2): the set of functions whose growth is on the order of N2
Efficiency of Algorithms Tue 10/2/12
22
Measuring EfficiencyMeasuring EfficiencyOrder of Magnitude—Order Order of Magnitude—Order NN22 (continued) (continued)
Eventually, every function with order N2 has greater values than any function with order N
Tue 10/2/12Efficiency of Algorithms
23Efficiency of Algorithms Tue 10/2/12
24
QuicksortQuicksort Quicksort is a faster searching algorithm
Divide and conquer approach: Pick a point in the list (the pivot) Toss everything smaller than the pivot to the left and
everything larger to the right Separately sort those two sublists (using quicksort! This is
an example of a recursive algorithm, which we’ll talk about more later in the semester...)
Quicksort is O(N log N) on average (but can be O(N2) in the worst case...) O(N log N) is slower than linear but faster than quadratic
Tue 10/2/12Efficiency of Algorithms
25
Getting Out of ControlGetting Out of Control Polynomially bounded: an algorithm that does work on
the order of O(Nk) (or less) linear, log, quadratic, ... degree k polynomial
Most common problems are polynomially bounded (in “P”)
Hamiltonian circuit (“traveling salesman problem”): no known polynomial solution—it is in “NP” Given a graph, find a path that passes through each vertex
exactly once and returns to its starting point
Efficiency of Algorithms Tue 10/2/12
26Efficiency of Algorithms Tue 10/2/12
# possible circuits grows exponentially in the number of cities takes a long time to find the best one!
27Efficiency of Algorithms Tue 10/2/12
28
SummarySummary We must evaluate the quality of algorithms, and compare
competing algorithms to each other
Attributes: correctness, efficiency, elegance, and ease of understanding
Compare competing algorithms for time and space efficiency (time/space tradeoffs are common)
Orders of magnitude capture work as a function of input size: O(log N), O(N), O(N2), O(2N)
Problems with only exponential algorithms are intractable
Efficiency of Algorithms Tue 10/2/12
29
Summary of Summary of Complexity ClassesComplexity Classes
Tue 10/2/12Efficiency of Algorithms