Efficiency & Complexity of AlgorithmsIntroduction

Desirable characteristics in an algorithm Correctness Ease of understanding (clarity) Elegance Efficiency

Attributes of Algorithms Correctness

o Does the algorithm solve the problem it is designed for?o Does the algorithm solve the problem correctly?

Ease of understanding (clarity)o How easy is it to understand or alter the algorithm?o Important for program maintenance

Eleganceo How clever or sophisticated is the algorithm?o Sometimes elegance and ease of understanding work at cross-purposes

Efficiencyo How much time and/or space does the algorithm require when executed?o Perhaps the most important desirable attribute

Measuring Efficiency Analysis of algorithms

o Study of the efficiency of various algorithms Efficiency measured as a function relating size of input to time or space used For one input size, best case, worst case, and average case behavior must be

considered The notation captures the order of magnitude of the efficiency function

1

Sequential Search Search for NAME among a list of n names Start at the beginning and compare NAME to each entry until a match is found

Sequential Search Algorithm

Comparison of the NAME being searched for against a name in the listo Central unit of worko Used for efficiency analysis

For lists with n entrieso Best case

NAME is the first name in the list 1 comparison (1)

o Worst case NAME is the last name in the list NAME is not in the list n comparisons (n)

o Average case Roughly n/2 comparisons (n)

Space efficiencyo Uses essentially no more memory storage than original input requireso Very space efficient

2

Order of Magnitude: Order n As n grows large, order of magnitude dominates running time, minimizing

effect of coefficients and lower-order terms All functions that have a linear shape are considered equivalent Order of magnitude n

o Written (n)o Functions vary as a constant times n

Work = cn for Various Values of c

3

Selection Sort Sorting

o Take a sequence of n values and rearrange them into order Selection sort algorithm

o Repeatedly searches for the largest value in a section of the data Moves that value into its correct position in a sorted section of the

listo Uses the Find Largest algorithm

Selection Sort Algorithm

Count comparisons of largest so far against other values Find Largest, given m values, does m-1 comparisons Selection sort calls Find Largest n times,

o Each time with a smaller list of valueso Cost = n-1 + (n-2) + … + 2 + 1 = n(n-1)/2

Time efficiency o Comparisons: n(n-1)/2o Exchanges: n (swapping largest into place)o Overall: (n2), best and worst cases

Space efficiency o Space for the input sequence, plus a constant number of local variables

4

Order of Magnitude – Order n2

All functions with highest-order term cn2 have similar shape An algorithm that does cn2 work for any constant c is order of magnitude n2,

or (n2) Anything that is (n2) will eventually have larger values than anything that is

(n), no matter what the constants are An algorithm that runs in time (n) will outperform one that runs in (n2)

Work = cn2 for Various Values of c

A Comparison of n and n2

5

Analysis of Algorithms Multiple algorithms for one task may be compared for efficiency and other

desirable attributes Data cleanup problem Search problem Pattern matching

Data Cleanup Algorithms Given a collection of numbers, find and remove all zeros Possible algorithms

o Shuffle-lefto Copy-overo Converging-pointers

What are they?o A systematic strategy for removing errors from data.

Why are they important?o Errors occur in all real computing situations.

How are they related to the search algorithm?o To remove errors from a series of values, each value must be examined

to determine if it is an error. Example

o suppose we have a list d of data values, from which we want to remove all the zeroes (they mark errors), and pack the good values to the left. Legit is the number of good values remaining when we are done.

6

d1 d2 d3 d4 d5 d6 d7 d8

5 3 4 0 6 2 4 0Legit

The Shuffle-Left Algorithm go over the list from left to right. Every time we see a zero, shift all subsequent elements one position to the left.

Keep track of legitimate (non-zero) entries

The Shuffle-Left Algorithm for Data Cleanup

Time efficiencyo Count examinations of list values and shiftso Best case

No shifts, n examinations (n)

o Worst case Shift at each pass, n passes n2 shifts plus n examinations (n2)

Space efficiencyo n slots for n values, plus a few local variableso (n)

7

The Copy-Over Algorithm Uses a second list

o Copy over each nonzero element in turn Time efficiency

o Count examinations and copieso Best case

All zeros n examinations and 0 copies (n)

o Worst case No zeros n examinations and n copies (n)

The Copy-Over Algorithm for Data Cleanup

// Draw what this algorithm would look like after a few iterations (collected)

Space efficiencyo 2n slots for n values, plus a few extraneous variables

Time/space tradeoffo Algorithms that solve the same problem offer a tradeoff

One algorithm uses more time and less memory Its alternative uses less time and more memory

The Converging-Pointers Algorithm8

Swap zero values from left with values from right until pointers converge in the middle

In plain English:o One finger moving left to right, one moving right to lefto Move left finger over non-zero values;o If encounter a zero value then o Copy element at right finger into this positiono Shift right finger to the left

The Converging-Pointers Algorithm for Data Cleanup

// Draw what this algorithm would look like after a few iterations (collected)

Time efficiencyo Count examinations and swapso Best case

n examinations, no swaps (n)

o Worst case n examinations, n swaps (n)

Space efficiencyo n slots for the values, plus a few extra variablesAnalysis of Three Data Cleanup Algorithms

9

Binary Search Algorithm Given ordered data

o Search for NAME by comparing to middle elemento If not a match, restrict search to either lower or upper half onlyo Each pass eliminates half the data

Binary Search Algorithm (list must be sorted)

Efficiency o Best case

1 comparison (1)

o Worst case lg n comparisons

10

lg n: The number of times n can be divided by two before reaching 1

(lg n) Tradeoff

o Sequential search Slower, but works on unordered data

o Binary search Faster (much faster), but data must be sorted first

A Comparison of n and lg n

Pattern-Matching Algorithm Analysis involves two measures of input size

o m: length of pattern string o n: length of text string

Unit of worko Comparison of a pattern character with a text character

Efficiencyo Best case

Pattern does not match at all n - m + 1 comparisons (n)

o Worst case Pattern almost matches at each point (m -1)(n - m + 1) comparisons

11

(m x n)

Order-of-Magnitude Time Efficiency Summary

When Things Get Out of Hand Polynomially bound algorithms

o Work done is no worse than a constant multiple of n2

Intractable algorithmso Run in worse than polynomial timeo Examples

Hamiltonian circuit Bin-packing

Exponential algorithmo (2n)o More work than any polynomial in n

Approximation algorithmso Run in polynomial time but do not give optimal solutions

12

Comparisons of lg n, n, n2 , and 2n

A Comparison of Four Orders of Magnitude

Sources:

http://www.personal.kent.edu/~aguercio/lab51/Labs/Lab5/Lab5inst.html

13