# chapter 5 greedy algorithms

Post on 02-Jan-2016

86 views

Embed Size (px)

DESCRIPTION

Chapter 5 Greedy Algorithms. Optimization Problems. Optimization problem: a problem of finding the best solution from all feasible solutions. Two common techniques: Greedy Algorithms (local) Dynamic Programming (global). Greedy Algorithms. Greedy algorithms typically consist of - PowerPoint PPT PresentationTRANSCRIPT

PowerPoint Presentation

Chapter 5Greedy AlgorithmsOptimization ProblemsOptimization problem: a problem of finding the best solution from all feasible solutions.

Two common techniques:Greedy Algorithms (local)Dynamic Programming (global)

Greedy AlgorithmsGreedy algorithms typically consist of

A set of candidate solutions Function that checks if the candidates are feasibleSelection function indicating at a given time which is the most promising candidate not yet usedObjective function giving the value of a solution; this is the function we are trying to optimizeStep by Step ApproachInitially, the set of chosen candidates is emptyAt each step, add to this set the best remaining candidate; this is guided by selection function.If increased set is no longer feasible, then remove the candidate just added; else it stays.Each time the set of chosen candidates is increased, check whether the current set now constitutes a solution to the problem.

When a greedy algorithm works correctly, the first solution found in this way is always optimal.Examples of Greedy AlgorithmsGraph AlgorithmsBreath First Search (shortest path 4 un-weighted graph)Dijkstras (shortest path) AlgorithmMinimum Spanning TreesData compressionHuffman codingSchedulingActivity SelectionMinimizing time in systemDeadline schedulingOther HeuristicsColoring a graphTraveling SalesmanSet-coveringElements of Greedy StrategyGreedy-choice property: A global optimal solution can be arrived at by making locally optimal (greedy) choices

Optimal substructure: an optimal solution to the problem contains within it optimal solutions to sub-problemsBe able to demonstrate that if A is an optimal solution containing s1, then the set A = A - {s1} is an optimal solution to a smaller problem w/o s1. AnalysisThe selection function is usually based on the objective function; they may be identical. But, often there are several plausible ones.

At every step, the procedure chooses the best candidate, without worrying about the future. It never changes its mind: once a candidate is included in the solution, it is there for good; once a candidate is excluded, its never considered again.

Greedy algorithms do NOT always yield optimal solutions, but for many problems they do.Huffman CodingHuffman codes - very effective technique for compressing data, saving 20% - 90%.

8Another application of binary treeCodingProblem:Consider a data file of 100,000 charactersYou can safely assume that there are many a,e,i,o,u, blanks, newlines, few q, x, zsWant to store it compactly

Solution:Fixed-length code, ex. ASCII, 8 bits per character Variable length code, Huffman code (Can take advantage of relative freq of letters to save space)Example2ZTotal BitsCodeFrequencyChar7K24M32C37U 42D42L111110101100011010001000120E Fixed-length code, need ? bits for each char69182172961111261263603Example (cont.)37L:42U:37C:32M:24K:7Z:2D:42E:1200 0 0 0 0 1 1 1 1 1 1 1 0 0 ELDUCMKZ000001010011100101110111CharCode Complete binary treeExample (cont.)Variable length code (Can take advantage of relative freq of letters to save space)

- Huffman codes

ELDUCMKZCharCodeHuffman Tree Construction (1)Associate each char with weight (= frequency) to form a subtree of one node (char, weight)Group all subtrees to form a forest Sort subtrees by ascending weight of subrootsMerge the first two subtrees (ones with lowest weights)Assign weight of subroot with sum of weights of two children.Repeat 3,4,5 until only one tree in the forestHuffman Tree Construction (2)

14Huffman Tree Construction (3)

M15Assigning CodesCompare with: 918~15% less

1111001001101111011111101011110BitsCode2Z37U42L7K24M120E42D32CFreqchar127851111264212012012612816No code is a prefix of anothe code --- prefix codeC: 1110D: 101E: 0M: 11111K: 111101L: 110U: 100Z: 111100Huffman Coding Tree

Coding and DecodingDEED: MUCK: ELDUCMKZ000001010011100101110111CharCodeELDUCMKZ0110101100111011111111101111100CharCodeDEED:MUCK:0100000000101010111001101010010111111100111011110118Why code for Muck longer?How to decodePrefix codesA set of codes is said to meet the prefix property if no code in the set is the prefix of another. Such codes are called prefix codes.Huffman codes are prefix codes.

ELDUCMKZ0110101100111011111111101111100CharCode19Code for DEED: 101 0 0 101Decode: DUCKExpected cost: (1 * 120 + 3 * 121 + 4 * 32 + 5 * 24 + 6 * 9)/ 306 = 785/306 = 2.57Coin ChangingCoin ChangingGoal. Given currency denominations: 1, 5, 10, 25, 100, devise a method to pay amount to customer using fewest number of coins.

Ex: 34.

Cashier's algorithm. At each iteration, add coin of the largest value that does not take us past the amount to be paid.

Ex: $2.89.

21most of us solve this kind of problem every day without thinking twice, unconsciously using an obvious greedy algorithmhttp://www.frbatlanta.org/publica/brochure/fundfac/html/coinfaces.htmlCoin-Changing: Greedy AlgorithmCashier's algorithm. At each iteration, add coin of the largest value that does not take us past the amount to be paid.

Q. Is cashier's algorithm optimal?Sort coins denominations by value: c1 < c2 < < cn.

S while (x 0) { let k be largest integer such that ck x if (k = 0) return "no solution found" x x - ck S S {k}}return Scoins selected 22takes O(n log n) + |S| where S is the number of coins taken by greedyNote: can be sped up somewhat by using integer divisionCoin-Changing: Analysis of Greedy AlgorithmObservation. Greedy algorithm is sub-optimal for US postal denominations: 1, 10, 21, 34, 37, 44, 70, 100, 350, 1225, 1500.

Counterexample. 140.Greedy: 100, 37, 1, 1, 1.Optimal: 70, 70.Greedy algorithm failed!

23note: may not even lead to *feasible* solution if c_1 > 1!Ex. C = {7, 8, 9}, x = 15Coin-Changing: Analysis of Greedy AlgorithmTheorem. Greed is optimal for U.S. coinage: 1, 5, 10, 25, 100.Proof. (by induction on x)Let ck be the kth smallest coinConsider optimal way to change ck x < ck+1 : greedy takes coin k.We claim that any optimal solution must also take coin k.if not, it needs enough coins of type c1, , ck-1 to add up to xtable below indicates no optimal solution can do thisProblem reduces to coin-changing x - ck cents, which, by induction, is optimally solved by greedy algorithm. 1ck1025100P 4All optimal solutionsmust satisfyN + D 2Q 35N 1no limitk13452-Max value of coins1, 2, , k-1 in any OPT4 + 5 = 920 + 4 = 24475 + 24 = 9924P = pennies, N = nickels, D = dimes, Q = quartersCoin-Changing: Analysis of Greedy AlgorithmTheorem. Greed is optimal for U.S. coinage: 1, 5, 10, 25, 100.Consider optimal way to change ck x < ck+1 : greedy takes coin k.We claim that any optimal solution must also take coin k.

1ck1025100P 4All optimal solutionsmust satisfyN + D 2Q 35N 1no limitk13452-Max value of coins1, 2, , k-1 in any OPT4 + 5 = 920 + 4 = 24475 + 24 = 991ck1025100P 9All optimal solutionsmust satisfyP + D 8Q 3no limitk1234-Max value of coins1, 2, , k-1 in any OPT940 + 4 = 4475 + 44 = 119Kevins problem25P = pennies, N = nickels, D = dimes, Q = quartersKnapsack ProblemKnapsack Problem0-1 knapsack: A thief robbing a store finds n items; the ith item is worth vi dollars and weighs wi pounds, where vi and wi are integers. He wants to take as valuable a load as possible, but he can only carry at most W pounds. What items should he take? Dynamic programming

Fractional knapsack: Same set up. But, the thief can take fractions of items, instead of making a binary (0-1) choice for each item.Fractional Knapsack ProblemGreedy method: repeatedly add item with max ratio vi/wi.

Value:1Value1822281Weight56627Item13452Let W=11Answer for 0-1 knapsack problem:

OPT: { 4, 3 }value = 22 + 18 = 40> 45Activity-selection ProblemActivity-selection ProblemInput: Set S of n activities, a1, a2, , an.si = start time of activity i.fi = finish time of activity i.Output: Subset A of max # of compatible activities.Two activities are compatible, if their intervals dont overlap.Time01234567891011fgheabcdInterval Scheduling: Greedy AlgorithmsGreedy template. Consider jobs in some order. Take each job provided it's compatible with the ones already taken.Earliest start time: Consider jobs in ascending order of start time sj.Earliest finish time: Consider jobs in ascending order of finish time fj.Shortest interval: Consider jobs in ascending order of interval length fj - sj.Fewest conflicts: For each job, count the number of conflicting jobs cj. Schedule in ascending order of conflicts cj.Greedy algorithm. Consider jobs in increasing order of finish time. Take each job provided it's compatible with the ones already taken.

Implementation. O(n log n).Remember job j* that was added last to A.Job j is compatible with A if sj fj*.Interval Scheduling: Greedy AlgorithmSort jobs by finish times so that f1 f2 ... fn.

A for j = 1 to n { if (job j compatible with A) A A {j}}return A jobs selected 32Interval Scheduling: AnalysisTheorem. Greedy algorithm is optimal.Proof: (by contradiction)Assume greedy is not optimal, and let's see what happens.Let i1, ... ik denote set of jobs selected by greedy.Let j1, ... jm denote set of jobs in the optimal solution withi1 = j1, i2 = j2, ..., ir = jr for the largest possible value of r. j1j2jri1i1irir+1. . .Greedy:OPT:jr+1why not replace job jr+1 with job ir+1?job ir+1 finishes before jr+1ir+1Still optimal with a bigger value than r : ir+1=jr+1 contradiction!33Weighted Interval SchedulingWeighted interval scheduling problem.Job j starts at sj, finishes at fj, and has weight or value vj . Two jobs compatib