5-1-5-1-11

CSC401 – Analysis of AlgorithmsCSC401 – Analysis of Algorithms Chapter 5--1Chapter 5--1

The Greedy MethodThe Greedy MethodObjectivesObjectives

Introduce the Brute Force method and Introduce the Brute Force method and the Greedy Methodthe Greedy MethodCompare the solutions to the same Compare the solutions to the same problem using different methodsproblem using different methodsFractional Knapsack problemFractional Knapsack problemTask scheduling problemTask scheduling problem

5-1-5-1-22

Brute ForceBrute ForceA straightforward approach, usually based directly A straightforward approach, usually based directly on the problem’s statement and definitions of the on the problem’s statement and definitions of the concepts involvedconcepts involved

Examples:Examples:1.1. Computing Computing aan n ((a a > 0, > 0, nn a nonnegative integer) a nonnegative integer)

2.2. Computing Computing nn!!

3.3. Multiplying two matricesMultiplying two matrices

4.4. Selection sortSelection sort

5.5. Searching for a key of a given value in a listSearching for a key of a given value in a list

5-1-5-1-33

Brute-Force Sorting AlgorithmBrute-Force Sorting Algorithm

Selection SortSelection Sort Scan the array to find its smallest Scan the array to find its smallest element and swap it with the first element. Then, starting element and swap it with the first element. Then, starting with the second element, scan the elements to the right of it with the second element, scan the elements to the right of it to find the smallest among them and swap it with the second to find the smallest among them and swap it with the second elements. Generally, on pass elements. Generally, on pass i i (0 (0 i i n-n-2), find the smallest 2), find the smallest element in element in AA[[i..n-i..n-1] and swap it with 1] and swap it with AA[[ii]:]:

AA[0] [0] . . . . . . AA[[ii-1] | -1] | AA[[ii], . . . , ], . . . , AA[[minmin], . . ., ], . . ., AA[[nn-1] -1]

in their final positionsin their final positions

Example: 7 3 2 5Example: 7 3 2 5

5-1-5-1-44

Analysis of Selection SortAnalysis of Selection Sort

Time efficiency: (n-1)n/2 Time efficiency: (n-1)n/2 O(n O(n22))

Space efficiency: 2 Space efficiency: 2 O(1) O(1)

5-1-5-1-55

Brute-Force String MatchingBrute-Force String Matchingpatternpattern: a string of : a string of mm characters to search for characters to search for

texttext: a (longer) string of : a (longer) string of nn characters to search in characters to search in

problem: find a substring in the text that matches problem: find a substring in the text that matches the patternthe pattern

Brute-force algorithmBrute-force algorithmStep 1 Align pattern at beginning of textStep 1 Align pattern at beginning of text

Step 2 Moving from left to right, compare each character ofStep 2 Moving from left to right, compare each character of pattern to the corresponding character in text until pattern to the corresponding character in text until

– all characters are found to match (successful search); orall characters are found to match (successful search); or– a mismatch is detecteda mismatch is detected

Step 3 While pattern is not found and the text is not yetStep 3 While pattern is not found and the text is not yet exhausted, realign pattern one position to the right exhausted, realign pattern one position to the right andand repeat Step 2 repeat Step 2

5-1-5-1-66

Brute-Force String MatchingBrute-Force String Matching: : Examples Examples

1.1. Pattern: Pattern: 001011 001011 Text: Text: 10010101101001100101111010 10010101101001100101111010

2.2. Pattern: Pattern: happyhappy Text: Text: It It is never too late to have a happy is never too late to have a happy childhood.childhood.

5-1-5-1-77

Pseudocode and Efficiency Pseudocode and Efficiency

Efficiency: O(nm)Efficiency: O(nm)

5-1-5-1-88

Brute-Force Polynomial EvaluationBrute-Force Polynomial EvaluationProblem: Find the value of polynomial Problem: Find the value of polynomial

pp((xx) = ) = aannxxnn + + aann-1-1xxnn-1 -1 +… ++… + a a11xx1 1 + + aa0 0

at a point at a point xx = = xx00

Brute-force algorithmBrute-force algorithm

Efficiency: O(nEfficiency: O(n22))

pp 0.00.0for for ii nn downto 0 do downto 0 do powerpower 1 1

for for jj 1 to 1 to ii do do //compute //compute xxii powerpower powerpower xx pp pp + + aa[[ii] ] powerpower

returnreturn pp

5-1-5-1-99

Polynomial Evaluation: ImprovementPolynomial Evaluation: Improvement

We can do better by evaluating from right to left:We can do better by evaluating from right to left:

Better brute-force algorithmBetter brute-force algorithm

Efficiency: O(n)Efficiency: O(n)

pp aa[0][0]powerpower 1 1forfor ii 1 1 toto nn dodo

powerpower powerpower x x pp p p + + aa[[ii] ] powerpowerreturnreturn pp

5-1-5-1-1010

Closest-Pair ProblemClosest-Pair ProblemFind the two closest points in a set Find the two closest points in a set of of nn points (in the two-dimensional points (in the two-dimensional Cartesian plane).Cartesian plane).

Brute-force algorithmBrute-force algorithm– Compute the distance between every Compute the distance between every

pair of distinct points and return the pair of distinct points and return the indexes of the points for which the indexes of the points for which the distance is the smallest.distance is the smallest.

5-1-5-1-1111

Closest-Pair Brute-Force Algorithm Closest-Pair Brute-Force Algorithm (cont.)(cont.)

Efficiency:Efficiency: O(n O(n22)) How to make it faster?How to make it faster?

5-1-5-1-1212

Convex hull Convex hull ProblemProblem– Find smallest convex polygon Find smallest convex polygon

enclosing n points on the planeenclosing n points on the plane

AlgorithmAlgorithm– Find each pair of points p1 and p2, Find each pair of points p1 and p2,

determine whether all other points lie determine whether all other points lie to the same side of the straight line to the same side of the straight line through p1 and p2through p1 and p2

Efficiency: Efficiency: – O(n3)

5-1-5-1-1313

Brute-Force Strengths and WeaknessesBrute-Force Strengths and WeaknessesStrengthsStrengths– wide applicabilitywide applicability– simplicitysimplicity– yields reasonable algorithms for some yields reasonable algorithms for some

important problemsimportant problems(e.g., matrix multiplication, sorting, searching, (e.g., matrix multiplication, sorting, searching, string matching)string matching)

WeaknessesWeaknesses– rarely yields efficient algorithms rarely yields efficient algorithms – some brute-force algorithms are unacceptably some brute-force algorithms are unacceptably

slow slow – not as constructive as some other design not as constructive as some other design

techniquestechniques

5-1-5-1-1414

Exhaustive SearchExhaustive SearchA brute force solution to a problem involving A brute force solution to a problem involving search for an element with a special property, search for an element with a special property, usually among combinatorial objects such as usually among combinatorial objects such as permutations, combinations, or subsets of a set.permutations, combinations, or subsets of a set.Method:Method:– generate a list of all potential solutions to the generate a list of all potential solutions to the

problem in a systematic manner problem in a systematic manner – evaluate potential solutions one by one, evaluate potential solutions one by one,

disqualifying infeasible ones and, for an disqualifying infeasible ones and, for an optimization problem, keeping track of the best optimization problem, keeping track of the best one found so farone found so far

– when search ends, announce the solution(s) when search ends, announce the solution(s) foundfound

5-1-5-1-1515

Example 1: Traveling Salesman ProblemExample 1: Traveling Salesman Problem Given Given nn cities with known distances between cities with known distances between each pair, find the shortest tour that passes each pair, find the shortest tour that passes through all the cities exactly once before through all the cities exactly once before returning to the starting cityreturning to the starting city

Alternatively: Find shortest Alternatively: Find shortest Hamiltonian circuitHamiltonian circuit in in a weighted connected grapha weighted connected graph

Example:Example: a b

c d

8

2

7

5 34 TourTour Cost Cost

a→a→bb→→cc→→dd→→a 2+3+7+5 = 17a 2+3+7+5 = 17a→a→bb→→dd→→cc→→a 2+4+7+8 = 21a 2+4+7+8 = 21a→a→cc→→bb→→dd→→a 8+3+4+5 = 20a 8+3+4+5 = 20a→a→cc→→dd→→bb→→a 8+7+4+2 = 21a 8+7+4+2 = 21a→a→dd→→bb→→cc→→a 5+4+3+8 = 20a 5+4+3+8 = 20a→a→dd→→cc→→bb→→a 5+7+3+2 = 17a 5+7+3+2 = 17

5-1-5-1-1616

Example 2: Knapsack ProblemExample 2: Knapsack ProblemGiven Given nn items: items:– weights: weights: ww11ww2 2 …w…wnn

– values: values: vv11vv22 … v … vnn

– a knapsack of a knapsack of capacity capacity W W

Find most valuable Find most valuable subset of the items that subset of the items that fit into the knapsackfit into the knapsackEfficiency: O(2Efficiency: O(2nn))Example: Knapsack Example: Knapsack capacity W=16capacity W=16

item weight valueitem weight value11 2 $202 $2022 5 $305 $3033 10 $5010 $5044 5 $105 $10

SubsetSubset W Weighteight V Valuealue {1} 2 $20{1} 2 $20 {2} 5 $30{2} 5 $30 {3} 10 $50{3} 10 $50 {4} 5 $10{4} 5 $10 {1,2} 7 $50{1,2} 7 $50 {1,3} 12 $70{1,3} 12 $70 {1,4} 7 $30{1,4} 7 $30 {2,3} 15 $80{2,3} 15 $80 {2,4} 10 $40{2,4} 10 $40 {3,4} 15 $60{3,4} 15 $60 {1,2,3} 17 not feasible{1,2,3} 17 not feasible {1,2,4} 12 $60{1,2,4} 12 $60 {1,3,4} 17 not feasible{1,3,4} 17 not feasible {2,3,4} 20 not feasible{2,3,4} 20 not feasible{1,2,3,4} 22 not feasible{1,2,3,4} 22 not feasible

5-1-5-1-1717

Example 3: The Assignment ProblemExample 3: The Assignment Problem

There are There are n n people who need to be assigned to people who need to be assigned to nn jobs, one jobs, one person per job. The cost of assigning person person per job. The cost of assigning person i i to job to job jj is C[ is C[ii,,jj]. ]. Find an assignment that minimizes the total cost.Find an assignment that minimizes the total cost.

Job 0 Job 1 Job 2 Job 3Job 0 Job 1 Job 2 Job 3Person 0 9Person 0 9 2 7 8 2 7 8Person 1 6 4 3 7Person 1 6 4 3 7Person 2 5 8 1 8Person 2 5 8 1 8Person 3 7 6 9 4Person 3 7 6 9 4

Algorithmic Plan:Algorithmic Plan: Generate all legitimate assignments, compute Generate all legitimate assignments, compute their costs, and select the cheapest one.their costs, and select the cheapest one.How many assignments are there? How many assignments are there?

AssignmentAssignment (col.#s) (col.#s) Total CostTotal Cost 1, 2, 3, 41, 2, 3, 4 9+4+1+4=189+4+1+4=18 1, 2, 4, 31, 2, 4, 3 9+4+8+9=309+4+8+9=30 1, 3, 2, 41, 3, 2, 4 9+3+8+4=249+3+8+4=24 1, 3, 4, 21, 3, 4, 2 9+3+8+6=269+3+8+6=26 1, 4, 2, 31, 4, 2, 3 9+7+8+9=339+7+8+9=33 1, 4, 3, 21, 4, 3, 2 9+7+1+6=239+7+1+6=23 etc.etc.

5-1-5-1-1818

Final Comments on Exhaustive SearchFinal Comments on Exhaustive Search

Exhaustive-search algorithms run in a realistic Exhaustive-search algorithms run in a realistic amount of time amount of time only on very small instancesonly on very small instances

In some cases, there are much better In some cases, there are much better alternatives! alternatives! – Euler circuitsEuler circuits– shortest pathsshortest paths– minimum spanning treeminimum spanning tree– assignment problemassignment problem

In many cases, exhaustive search or its variation In many cases, exhaustive search or its variation is the only known way to get exact solutionis the only known way to get exact solution

5-1-5-1-1919

The Greedy Method TechniqueThe Greedy Method TechniqueThe greedy methodThe greedy method is a general algorithm is a general algorithm design paradigm, built on the following elements:design paradigm, built on the following elements:– configurationsconfigurations: different choices, collections, : different choices, collections,

or values to findor values to find– objective functionobjective function: a score assigned to : a score assigned to

configurations, which we want to either configurations, which we want to either maximize or minimizemaximize or minimize

It works best when applied to problems with It works best when applied to problems with the the greedy-choicegreedy-choice property: property: – a globally-optimal solution can always be found a globally-optimal solution can always be found

by a series of local improvements from a by a series of local improvements from a starting configuration.starting configuration.

5-1-5-1-2020

Making ChangeMaking ChangeProblem:Problem: A dollar amount to reach and a collection A dollar amount to reach and a collection of coin amounts to use to get there.of coin amounts to use to get there.Configuration:Configuration: A dollar amount yet to return to a A dollar amount yet to return to a customer plus the coins already returnedcustomer plus the coins already returnedObjective function:Objective function: Minimize number of coins Minimize number of coins returned.returned.Greedy solution:Greedy solution: Always return the largest coin you Always return the largest coin you cancanExample 1:Example 1: Coins are valued $.25, $.10, $0.05, $0.01 Coins are valued $.25, $.10, $0.05, $0.01– Has the greedy-choice property, since no amount over $.25 Has the greedy-choice property, since no amount over $.25

can be made with a minimum number of coins by omitting a can be made with a minimum number of coins by omitting a $.25 coin (similarly for amounts over $.10, but under $.25).$.25 coin (similarly for amounts over $.10, but under $.25).

Example 2:Example 2: Coins are valued $.30, $.20, $.05, $.01 Coins are valued $.30, $.20, $.05, $.01– Does not have greedy-choice property, since $.40 is best Does not have greedy-choice property, since $.40 is best

made with two $.20’s, but the greedy solution will pick three made with two $.20’s, but the greedy solution will pick three coins (which ones?)coins (which ones?)

5-1-5-1-2121

The Fractional Knapsack ProblemThe Fractional Knapsack ProblemGiven:Given: A set S of n items, with each item i having A set S of n items, with each item i having– bbii - a positive benefit (value) - a positive benefit (value)– wwii - a positive weight - a positive weight

Goal:Goal: Choose items with maximum total benefit Choose items with maximum total benefit (value) but with weight at most W.(value) but with weight at most W.If we are allowed to take fractional amounts, then If we are allowed to take fractional amounts, then this is the this is the fractional knapsack problemfractional knapsack problem..– In this case, we let xIn this case, we let xi i denote the amount we take of item idenote the amount we take of item i

– Objective: maximizeObjective: maximize

– Constraint:Constraint:

Si

iii wxb )/(

iiSi

i wxWx

0,

5-1-5-1-2222

ExampleExampleGiven: A set S of n items, with each item i havingGiven: A set S of n items, with each item i having– bbii - a positive benefit - a positive benefit– wwii - a positive weight - a positive weight

Goal: Choose items with maximum total benefit but Goal: Choose items with maximum total benefit but with total weight at most W.with total weight at most W.

Weight:Benefit:

1 2 3 4 5

4 ml 8 ml 2 ml 6 ml 1 ml

$12 $32 $40 $30 $50

Items:

Value: 3($ per ml)

4 20 5 5010 ml

Solution:• 1 ml of 5• 2 ml of 3• 6 ml of 4• 1 ml of 2

“knapsack”

5-1-5-1-2323

The Fractional Knapsack AlgorithmThe Fractional Knapsack AlgorithmGreedy choice: Keep taking item with highest Greedy choice: Keep taking item with highest valuevalue (benefit to weight ratio) (benefit to weight ratio)– Since Since

Algorithm fractionalKnapsack(S, W)

Input: set S of items w/ benefit bi and weight wi; max. weight W

Output: amount xi of each item i to maximize benefit w/ weight at most W

for each item i in S

xi 0

vi bi / wi {value}w 0 {total weight}

while w < W

remove item i with highest vi

xi min{wi , W - w}

w w + xi

Si

iiiSi

iii xwbwxb )/()/(

5-1-5-1-2424

The Fractional Knapsack AlgorithmThe Fractional Knapsack AlgorithmRunning time:Running time: Given a collection S of n items, such Given a collection S of n items, such that each item i has a benefit bthat each item i has a benefit bii and weight w and weight wii, we , we can construct a maximum-benefit subset of S, can construct a maximum-benefit subset of S, allowing for fractional amounts, that has a total allowing for fractional amounts, that has a total weight W in O(nlogn) time.weight W in O(nlogn) time. – Use heap-based priority queue to store SUse heap-based priority queue to store S– Removing the item with the highest value takes Removing the item with the highest value takes

O(logn) timeO(logn) time– In the worst case, need to remove all itemsIn the worst case, need to remove all itemsCorrectness: Correctness: Suppose there is a better solutionSuppose there is a better solution– there is an item i with higher value than a chosen there is an item i with higher value than a chosen

item j, but xitem j, but xii<w<wii, x, xjj>0 and v>0 and vii<v<vjj

– If we substitute some i with j, we get a better If we substitute some i with j, we get a better solutionsolution

– How much of i: min{wHow much of i: min{wii-x-xii, x, xjj}}– Thus, there is no better solution than the greedy Thus, there is no better solution than the greedy

oneone

5-1-5-1-2525

Task SchedulingTask SchedulingGiven: a set T of n tasks, each having:Given: a set T of n tasks, each having:– A start time, sA start time, sii

– A finish time, fA finish time, fii (where s (where sii < f < fii))

Goal: Perform all the tasks using a minimum Goal: Perform all the tasks using a minimum number of “machines.”number of “machines.”

1 98765432

Machine 1

Machine 3

Machine 2

5-1-5-1-2626

Task Scheduling AlgorithmTask Scheduling AlgorithmGreedy choice: consider tasks by their start time and Greedy choice: consider tasks by their start time and use as few machines as possible with this order.use as few machines as possible with this order.

Algorithm taskSchedule(T)

Input: set T of tasks with start time si and finish time fi

Output: non-conflicting schedule with minimum number of machines

m 0 {no. of machines}

while T is not empty

remove task i with smallest si

if there’s a machine j for i thenschedule i on machine j

else m m + 1schedule i on machine m

5-1-5-1-2727

Task Scheduling AlgorithmTask Scheduling AlgorithmRunning time:Running time: Given a set of n tasks specified by Given a set of n tasks specified by their start and finish times, Algorithm TaskSchedule their start and finish times, Algorithm TaskSchedule produces a schedule of the tasks with the minimum produces a schedule of the tasks with the minimum number of machines in O(nlogn) time.number of machines in O(nlogn) time.– Use heap-based priority queue to store tasks with the start Use heap-based priority queue to store tasks with the start

time as the prioritiestime as the priorities– Finding the earliest task takes O(logn) timeFinding the earliest task takes O(logn) time

Correctness:Correctness: (proof by contradiction) Suppose there (proof by contradiction) Suppose there is a better schedule.is a better schedule.– We can use k-1 machinesWe can use k-1 machines– The algorithm uses kThe algorithm uses k– Let i be first task scheduled on machine kLet i be first task scheduled on machine k– Machine i must conflict with k-1 other tasksMachine i must conflict with k-1 other tasks– But that means there is no non-conflicting But that means there is no non-conflicting

schedule using k-1 machinesschedule using k-1 machines

5-1-5-1-2828

ExampleExampleGiven: a set T of n tasks, each having:Given: a set T of n tasks, each having:– A start time, sA start time, sii

– A finish time, fA finish time, fii (where s (where sii < f < fii))– [1,4], [1,3], [2,5], [3,7], [4,7], [6,9], [7,8] (ordered by start)[1,4], [1,3], [2,5], [3,7], [4,7], [6,9], [7,8] (ordered by start)

Goal: Perform all tasks on min. number of machinesGoal: Perform all tasks on min. number of machines

1 98765432

Machine 1

Machine 3

Machine 2