![Page 1: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/1.jpg)
Greedy Algorithms
Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!
![Page 2: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/2.jpg)
CSE 3101 2
Optimization Problems
• Shortest path is an example of an optimization problem: we wish to find the path with lowest weight.
• What is the general character of an optimization problem?
![Page 3: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/3.jpg)
CSE 3101 3
Ingredients:
•Instances: The possible inputs to the problem.
•Solutions for Instance: Each instance has an exponentially large set of valid solutions.
•Cost of Solution: Each solution has an easy-to-compute cost or value.
Specification
•Preconditions: The input is one instance.
•Postconditions: A valid solution with optimal cost. (minimum or maximum)
Optimization Problems
![Page 4: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/4.jpg)
CSE 3101 4
Greedy Solutions to Optimization Problems
Surprisingly, many important and practical optimization problems can be solved this way.
Every two-year-old knows the greedy algorithm.
In order to get what you want, just start grabbing what looks best.
![Page 5: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/5.jpg)
CSE 3101 5
Example 1: Making Change
Problem: Find the minimum # of quarters, dimes, nickels, and pennies that total to a given amount.
![Page 6: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/6.jpg)
CSE 3101 6
The Greedy Choice
Commit to the object that looks the ``best''
Must prove that this locally greedy choice does not have negative global consequences.
![Page 7: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/7.jpg)
CSE 3101 7
Instance: A drawer full of coins and an amount of change to return
25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢
10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢
5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢
1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢
Amount = 92¢
Solutions for Instance: A subset of the coins in the drawer that total the amount
Making Change Example
![Page 8: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/8.jpg)
CSE 3101 8
Instance: A drawer full of coins and an amount of change to return
Solutions for Instance: A subset of the coins that total the amount.
25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢
10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢
5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢
1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢
Amount = 92¢
Cost of Solution: The number of coins in the solution = 14
Making Change Example
Goal: Find an optimal valid solution.
![Page 9: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/9.jpg)
CSE 3101 9
25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢
10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢
5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢
1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢
Amount = 92¢
Making Change Example
Greedy Choice:
Does this lead to an optimal # of coins?
Start by grabbing quarters until exceeds amount, then dimes, then nickels, then pennies.
Cost of Solution: 7
Instance: A drawer full of coins and an amount of change to return
![Page 10: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/10.jpg)
CSE 3101 10
Hard Making Change Example
Greedy Choice: Start by grabbing a 4-cent coin.
Problem: Find the minimum # of 4, 3, and 1 cent coins to make up 6 cents.
Consequences: 4+1+1 = 6 mistake 3+3=6 better
Greedy Algorithm does not work!
![Page 11: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/11.jpg)
CSE 3101 11
When Does It Work?
• Greedy Algorithms: Easy to understand and to code, but do they work?
• For most optimization problems, all greedy algorithms tried do not work (i.e. yield sub-optimal solutions)
• But some problems can be solved optimally by a greedy algorithm.
• The proof that they work, however, is subtle.
• As with all iterative algorithms, we use loop invariants.
![Page 12: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/12.jpg)
CSE 3101 12
Designing an Algorithm Define Problem Define Loop Invariants Define Measure of
Progress
Define Step Define Exit Condition Maintain Loop Inv
Make Progress Initial Conditions Ending
km
79 km
to school
Exit
Exit
79 km 75 km
Exit
Exit
0 km Exit
![Page 13: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/13.jpg)
CSE 3101 13
The algorithm chooses the “best” object from amongst those not considered so far and either commits to it or rejects it.
Define Step
Another object considered
Make Progress
79 km 75 km
Exit
All objects have been consideredExit
Exit Condition
![Page 14: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/14.jpg)
CSE 3101 14
Designing a Greedy Algorithm
while exit condition
pre-condition
lo
loop
end loop
CodeA
CodeB
C
op-invariant>
post-condit
deC
ion
o
![Page 15: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/15.jpg)
CSE 3101 15
We have not gone wrong. There is at least one optimal solution consistent with the choices made so far.
Loop Invariant
![Page 16: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/16.jpg)
CSE 3101 16
Initially no choices have been made and hence all optimal solutions are consistent with these choices.
<preCond>codeA
<loop-invariant>
Establishing Loop Invariant
Establishing the Loop Invariant
![Page 17: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/17.jpg)
CSE 3101 17
Maintaining Loop Invariant
loop-invarian CodeBMus t loot s p-ihow that nvar iant
LILI : optimal solution consistent with choices soOptS far
Commit to or reject next CodeB : object
Ours optimal soln consistent with prev objects + neLI : w obOptS ject
Ours LImay or may not be the same asOptS Op: tS ! Note
Proof must massage optSLI into optSours and prove that optSours:
• is a valid solution • is consistent both with previous and new choices.• is optimal
![Page 18: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/18.jpg)
CSE 3101 18
Algorithm:commits to or rejects next best object
His actions are not part of the algorithm
The algorithm and prover do not know optSLI.
Prover:Proves LI is maintained.
Three Players
Fairy God Mother:Holds the hypothetical optimal sol optSLI.
optSLI
![Page 19: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/19.jpg)
CSE 3101 19
Proving the Loop Invariant is Maintained
• We need to show that the action taken by the algorithm maintains the loop invariant.
• There are 2 possible actions:
– Case 1. Commit to current object
– Case 2. Reject current object
![Page 20: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/20.jpg)
Case 1. Committing to Current Object
![Page 21: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/21.jpg)
CSE 3101 21
I instruct how to massage optSLI into optSours so that it is consistent with previous
& new choice.
Massaging optSLI into optSours
25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢
10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢
5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢
1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢
Amount = 92¢
I have committed to these coins.
I commit to keeping another 25¢
I hold optSLI witnessing that there is an opt sol consistent
with previous choices.
I hold optSours witnessing that there is an opt sol consistent
with previous & new choices.
optSLI
![Page 22: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/22.jpg)
CSE 3101 22
I know that her optSLI
is consistent with these choices.
As Time Goes On
25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢
10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢
5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢
1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢
Amount = 92¢
I keep making more choices.
I always hold an opt sol optSLI but one that keeps
changing.
optSLI
Hence, I know more and more of optSLI
In the end, I know it all.
![Page 23: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/23.jpg)
optSLI
Case 1A. The object we commit to is already part of optSLI
partial solution
new object
oursoptS partial solution
new object
![Page 24: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/24.jpg)
CSE 3101 24
If it happens to be the case that the new object selected is
consistent with the solution held by the fairy godmother,
then we are done.
Massaging optSLI into optSours
25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢
10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢
5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢
1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢
Amount = 92¢
optSLI
![Page 25: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/25.jpg)
Case 1B. The object we commit to is not part of optSLI
optSLIpartial
solution
new object
![Page 26: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/26.jpg)
CSE 3101 26
Case 1B. The object we commit to is not part of optSLI
• This means that our partial solution is not consistent with optSLI.
• The Prover must show that there is a new optimal solution optSours that is consistent with our partial solution.
• This has two parts
– All objects previously committed to must be part of optSours.
– The new object must be part of optSours.
optSLIpartial
solution oursoptS
new object
![Page 27: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/27.jpg)
CSE 3101 27
Case 1B. The object we commit to is not part of optSLI
• Strategy of proof: construct a consistent optSours by replacing one or more objects in optSLI (but not in the partial solution) with another set of objects that includes the current object.
• We must show that the resulting optSours is still
– Valid
– Consistent
– Optimal
optSLIpartial
solution
current object
swap
![Page 28: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/28.jpg)
CSE 3101 28
Case 1B. The object we commit to is not part of optSLI
• Strategy of proof: construct a consistent optSours by replacing one or more objects in optSLI (but not in the partial solution) with another set of objects that includes the current object.
• We must show that the resulting optSours is still
– Valid
– Consistent
– Optimal
optSLIpartial
solution
new object
swap
oursoptS
![Page 29: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/29.jpg)
CSE 3101 29
Massaging optSLI into optSours
optSLI25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢
10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢
5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢
1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢
Amount = 92¢
Replace
•A different 25¢ •Alg’s 25¢With
![Page 30: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/30.jpg)
CSE 3101 30
Massaging optSLI into optSours
optSLI25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢
10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢
5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢
1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢
Amount = 92¢
Replace
•A different 25¢ •Alg’s 25¢With
•3×10¢ •Alg’s 25¢ + 5¢
![Page 31: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/31.jpg)
CSE 3101 31
Massaging optSLI into optSours
optSLI25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢
10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢
5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢
1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢
Amount = 92¢
Replace
•A different 25¢ •Alg’s 25¢With
•3×10¢
•2×10¢ + 1×5¢
•Alg’s 25¢ + 5¢
•Alg’s 25¢
![Page 32: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/32.jpg)
CSE 3101 32
Massaging optSLI into optSours
optSLI25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢
10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢
5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢
1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢
Amount = 92¢
Replace
•A different 25¢ •Alg’s 25¢With
•3×10¢
•2×10¢ + 1×5¢
•Alg’s 25¢ + 5¢
•1×10¢ + 3×5¢
•Alg’s 25¢
•Alg’s 25¢
![Page 33: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/33.jpg)
CSE 3101 33
Massaging optSLI into optSours
optSLI25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢
10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢
5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢
1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢
Amount = 92¢
Replace
•A different 25¢ •Alg’s 25¢With
•?? + 5×1¢
•3×10¢
•2×10¢ + 1×5¢
•Alg’s 25¢ + 5¢
•1×10¢ + 3×5¢
•Alg’s 25¢
•Alg’s 25¢
•Alg’s 25¢
![Page 34: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/34.jpg)
CSE 3101 34
#Coins #Coins1Q 1 1Q 13D 3 1Q 1N 22D 1N 3 1Q 12D 5P 7 1Q 11D 3N 4 1Q 11D 2N 5P 8 1Q 11D 1N 10P 12 1Q 11D 15P 16 1Q 15N 5 1Q 14N 5P 9 1Q 13N 10P 13 1Q 12N 15P 17 1Q 11N 20P 21 1Q 125P 25 1Q 1
OursoptS
Must Consider All Cases
• Note that in all cases our new solution optSours is:
– Valid: the sum is still correct
– Consistent with our previous choices (we do not alter these).
– Optimal: we never add more coins to the solution than we delete
LIoptS
![Page 35: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/35.jpg)
CSE 3101 35
optSLI
Done
optSours
Massaging optSLI into optSours
She now has something.We must prove that it is
what we want.
![Page 36: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/36.jpg)
CSE 3101 36
optSoursoptSours is valid
optSLI was valid and we introduced no new conflicts.
Massaging optSLI into optSours
Total remains unchanged.
Replace• A different 25¢ •Alg’s 25¢
With
•?? + 5×1¢
•3×10¢
•2×10¢ + 1×5¢
•Alg’s 25¢ + 5¢
•1×10¢ + 3×5¢
•Alg’s 25¢
•Alg’s 25¢
![Page 37: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/37.jpg)
CSE 3101 37
optSours is consistent
Massaging optSLI into optSours
optSLI was consistent with previous choices and we made it consistent with new.
25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢
10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢
5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢
1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢
Amount = 92¢
optSours
![Page 38: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/38.jpg)
CSE 3101 38
optSours is optimalWe do not even know the cost of an optimal solution.
Massaging optSLI into optSours
optSLI was optimal and optSours cost (# of coins) is not bigger.
•Alg’s 25¢
With
•?? + 5×1¢
•3×10¢
•2×10¢ + 1×5¢
•Alg’s 25¢ + 5¢
•1×10¢ + 3×5¢
•Alg’s 25¢
•Alg’s 25¢
optSours
Replace• A different 25¢
![Page 39: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/39.jpg)
CSE 3101 39
Committing to Other Coins
• Similarly, we must show that when the algorithm selects a dime, nickel or penny, there is still an optimal solution consistent with this choice.
dimeLI OursoptS optS
nickelL OursIo tSptS op
pennyI OursLo SptS opt
![Page 40: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/40.jpg)
CSE 3101 40
Example: Dimes
– We only commit to a dime when less than 25¢ is unaccounted for.
– Therefore the coins in optSLI that this dime replaces have to be dimes, nickels or pennies.
#Coins #Coins
1D 1 1D 12N 2 1D 11N 5P 6 1D 110P 10 1D 1
OursoptSLIoptS
![Page 41: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/41.jpg)
CSE 3101 41
Committing to Other Coins
• We must consider all possible coins we might select:
– Quarter: Swap for another quarter, 3 dimes (with a nickel) etc.
– Dime: Swap for another dime, 2 nickels, 1 nickel + 5 pennies etc.
– Nickel: Swap for another nickel or 5 pennies.
– Penny: Swap for another penny.
![Page 42: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/42.jpg)
CSE 3101 42
optSours is valid
optSours is consistent
optSours is optimal
<LI>optSours
Massaging optSLI into optSours Case 1
<LI>¬<exit Cond>codeB
<LI>
Exit Maintaining Loop Invariant
optSours
![Page 43: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/43.jpg)
Case 2. Rejecting the Current Object
![Page 44: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/44.jpg)
CSE 3101 44
Rejecting the Current Object
Strategy of Proof:
LI1. There is at least one optimal solution consistent with previous choioptS ces.
2. Any optimal solution consistent with previous choices cannot include current object.
LI3. Therefore cannot include current objoptS ect.
![Page 45: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/45.jpg)
CSE 3101 45
Rejecting an Object
• Making Change Example:
– We only reject an object when including it would make us exceed the total.
– Thus optSLI cannot include the object either.
![Page 46: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/46.jpg)
CSE 3101 46
optSLII hold optSLI witnessing that there is an opt sol consistent
with previous choices.
I must make sure that what the Fairy God Mother has is consistent with
this new choice.
Massaging optSLI into optSoursCase 2
25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢
10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢
5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢
1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢
Amount = 92¢
I reject the next 25¢
![Page 47: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/47.jpg)
CSE 3101 47
The Algorithm has 92¢-75¢ = 17¢ 25¢ unchoosen.
Massaging optSLI into optSours
25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢ 25¢
10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢ 10¢
5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢ 5¢
1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢
Amount = 92¢
optSLI
Fairy God Mother must have 25¢ that I don’t know about.
optSLI does not contain the 25¢ either.
![Page 48: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/48.jpg)
CSE 3101 48
<loop-invariant><exit Cond>codeC
<postCond>Exit
Clean up loose ends
Alg has committed to or rejected each object.
Has yielded a solution S.
<exit Cond>
<LI> opt sol consistent with these choices.
S must be optimal.
Alg returns S .
<postCond>
codeC
![Page 49: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/49.jpg)
CSE 3101 49
Making Change Example
Greedy Choice: Start by grabbing quarters until exceeds amount, then dimes, then nickels, then pennies.
Problem: Find the minimum # of quarters, dimes, nickels, and pennies that total to a given amount.
Does this lead to an optimal # of coins?
Yes
![Page 50: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/50.jpg)
CSE 3101 50
Hard Making Change Example
Greedy Choice: Start by grabbing a 4 coin.
Problem: Find the minimum # of 4, 3, and 1 cent coins to make up 6 cents.
![Page 51: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/51.jpg)
CSE 3101 51
I will now instruct how to massage optSLI into optSours so that it is consistent with
previous & new choice.
Massaging optSLI into optSours
4¢ 4¢ 4¢ 4¢ 4¢ 4¢ 4¢ 4¢ 4¢ 4¢
3¢ 3¢ 3¢ 3¢ 3¢ 3¢ 3¢ 3¢ 3¢ 3¢
1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢ 1¢
Amount = 6¢
optSLI
I commit to keeping a 4¢
I hold optSLI.
Oops!
![Page 52: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/52.jpg)
CSE 3101 52
Hard Making Change Example
Greedy Choice: Start by grabbing a 4 coin.
Problem: Find the minimum # of 4, 3, and 1 cent coins to make up 6 cents.
Consequences: 4+1+1 = 6 mistake 3+3=6 better
Greedy Algorithm does not work!
![Page 53: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/53.jpg)
CSE 3101 53
Analysing Arbitrary Systems of Denominations
• Suppose we are given a system of coin denominations. How do we decide whether the greedy algorithm is optimal?
• It turns out that this problem can be solved in O(D3) time, where D = number of denominations (e.g., D=6 in Canada) (Pearson 1994).
![Page 54: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/54.jpg)
CSE 3101 54
Designing Optimal Systems of Denominations
In Canada, we use a 6 coin system:
1 cent, 5 cents, 10 cents, 25 cents, 100 cents and 200 cents.
Assuming that , the change to be made, is uniformly distributed
over {1,...,499}, the expected number of coins per transaction is 5.9.
N
The optimal (but non-greedy) 6-coin systems are (1,6,14,62,99,140) and
(1,8,13,69,110,160), each of which give an expected 4.67 coins per transaction.
The optimal 6-coin systems are (1,3,8,26,64,{202 or 203 or 204})
and (1,3,10,25,79,{195 or 196 or 197}) with an expected cost of 5.036
coins per transaction.
greedy
![Page 55: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/55.jpg)
CSE 3101 55
Summary
• We must prove that every coin chosen or rejected in greedy fashion still leaves us with a solution that is
– Valid
– Consistent
– Optimal
• We prove this using an inductive ‘cut and paste’ method.
• We know from the previous iteration we have a partial solution Spart that is part of some complete optimal solution optSLI.
optSLIpartial
solution
new object
![Page 56: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/56.jpg)
CSE 3101 56
Summary• Selecting a coin: we show that we can replace a subset
of the coins in optSLI\ Spart with the selected coin (+
perhaps some additional coins).
– Valid because we ensure that the trade is fair (sums are equal)
– Consistent because we have not touched Spart
– Optimal because the number of the new coin(s) is no greater than the number of coins they replace.
• Rejecting a coin: we show that we only reject a coin when it could not be part of optSLI.
optSLIpartial
solution
new object
swap
oursoptS
![Page 57: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/57.jpg)
Example 2: Job/Event Scheduling
![Page 58: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/58.jpg)
CSE 3101 58
Ingredients:
•Instances: Events with starting and finishing times
<<s1,f1>,<s2,f2>,… ,<sn,fn>>.
•Solutions: A set of events that do not overlap.
•Value of Solution: The number of events scheduled.
•Goal: Given a set of events, schedule as many as possible.
•Example: Scheduling lectures in a lecture hall.
The Job/Event Scheduling Problem
![Page 59: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/59.jpg)
CSE 3101 59
Possible Criteria for Defining “Best”
The Shortest Event
Counter Example
Does not book the room for a long period of time.
Motivation:
OptimalSchedule first
Optimal
Greedy Criterion:
![Page 60: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/60.jpg)
CSE 3101 60
Possible Criteria for Defining “Best”
The Earliest Starting Time
Counter Example
Gets room in use as early as possibleMotivation:
OptimalSchedule first
Optimal
Greedy Criterion:
![Page 61: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/61.jpg)
CSE 3101 61
Possible Criteria for Defining “Best”
Conflicting with the Fewest Other Events
Counter Example
Leaves many that can still be scheduled.Motivation:
Schedule firstOptimal
Optimal
Greedy Criterion:
![Page 62: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/62.jpg)
CSE 3101 62
Possible Criteria for Defining “Best”
Earliest Finishing TimeSchedule the event that will free up your room for someone else as soon as possible.
Motivation:
Greedy Criterion:
![Page 63: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/63.jpg)
CSE 3101 63
The Greedy Algorithm
![Page 64: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/64.jpg)
CSE 3101 64
Massaging optSLI into optSours
optSLI
Start by adding new event i.
Delete events conflicting with job i.
![Page 65: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/65.jpg)
CSE 3101 65
Massaging optSLI into optSours
optSLI
optSLI was valid and we removed any new conflicts.
optSours is valid
![Page 66: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/66.jpg)
CSE 3101 66
Massaging optSLI into optSours
optSLI
optSLI was consistent with our prior choices. We added event i. Events in Commit don’t conflict with event i and hence were not deleted.
optSours is consistent with our choices.
![Page 67: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/67.jpg)
CSE 3101 67
Massaging optSLI into optSours
optSLI
optSLI was optimal.If we delete at most one eventthen optSours is optimal too.
optSours is optimal
![Page 68: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/68.jpg)
CSE 3101 68
Massaging optSLI into optSours
optSLI
Only one in optSLI.
Deleted at most one event j
j
i<j
j runs at time fi.
Two such j conflict with each other.
j’
[j conflicts with i] sj fi
fi fj
![Page 69: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/69.jpg)
CSE 3101 69
optSLI
optSours
optSours is valid
optSours is consistent
optSours is optimal
<LI>optSours
Massaging optSLI into optSoursCase 1
<LI>¬<exit Cond>codeB
<LI>
Exit Maintaining Loop Invariant
![Page 70: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/70.jpg)
CSE 3101 70
optSLI
I hold optSLI witnessing that there is an opt sol consistent with
previous choices.
I reject next event i.
Massaging optSLI into optSours Case 2
Event i conflicts with events committed to so it can’t be in optSLI either.
![Page 71: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/71.jpg)
CSE 3101 71
optSLI
Massaging optSLI into optSours
<LI>¬<exit Cond>codeB
<LI>
Exit Maintaining Loop Invariant
![Page 72: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/72.jpg)
CSE 3101 72
<loop-invariant><exit Cond>codeC
<postCond>Exit
Clean up loose ends
Alg commits to or reject each event. Has a solution S.
<exit Cond>
<LI> opt sol consistent with these choices.S must be optimal.
Alg returns optS .
<postCond>
codeC
![Page 73: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/73.jpg)
CSE 3101 73
Running TimeGreedy algorithms are very fast because they only consider each object once.
Checking whether next event i conflicts with previously committed events requires only comparing it with the last such event.
![Page 74: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/74.jpg)
CSE 3101 74
Running Time
( ) ( log )T n n n
( log )n n
( )n
![Page 75: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/75.jpg)
Example 3: Minimum Spanning Trees
![Page 76: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/76.jpg)
CSE 3101 76
Minimum Spanning Trees• Example Problem
– You are planning a new terrestrial telecommunications network to connect a number of remote mountain villages in a developing country.
– The cost of building a link between pairs of neighbouring villages (u,v) has been estimated: w(u,v).
– You seek the minimum cost design that ensures each village is connected to the network.
– The solution is called a minimum spanning tree (MST).
![Page 77: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/77.jpg)
CSE 3101 77
Minimum Spanning Trees
The weight of a subset of a weighted graph is defined as:T
( , )
( ) ( , )u v T
w T w u v
Thus the MST is the spanning tree that minimizes ( )T w T
The problem is defined for any undirected, connected, weighted graph.
![Page 78: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/78.jpg)
CSE 3101 78
Building the Minimum Spanning Tree
• Iteratively construct the set of edges A in the MST.
• Initialize A to {}
• As we add edges to A, maintain a Loop Invariant:
– A is a subset of some MST
• Maintain loop invariant and make progress by only adding safe edges.
• An edge (u,v) is called safe for A iff A{u,v}) is also a subset of some MST.
![Page 79: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/79.jpg)
CSE 3101 79
Finding a safe edge
• Idea: Every 2 disjoint subsets of vertices must be connected by at least one edge.
• Which one should we choose?
![Page 80: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/80.jpg)
CSE 3101 80
Some definitions• A cut (S,V-S) is a partition of vertices into disjoint sets
S and V-S.
• Edge (u,v)E crosses cut (S, V-S) if one endpoint is in S and the other is in V-S.
• A cut respects a set of edges A iff no edge in A crosses the cut.
• An edge is a light edge crossing a cut iff its weight is minimum over all edges crossing the cut.
A
![Page 81: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/81.jpg)
CSE 3101 81
Minimum Spanning Tree Theorem
• Let
– A be a subset of some MST
– (S,V-S) be a cut that respects A
– (u,v) be a light edge crossing (S,V-S)
• Then
– (u,v) is safe for A.
A
Basis for a greedy algorithm
![Page 82: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/82.jpg)
CSE 3101 82
Proof• Let G be a connected, undirected, weighted graph.
• Let T be an MST that includes A.
• Let (S,V-S) be a cut that respects A.
• Let (u,v) be a light edge between S and V-S.
• If T contains (u,v) then we’re done.
u
v
S
V - S
Edge T
Edge T
![Page 83: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/83.jpg)
CSE 3101 83
ux
vy
S
V - S
• Suppose T does not contain (u,v)
– Can construct different MST T' that includes Au,v)
– The edge (u,v) forms a cycle with the edges on the path p from u to v in T.
– There is at least one edge in p that crosses the cut: let that edge be (x,y)
– (x,y) is not in A, since the cut (S,V-S) respects A.
– Form new spanning tree T' by deleting (x,y) from T and adding (u,v).
– w(T') w(T), since w(u,v) w(x,y) T' is an MST.
– A T', since A T and (x,y)A Au,v) T'
– Thus (u,v) is safe for A.
p
![Page 84: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/84.jpg)
End of Lecture 17
Nov 13, 2007
![Page 85: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/85.jpg)
CSE 3101 85
Kruskal’s Algorithm for computing MST
• Starts with each vertex being its own component.
• Repeatedly merges two components into one by choosing the light edge that crosses the cut between them.
• Scans the set of edges in monotonically increasing order by weight (greedy).
![Page 86: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/86.jpg)
CSE 3101 86
Kruskal’s Algorithm: Loop Invariant
Let solution under construction.
Let the subset of lowest-weight edges thus f
loop-invariant :
MST :
1) ,
2) ( ,
ar considered
) :
( , ) ( , ) i
i
T
A T
u v E
u v A o
A
E
r u v T
i
![Page 87: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/87.jpg)
CSE 3101 87
Kruskal’s Algorithm: Example
![Page 88: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/88.jpg)
CSE 3101 88
Kruskal’s Algorithm: Example
![Page 89: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/89.jpg)
CSE 3101 89
Kruskal’s Algorithm: Example
![Page 90: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/90.jpg)
CSE 3101 90
Kruskal’s Algorithm: Example
![Page 91: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/91.jpg)
CSE 3101 91
Kruskal’s Algorithm: Example
![Page 92: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/92.jpg)
CSE 3101 92
Kruskal’s Algorithm: Example
![Page 93: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/93.jpg)
CSE 3101 93
Kruskal’s Algorithm: Example
![Page 94: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/94.jpg)
CSE 3101 94
Kruskal’s Algorithm: Example
![Page 95: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/95.jpg)
CSE 3101 95
Kruskal’s Algorithm: Example
![Page 96: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/96.jpg)
CSE 3101 96
Kruskal’s Algorithm: Example
![Page 97: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/97.jpg)
CSE 3101 97
Kruskal’s Algorithm: Example
![Page 98: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/98.jpg)
CSE 3101 98
Kruskal’s Algorithm: Example
![Page 99: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/99.jpg)
CSE 3101 99
Kruskal’s Algorithm: Example
![Page 100: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/100.jpg)
CSE 3101 100
Kruskal’s Algorithm: Example
Finished!
![Page 101: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/101.jpg)
CSE 3101 101
Disjoint Set Data Structures
• Disjoint set data structures can be used to represent the disjoint connected components of a graph.
• Make-Set(x) makes a new disjoint component containing only vertex x.
• Union(x,y) merges the disjoint component containing vertex x with the disjoint component containing vertex y.
• Find-Set(x) returns a vertex that represents the disjoint component containing x.
![Page 102: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/102.jpg)
CSE 3101 102
Disjoint Set Data Structures• Most efficient representation represents each disjoint set
(component) as a tree.
• Time complexity of a sequence of m operations, n of which are Make-Set operations, is:
( ( ))O m n
where ( ) is Ackerman's function, which grows extremely slowly.n
80
( )
3 1
7 2
2047 3
10 4
n n
![Page 103: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/103.jpg)
CSE 3101 103
Kruskal’s Algorithm for computing MST
Running Time =
O(ElogE)
= O(ElogV)
Kruskal( , )
for each vertex [ ]
Make-Set(v)
sort E[G] into nondecreasing order:
loop-invariant :
MST :1)
[1... ]
for 1:
( , ) [ ]
if Find-Set( ) Fi
,
2) ( , ) [1... 1]: ( , ) ,
n
( )
G w
A
v V G
E n
i n
T A T
u v E i u v A or u v
v i
T
u E
u
d-Set( )
{( , )}
Union( , )
return
v
A A u v
u v
A
![Page 104: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/104.jpg)
CSE 3101 104
Prim’s Algorithm for Computing MST• Build one tree A
• Start from arbitrary root r
• At each step, add light edge connecting VA to V- VA
(greedy)
![Page 105: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/105.jpg)
CSE 3101 105
Prim’s Algorithm: Example
![Page 106: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/106.jpg)
CSE 3101 106
Prim’s Algorithm: Example
![Page 107: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/107.jpg)
CSE 3101 107
Prim’s Algorithm: Example
![Page 108: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/108.jpg)
CSE 3101 108
Prim’s Algorithm: Example
![Page 109: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/109.jpg)
CSE 3101 109
Prim’s Algorithm: Example
![Page 110: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/110.jpg)
CSE 3101 110
Prim’s Algorithm: Example
![Page 111: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/111.jpg)
CSE 3101 111
Prim’s Algorithm: Example
![Page 112: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/112.jpg)
CSE 3101 112
Prim’s Algorithm: Example
![Page 113: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/113.jpg)
CSE 3101 113
Prim’s Algorithm: Example
Finished!
![Page 114: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/114.jpg)
CSE 3101 114
Finding light edges quickly
• All vertices not in the partial MST formed by A reside in a min-priority queue.
• Key(v) is minimum weight of any edge (u,v), u VA.
• Priority queue can be implemented as a min heap on key(v).
• Each vertex in queue knows its potential parent in partial MST by [v].
![Page 115: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/115.jpg)
CSE 3101 115
Prim’s Algorithm
Let {( , [ ]) : - { } - }
Let
<loop-invariant>:
1. MST :
2. v Q, if [ ] NIL
then [ ] weight of light edge connecting to
A
A
A v v v V r Q
V V Q
T A T
v
key v v V
![Page 116: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/116.jpg)
CSE 3101 116
Prim’s Algorithm
( )O V
Executed | | timesV(log )O V
Executed |E | times
(log )O V
Running Time = ( log )O E V
![Page 117: Greedy Algorithms Credits: Many of these slides were originally authored by Jeff Edmonds, York University. Thanks Jeff!](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d205503460f949f58e6/html5/thumbnails/117.jpg)
CSE 3101 117
Algorithm Comparison
• Both Kruskal’s and Prim’s algorithm are greedy.
– Kruskal’s: Queue is static (constructed before loop)
– Prim’s: Queue is dynamic (keys adjusted as edges are encountered)