cs 4407 algorithms greedy algorithms
TRANSCRIPT
1
CS 4407
Algorithms
Greedy Algorithms
Prof. Gregory Provan
Department of Computer Science
University College Cork
CS 4407, Algorithms University College Cork,
Gregory M. Provan
Overview
Motivation
Abstract greedy algorithm
Examples
– Minimum spanning tree
– Scheduling
– Activity selection
CS 4407, Algorithms University College Cork,
Gregory M. Provan
3
Fundamental Techniques
There are some algorithmic tools that are
quite specialised. They are good for
problems they are intended to solve, but
they are not very versatile.
There are also more fundamental (general)
algorithmic tools that can be applied to a
wide variety of different data structure and
algorithm design problems.
CS 4407, Algorithms University College Cork,
Gregory M. Provan
Motivation
Greedy algorithms are efficient ways to solve
hard problems
– May not be optimal though
Objectives
– When is a greedy algorithm optimal?
– Can we prove bounded optimality?
• Approximation algorithms
– Can we show sub-classes for optimality?
CS 4407, Algorithms University College Cork,
Gregory M. Provan
5
The Greedy Method
An optimisation problem (OP) is a problem that involves searching through a set of configurations to find one that minimises or maximizes an objective function defined on these configurations
The greedy method solves a given OP going through a sequence of (feasible) choices
The sequence starts from well-understood starting configuration, and then iteratively makes the decision that seems best from all those that are currently possible.
CS 4407, Algorithms University College Cork,
Gregory M. Provan
6
The Greedy Method
The greedy approach does not always lead to an optimal solution.
The problems that have a greedy solution are said to posses the greedy-choice property.
The greedy approach is also used in the context of hard (difficult to solve) problems in order to generate an approximate solution.
CS 4407, Algorithms University College Cork,
Gregory M. Provan
Greedy Algorithms
A greedy algorithm always makes the choice that looks best at the moment.
It makes a local optimal choice in the hope that this choice will lead to a globally optimal solution.
Greedy algorithms yield optimal solutions for many (but not all) problems.
2020/2/6
CS 4407, Algorithms University College Cork,
Gregory M. Provan
The Greedy Method
The greedy method is a general algorithm design paradigm, built
on the following elements:
– configurations: different choices, collections, or values to find
– objective function: a score assigned to configurations, which we want to
either maximize or minimize
It works best when applied to problems with the greedy-choice
property:
– a globally-optimal solution can always be found by a series of local
improvements from a starting configuration.
– Never change a previous choice
8
CS 4407, Algorithms University College Cork,
Gregory M. Provan
Greedy Algorithms
When are greedy algorithms optimal?
Must satisfy the optimal substructure
property
– Optimal sub-problem must participate in optimal
problem
CS 4407, Algorithms University College Cork,
Gregory M. Provan
What makes a greedy algorithm?
Feasible – Has to satisfy the problem’s constraints
Locally Optimal – The greedy part
– Has to make the best local choice among all feasible choices available on that step
• If this local choice results in a global optimum then the problem has optimal substructure
Irrevocable – Once a choice is made it can’t be un-done on subsequent steps of the
algorithm
Simple examples: – Playing chess by making best move without lookahead
– Giving fewest number of coins as change
Simple and appealing, but don’t always give the best solution
CS 4407, Algorithms University College Cork,
Gregory M. Provan
Abstract Greedy Algorithm
Inputs: heuristic H, Loop Invariant LI, Cost
Function CF
Greedy (list)
– L Sort list by H, CF, R //R is result//
– Do while L
• R R head(L) if addition does not violate LI
• L L - head(L)
– End do
– Return R
CS 4407, Algorithms University College Cork,
Gregory M. Provan
Variants of Greedy Method
Greedy method sorts the objects in our task
Two main methods
– Global sort: sort at the beginning
– Repeated sort: sort for every iteration
CS 4407, Algorithms University College Cork,
Gregory M. Provan
Objectives
Define greedy algorithm
– Heuristic function
Determine optimality
Prove optimality (or sub-optimality)
Page 14
CS 4407, Algorithms University College Cork,
Gregory M. Provan
Proving Optimality
Sub-optimality is easy to show (if present)
– Provide counter-example
Optimality is more challenging
Techniques
– Greedy property always holds
– “greedy stays ahead”
– Exchange methods
CS 4407, Algorithms University College Cork,
Gregory M. Provan
Examples
Counting money
Scheduling
Minimum Spanning tree
Huffman encoding
Dijkstra’s algorithm (single-source shortest
paths)
CS 4407, Algorithms University College Cork,
Gregory M. Provan
Example: Counting money
Suppose you want to count out a certain amount of money, using the fewest possible bills and coins
A greedy algorithm would do this would be: At each step, take the largest possible bill or coin that does not overshoot – Example: To make $6.39, you can choose:
• a $5 bill
• a $1 bill, to make $6
• a 25¢ coin, to make $6.25
• A 10¢ coin, to make $6.35
• four 1¢ coins, to make $6.39
For US money, the greedy algorithm always gives the optimum solution
CS 4407, Algorithms University College Cork,
Gregory M. Provan
A failure of the greedy algorithm
In some (fictional) monetary system, “krons” come in 1 kron, 7 kron, and 10 kron coins
Using a greedy algorithm to count out 15 krons, you would get – A 10 kron piece
– Five 1 kron pieces, for a total of 15 krons
– This requires six coins
A better solution would be to use two 7 kron pieces and one 1 kron piece – This only requires three coins
The greedy algorithm results in a solution, but not in an optimal solution
CS 4407, Algorithms University College Cork,
Gregory M. Provan
Sample MST
6
5
4
2
9
15
14
10
38
CS 4407, Algorithms University College Cork,
Gregory M. Provan
Spanning Tree
Definition
– A spanning tree of a graph G is a tree (acyclic) that
connects all the vertices of G once
• i.e. the tree “spans” every vertex in G
– A Minimum Spanning Tree (MST) is a spanning tree on a
weighted graph that has the minimum total weight
w T w u vu v T
( ) ( , ),
such that w(T) is minimum
CS 4407, Algorithms University College Cork,
Gregory M. Provan
Sample MST
Which links to make this a MST?
6
5
4
2
9
15
14
10
38
Optimal substructure: A subtree of the MST must in turn be a
MST of the nodes that it spans.
CS 4407, Algorithms University College Cork,
Gregory M. Provan
MST Claim
Claim: Say that M is a MST
– If we remove any edge (u,v) from M then this
results in two trees, T1 and T2.
– T1 is a MST of its subgraph while T2 is a MST of
its subgraph.
– Then the MST of the entire graph is T1 + T2 + the
smallest edge between T1 and T2
– If some other edge was used, we wouldn’t have
the minimum spanning tree overall
CS 4407, Algorithms University College Cork,
Gregory M. Provan
Greedy Algorithm
We can use a greedy algorithm to find the
MST.
– Two common algorithms
• Kruskal
• Prim
CS 4407, Algorithms University College Cork,
Gregory M. Provan
Example: Kruskal’s MST Algorithm
Idea: Greedily construct the MST
– Go through the list of edges and make a forest
that is a MST
– At each vertex, sort the edges
– Edges with smallest weights examined and
possibly added to MST before edges with higher
weights
– Edges added must be “safe edges” that do not
ruin the tree property.
CS 4407, Algorithms University College Cork,
Gregory M. Provan
Scheduling Problem Statement
INPUT: A set S = { I1, I2, ···, In} of n event time-intervals Ik = sk , fk, k =1..n, where sk = start time of Ik , fk = finishing time of Ik , ( sk < fk ).
OUTPUT: A maximum cardinality subset C S of mutually compatible intervals
(i.e., no overlapping pairs).
Example: S = the intervals shown below, C = the blue intervals, C is not the unique optimum. |C| =4. Can you spot another optimum solution?
time
CS 4407, Algorithms University College Cork,
Gregory M. Provan
Task Scheduling
Given: a set T of n tasks, each having: – A start time, si
– A finish time, fi (where si < fi)
Goal: Perform all the tasks using a minimum number of “machines.”
27
1 9 8 7 6 5 4 3 2
Machine 1
Machine 3
Machine 2
CS 4407, Algorithms University College Cork,
Gregory M. Provan
Task Scheduling Algorithm
Greedy choice: consider tasks by their start time and use as few machines as possible with this order.
– Run time: O(n log n).
Correctness: Suppose there is a better schedule.
– We can use k-1 machines
– The algorithm uses k
– Let i be first task scheduled on machine k
– Task i must conflict with k-1 other tasks
– K mutually conflict tasks
– But that means there is no non-conflicting schedule using k-1 machines
28
Algorithm taskSchedule(T)
Input: set T of tasks w/ start time si and finish time fi
Output: non-conflicting schedule with minimum number of machines
m 0 {no. of machines}
while T is not empty
remove task i w/ smallest si
if there’s a machine j for i then
schedule i on machine j
else
m m + 1
schedule i on machine m
CS 4407, Algorithms University College Cork,
Gregory M. Provan
Example
Given: a set T of n tasks, each having: – A start time, si
– A finish time, fi (where si < fi)
– [1,4], [1,3], [2,5], [3,7], [4,7], [6,9], [7,8] (ordered by start)
Goal: Perform all tasks on min. number of machines
29
1 9 8 7 6 5 4 3 2
Machine 1
Machine 3
Machine 2
A banquet hall manager has received a list of reservation requests
for the exclusive use of her hall for specified time intervals
She wishes to grant the maximum number of reservation requests
that have no time overlap conflicts
Help her select the maximum number of conflict free time intervals
Event Scheduling Example
CS 4407, Algorithms University College Cork,
Gregory M. Provan
Activity-Selection Problem
Problem: get your money’s worth out of a
carnival
– Buy a wristband that lets you onto any ride
– Lots of rides, each starting and ending at
different times
– Your goal: ride as many rides as possible
• Another, alternative goal that we don’t solve here:
maximize time spent on rides
Welcome to the activity selection problem
CS 4407, Algorithms University College Cork,
Gregory M. Provan
Activity-Selection
Formally:
– Given a set S of n activities
si = start time of activity i
fi = finish time of activity i
– Find max-size subset A of compatible activities
Assume (wlog) that f1 f2 … fn
1 2
3 4
5
6
CS 4407, Algorithms University College Cork,
Gregory M. Provan
Activity Selection Problem
Problem: – Schedule an exclusive resource in competition with other
entities.
Formal Definition: – Set S={1,2, … n} of activities.
– Each activity has a start time si and a finish time fj, where si <fj.
– Activities i and j are compatible if they do not overlap.
The activity selection problem – select a maximum-size set of mutually compatible
activities.
CS 4407, Algorithms University College Cork,
Gregory M. Provan
Greedy Choice Property
Activity selection problem exhibits the
greedy choice property:
– Locally optimal choice globally optimal sol’n
Theorem: if S is an activity selection problem
sorted by finish time, then optimal solution
A S such that {1} A
– Sketch of proof: if optimal solution B that does
not contain {1}, can always replace first activity
in B with {1} (Why?). Same number of activities,
thus optimal.
CS 4407, Algorithms University College Cork,
Gregory M. Provan
Activity Selection: A Greedy Algorithm
So actual algorithm is simple:
– Sort the activities by finish time
– Schedule the first activity
– Then schedule the next activity in sorted list
which starts after previous activity finishes
– Repeat until no more activities
Intuition is even more simple:
– Always pick the shortest ride available at the
time
CS 4407, Algorithms University College Cork,
Gregory M. Provan
Greedy Activity Selection
Just march through each activity by finish
time and schedule it if possible:
CS 4407, Algorithms University College Cork,
Gregory M. Provan
Activity Selection Example
Schedule job 1, then try rest: (end up with 1, 4, 8):
T=1 T=2 T=3 T=4 T=5 T=6 T=7 T=8 T=9 T=10 T=11 T=12 T=13
1 1 1 1
Runtime?
CS 4407, Algorithms University College Cork,
Gregory M. Provan
Activity Selection: Optimal Substructure
Let k be the minimum activity in A (i.e., the
one with the earliest finish time). Then A - {k}
is an optimal solution to S’ = {i S: si fk}
– In words: once activity #1 is selected, the
problem reduces to finding an optimal solution
for activity-selection over activities in S
compatible with #1
– Proof: if we could find optimal solution B’ to S’
with |B| > |A - {k}|,
• Then B U {k} is compatible
• And |B U {k}| > |A|
CS 4407, Algorithms University College Cork,
Gregory M. Provan
Some Greedy Heuristics
Greedy iteration step: From among undecided intervals, select the interval I that looks BEST.
Commit to I if it’s conflict-free (i.e., doesn’t overlap with the committed ones so far).
Reject I otherwise.
Greedy 1: BEST = earliest start time (min sk).
Greedy 2: BEST = latest finishing time (max fk).
Greedy 3: BEST = shortest interval (min fk – sk).
Greedy 4: BEST = overlaps with fewest # of undecided intervals.