1

Greedy Algorithm

2

Introduction

• Similar to dynamic programming.

• Used for optimization problems.

• Not always yield an optimal solution.

• Make choice for the one looks best right now.

• Make a locally optimal choice in hope of getting

a globally optimal solution.

3

Activity selection

n activities require exclusive use of a common resource.

For example, scheduling the use of a classroom.

Set of activities S = {a1 . . . , an}.

a1 needs resource during period [si, fi), which is a half-open interval,

where si = start time and fi = finish time.

Goal: Select the largest possible set of nonoverlapping (mutually compatible) activities.

Note: Could have many other objectives:

• Schedule room for longest time. • Maximize income rental fees.

4

Activity selection

i 1 2 3 4 5 6 7 8 9 10 11

si fi

1 3 0 5 3 5 6 8 8 2 12 4 5 6 7 8 9 10 11 12 13 14

Example: S sorted by finish time

Maximum-size mutually compatible set: {a1 , a4 , a8 , a11}.

Not unique: also {a2 , a4 , a9, a11}.

5

Activity selection

Optimal substructure of activity selection

Sij = {ak∈ S : fi ≤ sk fk ≤ sj}

= activities that start after ai finishes and finish

before aj starts.

… … ak ai aj

fk fi sk sj

6

Activity selection

Activities in Sij are compatible with

• all activities that finish by fi, and

• all activities that start no earlier than sj.

To represent the entire problem, add fictitious activities:

a0 = [ – ∞, 0)

an+1 = [∞, “∞ + 1”)

we don’t care about – ∞ and “∞ + 1”.

Then S = S0,n+1.

Range for Sij is 0 ≤ i, j ≤ n+1.

7

Activity selection

Assume that activities are sorted by monotonically increasing

finish time:

f0 ≤ f1 ≤ f2 ≤ ··· ≤ fn ≤ fn+1 .

Then i ≥ j ⇒ Sij = Ø .

• If there exists ak ∈ Sij :

fi ≤ sk < fk ≤ sj < fj ⇒ fi < fj .

• But i ≥ j ⇒ fi ≥ fj . Contradiction.

So only need to worry about Sij with 0 ≤ i < j ≤ n + 1.

All other Sij are Ø .

8

Activity selection

Suppose that a solution to Sij includes ak. Have 2 subproblems:

• Sik (start after ai finishes, finish before ak starts)

• Skj (start after ak finishes, finish before a aj starts)

Solution to Sij is (solution to Sik) {ak} ( solution to Skj).

Since ak is in neither subproblem, and the subproblems are disjoint,

|solution to S| = |solution to Sik | + 1 + |solution to Skj | .

∩ ∩

9

Activity selection

If an optimal solution to Sij includes ak, then the solutions to

Sik and Skj used within this solution must be optimal as well.

Use the usual cut-and-paste argument.

Let Aij = optimal solution to Sij .

So Aij = Aik {ak} Akj , assuming:

• Sij is nonempty, and

• we know ak.

∩ ∩

10

Activity selection

Recursive solution to activity selection

c[i, j]=size of maximum-size subset of mutually compatible activities in Sij.

• i ≥ j ⇒ Sij = Ø ⇒ c[i, j] = 0.

If Sij ≠ Ø , suppose we know that ak is in the subset. Then

c[i, j] = c[i, k] + 1 + c[k, j].

But of course we don’t know which k to use, and so

C[i, j] =

i < k< j

{ 0 if Sij = Ø ,

max {c[i, k] + c[k, j] + 1} if Sij ≠ Ø. ak ∈ Sij

11

Activity selection

Theorem

Let Sij ≠ Ø , and let am be the activity in Sij with the earliest

finish time: fm = min {fk : ak Sij}. Then:

1. am is used in some maximum-size subset of mutually

compatible activities of Sij .

2. Sim = Ø , so that choosing am leaves Smj as the only

nonempty subproblem.

12

Activity selection

2. Suppose there is some ak Sim. Then fi sk fk sm fm fk

fm. Then ak Sij and it has an earlier finish than fm , which

contradicts our choice of am. Therefore, there is no ak Sim

Sim .

Proof

1. Let Aij be a maximum-size subset of mutually compatible

activities in Sij.

Order activities in Aij in monotonically increasing order of finish

time.

Let ak be the first activity in Aij.

If ak am , done (am is used in a maximum-size subset).

Otherwise, construct A’ij Aij−{ak}{am} (replace ak by am)

13

Activity selection

Claim

Activities in A'ij are disjoint.

Proof Activities in Aij are disjoint, ak is the first activity in

Aij to finish, fm ≤ fk (so am doesn’t overlap with anything

else in A'ij).

■ (claim)

Since |A'ij| = |Aij| and Aij is a maximum-size subset, so is A'ij.

■ (theorem)

14

Activity selection

This is great:

before theorem after theorem

# of subproblems in optimal solution 2 1

# of choices to consider j – i – 1 1

Now we can solve top down:

• To solve a problem Sij,

• Choose am ∈ Sij with earliest finish time: the greedy choice.

• Then solve Smj.

15

Activity selection

What are the subproblems?

• Original problem is S0,n+1 .

• Suppose our first choice is am1 .

• Then next subproblem is Sm1,n+1 .

• Suppose next choice is am2 .

• Next subproblem is Sm2,n+1 .

• And so on.

Each subproblem is Smi,n+1, i.e., the last activities to finish.

And the subproblems chosen have finish times that increase.

Therefore, we can consider each activity just once, in monotonically

increasing order of finish time.

16

Activity selection

Assumes activities already sorted by monotonically in-

creasing finish time. (If not, then sort in O (n lg n) time.)

Return an optimal solution for Si,n+1:

Easy recursive algorithm:

REC-ACTIVITY-SELECTOR(s, f, i, n)

m ← i + 1

while m ≤ n and sm < fi ▷ Find first activity in Si,n+1. do m ← m + 1

If m ≤ n

then return {am} REC-ACTIVITY-SELECTOR(s, f, m, n)

else return

17

Activity selection

Initial call: REC-ACTIVITY-SELECTOR(s, f, 0, n)

Idea: The while loop checks ai+1, ai+2, …, an until it finds an

activity am that is compatible with ai (need sm ≥ fi).

• If the loop terminates because am is found (m ≤ n), then re-

cursively solve Sm,n+1, and return this solution, along with am.

• If the loop never finds a compatible am (m > n), then just

return empty set.

Go through example given earlier. Should get {a1, a4, a8, a11}.

Time: Θ (n) – each activity examined exactly once.

18 time

a 1

k sk f

k

0 - 0

1 1 4

2 3 5

3 0 6

4 5 7

5 3 8

6 5 9

7 6 10

8 8 11

9 8 12

10 2 13

11 12 14

12 ∞ -

m = 1

m = 4

m = 8

m = 11

R E C U R SI V E - A C T I V I T Y - SE L E C T O R (s, f , 0, 11 )

R E C U R SI V E - A C T I V I T Y - SE L E C T O R (s, f , 1, 11 )

R E C U R SI V E - A C T I V I T Y - SE L E C T O R (s, f , 4, 11 )

R E C U R SI V E - A C T I V I T Y - SE L E C T O R (s, f , 8, 11 )

R E C U R SI V E - A C T I V I T Y - SE L E C T O R (s, f , 11, 11)

0 1 2 3 4 5 6 7 8 9 10 11 13 14

a 1

a 2

a 3

a 1

a 1

a 1

a 1

a 1

a 4

a 4

a 4

a 6

a 5

a 7

a 8

a 4

a 4

a 11

a 10

a 9

a 4

a 4

a 0

a 0

a 11a 8

a 8

a 8

a 8

a 4a 1

a 1

a 1

a 1

a 4a 1

19

Activity selection

Can make this iterative.

GREEDY-ACTIVITY-SELECTOR(s, f, n)

A ← {a1}

i ← 1

for m ← 2 to n

do if sm ≥ fi

then A ← A {am}

i ← m ▷ ai is most recent addition to A.

return A

Go through example given earlier. Should again get {a1, a4, a8, a11}.

Time: Θ (n).

20

Greedy strategy

1. Determine the optimal substructure.

2. Develop a recursive solution.

3. Prove that at any stage of recursion, one of the optimal

choices is the greedy choice. Therefore, it’s always safe

to make the greedy choice.

4. Show that all but one of the subproblems resulting from

the greedy choice are empty.

5. Develop a recursive greedy algorithm.

6. Convert it to an iterative algorithm.

The choice that seems best at the moment is the one we go with. What did we do for activity selection?

21

Greedy strategy

No general way to tell if a greedy algorithm is optimal, but two key ingredients are

1. greedy-choice property and

2. optimal substructure.

Typical streamlined steps:

1. Cast the optimization problem as one in which we make a choice and are left with one subproblem to solve.

2. Prove that there’s always an optimal solution that makes the greedy choice, so that the greedy choice is always safe.

3. Show that greedy choice and optimal solution to subproblem optimal solution to the problem.

22

Greedy strategy

Dynamic programming:

• Make a choice at each step.

• Choice depends on knowing optimal solutions to

subproblems. Solve subproblems first.

• Solve bottom-up.

Greedy:

• Make a choice at each step.

• Make the choice before solving the subproblems.

• Solve top-down

23

The knapsack problem is a good example of the difference.

0-1 knapsack problem:

• n items.

• Item i is worth $vi, weighs wi pounds.

• Find a most valuable subset of items with total weight ≤ W.

• Have to either take an item or not take it – can’t take part of it.

Greedy strategy

Greedy vs. dynamic programming

24

Fractional knapsack problem: Like the 0-1 kanpsack

problem, but can take fraction of an item.

Both have optimal substructure.

But the fractional kanpsack problem has the greedy-choice

property, and the 0-1 knapsack problem does not have

greedy-choice that returns optimal solution.

To solve the fractional problem, rank items by value/weight:

vi/wi.

Let vi/wi ≥ vi+1/wi+1 for all i.

Greedy strategy

25

Fractional knapsack (v, w, W)

load ← 0

i ← 1

while load ≤ W and i ≤ n

do if wi ≤ W – load

then take all of item i

else take (W – load)/wi of item i

add what was taken to load

i ← i + 1 ▷ move to the next valuable item.

Greedy strategy

Taking the items in order of greatest value per pound yields

an optimal solution.

26

Time: O (n lg n) to sort, O (n) thereafter

Greedy doesn’t work for the 0-1 knapsack problem. Might

get empty space, which lowers the average value per pound

of the items taken.

i

vi

wi

vi /wi

1 2 3

60 100 120

10 20 30

6 5 4

W = 50

Greedy strategy

27

Greedy solution:

• Take items 1 and 2.

• value = 160, weight = 30.

Have 20 pounds of capacity left over.

• 0-1 knapsack problem can’t be solved w/ greedy strategy.

(i.e., the optimal solution can’t be obtained w/ greedy strategy)

• Fraction knapsack problem can be solved w/ greedy strategy by

adding 20 pounds (2/3 of item 3) ×4 = $80 value.

Optimal solution (for 0-1 knapsack w/o greedy strategy):

• Take items 2 and 3.

• value = 220, weight = 50.

No leftover capacity.

Greedy strategy