greedy algorithm.ppt
Post on 14-Apr-2018
254 Views
Preview:
TRANSCRIPT
-
7/27/2019 greedy algorithm.ppt
1/30
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong 5. Greedy Algorithms - 1http://www.cs.cityu.edu.hk/~helena
Greedy Algorithms
-
7/27/2019 greedy algorithm.ppt
2/30
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong 5. Greedy Algorithms - 2http://www.cs.cityu.edu.hk/~helena
Greedy Algorithms
Coming upCasual Introduction: Two Knapsack Problems
An Activity-Selection Problem
Greedy Algorithm Design
Huffman Codes
(Chap 16.1-16.3)
-
7/27/2019 greedy algorithm.ppt
3/30
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong 5. Greedy Algorithms - 3http://www.cs.cityu.edu.hk/~helena
2 Knapsack Problems
A thief robbing a store finds n items.
ith item: worth vi dollars
wi pounds
W, wi, vi are integers.
He can carry at most W pounds.
1. 0-1 Knapsack Problem:
Which itemsshould I
take?
-
7/27/2019 greedy algorithm.ppt
4/30
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong 5. Greedy Algorithms - 4http://www.cs.cityu.edu.hk/~helena
2 Knapsack Problems
A thief robbing a store finds n items.
ith item: worth vi dollars
wi pounds
W, wi, vi are integers.
He can carry at most W pounds.
He can take fractions of items.
2. Fractional Knapsack Problem:
?
-
7/27/2019 greedy algorithm.ppt
5/30
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong 5. Greedy Algorithms - 5http://www.cs.cityu.edu.hk/~helena
2 Knapsack ProblemsDynamic Programming Solution
If jth item isremovedfrom his load,
Both problems exhibit the optimal-substructure property:
Consider the mostvaluable load thatweighs at most Wpounds.
the remaining load must bethe most valuable load
weighting at most W-wj thathe can take from the n-1original items excluding j.
=> Can be solved by dynamic programming
-
7/27/2019 greedy algorithm.ppt
6/30
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong 5. Greedy Algorithms - 6http://www.cs.cityu.edu.hk/~helena
2 Knapsack ProblemsDynamic Programming Solution
Example: 0-1 Knapsack Problem
Suppose there are n=100 ingots:30 Gold ingots: each $10000, 8 pounds (most expensive)20 Silver ingots: each $2000, 3 pound per piece50 Copper ingots: each $500, 5 pound per piece
Then, the most valuable load for to fill W pounds
= The most valuable way among the followings:
(1) take 1 gold ingot + the most valuable way to fill W-8 poundsfrom 29 gold ingots, 20 silver ingots and 50 copper ingots
(2) take 1 silveringot + the most valuable way to fill W-3 poundsfrom 30 gold ingots, 19 silver ingots and 50 copper ingots
(3) take 1 copperingot + the most valuable way to fill W-5 pounds
from 30 gold ingots, 20 silver ingots and 49copper ingots
-
7/27/2019 greedy algorithm.ppt
7/30
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong 5. Greedy Algorithms - 7http://www.cs.cityu.edu.hk/~helena
2 Knapsack ProblemsDynamic Programming Solution
Example: Fractional Knapsack Problem
Suppose there are totally n = 100 pounds of metal dust:30 pounds Gold dust: each pound $10000 (most expensive)20 pounds Silver dust: each pound $200050 pounds Copper dust: each pound $500
Then, the most valuable way to fill a capacity of W pounds
= The most valuable way among the followings:
(1) take 1 pound ofgold + the most valuable way to fill W-1 poundsfrom 29 pounds of gold, 20 pounds of silver, 50 pounds of copper
(2) take 1 pound ofsilver+ the most valuable way to fill W-1 poundsfrom 30 pounds of gold, 19 pounds of silver, 50 pounds of copper
(3) take 1 pound copper+ the most valuable way to fill W-1 pounds
from 30 pounds of gold, 20 pounds of silver, 49 pounds of copper
-
7/27/2019 greedy algorithm.ppt
8/30
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong 5. Greedy Algorithms - 8http://www.cs.cityu.edu.hk/~helena
2 Knapsack ProblemsBy Greedy Strategy
Both problems are similar. But Fractional Knapsack Problem
can be solved in a greedy strategy.
Step 1. Compute the value per pound for each item
Eg. gold dust: $10000 per pound (most expensive)
Silver dust: $2000 per poundCopper dust: $500 per pound
Step 2. Take as much as possible of the most expensive(ie. Gold dust)
Step 3. If the supply of that item is exhausted (ie. no moregold) and he can still carry more, he takes asmuch as possible of the item that is next most
expensive and so forth until he cant carry anymore.
-
7/27/2019 greedy algorithm.ppt
9/30
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong 5. Greedy Algorithms - 9http://www.cs.cityu.edu.hk/~helena
Knapsack ProblemsBy Greedy Strategy
We can solve the Fractional KnapsackProblem by a greedy algorithm:
Always makes the choice that looks best
at the moment.
ie. A locally optimal Choice
To see why we cant
solve 0-1 Knapsack
Problem by greedy
strategy, read Chp 16.2.
-
7/27/2019 greedy algorithm.ppt
10/30
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong 5. Greedy Algorithms - 10http://www.cs.cityu.edu.hk/~helena
Greedy Algorithms
2 techniques for solving optimization problems:1. Dynamic Programming
2. Greedy Algorithms (Greedy Strategy)
Greedy Approach can solve
these problems:
For the optimization problems:
Dynamic Programming can
solve these problems:
For some optimization problems,Dynamic Programming is overkillGreedy Strategy is simpler and more efficient.
-
7/27/2019 greedy algorithm.ppt
11/30
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong 5. Greedy Algorithms - 11http://www.cs.cityu.edu.hk/~helena
Activity-Selection Problem
For a set of proposed activities that wish touse a lecture hall, select a maximum-sizesubset of compatible activities.
Set of activities: S={a1,a2,an}
Duration of activity ai: [start_timei, finish_timei)
Activities sorted in increasing o rder of f in ish t im e:
i 1 2 3 4 5 6 7 8 9 10 11
start_timei 1 3 0 5 3 5 6 8 8 2 12
finish_timei 4 5 6 7 8 9 10 11 12 13 14
-
7/27/2019 greedy algorithm.ppt
12/30
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong 5. Greedy Algorithms - 12http://www.cs.cityu.edu.hk/~helena
Activity-Selection Problem
i 1 2 3 4 5 6 7 8 9 10 11start_timei 1 3 0 5 3 5 6 8 8 2 12
finish_timei 4 5 6 7 8 9 10 11 12 13 14
Compatible activities:{a3, a9, a11},{a
1
,a4
,a8
,a11
},{a2,a4,a9,a11}
0123456789
1011121314
time a1 a2 a3 a4 a5 a6 a7 a8 a9 a10a11
-
7/27/2019 greedy algorithm.ppt
13/30
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong 5. Greedy Algorithms - 13http://www.cs.cityu.edu.hk/~helena
Activity-Selection ProblemDynamic Programming Solution (Step 1)
Step 1. Characterize the structure of an optimal solution.
S: i 1 2 3 4 5 6 7 8 9 10 11(=n)start_timei 1 3 0 5 3 5 6 8 8 2 12finish_timei 4 5 6 7 8 9 10 11 12 13 14
eg
Definition:
Sij={akS: finish_timeistart_timek
-
7/27/2019 greedy algorithm.ppt
14/30
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong 5. Greedy Algorithms - 14http://www.cs.cityu.edu.hk/~helena
Activity-Selection ProblemDynamic Programming Solution (Step 1)
S: i 1 2 3 4 5 6 7 8 9 10 11(=n)start_timei 1 3 0 5 3 5 6 8 8 2 12finish_timei 4 5 6 7 8 9 10 11 12 13 14
Add fictitious activities: a0
and an+1
:
S: i 0 1 2 3 4 5 6 7 8 9 10 11 12start_timei 1 3 0 5 3 5 6 8 8 2 12 finish_timei 0 4 5 6 7 8 9 10 11 12 13 14
ie. S0,n+1={a1,a2,a3,a4,a5,a6,a7,a8,a9,a10,a11}= S
Note: If i>=j then Si,j=
01234567891011121314
time a1 a2 a3 a4 a5 a6 a7 a8 a9 a10a11a0 a12
-
7/27/2019 greedy algorithm.ppt
15/30
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong 5. Greedy Algorithms - 15http://www.cs.cityu.edu.hk/~helena
Substructure:
Activity-Selection ProblemDynamic Programming Solution (Step 1)
Suppose a solution to Si,jincludes activity ak,
then,2 subproblems aregenerated: Si,k, Sk,j
The problem:For a set of proposed activities that wishto use a lecture hall, select a maximum-size subset of compatible activities
Select a maximum-sizesubset of compatibleactivities from S0,n+1.
=
0123456789
1011121314
time a1 a2 a3 a4 a5 a6 a7 a8 a9 a10a11
0123456789
1011121314
time a1 a2 a3 a4 a5 a6a7a8 a9 a10a11
The maximum-size subset Ai,jof compatible activities is:
Ai,j=Ai,k U {ak} U Ak,jSuppose a solution to S0,n+1 contains a7, then,
2 subproblems are generated: S0,7 and S7,n+1
-
7/27/2019 greedy algorithm.ppt
16/30
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong 5. Greedy Algorithms - 16http://www.cs.cityu.edu.hk/~helena
Activity-Selection ProblemDynamic Programming Solution (Step 2)
Step 2. Recursively define an optimal solution
Let c[i,j] = number of activities in a maximum-size subset ofcompatible activities in Si,j.
If i>=j, then Si,j
=, ie. c[i,j]=0.
0 if Si,j=
Maxi
-
7/27/2019 greedy algorithm.ppt
17/30
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong 5. Greedy Algorithms - 17http://www.cs.cityu.edu.hk/~helena
Activity-Selection ProblemGreedy Strategy Solution
Consider any nonempty subproblemS
i,j, and let a
mbe the activity in S
i,j
with the earliest finish time.
01234567891011121314
time a1 a2 a3
okok
a4 a5
okokokok
a6
okokokok
a7
okokok
a8
okokokok
a9 a10a11
eg. S2,11={a4,a6,a7,a8,a9}
Among {a4,a6,a7,a8,a9}, a4 willfinish earliest
1. A4 is used in the solution
2. After choosing A4, there are 2subproblems: S2,4 and S4,11.But S2,4 is empty. Only S4,11
remains as a subproblem.
Then,
1. Am is used in some maximum-size subset of compatible
activities of Si,j.2. The subproblem Si,m is empty,
so that choosing am leaves thesubproblem Sm,j as the onlyone that may be nonempty.
0 if Si,j=
Maxi
-
7/27/2019 greedy algorithm.ppt
18/30
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong 5. Greedy Algorithms - 18http://www.cs.cityu.edu.hk/~helena
Activity-Selection ProblemGreedy Strategy Solution
That is,
To solve S0,12, we select a1 that will finish earliest, and solve for S1,12.To solve S1,12, we select a4 that will finish earliest, and solve for S4,12.
To solve S4,12, we select a8 that will finish earliest, and solve for S8,12.
Greedy Choices (Locally optimal choice)
To leave as much opportunity as possible for the
remaining activities to be scheduled.
Solve the problem in a
top-down fashion
Hence, to solve the Si,j:
1. Choose the activity am with the earliestfinish time.
2. Solution of Si,j = {am} U Solution ofsubproblem Sm,j
0
123456789
1011121314
time a1 a2 a3 a4 a5 a6 a7 a8 a9 a10 a11a0 a12
-
7/27/2019 greedy algorithm.ppt
19/30
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong 5. Greedy Algorithms - 19http://www.cs.cityu.edu.hk/~helena
Recursive-Activity-Selector(i,j)
1 m = i+1// Find first activity in Si,j
2 while m < j and start_timem < finish_timei
3 do m = m + 1
4 if m < j5 then return {am} U Recursive-Activity-Selector(m,j)
6 else return
Activity-Selection ProblemGreedy Strategy Solution
Order of calls:
Recursive-Activity-Selector(0,12)Recursive-Activity-Selector(1,12)
Recursive-Activity-Selector(4,12)
Recursive-Activity-Selector(8,12)
Recursive-Activity-Selector(11,12)
0123456789
1011121314
time a1 a2 a3 a4 a5 a6 a7 a8 a9 a10 a11a0 a12
m=2
Okay
m=3
Okay
m=4
break
the loop
{11}
{8,11}
{4,8,11}
{1,4,8,11}
-
7/27/2019 greedy algorithm.ppt
20/30
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong 5. Greedy Algorithms - 20http://www.cs.cityu.edu.hk/~helena
Iterative-Activity-Selector()
1 Answer = {a1}2 last_selected=1
3 for m = 2 to n
4 if start_timem>=finish_timelast_selected
5 then Answer = Answer U {am}6 last_selected = m
7 return Answer
Activity-Selection ProblemGreedy Strategy Solution
0123456789
1011121314
time a1 a2 a3 a4 a5 a6 a7 a8 a9 a10 a11a0 a12
-
7/27/2019 greedy algorithm.ppt
21/30
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong 5. Greedy Algorithms - 21http://www.cs.cityu.edu.hk/~helena
Activity-Selection ProblemGreedy Strategy Solution
For both Recursive-Activity-Selector andIterative-Activity-Selector,
Running times are (n)Reason: each am are examined once.
-
7/27/2019 greedy algorithm.ppt
22/30
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong 5. Greedy Algorithms - 22http://www.cs.cityu.edu.hk/~helena
Greedy Algorithm Design
Steps of Greedy Algorithm Design:
1. Formulate the optimization problem in theform: we make a choice and we are leftwith one subproblem to solve.
2. Show that the greedy choice can lead toan optimal solution, so that the greedychoice is always safe.
3. Demonstrate thatan optimal solution to original problem =greedy choice + an optimal solution to thesubproblem
Optimal
Substructure
Property
Greedy-
Choice
Property
A good clue that
that a greedy
strategy will solve
the problem.
-
7/27/2019 greedy algorithm.ppt
23/30
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong 5. Greedy Algorithms - 23http://www.cs.cityu.edu.hk/~helena
Greedy Algorithm Design
Comparison:
Dynamic Programming Greedy Algorithms
At each step, the choice isdetermined based on
solutions of subproblems.
At each step, we quickly make achoice that currently looks best.
--A local optimal (greedy) choice.
Bottom-up approach Top-down approach
Sub-problems are solved first. Greedy choice can be made firstbefore solving further sub-problems.
Can be slower, more complex Usually faster, simpler
-
7/27/2019 greedy algorithm.ppt
24/30
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong 5. Greedy Algorithms - 24http://www.cs.cityu.edu.hk/~helena
Huffman Codes
Huffman Codes For compressing data (sequence of characters)
Widely used
Very efficient (saving 20-90%)
Use a table to keep frequencies of occurrence
of characters.
Output binary string.
Todays
weather is nice
001 0110 0 0 100
1000 1110
-
7/27/2019 greedy algorithm.ppt
25/30
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong 5. Greedy Algorithms - 25http://www.cs.cityu.edu.hk/~helena
Huffman Codes
Frequency Fixed-length Variable-length
codeword codeword
a 45000 000 0
b 13000 001 101
c 12000 010 100
d 16000 011 111e 9000 100 1101
f 5000 101 1100
Example:
A file of 100,000 characters.
Containing only a to e
300,000 bits
1*45000 + 3*13000 + 3*12000 +3*16000 + 4*9000 + 4*5000
= 224,000 bits
1*45000 + 3*13000 + 3*12000 +3*16000 + 4*9000 + 4*5000
= 224,000 bits
eg. abc = 000001010 eg. abc = 0101100
-
7/27/2019 greedy algorithm.ppt
26/30
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong 5. Greedy Algorithms - 26http://www.cs.cityu.edu.hk/~helena
01
a:45 b:13 c:12 d:16 e:9 f:5
0
0
0
0
0
1
1
1 1
Huffman CodesThe coding schemes can be represented by trees:
Frequency Fixed-length(in thousands) codeword
a 45 000
b 13 001
c 12 010
d 16 011e 9 100
f 5 101
100
86 14
1401
58 28
a:45 b:13 c:12 d:16 e:9 f:5
0
0
0
0
0
1
1
1 1
Frequency Variable-length(in thousands) codeword
a 45 0
b 13 101
c 12 100
d 16 111e 9 1101
f 5 1100
100
55
0125 30
0
0
0
1
1
1
a:45
14
e:9 f:5
0 1d:16b:13 c:12
01
a:45 b:13 c:12 d:16 e:9 f:5
0
0
0
0
0
1
1
1 114
0158 28
a:45 b:13 c:12 d:16 e:9 f:5
0
0
0
0
0
1
1
1 1
86 14
1401
58 28
a:45 b:13 c:12 d:16 e:9 f:5
0
0
0
0
0
1
1
1 1
Not a fullbinary tree
A full binary treeevery nonleaf node
has 2 children
A file of 100,000
characters.
-
7/27/2019 greedy algorithm.ppt
27/30
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong 5. Greedy Algorithms - 27http://www.cs.cityu.edu.hk/~helena
Huffman Codes
Frequency Codeword
a 45000 0b 13000 101
c 12000 100
d 16000 111
e 9000 1101
f 5000 1100
100
55
0125 30
0
0
0
1
1
1
a:45
14
e:9 f:5
0 1d:16b:13 c:12
To find an optimal code for a file:
1. The coding must be unambiguous.Consider codes in which no codeword is also a
prefix of other codeword. => Prefix Codes
Prefix Codes are unambiguous.
Once the codewords are decided, it is easy tocompress (encode) and decompress (decode).
2. File size must be smallest.=> Can be represented by a full binary tree.=> Usually less frequent characters are at bottom
Let C be the alphabet (eg. C={a,b,c,d,e,f})For each character c, no. of bits to encode all cs
occurrences = freqc*depthcFile size B(T) = cCfreqc*depthcEg. abc is coded as 0101100
-
7/27/2019 greedy algorithm.ppt
28/30
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong 5. Greedy Algorithms - 28http://www.cs.cityu.edu.hk/~helena
Huffman Codes
Huffman code (1952) was invented to solve it.
A Greedy Approach.
Q: A min-priority queue f:5 e:9 c:12 b:13 d:16 a:45
100
55
25 30
a:45
14
e:9 f:5
d:16b:13 c:12
c:12 b:13 d:16 a:4514
f:5 e:9
d:16
a:45
14
25
c:12 b:13
30
f:5 e:9
a:45
d:1614
25
c:12 b:13
30
55
f:5 e:9
d:16 a:4514 25
c:12 b:13f:5 e:9
How do we find the
optimal prefix code?
-
7/27/2019 greedy algorithm.ppt
29/30
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong 5. Greedy Algorithms - 29http://www.cs.cityu.edu.hk/~helena
Huffman Codes
HUFFMAN(C)
1 Build Q from C
2 For i = 1 to |C|-1
3 Allocate a new node z
4 z.left = x = EXTRACT_MIN(Q)5 z.right = y = EXTRACT_MIN(Q)
6 z.freq = x.freq + y.freq
7 Insert z into Q in correct position.
8 Return EXTRACT_MIN(Q)
Q: A min-priority queue f:5 e:9 c:12 b:13 d:16 a:45
c:12 b:13 d:16 a:4514
f:5 e:9
d:16 a:4514 25
c:12 b:13f:5 e:9
.
If Q is implemented as a binary
min-heap,
Build Q from C is O(n)
EXTRACT_MIN(Q) is O(lg n)
Insert z into Q is O(lg n)
Huffman(C) is O(n lg n)
How is it greedy?
-
7/27/2019 greedy algorithm.ppt
30/30
CS3381 Des & Anal of Alg (2001-2002 SemA)
Greedy Algorithms
SummaryCasual Introduction: Two Knapsack Problems
An Activity-Selection Problem
Greedy Algorithm Design
Steps of Greedy Algorithm Design
Optimal Substructure Property
Greedy-Choice PropertyComparison with Dynamic Programming
Huffman Codes
top related