the greedy approachgsyoung/cs5300/cs530 hybrid notes/cs530 … · greedy 3 young cs 530 adv. algo....

Greedy

1

YoungCS 530 Adv. Algo.

Greedy 1

The Greedy Approach


Greedy 2

General idea:

Given a problem with n inputs, we are required to obtain a subset that maximizes or minimizes a given objective function subject to some constraints.

Feasible solution — any subset that satisfies some constraints

Optimal solution — a feasible solution that maximizes or minimizes the objective function

Greedy

2


Greedy 3

procedure Greedy (A, n)begin

solution ← Ø;for i ← 1 to n do

x ← Select (A); // based on the objective // function

if Feasible (solution, x), then solution ← Union (solution, x);

end;Select: A greedy procedure, based on a given objective function, which selects input from A, removes it and assigns its value to x.

Feasible: A boolean function to decide if x can be included into solution vector (without violating any given constraint).


Greedy 4

About Greedy method

The n inputs are ordered by some selection procedure which is based on some optimization measures.

It works in stages, considering one input at a time. At each stage, a decision is made regarding whether or not a particular input is in an optimal solution.

Greedy

3


Greedy 5

1. Minimum Spanning Tree ( For Undirected Graph)

The problem:1) Tree

A Tree is connected graph with no cycles.

2) Spanning Tree

A Spanning Tree of G is a tree which contains all verticesin G.Example:

G:


Greedy 6

b) Is G a Spanning Tree?

Key: Yes Key: No

Note: Connected graph with n vertices and exactly n – 1 edges is Spanning Tree.

3) Minimum Spanning Tree

Assign weight to each edge of G, then Minimum Spanning Tree is the Spanning Tree with minimum total weight.

Greedy

4


Greedy 7

Example:a) Edges have the same weight

G: 1

2 3

4 5 6 7

8


Greedy 8

DFS (Depth First Search)

1

2 3

4

5

6

7

8

Greedy

5


Greedy 9

BFS (Breadth First Search)

1

2 3

4 5 6 7

8


Greedy 10

b) Edges have different weights

G:

DFS

1 2

3

45

6

16

19

21

33

11

14

18

10

5

6

1

5

2

3

4

6

16

5

10

143378

331410516=

++++=Cost

Greedy

6


Greedy 11

BFS

Minimum Spanning Tree (with the least total weight)

1 2 3

45

6

16

1921

5

6

6765211916

=++++=Cost

1 2

3

4 56

165

611

18 5618116516

=++++=Cost


Greedy 12

Algorithms:1) Prim’s Algorithm (Minimum Spanning Tree)

Basic idea:Start from vertex 1 and let T ← Ø (T will contain all edges in the S.T.); the next edge to be included in T is the minimum cost edge(u, v), s.t. u is in the tree and v is not.

Example: G1 2

3

45

6

16

19

21

33

11

14

18

10

5

6

Greedy

7


Greedy 13

1

2 5 6

1619 21

1 2

5 6 3 64

19 21 5 6 11

1 1 2

1 2 3

5 6 4 6 4

1921 6 11 10

1 2 3 4

5 6 6 5 6

19 21 11 18 14

1 2 3 1 2 3

4

S.T. S.T.

S.T.S.T.

(Spanning Tree)


Greedy 14

1 2 3 4 6

5 5 5

19 18 33

1 2 3

4 6

S.T.

(n – # of vertices, e – # of edges)It takes O(n) steps. Each step takes O(e) and e ≤ n(n-1)/2Therefore, it takes O(n3) time.

With clever data structure, it can be implemented in O(n2).

).( 2nO⇒

1 2 3

4 6

5

S.T.

Cost = 16+5+6+11+18=56Minimum Spanning Tree

Greedy

8


Greedy 15

Example:

Step 1: Sort all of edges(1,2) 10 √(3,6) 15 √(4,6) 20 √(2,6) 25 √(1,4) 30 × reject Qcreate cycle(3,5) 35 √

1 2

3

5

4

6

1050

35

40

55

15

4525

30

20

2) Kruskal’s AlgorithmBasic idea:

Don’t care if T is a tree or not in the intermediate stage, as long as the including of a new edge will not create a cycle, we include the minimum cost edge


Greedy 16

Step 2: T

1 2

1 2 3 6

1 2 3 6 4

1 2 3 6 4

1 2 3 6 4

5

Greedy

9


Greedy 17

How to check:

adding an edge will create a cycle or not?

If Maintain a set for each group(initially each node represents a set)Ex: set1 set2 set3

∴ new edge

from different groups ⇒ no cycle created

Data structure to store sets so that:i. The group number can be easily found, and

ii. Two sets can be easily merged

1 2 3 6 4 5

2 6


Greedy 18

While (T contains fewer than n-1 edges) and (E ≠ ∅ ) doBegin

Choose an edge (v,w) from E of lowest cost;Delete (v,w) from E;If (v,w) does not create a cycle in Tthen add (v,w) to T else discard (v,w);

End;

With clever data structure, it can be implemented in O(e log e).

Kruskal’s algorithm

Greedy

10


Greedy 19

So, complexity of Kruskal is )(eLogeO

LognLognLogenne 22

)1( 2 =≤⇒−

≤Q

)()( eLognOeLogeO =⇒

3) Comparing Prim’s Algorithm with Kruskal’s Algorithm

i. Prim’s complexity is

ii. Kruskal’s complexity is

if G is a complete (dense) graph, Kruskal’s complexity is .

if G is a sparse graph, Kruskal’s complexity is

)( 2nO

)(eLognO

)( 2LognnO

).(nLognO


Greedy 20

2. Dijkstra’s Algorithm for Single-Source Shortest PathsThe problem: Given directed graph G = (V, E),

a weight for each edge in G,a source node v0,

Goal: determine the (length of) shortest paths from v0 to all the remaining vertices in G

Def: Length of the path: Sum of the weight of the edgesObservation:

May have more than 1 paths between w and x (y and z)But each individual path must be minimal length

(in order to form an overall shortest path form v0 to vi )

V0 Vi

w x y z

shortest paths from v0 to vi

Greedy

11


Greedy 21

1 if shortest path (v, w) is defined0 otherwises(w) =

jjDist ∀)( in the vertex set V= the length of the shortest path from v to j

ijFrom =)( if i is the predecessor of jalong the shortest path from v to j

Notation

cost adjacency matrix Cost, ∀1 ≤ a,b ≤ V

Cost (a, b) = cost from vertex i to vertex j if there is a edge0 if a = b∞ otherwise


Greedy 22

Example:V0 V1

V2 V3

V4

V5

50 10

2010

15

15

20

3

35

30

45

a) Cost adjacent matrix

∞∞∞∞∞∞∞∞∞∞∞∞∞∞∞∞∞∞∞

03030

3502015020

101504510500

5

4

3

2

1

0

543210

vvvvvv

vvvvvv

Greedy

12


Greedy 23

b) Steps in Dijkstra’s Algorithm

1. Dist (v0) = 0, From (v0) = v0 2. Dist (v2) = 10, From (v2) = v0

V0 V1

V2 V3

V4

V5

50 10

315

2035

30

1520 10

45

V0

V2

V1

V3

V4

V5

50 10

2010

15

15

20 35

30

3

45


Greedy 24

3. Dist (v3) = 25, From (v3) = v2 4. Dist (v1) = 45, From (v1) = v3

V0

V2 V3

V1 V4

V5

50 10

45

20 10

15

15

2035

3

V0

V2

V1

V3

V4

V5

50

45

10

20 1015

15

20

35

30

3

30

Greedy

13


Greedy 25

5. Dist (v4) = 45, From (v4) = v0 6. Dist (5) = ∞

V0 V1

V2 V3

V4

V5

45

50 10

20 10

15

20

15

35

30

3

V0 V1

V2 V3

V4

V5

45

50 10

20 10

15

20

15

35

30

3


Greedy 26

c) Shortest paths from source v0

v0 → v2 → v3 → v1 45

v0 → v2 10

v0 → v2 → v3 25

v0 → v4 45

v0 → v5 ∞

Greedy

14


Greedy 27

Dijkstra’s algorithm:

procedure Dijkstra (Cost, n, v, Dist, From)// Cost, n, v are input, Dist, From are output

beginfor i ← 1 to n do

for num ← 1 to (n – 1) do choose u s.t. s(u) = 0 and Dist(u) is minimum;

for all w with s(w) = 0 doif

end;

(cont. next page)

;)();,()(

;0)(

viFromivCostiDist

is

←←

←

;1)( ←vs

;1)( ←us

;)();,()()(

uwFromwuCostuDistwDist

←+←

))(),()(( wDistwuCostuDist <+


Greedy 28

Ex: 1

25

34

10 50

10

100 30

20

50

5

a) Cost adjacent matrix

∞∞∞∞∞∞∞∞∞∞∞∞

010020

50050

1010030500

54321

0504030201

Greedy

15


Greedy 29

b) Steps in Dijkstra’s algorithm

1. Dist (1) = 0, From (1) = 1 2. Dist (5) = 10, From (5) = 1

1

25

34

10 50

10

100 30

20

50

5

50

1

25

34

10 50

10

100 30

205


Greedy 30

3. Dist (4) = 20, From (4) = 5 4. Dist (3) = 30, From (3) = 11

25

34

10 50

10

100 30

20

50

5

1

25

34

10 50

10

100 30

20

50

5

5. Dist (2) = 35, From (2) = 31

25

34

10 50

10

100 30

20

50

5

Shortest paths from source 1

1 → 3 → 2 35

1 → 3 30

1 → 5→ 4 20

1 → 5 10

Greedy

16


Greedy 31

3. Optimal Storage on Tapes

The problem:

Given n programs to be stored on tape, the lengths of these n programs are l1, l2 , . . . , ln respectively.Suppose the programs are stored in the order of i1, i2 , . . . , in

Let tj be the time to retrieve program ij.

Assume that the tape is initially positioned at the beginning.

tj is proportional to the sum of all lengths of programs stored infront of the program ij.


Greedy 32

The goal is to minimize MRT (Mean Retrieval Time),

i.e. want to minimize

∑=

n

jjt

n 1

1

∑∑= =

n

j

j

kkil

1 1

Ex:There are n! = 6 possible orderings for storing them.

)3,10,5(),,(,3 321 == llln

order total retrieval time MRT

123456

1 2 31 3 22 1 32 3 13 1 23 2 1

5+(5+10)+(5+10+3)=385+(5+3)+(5+3+10)=31

10+(10+5)+(10+5+3)=4310+(10+3)+(10+3+5)=413+(3+5)+(3+5+10)=293+(3+10)+(3+10+5)=34

38/331/343/341/329/334/3

Note: The problem can be solved using greedy strategy,just always let the shortest program goes first.

( Can simply get the right order by using any sorting algorithm)

Smallest

Greedy

17


Greedy 33

Analysis:

Try all combination: O( n! )

Shortest-length-First Greedy method: O (nlogn)

Shortest-length-First Greedy method: Sort the programs s.t. and call this ordering L.

Next is to show that the ordering L is the best

Proof by contradiction:Suppose Greedy ordering L is not optimal, then there exists some other permutation I that is optimal.I = (i1, i2, … in) ∃ a < b, s.t. (otherwise I = L)

ba ii ll >

nlll ≤≤≤ . . .21


Greedy 34

Interchange ia and ib in and call the new list I′ :I

I′

… ia ia+1 ia+2 … ib …

x

… ib ia+1 ia+2 … ia …

xIn I′, Program ia+1 will take less (lia

- lib) time than in I to be retrieved

In fact, each program ia+1 , …, ib-1 will take less (lia- lib

) time For ib, the retrieval time decreases x + liaFor ia, the retrieval time increases x + lib

)(0))((

)()())(1()()(

→←>−−=

+−++−−−=′−

ba

baba

ii

iiii

llab

lxlxllabItotalRTItotalRT

Therefore, greedy ordering L is optimal Contradiction!!

SWAP

Greedy

18


Greedy 35

4. Knapsack Problem

The problem:Given a knapsack with a certain capacity M,n objects, are to be put into the knapsack, each has a weight and a profit if put in the knapsack .

The goal is find where

s.t. is maximized and

Note: All objects can break into small piecesor xi can be any fraction between 0 and 1.

nwww ,,, 21 L

nppp ,,, 21 L

),,,( 21 nxxx L 10 ≤≤ ix

∑=

n

iii xp

1∑=

≤n

iii Mxw

1


Greedy 36

Example:

)15,24,25(),,()10,15,18(),,(

203

321

321

==

==

pppwww

Mn

Greedy Strategy#1: Profits are ordered in nonincreasing order (1,2,3)

2.2801515224125

)0,152,1(),,(

3

1

321

=×+×+×=

=

∑=i

ii xp

xxx

Greedy

19


Greedy 37

Greedy Strategy#2: Weights are ordered in nondecreasing order (3,2,1)

311153224025

)1,32,0(),,(

3

1

321

=×+×+×=

=

∑=i

ii xp

xxx

Greedy Strategy#3: p/w are ordered in nonincreasing order (2,3,1)

5.312115124025

)21,1,0(),,(

)1,3,2(

5.11015

6.11524

4.11825

3

1

321

3

3

2

2

1

1

=×+×+×=

=

⇒

==

==

==

∑=i

ii xp

xxx

wpwpwp

Optimal solution


Greedy 38

Analysis:Sort the p/w, such that Show that the ordering is the best.

Proof by contradiction:Given some knapsack instanceSuppose the objects are ordered s.t.

let the greedy solution be

Show that this ordering is optimalCase1: it’s optimalCase2: s.t.

where

n

n

wp

wp

wp

≥≥≥ L2

2

1

1

n

nwp

wp

wp

≥≥≥ L2

2

1

1

),,( 21 nxxxX L=

)1,,1,1( L=X)0,,0,,,1,1( LL jxX = ∑

=

=n

iii Mxw

1

10 ≤≤ jx

Greedy

20


Greedy 39

Assume X is not optimal, and then there exists

s.t. and Y is optimal

examine X and Y, let yk be the 1st one in Y that yk ≠ xk.

),,,( 21 nyyyY L=

∑∑==

>n

iii

n

iii xpyp

11

),,,,,,( 121 nkk xxxxxX LLLL −=

),,,,,,( 121 nkk yyyyyY LLLL −=

Now we increase yk to xk and decrease as many of as necessary, so that the capacity is still M.

),,( 1 nk yy L+

same yk ≠ xk⇒ yk < xk


Greedy 40

Let this new solution be

where

),,,( 21 nzzzZ L=kixz ii ≤≤∀= 1

∑+=

⋅−=⋅−n

kiiiikkk wzywyzand

1

)()(

∑∑∑∑

∑∑∑

=+===

+===

=⋅

⋅−−⋅−+≥

⋅≤∴

+≥∀≥

⋅−−⋅−+=

n

iii

k

kn

kiiiikkk

n

iii

n

iii

ik

ki

i

i

k

k

n

kiiiikkk

n

iii

n

iii

ypwpwzywyzypzp

wwpp

kiwp

wp

pzypyzypzp

1111

111

)()(

)(

1

)()(

Q

0

Greedy

21


Greedy 41

So,if

else (Repeat the same process. At the end, Y can be transformed into X.⇒ X is also optimal.Contradiction! )

)()()( →←> yprofitzprofit

)()( yprofitzprofit =

→←

the greedy approachgsyoung/cs5300/cs530 hybrid notes/cs530 … · greedy 3 young cs 530 adv. algo....

Documents