lecture8 - greedy algorithms

8/14/2019 Lecture8 - Greedy Algorithms

1/61

COMP2230 Introduction to

Algorithmics

Lecture 8

Slides based on:A. Levitin. The Design and Analysis of Algorithms

R. Johnsonbaugh & M. Schefer. Algorithms

Slides by P. Moscato and Y. Lin


2/61

Lecture Overview

Greedy Algorithms Text, Chapter 7


3/61

Change-Making Problem

We have an infinite supply of coins of differentdenominations d1> d2> . . . > dm.

We want to pay any given amount, using theminimumnumber of coins.

(This problem is faced every day by millions andmillions of cahiers . . .)


4/61

Change-Making Problem

For example, we have an infinite supply of each of thefollowing

1c, 2c, 5c, 10c, 20c, 50c,1,2

How would you pay for

1.05 2.40 30c 85c
http://en.wikipedia.org/wiki/2_euro_coinshttp://en.wikipedia.org/wiki/2_euro_coinshttp://en.wikipedia.org/wiki/2_euro_coinshttp://en.wikipedia.org/wiki/2_euro_coinshttp://en.wikipedia.org/wiki/2_euro_coinshttp://en.wikipedia.org/wiki/2_euro_coinshttp://en.wikipedia.org/wiki/2_euro_coinshttp://en.wikipedia.org/wiki/2_euro_coins


5/61

Making change (cont)

This algorithm makes change for an amount Ausing coins of denominationsdenom[1] > denom[2] > > denom[n] = 1.

Input Parameters: denom, AOutput Parameters: None

greedy_coin_change(denom,A) {i= 1

while (A> 0) {

c= A/denom[i]

println(use+ c + coins of denomination+ denom[i])

A= A- c* denom[i]

i= i+ 1}

}


6/61

Making change (cont)

greedy_coin_change is a Greedy Algorithm keeps track of sum of coins paid so far checks whether there is still more to pay

If there is more to pay, algorithm adds coins of largest amountpossible (so as not to exceed amount needed)

It never changes its mind

Be greedy seems like a good idea! Could you see when greed is bad?


7/61

You should expect thisBad cases

Greed might end up with a solution which is not optimal (checkcoin denominations 1, 6 and 10 and use greedy for 22).

Greed might end up with a solution which is not feasible (if

the smallest coin denomination is larger than 1, solution mightnot always be feasible, e.g. coin denominations 3, 4 and 5 andamount 6).


8/61

General characteristicsof greedy algorithms

On each step, Greedy algorithms make a choice that is:

Feasible (satisfies the constraints)

Locally Optimal (the best choice among all feasible choices)

Irrevocable (cannot be changed latter)


9/61

General characteristicsof greedy algorithms

Greedy algorithms generally involve:

A computable selection functionused to prioritize (whichcandidate is most desirable?)

set of candidates to be considered for inclusion in solution a set of candidates which have already been chosen a set of candidates which have already been rejected

solution function (to check if our set isa solution) feasible function, to check if our set is feasible, i.e. can be

extended to make a solution For optimization problems an objective function

(what is our score ?)


10/61

Greed: the form

Make-change version: candidates

(large) set of coins

solution function check if coins chosen =

exact amount paid feasible set

total value of coins < totalamount to be paid

selection function choose highest valued coin

of remaining set objective function count # of coins

(C,S)

{C is the set of candidates}

S { will hold the solution}

C and not (S)

x (C)

C C \ {x}

(S {x})

S (S {x})

(S)

S

no solution


11/61

Greed: the general form

(C,S)

{C is the set of candidates}

S { will hold the solution}

C and not (S)

x (C)

C C \ {x}

(S {x})

S (S {x})

(S)

S

no solution

( )

const = {c0, c1, ....}

{ will hold the solution}

0 {sum of coins in S}

x largest item in Csuch that s + x val

no such item

No solution

S S {a coin of value x}

s s + x


12/61

Minimum Spanning Trees

MST-Problem

Input:G=(V,E,W) a connectedundirected graph with non-negative weights on edges

Output:a connected sub-graphG=(V,T) of G such that the sum

of the edge lengths of G isminimized.

10

5

8

7

16

2

1

8


13/61

MST (cont)

MST must be a tree. Why ?

Applications: minimum cost networks

cabling

transport 5

8

2

1


14/61

Greedy strategies for MST

Identify what are thecandidates

What would the startingpoint be?

How do we select the nextcandidate? what about rejection?

When are we finished?

edges? vertices?something else?


15/61

Greedy by edges

start with an empty edgeset T

select next shortest edge(not already seen)

reject if creates a cycle

stop when we haveselected n-1 edges

what does the selectedset represent?


What are our candidates?





16/61

Greedy by vertices

Start with a single node is there a special one we

should start at?

Select shortest edge Reject if it doesn't connect

a node in current set to anode we havent seen yet

Stop when we haveconnected all |V| nodes






K k l G d b d


17/61

Kruskal: Greedy by edgesExample

Connected Components:

start: {1} {2} {3} {4} {5} {6} {7} {8}

1 2

5643

4

8

64

37

1 2 3

4 5 6

7


18/61

Prims algorithm: Greedy byvertices - Example

starting set: VT= {1}

1 2

5643

4

8

64

37

1 2 3

4 5 6

7


19/61

The greedy template on MST

Candidates are edgesin G Set of edges is a solution if it is a tree which spans V Set of edges is feasible if doesnt contain a cycle Objective function:

minimize total length of edges in the solution

Terminology:

Promising set of edges: one which can be extended to form optimal

solution

Leaving edge: edge with exactly one end in particular set of vertices


20/61

Promising Edges

Promising Lemma BTW, lemmas are auxiliary statements used to form a basis for more

relevant statements (theorems) to be used later on; both lemmas andtheorems need to be proved otherwise they are just conjectures

Given: G = (V,E,W) as before Given VTV Given T E a promising set of edges, such that no edge

leaves VT Let e be the shortest edge leaving VT

Then T {e} is promising


21/61

Proof of Promising Lemma

The proof will be by contradiction we assume that what were

going to prove is false, and thenshow that thatassumption isfalse

assume T {e} is notpromising

We have a promising set (T)

and the edge e leaving VT

T

eVT


22/61

Proof (cont) T is promising, so there is a MST U containing

edges of T

T is the section of the MST U that weve alreadyfound consider U T {e} not promising, so U {e} must have a cycle

- why? because T {e} not promising and thus U must

have another edge uleaving VTon the path

between the end vertices of e AND umust be atleast as large as e, that is, w(u) w(e) (because eis the shortest edge leaving VT)

So, lets create a new tree T, by deleting u fromU and adding in e

T= U\{u}{e}

Since U was MST, so is T

Now T{e} T,

So T

{e} is promising - so we have our proof !

T

e

u


23/61

Kruskals Algorithm

Initially, set of edges T is empty

Each node of G = (V,T) is in itself a connected component

Each step: smallest not-yet-chosen candidate edge e is considered

If T{e}contains no cycle, that is, if e bridges two differentconnected components, then e is added to T

Otherwise, e is rejected (and never considered again!)

Each addition to T reduces the number of connected components by 1(why?)

Terminate when only one connected component remains.


24/61

Kruskals Algorithm - Pseudocode

Algorithm Kruskal(G)

//Input: A weighted connected graph G//Output: A minimum spanning tree of G, given by its set of edges ETSort E in non-decresing order of the edge weights

w(ei1) . . . w(ei|E|)

ET; //initializing the set of tree edgescounter 0; //initializing the number of tree edgesk 0; //initializing the number of processed edgeswhilecounter < |V|-1 do

k k + 1;

ifET

{eik

} is acyclic

ETET {eik};

counter counter + 1;

returnET


25/61

Kruskal - an example

Connected Components: start: {1} {2} {3} {4} {5} {6} {7} Shortest edge: (1,2); no cycle accept; connected components: {1,2} {3} {4} {5} {6} {7}

Shortest edge: (2,3); no cycle accept; connected components: {1,2,3} {4} {5} {6} {7} Shortest edge: (4,5); no cycle accept; connected components: {1,2,3} {4,5} {6} {7} Shortest edge: (6,7); no cycle accept; connected components: {1,2,3} {4,5} {6,7} Shortest edge: (1,4); no cycle accept; connected components: {1,2,3,4,5} {6,7} Shortest edge: (2,5); creates cycle reject Shortest edge: (4,7); no cycle accept; connected components:{1,2,3,4,5,6,7}

1 2

5643

4

8

64

37

1 2 3

4 5 6

7


26/61

Correctness of Kruskals Algorithm

Apply Promising Lemma and use induction

Empty set is promising

Assume by inductive hypothesis that set Tof kedgesselected by the Kruskals algorithm is promising. If ashortest edge edoes not create a cycle then byPromising Lemma T{e} is also promising.


27/61

Implementing Kruskals Algorithm

Kruskals algorithm can be efficiently implementedusing disjoint sets.

Remember operations on disjoint sets: makeset(x) find(x) tells us which component x is in

union(a,b) merges two components


28/61

Kruskal algorithm

Kruskals algorithm finds a minimum spanning tree in aconnected, weighted graph with vertex set {1, ... , n} .The input to the algorithm is edgelist, an array ofedge, and n. The members of edge are

vand w, the vertices on which the edge is incident. weight, the weight of the edge.

The output lists the edges in a minimum spanning

tree. The function sortsorts the array edgelistinnondecreasing order of weight.


29/61

Algorithm 7.2.4 Kruskals Algorithm

Input Parameters: edgelist, nOutput Parameters: Nonekruskal(edgelist,n) {

sort(edgelist)

fori= 1 to n

makeset(i)

count= 0

i= 1

while(count< n- 1) {

if(findset(edgelist[i].v) != findset(edgelist[i].w)) {println(edgelist[i].v+ + edgelist[i].w)

count= count+ 1

union(edgelist[i].v,edgelist[i].w)

}

i= i+ 1

}

}


30/61

Analysis of Kruskals AlgorithmAssume we have n nodes and |E| edges.

Worst case: we have to look at all edges and nodes.

Q(|E| log |E|) to sort edges

Q(n) to initialize disjoint node sets

O(|E|log n) for findset and union operations At most 2 |E| findset

n-1 unions

|E| n-1, so n-1 = O(|E|)

Each findset or union takes O(log n) log |E| is Q( log n)

the total is:

Q(|E| log n)


31/61

Prims Algorithm

Greedy by verteces Initially set VTconsists of a single vertex of V and T is empty

Each step: the smallest not-yet-chosen candidate edge e isconsidered

If e leaves VTthen e is added to T and the endpoint not in VTisadded to VT

otherwise e is rejected.

Terminate when VT=V (all nodes used up)


32/61

Prims algorithm Pseudocode

algorithm Prim(G)

//Input: A weighted connected graph G

//Output: A minimum spanning tree of G, given by its set of edges ETVT{v0}

ETfori = 1 to |V|-1

find a minimum weight edge e*=(v*,u*), among all

edges leaving VT (that is, v*VT, u* VT)

VTVT{u*}

ETET {e*}

returnET


33/61

Prims algorithm - example

starting set: VT= {1} {1,2}

{1,2,3} {1,2,3,4} {1,2,3,4,5} {1,2,3,4,5,7} {1,2,3,4,5,7,6}

1 2

5643

4

8

64

37

1 2 3

4 5 6

7


34/61

Is Prims Algorithm Correct?

Apply Promising Lemma anduse induction

Tree containing a singlevertex is promising

Assume by inductivehypothesis that T= (VT, ET)of kedges selected by thePrims algorithm is promising.

Let e*=(v*,u*), be a shortestedge leaving VT. Then byPromising Lemma T = (VT{u*}, ET{e*} ) is alsopromising.

e

v

u


35/61

Analysis of Prims algorithm

To improve complexity, for each vertex u not in the current treewe can maintain the nearest tree vertex and the weight of thecorresponding edge; we call this the label of vertex u

When we add a new vertex u* to the tree, we need to: add u* to VT

for each vertex u in V-VT which is connected to u* by a shorteredgethen its label, we need to update the label

Time Complexity of Prims algorithm will depend on the datastructures used

If we use adjacency matrix for the graph and an unorderedarray for the list of labels then time complexity is Q(n2) If we use adjacency lists for graph G and min-heap for list of

labels, then the time complexity is O(|E| log n)


36/61

Greedy Shortest Path

Whats difference in this toPrim/Kruskal Algorithms?

Is this just another MST?


What do we start with?When are we finished?

What do we return?

What does a general step in thealgorithm look like?

Set of C = candidate nodes (yetto be considered)

Set of S = nodes already chosen

S initially contains source

S maintained so that currentshortest paths are stored

At each step, select closestnode to S in C

Stop when all nodes in S


37/61

Dijkstras Algorithmalgorithm Dijakstra(G,s)

//Diajkstras algorithm for a single source (s) shortest paths

//Input: A weighted connected graph G = (V,E,W)with non-negative weights and a vertex s

//Output: The length dvof a shortest path from sto vfor eachvertex v

forevery vertex v in V do

dv; pvnull

Penqueue(Q,v,d) //initialise vertex priority in the priority queueds0; decrease(Q,v,dv)//updatepriority of s with dsVT

fori = 1 to |V|-1 do

u* Pdequeue(Q) //dequeuethe minimum priority element

VTVT{u*}

forevery vertex in V-VTthat is adjacent to u* do

ifdu*+ w(u*,u) < du

dudu*+ w(u*,u); puu*;decrease(Q,v,dv)
http://localhost/var/www/apps/conversion/tmp/scratch_3//updatehttp://localhost/var/www/apps/conversion/tmp/scratch_3//dequeuehttp://localhost/var/www/apps/conversion/tmp/scratch_3//dequeuehttp://localhost/var/www/apps/conversion/tmp/scratch_3//update


38/61

Dijkstra Example

6

5

1 2

4 3

25

10

50

200

20

30

20

5

12

1

0

4


39/61

6

5

1 2

4 3

25 50

200

20

30

20

5

1

0

1 node 1 PQ = [50,200,10,,25]

4

node 4 PQ= [40,30,-x-,,22]

6

node 6 PQ= [40,30,-x-,22,-x-]

5

node 5 PQ= [40,30,-x-,-x-,-x-]

3

node 3 PQ= [35,-x-,-x-,-x-,-x-]

stop

10

12


40/61

Is it correct?

Terminology: A path from source to v is

specialif all intermediate nodesbelong to S

We use an array D to storelength of shortest specialpaths

We need to prove 2 things: If node i 1 is in S, then D[i]

gives length of shortest pathfrom 1 to i

If node i is not in S, then D[i]gives shortest specialpath from1 to i

How could we prove this? consider the proof techniques

weve seen so far, which style ismore appropriate?

Why is this a good thing to prove? Why might this proof be more

straight forward than provingcorrectness after its all finished?


41/61

Proof

(1) If node i 1 is in S, then D[i]gives length of shortest path from1 to i

(2) If node i is not in S, then D[i]

gives shortest specialpath from 1to i

Basis?Inductive assumption?

assume both conditions are truejustbeforewe add node v

(1) Prove path to v passes onlythroughnodes in S.

How? What other options are there?

(2) Consider node w v which is not in S How do we know the special path to

w is still shortest if v is included in it? if v is not included?


42/61


43/61

Prims algorithm Pseudocode

algorithm Prim(G)//Input: A weighted connected graph G//Output: A minimum spanning tree of G, given by its

set of edges ETVT{v0}

ET

fori = 1 to |V|-1

find a minimum weight edge e*=(v*,u*), among all edges leaving VT(that is, v*V

T, u* V

T)

VTVT{u*}

ETET {e*}

returnET


44/61

To improve complexity, for each vertex u not in thecurrent tree we can maintain the nearest tree vertex

and the weight of the corresponding edge; we callthis the label of vertex u.

When we add a new vertex u* to the tree, we need to: add u* to VT

for each vertex u in V-VT which is connected to u*by a shorter edgethen its label, we need to updatethe label


45/61

Example

Tree vertices:a(-,-)

Fringe vertices:b(a,3), e(a,6), f(a,5)

Unseen vertices:

c(-,), d(-, )

Nearest vertex:b(a,3)

a f d

b

e

c1

3 4 4 6

55

62

8


46/61

Example

Tree vertices:a(-,-), b(a,3)

Fringe vertices:c(b,1),e(a,6), f(b,4)

Unseen vertices:

d(-, )

Nearest vertex:c(b,1)

a f d

b

e

c1

3 4 4 6

55

62

8


47/61

Example

Tree vertices:a(-,-), b(a,3), c(b,1)

Fringe vertices:e(a,6), f(b,4), d(c, 6)

Unseen vertices:

Nearest vertex:f(b,4)

a f d

b

e

c1

3 4 4 6

55

62

8


48/61

Example

Tree vertices:a(-,-), b(a,3), c(b,1), f(b,4)

Fringe vertices:e(f,2), d(f, 5)

Unseen vertices:

Nearest vertex:e(f,2)

a f d

b

e

c1

3 4 4 6

55

62

8


49/61

Example

Tree vertices:a(-,-), b(a,3), c(b,1), f(b,4),

e(f,2)

Fringe vertices:d(f, 5)

Unseen vertices:

Nearest vertex:d(f, 5)

a f d

b

e

c1

3 4 4 6

55

62

8


50/61

Example

Tree vertices:a(-,-), b(a,3), c(b,1), f(b,4),

e(f,2), d(f, 5)

Fringe vertices:

Unseen vertices:

Nearest vertex:

a f d

b

e

c1

3 4 4 6

55

62

8


51/61

Analysis of Prims algorithm

Time Complexity of Prims algorithm will depend onthe data structures used; note that the list of labelson fringe vertices is a priority queue

If we use adjacency matrix for the graph and anunordered array for the list of labels then timecomplexity is Q(n2): To create a list of labels Q(n)

At each step finding a minimum in the list and updating thelist takes Q(n)

There are n-1 steps, thus Q(n2) in total


52/61


53/61

Dijkstras algorithm is very similar to Prims. Toimprove complexity, for each vertex u not in thecurrent tree we can maintain the parent tree vertexand the distance from the source; we call this thelabel of vertex u.

When we add a new vertex u* to the tree, we need to: add u* to VT for each vertex u in V-VT which is connected to u*

by an edgeof the weight w(u*,u) such that du*+w(u*,u) < duthen we need to update the label of uby du*+ w(u*,u) and set parent to u*

Dijkstras Algorithm for computing a single-source shortest paths


54/61


55/61


56/61

Example

Tree vertices:a(-,0)

Fringe vertices:b(a,3), e(a,6), f(a,5)

Unseen vertices:

c(-,), d(-, )

Nearest vertex:b(a,3)

a f d

b

e

c1

3 4 4 6

55

62

8


57/61

Example

Tree vertices:a(-,0), b(a,3)

Fringe vertices:e(a,6), f(a,5), c(b,1+3)

Unseen vertices:

d(-, )

Nearest vertex:c(b,1+3)

a f d

b

e

c1

3 4 4 6

55

62

8


58/61

Example

Tree vertices:a(-,0), b(a,3), c(b,4)

Fringe vertices:e(a,6), f(a,5), d(c,6+4)

Unseen vertices:

Nearest vertex:f(a,5)

a f d

b

e

c1

3 4 4 6

55

62

8


59/61

Example

Tree vertices:a(-,0), b(a,3), c(b,4), f(a,5)

Fringe vertices:e(a,6), d(c,6+4)

Unseen vertices:

Nearest vertex:e(a,6)

a f d

b

e

c1

3 4 4 6

55

62

8


60/61

Example

Tree vertices:a(-,0), b(a,3), c(b,4), f(a,5), e(a,6)

Fringe vertices:

d(c,6+4)

Unseen vertices:

Nearest vertex:d(c,10)

a f d

b

e

c1

3 4 4 6

55

62

8


61/61

Example

Tree vertices:a(-,0), b(a,3), c(b,4),

f(a,5), e(a,6), d(c,10)

Fringe vertices:

Unseen vertices:

Nearest vertex:

a f d

b

e

c1

3 4 4 6

55

62

8

lecture8 - greedy algorithms

Documents