lecture9 - dynamic programming

8/14/2019 Lecture9 - Dynamic Programming

1/41

COMP2230

Introduction to Algorithmics

Lecture 9

A/Prof Ljiljana Brankovic


2/41

Lecture Overview

Dynamic Programming

Chapter 8, Text

Slides based on:Levitin. The Design and Analysis of AlgorithmsR. Johnsonbaugh & M. Schefer. Algorithms

G. Brassard & P. Bratley. Fundamentals of AlgorithmsSlides by Y. Lin and P. Moscato


3/41

Dynamic Programming

Use a bottom-up approach small answers build up to large ones

Create a table of (partial) answers The sub-answers are combined to form full answers Trade some space for either better time or reliability

Theres an implicit assumption

What are we assuming about the sub-answers, to ensure thefull answer is correct?


4/41

Principal of OptimalityIf we have a set of optimal solutions to a group of sub-

problems, and we have an optimal method of combiningsolutions, then we have an optimal solution to aproblem made up from the sub-problems.

or

If S is an optimal solution to a problem than componentsof S are optimal solutions to subproblems.

Note that this principle does not always apply forexample, it does not apply to the longest path problem.We need this principle to apply for DynamicProgramming to work.


5/41

A Risky bet...

Fred is a notoriously good cards player. He makes youan interesting (though possibly expensive) proposition:

You and Fred play 10 games per week.

First to win 100 games is the champion. If Fred wins, you pay Fred $100

If you win, Fred pays you $1000

You estimate Fred has a 60% chance of winning each game,

based on previous games.

Should you accept this bet?


6/41

Accept? If Fred and you are to play a single game then you could

easily calculate the expected gain:

E(gain) = 0.41000 + 0.6(-100) =340 - accept!

Let P(i,j) be the probability of you winning the bet, giventhat you need to win i more games, and Fred needs j more- we need to calculate P(100,100)

E(gain) = P(100,100)1000 + (1- P(100,100))(-100)

This is similar to the calculations performed by casinosaround the world, to work out how much to charge per bet.

(0.4 1000 + 0.6(-x) < 0 x > 667)


7/41

Setting up a recursion

Let P(i,j) be the probabilityof you winning the bet, giventhat you need to win imoregames, and Fred needsjmoregames.

We need to calculateP(100,100)

Probability that Fred wins agame isp = 0.6

Probability that Fred loses agame is q = 1-p = 0.4

Boundary conditions need tobe specified

P(0,j) = 1 for all 0 < j 100 Youve already won!

P(i,0) = 0 for all 0 < i 100 Freds already won!

P(0,0)cant exist both cant have already won


8/41

Probability formula How we relate what has happened with the future ?

We have the following formula:

P(i,j) = p P(i,j-1) + (1-p) P(i-1,j) i,j 1

functionP(i,j)

ifi=0thenreturn1ifj=0thenreturn0

returnpP(i,j-1) + (1-p)P(i-1,j)


9/41

How long will this take?

Let C(a,b) be the number of basis caseexecutions performed, given inputs a & b.

otherwise

0

0

11

1

1

),(

b

a

),b)+C(a,bC(a

baC


10/41

How long (cont)

We can show by induction:

For n = 100, this is approx. 91058.. If you execute1010instructions per second, thats about 2.81041years (which is a really long time...3.4 1010years is

the estimated lifetime of the universe)

Why is it so bad? What can be done?

n

nbaC

2),(


11/41

Solution: Save the work wevedone

Recall that D&C Gives a top-down approach

we should try to work frombottom up - and re-use thework weve already done. avoid calculating the same

thing twice (or many times)

programming refers to amatrix storage

Pi,j

execution tree

Pi-1,j Pi,j-1

Pi-2,j Pi-1,j-1 Pi-1,j-1 Pi,j-2


12/41


13/41

Calculating P(i,j) using array P

0 1 2 n

0

1

n

array P

ji

need a way to fill in array, from (0,0) to (n,n)


14/41

Then we fill the array0 1 2 n

0

1

n

we know all the boundaryvalues: P(i,0) and P(0,j)

So fill them in first.

Now,How do we get P(1,1)?

P(1,2)?

P(1,3)?

P(n,n)?

P(100,100)?


15/41

The tablej = 0 1 2 3 4 5 6 7 8 9 10

i = 0 1 1 1 1 1 1 1 1 1 11 0 0.4 0.64 0.784 0.87 0.922 0.953 0.972 0.983 0.99 0.9942 0 0.16 0.352 0.525 0.663 0.767 0.841 0.894 0.929 0.954 0.973 0 0.064 0.179 0.317 0.456 0.58 0.685 0.768 0.833 0.881 0.9174 0 0.026 0.087 0.179 0.29 0.406 0.517 0.618 0.704 0.775 0.8315 0 0.01 0.041 0.096 0.174 0.267 0.367 0.467 0.562 0.647 0.7216 0 0.004 0.019 0.05 0.099 0.166 0.247 0.335 0.426 0.514 0.5977 0 0.002 0.009 0.025 0.055 0.099 0.158 0.229 0.308 0.39 0.4738 0 7E-04 0.004 0.012 0.029 0.057 0.098 0.15 0.213 0.284 0.3599 0 3E-04 0.002 0.006 0.015 0.032 0.058 0.095 0.142 0.199 0.263

10 0 1E-04 7E-04 0.003 0.008 0.018 0.034 0.058 0.092 0.135 0.186


16/41

The dynamic programmingapproach

function(a,b,p)//{Calculate P[a,b] with prob. p}

n max(a,b); array P[0..n,0..n]fori=1ton

P[i,0]0

forj=1 tonP[0,j]1

fori=1tonforj=1ton

P[i,j] p P[i, j-1] + (1-p) P[i-1, j]

returnP[a,b]

Set up array with theboundary conditions

Work to complete the array


17/41

So, how long now?

Analysis is straight forward: fill the array takes Q(n2)

array takes space Q(n2)

assumes addition is Q(1)

could we make it take less space? Q(n)?


18/41

Benefits

Improved Efficiency as shown, we avoid calculating things more than once

Improved reliability some problems weve seen can be better solved with

Dynamic Programming consider some of the greedy problems

sometimes Dynamic Programming gives a solution whereGreedy wouldnt


19/41

Making change revisited

Instance 1:

denominations: 1, 4, 6 find change for 8

Greedy: 6, 1, 1

optimal: 4, 4

Instance 2:

denominations : 1, 1 3/4, 4 find change for 3

Greedy: (1 3/4) + 1.... fail

optimal: 1, 1, 1

Sometimes greedy gives non-optimal solution,

and sometimes it completely fails

Making change problem: For a given amount A and given

denominations, make amount A using smallest total numberof coins


20/41

Dynamic Programming for change...

What would a bottom-up approach to makingchange look like?

Some sort of table of sub-solutions

Problem reduced by considering reduced sets of

denominations; for example, for Instance 1: only coins of value 1

then coins of value 1 & 4

then coins of value 1 & 4 & 6....


21/41

Dynamic making change

Imagine we have found an optimal solution using onlythe first k-1 denominations (dont care how yet):

Coins: d1d2 d3.....dk-1 we know how many, and what types of coins are needed togive optimal results for all change amounts

Say we now consider the case d1d2... dk-1dk what choices do we have to update the old solution?

all old denominations

+ next one


22/41

Dynamic Programming for Change Thearray

Create a table of answersC[k,v] such that

k = number of denominationsconsidered so far.v = amount of change tomake.

C[k,v] =total numberofcoins required to make changevalue v, if only the first kdenominations are used.

v v+1

k

k+1

C[k,v]

optimal #

We will assume we have a listof all coin denominations,stored as a vector, d[k]or dkfor clarity


23/41


24/41


25/41

A back-of-envelope test

Try out the ideas before we write the pseudo-code.

Amount:

d1 = 1

0 1 2 3 4

d2 = 4

d3

= 6

0

0

0

0 1 2 3 4

compare C[k-1,v] and 1+C[k,v-d[k]]


26/41

Dynamic Change

functionCoins(A, d[1..n])

//Finds minimum # coins to make $A change

//Array d gives coin denominations

//Assume infinite amount of coins in each denomination

arrayC[1..n, 0..n]

fori 1 ton

C[i,0] = 0

fori 1 ton

forj 1 toA

C[i,j] ifi=1 &j < d[i] then+

elseifi=1then1 + C[1, j-d[1]]

elseifj < d[i] thenC[i-1, j]

elsemin{

C[i-1, j], 1 + C[i, j-d[i]]}


27/41

Which coins?

This gives (as greedy did) a solution whichonly tells us how many coins are needed, butnot which ones.

Questions for you to answer:

How do we interpret the table to find the coins to

use? What is O( ) of making change?

What is O( ) of finding coins?


28/41

Dynamic Progamming Summary

Build up solutions from little ones may have to fill up array and waste space/time

results may be faster than straight Divide & Conquer results may be more reliable than straight Greedy

Based on the Principle of Optimality: In an optimalsequence of decisions, each subsequence must also beoptimal.


29/41

WarshallsAlgorithm

Warshallsalgorithm is a dynamicprogramming algorithm for computing a thetransitive closure of a directed graph.

The transitive closure of a directed graph Gis an n x n matrix T of zeros and ones, wheretij = 1 if and only if there is a directednontrivial path from vertex i to vertex j.


30/41

Warshalls algorithm - example

Aadjacency matrix

T transitive closure

A=

T =

ba

dc

0101d

0000c

1000b

0010a

dcba

1111d

0000c

1111b

1111a

dcba


31/41

Warshalls Algorithm

One way to solve this problem would be to run a depth-first search (or breath-first search) from each vertexin digraph G.

Another way to solve this problem is Warshallsalgorithm which starts from adjacency matrix M0ofthe digraph G and then through a series of matricesM1, M2, . . . , Mn constructs the transitive closure of G.

In the matrix Mk, mij=1 if and only if there is a directedpath from i to j such that no vertex on the path haslabel greater than k.


32/41

Warshalls Algorithm

The matrix Mkcan be obtained directly fromMk-1as follows. The element mijin Mkis equalto one if and only if there is a path from i to jwhich does not contain any vertices with labelgreater than k. This will be the case if one ofthe following is true:

1. There is a path from i to j which does not contain anyvertex with label greater than k-1; in this case mij= 1 inM

k-12. There is a path from i to j that contains vertex k butdoes not contain any vertex with a label greater than k;then there is a path from i to k and from k to j and thusin Mk-1we have mik=1 and mkj=1.


33/41

Warshalls algorithm - example

A=M0= M1=

M2= M3= T=M4=

ba

dc0101d

0000c

1000b

0010a

dcba

0111d

0000c

1000b

0010a

dcba

1111d

0000c

1000b1010a

dcba

1111d

0000c

1000b1010a

dcba

1111d

0000c

1111b1111a

dcba


34/41

Algorithm 8.5.12 WarshallsAlgorithmThis algorithm computes the transitive closure of a relation Ron {1, ... , n}.The input is the matrix Aof R. The output is the matrix Aof the transitiveclosure of R.

Input Parameters: AOutput Parameters: A

transitive_closure(A) {n= A.lastfor k= 1 to n

for i= 1 to nfor j= 1 to n

A[i][j] = A[i][j] (A[i][k] A[k][j])

}


35/41

Floyds Algorithm

Given a graph G , either directed or undirected I want to know the shortest path from each node

to every other node. brute force: run Dijkstra from every node.

What is the complexity of this?

Dynamic programming may be helpful. Does optimality apply?

what do we need for optimality to apply?


36/41

Shortest paths (cont)

Consider a single shortestpath through three nodesi-k-j (plus otherintermediates)

Say k is on the shortestpath from i to j

What can we say about the

paths from i to k, and k to j?

i

k

j


37/41

i to j via k

We can ask: for a given pair of nodes, (i,j) is it shorter to go via the

current path, or is it better to go via some node k?

Assume we have a matrix of lengths Dk-1[1..n,1..n]

Dk-1holds the best path lengths for intermediate nodes{1, 2, .., k-1}

If we now consider node k, how do we update Dk-1?

We can do this using similar approach as Warshallsalgorithm


38/41

via k, or not via k

If we go via node k, the path will be from i k, and thenfrom k j, using the previous best paths.

Dk[i,j] = Dk-1[i,k] + Dk-1[k,j]

If we dont go via k, then the path length is unchanged

Dk [i,j] = Dk-1[i,j]

what would we do if we wanted to know the actual path from i to j? What is D0?


39/41

Working up to the pseudo-code

We need to check all pairs (i,j) and ask is thenode k (constant) on a shorter path.

So well have a nested for loop.

for i 1 to nfor j 1 to nD[i,j] min{D[i,j] , D[i,k] + D[k,j]}

Note: we can just overwrite D as we go, so wedo not need to store all the matrices.


40/41

Floyds algorithm - example

A=D0= D1=

D2= D3= D4=

ba

dc 06d

107c

02b

30a

dcba

096d

107c

502b

30a

dcba

096d

1079c

502b

3

0a

dcba

09166d

1079c

6502b43100a

dcba

09166d

1077c

6502b

43100a

dcba

2

3

1

76


41/41

Algorithm 8.5.3 Floyds Algorithm,Version 1This algorithm computes the length of a shortest path between each pairof vertices in a simple, directed or undirected, weighted graph G, whichdoes not contain a negative cycle. The input is the adjacency matrix AofG. The output is the matrix Awhose ijth entry is the length of a shortestpath from vertex ito vertex j.

Input Parameter: A

Output Parameter: Aall_paths(A) {

n= A.lastfor k= 1 to n// compute A(k)

for i= 1 to nfor j= 1 to n

if (A[i][k] + A[k][j] < A[i][j])A[i][j] = A[i][k] + A[k][j]

}

lecture9 - dynamic programming

Documents