programming - ivor

8/3/2019 Programming - Ivor

1/5

Dynamic Programming 1

Dynamic Programming

Ivor Page1

This note explains the principles behind dynamic program through some simple examples.In some of the examples, a greedy algorithm exists that may run in less time than thedynamic programming solution given. The solutions are presented to help you understandthe principles of dynamic programming.

A problem can be solved by Dynamic Programming techniques if the solution to the problemcan be constructed from solutions to its sub-problems. Typically such problems suggest arecursive solution. For example, to find the nth Fibonacci number, F ibn we know that

F ibn = F ibn1 + F ibn2

The definition of Fibonacci numbers shows how to construct a solution from solutions tosub-problems. However, it is easy to show that a purely recursive solution runs in O(2n), orexponential time.

A dynamic programming solution collects solutions to all sub-problems in an array. For theFibonacci problem, we might start by computing F ib2 (after filling in the entries for F ib0 = 1and F ib1 = 1) and compute values for F ib3, F i b4, up to the required value, recording allcomputed values in an array. However, we observe that, to compute F ibn, we only need theprevious two Fibonacci numbers, F ibn1 and F ibn2. Our array reduces to a queue of sizetwo.

The Ways to Give Change Problem

This problem requires us to compute the number of ways that a certain dollar amount canbe given, using only the set of coin and note denominations stated in the problem. Forexample, we might restrict the denominations to the set {1, 5, 10, 25, 50} and ask how manyways can we use any number of coins of each type such that the sum equals $100. Onceagain, a recursive solution is suggested in which solutions to sub-problems can be combined toproduce the solution to the main problem. An array, d[] is initiallized with the denominationsin incresing order.

ways(amount, d[1 k]) = ways(amount, d[1 k 1])

+ways(amount d[k], d[1 k])

For example the number of ways to make up 10 cents using the set {1, 5} is given by:

ways(10, {1, 5}) = ways(10, {1}) + ways(5, {1, 5})

= 1 + 2

1University of Texas at Dallas


2/5


A recursive method can easily be developed from this recurrence, but the running time isexponential for large values. As is usually the case with such problems that yield to dynamic

programming techniques, a straight recursive approach solves each sub-problems many times.Dynamic programming replaces the recursion with iteration. We build an array, computingsolutions to ALL sub-problems as we go. The array will have columns equal to the num-ber of denominations and rows equal to the amount. For example, to solve the problemways(15, {1, 2, 4, 5}), we build an array of size 15 4 and begin by solving entries in the toprow, ways(1, {1}), ways(1, {1, 2}), ways(1, {1, 2, 4}) ways(1, {1, 2, 4, 5}) Using these re-sults, we compute entries in the second row, ways(2, {1}), ways(2, {1, 2}), ways(2, {1, 2, 5}),and so on.

In the program, an array w will be used to store the solutions to sub-problems.

Note that we can construct the array in this order since the update mechanism is as follows:

w[i][j] = w[i][j 1] + w[i denom[k]][j 1]

where denom[k] is denomination added in column j. A new entry is computed from theentry to its left and an entry in the same column above it denom[k] positions.

Here is the resulting array w:

Denoms

Amount 1 2 4 5

0 1 1 1 1

1 1 1 1 1

2 1 2 2 2

3 1 2 2 2

4 1 3 4 4

5 1 3 4 4

6 1 4 6 7

7 1 4 6 8

8 1 5 9 11

9 1 5 9 13

10 1 6 12 17

11 1 6 12 19

12 1 7 16 24

13 1 7 16 27

14 1 8 20 33

15 1 8 20 37

A row for amount = 0 has been added because, in the logic of the solution, ways(0, S) = 1,where S is any non-empty set of denominations. However, ways(n, S) = 0 if n < 0. Entriesin the first row and column are initially set equal to 1.

Look at the bottom right entry (the solution). ways[15][4] = ways[15][3]+ ways[155][4] =20 + 17 = 37. All entries are calculated in this way.


3/5


The Desert Traversal Problem

Here is another simple problem. You are given a n m grid of squares G with positiveintegers in the squares:

1 6 3 4 28 5 2 1 34 6 1 9 41 5 1 3 6

The object is to find the least cost path from any position above the top row to any positionbelow the bottom row. That is, we can begin our path in any of the squares in the top rowand end it in any of the squares in the bottom row. At each step, we can move verticallydown, or diagonally down to one square to the left or the right in the next row down. Thecost of a step is equal to the positive difference between the cell weights. So, if the chosenpath is vertically down the left edge, where the cell weights are 1, 8, 4, 1, the cost of thatpath is 1 + |1 8| + |8 4| + |4 1| = 1 + 7 + 4 + 3 = 15.

To solve this problem we construct an array C of the same size as the above grid, where ci,jis the minimum cost of getting from above the grid to cell (i, j) in the grid. We wont worryabout storing the path yet. Obviously, the first row of C is equal to the first row of G. Thesecond row of C is added as follows:

1 6 3 4 28 5 4 3 3

Consider the middle entry of the 2nd row. There are 3 ways to get to that cell from itsneighbors in the first row, from the 6, the 3, and the 4. The costs of those paths are 6+4,3+1, and 4+2. The middle path is the best, so the corresponding entry in C is 4. Note thatno entry of C can be less than the corresponding entry in G.

Each row of the array is developed in the same way. Its easy to complete on paper. Check

my 1 minute answer:

1 6 3 4 2

8 5 4 3 3

6 6 3 9 4

7 7 3 5 6

How do we use the table to construct the path? Clearly the minimum cost path ends inthe middle square of the bottom row. The previous square on the path must be one of itsthree neighbors in the row above, the middle one. Before that square we have the 3 square


4/5


above and to the right, and then the 2 square in the top right-hand corner. The best pathis through squares 2, 3, 3, 3, with total cost 3.

Floyds All pairs Shortest Path Algorithm

Finally, we have the famous all-pairs shortest path in a graph problem. We have an undi-rected graph G(V, E) where |V| = n, and every edge ei,j E has a positive weight associatedwith it. A shortest path between vertices vi and vj is a path with minimum sum of its weights.

The algorithm is frequently used to find a minimal cost path between two specified verticesbut, in the usual requirement for dynamic programming, it solves all sub-problems and, indoing so, finds minimal paths between all pairs of vertices. Dijkstras greedy algorithm isfaster for the minimal cost path between two specific vertices. A trivial version of Dijkstraruns in time O(n2), while a version employing a Fibonacci heap runs in time O(|E|+ nlog n).Floyds algorithm runs in time O(n3), but the computations for each iteration are very fast.A 2GHz PC can run Floyds algorithm with graphs of 200 nodes in a few seconds.

Floyds dynamic programming algorithm works as follows:

A is the adjacency matrix, ai,j = cost of edge ei,j. For all 0 i < n, ai,i = 0. The verticesare numbered starting at zero.

We maintain an array D of dimensions n n where di,j is the minimum cost of a path fromvi to vj using the set of vertices considered so far. Initially, for all i, j, di,j = ai,j. Initially,only those vertices connected by an edge will have a non-zero element of D.

Now add one vertex at a time and see if a better path can be found using the new vertex.Say we have already computed the minimum cost of getting from vi to vj using only verticesin the set {v0, v1, vk1}. Now we consider whether going via the new vertex vk is betterthan the path that we have already found. The decision is simply,

if(d[i][k] + d[k][j] < d[i][j]) {

d[i][j] = d[i][k] + d[k]j];

update the path information for i to j to

include k

}

Having considered whether going via vertex vk is a better route for all pairs vertices thanusing only vertices in the set {v0, v1, , vk1}, we repeat the process considering vertex vk+1and so on.


5/5


Here is the complete algorithm in the context of the problem Frogger:

// calculate all pairs direct distancesfor(i=0;i

programming - ivor

Documents