Shortest path problems and algorithms1. Shortest path problems

Shortest path problem Single-source shortest path problem Single-destination shortest path problem All-pairs shortest path problem

2. Algorithms Dijkstra’s algorithm for single source shortest path problem

(positive weight) Bellman–Ford algorithm for single source shortest path problem

(allow negative weight) Floyd–Warshall algorithm for all pairs shortest paths Johnson's algorithm for all pairs shortest paths

CP412 ADAII

1. Shortest path problems

Shortest path problem:Input: weight graph G = (V, E, W),

source vertex s and destination vertex t. Output: a shortest path of G from s to t.

Single-source shortest path problem:Output: shortest paths from source vertex s to all other vertices of G.

Single-destination shortest path problem: Output: shortest paths from all vertices of G to a single destination t.

All-pairs shortest path problem:Output: shortest paths between every pair of vertices u, v in the graph.

CP412 ADAII

History of SPP and algorithms

Shimbel (1955). Information networks.

Ford (1956). RAND, economics of transportation.

Leyzorek, Gray, Johnson, Ladew, Meaker, Petry, Seitz (1957). Combat Development Dept. of the Army Electronic Proving Ground.

Dantzig (1958). Simplex method for linear programming.

Bellman (1958). Dynamic programming.

Moore (1959). Routing long-distance telephone calls for Bell Labs.

Dijkstra (1959). Simpler and faster version of Ford's algorithm.

CP412 ADAII

Dijkstra’s algorithms

Edger W. Dijkstra’s quotes The question of whether computers can think is

like the question of whether submarines can swim. In their capacity as a tool, computers will be but a ripple on

the surface of our culture. In their capacity as intellectual challenge, they are without precedent in the cultural history of mankind.

Do only what only you can do. (https://en.wikipedia.org/wiki/Turing_Award)

Dijkstra’s algorithm: solve the single-source shortest path problemGREEDY strategy Starting from source s, set visited set R = {s} Always pick the next closest vertex and add to R Use priority queue to store unvisited vertices by distance from s After deleteMin v, update distances of remaining vertices adjacent

to v using decreaseKey

CP412 ADAII

Example

CP412 ADAII4

5

0

1 2 3

48

7 6 5

4

8

8 7

117

1

2

6

4 14

9

10

2

0

1 2 3

48

7 6 5

4

8

8 7

117

1

2

6

4 14

9

10

4

8

12

2

0

1 2 3

48

7 6 5

4

8

8 7

117

1

2

6

4 14

9

10

4

8

12

9

15

2

0

1 2 3

48

7 6 5

4

8

8 7

117

1

2

6

4 14

9

10

4

8

12

9

15

2 11

0

0 0

Example

CP412 ADAII4

6

0

1 2 3

48

7 6 5

4

8

8 7

117

1

2

6

4 14

9

10

4

8

12

9

15

2 11

0

1 2 3

48

7 6 5

4

8

8 7

117

1

2

6

4 14

9

10

4

8

12

9

15

2 11

25

21

0

1 2 3

48

7 6 5

4

8

8 7

117

1

2

6

4 14

9

10

4

8

12

9

14

2 11

19

21

0

1 2 3

48

7 6 5

4

8

8 7

117

1

2

6

4 14

9

10

4

8

12

9

14

2 11

19

21

0 0

0 0

Example

Shortest Path Tree (SPT)

CP412 ADAII4

7

0

1 2 3

48

7 6 5

4

8

8 7

117

1

2

6

4 14

9

10

4

8

12

9

14

2 11

19

21

0

1 2 3

48

7 6 5

4

8

8 7

117

1

2

6

4 14

9

10

4

8

12

9

14

2 11

19

210 0

Correctness of Dijkstra’s algorithmLemma: (optimal substructure property) A least-cost path from X to Y contains least-cost paths from X to every vertex on the path to Y. (prove by contradiction)

Let d(v) be the label found by the algorithm and let δ(v) be the shortest path distance from s-to-v. We want to show that d(v) = δ(v) for every vertex v at the end of the algorithm, showing that the algorithm correctly computes the distances. We prove this by induction on |R| via the following lemma

Lemma: For each x ∈ R, d(x) = δ(x).

CP412 ADAII

Correctness of Dijkstra’s algorithmProof by Induction on |R|. Base case (|R| = 1): Since R only grows in size, the only time |R| = 1 is when R = {s} and d(s) = 0 = δ(s), which is correct.

Suppose it is true for |R| < k. Now assume |R| = k. Let u be the last vertex added to R. We need only show that d(u) = δ(u). Suppose for a contradiction that the shortest path from s-to-u is Q and has length w(Q) < d(u). Let R0 = R \ {u}. Q starts in R0 and at some leaves R0 (to get to u which is not in R0 ). Let xy be the first edge along Q that leaves R0 . Let Qx be the s-to-x subpath of Q. Clearly: w(Qx) + w(xy) ≤ w(Q). Since d(x) is the length of the shortest s-to-x path, d(x) ≤ w(Qx), d(x) + w(xy) ≤ w(Q). Since y is adjacent to x, d(y) must have been updated by the algorithm, so d(y) ≤ d(x) + w(xy), and d(y) < d(u). Finally, since u was picked by the algorithm, u must have the smallest distance label: d(u) ≤ d(y). Combining these inequalities gives us the contradiction that d(u) < d(u). Therefore, no such shorter path Q must exist and so d(u) = δ(u).

CP412 ADAII

Implementation and analysis

CP412 ADAII

CP412 ADAII

BuildHeap: O(|V|)

DeleteMin: O(|V| log |V|)

DecreaseKey: O(|E| log |V|)

Total running time: O(|E| log |V|)

What about graphs with negative edges?

CP412 ADAII

Do not mark any vertex as “known”.Instead allow multiple updates.

Bellman-Ford Algorithm

CP412 ADAII

The idea of Bellman-Ford algorithm is from dynamic programming.

Consider problem of shortest path using k number of edges for given source s and any destination v. Let dk(s, v) denote the weight of shortest path from s to v using at most k edges.

We can start by determining d0(s, v) for all v ϵ V, which is infinite for all vertices except s itself, i.e., d0(s, s) = 0, d0(s, v) = ∞, v≠s. Then we can use this to determine d1(s, v) for all v ϵ V. In general we want to determine dk+1(s, v) based on all dk(s, v).

Observation: if a dk+1(s, v) shortest path uses at most k edges, then dk+1(s, v) = dk(s, v), otherwise it uses k+1 edges, then it reaches of v’s neighbor x with k edges and plus edge (x, v). Therefore,

dk+1(s, v) = min{ dk(s, v), min{dk(s, x)+w(x, v) : x ϵ N-(v)} }

Bellman-Ford algorithm and analysis

CP412 ADAII

distance[|V|], predecessor[|V|], source vertex s // Step 1: initialize graph

for each vertex v in V: // O(|V|)distance[v] := inf // At the beginning , all vertices have a weight of infinitypredecessor[v] := null // And a null predecessor

distance[s] := 0

// Step 2: relax edges repeatedlyfor i from 1 to |V|-1:

for each edge (u, v) with weight w in edges: // O(|V||E|)if distance[u] + w < distance[v]:

distance[v] := distance[u] + wpredecessor[v] := u

// Step 3: check for negative-weight cyclesfor each edge (u, v) with weight w in edges:

if distance[u] + w < distance[v]:error "Graph contains a negative-weight cycle"

return distance[], predecessor[]

Time: O(|V||E|), Space : O(|V|)

Example

CP412 ADAII

Correctness of Bellman-Ford Algorithm

CP412 ADAII

Theorem. Given a directed weighted graph G = (V, E, w), w : E → R, and a source s, the BellmanFord algorithm returns the shortest path length from s to every vertex or indicates that there is a negative weight cycle in G reachable from s.

Proof. By induction on the number of edges k in a path. The base case is correct since d0(s,s) = 0. For all v in V \ s, on each step a shortest (s, v) path with up to k +1 edges must consist of a shortest (s, u) path of up to k edges followed by a single edge (u, v). Therefore if we take the minimum of these we get the overall shortest path with up to k+1 edges.

The algorithm only proceeds to |V| rounds (step 3). If there is a reachable negative-weight cycle, error returns; otherwise shortest path to every v is simple and consist of at most n vertices and hence |V|-1 edges.

CP412 ADAII

Also known as Floyd's algorithm, the Roy–Warshall algorithm, the Roy–Floyd algorithm, is an algorithm for finding all pair shortest paths in a weighted graph with positive or negative edge weights (but with no negative cycles). Based on the idea of dynamic programming.

Let G be graph with vertices numbered 1 through N. Consider a function sp(i,j,k) that returns the shortest possible path from i to j using vertices only from the set {1,2, …,k} as intermediate points along the way. • Now, given this function, find the shortest path from each i to each j

using only vertices in {1, …, k+1}. • For each of these pairs of vertices, the true shortest path could be

either (1) a path that only uses vertices in the set {1,… ,k} or (2) a path that goes from i to k+1 and then from k+1 to j.

Floyd–Warshall algorithm

CP412 ADAII 18

• The best path from i to j that only uses vertices 1 through k is defined by sp(i,j,k). If there were a better path from i to k+1 to j, then the length of this path is the concatenation of the shortest path from i to k+1 using vertices in {1, … ,k} and the shortest path from k+1 to j (also using vertices in {1, … ,k}.

Then sp(i, j, k+1) satisfies the following recursion formula:

sp(i,j,0) = w(i, j)

sp(i,j,k+1)=min{sp(i,j,k), sp(i,k+1,k)+sp(k+1,j,k) }.

Floyd–Warshall algorithm and analysis

CP412 ADAII 19

D[|V|] [|V|] = { ∞ }

for each vertex v in VD[v][v] ← 0for each edge (u,v)

D[u][v] ← w(u,v) // the weight of the edge (u,v) // O(|V|2)

for k from 1 to |V|for i from 1 to |V|

for j from 1 to |V|if D[i][j] > D[i][k] + D[k][j]

D[i][j] ←D[i][k] + D[k][j] // O(|V|3)end if

Time: O(|V|3), Space: O(|V|2)

Floyd–Warshall algorithm and recovery pathD[|V|] [|V|] = { ∞ }, n[|V|] [|V|] = { null }FloydWarshall()

for each edge (u,v)D[u][v] ← w(u,v) n[u][v] ← u

for k from 1 to |V| // standard Floyd-Warshall implementationfor i from 1 to |V|

for j from 1 to |V|if D[i][k] + D[k][j] < D[i][j] then

D[i][j] ← D[i][k] + D[k][j]n[i][j] ← n[i][k]

procedure Path(u, v)if n[u][v] = null then

return []path = [u]while u ≠ v

u ← n[u][v]path.append(u)

return path

20CP412 ADAII

Example

CP412 ADAII 21

Initialization: (k = 0)

CP412 ADAII 22

Iteration 1: (k = 1) Shorter paths from 2 ↝ 3 and 2 ↝ 4 are found through vertex 1

Iteration 2: (k = 2) Shorter paths from 4 ↝ 1, 5 ↝ 1, and 5 ↝ 3 are found through vertex 2

CP412 ADAII 23

Iteration 3: (k = 3) No shorter paths are found through vertex 3

CP412 ADAII 24

Iteration 4: (k = 4) Shorter paths from 1 ↝ 2, 1 ↝ 3, 2 ↝ 3, 3 ↝ 1, 3 ↝ 2, 5 ↝ 1, 5 ↝ 2, 5 ↝ 3, and 5 ↝ 4 are found through vertex 4

Iteration 5: (k = 5) No shorter paths are found through vertex 5