introduction to algorithms

Introduction to Algorithms

Jiafen Liu

Sept. 2013

Today’s Tasks

• Shortest Paths – Properties of shortest paths

– Dijkstra’s algorithm

– Correctness

– Analysis

– Breadth-first search

Paths in graphs

• Consider a digraph G= (V, E) with edge-weight function w: E→R. The weight of path p = v1→v2→…→vk is defined to be

• Example:

w(p)=?

Shortest paths

• A shortest path from u to v is a path of minimum weight from u to v.

• The shortest path weight from u to v is defined as

δ(u, v) = min{w(p): p is a path from u to v}.

• In what case there is no shortest path?– Note: δ(u, v) = ∞ if no path from u to v exists.– There is a negative cycle.

• In other cases, usually a finite set.

Well-definedness of shortest paths

• If a graph G contains a negative-weight cycle, then some shortest paths may not exist.

• Example:

Thought of solving

• Dynamic Programming

• Greedy Algorithms

• Characteristics：– Optimal Substructure– Overlapping Subproblems– Greedy choice property

Optimal substructure

• What is the optimal substructure for this problem?

• Theorem. A subpath of a shortest path is a shortest path.

• How to prove?– Cut and paste

What’s the subproblems？

• Shortest path going from u to v, maybe involves computing shortest path from u to some intermediate point x, and then from x to v.

• Different choosing of x1,x2…xn consists overlapping subproblems.

• How many subproblems are there?

• |v|2

Triangle inequality

• Important for us to make greedy choice

• Theorem:

For all u, v, x V, we have∈

• Figure Illustration:

Single-source shortest paths

• Problem. From a given source vertex s V∈ , find the shortest-path weights δ(s, v) for all v V∈ .

• If all edge weights w(u, v) are nonnegative, all shortest-path weights must exist.

• IDEA: Greedy.

1.Maintain a set S of vertices whose shortest-path distances from s are known. (s S∈ )

2.At each step add the vertex v V–S ∈ whose distance estimate from s is minimal to S .

3.Update the distance estimates of vertices adjacent to v.

Dijkstra’s algorithm

Example

Correctness —Part I

• Lemma. Initializing d[s]←0 and d[v]←∞ for all v V–{s} ∈ establishes d[v] ≥ δ(s,v) for all v V∈ , and this invariant is maintained over any sequence of relaxation steps.

• Any suggestions about how to prove?– Induction

• Proof.

• Base case: check the initializing term

• Induction: suppose that this fails to hold at some point.

Correctness —Part I

• Suppose not. Let v be the first vertex for which d[v] < δ(s, v), and let u be the vertex that caused d[v] to change: d[v] = d[u] + w(u, v). Then how to induct a contradition?

d[v]< δ(s, v)

≤ δ(s, u) + δ(u, v)

≤ δ(s,u) + w(u, v)

d[v] = d[u] + w(u, v)

• Contradiction.

supposition

triangle inequality

sh. path ≤ specific path

v is first violation

Correctness —Part II

• Lemma. Let u be v’s predecessor on a shortest path from s to v: s→…u→v.

if d[u]=δ(s, u) and if we relax edge(u, v),

Then, we have d[v] = δ(s, v) after the relaxation.

• Proof.

• s→…u→v is the shortest path from s to v, so δ(s, v) = w(s→… →u)+ w(u, v)

= δ(s, u)+ w(u, v).

Correctness —Part II

• Suppose that d[v] > δ(s, v) before the relaxation. – By Correctness I, d[v] ≥ δ(s,v) – Otherwise d[v] = δ(s,v) , we’re done.

• Then test d[v] > d[u] + w(u, v) succeeds – because d[v] > δ(s, v)

= δ(s, u)+ w(u, v)

= d[u] + w(u, v)

• Then the algorithm sets d[v] = d[u]+ w(u,v)

= δ(s, v).

Correctness —Part III

• Theorem. Dijkstra’s algorithm terminates with d[v] = δ(s, v) for all v V∈ .

• Something to be addressed:– Once we add a vertex to S, we never change

its weight

• Proof.

• It suffices to show that d[v] = δ(s, v) for every v V ∈ when v is added to S.


• Suppose for contradiction: Suppose u is the first vertex added to S for which d[u] ≠ δ(s, u). – d[u] > δ(s, u).

• Let p be a shortest path from s to u.

• Now look at the first place here where the path p exits S


• Let y be the first vertex in V–S along a shortest path from s to u, and let x be its predecessor.

• Since u is the first vertex violating the claimed invariant, we have d[x] = δ(s, x) .


• When x was added to S, the edge (x, y) was relaxed, which implies that

d[y] = δ(s, y) ≤ δ(s, u) < d[u]

• But, d[u] ≤ d[y] by our choice of u. Contradiction.

Analysis of Dijkstra

Analysis of Dijkstra

• Note: Same formula as in the analysis of Prim’s minimum spanning tree algorithm.

Unweighted graphs

• Suppose that w(u, v) =1 for all (u, v) E ∈ .

Can Dijkstra’s algorithm be improved?– Use a simple FIFO queue instead of a priority

queue.

– Analysis: Time = O(V+ E).

Example

Correctness of BFS

• IDEA: The FIFO Q in breadth-first search mimics the priority queue Q in Dijkstra.

• Invariant: v comes after u in Q implies that d[v] = d[u] or d[v] = d[u] + 1.

Step Further I

Bellman-Ford algorithm

Example of Bellman-Ford

Step Further II

• All-pairs shortest paths

• Input: Digraph G= (V, E), where V= {1, 2, …, n}, with edge-weight function w: E→R.

• Output: n ×nmatrix of shortest-path lengths δ(i, j)for all i, j∈V.

• Floyd alglrithm

Any idea of this problem?

• Dynamic programming again!

• Consider the n×n adjacency matrix A= (aij) of the digraph, and define dij

(m)=weight of a shortest path from i to j that uses at most m edges

• Claim: We have

and for m= 1, 2, …, n–1,

Proof of Claim

Matrix multiplication

• Compute C= A·B, where C, A, and B are n×n matrices

• Time = Θ(n3) using the standard algorithm.

• What if we map “+”→“min” and “·”→“+”?

introduction to algorithms

Documents