Shortest paths: models and algorithmsComplements of Operations Research

Giovanni RighiniUniversita degli Studi di Milano

Outline

• Part I. Basic results in LP duality.

• Part II. The shortest path problem: primal and dual formulations.

• Part III. The shortest path problem: algorithms.

• Conclusion. Lessons learned.

Linear Programming (LP) duality

Every linear program P has a dual linear program D.

P) minimize z =c′x ′ + c′′x ′′

s.t. a′x ′ + a′′x ′′ ≥ b′

d ′x ′ + d ′′x ′′ = b′′

x ′ ≥ 0

x ′′ free

D) maximize w =b′y ′ + b′′y ′′

s.t. a′y ′ + d ′y ′′ ≤ c′

a′′y ′ + d ′′y ′′ = c′′

y ′ ≥ 0

y ′′ free

The fundamental theorem of LP duality

One of these four cases can occur (and the simplex algorithm candetect it in a finite number of steps):

• both P and D have a finite optimal solution;

• P is unbounded and D is infeasible;

• D is unbounded and P is infeasible;

• both P and D are infeasible.

Three more theorems

Weak duality theorem.For each feasible solution x of P and for each feasible solution y of D,z(x) ≥ w(y).

Strong duality theorem.If there exist a feasible and optimal solution x∗ for P and a feasibleand optimal solution y∗ for D, then z(x∗) = w(y∗).

Complementary slackness theorem.Given a feasible solution x for P and a feasible solution y for D,necessary and sufficient condition for them to be optimal is:

x ′ (c′ − a′y ′) = 0

y ′ (a′x ′ − b′) = 0.

The Shortest Path Problem

Data:• a digraph D = (N ,A) with |N | = n nodes and |A| = m arcs;

• a source node s ∈ N ;

• a cost function c : A 7→ ℜ.

Shortest Path Problem (SPP).Find all minimum cost (i.e. shortest) paths from s to all nodes in N .

The problem is also called Shortest Path Tree/ArborescenceProblem, because of a property of its solution: the set of all shortestpaths forms a spanning arborescence rooted in s.

The shortest paths arborescence

1

2

3

4

5

6 7

1

1

2

2

1

2

2

4

1

3

1

1

0

1 2

2 4

5

6

Figura: A shortest paths arborescence (s = 1). Costs are black. Flows arered. Distances are blue.

Bellman’s optimality principle

Bellman’s optimality principle states that each optimal policy is madeby optimal sub-policies.

Translating this statement for the SPP:every shortest path from s to t ∈ N visiting i ∈ N is made by theshortest path from s to i and the shortest path from i to t .

As a consequence of this principle, the set of all the shortest pathsfrom s to N forms a spanning arborescence rooted in s.

But this “principle” is indeed a theorem: it can be proved, instead ofassumed.

We do not assume a priori that we are looking for a spanningarborescence rooted in s.

The mathematical model: variables

Variables. We associate an integer variable xij ∈ Z+ with each arc ofthe digraph. It indicates the number of shortest paths that use arc(i, j) ∈ A.

1

2

3

4

5

6 7

1

1

2

2

1

2

2

4

1

3

1

1

0

1 2

2 4

5

6

Figura: One unit of flow goes from s to each other node. The flow on eacharc equals the number of nodes that are reached through it.

The mathematical model: obj. function

Objective function. We want to minimize each path from s to t ∈ N :

minimize∑

(i,j)∈A

cijxij .

1

2

3

4

5

6 7

1

1

2

2

1

2

2

4

1

3

1

1

0

1 2

2 4

5

6

Figura: The sum of the costs times the flow equals the sum of the distances:1×2+1×1+2×4+2×3+1×1+2×1 = 0+ 1+ 2+ 2+ 4+ 5+ 6 = 20.

The mathematical model: constraints

Constraints. For each shortest path from s to t ∈ N we impose theflow conservation constraints:

∑

(j,i)∈δ−

i

xji −∑

(i,j)∈δ+i

xij = 0 ∀i ∈ N\{s, t}

∑

(j,s)∈δ−

s

xjs −∑

(s,j)∈δ+s

xsj = −1

∑

(j,t)∈δ−

t

xjt −∑

(t,j)∈δ+t

xtj = +1

Summing up for all t ∈ N we obtain:

∑

(j,i)∈δ−

i

xji −∑

(i,j)∈δ+i

xij = 1 ∀i ∈ N\{s}

∑

(j,s)∈δ−

s

xjs −∑

(s,j)∈δ+s

xsj = 1 − n

SPP: primal formulation (ILP)

P) minimize∑

(i,j)∈A

cijxij

s.t.∑

(j,i)∈δ−

i

xji −∑

(i,j)∈δ+i

xij = 1 ∀i ∈ N\{s}

∑

(j,s)∈δ−

s

xjs −∑

(s,j)∈δ+s

xsj = 1 − n

xij ∈ Z+ ∀(i, j) ∈ A.

Observation 1. The constraint matrix is totally unimodular, i.e. everysub-matrix has a determinant equal to -1, 0 or 1.

Observation 2. The right-hand-sides of the constraints are all integernumbers.

In this case we have the guarantee that every base solution of thecontinuous relaxation of P has integer coordinates.

Reformulation (relaxation) of the primal problem (LP)

Hence we can relax the integrality restrictions:

P) minimize∑

(i,j)∈A

cijxij

s.t.∑

(j,i)∈δ−

i

xji −∑

(i,j)∈δ+i

xij = 1 ∀i ∈ N\{s}

∑

(j,s)∈δ−

s

xjs −∑

(s,j)∈δ+s

xsj = 1 − n

xij ≥ 0 ∀(i, j) ∈ A.

This linear problem P has a dual problem, D, and for the primal-dualpair (P,D) the duality theorems hold.

SPP: Dual formulation (LP)

P) minimize∑

(i,j)∈A

cijxij

s.t.∑

(j,i)∈δ−

i

xji −∑

(i,j)∈δ+i

xij = 1 ∀i ∈ N\{s}

∑

(j,s)∈δ−

s

xjs −∑

(s,j)∈δ+s

xsj = 1 − n

xij ≥ 0 ∀(i, j) ∈ A.

D) maximize∑

i∈N\{s}

yi + (1 − n)ys

s.t. yj − yi ≤ cij ∀(i, j) ∈ A

yi free ∀i ∈ N .

An equivalent dual formulation (LP)

D) maximize∑

i∈N\{s}

yi+(1 − n)ys

s.t. yj − yi ≤ cij ∀(i, j) ∈ A

yi free ∀i ∈ N .

Observation 1. If we add a constant α to each y variable, nothingchanges. Hence we are allowed to fix one variable:

ys = 0

Observation 2. We have m inequality constraints, n − 1 original yvariables and m slack variables. The LP tableau of the dual problemhas m rows and n − 1 + m columns. Hence in each base solution ofD there should be m basic variables and n − 1 non-basic (null)variables. For the complementary slackness theorem, there shouldbe red n − 1 basic (positive) variables in the primal problem.

An equivalent primal formulation (LP)

P) minimize∑

(i,j)∈A

cijxij

s.t.∑

(j,i)∈δ−

i

xji −∑

(i,j)∈δ+i

xij = 1 ∀i ∈ N\{s}

∑

(j,s)∈δ−

s

xjs −∑

(s,j)∈δ+s

xsj = 1 − n

xij ≥ 0 ∀(i, j) ∈ A.

Observation 3. We have n equality constraints that are not linearlyindependent: summing up all the rows we obtain 0 = 0. Hence weare allowed to delete one constraint: we delete the flow conservationconstraint for s.Observation 4. We have now n − 1 equality constraints and mvariables. The LP tableau of P has n − 1 rows and m columns.Hence in each base solution of P there are n − 1 basic variables andm − (n − 1) non-basic variables.

Complementary slackness conditions (CSC)

P′) minimize z =∑

(i,j)∈A

cijxij

s.t.∑

(j,i)∈δ−

i

xji −∑

(i,j)∈δ+i

xij = 1 ∀i ∈ N\{s}

xij ≥ 0 ∀(i, j) ∈ A.

D′) maximize w =∑

i∈N\{s}

yi

s.t. yj − yi ≤ cij ∀(i, j) ∈ A

yi free ∀i ∈ N\{s}.

Primal CSCs: xij(yi + cij − yj)= 0Basic variables in P′ correspond to active constraints in D′.Only arcs (i, j) for which yi + cij = yj can carry flow xij .

The Ford-Fulkerson algorithm (1962)

Data structures:• a predecessor label, πi for each node i ∈ N ;

• a cost label, yi for each node i ∈ N .

Algorithm:

Step FF1 (Initialization):Set ys:=0 and yi :=∞ ∀i ∈ N\{s}.Set πi :=nil ∀i ∈ N .

Step FF2 (Iteration):Select an arc (i, j) ∈ A such that yj − yi > cij .If such an arc exists

then Set yj := yi + cij , πj := i and repeat;else Terminate;

Different algorithms with different worst-case time complexity areobtained from different implementations of the Select function.

Feasibility

After initialization (Step FF1) we have neither primal feasibility nordual feasibility.

Primal viewpoint:We have πi = nil for all i ∈ N ; hence no flow enters any node.

Dual viewpoint:We have yi = ∞ for all i ∈ N\{s}; hence all constraints yi − ys ≤ csi

are violated.

The FF algorithm maintains the CSCs and iteratively enforces primaland dual feasibility.

Dual descent

After each iteration (Step FF2) one of the dual values yj is decreased• from a value such that yj − yi > cij

• to a value such that yj − yi = cij

so that arc (i, j) becomes tight and xij enters the primal basis.

What happens to the other constraints (dual viewpoint) and arcs(primal viewpoint)?

There are two cases:• Case I: before the iteration, yj = ∞ and πj = nil. In this case arc(i, j) becomes tight and nothing else changes. Flow can nowreach j from i (node j has been appended to the arborescence).

• Case II: before the iteration, yj 6= ∞ and πj = k . In this casenode j was already in the arborescence and was receiving flowfrom node k along a tight arc (k , j), i.e. yj − yk = wkj . After theiteration, arc (k , j) is no longer tight, i.e. yj − yk < wkj and cannotcarry flow any more. Node j now receives flow from i and notfrom k .

Case I

1

2

3

6

5

4

7

1

1

2

5

2 1

1

0

1

2

2

�

1

3

5

6

7

1

1

2

3 1

1

0

1

2

2

5 �

6

1 � 2 4

� �

Figura: Arc (5,6) becomes tight and y6 takes a finite value.

Case II

1

2

3

4

5

6

7

1

1

2

5

1 2

5 1

1 3

1 1

0

1

2

2

6

7

8

1

2

3

4

5

6

7

1

1

2

2

1 2

2 4

1 3

1 1

0

1

2

2

4

7

8

5

2

Figura: Arc (7,6) replaces arc (5, 6). Arcs (6,2) and (6,4) become infeasibleagain.

A mechanical analogy

Assume we have n balls to be linked together by a set of m strings ofgiven lengths. Let ball s to be fixed at the ceiling and let yi be thedistance of ball i from the ceiling (ys = 0).

Initially all balls but s are put on the floor, very far from the ceiling(yi = ∞), and they are not connected to the ceiling (πi = nil).

Iteratively take a string (i, j) and append ball j to ball i. Select one forwhich the distance between the balls exceeds the length of the string(so you will never select i on the floor).In doing this you can either link a ball j that was on the floor to a ball ihanging from above (Case I) or pull up a ball j already hanging fromthe ceiling by connecting it to a ball i over it (Case II).

When all strings have been used (all dual constraints have beenenforced), there are n − 1 tight strings (the spanning arborescence).

This analogy holds for the case in which cij ≥ 0 ∀(i, j) ∈ A.

Termination

Does the algorithm always achieve primal and dual feasibility?

If there is a node t ∈ N not reachable from s, the algorithm does notfind any arc (i, t) ∈ A corresponding to a violated dual constraint.Hence yt remains equal to ∞; no arc entering t becomes tight; noflow can reach t : the primal problem is infeasible and the dualproblem is unbounded.

If there is a negative-cost cycle reachable from s, the algorithm keepsfinding a violated dual constraint corresponding to one of the arcs inthe cycle. Hence the algorithm enters a never-ending loop in whichthe y values of the nodes in the cycle are decreased to −∞ and itnever finds a feasible dual solution: the dual problem is infeasible andthe primal problem is unbounded.

The two things can also happen independently: both problems areinfeasible.

Dijkstra’s algorithm (1959)

T :=∅;for i:=1 to n do flag[i]:=0;flag[s]:=1; for i:=1 to n do y[i]:=c(s, i); pred[i]:=s;for iteration:=1 to n − 1 do

ymin:=∞;for i:=1 to n do

if (flag[i]=0) and (y[i] < ymin) theni∗:=i; ymin := y[i];

T := T ∪ {[pred[i∗],i∗]};flag[i∗] := 1;for i:=1 to n do

if (flag[i]=0) and (y[i∗] + c(i∗, i) < y[i]) thenpred[i] := i∗; y[i] := y[i∗] + c(i∗, i);

The time complexity is O(n2). It requires c ≥ 0.

Bellman-Ford’s algorithm (1956,1958)

for i:=1 to n do y[i]:=c(s, i); pred[i]:=s;for iteration:=1 to n − 1 do

for (i, j) ∈ A doif (y[i] + c(i, j) < y[j]) then

pred[j] := i;y[j] := y[i] + c(i, j);

The time complexity is O(nm) because it requires O(n) iterations,each one with complexity O(m).

Lessons learned

Theory is when one knows everything but nothing works. Practice iswhen everything works but one doesn’t know why. In any case onealways ends up with joining theory with practice: nothing works andone doesn’t know why. [A. Einstein]

What have we (hopefully) learned from this lecture?

Mainly two things:

• Lesson 1. Duality theory works: it provides us not only beautifultheorems but also efficient algorithms.

• Lesson 2. We know why algorithms work: they are based on asolid theoretical background.

Lessons learned

Theory is when one knows everything but nothing works. Practice iswhen everything works but one doesn’t know why. In any case onealways ends up with joining theory with practice: nothing works andone doesn’t know why. [A. Einstein]

What have we (hopefully) learned from this lecture?

Mainly two things:

• Lesson 1. Duality theory works: it provides us not only beautifultheorems but also efficient algorithms.

• Lesson 2. We know why algorithms work: they are based on asolid theoretical background.

Nothing is more practical than a good theory. [A. Einstein]