graph search algorithms - york university

CSE 31011

Graph Search Algorithms

CSE 31012

Graph

a

c

b Node ~ city or computer

Edge ~ road or data cable

Undirected or Directed

A surprisingly large number of computational problems can be expressed as graph problems.

CSE 31013

Directed and Undirected Graphs

(c) The subgraph of the graph in part (a) induced by the vertex set {1,2,3,6}.

(a) A directed graph G = (V, E), where V = {1,2,3,4,5,6} and E = {(1,2), (2,2), (2,4), (2,5), (4,1), (4,5), (5,4), (6,3)}. The edge (2,2) is a self-loop.

(b) An undirected graph G = (V,E), where V = {1,2,3,4,5,6} and E = {(1,2), (1,5), (2,5), (3,6)}. The vertex 4 is isolated.

CSE 31014

Trees

Tree Forest Graph with Cycle

A tree is a connected, acyclic, undirected graph.

A forest is a set of trees (not necessarily connected)

CSE 31015

Running Time of Graph Algorithms

• Running time often a function of both |V| and |E|.

• For convenience, drop the | . | in asymptotic notation, e.g. O(V+E).

End of Lecture 10

April 6, 2009

CSE 31017

Representations: Undirected Graphs

Adjacency List Adjacency Matrix

Space complexity:

Time to find all neighbours of vertex :u

Time to determine if ( , ) : ∈u v E

( )θ +V E

(degree( ))θ u

(degree( ))θ u

2( )θ V

( )θ V

(1)θ

CSE 31018

Representations: Directed Graphs

Adjacency List Adjacency Matrix

Space complexity:

Time to find all neighbours of vertex :u

Time to determine if ( , ) : ∈u v E

( )θ +V E

(degree( ))θ u

(degree( ))θ u

2( )θ V

( )θ V

(1)θ

CSE 31019

Breadth-First Search

• Goal: To recover the shortest paths from a source node s to all other reachable nodes v in a graph.– The length of each path and the paths themselves are returned.

• Notes:– There are an exponential number of possible paths

– This problem is harder for general graphs than trees because of cycles!

s

?

CSE 310110

Breadth-First Search

• Idea: send out search ‘wave’ from s.

• Keep track of progress by colouring vertices:– Undiscovered vertices are coloured black

– Just discovered vertices (on the wavefront) are coloured red.

– Previously discovered vertices (behind wavefront) are coloured grey.

Graph ( , ) (directed or undirected) and sourceInput: vertex .G V E s V= ∈

[ ] shortest path distance ( , ) from to , . [ ] such that ( , ) is las

Outpu

t edg

t:

e on shortest path from a to

.d v s v s v v V

v u u v s vδ

π= ∀ ∈

=

CSE 310111

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

First-In First-Out (FIFO) queuestores ‘just discovered’ vertices

CSE 310112

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

s

d=0

d=0

CSE 310113

BFS FoundNot Handled

Queue

d=0a

bgd

d=1

s

a

c

h

k

f

i

l

m

j

e

b

gd

d=0d=1

CSE 310114

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

a

bgd

d=0d=1

d=1

CSE 310115

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

bgd

cf

d=0d=1

d=2

d=1

d=2

CSE 310116

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

bg

cfme

d=0d=1

d=2

d=1

d=2

CSE 310117

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queued=0

d=1

d=2

b

j

cfme

d=1

d=2

CSE 310118

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queued=0

d=1

d=2

j

cfme

d=1

d=2

CSE 310119

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

cfmej

d=0d=1

d=2

d=2

CSE 310120

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

fmejhi

d=0d=1

d=2

d=3

d=2

d=3

CSE 310121

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

mejhi

d=0d=1

d=2

d=3

d=2

d=3

CSE 310122

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

ejhil

d=0d=1

d=2

d=3

d=2

d=3

CSE 310123

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

jhil

d=0d=1

d=2

d=3

d=2

d=3

CSE 310124

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

hil

d=0d=1

d=2

d=3

d=2

d=3

CSE 310125

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

h

d=0d=1

d=3

d=2

il

d=3

CSE 310126

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

ilk

d=0d=1

d=3d=4

d=2

d=3

d=4

CSE 310127

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

lk

d=0d=1

d=3d=4

d=2

d=3

d=4

CSE 310128

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

k

d=0d=1

d=3d=4

d=2

d=3

d=4

CSE 310129

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queue

k

d=0d=1

d=3d=4

d=2

d=4

CSE 310130

BFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Queued=0

d=1

d=3d=4

d=2

d=4d=5

CSE 310131

Breadth-First Search Algorithm

• Q is a FIFO queue.

• Each vertex assigned finite dvalue at most once.

• Q contains vertices with d values {i, …, i, i+1, …, i+1}

• d values assigned are monotonically increasing over time.

BLACK

RED

BLACKRED

GRAY

CSE 310132

Breadth-First-Search is Greedy

• Vertices are handled:– in order of their discovery (FIFO queue)

– Smallest d values first

CSE 310133

Basic Steps:

su

The shortest path to uhas length d

v

& there is an edge from u to v

d

There is a path to v with length d+1.

Correctness

CSE 310134

Correctness

• Vertices are discovered in order of their distance from the source vertex s.

• When we discover v, how do we know there is not a shorter path to v?– Because if there was, we would already have discovered it!

su vd

CSE 310135

Correctness

Graph ( , ) (directed or undirected) and sourceInput: vertex .G V E s V= ∈

[ ] distance from to , . [ ] such that ( , ) is last edge on s

Output:

hortest path from to . d v s v v V

v u u v s vπ= ∀ ∈

=

1. [ ] ( , )d v s v v Vδ≥ ∀ ∈

2. [ ] ( , ) d v s v v Vδ> ∀ ∈/

Two-step proof:

On exit:

CSE 310136

δ≥ ∀ ∈Claim 1. is never too small: [ ] ( , )d d v s v v V There exists a path from to of length [ ].Proof: v vs d

By Induction:

Suppose it is true for all vertices thus far discovered ( an grre d d ey).

is discovered from some adjacent vertex being handled.uv

→ = +[ ] [ ] 1d v d uδ≥ +( , ) 1usδ≥ ( , )s v u v

s

since each vertex is assigned a value exactly once, it follows that o [ ]n exit, ( ., )d v s v

vv V

dδ≥ ∀ ∈

CSE 310137

<LI>: [ ] ( , ) 'disco rvered' ( o gr )eyred d v s v v Vδ← ≥ ∀ ∈

( , ) 1s uδ≥ + ( , )s vδ≥

δ≥ ∀ ∈Claim 1. is never too small: [ ] ( , )d d v s v v V

BLACK

RED

BLACKRED

GRAY

There exists a path from to of length [ ].Proof: v vs d

su v

CSE 310138

δ≤ ∀ ∈Claim 2. is never too big: [ ] ( , ) d d v s v v VProof by contradiction:

δSuppose one or more vertices receive a value greater than .d

δLet be the vertex with minimum ( , ) that receives such a value.s dv v

Let be 's predecessor on a shortest path from to .u sv v

su v

Suppose that is discovered and assigned this d value when vertex is dequeued.v x

= −[ ] [ ] 1d x d v

δ= −[ ] ( , ) 1d s vu

δ <( , ) [ ]vs d v

vertices are dequeued in increasing order of Reca v .ll: alued

→ u was dequeued before x.

δ→ = + =[ ] [ ] 1 ( , )dvd u s v

xδ→ − < −( , ) 1 [ ] 1v d vs

Then

→ <[ ] [ ]d u d x

Contradiction!

CSE 310139

Correctness

δ≥ ∀ ∈Claim 1. is never too small: [ ] ( , )d d v s v v V

δ≤ ∀ ∈Claim 2. is never too big: [ ] ( , ) d d v s v v V

δ⇒ = ∀ ∈ is just right: [ ] ( , ) d d v s v v V

CSE 310140

Progress?

• On every iteration one vertex is processed (turns gray).

BLACK

RED

BLACKRED

GRAY

CSE 310141

Running Time

Each vertex is enqueued at most once ( )O V→

Each entry in the adjacency lists is scanned at most once O(E)→

Thus run time is ( ).O V E+BLACK

RED

RED

GRAY

BLACK

CSE 310142

• The shortest path problem has the optimal substructure property:– Every subpath of a shortest path is a shortest path.

• The optimal substructure property– is a hallmark of both greedy and dynamic programming algorithms.

– allows us to compute both shortest path distance and the shortest paths themselves by storing only one d value and one predecessor value per vertex.

Optimal Substructure Property

u vs

shortest path

shortest path shortest path

CSE 310143

Recovering the Shortest PathFor each node v, store predecessor of v in π(v).

su v

π(v)

π(v) = u.Predecessor of v is

CSE 310144

Recovering the Shortest Path

Precondition: and are vertices of graph Postcondition: the vertices on the shortest path from to have been prin

P

if then

RINT-PATH( , , )

pr

print

ted in o

else

intif

rd

then [ ] I "

e

L

r

N

s v Gs v

s

v

v

ss

G

v

π

=

=

elseno path from" "to" "exists"

PRINT-PATH( , , [ ])print

s v

G s vv

π

CSE 310145

Colours are actually not required

CSE 310146

Depth First Search (DFS)

• Idea:– Continue searching “deeper” into the graph, until we get

stuck.

– If all the edges leaving v have been explored we “backtrack”to the vertex from which v was discovered.

• Does not recover shortest paths, but can be useful for extracting other properties of graph, e.g.,– Topological sorts

– Detection of cycles

– Extraction of strongly connected components

CSE 310147

Depth-First Search

• Explore every edge, starting from different vertices if necessary.

• As soon as vertex discovered, explore from it.

• Keep track of progress by colouring vertices:– Black: undiscovered vertices

– Red: discovered, but not finished (still exploring from it)

– Gray: finished (found everything reachable from it).

Graph ( , ) (directed or In undirectep : )t du G V E=

2 timestamps on each vertex: [ ] discovery time. [ ] finishing tim

Output

.

:

ed vf v

=

=1 [ ] [ ] 2| |d v f v V≤ < ≤

CSE 310148

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled

Stack <node,# edges>

/

/

//

/

/

/

/

/

/

/

/

/

/

d fNote: Stack is Last-In First-Out (LIFO)

CSE 310149

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

FoundNot Handled


s,0

/

1/

//

/

/

/

/

/

/

/

/

/

/

CSE 310150

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,0/

1/

/2/

/

/

/

/

/

/

/

/

/

/

CSE 310151

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,0/

1/

/2/

3/

/

/

/

/

/

/

/

/

/

CSE 310152

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,1h,0

/

1/

/2/

3/

/

/

/

/

/

/

/

4/

/

CSE 310153

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,1h,1k,0

/

1/

/2/

3/

/

/

/

/

/

/

5/

4/

/

CSE 310154

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,1h,1

Tree Edge

Path on Stack /

1/

/2/

3/

/

/

/

/

/

/

5/6

4/

/

CSE 310155

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,1/

1/

/2/

3/

/

/

/

/

/

/

5/6

4/7

/

CSE 310156

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,2i,0

8/

1/

/2/

3/

/

/

/

/

/

/

5/6

4/7

/

CSE 310157

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,2i,1

Cross Edge to handled node: d[h]<d[i]

8/

1/

/2/

3/

/

/

/

/

/

/

5/6

4/7

/

CSE 310158

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,2i,2

8/

1/

/2/

3/

/

/

/

/

/

/

5/6

4/7

/

CSE 310159

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,2i,3l,0

8/

1/

/2/

3/

/

/

/

/

/

9/

5/6

4/7

/

CSE 310160

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,2i,3l,1

8/

1/

/2/

3/

/

/

/

/

/

9/

5/6

4/7

/

CSE 310161

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,2i,3

8/

1/

/2/

3/

/

/

/

/

/

9/10

5/6

4/7

/

CSE 310162

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,2i,4g,0

8/

1/

/2/

3/

/

/

11/

/

/

9/10

5/6

4/7

/

CSE 310163

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,2i,4g,1j,0

8/

1/

/2/

3/

/

/

11/

12/

/

9/10

5/6

4/7

/

CSE 310164

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,2i,4g,1j,1

Back Edge to node on Stack:

8/

1/

/2/

3/

/

/

11/

12/

/

9/10

5/6

4/7

/

CSE 310165

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,2i,4g,1j,2m,0

8/

1/

/2/

3/

/

/

11/

12/

13/

9/10

5/6

4/7

/

CSE 310166

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,2i,4g,1j,2m,1

8/

1/

/2/

3/

/

/

11/

12/

13/

9/10

5/6

4/7

/

CSE 310167

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,2i,4g,1j,2

8/

1/

/2/

3/

/

/

11/

12/

13/14

9/10

5/6

4/7

/

CSE 310168

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,2i,4g,1

8/

1/

/2/

3/

/

/

11/

12/15

13/14

9/10

5/6

4/7

/

CSE 310169

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,2i,4

8/

1/

/2/

3/

/

/

11/16

12/15

13/14

9/10

5/6

4/7

/

CSE 310170

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,2i,5f,0

8/

1/

/2/

3/

17/

/

11/16

12/15

13/14

9/10

5/6

4/7

/

CSE 310171

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,2i,5f,1

8/

1/

/2/

3/

17/

/

11/16

12/15

13/14

9/10

5/6

4/7

/

CSE 310172

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,2i,5

8/

1/

/2/

3/

17/18

/

11/16

12/15

13/14

9/10

5/6

4/7

/

CSE 310173

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,28/19

1/

/2/

3/

17/18

/

11/16

12/15

13/14

9/10

5/6

4/7

/

CSE 310174

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,1c,38/19

1/

/2/

3/

17/18

/

11/16

12/15

13/14

9/10

5/6

4/7

/

Forward Edge

CSE 310175

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,18/19

1/

/2/

3/19

17/18

/

11/16

12/15

13/14

9/10

5/6

4/7

/

CSE 310176

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


a,28/19

1/

/2/

3/19

17/18

/

11/16

12/15

13/14

9/10

5/6

4/7

/

CSE 310177

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,1

FoundNot Handled


8/19

1/

/2/20

3/19

17/18

/

11/16

12/15

13/14

9/10

5/6

4/7

/

CSE 310178

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,2

FoundNot Handled


d,08/19

1/

/2/20

3/19

17/18

21/

11/16

12/15

13/14

9/10

5/6

4/7

/

CSE 310179

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,2

FoundNot Handled


d,18/19

1/

/2/20

3/19

17/18

21/

11/16

12/15

13/14

9/10

5/6

4/7

/

CSE 310180

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,2

FoundNot Handled


d,28/19

1/

/2/20

3/19

17/18

21/

11/16

12/15

13/14

9/10

5/6

4/7

/

CSE 310181

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,2

FoundNot Handled


d,3e,08/19

1/

/2/20

3/19

17/18

21/

11/16

12/15

13/14

9/10

5/6

4/7

22/

CSE 310182

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,2

FoundNot Handled


d,3e,18/19

1/

/2/20

3/19

17/18

21/

11/16

12/15

13/14

9/10

5/6

4/7

22/

CSE 310183

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,2

FoundNot Handled


d,38/19

1/

/2/20

3/19

17/18

21/

11/16

12/15

13/14

9/10

5/6

4/7

22/23

CSE 310184

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,2

FoundNot Handled


8/19

1/

/2/20

3/19

17/18

21/24

11/16

12/15

13/14

9/10

5/6

4/7

22/23

CSE 310185

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,3

FoundNot Handled


8/19

1/

/2/20

3/19

17/18

21/24

11/16

12/15

13/14

9/10

5/6

4/7

22/23

CSE 310186

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,4

FoundNot Handled


b,08/19

1/

25/2/20

3/19

17/18

21/24

11/16

12/15

13/14

9/10

5/6

4/7

22/23

CSE 310187

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,4

FoundNot Handled


b,18/19

1/

25/2/20

3/19

17/18

21/24

11/16

12/15

13/14

9/10

5/6

4/7

22/23

CSE 310188

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,4

FoundNot Handled


b,28/19

1/

25/2/20

3/19

17/18

21/24

11/16

12/15

13/14

9/10

5/6

4/7

22/23

CSE 310189

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,4

FoundNot Handled


b,38/19

1/

25/2/20

3/19

17/18

21/24

11/16

12/15

13/14

9/10

5/6

4/7

22/23

CSE 310190

DFS

s

a

c

h

k

f

i

l

m

j

e

b

gd

s,4

FoundNot Handled


8/19

1/

25/262/20

3/19

17/18

21/24

11/16

12/15

13/14

9/10

5/6

4/7

22/23

CSE 310191

DFS

s

FoundNot Handled


Finished!

Tree EdgesBack Edges

a

c

h

k

f

i

l

m

j

e

b

gd

Cross Edges

8/19

1/27

25/262/20

3/19

17/18

21/24

11/16

12/15

13/14

9/10

5/6

4/7

22/23

Forward Edges

CSE 310192

Classification of Edges in DFS1. Tree edges are edges in the depth-first forest Gπ. Edge (u, v) is a tree edge if

v was first discovered by exploring edge (u, v).

2. Back edges are those edges (u, v) connecting a vertex u to an ancestor v in a depth-first tree.

3. Forward edges are non-tree edges (u, v) connecting a vertex u to a descendant v in a depth-first tree.

4. Cross edges are all other edges. They can go between vertices in the same depth-first tree, as long as one vertex is not an ancestor of the other.

s

a

c

h

k

f

i

l

m

j

e

b

gd

8/19

1/27

25/262/20

3/19

17/18

21/24

11/16

12/15

13/14

9/10

5/6

4/7

22/23

CSE 310193

Classification of Edges in DFS1. Tree edges: Edge (u, v) is a tree edge if v was black when (u, v) traversed.

2. Back edges: (u, v) is a back edge if v was red when (u, v) traversed.

3. Forward edges: (u, v) is a forward edge if v was gray when (u, v) traversed and d[v] > d[u].

4. Cross edges (u,v) is a cross edge if v was gray when (u, v) traversed and d[v] < d[u].

s

a

c

h

k

f

i

l

m

j

e

b

gd

8/19

1/27

25/262/20

3/19

17/18

21/24

11/16

12/15

13/14

9/10

5/6

4/7

22/23

Classifying edges can help to identify properties of the graph, e.g., a graph is acyclic iff DFS yields no back edges.

CSE 310194

Undirected Graphs

• In a depth-first search of an undirected graph, every edge is either a tree edge or a back edge.

• Why?

CSE 310195

Undirected Graphs

• Suppose that (u,v) is a forward edge or a cross edge in a DFS of an undirected graph.

• (u,v) is a forward edge or a cross edge when vis already handled (grey) when accessed from u.

• This means that all vertices reachable from vhave been explored.

• Since we are currently handling u, u must be red.

• Clearly v is reachable from u.

• Since the graph is undirected, u must also be reachable from v.

• Thus u must already have been handled: u must be grey.

• Contradiction!

u

v

End of Lecture 11

Apr 8, 2009

CSE 310197

Depth-First Search Algorithm

RED

BLACK

GRAY GRAY

BLACK

Precondition: vertex is undiscoveredPostcondition: all vertices reachable from have

DFS-Vi

been p

si

ro

t

s

( )

ces edu

u

u

BLACK

BLACK

DFS(G)

CSE 310198

Depth-First Search Algorithm

BLACK

BLACK

RED

BLACK

GRAY GRAY

BLACK

Precondition: vertex is undiscoveredPostcondition: all vertices reachable from have

DFS-Vi

been p

si

ro

t

s

( )

ces edu

u

u

DFS(G)

θ∈

=∑total work = | [ ] | ( )v V

Adj v E

θtotal work = ( )V

θ +Thus running time = ( )V E

Topological Sorting (e.g., putting tasks in linear order)

An application of Depth-First Search

CSE 3101100

Linear Order

underwear

pants

socks

shoes

underwearpantssocksshoes

socks underwearpantsshoes

CSE 3101101

Linear Order

underwear

pants

socks

shoes

Too many video games?

CSE 3101102

Linear Order

a

b hc id je k

f l

g

Precondition:A Directed Acyclic Graph(DAG)

Post Condition:Find one valid linear order

Algorithm:•Find a terminal node (sink).•Put it last in sequence.•Delete from graph & repeat

Θ(V)Θ(V2)

….. lWe can do better!

CSE 3101103

Linear Order

a

b hc id je k

f

g

FoundNot Handled

Stack

Alg: DFS

degf

l

….. f

CSE 3101104

Linear Order

a

b hc id je k

f

g

FoundNot Handled

Stack

Alg: DFS

degl

lWhen node is popped off stack, insert at front of linearly-ordered “to do” list.

….. fLinear Order:

CSE 3101105

Linear Order

a

b hc id je k

f

g

FoundNot Handled

Stack

Alg: DFS

deg

l

l,fLinear Order:

CSE 3101106

Linear Order

a

b hc id je k

f

g

FoundNot Handled

Stack

Alg: DFS

del

g,l,fLinear Order:

CSE 3101107

Linear Order

a

b hc id je k

f

g

FoundNot Handled

Stack

Alg: DFS

dl

e,g,l,fLinear Order:

CSE 3101108

Linear Order

a

b hc id je k

f

g

FoundNot Handled

Stack

Alg: DFS

l

d,e,g,l,fLinear Order:

CSE 3101109

Linear Order

a

b hc id je k

f

g

FoundNot Handled

Stack

Alg: DFS

il

d,e,g,l,f

jk

Linear Order:

CSE 3101110

Linear Order

a

b hc id je k

f

g

FoundNot Handled

Stack

Alg: DFS

il

k,d,e,g,l,f

j

Linear Order:

CSE 3101111

Linear Order

a

b hc id je k

f

g

FoundNot Handled

Stack

Alg: DFS

il

j,k,d,e,g,l,fLinear Order:

CSE 3101112

Linear Order

a

b hc id je k

f

g

FoundNot Handled

Stack

Alg: DFS

l

i,j,k,d,e,g,l,fLinear Order:

CSE 3101113

Linear Order

a

b hc id je k

f

g

FoundNot Handled

Stack

Alg: DFS

blc

i,j,k,d,e,g,l,fLinear Order:

CSE 3101114

Linear Order

a

b hc id je k

f

g

FoundNot Handled

Stack

Alg: DFS

bl

c,i,j,k,d,e,g,l,fLinear Order:

CSE 3101115

Linear Order

a

b hc id je k

f

g

FoundNot Handled

Stack

Alg: DFS

l

b,c,i,j,k,d,e,g,l,fLinear Order:

CSE 3101116

Linear Order

a

b hc id je k

f

g

FoundNot Handled

Stack

Alg: DFS

alh

b,c,i,j,k,d,e,g,l,fLinear Order:

CSE 3101117

Linear Order

a

b hc id je k

f

g

FoundNot Handled

Stack

Alg: DFS

al

h,b,c,i,j,k,d,e,g,l,fLinear Order:

CSE 3101118

Linear Order

a

b hc id je k

f

g

FoundNot Handled

Stack

Alg: DFS

l

a,h,b,c,i,j,k,d,e,g,l,f Done!Linear Order:

CSE 3101119

Linear OrderFound

Not HandledStack

Proof:•Case 1: u goes on stack first before v.

•Because of edge,v goes on before u comes off

•v comes off before u comes off•v goes after u in order. ☺

u vv…u…

Consider each edge

v

…

u

…

CSE 3101120

Linear OrderFound

Not HandledStack

Proof:•Case 1: u goes on stack first before v.•Case 2: v goes on stack first before u.

v comes off before u goes on.•v goes after u in order. ☺

u vv…u…

Consider each edge

u

…

v

…

CSE 3101121

Linear OrderFound

Not HandledStack

Proof:•Case 1: u goes on stack first before v.•Case 2: v goes on stack first before u.

v comes off before u goes on.Case 3: v goes on stack first before u.

u goes on before v comes off.•Panic: u goes after v in order. •Cycle means linear order

is impossible ☺

u vu…v…

Consider each edge

u

…

v

…

The nodes in the stack form a path starting at s.

CSE 3101122

Linear Order

a

b hc id je k

f

g

FoundNot Handled

Stack

Alg: DFS

l

a,h,b,c,i,j,k,d,e,g,l,f Done!Linear Order:

ΘAnalysis: (V+E)

Shortest Paths Revisited

CSE 3101124

3

Back to Shortest Path

• BFS finds the shortest paths from a source node s to every vertex v in the graph.

• Here, the length of a path is simply the number of edges on the path.

• But what if edges have different ‘costs’?

s

v

( , ) 3s vδ = ( , ) 12s vδ =

2s

v517

2

Single-Source (Weighted) Shortest Paths

CSE 3101126

The Problem

• What is the shortest driving route from Toronto to Ottawa? (e.g.MAPQuest, Google Maps)

• Input:

0 1 11

Weight of path , ,..., ( , )k

i iki

p v v v w v v−=

=< > = ∑Shortest-path weight from to :u v

min{ ( ) : } if a path ,( , ) otherwise.

pw p u v u vu vδ

⎧⎪ → → ∃ → →= ⎨∞⎪⎩

Shortest path from to is any path such that ( ) ( , ).u v p w p u vδ=

Directed Graph ( , )G V E=

Edge weights :w E →

CSE 3101127

Example

Single-source shortest path search induces a search tree rooted at s.

This tree, and hence the paths themselves, are not necessarily unique.

CSE 3101128

Shortest path variants

• Single-source shortest-paths problem: – the shortest path from s to each vertex v. (e.g. BFS)

• Single-destination shortest-paths problem: Find a shortest path to a given destination vertex t from each vertex v.

• Single-pair shortest-path problem: Find a shortest path from u to v for given vertices u and v.

• All-pairs shortest-paths problem: Find a shortest path from u to v for every pair of vertices u and v.

CSE 3101129

Negative-weight edges

• OK, as long as no negative-weight cycles are reachable from the source.– If we have a negative-weight cycle, we can just keep going

around it, and get w(s, v) = −∞ for all v on the cycle.

– But OK if the negative-weight cycle is not reachable from the source.

– Some algorithms work only if there are no negative-weight edges in the graph.

CSE 3101130

Optimal substructure

• Lemma: Any subpath of a shortest path is a shortest path

• Proof: Cut and paste.

Now suppose there exists a shorter path .xyp

x y′

→ →

Then ( ) ( ).xy xyw p w p′ <

Construct p :′

Then ( ) ( ) ( ) ( )ux xy yvw p w p w p w p′ ′= + + ( ) ( ) ( )ux xy yvw p w p w p< + + ( ).w p=

So p wasn't a shortest path after all!

Suppose this path is a shortest path from to .p u v

Then ( , ) ( ) ( ) ( ) ( ).ux xy yvu v w p w p w p w pδ = = + +

CSE 3101131

Cycles

• Shortest paths can’t contain cycles:

– Already ruled out negative-weight cycles.

– Positive-weight: we can get a shorter path by omitting the cycle.

– Zero-weight: no reason to use them assume that our solutions won’t use them.

CSE 3101132

Output of a single-source shortest-path algorithm

• For each vertex v in V:

– d[v] = δ(s, v).

• Initially, d[v]=∞.

• Reduce as algorithm progresses. But always maintain d[v] ≥ δ(s, v).

• Call d[v] a shortest-path estimate.

– π[v] = predecessor of v on a shortest path from s.

• If no predecessor, π[v] = NIL.

• π induces a tree — shortest-path tree.

CSE 3101133

Initialization

• All shortest-paths algorithms start with the same initialization:

INIT-SINGLE-SOURCE(V, s)

for each v in Vdo d[v]←∞

π[v] ← NIL

d[s] ← 0

CSE 3101134

Relaxing an edge

• Can we improve shortest-path estimate for v by going through u and taking (u,v)?

RELAX(u, v,w)

if d[v] > d[u] + w(u, v) then

d[v] ← d[u] + w(u, v)

π[v]← u

CSE 3101135

General single-source shortest-path strategy

1. Start by calling INIT-SINGLE-SOURCE

2. Relax Edges

Algorithms differ in the order in which edges are taken and

how many times each edge is relaxed.

CSE 3101136

Example: Single-source shortest paths in a directed acyclic graph (DAG)

• Since graph is a DAG, we are guaranteed no negative-weight cycles.

CSE 3101137

Algorithm

Time: ( )V EΘ +

CSE 3101138

Example

CSE 3101139

Example

CSE 3101140

Example

CSE 3101141

Example

CSE 3101142

Example

CSE 3101143

Example

CSE 3101144

Correctness: Path relaxation property (Lemma 24.15)

0 1 0Let , , . . . , be a shortest path from .k kp v v v s v to v=< > =

0 1 1 2 -1If we relax, in order, ( , ), ( , ), . . . , ( , ), k kv v v v v v

even intermixed with other relaxations,

then [ ] ( , ).k kd v s vδ=

CSE 3101145

Correctness of DAG Shortest Path Algorithm

• Because we process vertices in topologically sorted order, edges of any path are relaxed in order of appearance in the path.

– Edges on any shortest path are relaxed in order.

– By path-relaxation property, correct.

CSE 3101146

Example: Dijkstra’s algorithm

• Applies to general weighted directed graph (may contain cycles).

• But weights must be non-negative.

• Essentially a weighted version of BFS.– Instead of a FIFO queue, uses a priority queue.

– Keys are shortest-path weights (d[v]).

• Maintain 2 sets of vertices:– S = vertices whose final shortest-path weights are determined.

– Q = priority queue = V-S.

CSE 3101147

Dijkstra’s algorithm

Dijkstra’s algorithm can be viewed as greedy, since it always chooses the “lightest” vertex in V − S to add to S.

CSE 3101148

Dijkstra’s algorithm: Analysis

Analysis:Using minheap, queue operations takes O(logV) time

( )O V

(log )O V ( ) iterationsO V×

(log )O V ( ) iterationsO E×

Running Time is ( log )O E V→

CSE 3101149

Example⇔

⇔⇔Grey Handling

B

White N

lack

ot Fo

Ha

und

ndled

Key:

CSE 3101150

Example

CSE 3101151

Example

CSE 3101152

Example

CSE 3101153

Example

CSE 3101154

Example

CSE 3101155

Correctness of Dijkstra’s algorithm

• Loop invariant: d[v] = δ(s, v) for all v in S.– Initialization: Initially, S is empty, so trivially true.

– Termination: At end, Q is empty S = V d[v] = δ(s, v) for all v in V.

– Maintenance:• Need to show that

– d[u] = δ(s, u) when u is added to S in each iteration.

– d[u] does not change once u is added to S.

CSE 3101156

Correctness of Dijkstra’s Algorithm: Upper Bound Property

• Upper Bound Property:1. [ ] ( , )d v s v v Vδ≥ ∀ ∈

• Proof:By induction.

[ ] ( , ) immediately after initialization, since[ ] 0 ( ,

Base Cas :

[ ]

e)

d v s v v Vd s s sd v v s

δδ

≥ ∀ ∈= == ∞∀ ≠

δ≥ ∀ ∈SupposeInductive Step:

[ ] ( , )d x s x x V

( , ) ( , )s u w u vδ≥ +

( , )s vδ≥

If [ ] changes, then [ ] [ ] ( , )d v d v d u w u v= +

Suppose we relax edge ( , ).u v

2. Once [ ] ( , ), it doesn't changed v s vδ=

CSE 3101157

Correctness of Dijkstra’s Algorithm

When is added to Clai , [ ] (m ): ,u S d u s uδ=

Let be first vertex in on shortest path to y V S u−

Let be the predecessor of on the shortest path to x y u

[ ] ( , ) when is added toCl : .aim d y s y u Sδ=

Proof:[ ] ( , ), since x .d x s x Sδ= ∈

( , ) was relaxed when was added to x y x S [ ] ( , ) ( , ) ( , )d y s x w x y s yδ δ→ = + =

Handled

Let be the first vertex added to such thaProof by Con

t [ ] ( , ) when is added.tradiction: u S

d u s u uδ≠

CSE 3101158

Correctness of Dijkstra’s AlgorithmThus [ ] ( , ) when is added to .d y s y u Sδ=

[ ] ( , ) ( , ) [ ] (upper bound property)d y s y s u d uδ δ→ = ≤ ≤

But [ ] [ ] when added to d u d y u S≤

Thus [ ] ( , ) ( , ) [ ]!d y s y s u d uδ δ= = =

Thus when is added to , [ ] ( , )u S d u s uδ=

There is a shortest path to such that the predecessor of [ ]Conse

whequences

n is added to .:

u u u S u Sπ ∈

π [ ]u

=2The path through can only be a shortest path if [ ] 0.y w p

Handled

CSE 3101159

Correctness of Dijkstra’s algorithm

• Loop invariant: d[v] = δ(s, v) for all v in S.– Maintenance:

• Need to show that – d[u] = δ(s, u) when u is added to S in each iteration.

– d[u] does not change once u is added to S.

δ=Thus once [ ] ( , ), it will not be changed.d v s v

can only decRelax(u rease ],v,w) [ .d v

δ≥upper bound prBy the , operty [ ] ( , ).d v s v

?

graph search algorithms - york university

Documents