cs 3343: analysis of algorithms lecture 21: introduction to graphs

CS 3343: Analysis of Algorithms

Lecture 21: Introduction to Graphs

Uniform-profit Restaurant location problem

5 2 2 6 6 63 10 7

d 0 5 7 9 150

6 9 150

100

7

Goal: maximize number of restaurants openSubject to: distance constraint (min-separation >= 10)

Events scheduling problem

Time

e1 e2

e3e4 e5

e6

e7

e8

e9

Goal: maximize number of non-conflicting events

Fractional knapsack problem• Goal: maximize value

without exceeding bag capacity

• Weight limit: 10LB

1.5

2

1.2

1

0.75

1

$ / LB

966

425

654

333

342

221

Value ($)

Weight (LB)

item

Example• Goal: maximize value

without exceeding bag capacity

• Weight limit: 10LB

• 2 + 6 + 2 = 10 LB• 4 + 9 + 1.2*2 = 15.4

item Weight (LB)

Value ($)

$ / LB

5 2 4 2

6 6 9 1.5

4 5 6 1.2

1 2 2 1

3 3 3 1

2 4 3 0.75

The remaining lectures

• Graph algorithms

• Very important in practice– Tons of computational problems can be

defined in terms of graphs– We’ll study a few interesting ones

• Minimum spanning tree• Shortest path• Graph search• Topological sort, connected components

Graphs

• A graph G = (V, E)– V = set of vertices– E = set of edges = subset of V V– Thus |E| = O(|V|2)

1

2 4

3

Vertices: {1, 2, 3, 4}

Edges: {(1, 2), (2, 3), (1, 3), (4, 3)}

Graph Variations (1)

• Directed / undirected:– In an undirected graph:

• Edge (u,v) E implies edge (v,u) E• Road networks between cities

– In a directed graph:• Edge (u,v): uv does not imply vu• Street networks in downtown

– Degree of vertex v:• The number of edges adjacency to v• For directed graph, there are in-degree and out-degree

1

2 4

3

Directed

1

2 4

3

Undirected

Degree = 3In-degree = 3Out-degree = 0


• Weighted / unweighted:– In a weighted graph, each edge or vertex has an

associated weight (numerical value)• E.g., a road map: edges might be weighted w/ distance

1

2 4

3

1

2 4

3Unweighted Weighted

0.3

0.4

1.2

1.9


• Connected / disconnected:– A connected graph has a path from every

vertex to every other– A directed graph is strongly connected if there

is a directed path between any two vertices

1

2 4

3

Connected but not strongly connected


• Dense / sparse:– Graphs are sparse when the number of edges is

linear to the number of vertices• |E| O(|V|)

– Graphs are dense when the number of edges is quadratic to the number of vertices

• |E| O(|V|2)

– Most graphs of interest are sparse– If you know you are dealing with dense or sparse

graphs, different data structures may make sense

Representing Graphs

• Assume V = {1, 2, …, n}• An adjacency matrix represents the graph as a n

x n matrix A:– A[i, j] = 1 if edge (i, j) E

= 0 if edge (i, j) E

• For weighted graph– A[i, j] = wij if edge (i, j) E

= 0 if edge (i, j) E

• For undirected graph– Matrix is symmetric: A[i, j] = A[j, i]

Graphs: Adjacency Matrix

• Example:

1

2 4

3

A 1 2 3 4

1

2

3 ??4


• Example:

1

2 4

3

A 1 2 3 4

1 0 1 1 0

2 0 0 1 0

3 0 0 0 0

4 0 0 1 0

How much storage does the adjacency matrix require?A: O(V2)


• Example:

1

2 4

3 4

3

2

0100

1011

0101

01101

4321A

Undirected graph


• Example:

1

2 4

3

5

6

9 4

4

3

2

0400

4096

0905

06501

4321A

Weighted graph


• Time to answer if there is an edge between vertex u and v: Θ(1)

• Memory required: Θ(n2) regardless of |E|– Usually too much storage for large graphs– But can be very efficient for small graphs

• Most large interesting graphs are sparse– E.g., road networks (due to limit on junctions)– For this reason the adjacency list is often a

more appropriate representation

Graphs: Adjacency List

• Adjacency list: for each vertex v V, store a list of vertices adjacent to v

• Example:– Adj[1] = {2,3}– Adj[2] = {3}– Adj[3] = {}– Adj[4] = {3}

• Variation: can also keep a list of edges coming into vertex

1

2 4

3

Graph representations

• Adjacency list

1

2 4

3

2 3

3

3

How much storage does the adjacency list require?A: O(V+E)


• Undirected graph

1

2 4

3 4

3

2

0100

1011

0101

01101

4321A

2 3

1

3

3

1 2 4


• Weighted graph

1

2 4

3

5

6

9 4 4

3

2

0400

4096

0905

06501

4321A

2,5 3,6

1,5 3,9

3,4

1,6 2,9 4,4

Tradeoffs between the two representations

Adj Matrix Adj List

test (u, v) E Θ(1) O(n)

Degree(u) Θ(n) O(n)

Memory Θ(n2) Θ(n+m)

Edge insertion Θ(1) Θ(1)

Edge deletion Θ(1) O(n)

Graph traversal Θ(n2) Θ(n+m)

|V| = n, |E| = m

Both representations are very useful and have different properties.

Minimum Spanning Tree

• Problem: given a connected, undirected, weighted graph:

1410

3

6 45

2

9

15

8


• Problem: given a connected, undirected, weighted graph, find a spanning tree using edges that minimize the total weight

1410

3

6 45

2

9

15

8

• A spanning tree is a tree that connects all vertices

• Number of edges = ?• A spanning tree has no

designated root.

How to find MST?

• Connect every node to the closest node?– Does not guarantee a spanning tree


• MSTs satisfy the optimal substructure property: an optimal tree is composed of optimal subtrees– Let T be an MST of G with an edge (u,v) in the middle

– Removing (u,v) partitions T into two trees T1 and T2

– w(T) = w(u,v) + w(T1) + w(T2)

• Claim 1: T1 is an MST of G1 = (V1, E1), and T2 is an MST of G2 = (V2, E2)

T1T2

u v

Proof by contradiction:

• if T1 is not optimal, we can replace T1 with a better spanning tree, T1’

• T1’, T2 and (u, v) form a new spanning tree T’

• W(T’) < W(T). Contradiction.

T1’


• MSTs satisfy the optimal substructure property: an optimal tree is composed of optimal subtrees– Let T be an MST of G with an edge (u,v) in the middle

– Removing (u,v) partitions T into two trees T1 and T2

– w(T) = w(u,v) + w(T1) + w(T2)

• Claim 2: (u, v) is the lightest edge connecting G1 = (V1, E1) and G2 = (V2, E2)

T1T2

u v

Proof by contradiction: • if (u, v) is not the lightest edge, we

can remove it, and reconnect T1 and T2 with a lighter edge (x, y)

• T1, T2 and (x, y) form a new spanning tree T’

• W(T’) < W(T). Contradiction.

x y

Algorithms

• Generic idea:– Compute MSTs for sub-graphs– Connect two MSTs for sub-graphs with the lightest

edge

• Two of the most well-known algorithms– Prim’s algorithm– Kruskal’s algorithm– Let’s first talk about the ideas behind the algorithms

without worrying about the implementation and analysis

Prim’s algorithm

• Basic idea:– Start from an arbitrary single node

• A MST for a single node has no edge

– Gradually build up a single larger and larger MST

65

7

Fully explored nodes

Discovered but not fully explored nodes

Not yet discovered

Prim’s algorithm




65

7

2

94

Fully explored nodes

Discovered but not fully explored nodes

Not yet discovered

Prim’s algorithm




65

7

2

94

Prim’s algorithm in words

• Randomly pick a vertex as the initial tree T

• Gradually expand into a MST:– For each vertex that is not in T but directly

connected to some nodes in T• Compute its minimum distance to any vertex in T

– Select the vertex that is closest to T• Add it to T

Example

aa

bb ff

cc ee

dd

gg

hh

6 125

14

3

8

10

15

9

7

Example

aa

bb ff

cc ee

dd

gg

hh

6 125

14

3

8

10

15

9

7

Total weight = 3 + 8 + 6 + 5 + 7 + 9 + 15 = 53

Kruskal’s algorithm

• Basic idea:– Grow many small trees – Find two trees that are closest (i.e., connected

with the lightest edge), join them with the lightest edge

– Terminate when a single tree forms

Claim

• If edge (u, v) is the lightest among all edges, (u, v) is in a MST

• Proof by contradiction:– Suppose that (u, v) is not in any MST– Given a MST T, if we connect (u, v), we create a cycle– Remove an edge in the cycle, have a new tree T’– W(T’) < W(T)

u v

By the same argument, the second, third, …, lightest edges, if they do not create a cycle, must be in MST

Kruskal’s algorithm in words

• Procedure:– Sort all edges into non-decreasing order– Initially each node is in its own tree– For each edge in the sorted list

• If the edge connects two separate trees, then– join the two trees together with that edge

Example

aa

bb ff

cc ee

dd

gg

hh

6 125

14

3

8

10

15

9

7

c-d: 3

b-f: 5

b-a: 6

f-e: 7

b-d: 8

f-g: 9

d-e: 10

a-f: 12

b-c: 14

e-h: 15

cs 3343: analysis of algorithms lecture 21: introduction to graphs

Documents