more chapter 7: greedy algorithms
DESCRIPTION
More Chapter 7: Greedy Algorithms. Kruskal’s Minimum Spanning Tree Algorithm. Minimum Spanning Tree (MST) Problem. Given a weighted graph, i.e. a graph with edge weights… try to find a sub-graph that (i) connects all the nodes and (ii) the sum of the edge weights is minimal. - PowerPoint PPT PresentationTRANSCRIPT
More Chapter 7: Greedy Algorithms
Kruskal’s Minimum Spanning Tree Algorithm.
Minimum Spanning Tree (MST) Problem
• Given a weighted graph, i.e. a graph with edge weights…
• try to find a sub-graphthat(i) connects all the nodes and (ii) the sum of the edge weights is minimal.
• This sub-graph will always be a tree.
A B C
D E F G
H I J
4 5
7 3 4
5 5
3 6
7
4
6 43
5 6
5
MST Example
• Given the followinggraph, find the MST
• First, we want allthe nodes connected
• Second, we want topick the lowest weight edges.
A B C
D E F G
H I J
4 5
7 3 4
5 5
3 6
7
4
6 43
5 6
5
MST Example
• Greedy step 1:
• Start with Aand select theminimum edge
• This would connect D.
A B C
D E F G
H I J
4 5
7 3 4
5 5
3 6
7
4
6 43
5 6
5
Total Weight: 3
MST Example
• Greedy step 2:
• From Dselect theminimum edge
• This would connect H.
A B C
D E F G
H I J
4 5
7 3 4
5 5
3 6
7
4
6 43
5 6
5
Total Weight: 6
MST Example
• Greedy step 3:
• From Hselect theminimum edge
• This would connect I.
A B C
D E F G
H I J
4 5
7 3 4
5 5
3 6
7
4
6 43
5 6
5
Total Weight: 11
MST Example
• Greedy step 4:
• From Iselect theminimum edge
• This would connect J.
A B C
D E F G
H I J
4 5
7 3 4
5 5
3 6
7
4
6 43
5 6
5
Total Weight: 16
MST Example
• Greedy step 5:
• From Jselect theminimum edge
• This would connect G.
A B C
D E F G
H I J
4 5
7 3 4
5 5
3 6
7
4
6 43
5 6
5
Total Weight: 20
MST Example
• Greedy step 6:
• From Gselect theminimum edge
• This would connect F.
A B C
D E F G
H I J
4 5
7 3 4
5 5
3 6
7
4
6 43
5 6
5
Total Weight: 24
MST Example
• Greedy step 6:
• From Gselect theminimum edge
• This would connect F.
A B C
D E F G
H I J
4 5
7 3 4
5 5
3 6
7
4
6 43
5 6
5
Total Weight: 24
MST Example
• What is the running timeof the algorithm?
A B C
D E F G
H I J
4 5
7 3 4
5 5
3 6
7
4
6 43
5 6
5
Total Weight: 27
MST Example
• N steps
• Each step mustfind the minimumedge from a node
• Worst case:N-1 + N-1 + N-1 + … + N-1 = O(N2)
A B C
D E F G
H I J
4 5
7 3 4
5 5
3 6
7
4
6 43
5 6
5
MST Example
• You might think the worst case is:
N-1 + N-2 + N-3 + … + 3 + 2 + 1 = O(N2)
because at every step you connect a node and don’t have to consider it’s edges.
• However, think about how the algorithm would actually be implemented, and how you would keep track of this info?
MST Example• You might think the worst case is:
N-1 + N-2 + N-3 + … + 3 + 2 + 1 = O(N2)
pick node v from the node_list.while node_list is not empty {
mark v as visited.min_hop = infinity;foreach of v’s edges (v,w)
if (w is not visited)if (edge (v,w) < min_hop) {min_hop = edge(v,w)min_edge = w;
}remove v from node from node_listv = w;
}The first while loop will always take N iterationsThe foreach loop could take N-1 iteration in a complete graph
MST Example
• Greedy step 8:
• From Eselect theminimum edge
• This would connect B.
A B C
D E F G
H I J
4 5
7 3 4
5 5
3 6
7
4
6 43
5 6
5
Total Weight: 33
MST Example
• Greedy step 9:
• From Bselect theminimum edge
• This would connect C.
A B C
D E F G
H I J
4 5
7 3 4
5 5
3 6
7
4
6 43
5 6
5
Total Weight: 38
MST Example
• This greedyalgorithm failed
• Why?
A B C
D E F G
H I J
4 5
7 3 4
5 5
3 6
7
4
6 43
5 6
5
Total Weight: 38
MST Example
• It makes a localdecision.
• From E, itchooses to go toB
• We have to consider other options.
A B C
D E F G
H I J
4 5
7 3 4
5 5
3 6
7
4
6 43
5 6
5
Total Weight: 38
Kruskal’s Algorithm
• Solves the Minimum Spanning Tree Problem using a better Greedy Approach
• Input:– List of edges in a graph– n – the number of vertices
• Output:– Prints the list of edges in the Minimum
Spanning Tree
A B C
D E F G
H I J
4 5
7 3 4
5 5
3 6
7
4
6 43
5 6
5
Kruskal’skruskal(e, n) {
sort(e);
A B C
D E F G
H I J
4 5
7 3 4
5 5
3 6
7
4
6 43
5 6
5
Kruskal’skruskal(e, n) {
sort(e);
A B C
D E F G
H I J
4 5
7 3 4
5 5
3 6
7
4
6 43
5 6
5
A
D
3
D
H
3
E F3 A B4
C
F
4
F G4
G
J
4
A
E
5
H I5
I J5
F
J
5
B C5
B
E
6
C
G
6
F
I
6
D E7
E
H
7
kruskal(e, n) {
sort(e);
for (i = A to J)
makeset(i)
A B C
D E F G
H I J
4 5
7 3 4
5 5
3 6
7
4
6 43
5 6
5
A
D
3
D
H
3
E F3 A B4
C
F
4
F G4
G
J
4
A
E
5
H I5
I J5
F
J
5
B C5
B
E
6
C
G
6
F
I
6
D E7
E
H
7
A B C D E F G H I J
kruskal(e, n) {
...
count = 0;
i = 1
A B C
D E F G
H I J
4 5
7 3 4
5 5
3 6
7
4
6 43
5 6
5
A
D
3
D
H
3
E F3 A B4
C
F
4
F G4
G
J
4
A
E
5
H I5
I J5
F
J
5
B C5
B
E
6
C
G
6
F
I
6
D E7
E
H
7
A B C D E F G H I J
Count
0
i
1
kruskal(e, n) {
while (count < n-1) {
if (findset(e[i].v) != findset(e[i].w)) {
print(e[i].v + “ ”+ e[i].w);
count++;
union(e[i].v, e[i].w);
}
i++;
}
A
D
3
D
H
3
E F3 A B4
C
F
4
F G4
G
J
4
A
E
5
H I5
I J5
F
J
5
B C5
B
E
6
C
G
6
F
I
6
D E7
E
H
7
A B C D E F G H I J
count
0
i
1
n
10
kruskal(e, n) {
while (count < n-1) {
if (findset(e[i].v) != findset(e[i].w)) {
print(e[i].v + “ ”+ e[i].w);
count++;
union(e[i].v, e[i].w);
}
i++;
}
A
D
3
D
H
3
E F3 A B4
C
F
4
F G4
G
J
4
A
E
5
H I5
I J5
F
J
5
B C5
B
E
6
C
G
6
F
I
6
D E7
E
H
7
A B C DH E F G I J
Count
1
i
2
n
10
kruskal(e, n) {
while (count < n-1) {
if (findset(e[i].v) != findset(e[i].w)) {
print(e[i].v + “ ”+ e[i].w);
count++;
union(e[i].v, e[i].w);
}
i++;
}
A
D
3
D
H
3
E F3 A B4
C
F
4
F G4
G
J
4
A
E
5
H I5
I J5
F
J
5
B C5
B
E
6
C
G
6
F
I
6
D E7
E
H
7
A B C DH EF G I J
Count
2
i
3
n
10
kruskal(e, n) {
while (count < n-1) {
if (findset(e[i].v) != findset(e[i].w)) {
print(e[i].v + “ ”+ e[i].w);
count++;
union(e[i].v, e[i].w);
}
i++;
}
A
D
3
D
H
3
E F3 A B4
C
F
4
F G4
G
J
4
A
E
5
H I5
I J5
F
J
5
B C5
B
E
6
C
G
6
F
I
6
D E7
E
H
7
ADH B C EF G I J
Count
3
i
4
n
10
kruskal(e, n) {
while (count < n-1) {
if (findset(e[i].v) != findset(e[i].w)) {
print(e[i].v + “ ”+ e[i].w);
count++;
union(e[i].v, e[i].w);
}
i++;
}
A
D
3
D
H
3
E F3 A B4
C
F
4
F G4
G
J
4
A
E
5
H I5
I J5
F
J
5
B C5
B
E
6
C
G
6
F
I
6
D E7
E
H
7
ADH B C EFG I J
Count
4
i
5
n
10
kruskal(e, n) {
while (count < n-1) {
if (findset(e[i].v) != findset(e[i].w)) {
print(e[i].v + “ ”+ e[i].w);
count++;
union(e[i].v, e[i].w);
}
i++;
}
A
D
3
D
H
3
E F3 A B4
C
F
4
F G4
G
J
4
A
E
5
H I5
I J5
F
J
5
B C5
B
E
6
C
G
6
F
I
6
D E7
E
H
7
ADHB C EFG I J
Count
5
i
6
n
10
kruskal(e, n) {
while (count < n-1) {
if (findset(e[i].v) != findset(e[i].w)) {
print(e[i].v + “ ”+ e[i].w);
count++;
union(e[i].v, e[i].w);
}
i++;
}
A
D
3
D
H
3
E F3 A B4
C
F
4
F G4
G
J
4
A
E
5
H I5
I J5
F
J
5
B C5
B
E
6
C
G
6
F
I
6
D E7
E
H
7
ADHB CEFG I J
Count
6
i
7
n
10
kruskal(e, n) {
while (count < n-1) {
if (findset(e[i].v) != findset(e[i].w)) {
print(e[i].v + “ ”+ e[i].w);
count++;
union(e[i].v, e[i].w);
}
i++;
}
A
D
3
D
H
3
E F3 A B4
C
F
4
F G4
G
J
4
A
E
5
H I5
I J5
F
J
5
B C5
B
E
6
C
G
6
F
I
6
D E7
E
H
7
ADHB CEFGJ I
Count
7
i
8
n
10
kruskal(e, n) {
while (count < n-1) {
if (findset(e[i].v) != findset(e[i].w)) {
print(e[i].v + “ ”+ e[i].w);
count++;
union(e[i].v, e[i].w);
}
i++;
}
A
D
3
D
H
3
E F3 A B4
C
F
4
F G4
G
J
4
A
E
5
H I5
I J5
F
J
5
B C5
B
E
6
C
G
6
F
I
6
D E7
E
H
7
ADHBCEFGJ I
Count
8
i
9
n
10
kruskal(e, n) {
while (count < n-1) {
if (findset(e[i].v) != findset(e[i].w)) {
print(e[i].v + “ ”+ e[i].w);
count++;
union(e[i].v, e[i].w);
}
i++;
}
A
D
3
D
H
3
E F3 A B4
C
F
4
F G4
G
J
4
A
E
5
H I5
I J5
F
J
5
B C5
B
E
6
C
G
6
F
I
6
D E7
E
H
7
ADHBCEFGJ I
Count
8
i
10
n
10
kruskal(e, n) {
while (count < n-1) {
if (findset(e[i].v) != findset(e[i].w)) {
print(e[i].v + “ ”+ e[i].w);
count++;
union(e[i].v, e[i].w);
}
i++;
}
A
D
3
D
H
3
E F3 A B4
C
F
4
F G4
G
J
4
A
E
5
H I5
I J5
F
J
5
B C5
B
E
6
C
G
6
F
I
6
D E7
E
H
7
ADHBCEFGJI
Count
9
i
11
n
10
A
D
3
D
H
3
E F3 A B4
C
F
4
F G4
G
J
4
A
E
5
H I5
I J5
F
J
5
B C5
B
E
6
C
G
6
F
I
6
D E7
E
H
7
A B C
D E F G
H I J
4 5
7 3 4
5 5
3 6
7
4
6 43
5 6
5
A
D
3
D
H
3
E F3 A B4
C
F
4
F G4
G
J
4
A
E
5
H I5
I J5
F
J
5
B C5
B
E
6
C
G
6
F
I
6
D E7
E
H
7
A B C
D E F G
H I J
4 5
7 3 4
5 5
3 6
7
4
6 43
5 6
5
A B C
D E F G
H I J
4 5
7 3 4
5 5
3 6
7
4
6 43
5 6
5
A
B
C
D E
F
G
H
I
J
Theorem 7.2.5 pp. 280
• Let G be a connected, weighted graph, and let G’ be a sub-graph of a minimal spanning tree of G. Let C be a component of G’, and let S be the set of all Edges with one vertex in C and the other not in C. If we add a minimum weight edge in S to G’, the resulting graph is also contained in a minimal spanning tree of G
Theorem 7.2.5 pp. 280• Let G be a connected, weighted graph, and let
G’ be a sub-graph of a minimal spanning tree of G. Let C be a component of G’, and let S be the set of all Edges with one vertex in C and the other not in C. If we add a minimum weight edge in S to G’, the resulting graph is also contained in a minimal spanning tree of G
A B C
D E F G
H I J
4 5
7 3 4
5 5
3 6
7
4
6 43
5 6
5
G A
B
C
D E
F
G
H
I
J
Minimal Spanning Tree of G
Theorem 7.2.5 pp. 280• G’ be a sub-graph of a minimal
spanning tree of G. Let C be a component of G’, and let S be the set of all Edges with one vertex in C and the other not in C.
A B C
D E F G
H I J
4 5
7 3 4
5 5
3 6
7
4
6 43
5 6
5
G G’ Subset of Minimal
Spanning Tree of G
S
C A
D E
A B4
A
E
5 D E7D
H
3
Theorem 7.2.5 pp. 280• If we add a minimum
weight edge from S to G’, the resulting graph is also contained in a minimal spanning tree of G
A B C
D E F G
H I J
4 5
7 3 4
5 5
3 6
7
4
6 43
5 6
5
G G’ Subset of Minimal
Spanning Tree of G
S
C A
D E
A B4
A
E
5 D E7D
H
3
Theorem 7.2.6: Kruskal’s Algorithm finds minimum spanning tree
Proof by induction
• G’ is a sub-graph constructed by Kruskal’s Algorithm
• G’ is initially empty but each step of the Algorithm increases the size of G’
• Inductive Assumption: G’ is contained in the MST.
Theorem 7.2.6: Kruskal’s Algorithm finds minimum spanning tree
Proof by induction• Let (v,w) be the next edge selected by Kruskal’s
Algorithm• Kruskal’s algorithm finds the minimum weight
edge (v,w) such that v and w are not already in G’• C can be any subset of the MST, so you can
always construct a C such that v is in C and w is not.
• Therefore, by Theorem 7.2.5, when (v,w) is added to G’, the resulting graph is also contained in the MST.