A Randomized Linear-Time A Randomized Linear-Time Algorithm to Find Minimum Algorithm to Find Minimum

Spanning TreesSpanning Trees

David R. David R. KargerKargerPhilip N. KleinPhilip N. Klein

Robert E. TarjanRobert E. Tarjan

Talk OutlineTalk Outline

Objective & related work from Objective & related work from literaturesliteratures

IntuitionIntuition DefinitionsDefinitions AlgorithmAlgorithm Proof & AnalysisProof & Analysis Conclusion and future workConclusion and future work

ObjectiveObjective

A minimum spanning tree is a tree A minimum spanning tree is a tree formed from a subset of the edges in a formed from a subset of the edges in a given undirected graph, with two given undirected graph, with two properties:properties:

1. it spans the graph, i.e., it includes 1. it spans the graph, i.e., it includes every every

vertex in the graph, andvertex in the graph, and

2. it is a minimum, i.e., the total weight of 2. it is a minimum, i.e., the total weight of

all the edges is as low as possible.all the edges is as low as possible.

Find a minimum spanning tree for a graph by linear time with very high probability!!

Related WorkRelated Work Boruvka 1926, textbook algorithms – Boruvka 1926, textbook algorithms – Yao 1975 – Yao 1975 – Cheriton and Tarjan 1976 –Cheriton and Tarjan 1976 – Fredman and Tarjan 1987 –Fredman and Tarjan 1987 – Gabow 1986 –Gabow 1986 – Chazelle 1995 – Chazelle 1995 –

)),(( nmmO

)),(( nmmO

)loglog( nmO

)log( nmO

)loglog( nmO

)),(log( nmmO

Deterministic results! How about the randomized one??

IntuitionIntuition Cycle PropertyCycle Property Cut PropertyCut Property RandomizationRandomization

IntuitionIntuition

For any cycle For any cycle CC in a graph, the heavies in a graph, the heaviest edge in t edge in CC does not apper in the mini does not apper in the minimum spanning tree.mum spanning tree.

Cycle PropertyCycle PropertyHeaviest edge

Cycle PropertyCycle Property

For any graph, find all possible For any graph, find all possible cycles and remove the heaviest edge cycles and remove the heaviest edge from each cycle. Then we can get a from each cycle. Then we can get a minimum spanning tree??minimum spanning tree??

How about the time complexity?How about the time complexity?

How to detect the cycles in the How to detect the cycles in the graph??graph??

Cut PropertyCut Property

For any proper nonempty subset For any proper nonempty subset XX of the vertices, the lightest edge of the vertices, the lightest edge with exactly one endpoint in with exactly one endpoint in XX belongs to the minimum spanning belongs to the minimum spanning tree.tree.

Cut PropertyCut Property

XV X

Boruvka AlgorithmBoruvka Algorithm For each vertex, select the minimum-For each vertex, select the minimum-

weight edge incident to the vertex. weight edge incident to the vertex. Contract all the selected edges, Contract all the selected edges, replacing by a single vertex each replacing by a single vertex each connected component defined by the connected component defined by the selected edges and deleting all resulting selected edges and deleting all resulting isolated vertices, loops (edges both of isolated vertices, loops (edges both of whose endpoints are the same), and all whose endpoints are the same), and all but the lowest-weight edge among each but the lowest-weight edge among each set of multiple edges.set of multiple edges. O(m log n)

RandomizationRandomization

How the randomization can help us to How the randomization can help us to achieve our goal?achieve our goal?

Boruvka + Cycle Property + RandomizBoruvka + Cycle Property + Randomizationation= Linear time with very high probabilit= Linear time with very high probabilityy

Definition

DefinitionDefinition

Let G be a graph with weighted edges. Let G be a graph with weighted edges. ww((xx, , yy) : the weight of edge {) : the weight of edge {xx, , yy} }

If If FF is a forest of a subgraph in is a forest of a subgraph in GG, , FF((xx, , yy) : the path (if any) connecting ) : the path (if any) connecting xx a and nd yy in in FF, , : the maximum weight : the maximum weight of an edge onof an edge on FF((xx, , yy), with the c), with the convention that onvention that if if xx and and yy are not are not connect connected in ed in FF..

),( yxwF

),( yxwF

DefinitionDefinition

FF-heavy-heavy

Otherwise, {Otherwise, {xx, , yy} is } is FF-light.-light.

),(),( yxwyxw F

FF-heavy & -heavy & FF-light-light

G

HF

FF-light-light

F

FG

FF-heavy-heavy

FF-heavy & -heavy & FF-light-light

Note that the edges of Note that the edges of FF are all are all FF-li-light.ght.For any forest For any forest FF, no , no FF-heavy edge -heavy edge can be in the minimum spanning fcan be in the minimum spanning forest of orest of GG..

Cycle Property!!

Recursive function call:

Input: A undirected graphOutput: A minimum spanning forestTime: for the worst case

O(m) with very high probability

Algorithm}) log , (min{ 2 nmnO

AlgorithmAlgorithm

Step 1. Apply two successive BoruStep 1. Apply two successive Boruvka steps to the graph, thereby redvka steps to the graph, thereby reducing the number of vertices by at ucing the number of vertices by at least a factor of four.least a factor of four.

AlgorithmAlgorithm

G

*G

AlgorithmAlgorithm

Step 2. In the contracted graph, choose Step 2. In the contracted graph, choose a subgraph a subgraph HH by selecting each edge in by selecting each edge independently with probability 1/2. Appldependently with probability 1/2. Apply the algorithm recursively to y the algorithm recursively to HH, produ, producing a minimum spanning forest cing a minimum spanning forest FF of of HH. Find all the . Find all the FF-heavy edges (both tho-heavy edges (both those in se in HH and those not in and those not in HH) and delete t) and delete them.hem.

AlgorithmAlgorithm

G

*G

HF

*G

HHF

Back to analysis

AlgorithmAlgorithm

Step 3. Apply the algorithm Step 3. Apply the algorithm recursively to the remaining graph recursively to the remaining graph to compute a spanning forest . to compute a spanning forest . Return those edges contracted in Return those edges contracted in Step 1 together with the edges Step 1 together with the edges of .of .

FF

AlgorithmAlgorithm

G

*G

)()()( * FEHEGE

*G

)()( * heavyFGE

HF

Back to analysis

FReturn

AlgorithmAlgorithm

F

Those not in H

F - light

F - heavy)()( HVFV Edges of H

)()(*)(

)()( *

FEHEGE

heavyFGE

AnalysisAnalysis

Correctness?Correctness? Worst-case time complexity?Worst-case time complexity? Expected time complexity?Expected time complexity?

CorrectnessCorrectness

CompletenessCompleteness

By the cut property, every edge By the cut property, every edge contracted during Step 1 is in the contracted during Step 1 is in the minimum spanning forest. Hence minimum spanning forest. Hence the remaining edges of the minimum the remaining edges of the minimum spanning forest of the original graph spanning forest of the original graph form a minimum spanning forest of form a minimum spanning forest of the contracted graph. the contracted graph.

CorrectnessCorrectness

SoundnessSoundness

By the cycle property, the edges By the cycle property, the edges deleted in Step 2 do not belong to deleted in Step 2 do not belong to minimum spanning forest. By the minimum spanning forest. By the inductive hypothesis, the minimum inductive hypothesis, the minimum spanning forest of the remaining spanning forest of the remaining graph is correctly determined in graph is correctly determined in recursive call of Step 3.recursive call of Step 3.

Worst-case time Worst-case time complexitycomplexity

The worst-case running time of the mini-sThe worst-case running time of the mini-spanning forest algorithm is panning forest algorithm is , the same as the bound for Boruvka’s alg , the same as the bound for Boruvka’s algorithm.orithm.

Count the total number of edges. Step 1 reCount the total number of edges. Step 1 reduces the size to ¼ as its original. A subproduces the size to ¼ as its original. A subproblem at depth blem at depth dd contains at most contains at most e edges. Summing all subproblems gives andges. Summing all subproblems gives an

bound on the total number of edge bound on the total number of edges. s.

}) log , (min{ 2 nmnO

2/)4/( 2dn)( 2nO

Worst-case time Worst-case time complexitycomplexity

Parent: Parent: EE((G)G) Left childLeft child: : EE((H)H) Right childRight child::

Number of edges in next recursion level Number of edges in next recursion level

= E(G*) + E(F)= E(G*) + E(F)= E(G) – V(G)/2 + V(G)/4= E(G) – V(G)/2 + V(G)/4

)(GE

Parent

Left Right

)()()(*)( HEFEHEGE

)()(*)( FEHEGE

Worst-case time Worst-case time complexitycomplexity

m edges

edges m

edges m

edges mnlog

Worst-case time Worst-case time complexitycomplexity

The total time spent in Steps 1-3 is linear in thThe total time spent in Steps 1-3 is linear in the number of edges: e number of edges:

Step 1 is just two steps of Boruvka’s algorithStep 1 is just two steps of Boruvka’s algorithm. m. Step 2 takes linear time using the modified DixStep 2 takes linear time using the modified Dixon-Rauch-Tarjan verification algorithm.on-Rauch-Tarjan verification algorithm.- - FF-heavy edges of -heavy edges of GG can be computed in time can be computed in time

linear in the number of edges of linear in the number of edges of GG..

}) log , (min{ 2 nmnO

Analysis

Given graph G with n vertices and m edges• After one Boruvka step, Boruvka step forms

connected components and replaces each by single vertex. Since each component connects more than 2 edges, there are at most n/2 vertices remained.

• For component with k vertices, exactly k – 1 edges are removed. Thus the edges removed is at least

where is set of connected components. Since there is at most n/2 components, there is at least n/2 edges removed.

ΓnΓv1vΓg

gΓg

g

Analysis

Given F = MST(H), for (x, y) in H

1. If (x, y) is in F, (x, y) is F-light

2. If (x, y) is not in F, assume (x, y) is F-light, the heaviest edge in cycle P (x, y) would be on P, and is belong to no MST according to cycle property. This causes contradiction, thus (x, y) is F-heavy.

3. Thus, each F-light edge in H is also in F, and vice versa.

F

P

X

Y

F-light

F-heavy

G* not selected

selected bysampling

F

G'{F-light edges in G*}

H

Analysis

According to the distribution of edges used by H and G', edges of F are used twice by calling MST(H)

and MST(G').

The binary tree represents the recursive invocation of MST:

Analysis

Left child represents invocation of MST(H).Right child represents invocation of MST(G').

Since 2 Borůvka step are performed before invocation of MST(H) and MST(G'), number of vertices is reduced in factor of 4. Thus, the height of invocation is at most log4n.

Analysis - Worst Case

Given graph G with m edges and n vertices

1. After 2 Borůvka steps, at most n/4 vertices and m – n /2 edges remain for G*. This is true also for H and G' which are subgraph of G*.

2. Since F = MST(H), F has at most vH – 1 edges, and thus less than n/4.

3. According to the edge distribution,eH + eG' eG* + eF

m – n/2 + vH

m – n/2 + n/4 mThus, the number of edges in subproblems is less than original’s.

FG'

H

G*

G*

G'

G

H

original problem

Borůvka

subproblem subproblem

sample

F-light

4. Since total edge number of subproblems at the same depth is bound by m, and the depth is at most log4n, the overall edge number is at most m log4n.

5. Since vertex number for submproblem at depth d is at most n/4d, the edge is at most (n/4d)2. Overall edge number is also bound by

Analysis - Worst Case

0d

22

dd n

78

4n2

6. Since running time of the algorithm is proportional to edge number, we could give time complexity as

O(min{n2, m log n})

Analysis – Average Case

Here, we analyze the average case by partitioning the invocations as “left paths” (red paths above). After reckoning edges of subproblems along each “left path”, sum them up and we will get the overall estimate.

G*

G'

G

H

original problem

Borůvka

subproblem subproblem

sample

F-light

Analysis – Average Case

1. For G* with k edges, after sampling with 1/2 probability for each edge, E[eH] = k/2.Since G* G, we have E[eG*] E(eG) and

E[eH] = E[eG*]/2 E[eG]/2.

2. Along the left path with starting E[eG] = k, the expected value of total edges is

k22

keEeE0d

d0d

)d(G0d

)d(G

G*

G'

G

H

original problem

Borůvka

subproblem subproblem

sample

F-light

Analysis – Average Case

1. For each F-light edge, there is 1/2 probability of being sampled into H.

2. Since each F-light edge in H is also in F and F includes no edges not in H, the chance that an F-light is in F is also 1/2.

3. For edge e with weight heavier than the lightest of F is never F-light since there would be cycle with e as heaviest edge.

4. Thus, the heaviest F-light edge is always in F. Given eF=k, eG' is the number trials before k successes (selected into H), and it forms a negative binomial distribution.

Given vG*= n, F = MST(H) where H G*

F

1

23

45

6 78

11

9

10

12 13

G*

G'

G

H

original problem

Borůvka

subproblem subproblem

sample

F-light

Analysis – Average Case

F

1

23

45

6 78

11

9

10

12 13

5. For eF = k, eG' is of negative binomial distribution with parameter 1/2 and k. Thus E[eG'] = k/(½) = 2k.

6. Summing all cases, we get

1n

0kF'GF'G ke|eE)ke(PeE

n2

n2)ke(P

k2)ke(P

1n

0kF

1n

0kF

Given vG*= n, F = MST(H) where H G*

Analysis – Average Case

1. For all right subproblems, expected sum of edges is at most

E(e) 2(n/4)

E(e) 2(n/42)

E(e) 2(n/43)

E(e) 2(n/44)E(e) 2(n/45)... ...

e=m

2. For each left path, the expected total number of edges is twice of the leading subproblem, which is root or right child. So the overall expected value is at most 2(m + n).

3. Since running time is proportional to overall edge number, so its expected value is O(m) = O(m + n).

n24n2 )1d(

1dd

Analysis – Probability of Linearity

Chernoff Bound:

Given xi as i.d.d. random variables and 0< i n, and X is the sum of all xi, for t > 0, we have

n

1i

tXAt ieEeAXPr

Thus, the probability that less than s successes (each with cance p) within k trail is

21

21)s(Ω

ktst

k

1i

tXst

p and t for ,e

)pe(e

eEesXPr i

G*

G'

G

H

original problem

Borůvka

subproblem subproblem

sample

F-light

Analysis – Probability of Linearity

Given a path with leading problem G, eG = k

• For each edge in G, it has 1/2 less chance to be kept in next subproblem. and each edge-keep contributes 1 to the total edge number. The path ends when the k-th edge-move occurs.

• The probability there are 3k more total edges is probability there are k less edge-remove in k+3k trail. According to Chernoff bound, the probability is exp(-(k)).

G*

G'

G

H

original problem

Borůvka

subproblem subproblem

sample

F-light

Analysis – Probability of Linearity

1. Given vG* = n'. For each edge in G', it has 1/2 chance to be in F. Since eF = n' – 1, the probability that eG' > 3n' is probability there are n' - 1 less F edge in 3k trail. According to Chernoff bound, the probability is exp(-(n')).

2. There is at most n/2 total vertices in all G*. If we take all the trail as a whole, the probability that there are more than 3n/2 edges in all right subproblem is exp(-(n)).

Analysis – Probability of Linearity

Combined with previous two analysis, there is at least probability as below that total edges never exceeds 3(m+3n/2), where is the set of all right problems:

)m(Ω

Γg

)v(Ω)m(Ω)n(Ω e1e1e1e1 g

Thus, the probability that time complexity is O(m) is1 – exp((m)).