randomized algorithms cs648
DESCRIPTION
Randomized Algorithms CS648. Lecture 17 Miscellaneous applications of Backward analysis. Minimum spanning tree. Minimum spanning tree. 16. d. 17. 9. v. x. 4. 2. 5. h. 19. 6. 11. y. 3. 3. 22. c. 12. a. 18. 15. 1. 10. u. b. 7. 13. Minimum spanning tree. 16. d. 17. - PowerPoint PPT PresentationTRANSCRIPT
Randomized AlgorithmsCS648
Lecture 17Miscellaneous applications of Backward analysis
1
MINIMUM SPANNING TREE
2
Minimum spanning tree
3
b
ac
d
h x
y
u
v
18
7
1
19
22
10
312
3
15
11
5
16
17
13
4 2
9
6
Minimum spanning tree
Algorithms: • Prim’s algorithm• Kruskal’s algorithm• Boruvka’s algorithm
4
b
ac
d
h x
y
u
v
18
7
1
19
22
10
312
3
15
11
5
16
17
13
4 2
9
6
Less known but it is the first algorithm for MST
Minimum spanning tree
: undirected graph with weights on edges, .
Deterministic algorithms:Prim’s algorithm 1. O(( + ) log ) using Binary heap2. O( + log ) using Fibonacci heapBest deterministic algorithm: O( + ) bound
– Too complicated to design and analyze– Fails to beat Prim’s algorithm using Binary heap
5
Minimum spanning tree
When finding an efficient solution of a problem appears hard, one should strive to design an efficient verification algorithm.
MST verification algorithm: [King, 1990]Given a graph and a spanning tree , it takes O( + ) time todetermine if is MST of .
Interestingly, no deterministic algorithm for MST could use this algorithm to achieve O( + ) time.
6
Minimum spanning tree
: undirected graph with weights on edges, .
Randomized algorithm:Karger-Klein-Tarjan’s algorithm [1995]1. Las Vegas algorithm2. O( + ) expected timeThis algorithm uses • Random sampling• MST verification algorithm• Boruvka’s algorithm• Elementary data structure
7
Minimum spanning tree
: undirected graph with weights on edges, .
Randomized algorithm:Karger-Klein-Tarjan’s algorithm [1994]1. Las Vegas algorithm2. O( + ) expected time
• Random sampling :How close is MST of a random sample of edges to MST of original graph ?
The notion of closeness is formalized in the following slide.
8
Light Edge
Definition: Let . An edge is said to be light with respect to if
Question: If and ||= , how many edges from are light with respect to on expectation ?Answer: ??
9
b
ac
d
h x
y
u
v
18
7
1
19
22
10
312
3
15
11
5
16
17
13
4 2
9
6 31
MST() MST()
¿𝒏𝒌 (𝒎−𝒌)
USING BACKWARD ANALYSIS FORMISCELLANEOUS APPLICATIONS
10
PROBLEM 1SMALLEST ENCLOSING CIRCLE
11
Smallest Enclosing Circle
12
Smallest Enclosing Circle
Question: Suppose we sample points randomly uniformly from a set of points, what is the expected number of points that remain outside the smallest circle enclosing the sample?
13
For = , the answer is
PROBLEM 2SMALLEST LENGTH INTERVAL
14
0 1
Sampling points from a unit interval
Question: Suppose we select points from interval [0,1], what is expected length of the smallest sub-interval ?• for , it is ?? • for , it is ??
• General solution : ??
This bound can be derived using two methods.• One method is based on establishing a relationship between uniform
distribution and exponential distribution.• Second method (for nearly same asymptotic bound) using Backward
analysis.
𝟏
(𝒌+𝟏 )𝟐
PROBLEM 3MINIMUM SPANNING TREE
16
Light Edge
Definition: Let . An edge is said to be light with respect to if
Question: If and ||= , how many edges from are light with respect to on expectation ?
17
b
ac
d
h x
y
u
v
18
7
1
19
22
10
312
3
15
11
5
16
17
13
4 2
9
6 31
MST() MST()
USING BACKWARD ANALYSIS FORTHE 3 PROBLEMS :
A GENERAL FRAMEWORK
18
A General framework
Let be the desired random variable in any of these problems/random experiment.
• Step 1: Define an event related to the random variable .
• Step 2: Calculate probability of event using standard method based on definition. (This establishes a relationship between )
• Step 3: Express the underlying random experiment as a Randomized incremental construction and calculate the probability of the event using Backward analysis.
• Step 4: Equate the expressions from Steps 1 and 2 to calculate E[].
19
PROBLEM 3MINIMUM SPANNING TREE
20
A BETTER UNDERSTANDING OF LIGHT EDGES
21
Minimum spanning tree
22
b
ac
d
h x
y
u
v
18
7
1
19
22
10
342
3
15
11
5
16
17
13
4 2
19
6
x
b
ac
d
h
y
u
v
181
19
22
10 15
11
5
16
4
31
31
Random sampling
(𝑽 ,𝑬 )
(𝑽 ,𝑹)
Minimum spanning tree
23
b
ac
d
h x
y
u
v
18
7
1
19
22
10
342
3
15
11
5
16
17
13
4 2
19
6
x
b
ac
d
h
y
u
v
1
19
10 15
11
5
16
4
31
18
2231
MST()
(𝑽 ,𝑬 )
(𝑽 ,𝑹)
Minimum spanning tree
24
b
ac
d
h x
y
u
v
18
7
1
19
22
10
342
3
15
11
5
16
17
23
4 2
19
6
b
ac
d
h x
y
u
v
1
19
10 15
11
5
16
4
7
342
3
17
23
2
19
6
31
MST()
(𝑽 ,𝑬 )
(𝑽 ,𝑹)𝑬 ¿
Minimum spanning tree
25
b
ac
d
h x
y
u
v
18
7
1
19
22
10
342
3
15
11
5
16
17
23
4 2
19
6
b
ac
d
h x
y
u
v
7
1
19
10
342
3
15
11
5
16
17
23
4 2
19
6
31
MST()
(𝑽 ,𝑬 )
(𝑽 ,𝑹)𝑬 ¿
Light
First useful insight
Lemma1: An edge is light with respect to if and only if belongs to MST().
26
Minimum spanning tree
27
b
ac
d
h x
y
u
v
18
7
1
19
22
10
342
3
15
11
5
16
17
23
4 2
19
6
b
ac
d
h x
y
u
v
7
1
19
10
342
3
15
11
5
16
17
23
4 2
19
6
31
MST()
(𝑽 ,𝑬 )
(𝑽 ,𝑹)𝑬 ¿
Light heavy
Minimum spanning tree
28
b
ac
d
h x
y
u
v
7
1
3 3
54 2
6
b
ac
d
h x
y
u
v
7
1
19
10
3 3
15
11
5
16
17
4 2
42
23
19
6
MST()
(𝑽 ,𝑬 )
(𝑽 ,𝑹)𝑬 ¿
Light heavy
MST()
Is there any relationship among MST(), MST()
and Light edges from ?
Second useful insight
Lemma2: Let • and • be the set of all edges from that are light with respect to . Then
MST() = MST()
This lemma is used in the design of randomized algorithm for MST as follows (just a sketch):• Compute MST of a sample of edges (recursively). Let it be T’.• There will be expected edges light edges among all unsampled edges.• Recursively compute MST of T’ edges which are less than on expectation.
29
Light Edge
Definition: Let . An edge is said to be light with respect to if
Question: If and ||= , how many edges from are light with respect to on expectation ?
30
b
ac
d
h x
y
u
v
18
7
1
19
22
10
312
3
15
11
5
16
17
13
4 2
9
6 31
MST() MST()
We shall answer the above question using the Generic framework. But before that, we need to get a better understanding of the
corresponding random variable.
31
𝒌
𝑹
𝑬 ¿MST()
32
𝒌
𝑹
𝑬 ¿MST()
Light
33
𝒌
𝑹
𝑬 ¿Light heavy
MST()
: random variable for the number of light edges in when is a random sample of edges.
: set of all subsets of of size . : number of light edges in when . = ??
34
𝟏¿𝑺∨¿ ∑
𝒂∈𝑺𝒇 (𝒂 ) ¿
Can you express in terms of and only ?
Step 1
Question: Let be a uniformly random sample of edges from .What is the prob. that an edge selected randomly from is a light edge ?
35
Two methods to find
Step 2Calculating using definition
36
Step 2 Calculating using definition
37
𝒌
𝒂
𝑬 ¿MST()
Light heavy
Light edges=
Step 2Calculating using definition
: set of all subsets of of size .The probability is equal to
38
𝒇 (𝒂)𝒎−𝒌
𝟏¿𝑺∨¿¿
Step 3
Expressing the entire experiment as Randomized Incremental Construction
A slight difficulty in this process is the following:• The underlying experiment talks about random sample from a set.• But RIC involves analyzing a random permutation of a set of elements. Question: What is relation between random sample from a set and a random permutation of the set ?
Spend some time on this question before proceeding further.
39
random sample and random permutation
Observation: is indeed a uniformly random sample of
40
Random permutation of
𝒓 𝒎−𝒓𝑨 𝑬 ¿
Step 3
The underlying random experiment as Randomized Incremental Construction: • Permute the edges randomly uniformly.• Find the probability that th edge is light relative to the first edges.
Question: Can you now calculate probability ?
Spend some time on this question before proceeding further.
41
Step 3
42
Random permutation of
𝒆𝒊𝒆𝟏𝒆𝟐 …
𝑬 𝒊−𝟏
Step 3
: a random variable taking value 1 if is a light edge with respect to MST().
43
Random permutation of
𝒆𝒊𝒆𝟏𝒆𝟐 …
𝑬 𝒊−𝟏 𝑬 {𝑬 ¿𝒊−𝟏
Step 3
: a random variable taking value 1 if is a light edge with respect to MST().
Question: What is relation between and ’s?Answer: ??
44
Random permutation of
𝒆𝒊𝒆𝟏𝒆𝟐 …
𝑬 𝒊−𝟏 𝑬 {𝑬 ¿𝒊−𝟏
𝒑=𝐏¿¿
Calculating ).
: set of all subsets of of size . ) =
depends upon
45
Forward analysis
𝒆𝒊𝒆𝟏𝒆𝟐 …
𝑬 𝒊−𝟏
MST()
Random permutation of
Calculating ).
: set of all subsets of of size . )=
46
Backward analysis
𝒆𝒊𝒆𝟏𝒆𝟐 …
𝑬 𝒊
Random permutation of
𝐏¿ ¿
= ??
??
47
Backward analysis
𝒆𝒊𝒆𝟏𝒆𝟐 …
𝑬 𝒊
MST()
¿MST (𝒂)∨¿𝒊 ¿
Random permutation of
Use Lemma 2.
𝐏 ( 𝒊 thedge𝐛𝐞𝐥𝐨𝐧𝐠𝐬 ¿MST (𝒂))
Calculating )
: set of all subsets of of size . )=
48
Backward analysis
𝒆𝒊𝒆𝟏𝒆𝟐 …
𝑬 𝒊
Random permutation of
Combining the two methods for calculating
Using method 1:
Using method 2:
)
Hence:
49
Theorem: If we sample edges uniformly randomly from an undirected graph on vertices and edges, the number of light edges among the unsampled edges will be less than on expectation.
50