The Best Algorithms are Randomized Algorithms
N. HarveyC&O Dept
WHY DO WE RANDOMIZE?
Reasons for Randomness
• Fooling Adversaries• Symmetry Breaking• Searching a Haystack• Sampling huge sets• ...
Picking Passwords
• Encryption keys are randomly chosen
The Hallway Dance
• Randomly go left or right• After a few twists you won’t collide
(usually!)
QuickSort17 5 23 9 12 4 19 10
• If I knew an approximate median, I could partition into 2 groups & recurse
• A random element is probably close to the median
• Like “searching for hay in a haystack”
175 239 12 4 19 10
Reasons for Randomness
• Fooling Adversaries
• Symmetry Breaking
• Searching a Haystack
• ...
Another Adversary Example
• Linux 2.4.20 denial-of-service attack• Kernel uses a hash table to cache info about IP
traffic flows• Adversary sends many IP packets that all have
same hash value• Hash table becomes linked list; Kernel very slow• Solution: Randomized hash function
IP
Another Symmetry Breaking Example• Ethernet Media Access Control• Try to send packet• If collision detected–Wait for a random delay–Retry sending packet
http://picasaweb.google.com/tomatobasil
Another Haystack Example
• Problem: Find a point (x,y) that is not a zero of f• If f0 then, for randomly chosen (x,y),
Pr[ f(x,y)=0 ] = 0• Even over finite fields, Pr[ f(x,y)=0 ] is small
• Consider the polynomial f(x,y)=(x-y2)(x2+2y+1)
Reasons for Randomness
• Fooling Adversaries
• Symmetry Breaking
• Searching a Haystack
• ...
• Beauty: randomization often gives really cool algorithms!
Graph Basics
• A graph is a collection of vertices and edges
• Every edge joins two vertices• The degree of a vertex is # edges attached to it
Graph ConnectivityA connected graph
A disconnected graph
• Problem: How many edges do you need to remove to make a graph disconnected?
Graph Connectivity• Problem: How many edges do you need to
remove to make a graph disconnected?
• Connectivity is minimum # edges whose removal disconnects the graph
• Observation:for every vertex v, degree(v) connectivity• Because removing all edges attached to vertex v
disconnects the graph
Graph Connectivity• Problem: How many edges do you need to
remove to make a graph disconnected?
• Connectivity is minimum # edges whose removal disconnects the graph
• Useful in many applications:• How many MFCF network cables must be
cut to disrupt MC’s internet connectivity?• How many bombs can my oil pipelines
withstand?
Graph Connectivity Example
• Removing red edges disconnects graph• Definition: A cut is a set of edges of the form
{ all edges with exactly one endpoint in S }where S is some set of vertices.
• Fact: min set of edges that disconnects graph is always a cut
S
Algorithms for Graph Connectivity• Given a graph, how to compute connectivity?• Standard Approach: Network Flow Theory– Ford-Fulkerson Algorithm computes an st min cut
(minimum # edges to disconnect vertices s & t)– Compute this value for all vertices s and t, then
take the minimum– Get min # edges to disconnect any two vertices
• CO 355 Approach:– Compute st min cuts by ellipsoid method instead
• Next: An amazing randomized approach
Algorithm Overview
• Input: A haystack• Output: A needle (maybe)
• While haystack not too small–Pick a random handful– Throw it away
• End While• Output whatever is left
Edge Contraction
uv
• Key operation: contracting an edge uv• Delete the edge uv• Combine u & v into a single vertex w• Any edge with endpoint u or v now ends at w
w
Edge Contraction
u v
• Key operation: contracting an edge uv• Delete the edge uv• Combine u & v into a single vertex w• Any edge with endpoint u or v now ends at w• This can create “parallel” edges
w
Randomized Algorithm for Connectivity
• Input: A graph• Output: Minimum cut (maybe)
• While graph has 2 vertices “Not too small”–Pick an edge uv at random “Random Handful”–Contract it “Throw it away”
• End While• Output remaining edges
Graph Connectivity Example
• We were lucky: Remaining edges are the min cut• How lucky must we be to find the min cut?• Theorem (Karger ‘93): The probability that this
algorithm finds a min cut is 1/(# vertices)2.
But does the algorithm work?• How lucky must we be to find the min cut?• Theorem (Karger ‘93): The probability that this
algorithm finds a min cut is 1/n2. (n = # vertices)
• Fairly low success probability. Is this useful?• Yes! Run the algorithm n2 times.
Pr[ fails to find min cut ] (1-1/n2)n2 1/e• So algorithm succeeds with probability > 1/2
• Fix some min cut. Say it has k edges.• If algorithm doesn’t contract any edge in this cut,
then the algorithm outputs this cut– When contracting edge uv, both u & v are on same side of cut
• So what is probability that this happens?
• While graph has 2 vertices “Not too small”– Pick an edge uv at random “Random Handful”– Contract it “Throw it away”
• End While• Output remaining edges
Proof of Main Theorem
• Initially there are n vertices.• Claim 1: # edges in min cut=k
every vertex has degree k total # edges nk/2
• Pr[random edge is in min cut] = # edges in min cut / total # edges k / (nk/2) = 2/n
• Now there are n-1 vertices.• Claim 2:
min cut in remaining graph is k• Why? Every cut in remaining graph is also a
cut in original graph.• So, Pr[ random edge is in min cut ] 2/(n-1)
• In general, when there are i vertices left Pr[ random edge is in min cut ] 2/i
• So Pr[ alg never contracts an edge in min cut ]= Q n
i=3 Pr[random edge not in cut when i vertices left]¸ Q n
i=3(1¡ 2=i)= ¡ n¡ 2
n¢¡ n¡ 3
n¡ 1¢¡ n¡ 4
n¡ 2¢¡ n¡ 5
n¡ 3¢¢¢¢¡ 2
4¢¡ 1
3¢= ¡n
2¢¡ 1
= Q ni=3
i ¡ 2i
• Input: Graph G• Output: min cut, with probability 1/2
• For i=1,..,n2
• Start from G• While graph has 2 vertices– Pick an edge uv at random– Contract it
• End While• Let Ei = { remaining edges }
• End For• Output smallest Ei
Final Algorithm
• Running time: O(m¢n2) (m = # edges, n = # vertices)
• With more beautiful ideas, improves to O(n2)
How Many Min Cuts?• Our analysis: for any particular min cut,
Pr[ algorithm finds that min cut ] 1/n2
• So suppose C1, C2, ..., Ct are all the min cuts
• 1/n2 fraction of the time the algorithm finds C1
• 1/n2 fraction of the time the algorithm finds C2
• ...• 1/n2 fraction of the time the algorithm finds Ct
• This is only possible if t · n2
• Corollary: Any connected graph on n verticeshas · n2 min cuts. (Actually, min cuts)
· ¡n2¢
• An n-cycle has exactly min cuts!¡n
2¢
How Many Min Cuts?• Corollary: Any connected graph on n vertices
has min cuts.• Is this optimal?
· ¡n2¢
How Many Approximate Min Cuts?• A similar analysis gives a nice generalization:
• Theorem (Karger-Stein ‘96):Let G be a graph with min cut size k.Then # { cuts with k edges } n2.
• No other proof of this theorem is known!
Books
Motwani-RaghavanMitzenmacher-Upfal Alon-Spencer
ClassesCS 466, CS 761 (still active?), C&O 738
