cse 202: design and analysis of algorithms · cse 202: design and analysis of algorithms lecture 14...

CSE 202: Design and Analysis of Algorithms

Lecture 14

Instructor: Kamalika Chaudhuri

Algorithm Design Paradigms

• Exhaustive Search

• Greedy Algorithms: Build a solution incrementally piece by piece

• Divide and Conquer: Divide into parts, solve each part, combine results

• Dynamic Programming: Divide into subtasks, perform subtask by size. Combine smaller subtasks to larger ones

• Hill-climbing: Start with a solution, improve it

• Randomized Algorithms: Algorithm can make random choices

Randomized Algorithms

• Algorithm can make random decisions

• Why randomized algorithms? Simple and efficient

• Examples: Symmetry-breaking, graph algorithms, quicksort, hashing, load balancing, cryptography, etc

Contention Resolution

Given n processors and a resource which they all wish to access, design a symmetry breaking access protocol

Restriction: Processors can’t communicate Simultaneous access blocks the resource

Processors

1

2

3

4

5Object

Need symmetry-breaking!




Processors

1

2

3

4

5Object

Need symmetry-breaking!

Protocol: At time t, each processor accesses resource w.p. p = 1/n





Goals:- Low contention - Low waiting time for all processors






S(i, t) = Event that processor i succeeds at time t = At time t, only processor i accesses






Pr[S(i, t)] = p x (1 - p)n-1

processor iaccesses

no other processors access

= (1/n) x (1 - 1/n)n-1







Independent Events A, B:Pr(A and B) = Pr(A) x Pr(B)

Pr[S(i, t)] = p x (1 - p)n-1

processor iaccesses


= (1/n) x (1 - 1/n)n-1






Pr[S(i, t)] = p x (1 - p)n-1

processor iaccesses


= (1/n) x (1 - 1/n)n-1






Pr[S(i, t)] = p x (1 - p)n-1

processor iaccesses


= (1/n) x (1 - 1/n)n-1


2 4 6 8 10 12 14 16 18 200.25

0.3

0.35

0.4

0.45

0.5

Fact: As n increases,(1) (1 - 1/n)n grows monotonically up to 1/e (2) (1 - 1/n)n-1 decreases monotonically down to 1/e





Pr[S(i, t)] = p x (1 - p)n-1

processor iaccesses


= (1/n) x (1 - 1/n)n-1


>= 1/en

<= 1/2n

Fact: As n increases,(1) (1 - 1/n)n grows monotonically up to 1/e (2) (1 - 1/n)n-1 decreases monotonically down to 1/e






Pr[S(i, t)] = p x (1 - p)n-1

processor iaccesses


= (1/n) x (1 - 1/n)n-1

>= (1/en)

<= (1/2n)






F(i, t) = Event that processor i has not succeeded after t steps

Pr[S(i, t)] = p x (1 - p)n-1

processor iaccesses


= (1/n) x (1 - 1/n)n-1

>= (1/en)

<= (1/2n)







Pr[F(i, t)] = (1 - S(i, 1))(1 - S(i, 2))...(1 - S(i, t)) <= (1 - 1/en)t

processor ifails at time 1

Pr[S(i, t)] = p x (1 - p)n-1

processor iaccesses


= (1/n) x (1 - 1/n)n-1

>= (1/en)

<= (1/2n)







Pr[F(i, t)] = (1 - Pr[S(i, 1)])(1 - Pr[S(i, 2)])...(1 - Pr[S(i, t)]) <= (1 - 1/en)t

processor ifails at time 1

Picking t = en, Pr[F(i, t)] <= 1/e Picking t = c. en. ln n, Pr[F(i, t)] <= n-c

Pr[S(i, t)] = p x (1 - p)n-1

processor iaccesses


= (1/n) x (1 - 1/n)n-1

>= (1/en)

<= (1/2n)





Properties:


Picking t = en, Pr[F(i, t)] <= 1/e

Picking t = c. en. ln n, Pr[F(i, t)] <= n-c





Properties:




Fact: W.p. 1 - 1/n, all the processors succeed within 2en. ln n rounds





Properties:





For t = 2en. ln n, Pr[F(i,t)] = n-2 for a fixed i





Properties:





For t = 2en. ln n, Pr[F(i,t)] = n-2 for a fixed iUnion Bound: Any A, BPr[A U B] <= Pr[A] + Pr[B]





Properties:





For t = 2en. ln n, Pr[F(i,t)] = n-2 for a fixed iUnion Bound: Any A, BPr[A U B] <= Pr[A] + Pr[B]

Pr[∪ni=1F (i, t)] ≤

n�

i=1

Pr[F (i, t)] ≤ n · n−2 ≤ 1/n

Therefore,

Summary: Contention Resolution



Protocol: At time t, each processor accesses resource w.p. p = 1/nF(i, t) = Event that processor i has not succeeded after t steps

Facts:

Facts we learnt:

Independent Events A, B: Pr(A and B) = Pr(A) x Pr(B)

As n increases,(1) (1 - 1/n)n converges monotonically up to 1/e (2) (1 - 1/n)n-1 converges monotonically down to 1/e


Wp 1 - 1/n, all processors succeed within 2en ln n rounds

Union Bound: For any two events A and B, Pr[A U B] <= Pr[A] + Pr[B]


• Contention Resolution

• Some Facts about Random Variables

Expectation

Given discrete random variable X, which takes m values xi w.p. pi, the expectation E[X] is defined as:

E[X] =m�

i=1

xi · Pr[X = xi] =m�

i=1

xipi

Expectation


Examples:

1. Let X = 1 if a fair coin toss comes up heads, 0 ow. What is E[X]?

E[X] =m�

i=1


i=1

xipi

Expectation


Examples:


2. We are tossing a coin with head probability p, tail probability 1 - p. Let X = #independent flips until first head. What is E[X]?

E[X] =m�

i=1


i=1

xipi

Expectation


Examples:


2. We are tossing a coin with head probability p, tail probability 1 - p. Let X = #independent flips until first head. What is E[X]?

Pr[X = j] = p x (1 - p)j - 1

first j-1 tailshead on toss j

E[X] =m�

i=1


i=1

xipi

E[X] =∞�

j=1

j · p(1− p)j−1 =p

1− p

∞�

j=0

j(1− p)j =p

1− p· 1− p

p2=

1

p

Expectation


E[X] =m�

i=1


i=1

xipi

Linearity of Expectation: E[X + Y] = E[X] + E[Y]

Expectation


E[X] =m�

i=1


i=1

xipi


Example: Guessing a card

Shuffle n cards, then turn them over one by one. Guess what the next card is.How many guesses are correct on expectation?

Expectation


E[X] =m�

i=1


i=1

xipi




Let Xi = 1 if guess i is correct, 0 otherwise

Expectation


E[X] =m�

i=1


i=1

xipi





Pr[Xi = 1] =1

n− i+ 1

Expectation


E[X] =m�

i=1


i=1

xipi





Pr[Xi = 1] =1

n− i+ 1E[Xi] =

1

n− i+ 1

Expectation


E[X] =m�

i=1


i=1

xipi





Pr[Xi = 1] =1

n− i+ 1E[Xi] =

1

n− i+ 1

Expected # of correct guesses = E[n�

i=1

Xi] =n�

i=1

E[Xi] =n�

i=1

1

i= Θ(log n)

Expectation


E[X] =m�

i=1


i=1

xipi





Pr[Xi = 1] =1

n− i+ 1E[Xi] =

1

n− i+ 1

Expected # of correct guesses = E[n�

i=1

Xi] =n�

i=1

E[Xi] =n�

i=1

1

i= Θ(log n)

What if we insert the selected card into the pile randomly and pull another ?

Expectation


E[X] =m�

i=1


i=1

xipi


Example: Coupon Collector’s Problem

Balls tossed randomly into n bins. How many balls in expectation before each bin has a ball?

Expectation


E[X] =m�

i=1


i=1

xipi




Let Xj = time spent when there are exactly j non-empty bins (Phase j)

Expectation


E[X] =m�

i=1


i=1

xipi





Let X = total #steps = X1 + X2 + ... + Xn-1

Expectation


E[X] =m�

i=1


i=1

xipi






We move from phase j to j+1 when a ball hits one of n-j bins, so w.p. (n-j) /n

Expectation


E[X] =m�

i=1


i=1

xipi







Therefore, E[Xj] = n/(n - j) [From previous slide on expected waiting times]

Expectation


E[X] =m�

i=1


i=1

xipi







Therefore, E[Xj] = n/(n - j) [From previous slide on expected waiting times]

E[X] = E[X1 + .. + Xn-1] = E[X1] + .. + E[Xn-1] = n + n/2 + .. + n/(n-1) = Θ(n log n)

Expectation


E[X] =m�

i=1


i=1

xipi


Example: Birthday Paradox

m balls tossed randomly into n bins. What is the expected #collisions?

Expectation


E[X] =m�

i=1


i=1

xipi




For 1 <= i < j <= m, let Xij = 1 if balls i and j land in the same bin, 0 otherwise

Expectation


E[X] =m�

i=1


i=1

xipi





Pr[ Xij = 1 ] = 1/n, so E[Xij] = 1/n

Expectation


E[X] =m�

i=1


i=1

xipi





Pr[ Xij = 1 ] = 1/n, so E[Xij] = 1/n

So, expected number of collisions from tossing m balls = �

i,j

E[Xij ] =

�m2

�

n=

m(m− 1)

2n

So when m < sqrt(2n), expected #collisions < 1; otherwise, it’s more

Summary: Expectation and Linearity of Expectation


E[X] =m�

i=1


i=1

xipi


Variance

Given a random variable X, its variance Var[X] is defined as: Var(X) = E[(X - E[X])2]

Variance of a random variable measures its “spread”

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

20

40

60

80

100

120

Student Version of MATLAB

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

100

200

300

400

500

600


High Variance Distribution Low Variance Distribution

Variance


Examples:

1. Let X = 1 if a fair coin toss comes up heads, 0 otherwise. What is Var(X)?


0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

20

40

60

80

100

120


0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

100

200

300

400

500

600



Variance


Examples:

1. Let X = 1 if a fair coin toss comes up heads, 0 otherwise. What is Var(X)?

2. Let X = outcome of a fair dice throw. What is Var(X)?


0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

20

40

60

80

100

120


0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

100

200

300

400

500

600



Variance


Properties of Variance:

1. Var(X) = E[X2] - (E[X])2

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

20

40

60

80

100

120


0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

100

200

300

400

500

600




Variance



1. Var(X) = E[X2] - (E[X])2

2. If X and Y are independent random variables, then, Var(X + Y) = Var(X) + Var(Y)

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

20

40

60

80

100

120


0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

100

200

300

400

500

600




Variance



1. Var(X) = E[X2] - (E[X])2

2. If X and Y are independent random variables, then, Var(X + Y) = Var(X) + Var(Y)

3. For any constants a and b, Var(aX + b) = a2 Var(X)

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

20

40

60

80

100

120


0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

100

200

300

400

500

600





• Contention Resolution

• Some Facts about Random Variables

• Global Minimum Cut Algorithm

Global Min-Cut

Example:

G G

A

V - A

Note: A global min-cut is not the same as an s-t min cut

How can we find a global min-cut using n - 1 max-flows?

We can do better for the global min-cut

Problem: Given undirected, unweighted graph G = (V, E), find a cut (A, V - A) which has the minimum number of edges across it

cse 202: design and analysis of algorithms · cse 202: design and analysis of algorithms lecture 14...

Documents