# randomized randomized algorithms 2 randomized algorithms 3 â€¢so far we dealt with deterministic...

Post on 15-Nov-2019

0 views

Embed Size (px)

TRANSCRIPT

Randomized algorithms Inge Li Gørtz

�1

• Today

• What are randomized algorithms?

• Properties of randomized algorithms

• Three examples:

• Median/Select.

• Quick-sort

• Closest pair of points

Randomized algorithms

�2

Randomized Algorithms

�3

• So far we dealt with deterministic algorithms:

• Giving the same input to the algorithm repeatedly results in:

• The same running time.

• The same output.

• Randomized algorithm:

• Can make random choices (using a random number generator)

Randomized Algorithms

�4

• A randomized algorithm has access to a random number generator rand(a, b), where a, b are integers, a < b.

• rand(a, b) returns a number x ∈ {a, (a + 1), . . . , (b − 1), b}.

• Each of the b − a + 1 numbers is equally likely.

• Calling rand(a, b) again results in a new, possibly different number.

• rand(a, b) “has no memory”. The current number does not depend on the results of previous calls.

• Statistically speaking: rand(a, b) generates numbers independently according to uniform distribution.

Random number generator

�5

• The algorithm can make random decisions using rand

Randomized algorithms

Running while (rand(0,1) = 0) do

print(“running”); end while print(“stopped”);

Julefrokost while (rand(1,4) = rand(1,4)) do have another Christmas beer;

end while go home;

�6

• A randomized algorithm might never stop.

• Example: Find the position of 7 in a three-element array containing the numbers 4, 3 and 7.

Properties of randomized algorithms

Find7(A[1…3]) goon := true; while (goon) do i := rand(1,3); if (A[i] = 7) then print(Bingo: 7 found at position i); goon := false; end if

end while

�7

• A randomized algorithm might find the wrong solution.

• Example: Find the position of the minimum element in a three-element array.

• If A = [5,6,4] and i = 2 and j = 1 then the algorithm will wrongly return 5.

Properties of randomized algorithms

MinOfThree(A[1…3]) i := rand(1,3); j := rand(1,3); return(min(A[i],A[j]);

�8

• Both examples are not so bad as they look.

• It is highly unlikely that Find7 runs a long time.

• It is more likely that MinOfThree returns a correct result than a wrong one.

Properties of randomized algorithms

�9

Analysis of Randomized Algorithms

�10

• Running time: O(1).

• What is the chance that the answer of MinOfThree is correct?

• 9 possible pairs all with probability 1/9:

• (1,1), (1,2), (1,3), (2,1), (2,2), (2,3), (3,1), (3,2), (3,3)

• Assume wlog that A[1] is the minimum:

• 5 of the 9 pairs contain the index 1 and give the correct minimum.

• P[correct] = 5/9 > 1/2.

Analysis of MinOfThree MinOfThree(A[1…3])

i := rand(1,3); j := rand(1,3); return(min(A[i],A[j]);

�11

• Running MinOfThree once gives the correct answer with probaility 5/9 and the incorrect one with probability 4/9.

• Idea: Run MinOfThree many times and pick the smallest value returned.

• The probability that MinOfThree fails every time in k runs is:

• To be 99% sure to find the minimum, choose k such that

Analysis of MinOfThree MinOfThree(A[1…3])

i := rand(1,3); j := rand(1,3); return(min(A[i],A[j]);

�12

• Running time is variable. What is the expected running time?

• Assume wlog that A[1] = 7.

• If rand(1, 3) = 1 then the while-loop is terminated.

• P[rand(1,3) = 1] = 1/3

• If rand(1, 3) ≠ 1 then the while-loop is not terminated.

• P[rand(1,3) ≠ 1] = 2/3

• The while-loop is terminated after one iteration with probability

P[rand(1,3) = 1] = 1/3.

Analysis of Find7 Find7(A[1…3])

goon := true; while (goon) do i := rand(1,3); if (A[i] = 7) then print(Bingo: 7 found at position i); goon := false; end if

end while

�13

• The while-loop stops after exactly two iterations if

• rand(1, 3) ≠ 1 in the first iteration.

• rand(1, 3) = 1 in the second iteration.

• This happens with probability 2/3 and 1/3, resp.

• The probability for both to happen is - by independence -

Analysis of Find7 Find7(A[1…3])

goon := true; while (goon) do i := rand(1,3); if (A[i] = 7) then print(Bingo: 7 found at position i); goon := false; end if

end while

�14

• The while-loop stops after exactly three iterations if

• rand(1, 3) ≠ 1 in the first two iterations.

• rand(1, 3) = 1 in the third iteration.

• This happens with probability 2/3, 2/3 and 1/3, resp.

• The probability for all three to happen is - by independence -

• The while-loop stops after exactly k iterations if

• rand(1, 3) ≠ 1 in the first k-1 iterations.

• rand(1, 3) = 1 in the kth iteration.

• This happens with probability 2/3, 2/3 and 1/3, resp.

• The probability for all three to happen is - by independence -

Analysis of Find7

�15

• We have

• The expected running time of Find7 is

• Idea: Stop Las Vegas algorithm if it runs “much longer” than the expected time and restart it.

Expected running time

�16

∞

∑ k=0

k ⋅ xk = x

(1 − x)2 for |x | < 1.

• We have seen two kinds of algorithms:

• Monte Carlo algorithms: stop after a fixed (polynomial) time and give

the correct answer with probability greater 50%.

• Las Vegas algorithms: have variable running time but always give the correct answer.

Types of randomized algorithms

�17

• Analyse the expected number of times running is printed:

• Analyse the expected number of beers you get if you follow the algorithm:

Randomized algorithms

Running while (rand(0,1) = 0) do

print(“running”); end while print(“stopped”);

Julefrokost while (rand(1,4) = rand(1,4)) do have another Christmas beer;

end while go home;

�18

Summations

∞

∑ k=0

k ⋅ xk = x

(1 − x)2 for |x | < 1.

∞

∑ k=0

xk = 1

(1 − x) for |x | < 1.

∞

∑ k=0

k ⋅ xk−1 = 1

(1 − x)2 for |x | < 1.

Median/Select

�20

• Given n numbers S = {a1, a2, …, an}.

• Median: number that is in the middle position if in sorted order.

• Select(S,k): Return the kth smallest number in S.

• Min(S) = Select(S,1), Max(S)= Select(S,n), Median = Select(S,n/2).

• Assume the numbers are distinct.

Select

Select(S, k) {

Choose a pivot s ∈ S uniformly at random.

For each element e in S if e < s put e in S’ if e > s put e in S’’

if |S’| = k-1 then return s

if |S’| ≥ k then call Select(S’, k)

if |S’| < k then call Select(S’’, k - |S’| - 1) }

�21

• Worst case running time:

• If there is at least an fraction of elements both larger and smaller than s:

• Limit number of bad pivots.

• Intuition: A fairly large fraction of elements are “well-centered” => random pivot

likely to be good.

Select Select(S, k) {

Choose a pivot s ∈ S uniformly at random.

For each element e in S if e < s put e in S’ if e > s put e in S’’

if |S’| = k-1 then return s

if |S’| ≥ k then call Select(S’, k)

if |S’| < k then call Select(S’’, k - |S’| - 1) }

T (n) = cn+ c(n� 1) + c(n� 2) + · · · = ⇥(n2). "

T (n) = cn+ (1� ")cn+ (1� ")2cn+ · · · =

� 1 + (1� ") + (1� ")2 + · · ·

� cn

cn/".

�22

• Phase j: Size of set at most and at least .

• Element is central if at least a quarter of the elements in the current call are smaller

and at least a quarter are larger.

• At least half the elements are central.

• Pivot central => size of set shrinks with by at least a factor 3/4 => current phase

ends.

• Pr[s is central] = 1/2.

• Expected number of iterations before a central pivot is found = 2 =>

expected number of iterations in phase j at most 2.

• X: random variable equal to number of steps taken by algorithm.

• Xj: expected number of steps in phase j.

• X = X1+ X2 + .…

• Number of steps in one iteration in phase j is at most .

• E[Xj] = .

• Expected running time:

Select n(3/4)j n(3/4)j+1

E[X] = X

j

E[Xj ] X

j

2cn

✓ 3

4

◆j = 2cn

X

j

✓ 3

4

◆j 8cn.

2cn(3/4)j cn(3/4)j

�23

Quicksort

�24

• Given n numbers S = {a1, a2, …, an} return the sorted list.

• Assume the numbers are dis

Recommended