foundations of computer science lecture 17 -...

89
Foundations of Computer Science Lecture 17 Independent Events Independence is a Powerful Assumption The Fermi Method Coincidence and the Birthday Paradox Application to Hashing Random Walks and Gambler’s Ruin

Upload: donhi

Post on 20-May-2018

213 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Foundations of Computer Science

Lecture 17

Independent EventsIndependence is a Powerful Assumption

The Fermi Method

Coincidence and the Birthday Paradox

Application to Hashing

Random Walks and Gambler’s Ruin

Page 2: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Last Time

1 New information changes a probability.

2 Conditional probability.

3 Conditional probability traps. Sampling bias, using P[A] instead of P[A | B]. Transposed conditional, using P[B | A] instead of P[A | B]. Medical testing.

4 Law of total probability. Case by case probability analysis.

Creator: Malik Magdon-Ismail Independent Events: 2 / 13 Today →

Page 3: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Today: Independent Events

1 Independence is an assumptionFermi method

Multiway independence

2 Coincidence and the birthday paradoxApplication to hashing

3 Random walk and gambler’s ruin

Creator: Malik Magdon-Ismail Independent Events: 3 / 13 Independence is an Assumption →

Page 4: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Independence is a Simplifying Assumption

Sex of first child has nothing to do with sex of second → independent.

Creator: Malik Magdon-Ismail Independent Events: 4 / 13 Definition of Independence →

Page 5: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Independence is a Simplifying Assumption

Sex of first child has nothing to do with sex of second → independent.

What about eyecolor? (Depends on genes of parent.) → not independent.

Creator: Malik Magdon-Ismail Independent Events: 4 / 13 Definition of Independence →

Page 6: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Independence is a Simplifying Assumption

Sex of first child has nothing to do with sex of second → independent.

What about eyecolor? (Depends on genes of parent.) → not independent.

Tosses of different coins have nothing to do with each other →independent.

Creator: Malik Magdon-Ismail Independent Events: 4 / 13 Definition of Independence →

Page 7: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Independence is a Simplifying Assumption

Sex of first child has nothing to do with sex of second → independent.

What about eyecolor? (Depends on genes of parent.) → not independent.

Tosses of different coins have nothing to do with each other →independent.

Cloudy and rainy days. When it rains, there must be clouds. → not independent.

Creator: Malik Magdon-Ismail Independent Events: 4 / 13 Definition of Independence →

Page 8: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Independence is a Simplifying Assumption

Sex of first child has nothing to do with sex of second → independent.

What about eyecolor? (Depends on genes of parent.) → not independent.

Tosses of different coins have nothing to do with each other →independent.

Cloudy and rainy days. When it rains, there must be clouds. → not independent.

Toss two coins.

P[Coin 1=H] = 12

P [Coin 2=H] = 12

P [Coin 1=H and Coin 2=H] = 14

Toss 100 times: Coin 1 ≈ 50H (of these)→ Coin 2 ≈ 25H (independent)

P[Coin 1=H and Coin 2=H] = 14

= 12× 1

2= P[Coin 1=H]× P[Coin 2=H].

Creator: Malik Magdon-Ismail Independent Events: 4 / 13 Definition of Independence →

Page 9: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Independence is a Simplifying Assumption

Sex of first child has nothing to do with sex of second → independent.

What about eyecolor? (Depends on genes of parent.) → not independent.

Tosses of different coins have nothing to do with each other →independent.

Cloudy and rainy days. When it rains, there must be clouds. → not independent.

Toss two coins.

P[Coin 1=H] = 12

P [Coin 2=H] = 12

P [Coin 1=H and Coin 2=H] = 14

Toss 100 times: Coin 1 ≈ 50H (of these)→ Coin 2 ≈ 25H (independent)

P[Coin 1=H and Coin 2=H] = 14

= 12× 1

2= P[Coin 1=H]× P[Coin 2=H].

P[rain and clouds] = P[rain] = 17≫ 1

35= P[rain]× P[clouds]. (not independent)

Creator: Malik Magdon-Ismail Independent Events: 4 / 13 Definition of Independence →

Page 10: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Definition of Independence

Events A and B are independent if “They have nothing to do with each other.”Knowing the outcome is in B does not change the probability that the outcome is in A.

Creator: Malik Magdon-Ismail Independent Events: 5 / 13 Fermi-Method →

Page 11: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Definition of Independence

Events A and B are independent if “They have nothing to do with each other.”Knowing the outcome is in B does not change the probability that the outcome is in A.

The events A and B are independent if

P[A and B] = P[A ∩B] = P[A]× P[B].

In general, P[A ∩ B] = P[A | B]× P[B]. Independence means that

P[A | B] = P[A].

Creator: Malik Magdon-Ismail Independent Events: 5 / 13 Fermi-Method →

Page 12: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Definition of Independence

Events A and B are independent if “They have nothing to do with each other.”Knowing the outcome is in B does not change the probability that the outcome is in A.

The events A and B are independent if

P[A and B] = P[A ∩B] = P[A]× P[B].

In general, P[A ∩ B] = P[A | B]× P[B]. Independence means that

P[A | B] = P[A].

Independence is a non-trivial assumption, and you can’t always assume it.

When you can assume independence

PROBABILITIES MULTIPLY

Creator: Malik Magdon-Ismail Independent Events: 5 / 13 Fermi-Method →

Page 13: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Fermi-Method: How Many Dateable Girls Are Out There?

Creator: Malik Magdon-Ismail Independent Events: 6 / 13 Multiway Independence →

Page 14: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Fermi-Method: How Many Dateable Girls Are Out There?

A1 = “Lives nearby”; A2 = “Right sex”; A3 = “Right age”; A4 = “Single”;

A5 = “Educated”; A6 = “Attractive”; A7 = “Finds me attractive”; A8 = “We get along”.

Creator: Malik Magdon-Ismail Independent Events: 6 / 13 Multiway Independence →

Page 15: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Fermi-Method: How Many Dateable Girls Are Out There?

A1 = “Lives nearby”; A2 = “Right sex”; A3 = “Right age”; A4 = “Single”;

A5 = “Educated”; A6 = “Attractive”; A7 = “Finds me attractive”; A8 = “We get along”.

A = A1 ∩ A2 ∩A3 ∩ A4 ∩ A5 ∩A6 ∩A7 ∩ A8 (all criteria must be met)

Creator: Malik Magdon-Ismail Independent Events: 6 / 13 Multiway Independence →

Page 16: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Fermi-Method: How Many Dateable Girls Are Out There?

A1 = “Lives nearby”; A2 = “Right sex”; A3 = “Right age”; A4 = “Single”;

A5 = “Educated”; A6 = “Attractive”; A7 = “Finds me attractive”; A8 = “We get along”.

A = A1 ∩ A2 ∩A3 ∩ A4 ∩ A5 ∩A6 ∩A7 ∩ A8 (all criteria must be met)

Independence:P[A] = P[A1 ∩ A2 ∩ A3 ∩A4 ∩ A5 ∩ A6 ∩A7 ∩A8].

Creator: Malik Magdon-Ismail Independent Events: 6 / 13 Multiway Independence →

Page 17: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Fermi-Method: How Many Dateable Girls Are Out There?

A1 = “Lives nearby”; A2 = “Right sex”; A3 = “Right age”; A4 = “Single”;

A5 = “Educated”; A6 = “Attractive”; A7 = “Finds me attractive”; A8 = “We get along”.

A = A1 ∩ A2 ∩A3 ∩ A4 ∩ A5 ∩A6 ∩A7 ∩ A8 (all criteria must be met)

Independence:P[A] = P[A1 ∩ A2 ∩ A3 ∩A4 ∩ A5 ∩ A6 ∩A7 ∩A8].

P[“Lives nearby”]number(nearby)

number(world)≈

20 million

7 billion≈

3

1000

Creator: Malik Magdon-Ismail Independent Events: 6 / 13 Multiway Independence →

Page 18: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Fermi-Method: How Many Dateable Girls Are Out There?

A1 = “Lives nearby”; A2 = “Right sex”; A3 = “Right age”; A4 = “Single”;

A5 = “Educated”; A6 = “Attractive”; A7 = “Finds me attractive”; A8 = “We get along”.

A = A1 ∩ A2 ∩A3 ∩ A4 ∩ A5 ∩A6 ∩A7 ∩ A8 (all criteria must be met)

Independence:P[A] = P[A1 ∩ A2 ∩ A3 ∩A4 ∩ A5 ∩ A6 ∩A7 ∩A8].

P[“Lives nearby”]number(nearby)

number(world)≈

20 million

7 billion≈

3

1000

P[“Right sex”] 12 (there are about 50% male and 50% female in the world)

Creator: Malik Magdon-Ismail Independent Events: 6 / 13 Multiway Independence →

Page 19: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Fermi-Method: How Many Dateable Girls Are Out There?

A1 = “Lives nearby”; A2 = “Right sex”; A3 = “Right age”; A4 = “Single”;

A5 = “Educated”; A6 = “Attractive”; A7 = “Finds me attractive”; A8 = “We get along”.

A = A1 ∩ A2 ∩A3 ∩ A4 ∩ A5 ∩A6 ∩A7 ∩ A8 (all criteria must be met)

Independence:P[A] = P[A1 ∩ A2 ∩ A3 ∩A4 ∩ A5 ∩ A6 ∩A7 ∩A8].

P[“Lives nearby”]number(nearby)

number(world)≈

20 million

7 billion≈

3

1000

P[“Right sex”] 12 (there are about 50% male and 50% female in the world)

P[“Right age”] 15100 (about 15% of people between 20 and 30)

Creator: Malik Magdon-Ismail Independent Events: 6 / 13 Multiway Independence →

Page 20: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Fermi-Method: How Many Dateable Girls Are Out There?

A1 = “Lives nearby”; A2 = “Right sex”; A3 = “Right age”; A4 = “Single”;

A5 = “Educated”; A6 = “Attractive”; A7 = “Finds me attractive”; A8 = “We get along”.

A = A1 ∩ A2 ∩A3 ∩ A4 ∩ A5 ∩A6 ∩A7 ∩ A8 (all criteria must be met)

Independence:P[A] = P[A1 ∩ A2 ∩ A3 ∩A4 ∩ A5 ∩ A6 ∩A7 ∩A8].

P[“Lives nearby”]number(nearby)

number(world)≈

20 million

7 billion≈

3

1000

P[“Right sex”] 12 (there are about 50% male and 50% female in the world)

P[“Right age”] 15100 (about 15% of people between 20 and 30)

P[“Single”] 12 (about 50% of people are single)

Creator: Malik Magdon-Ismail Independent Events: 6 / 13 Multiway Independence →

Page 21: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Fermi-Method: How Many Dateable Girls Are Out There?

A1 = “Lives nearby”; A2 = “Right sex”; A3 = “Right age”; A4 = “Single”;

A5 = “Educated”; A6 = “Attractive”; A7 = “Finds me attractive”; A8 = “We get along”.

A = A1 ∩ A2 ∩A3 ∩ A4 ∩ A5 ∩A6 ∩A7 ∩ A8 (all criteria must be met)

Independence:P[A] = P[A1 ∩ A2 ∩ A3 ∩A4 ∩ A5 ∩ A6 ∩A7 ∩A8].

P[“Lives nearby”]number(nearby)

number(world)≈

20 million

7 billion≈

3

1000

P[“Right sex”] 12 (there are about 50% male and 50% female in the world)

P[“Right age”] 15100 (about 15% of people between 20 and 30)

P[“Single”] 12 (about 50% of people are single)

P[“Educated”] 14 (about 25% in the US have a college degree)

Creator: Malik Magdon-Ismail Independent Events: 6 / 13 Multiway Independence →

Page 22: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Fermi-Method: How Many Dateable Girls Are Out There?

A1 = “Lives nearby”; A2 = “Right sex”; A3 = “Right age”; A4 = “Single”;

A5 = “Educated”; A6 = “Attractive”; A7 = “Finds me attractive”; A8 = “We get along”.

A = A1 ∩ A2 ∩A3 ∩ A4 ∩ A5 ∩A6 ∩A7 ∩ A8 (all criteria must be met)

Independence:P[A] = P[A1 ∩ A2 ∩ A3 ∩A4 ∩ A5 ∩ A6 ∩A7 ∩A8].

P[“Lives nearby”]number(nearby)

number(world)≈

20 million

7 billion≈

3

1000

P[“Right sex”] 12 (there are about 50% male and 50% female in the world)

P[“Right age”] 15100 (about 15% of people between 20 and 30)

P[“Single”] 12 (about 50% of people are single)

P[“Educated”] 14 (about 25% in the US have a college degree)

P[“Attractive”] 15 (you find 1 in 5 people attractive)

Creator: Malik Magdon-Ismail Independent Events: 6 / 13 Multiway Independence →

Page 23: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Fermi-Method: How Many Dateable Girls Are Out There?

A1 = “Lives nearby”; A2 = “Right sex”; A3 = “Right age”; A4 = “Single”;

A5 = “Educated”; A6 = “Attractive”; A7 = “Finds me attractive”; A8 = “We get along”.

A = A1 ∩ A2 ∩A3 ∩ A4 ∩ A5 ∩A6 ∩A7 ∩ A8 (all criteria must be met)

Independence:P[A] = P[A1 ∩ A2 ∩ A3 ∩A4 ∩ A5 ∩ A6 ∩A7 ∩A8].

P[“Lives nearby”]number(nearby)

number(world)≈

20 million

7 billion≈

3

1000

P[“Right sex”] 12 (there are about 50% male and 50% female in the world)

P[“Right age”] 15100 (about 15% of people between 20 and 30)

P[“Single”] 12 (about 50% of people are single)

P[“Educated”] 14 (about 25% in the US have a college degree)

P[“Attractive”] 15 (you find 1 in 5 people attractive)

P[“Finds me attractive”] 110

(you are modest)

Creator: Malik Magdon-Ismail Independent Events: 6 / 13 Multiway Independence →

Page 24: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Fermi-Method: How Many Dateable Girls Are Out There?

A1 = “Lives nearby”; A2 = “Right sex”; A3 = “Right age”; A4 = “Single”;

A5 = “Educated”; A6 = “Attractive”; A7 = “Finds me attractive”; A8 = “We get along”.

A = A1 ∩ A2 ∩A3 ∩ A4 ∩ A5 ∩A6 ∩A7 ∩ A8 (all criteria must be met)

Independence:P[A] = P[A1 ∩ A2 ∩ A3 ∩A4 ∩ A5 ∩ A6 ∩A7 ∩A8].

P[“Lives nearby”]number(nearby)

number(world)≈

20 million

7 billion≈

3

1000

P[“Right sex”] 12 (there are about 50% male and 50% female in the world)

P[“Right age”] 15100 (about 15% of people between 20 and 30)

P[“Single”] 12 (about 50% of people are single)

P[“Educated”] 14 (about 25% in the US have a college degree)

P[“Attractive”] 15 (you find 1 in 5 people attractive)

P[“Finds me attractive”] 110

(you are modest)

P[“We get along”] 116 (you get along with 1 in 4 people and assume so for her)

Creator: Malik Magdon-Ismail Independent Events: 6 / 13 Multiway Independence →

Page 25: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Fermi-Method: How Many Dateable Girls Are Out There?

A1 = “Lives nearby”; A2 = “Right sex”; A3 = “Right age”; A4 = “Single”;

A5 = “Educated”; A6 = “Attractive”; A7 = “Finds me attractive”; A8 = “We get along”.

A = A1 ∩ A2 ∩A3 ∩ A4 ∩ A5 ∩A6 ∩A7 ∩ A8 (all criteria must be met)

Independence:P[A] = P[A1 ∩ A2 ∩ A3 ∩A4 ∩ A5 ∩ A6 ∩A7 ∩A8].

P[“Lives nearby”]number(nearby)

number(world)≈

20 million

7 billion≈

3

1000

P[“Right sex”] 12 (there are about 50% male and 50% female in the world)

P[“Right age”] 15100 (about 15% of people between 20 and 30)

P[“Single”] 12 (about 50% of people are single)

P[“Educated”] 14 (about 25% in the US have a college degree)

P[“Attractive”] 15 (you find 1 in 5 people attractive)

P[“Finds me attractive”] 110

(you are modest)

P[“We get along”] 116 (you get along with 1 in 4 people and assume so for her)

P[“Dateable”] = 31000× 1

2× 15

100× 1

2× 1

4× 1

5× 1

10× 1

16× ≈ 3.5× 10−8,

1-in-30 million (or 250) dateable girls.

Creator: Malik Magdon-Ismail Independent Events: 6 / 13 Multiway Independence →

Page 26: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Multiway Independence

Ω HHH HHT HTH HTT THH THT TTH TTT

P (ω) 18

18

18

18

18

18

18

18

A1 =coins 1,2 match

A2 =coins 2,3 match

A3 =coins 1,3 match

Creator: Malik Magdon-Ismail Independent Events: 7 / 13 Coincidence and FOCS-Twins →

Page 27: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Multiway Independence

Ω HHH HHT HTH HTT THH THT TTH TTT

P (ω) 18

18

18

18

18

18

18

18

A1 =coins 1,2 match

A2 =coins 2,3 match

A3 =coins 1,3 match

P[A1] = P[A2] = P[A3] = 12.

Creator: Malik Magdon-Ismail Independent Events: 7 / 13 Coincidence and FOCS-Twins →

Page 28: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Multiway Independence

Ω HHH HHT HTH HTT THH THT TTH TTT

P (ω) 18

18

18

18

18

18

18

18

A1 =coins 1,2 match

A2 =coins 2,3 match

A3 =coins 1,3 match

P[A1] = P[A2] = P[A3] = 12.

P[A1 ∩A2] = P[A2 ∩ A3] = P[A1 ∩ A3] = 14. (independent)

Creator: Malik Magdon-Ismail Independent Events: 7 / 13 Coincidence and FOCS-Twins →

Page 29: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Multiway Independence

Ω HHH HHT HTH HTT THH THT TTH TTT

P (ω) 18

18

18

18

18

18

18

18

A1 =coins 1,2 match

A2 =coins 2,3 match

A3 =coins 1,3 match

P[A1] = P[A2] = P[A3] = 12.

P[A1 ∩A2] = P[A2 ∩ A3] = P[A1 ∩ A3] = 14. (independent)

P[A1 ∩A2 ∩ A3] = 14. (1,2) match and (2,3) match → (1,3) match.

2-way independent, not 3-way independent.

Creator: Malik Magdon-Ismail Independent Events: 7 / 13 Coincidence and FOCS-Twins →

Page 30: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Multiway Independence

Ω HHH HHT HTH HTT THH THT TTH TTT

P (ω) 18

18

18

18

18

18

18

18

A1 =coins 1,2 match

A2 =coins 2,3 match

A3 =coins 1,3 match

P[A1] = P[A2] = P[A3] = 12.

P[A1 ∩A2] = P[A2 ∩ A3] = P[A1 ∩ A3] = 14. (independent)

P[A1 ∩A2 ∩ A3] = 14. (1,2) match and (2,3) match → (1,3) match.

2-way independent, not 3-way independent.

A1, . . . , An are independent if the probability of any intersection of distinctevents is the product of the event-probabilities of those events,

P[Ai1∩ Ai2

∩ · · · ∩ Aik] = P[Ai1

] · P[Ai2] · · · P [Aik

].

Creator: Malik Magdon-Ismail Independent Events: 7 / 13 Coincidence and FOCS-Twins →

Page 31: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Coincidence: Let’s Try to Find a FOCS-Twin

Two hundred students S = s1, . . . , s200,

Birthdays are independent (no twins, triplets, . . . ) and all birthdays are equally likely.

1 2 3 4 · · · B = 366

Creator: Malik Magdon-Ismail Independent Events: 8 / 13 The Birthday Paradox→

Page 32: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Coincidence: Let’s Try to Find a FOCS-Twin

Two hundred students S = s1, . . . , s200,

Birthdays are independent (no twins, triplets, . . . ) and all birthdays are equally likely.

1 2 3 4 · · · B = 366

s1

Creator: Malik Magdon-Ismail Independent Events: 8 / 13 The Birthday Paradox→

Page 33: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Coincidence: Let’s Try to Find a FOCS-Twin

Two hundred students S = s1, . . . , s200,

Birthdays are independent (no twins, triplets, . . . ) and all birthdays are equally likely.

1 2 3 4 · · · B = 366

s1

P [s1 has no FOCS-twin] =(

B−1B

)N−1=

(

365366

)199

Creator: Malik Magdon-Ismail Independent Events: 8 / 13 The Birthday Paradox→

Page 34: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Coincidence: Let’s Try to Find a FOCS-Twin

Two hundred students S = s1, . . . , s200,

Birthdays are independent (no twins, triplets, . . . ) and all birthdays are equally likely.

1 2 3 4 · · · B = 366

s1 s2

P [s1 has no FOCS-twin] =(

B−1B

)N−1=

(

365366

)199

Creator: Malik Magdon-Ismail Independent Events: 8 / 13 The Birthday Paradox→

Page 35: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Coincidence: Let’s Try to Find a FOCS-Twin

Two hundred students S = s1, . . . , s200,

Birthdays are independent (no twins, triplets, . . . ) and all birthdays are equally likely.

1 2 3 4 · · · B = 366

s1 s2

P [s1 has no FOCS-twin] =(

B−1B

)N−1=

(

365366

)199

P [s2 has no FOCS-twin | s1 has no FOCS-twin] =(

B−2B−1

)N−2=

(

364365

)198

Creator: Malik Magdon-Ismail Independent Events: 8 / 13 The Birthday Paradox→

Page 36: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Coincidence: Let’s Try to Find a FOCS-Twin

Two hundred students S = s1, . . . , s200,

Birthdays are independent (no twins, triplets, . . . ) and all birthdays are equally likely.

1 2 3 4 · · · B = 366

s1 s2 s3

P [s1 has no FOCS-twin] =(

B−1B

)N−1=

(

365366

)199

P [s2 has no FOCS-twin | s1 has no FOCS-twin] =(

B−2B−1

)N−2=

(

364365

)198

Creator: Malik Magdon-Ismail Independent Events: 8 / 13 The Birthday Paradox→

Page 37: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Coincidence: Let’s Try to Find a FOCS-Twin

Two hundred students S = s1, . . . , s200,

Birthdays are independent (no twins, triplets, . . . ) and all birthdays are equally likely.

1 2 3 4 · · · B = 366

s1 s2 s3

P [s1 has no FOCS-twin] =(

B−1B

)N−1=

(

365366

)199

P [s2 has no FOCS-twin | s1 has no FOCS-twin] =(

B−2B−1

)N−2=

(

364365

)198

P [s3 has no FOCS-twin | s1, s2 have no FOCS-twin] =(

B−3B−2

)N−3=

(

363364

)197

Creator: Malik Magdon-Ismail Independent Events: 8 / 13 The Birthday Paradox→

Page 38: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Coincidence: Let’s Try to Find a FOCS-Twin

Two hundred students S = s1, . . . , s200,

Birthdays are independent (no twins, triplets, . . . ) and all birthdays are equally likely.

1 2 3 4 · · · B = 366

s1 s2 s3

P [s1 has no FOCS-twin] =(

B−1B

)N−1=

(

365366

)199

P [s2 has no FOCS-twin | s1 has no FOCS-twin] =(

B−2B−1

)N−2=

(

364365

)198

P [s3 has no FOCS-twin | s1, s2 have no FOCS-twin] =(

B−3B−2

)N−3=

(

363364

)197

...

P[sk has no FOCS-twin | s1, . . . , sk−1 have no FOCS-twin] =(

B−kB−k+1

)N−k=

(

366−k366−k+1

)N−k

Creator: Malik Magdon-Ismail Independent Events: 8 / 13 The Birthday Paradox→

Page 39: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Coincidence: Let’s Try to Find a FOCS-Twin

Two hundred students S = s1, . . . , s200,

Birthdays are independent (no twins, triplets, . . . ) and all birthdays are equally likely.

1 2 3 4 · · · B = 366

s1 s2 s3

P [s1 has no FOCS-twin] =(

B−1B

)N−1=

(

365366

)199

P [s2 has no FOCS-twin | s1 has no FOCS-twin] =(

B−2B−1

)N−2=

(

364365

)198

P [s3 has no FOCS-twin | s1, s2 have no FOCS-twin] =(

B−3B−2

)N−3=

(

363364

)197

...

P[sk has no FOCS-twin | s1, . . . , sk−1 have no FOCS-twin] =(

B−kB−k+1

)N−k=

(

366−k366−k+1

)N−k

P[s1, . . . , sk have no FOCS-twin] =(

365366

)199×

(

364365

)198×

(

363364

)197× · · · ×

(

366−k366−k+1

)N−k≈ 0.58

Creator: Malik Magdon-Ismail Independent Events: 8 / 13 The Birthday Paradox→

Page 40: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Coincidence: Let’s Try to Find a FOCS-Twin

Two hundred students S = s1, . . . , s200,

Birthdays are independent (no twins, triplets, . . . ) and all birthdays are equally likely.

1 2 3 4 · · · B = 366

s1 s2 s3

P [s1 has no FOCS-twin] =(

B−1B

)N−1=

(

365366

)199

P [s2 has no FOCS-twin | s1 has no FOCS-twin] =(

B−2B−1

)N−2=

(

364365

)198

P [s3 has no FOCS-twin | s1, s2 have no FOCS-twin] =(

B−3B−2

)N−3=

(

363364

)197

...

P[sk has no FOCS-twin | s1, . . . , sk−1 have no FOCS-twin] =(

B−kB−k+1

)N−k=

(

366−k366−k+1

)N−k

P[s1, . . . , sk have no FOCS-twin] =(

365366

)199×

(

364365

)198×

(

363364

)197× · · · ×

(

366−k366−k+1

)N−k≈ 0.58

Finding a FOCS-twin by the kth student with class size 200

k 1 2 3 4 5 6 7 8 9 10 23 25

chances (%) 42.0 66.3 80.4 88.6 93.3 96.1 97.7 98.7 99.2 99.5 99.999 100

Creator: Malik Magdon-Ismail Independent Events: 8 / 13 The Birthday Paradox→

Page 41: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

The Birthday Paradox

In a party of 50 people, what are the chances that two have the same birthday?

Creator: Malik Magdon-Ismail Independent Events: 9 / 13 Search and Hashing →

Page 42: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

The Birthday Paradox

In a party of 50 people, what are the chances that two have the same birthday?

Same as asking forP[s1, . . . , s50 have no FOCS-twin].

Creator: Malik Magdon-Ismail Independent Events: 9 / 13 Search and Hashing →

Page 43: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

The Birthday Paradox

In a party of 50 people, what are the chances that two have the same birthday?

Same as asking forP[s1, . . . , s50 have no FOCS-twin].

Answer:P[no social twins] =

(

365366

)49×

(

364365

)48×

(

363364

)47× · · · ×

(

315316

)0≈ 0.03.

Creator: Malik Magdon-Ismail Independent Events: 9 / 13 Search and Hashing →

Page 44: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

The Birthday Paradox

In a party of 50 people, what are the chances that two have the same birthday?

Same as asking forP[s1, . . . , s50 have no FOCS-twin].

Answer:P[no social twins] =

(

365366

)49×

(

364365

)48×

(

363364

)47× · · · ×

(

315316

)0≈ 0.03.

Chances are about 97% that two people share a birthday!

Moral: when searching for something among many options(1225 pairs of people), do not be surprised when you find it.

Creator: Malik Magdon-Ismail Independent Events: 9 / 13 Search and Hashing →

Page 45: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Search and Hashing

http://page.1 http://page.2 http://page.3

dirty apples

hurt health

health freaks

hate dirty

apples

survey: people

hate bananas

Example Queries

search(apples) = page.1, page.2

search(hate) = page.2, page.3

search(bananas) = page.3

Creator: Malik Magdon-Ismail Independent Events: 10 / 13 Hashing and FOCS-twins →

Page 46: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Search and Hashing

http://page.1 http://page.2 http://page.3

dirty apples

hurt health

health freaks

hate dirty

apples

survey: people

hate bananas

apples → page.1, page.2bananas→ page.3dirty → page.1, page.2freaks → page.2hate → page.2, page.3health → page.1, page.2hurt → page.1people → page.3survey → page.3

O(log N) search

Web-address Directory

Example Queries

search(apples) = page.1, page.2

search(hate) = page.2, page.3

search(bananas) = page.3

Creator: Malik Magdon-Ismail Independent Events: 10 / 13 Hashing and FOCS-twins →

Page 47: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Search and Hashing

http://page.1 http://page.2 http://page.3

dirty apples

hurt health

health freaks

hate dirty

apples

survey: people

hate bananas

apples → page.1, page.2bananas→ page.3dirty → page.1, page.2freaks → page.2hate → page.2, page.3health → page.1, page.2hurt → page.1people → page.3survey → page.3

O(log N) search

Web-address Directory

Example Queries

search(apples) = page.1, page.2

search(hate) = page.2, page.3

search(bananas) = page.3

Hash words into a table (array) using a hash function h(w), e.g:

h(hate) = 817 + 117 + 2017 + 517 (mod 11) = 70 bananas→ page.3

1

2 hurt→ page.1

3 people→ page.3

4 dirty→ page.1, page.2

5

6

7 freaks→ page.2

hate→ page.2, page.3

8

9 apples→ page.1, page.2

survey→ page.3

10 health→ page.1, page.2

Creator: Malik Magdon-Ismail Independent Events: 10 / 13 Hashing and FOCS-twins →

Page 48: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Search and Hashing

http://page.1 http://page.2 http://page.3

dirty apples

hurt health

health freaks

hate dirty

apples

survey: people

hate bananas

apples → page.1, page.2bananas→ page.3dirty → page.1, page.2freaks → page.2hate → page.2, page.3health → page.1, page.2hurt → page.1people → page.3survey → page.3

O(log N) search

Web-address Directory

Example Queries

search(apples) = page.1, page.2

search(hate) = page.2, page.3

search(bananas) = page.3

Hash words into a table (array) using a hash function h(w), e.g:

h(hate) = 817 + 117 + 2017 + 517 (mod 11) = 7

search(w): goto hash-table row h(w).

0 bananas→ page.3

1

2 hurt→ page.1

3 people→ page.3

4 dirty→ page.1, page.2

5

6

7 freaks→ page.2

hate→ page.2, page.3

8

9 apples→ page.1, page.2

survey→ page.3

10 health→ page.1, page.2

Creator: Malik Magdon-Ismail Independent Events: 10 / 13 Hashing and FOCS-twins →

Page 49: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Search and Hashing

http://page.1 http://page.2 http://page.3

dirty apples

hurt health

health freaks

hate dirty

apples

survey: people

hate bananas

apples → page.1, page.2bananas→ page.3dirty → page.1, page.2freaks → page.2hate → page.2, page.3health → page.1, page.2hurt → page.1people → page.3survey → page.3

O(log N) search

Web-address Directory

Example Queries

search(apples) = page.1, page.2

search(hate) = page.2, page.3

search(bananas) = page.3

Hash words into a table (array) using a hash function h(w), e.g:

h(hate) = 817 + 117 + 2017 + 517 (mod 11) = 7

search(w): goto hash-table row h(w).

Collisions: (hate,freaks), (survey,apples)

Problem: What if you search for hate or survey?

0 bananas→ page.3

1

2 hurt→ page.1

3 people→ page.3

4 dirty→ page.1, page.2

5

6

7 freaks→ page.2

hate→ page.2, page.3

8

9 apples→ page.1, page.2

survey→ page.3

10 health→ page.1, page.2

Creator: Malik Magdon-Ismail Independent Events: 10 / 13 Hashing and FOCS-twins →

Page 50: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Search and Hashing

http://page.1 http://page.2 http://page.3

dirty apples

hurt health

health freaks

hate dirty

apples

survey: people

hate bananas

apples → page.1, page.2bananas→ page.3dirty → page.1, page.2freaks → page.2hate → page.2, page.3health → page.1, page.2hurt → page.1people → page.3survey → page.3

O(log N) search

Web-address Directory

Example Queries

search(apples) = page.1, page.2

search(hate) = page.2, page.3

search(bananas) = page.3

Hash words into a table (array) using a hash function h(w), e.g:

h(hate) = 817 + 117 + 2017 + 517 (mod 11) = 7

search(w): goto hash-table row h(w).

Collisions: (hate,freaks), (survey,apples)

Problem: What if you search for hate or survey?

Good hash function maps words independently and randomly.

No collisions→ O(1) search (constant time, not log N).

0 bananas→ page.3

1

2 hurt→ page.1

3 people→ page.3

4 dirty→ page.1, page.2

5

6

7 freaks→ page.2

hate→ page.2, page.3

8

9 apples→ page.1, page.2

survey→ page.3

10 health→ page.1, page.2

Creator: Malik Magdon-Ismail Independent Events: 10 / 13 Hashing and FOCS-twins →

Page 51: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Hashing and FOCS-twins

Words w1, w2 . . . , wN and Hashing ↔ Students s1, s2, . . . , sN and Birthdays

Page 52: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Hashing and FOCS-twins

Words w1, w2 . . . , wN and Hashing ↔ Students s1, s2, . . . , sN and Birthdays

w1, . . . , wN hashed to rows 0, 1, . . . , B − 1 ↔ s1, . . . , sN born on days 0, 1, . . . , B − 1

Page 53: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Hashing and FOCS-twins

Words w1, w2 . . . , wN and Hashing ↔ Students s1, s2, . . . , sN and Birthdays

w1, . . . , wN hashed to rows 0, 1, . . . , B − 1 ↔ s1, . . . , sN born on days 0, 1, . . . , B − 1

No collisions, or hash-twins ↔ No FOCS-twins

Creator: Malik Magdon-Ismail Independent Events: 11 / 13 Random Walk →

Page 54: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Hashing and FOCS-twins

Words w1, w2 . . . , wN and Hashing ↔ Students s1, s2, . . . , sN and Birthdays

w1, . . . , wN hashed to rows 0, 1, . . . , B − 1 ↔ s1, . . . , sN born on days 0, 1, . . . , B − 1

No collisions, or hash-twins ↔ No FOCS-twins

Example: Suppose you have N = 10 words w1, w2, . . . , w10.

B = 10 (hash table has as many rows as words).

Creator: Malik Magdon-Ismail Independent Events: 11 / 13 Random Walk →

Page 55: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Hashing and FOCS-twins

Words w1, w2 . . . , wN and Hashing ↔ Students s1, s2, . . . , sN and Birthdays

w1, . . . , wN hashed to rows 0, 1, . . . , B − 1 ↔ s1, . . . , sN born on days 0, 1, . . . , B − 1

No collisions, or hash-twins ↔ No FOCS-twins

Example: Suppose you have N = 10 words w1, w2, . . . , w10.

B = 10 (hash table has as many rows as words).

P[no collisions] =(

910

)9×

(

89

)8×

(

78

)7×

(

67

)6×

(

56

)5×

(

45

)4×

(

34

)3×

(

23

)2×

(

12

)1×

(

01

)0≈ 0.0004.

Creator: Malik Magdon-Ismail Independent Events: 11 / 13 Random Walk →

Page 56: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Hashing and FOCS-twins

Words w1, w2 . . . , wN and Hashing ↔ Students s1, s2, . . . , sN and Birthdays

w1, . . . , wN hashed to rows 0, 1, . . . , B − 1 ↔ s1, . . . , sN born on days 0, 1, . . . , B − 1

No collisions, or hash-twins ↔ No FOCS-twins

Example: Suppose you have N = 10 words w1, w2, . . . , w10.

B = 10 (hash table has as many rows as words).

P[no collisions] =(

910

)9×

(

89

)8×

(

78

)7×

(

67

)6×

(

56

)5×

(

45

)4×

(

34

)3×

(

23

)2×

(

12

)1×

(

01

)0≈ 0.0004.

B = 20 (hash table has as twice many rows as words).

P[no collisions] =(

1920

)9×

(

1819

)8×

(

1718

)7×

(

1617

)6×

(

1516

)5×

(

1415

)4×

(

1314

)3×

(

1213

)2×

(

1112

)1×

(

1011

)0≈ 0.07.

Creator: Malik Magdon-Ismail Independent Events: 11 / 13 Random Walk →

Page 57: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Hashing and FOCS-twins

Words w1, w2 . . . , wN and Hashing ↔ Students s1, s2, . . . , sN and Birthdays

w1, . . . , wN hashed to rows 0, 1, . . . , B − 1 ↔ s1, . . . , sN born on days 0, 1, . . . , B − 1

No collisions, or hash-twins ↔ No FOCS-twins

Example: Suppose you have N = 10 words w1, w2, . . . , w10.

B = 10 (hash table has as many rows as words).

P[no collisions] =(

910

)9×

(

89

)8×

(

78

)7×

(

67

)6×

(

56

)5×

(

45

)4×

(

34

)3×

(

23

)2×

(

12

)1×

(

01

)0≈ 0.0004.

B = 20 (hash table has as twice many rows as words).

P[no collisions] =(

1920

)9×

(

1819

)8×

(

1718

)7×

(

1617

)6×

(

1516

)5×

(

1415

)4×

(

1314

)3×

(

1213

)2×

(

1112

)1×

(

1011

)0≈ 0.07.

B 10 20 30 40 50 60 70 80 90 100 500 1000

P[no collisions] 0.0004 0.07 0.18 0.29 0.38 0.45 0.51 0.56 0.60 0.63 0.91 0.96

B large enough → chances of no collisions are high (that’s good). How large should B be?

Creator: Malik Magdon-Ismail Independent Events: 11 / 13 Random Walk →

Page 58: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Hashing and FOCS-twins

Words w1, w2 . . . , wN and Hashing ↔ Students s1, s2, . . . , sN and Birthdays

w1, . . . , wN hashed to rows 0, 1, . . . , B − 1 ↔ s1, . . . , sN born on days 0, 1, . . . , B − 1

No collisions, or hash-twins ↔ No FOCS-twins

Example: Suppose you have N = 10 words w1, w2, . . . , w10.

B = 10 (hash table has as many rows as words).

P[no collisions] =(

910

)9×

(

89

)8×

(

78

)7×

(

67

)6×

(

56

)5×

(

45

)4×

(

34

)3×

(

23

)2×

(

12

)1×

(

01

)0≈ 0.0004.

B = 20 (hash table has as twice many rows as words).

P[no collisions] =(

1920

)9×

(

1819

)8×

(

1718

)7×

(

1617

)6×

(

1516

)5×

(

1415

)4×

(

1314

)3×

(

1213

)2×

(

1112

)1×

(

1011

)0≈ 0.07.

B 10 20 30 40 50 60 70 80 90 100 500 1000

P[no collisions] 0.0004 0.07 0.18 0.29 0.38 0.45 0.51 0.56 0.60 0.63 0.91 0.96

B large enough → chances of no collisions are high (that’s good). How large should B be?

Theorem. If B ∈ ω(N 2), then P[no collisions]→ 1

Creator: Malik Magdon-Ismail Independent Events: 11 / 13 Random Walk →

Page 59: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Random Walk: What are the Chances the Drunk Gets Home?

0 1 2 3

BAR

12

12

Creator: Malik Magdon-Ismail Independent Events: 12 / 13 Gambler’s Ruin →

Page 60: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Random Walk: What are the Chances the Drunk Gets Home?

0 1 2 3

BAR

12

12

Infinite Outcome Tree

Creator: Malik Magdon-Ismail Independent Events: 12 / 13 Gambler’s Ruin →

Page 61: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Random Walk: What are the Chances the Drunk Gets Home?

0 1 2 3

BAR

12

12

Infinite Outcome Tree

Sequences leading to home:L RLL RLRLL RLRLRLL RLRLRLRLL · · ·12 (1

2)3 (12)5 (1

2)7 (12)9 · · ·

P ((RL)•iL) = (12)2i+1

Creator: Malik Magdon-Ismail Independent Events: 12 / 13 Gambler’s Ruin →

Page 62: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Random Walk: What are the Chances the Drunk Gets Home?

0 1 2 3

BAR

12

12

Infinite Outcome Tree

Sequences leading to home:L RLL RLRLL RLRLRLL RLRLRLRLL · · ·12 (1

2)3 (12)5 (1

2)7 (12)9 · · ·

P ((RL)•iL) = (12)2i+1

P[home] = 12

+ (12)3 + (1

2)5 + (1

2)7 + (1

2)9 + · · ·

=12

1−14

= 23.

Creator: Malik Magdon-Ismail Independent Events: 12 / 13 Gambler’s Ruin →

Page 63: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Random Walk: What are the Chances the Drunk Gets Home?

0 1 2 3

BAR

12

12

Infinite Outcome Tree

Sequences leading to home:L RLL RLRLL RLRLRLL RLRLRLRLL · · ·12 (1

2)3 (12)5 (1

2)7 (12)9 · · ·

P ((RL)•iL) = (12)2i+1

P[home] = 12

+ (12)3 + (1

2)5 + (1

2)7 + (1

2)9 + · · ·

=12

1−14

= 23.

Total Probability

Creator: Malik Magdon-Ismail Independent Events: 12 / 13 Gambler’s Ruin →

Page 64: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Random Walk: What are the Chances the Drunk Gets Home?

0 1 2 3

BAR

12

12

Infinite Outcome Tree

Sequences leading to home:L RLL RLRLL RLRLRLL RLRLRLRLL · · ·12 (1

2)3 (12)5 (1

2)7 (12)9 · · ·

P ((RL)•iL) = (12)2i+1

P[home] = 12

+ (12)3 + (1

2)5 + (1

2)7 + (1

2)9 + · · ·

=12

1−14

= 23.

Total Probability

P [home] = P[L] · P[home | L]

Creator: Malik Magdon-Ismail Independent Events: 12 / 13 Gambler’s Ruin →

Page 65: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Random Walk: What are the Chances the Drunk Gets Home?

0 1 2 3

BAR

12

12

Infinite Outcome Tree

Sequences leading to home:L RLL RLRLL RLRLRLL RLRLRLRLL · · ·12 (1

2)3 (12)5 (1

2)7 (12)9 · · ·

P ((RL)•iL) = (12)2i+1

P[home] = 12

+ (12)3 + (1

2)5 + (1

2)7 + (1

2)9 + · · ·

=12

1−14

= 23.

Total Probability

P [home] = P[L] · P[home | L]

+ P[RR] · P[home | RR]

Creator: Malik Magdon-Ismail Independent Events: 12 / 13 Gambler’s Ruin →

Page 66: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Random Walk: What are the Chances the Drunk Gets Home?

0 1 2 3

BAR

12

12

Infinite Outcome Tree

Sequences leading to home:L RLL RLRLL RLRLRLL RLRLRLRLL · · ·12 (1

2)3 (12)5 (1

2)7 (12)9 · · ·

P ((RL)•iL) = (12)2i+1

P[home] = 12

+ (12)3 + (1

2)5 + (1

2)7 + (1

2)9 + · · ·

=12

1−14

= 23.

Total Probability

P [home] = P[L] · P[home | L]

+ P[RR] · P[home | RR]

+ P[RL] · P[home | RL]

Creator: Malik Magdon-Ismail Independent Events: 12 / 13 Gambler’s Ruin →

Page 67: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Random Walk: What are the Chances the Drunk Gets Home?

0 1 2 3

BAR

12

12

Infinite Outcome Tree

Sequences leading to home:L RLL RLRLL RLRLRLL RLRLRLRLL · · ·12 (1

2)3 (12)5 (1

2)7 (12)9 · · ·

P ((RL)•iL) = (12)2i+1

P[home] = 12

+ (12)3 + (1

2)5 + (1

2)7 + (1

2)9 + · · ·

=12

1−14

= 23.

Total Probability

P [home] = P[L] · P[home | L] ←12×1

+ P[RR] · P[home | RR]

+ P[RL] · P[home | RL]

Creator: Malik Magdon-Ismail Independent Events: 12 / 13 Gambler’s Ruin →

Page 68: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Random Walk: What are the Chances the Drunk Gets Home?

0 1 2 3

BAR

12

12

Infinite Outcome Tree

Sequences leading to home:L RLL RLRLL RLRLRLL RLRLRLRLL · · ·12 (1

2)3 (12)5 (1

2)7 (12)9 · · ·

P ((RL)•iL) = (12)2i+1

P[home] = 12

+ (12)3 + (1

2)5 + (1

2)7 + (1

2)9 + · · ·

=12

1−14

= 23.

Total Probability

P [home] = P[L] · P[home | L] ←12×1

+ P[RR] · P[home | RR] ←14×0

+ P[RL] · P[home | RL]

Creator: Malik Magdon-Ismail Independent Events: 12 / 13 Gambler’s Ruin →

Page 69: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Random Walk: What are the Chances the Drunk Gets Home?

0 1 2 3

BAR

12

12

Infinite Outcome Tree

Sequences leading to home:L RLL RLRLL RLRLRLL RLRLRLRLL · · ·12 (1

2)3 (12)5 (1

2)7 (12)9 · · ·

P ((RL)•iL) = (12)2i+1

P[home] = 12

+ (12)3 + (1

2)5 + (1

2)7 + (1

2)9 + · · ·

=12

1−14

= 23.

Total Probability

P [home] = P[L] · P[home | L] ←12×1

+ P[RR] · P[home | RR] ←14×0

+ P[RL] · P[home | RL] ←14×P[home]

Creator: Malik Magdon-Ismail Independent Events: 12 / 13 Gambler’s Ruin →

Page 70: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Random Walk: What are the Chances the Drunk Gets Home?

0 1 2 3

BAR

12

12

Infinite Outcome Tree

Sequences leading to home:L RLL RLRLL RLRLRLL RLRLRLRLL · · ·12 (1

2)3 (12)5 (1

2)7 (12)9 · · ·

P ((RL)•iL) = (12)2i+1

P[home] = 12

+ (12)3 + (1

2)5 + (1

2)7 + (1

2)9 + · · ·

=12

1−14

= 23.

Total Probability

P [home] = P[L] · P[home | L] ←12×1

+ P[RR] · P[home | RR] ←14×0

+ P[RL] · P[home | RL] ←14×P[home]

= 12

+ 14P [home].

Creator: Malik Magdon-Ismail Independent Events: 12 / 13 Gambler’s Ruin →

Page 71: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Random Walk: What are the Chances the Drunk Gets Home?

0 1 2 3

BAR

12

12

Infinite Outcome Tree

Sequences leading to home:L RLL RLRLL RLRLRLL RLRLRLRLL · · ·12 (1

2)3 (12)5 (1

2)7 (12)9 · · ·

P ((RL)•iL) = (12)2i+1

P[home] = 12

+ (12)3 + (1

2)5 + (1

2)7 + (1

2)9 + · · ·

=12

1−14

= 23.

Total Probability

P [home] = P[L] · P[home | L] ←12×1

+ P[RR] · P[home | RR] ←14×0

+ P[RL] · P[home | RL] ←14×P[home]

= 12

+ 14P [home].

That is, (1− 14) P [home] = 1

2. Solve for P[home]:

Creator: Malik Magdon-Ismail Independent Events: 12 / 13 Gambler’s Ruin →

Page 72: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Random Walk: What are the Chances the Drunk Gets Home?

0 1 2 3

BAR

12

12

Infinite Outcome Tree

Sequences leading to home:L RLL RLRLL RLRLRLL RLRLRLRLL · · ·12 (1

2)3 (12)5 (1

2)7 (12)9 · · ·

P ((RL)•iL) = (12)2i+1

P[home] = 12

+ (12)3 + (1

2)5 + (1

2)7 + (1

2)9 + · · ·

=12

1−14

= 23.

Total Probability

P [home] = P[L] · P[home | L] ←12×1

+ P[RR] · P[home | RR] ←14×0

+ P[RL] · P[home | RL] ←14×P[home]

= 12

+ 14P [home].

That is, (1− 14) P [home] = 1

2. Solve for P[home]:

P[home] =12

1−14

= 23.

Creator: Malik Magdon-Ismail Independent Events: 12 / 13 Gambler’s Ruin →

Page 73: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Doubling Up: A Random Walk at the Casino

$0 $1 $2 $3 $4

start

q = 0.6 p = 0.4

Creator: Malik Magdon-Ismail Independent Events: 13 / 13

Page 74: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Doubling Up: A Random Walk at the Casino

$0 $1 $2 $3 $4

start

q = 0.6 p = 0.4

P0 = 0 P4 = 1

Creator: Malik Magdon-Ismail Independent Events: 13 / 13

Page 75: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Doubling Up: A Random Walk at the Casino

$0 $1 $2 $3 $4

start

q = 0.6 p = 0.4

P0 = 0 P1 =? P4 = 1

P1

Creator: Malik Magdon-Ismail Independent Events: 13 / 13

Page 76: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Doubling Up: A Random Walk at the Casino

$0 $1 $2 $3 $4

start

q = 0.6 p = 0.4

P0 = 0 P1 =? P4 = 1

P1 = qP0 + pP2

Creator: Malik Magdon-Ismail Independent Events: 13 / 13

Page 77: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Doubling Up: A Random Walk at the Casino

$0 $1 $2 $3 $4

start

q = 0.6 p = 0.4

P0 = 0 P1 =? P4 = 1

P1 = qP0 + pP2 = pP2.

Creator: Malik Magdon-Ismail Independent Events: 13 / 13

Page 78: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Doubling Up: A Random Walk at the Casino

$0 $1 $2 $3 $4

start

q = 0.6 p = 0.4

P0 = 0 P1 =? P2 =? P4 = 1

P1 = qP0 + pP2 = pP2.

P2

Creator: Malik Magdon-Ismail Independent Events: 13 / 13

Page 79: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Doubling Up: A Random Walk at the Casino

$0 $1 $2 $3 $4

start

q = 0.6 p = 0.4

P0 = 0 P1 =? P2 =? P4 = 1

P1 = qP0 + pP2 = pP2.

P2 = qP1 + pP3

Creator: Malik Magdon-Ismail Independent Events: 13 / 13

Page 80: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Doubling Up: A Random Walk at the Casino

$0 $1 $2 $3 $4

start

q = 0.6 p = 0.4

P0 = 0 P1 =? P2 =? P4 = 1

P1 = qP0 + pP2 = pP2.

P2 = qP1 + pP3 = pqP2 + pP3

Creator: Malik Magdon-Ismail Independent Events: 13 / 13

Page 81: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Doubling Up: A Random Walk at the Casino

$0 $1 $2 $3 $4

start

q = 0.6 p = 0.4

P0 = 0 P1 =? P2 =? P4 = 1

P1 = qP0 + pP2 = pP2.

P2 = qP1 + pP3 = pqP2 + pP3 → P2 =pP3

1− pq.

Creator: Malik Magdon-Ismail Independent Events: 13 / 13

Page 82: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Doubling Up: A Random Walk at the Casino

$0 $1 $2 $3 $4

start

q = 0.6 p = 0.4

P0 = 0 P1 =? P2 =? P3 =? P4 = 1

P1 = qP0 + pP2 = pP2.

P2 = qP1 + pP3 = pqP2 + pP3 → P2 =pP3

1− pq.

P3

Creator: Malik Magdon-Ismail Independent Events: 13 / 13

Page 83: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Doubling Up: A Random Walk at the Casino

$0 $1 $2 $3 $4

start

q = 0.6 p = 0.4

P0 = 0 P1 =? P2 =? P3 =? P4 = 1

P1 = qP0 + pP2 = pP2.

P2 = qP1 + pP3 = pqP2 + pP3 → P2 =pP3

1− pq.

P3 = qP2 + pP4

Creator: Malik Magdon-Ismail Independent Events: 13 / 13

Page 84: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Doubling Up: A Random Walk at the Casino

$0 $1 $2 $3 $4

start

q = 0.6 p = 0.4

P0 = 0 P1 =? P2 =? P3 =? P4 = 1

P1 = qP0 + pP2 = pP2.

P2 = qP1 + pP3 = pqP2 + pP3 → P2 =pP3

1− pq.

P3 = qP2 + pP4 =pqP3

1− pq+ p

Creator: Malik Magdon-Ismail Independent Events: 13 / 13

Page 85: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Doubling Up: A Random Walk at the Casino

$0 $1 $2 $3 $4

start

q = 0.6 p = 0.4

P0 = 0 P1 =? P2 =? P3 =? P4 = 1

P1 = qP0 + pP2 = pP2.

P2 = qP1 + pP3 = pqP2 + pP3 → P2 =pP3

1− pq.

P3 = qP2 + pP4 =pqP3

1− pq+ p → P3 =

p(1− pq)

1− 2pq.

Creator: Malik Magdon-Ismail Independent Events: 13 / 13

Page 86: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Doubling Up: A Random Walk at the Casino

$0 $1 $2 $3 $4

start

q = 0.6 p = 0.4

P0 = 0 P1 =? P2 =? P3 =? P4 = 1

P1 = qP0 + pP2 = pP2.

P2 = qP1 + pP3 = pqP2 + pP3 → P2 =pP3

1− pq.

P3 = qP2 + pP4 =pqP3

1− pq+ p → P3 =

p(1− pq)

1− 2pq.

Conclusion:

P2 =p2

1− 2pq≈ 0.31 (69% chances of RUIN)

Creator: Malik Magdon-Ismail Independent Events: 13 / 13

Page 87: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Doubling Up: A Random Walk at the Casino

$0 $1 $2 $3 $4

start

q = 0.6 p = 0.4

P0 = 0 P1 =? P2 =? P3 =? P4 = 1

P1 = qP0 + pP2 = pP2.

P2 = qP1 + pP3 = pqP2 + pP3 → P2 =pP3

1− pq.

P3 = qP2 + pP4 =pqP3

1− pq+ p → P3 =

p(1− pq)

1− 2pq.

Conclusion:

P2 =p2

1− 2pq≈ 0.31 (69% chances of RUIN)

Exercise.

What if you are trying to double up from $3? (Answer: 77% chance of RUIN).

Creator: Malik Magdon-Ismail Independent Events: 13 / 13

Page 88: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Doubling Up: A Random Walk at the Casino

$0 $1 $2 $3 $4

start

q = 0.6 p = 0.4

P0 = 0 P1 =? P2 =? P3 =? P4 = 1

P1 = qP0 + pP2 = pP2.

P2 = qP1 + pP3 = pqP2 + pP3 → P2 =pP3

1− pq.

P3 = qP2 + pP4 =pqP3

1− pq+ p → P3 =

p(1− pq)

1− 2pq.

Conclusion:

P2 =p2

1− 2pq≈ 0.31 (69% chances of RUIN)

Exercise.

What if you are trying to double up from $3? (Answer: 77% chance of RUIN).

What if you are trying to double up from $10? (Answer: 98% chance of RUIN).

Creator: Malik Magdon-Ismail Independent Events: 13 / 13

Page 89: Foundations of Computer Science Lecture 17 - cs.rpi.edugoldsd/docs/csci2200-s18/csci2200-s18-lecture17.pdf · Foundations of Computer Science Lecture 17 Independent Events Independence

Doubling Up: A Random Walk at the Casino

$0 $1 $2 $3 $4

start

q = 0.6 p = 0.4

P0 = 0 P1 =? P2 =? P3 =? P4 = 1

P1 = qP0 + pP2 = pP2.

P2 = qP1 + pP3 = pqP2 + pP3 → P2 =pP3

1− pq.

P3 = qP2 + pP4 =pqP3

1− pq+ p → P3 =

p(1− pq)

1− 2pq.

Conclusion:

P2 =p2

1− 2pq≈ 0.31 (69% chances of RUIN)

Exercise.

What if you are trying to double up from $3? (Answer: 77% chance of RUIN).

What if you are trying to double up from $10? (Answer: 98% chance of RUIN).

The richer the Gambler, the greater the chances of RUIN!

Creator: Malik Magdon-Ismail Independent Events: 13 / 13