comp 182 algorithmic thinking discrete probability luay...
TRANSCRIPT
COMP 182 Algorithmic Thinking
Discrete Probability Luay NakhlehComputer ScienceRice University
Reading Material
❖ Chapter 7, Sections 1-4
Experiment, Sample Space, and Event❖ An experiment is a procedure that yields one of a given
set of possible outcomes (tossing a coin, rolling a die five times, etc.).
❖ The sample space of the experiment is the set of possible outcomes (for the coin toss experiment, the sample space is {H,T}).
❖ An event is a subset of the sample space (the possible events that correspond to the coin toss experiment are {}, {H}, {T}, {H,T}).
Experiment, Sample Space, and Event
❖ For each of the following, what is the sample space? What is an example of an event? ❖ Experiment 1: Rolling a 6-sided die twice.❖ Experiment 2: Generating an Erdos-Renyi
graph with parameters n and p.
Assigning Probabilities
❖ Let S be a sample space of an experiment with finite number of outcomes. We assign a probability p(s) to every outcome s, so that
1. 0≤p(s)≤1 for each s∈S, and
2. X
s2S
p(s) = 1.
Assigning Probabilities
❖The function p from the set of all outcomes of the sample space S is called a probability distribution.
Uniform Distribution
❖The uniform distribution on a set S with n elements assigns probability 1/n to each element of S (all n elements are equally likely).
Probability of an Event: Laplace’s Definition
❖ If S is a finite sample space of equally likely outcomes, and E is an event, that is, a subset of S, then the probability of E is p(E)=|E|/|S|.
Pierre-Simon Laplace
Probability of an Event
❖ Back to the experiment of rolling a die twice:❖ What is the probability of the event “the
sum of the two numbers is 8”?
Probability of an Event
❖ When the elements of the sample space are not all equally likely, Laplace’s definition doesn’t work!
❖ The probability of an event E is then defined in terms of the probabilities of its elements.
Probability of an Event
❖ The probability of the event E is
p(E) =X
s2E
p(s)
Probability of an Event
❖ Back to the experiment of ER graphs with n and p:❖ What is the probability of the event “the
generated graph has exactly k edges”?
Combinations of Events
❖ If E1,E2,… is a sequence of pairwise disjoint events in a sample space S, then
p
[
i
Ei
!=X
i
p(Ei)
Conditional Probability
❖ Let E and F be two events with p(F)>0. The conditional probability of E given F, denoted by p(E|F), is defined as
p(E|F ) =p(E \ F )
p(F )
Independence
❖ The events E and F are independent if and only if p(E∩F)=p(E)p(F).
❖ For example, consider the space of randomly generated bit strings of length four (all 16 have the same probability), and consider: ❖ E: the string begins with 1❖ F: the string contains an even number of 1s.
Pairwise and Mutual Independence❖ The events E1,E2,…,En are pairwise
independent if and only if p(Ei∩Ej)=p(Ei)p(Ej) for all pairs i and j with i≠j.
❖ The events E1,E2,…,En are mutually independent if and only if for every subset E’⊆{E1,E2,…,En} with |E’|≥2 we have
p
\
Ei2E0
Ei
!=
Y
Ei2E0
p(Ei)
❖ Mutually independent events are also pairwise independent events.
❖ Pairwise independent events are not necessarily mutually independent events. ❖ Experiment: Toss a fair coin twice.❖ Events:
❖ E1: The first toss is H.❖ E2: The second toss is H.❖ E3: Both tosses give the same outcome.
Bernoulli Trials
❖ Suppose an experiment can have only two possible outcomes, e.g., the flipping of a coin or the random generation of a bit (recall the random ER graphs?).
❖ Each performance of the experiment is called a Bernoulli trial.
❖ One outcome is called a success and the other a failure.
❖ If p and q are the probabilities of success and failure, respectively, then p+q=1.
The Binomial Distribution B(k:n,p)
❖ The probability of exactly k successes in n independent Bernoulli trials, with probability of success p and probability of failure q=1-p, is ✓
n
k
◆pkqn�k
Bayes’ Theorem
Bayes’ Theorem
❖ Suppose that E and F are events from a sample space S such that p(E)≠0 and p(F)≠0. Then:
p(F |E) =p(E|F )p(F )
p(E|F )p(F ) + p(E|F )p(F )
Bayes’ Theorem
❖ Suppose that E and F are events from a sample space S such that p(E)≠0 and p(F)≠0. Then:
p(F |E) =p(E|F )p(F )
p(E|F )p(F ) + p(E|F )p(F )
p(E)
❖ Suppose that one person in 100,000 has a particular disease. There is a test for the disease that gives a positive result 99% of the time when given to someone with the disease. When given to someone without the disease, 99.5% of the time it gives a negative result. Find
1. the probability that a person who tests positive has the disease.
2. the probability that a person who tests negative does not have the disease.
Generalized Bayes’ Theorem
❖Suppose that {F1,F2,…,Fn} is a partition of the sample space S. Then, for an event E, we have
p(Fj |E) =p(E|Fj)p(Fj)Pni=1 p(E|Fi)p(Fi)
Random Variables, Expected Value and Variance
Random Variables❖ A random variable is a function from the sample space
of an experiment to the set of real numbers.
❖ A coin is tossed twice. Let X(t) be the random variable that equals that number of heads that appear when t is the outcome. Then X(t) takes on the following values:
❖ X(HH)=2
❖ X(HT)=X(TH)=1
❖ X(TT)=0
Random Variables
❖ The distribution of a random variable X on a sample space S is the set of pairs (r,p(X=r)) for all r∈X(S), where p(X=r) is the probability that X takes the value r.
❖ Example: A fair coin is tossed twice and X(t) is the number of heads in outcome t. The distribution of X is
❖ {(0,0.25),(1,0.5),(2,0.25)}
Random Variables: Illustration on Random Graphs
❖ Experiment: Given a set of n nodes, for every set of two nodes, connect them with an edge with probability p (recall Erdos-Renyi?)
❖ What is the sample space? What is its size?
❖ Define a random variable X(g) that equals the number of edges in g.
❖ What are the possible values of X(g)?
❖ What is the probability distribution of X(g)?
Random Variables: Illustration on Random Graphs
❖ Experiment: Given a set of n nodes, for every set of two nodes, connect them with an edge with probability p (recall Erdos-Renyi?)
❖ What is the sample space? What is its size?
❖ S is the set of all graphs with n nodes.
❖ S contains the set of all n-node graphs with 0 edges, 1 edge, 2 edges,…, n(n-1)/2 edges. So, the size of S is
|S| =n(n�1)/2X
k=0
✓n(n� 1)/2
k
◆= 2n(n�1)/2
Random Variables: Illustration on Random Graphs
❖ Experiment: Given a set of n nodes, for every set of two nodes, connect them with an edge with probability p (recall Erdos-Renyi?)
❖ Define a random variable X(g) that equals the number of edges in g.
❖ What are the possible values of X(g)?
❖ X(g)∈{0,1,…,n(n-1)/2}.
Random Variables: Illustration on Random Graphs
❖ Experiment: Given a set of n nodes, for every set of two nodes, connect them with an edge with probability p (recall Erdos-Renyi?)
❖ Define a random variable X(g) that equals the number of edges in g.
❖ What is the probability distribution of X(g)?
p(X(g) = k) =
✓n(n� 1)/2
k
◆pk(1� p)
n(n�1)2 �k k 2
⇢0, 1, . . . ,
n(n� 1)
2
�
Expected Values
❖ The expected value (also called the expectation or mean) of a (discrete) random variable X on the sample space S is
E(X) =X
s2S
p(s)X(s)
Expected Values
❖ Example: A fair coin is tossed twice and X(t) is the number of heads.
E(X) = p(HH) ·X(HH) + p(HT ) ·X(HT ) + p(TT ) ·X(TT ) + p(TH) ·X(TH)= 0.25 · 2 + 0.25 · 1 + 0.25 · 0 + 0.25 · 1= 1
Expected Values❖ What is the expected number of edges in a random
graph with n nodes and probability p (using the ER procedure)?
❖ Based on the results we saw before, we have
E(X(g)) =Pn(n�1)/2
k=0 k · p(X(g) = k)
=Pn(n�1)/2
k=0 k�n(n�1)/2
k
�pk(1� p)
n(n�1)2 �k
= n(n�1)2 p
Linearity of Expectations
❖ If Xi, i=1,2,…,n, are random variables on S, and if a and b are real numbers, then
E(X1 +X2 + · · ·+Xn) = E(X1) + E(X2) + · · ·+ E(Xn)
E(aXi + b) = aE(Xi) + b
Average-case Analysis❖ What is the average-case running time of LinearSearch
(for finding whether an element x exists in an array of n elements)?
❖ Assume x is in the input array with probability p.
❖ Assume that if x is in the array, it can be in any position with equal probability.
❖ What random variable do we define? What is the expected value of the variable?
The Geometric Distribution
❖ So far, we have discussed random variables with a finite number of possible outcomes.
❖ Consider the following experiment: A coin with probability of tails being p is tossed repeatedly until it comes up tails. What is the expected number of tosses until this coin comes up tails?
❖ The sample space here is {T,HT,HHT,HHHT,…}, which is infinite.
The Geometric Distribution
❖ Let X be the random variable equal to the number of tosses in an element in the sample space.
❖ We have p(X=k)=(1-p)k-1p.
❖ Then,
E(X) =1X
k=1
kp(X = k) =1X
k=1
k(1� p)k�1p = p1X
k=1
k(1� p)k�1 = p1
p2=
1
p
The Geometric Distribution
❖ Let X be the random variable equal to the number of tosses in an element in the sample space.
❖ We have p(X=k)=(1-p)k-1p.
❖ Then,
E(X) =1X
k=1
kp(X = k) =1X
k=1
k(1� p)k�1p = p1X
k=1
k(1� p)k�1 = p1
p2=
1
p
1X
k=1
kxk�1 =1
(1� x)2(|x| < 1)
The Geometric Distribution
❖ A random variable X has a geometric distribution with parameter p if p(X=k)=(1-p)k-1p for k=1,2,3,…, where p is a real number with 0≤p≤1.
❖ If , then X ⇠ Geometric(p) E(X) = 1/p.
The Geometric Distribution
0 1
p
q
1-p 1-q
How is the duration in state 0 distributed?
Independent Random Variables
❖ The random variables X and Y on a sample space S are independent if
p(X = x and Y = y) = p(X = x)p(Y = y)
Independent Random Variables
❖ If random variables X and Y on sample space S are independent, then
E(XY ) = E(X)E(Y )
Variance
❖ The expected value of a random variable tells us its average value, but nothing about how widely its values are distributed.
❖ Contrast X and Y on S={1,2,3,4,5,6}, where
❖ X(s)=0 for all s∈S
❖ Y(s)=-1 for s∈{1,2,3} and Y(s)=1 for s∈{4,5,6}
Variance
❖ Let X be a random variable on a sample space S. The variance of X, denoted by V(X), is
V (X) =X
s2S
(X(s)� E(X))2p(s)
❖ The standard deviation of X, denoted by σ(X), is defined to be
�(X) =pV (X)
Variance
V (X) =P
s2S(X(s)� E(X))2p(s)=
Ps2S X(s)2p(s)� 2E(X)
Ps2S X(s)p(s) + E(X)2
Ps2S p(s)
= E(X2)� 2E(X)E(X) + E(X)2
= E(X2)� E(X)2
Variance
❖ If X is a random variable on a sample space S and , then E(X) = µ
V (X) = E((X � µ)2)
Bienayme’s Formula
❖ If Xi, i=1,2,…,n, are pairwise independent random variables on S, then
V (nX
i=1
Xi) =nX
i=1
V (Xi)
Variance
❖ If X is a random variable and c is a constant then,
V (cX) = c2V (X)
Bienayme’s Formula
❖ If Xi, i=1,2,…,n, are random variables (not necessarily independent) on S, then
where
Cov(X,Y ) = E(XY )� E(X)E(Y )
V
nX
i=1
Xi
!=
nX
i=1
V (Xi) + 2nX
i=1
X
j>i
Cov(Xi, Xj)
Tail Bounds
What Is This About?
❖ How large can a random variable get?
❖ In other words, how far from the mean can a value that the random variable takes be?
mean
mean
tails
mean
tails
how large?
Why Do We Care?
❖ Example:
❖ is the number of steps an algorithm takes.
❖ is the average-case running-time of the algorithm.
❖ Can the algorithm, on average, take 2n steps, but on some inputs take, say, 500n2 steps?
E(X)<latexit sha1_base64="kTVjQRU6iUov3VUghL8k1P3M2A8=">AAAB9XicbVDLSgMxFL1TX7W+qi7dBItQN2VGBF0WRXBZwT6gHUsmzbShmWRIMkoZ+h9uXCji1n9x59+YaWehrQcCh3Pu5Z6cIOZMG9f9dgorq2vrG8XN0tb2zu5eef+gpWWiCG0SyaXqBFhTzgRtGmY47cSK4ijgtB2MrzO//UiVZlLcm0lM/QgPBQsZwcZKD2kvwmYUBOhmWu2c9ssVt+bOgJaJl5MK5Gj0y1+9gSRJRIUhHGvd9dzY+ClWhhFOp6VeommMyRgPaddSgSOq/XSWeopOrDJAoVT2CYNm6u+NFEdaT6LATmYh9aKXif953cSEl37KRJwYKsj8UJhwZCTKKkADpigxfGIJJorZrIiMsMLE2KJKtgRv8cvLpHVW89yad3deqV/ldRThCI6hCh5cQB1uoQFNIKDgGV7hzXlyXpx352M+WnDynUP4A+fzB395kdY=</latexit><latexit sha1_base64="kTVjQRU6iUov3VUghL8k1P3M2A8=">AAAB9XicbVDLSgMxFL1TX7W+qi7dBItQN2VGBF0WRXBZwT6gHUsmzbShmWRIMkoZ+h9uXCji1n9x59+YaWehrQcCh3Pu5Z6cIOZMG9f9dgorq2vrG8XN0tb2zu5eef+gpWWiCG0SyaXqBFhTzgRtGmY47cSK4ijgtB2MrzO//UiVZlLcm0lM/QgPBQsZwcZKD2kvwmYUBOhmWu2c9ssVt+bOgJaJl5MK5Gj0y1+9gSRJRIUhHGvd9dzY+ClWhhFOp6VeommMyRgPaddSgSOq/XSWeopOrDJAoVT2CYNm6u+NFEdaT6LATmYh9aKXif953cSEl37KRJwYKsj8UJhwZCTKKkADpigxfGIJJorZrIiMsMLE2KJKtgRv8cvLpHVW89yad3deqV/ldRThCI6hCh5cQB1uoQFNIKDgGV7hzXlyXpx352M+WnDynUP4A+fzB395kdY=</latexit><latexit sha1_base64="kTVjQRU6iUov3VUghL8k1P3M2A8=">AAAB9XicbVDLSgMxFL1TX7W+qi7dBItQN2VGBF0WRXBZwT6gHUsmzbShmWRIMkoZ+h9uXCji1n9x59+YaWehrQcCh3Pu5Z6cIOZMG9f9dgorq2vrG8XN0tb2zu5eef+gpWWiCG0SyaXqBFhTzgRtGmY47cSK4ijgtB2MrzO//UiVZlLcm0lM/QgPBQsZwcZKD2kvwmYUBOhmWu2c9ssVt+bOgJaJl5MK5Gj0y1+9gSRJRIUhHGvd9dzY+ClWhhFOp6VeommMyRgPaddSgSOq/XSWeopOrDJAoVT2CYNm6u+NFEdaT6LATmYh9aKXif953cSEl37KRJwYKsj8UJhwZCTKKkADpigxfGIJJorZrIiMsMLE2KJKtgRv8cvLpHVW89yad3deqV/ldRThCI6hCh5cQB1uoQFNIKDgGV7hzXlyXpx352M+WnDynUP4A+fzB395kdY=</latexit><latexit sha1_base64="kTVjQRU6iUov3VUghL8k1P3M2A8=">AAAB9XicbVDLSgMxFL1TX7W+qi7dBItQN2VGBF0WRXBZwT6gHUsmzbShmWRIMkoZ+h9uXCji1n9x59+YaWehrQcCh3Pu5Z6cIOZMG9f9dgorq2vrG8XN0tb2zu5eef+gpWWiCG0SyaXqBFhTzgRtGmY47cSK4ijgtB2MrzO//UiVZlLcm0lM/QgPBQsZwcZKD2kvwmYUBOhmWu2c9ssVt+bOgJaJl5MK5Gj0y1+9gSRJRIUhHGvd9dzY+ClWhhFOp6VeommMyRgPaddSgSOq/XSWeopOrDJAoVT2CYNm6u+NFEdaT6LATmYh9aKXif953cSEl37KRJwYKsj8UJhwZCTKKkADpigxfGIJJorZrIiMsMLE2KJKtgRv8cvLpHVW89yad3deqV/ldRThCI6hCh5cQB1uoQFNIKDgGV7hzXlyXpx352M+WnDynUP4A+fzB395kdY=</latexit>
X<latexit sha1_base64="hf6hOeTjseL13iz+i/MO/ptaY5E=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dzvPKHSPJYPZpqgH9GR5CFn1Fip2R2UK27VXYCsEy8nFcjRGJS/+sOYpRFKwwTVuue5ifEzqgxnAmelfqoxoWxCR9izVNIItZ8tDp2RC6sMSRgrW9KQhfp7IqOR1tMosJ0RNWO96s3F/7xeasIbP+MySQ1KtlwUpoKYmMy/JkOukBkxtYQyxe2thI2poszYbEo2BG/15XXSvqp6btVrXlfqt3kcRTiDc7gED2pQh3toQAsYIDzDK7w5j86L8+58LFsLTj5zCn/gfP4AtbmM3A==</latexit><latexit sha1_base64="hf6hOeTjseL13iz+i/MO/ptaY5E=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dzvPKHSPJYPZpqgH9GR5CFn1Fip2R2UK27VXYCsEy8nFcjRGJS/+sOYpRFKwwTVuue5ifEzqgxnAmelfqoxoWxCR9izVNIItZ8tDp2RC6sMSRgrW9KQhfp7IqOR1tMosJ0RNWO96s3F/7xeasIbP+MySQ1KtlwUpoKYmMy/JkOukBkxtYQyxe2thI2poszYbEo2BG/15XXSvqp6btVrXlfqt3kcRTiDc7gED2pQh3toQAsYIDzDK7w5j86L8+58LFsLTj5zCn/gfP4AtbmM3A==</latexit><latexit sha1_base64="hf6hOeTjseL13iz+i/MO/ptaY5E=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dzvPKHSPJYPZpqgH9GR5CFn1Fip2R2UK27VXYCsEy8nFcjRGJS/+sOYpRFKwwTVuue5ifEzqgxnAmelfqoxoWxCR9izVNIItZ8tDp2RC6sMSRgrW9KQhfp7IqOR1tMosJ0RNWO96s3F/7xeasIbP+MySQ1KtlwUpoKYmMy/JkOukBkxtYQyxe2thI2poszYbEo2BG/15XXSvqp6btVrXlfqt3kcRTiDc7gED2pQh3toQAsYIDzDK7w5j86L8+58LFsLTj5zCn/gfP4AtbmM3A==</latexit><latexit sha1_base64="hf6hOeTjseL13iz+i/MO/ptaY5E=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dzvPKHSPJYPZpqgH9GR5CFn1Fip2R2UK27VXYCsEy8nFcjRGJS/+sOYpRFKwwTVuue5ifEzqgxnAmelfqoxoWxCR9izVNIItZ8tDp2RC6sMSRgrW9KQhfp7IqOR1tMosJ0RNWO96s3F/7xeasIbP+MySQ1KtlwUpoKYmMy/JkOukBkxtYQyxe2thI2poszYbEo2BG/15XXSvqp6btVrXlfqt3kcRTiDc7gED2pQh3toQAsYIDzDK7w5j86L8+58LFsLTj5zCn/gfP4AtbmM3A==</latexit>
Markov’s Inequality
❖ Let X be a random variable that takes only nonnegative values. Then, for every real number a>0 we have
P (X � a) E(X)
a<latexit sha1_base64="31UkQukhjmsnC9djG5Wb82HiHoE=">AAACEnicbVDLSsNAFJ3UV62vqEs3g0VoNyURQZdFEVxWsA9oQrmZTtqhk4czE6GEfIMbf8WNC0XcunLn3zhps9DWA8MczrmXe+/xYs6ksqxvo7Syura+Ud6sbG3v7O6Z+wcdGSWC0DaJeCR6HkjKWUjbiilOe7GgEHicdr3JVe53H6iQLArv1DSmbgCjkPmMgNLSwKy3aj3sjOg9hjp2uP4dXwBJUycANfY8fJ3VevUshWxgVq2GNQNeJnZBqqhAa2B+OcOIJAENFeEgZd+2YuWmIBQjnGYVJ5E0BjKBEe1rGkJApZvOTsrwiVaG2I+EfqHCM/V3RwqBlNPA05X5onLRy8X/vH6i/As3ZWGcKBqS+SA/4VhFOM8HD5mgRPGpJkAE07tiMgYdidIpVnQI9uLJy6Rz2rCthn17Vm1eFnGU0RE6RjVko3PURDeohdqIoEf0jF7Rm/FkvBjvxse8tGQUPYfoD4zPH/qYnG8=</latexit><latexit sha1_base64="31UkQukhjmsnC9djG5Wb82HiHoE=">AAACEnicbVDLSsNAFJ3UV62vqEs3g0VoNyURQZdFEVxWsA9oQrmZTtqhk4czE6GEfIMbf8WNC0XcunLn3zhps9DWA8MczrmXe+/xYs6ksqxvo7Syura+Ud6sbG3v7O6Z+wcdGSWC0DaJeCR6HkjKWUjbiilOe7GgEHicdr3JVe53H6iQLArv1DSmbgCjkPmMgNLSwKy3aj3sjOg9hjp2uP4dXwBJUycANfY8fJ3VevUshWxgVq2GNQNeJnZBqqhAa2B+OcOIJAENFeEgZd+2YuWmIBQjnGYVJ5E0BjKBEe1rGkJApZvOTsrwiVaG2I+EfqHCM/V3RwqBlNPA05X5onLRy8X/vH6i/As3ZWGcKBqS+SA/4VhFOM8HD5mgRPGpJkAE07tiMgYdidIpVnQI9uLJy6Rz2rCthn17Vm1eFnGU0RE6RjVko3PURDeohdqIoEf0jF7Rm/FkvBjvxse8tGQUPYfoD4zPH/qYnG8=</latexit><latexit sha1_base64="31UkQukhjmsnC9djG5Wb82HiHoE=">AAACEnicbVDLSsNAFJ3UV62vqEs3g0VoNyURQZdFEVxWsA9oQrmZTtqhk4czE6GEfIMbf8WNC0XcunLn3zhps9DWA8MczrmXe+/xYs6ksqxvo7Syura+Ud6sbG3v7O6Z+wcdGSWC0DaJeCR6HkjKWUjbiilOe7GgEHicdr3JVe53H6iQLArv1DSmbgCjkPmMgNLSwKy3aj3sjOg9hjp2uP4dXwBJUycANfY8fJ3VevUshWxgVq2GNQNeJnZBqqhAa2B+OcOIJAENFeEgZd+2YuWmIBQjnGYVJ5E0BjKBEe1rGkJApZvOTsrwiVaG2I+EfqHCM/V3RwqBlNPA05X5onLRy8X/vH6i/As3ZWGcKBqS+SA/4VhFOM8HD5mgRPGpJkAE07tiMgYdidIpVnQI9uLJy6Rz2rCthn17Vm1eFnGU0RE6RjVko3PURDeohdqIoEf0jF7Rm/FkvBjvxse8tGQUPYfoD4zPH/qYnG8=</latexit><latexit sha1_base64="31UkQukhjmsnC9djG5Wb82HiHoE=">AAACEnicbVDLSsNAFJ3UV62vqEs3g0VoNyURQZdFEVxWsA9oQrmZTtqhk4czE6GEfIMbf8WNC0XcunLn3zhps9DWA8MczrmXe+/xYs6ksqxvo7Syura+Ud6sbG3v7O6Z+wcdGSWC0DaJeCR6HkjKWUjbiilOe7GgEHicdr3JVe53H6iQLArv1DSmbgCjkPmMgNLSwKy3aj3sjOg9hjp2uP4dXwBJUycANfY8fJ3VevUshWxgVq2GNQNeJnZBqqhAa2B+OcOIJAENFeEgZd+2YuWmIBQjnGYVJ5E0BjKBEe1rGkJApZvOTsrwiVaG2I+EfqHCM/V3RwqBlNPA05X5onLRy8X/vH6i/As3ZWGcKBqS+SA/4VhFOM8HD5mgRPGpJkAE07tiMgYdidIpVnQI9uLJy6Rz2rCthn17Vm1eFnGU0RE6RjVko3PURDeohdqIoEf0jF7Rm/FkvBjvxse8tGQUPYfoD4zPH/qYnG8=</latexit>
Markov’s Inequality
❖ Let X be a random variable that takes only nonnegative values. Then, for every real number a>0 we have
How large a value can X take?
P (X � a) E(X)
a<latexit sha1_base64="31UkQukhjmsnC9djG5Wb82HiHoE=">AAACEnicbVDLSsNAFJ3UV62vqEs3g0VoNyURQZdFEVxWsA9oQrmZTtqhk4czE6GEfIMbf8WNC0XcunLn3zhps9DWA8MczrmXe+/xYs6ksqxvo7Syura+Ud6sbG3v7O6Z+wcdGSWC0DaJeCR6HkjKWUjbiilOe7GgEHicdr3JVe53H6iQLArv1DSmbgCjkPmMgNLSwKy3aj3sjOg9hjp2uP4dXwBJUycANfY8fJ3VevUshWxgVq2GNQNeJnZBqqhAa2B+OcOIJAENFeEgZd+2YuWmIBQjnGYVJ5E0BjKBEe1rGkJApZvOTsrwiVaG2I+EfqHCM/V3RwqBlNPA05X5onLRy8X/vH6i/As3ZWGcKBqS+SA/4VhFOM8HD5mgRPGpJkAE07tiMgYdidIpVnQI9uLJy6Rz2rCthn17Vm1eFnGU0RE6RjVko3PURDeohdqIoEf0jF7Rm/FkvBjvxse8tGQUPYfoD4zPH/qYnG8=</latexit><latexit sha1_base64="31UkQukhjmsnC9djG5Wb82HiHoE=">AAACEnicbVDLSsNAFJ3UV62vqEs3g0VoNyURQZdFEVxWsA9oQrmZTtqhk4czE6GEfIMbf8WNC0XcunLn3zhps9DWA8MczrmXe+/xYs6ksqxvo7Syura+Ud6sbG3v7O6Z+wcdGSWC0DaJeCR6HkjKWUjbiilOe7GgEHicdr3JVe53H6iQLArv1DSmbgCjkPmMgNLSwKy3aj3sjOg9hjp2uP4dXwBJUycANfY8fJ3VevUshWxgVq2GNQNeJnZBqqhAa2B+OcOIJAENFeEgZd+2YuWmIBQjnGYVJ5E0BjKBEe1rGkJApZvOTsrwiVaG2I+EfqHCM/V3RwqBlNPA05X5onLRy8X/vH6i/As3ZWGcKBqS+SA/4VhFOM8HD5mgRPGpJkAE07tiMgYdidIpVnQI9uLJy6Rz2rCthn17Vm1eFnGU0RE6RjVko3PURDeohdqIoEf0jF7Rm/FkvBjvxse8tGQUPYfoD4zPH/qYnG8=</latexit><latexit sha1_base64="31UkQukhjmsnC9djG5Wb82HiHoE=">AAACEnicbVDLSsNAFJ3UV62vqEs3g0VoNyURQZdFEVxWsA9oQrmZTtqhk4czE6GEfIMbf8WNC0XcunLn3zhps9DWA8MczrmXe+/xYs6ksqxvo7Syura+Ud6sbG3v7O6Z+wcdGSWC0DaJeCR6HkjKWUjbiilOe7GgEHicdr3JVe53H6iQLArv1DSmbgCjkPmMgNLSwKy3aj3sjOg9hjp2uP4dXwBJUycANfY8fJ3VevUshWxgVq2GNQNeJnZBqqhAa2B+OcOIJAENFeEgZd+2YuWmIBQjnGYVJ5E0BjKBEe1rGkJApZvOTsrwiVaG2I+EfqHCM/V3RwqBlNPA05X5onLRy8X/vH6i/As3ZWGcKBqS+SA/4VhFOM8HD5mgRPGpJkAE07tiMgYdidIpVnQI9uLJy6Rz2rCthn17Vm1eFnGU0RE6RjVko3PURDeohdqIoEf0jF7Rm/FkvBjvxse8tGQUPYfoD4zPH/qYnG8=</latexit><latexit sha1_base64="31UkQukhjmsnC9djG5Wb82HiHoE=">AAACEnicbVDLSsNAFJ3UV62vqEs3g0VoNyURQZdFEVxWsA9oQrmZTtqhk4czE6GEfIMbf8WNC0XcunLn3zhps9DWA8MczrmXe+/xYs6ksqxvo7Syura+Ud6sbG3v7O6Z+wcdGSWC0DaJeCR6HkjKWUjbiilOe7GgEHicdr3JVe53H6iQLArv1DSmbgCjkPmMgNLSwKy3aj3sjOg9hjp2uP4dXwBJUycANfY8fJ3VevUshWxgVq2GNQNeJnZBqqhAa2B+OcOIJAENFeEgZd+2YuWmIBQjnGYVJ5E0BjKBEe1rGkJApZvOTsrwiVaG2I+EfqHCM/V3RwqBlNPA05X5onLRy8X/vH6i/As3ZWGcKBqS+SA/4VhFOM8HD5mgRPGpJkAE07tiMgYdidIpVnQI9uLJy6Rz2rCthn17Vm1eFnGU0RE6RjVko3PURDeohdqIoEf0jF7Rm/FkvBjvxse8tGQUPYfoD4zPH/qYnG8=</latexit>
Markov’s Inequality: Proof
E(X) =P
x xP (x)=
Px<a xP (x) +
Px�a xP (x)
�P
x<a 0P (x) +P
x�a aP (x)= a
Px�a P (x)
= aP (x � a)<latexit sha1_base64="zdBcIo93Nxa1qsLtbPixCAD4rZU=">AAAC03icbVJdi9NAFJ1k/VjjV10ffblYXFqEkoigDwqLIvhYwe4WmlJuprfdYSeTODNZWkLYRXz1z/nmX/BXOGnistt6YeDMOffcO3NnklwKY8Pwt+fv3bp95+7+veD+g4ePHneeHBybrNCcRjyTmR4naEgKRSMrrKRxrgnTRNJJcvax1k/OSRuRqa92ndM0xaUSC8HROmrW+RMntBSqRK1xXZVSyioo4xTtaZLAp6o37sMhvHcrNkU6K1cVrGDYW/UhjgO4KcE7rFYb7eUVFS/pG/yja8thQ10zORnCpmaw68Srbk0v3E7Y0d2+1fqOjUnN28vNOt1wEG4CdkHUgi5rYzjr/IrnGS9SUpZLNGYShbmdumpWcElVEBeGcuRnuKSJgwpTMtNy8yYVvHDMHBaZdktZ2LDXHSWmxqzTxGXW0zbbWk3+T5sUdvF2WgqVF5YUbxotCgk2g/qBYS40cSvXDiDXwp0V+Clq5NZ9g8ANIdq+8i44fjWIwkH05XX36EM7jn32jD1nPRaxN+yIfWZDNmLcG3rn3oV36Y/80v/u/2hSfa/1PGU3wv/5F1RA1K8=</latexit><latexit sha1_base64="zdBcIo93Nxa1qsLtbPixCAD4rZU=">AAAC03icbVJdi9NAFJ1k/VjjV10ffblYXFqEkoigDwqLIvhYwe4WmlJuprfdYSeTODNZWkLYRXz1z/nmX/BXOGnistt6YeDMOffcO3NnklwKY8Pwt+fv3bp95+7+veD+g4ePHneeHBybrNCcRjyTmR4naEgKRSMrrKRxrgnTRNJJcvax1k/OSRuRqa92ndM0xaUSC8HROmrW+RMntBSqRK1xXZVSyioo4xTtaZLAp6o37sMhvHcrNkU6K1cVrGDYW/UhjgO4KcE7rFYb7eUVFS/pG/yja8thQ10zORnCpmaw68Srbk0v3E7Y0d2+1fqOjUnN28vNOt1wEG4CdkHUgi5rYzjr/IrnGS9SUpZLNGYShbmdumpWcElVEBeGcuRnuKSJgwpTMtNy8yYVvHDMHBaZdktZ2LDXHSWmxqzTxGXW0zbbWk3+T5sUdvF2WgqVF5YUbxotCgk2g/qBYS40cSvXDiDXwp0V+Clq5NZ9g8ANIdq+8i44fjWIwkH05XX36EM7jn32jD1nPRaxN+yIfWZDNmLcG3rn3oV36Y/80v/u/2hSfa/1PGU3wv/5F1RA1K8=</latexit><latexit sha1_base64="zdBcIo93Nxa1qsLtbPixCAD4rZU=">AAAC03icbVJdi9NAFJ1k/VjjV10ffblYXFqEkoigDwqLIvhYwe4WmlJuprfdYSeTODNZWkLYRXz1z/nmX/BXOGnistt6YeDMOffcO3NnklwKY8Pwt+fv3bp95+7+veD+g4ePHneeHBybrNCcRjyTmR4naEgKRSMrrKRxrgnTRNJJcvax1k/OSRuRqa92ndM0xaUSC8HROmrW+RMntBSqRK1xXZVSyioo4xTtaZLAp6o37sMhvHcrNkU6K1cVrGDYW/UhjgO4KcE7rFYb7eUVFS/pG/yja8thQ10zORnCpmaw68Srbk0v3E7Y0d2+1fqOjUnN28vNOt1wEG4CdkHUgi5rYzjr/IrnGS9SUpZLNGYShbmdumpWcElVEBeGcuRnuKSJgwpTMtNy8yYVvHDMHBaZdktZ2LDXHSWmxqzTxGXW0zbbWk3+T5sUdvF2WgqVF5YUbxotCgk2g/qBYS40cSvXDiDXwp0V+Clq5NZ9g8ANIdq+8i44fjWIwkH05XX36EM7jn32jD1nPRaxN+yIfWZDNmLcG3rn3oV36Y/80v/u/2hSfa/1PGU3wv/5F1RA1K8=</latexit><latexit sha1_base64="zdBcIo93Nxa1qsLtbPixCAD4rZU=">AAAC03icbVJdi9NAFJ1k/VjjV10ffblYXFqEkoigDwqLIvhYwe4WmlJuprfdYSeTODNZWkLYRXz1z/nmX/BXOGnistt6YeDMOffcO3NnklwKY8Pwt+fv3bp95+7+veD+g4ePHneeHBybrNCcRjyTmR4naEgKRSMrrKRxrgnTRNJJcvax1k/OSRuRqa92ndM0xaUSC8HROmrW+RMntBSqRK1xXZVSyioo4xTtaZLAp6o37sMhvHcrNkU6K1cVrGDYW/UhjgO4KcE7rFYb7eUVFS/pG/yja8thQ10zORnCpmaw68Srbk0v3E7Y0d2+1fqOjUnN28vNOt1wEG4CdkHUgi5rYzjr/IrnGS9SUpZLNGYShbmdumpWcElVEBeGcuRnuKSJgwpTMtNy8yYVvHDMHBaZdktZ2LDXHSWmxqzTxGXW0zbbWk3+T5sUdvF2WgqVF5YUbxotCgk2g/qBYS40cSvXDiDXwp0V+Clq5NZ9g8ANIdq+8i44fjWIwkH05XX36EM7jn32jD1nPRaxN+yIfWZDNmLcG3rn3oV36Y/80v/u/2hSfa/1PGU3wv/5F1RA1K8=</latexit>
Markov’s Inequality: An Example
❖ A fair coin is tossed n times. Give an upper bound on the probability that at least 3n/4 of the tosses yield heads.
P (X � 3n
4) E(X)
3n/4=
n/2
3n/4=
2
3<latexit sha1_base64="HcuvOFcpjVlkBhuZxxBdEiP5M9o=">AAACQXicbZDNS8MwGMZTP+f8mnr0EhzCdtnaOdCLMBTB4wS3FdYx0izdwtK0JqkwSv81L/4H3rx78aCIVy+mWxHdfCHw8Hvelzfv44aMSmWaz8bS8srq2npuI7+5tb2zW9jbb8sgEpi0cMACYbtIEkY5aSmqGLFDQZDvMtJxx5ep37knQtKA36pJSHo+GnLqUYyURv2C3SzZ0BmSO+h4AuH4hCdxPSlDh/2g2PGRGrkuvEpKdjnRLdV6As8zl1dr8ygFSb9QNCvmtOCisDJRBFk1+4UnZxDgyCdcYYak7FpmqHoxEopiRpK8E0kSIjxGQ9LVkiOfyF48TSCBx5oMoBcI/biCU/p7Ika+lBPf1Z3pMXLeS+F/XjdS3lkvpjyMFOF4tsiLGFQBTOOEAyoIVmyiBcKC6r9CPEI6BaVDz+sQrPmTF0W7VrHMinVTLzYusjhy4BAcgRKwwClogGvQBC2AwQN4AW/g3Xg0Xo0P43PWumRkMwfgTxlf3zaMrkY=</latexit><latexit sha1_base64="HcuvOFcpjVlkBhuZxxBdEiP5M9o=">AAACQXicbZDNS8MwGMZTP+f8mnr0EhzCdtnaOdCLMBTB4wS3FdYx0izdwtK0JqkwSv81L/4H3rx78aCIVy+mWxHdfCHw8Hvelzfv44aMSmWaz8bS8srq2npuI7+5tb2zW9jbb8sgEpi0cMACYbtIEkY5aSmqGLFDQZDvMtJxx5ep37knQtKA36pJSHo+GnLqUYyURv2C3SzZ0BmSO+h4AuH4hCdxPSlDh/2g2PGRGrkuvEpKdjnRLdV6As8zl1dr8ygFSb9QNCvmtOCisDJRBFk1+4UnZxDgyCdcYYak7FpmqHoxEopiRpK8E0kSIjxGQ9LVkiOfyF48TSCBx5oMoBcI/biCU/p7Ika+lBPf1Z3pMXLeS+F/XjdS3lkvpjyMFOF4tsiLGFQBTOOEAyoIVmyiBcKC6r9CPEI6BaVDz+sQrPmTF0W7VrHMinVTLzYusjhy4BAcgRKwwClogGvQBC2AwQN4AW/g3Xg0Xo0P43PWumRkMwfgTxlf3zaMrkY=</latexit><latexit sha1_base64="HcuvOFcpjVlkBhuZxxBdEiP5M9o=">AAACQXicbZDNS8MwGMZTP+f8mnr0EhzCdtnaOdCLMBTB4wS3FdYx0izdwtK0JqkwSv81L/4H3rx78aCIVy+mWxHdfCHw8Hvelzfv44aMSmWaz8bS8srq2npuI7+5tb2zW9jbb8sgEpi0cMACYbtIEkY5aSmqGLFDQZDvMtJxx5ep37knQtKA36pJSHo+GnLqUYyURv2C3SzZ0BmSO+h4AuH4hCdxPSlDh/2g2PGRGrkuvEpKdjnRLdV6As8zl1dr8ygFSb9QNCvmtOCisDJRBFk1+4UnZxDgyCdcYYak7FpmqHoxEopiRpK8E0kSIjxGQ9LVkiOfyF48TSCBx5oMoBcI/biCU/p7Ika+lBPf1Z3pMXLeS+F/XjdS3lkvpjyMFOF4tsiLGFQBTOOEAyoIVmyiBcKC6r9CPEI6BaVDz+sQrPmTF0W7VrHMinVTLzYusjhy4BAcgRKwwClogGvQBC2AwQN4AW/g3Xg0Xo0P43PWumRkMwfgTxlf3zaMrkY=</latexit><latexit sha1_base64="hP+6LrUf2d3tZaldqaQQvEKMXyw=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odBu3wMYA6nMMFXEEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit><latexit sha1_base64="cJVIq6fDH9Dtb1vDnmGkmMNTnL8=">AAACNnicbZBPS8MwGMbfzn9z/qtevQRFmJfZzoFeBEEEjxPcLKxlpFk6g2lak1QYpV/Ni9/Am3cvHhTxE5huQ3T6QODh97wheZ8w5Uxpx3m2KnPzC4tL1eXayura+oa9udpVSSYJ7ZCEJ9ILsaKcCdrRTHPqpZLiOOT0Orw9K/PreyoVS8SVHqU0iPFQsIgRrA3q21677iF/SO+QH0lM8kNR5K1iH/n8G+V+jPVNGKLzou7tF2bkoFWgk2kqDpqzqARF3951Gs5Y6K9xp2YXpmr37Sd/kJAspkITjpXquU6qgxxLzQinRc3PFE0xucVD2jNW4JiqIB83UKA9QwYoSqQ5QqMx/Xkjx7FSozg0k+UyajYr4X9ZL9PRcZAzkWaaCjJ5KMo40gkq60QDJinRfGQMJpKZvyJyg00L2pReMyW4syv/Nd1mw3Ua7qUDVdiGHaiDC0dwChfQhg4QeIAXeIN369F6tT4mdVWsaW9b8EvW5xfDmKya</latexit><latexit sha1_base64="cJVIq6fDH9Dtb1vDnmGkmMNTnL8=">AAACNnicbZBPS8MwGMbfzn9z/qtevQRFmJfZzoFeBEEEjxPcLKxlpFk6g2lak1QYpV/Ni9/Am3cvHhTxE5huQ3T6QODh97wheZ8w5Uxpx3m2KnPzC4tL1eXayura+oa9udpVSSYJ7ZCEJ9ILsaKcCdrRTHPqpZLiOOT0Orw9K/PreyoVS8SVHqU0iPFQsIgRrA3q21677iF/SO+QH0lM8kNR5K1iH/n8G+V+jPVNGKLzou7tF2bkoFWgk2kqDpqzqARF3951Gs5Y6K9xp2YXpmr37Sd/kJAspkITjpXquU6qgxxLzQinRc3PFE0xucVD2jNW4JiqIB83UKA9QwYoSqQ5QqMx/Xkjx7FSozg0k+UyajYr4X9ZL9PRcZAzkWaaCjJ5KMo40gkq60QDJinRfGQMJpKZvyJyg00L2pReMyW4syv/Nd1mw3Ua7qUDVdiGHaiDC0dwChfQhg4QeIAXeIN369F6tT4mdVWsaW9b8EvW5xfDmKya</latexit><latexit sha1_base64="hSgK9dyjj0PqU3eWocHf+/5PwWA=">AAACQXicbZDNS8MwGMZTP+f8mnr0EhzCdtnaOdCLMBTB4wS3FdYx0izdwtK0JqkwSv81L/4H3rx78aCIVy+mWxHdfCHw8Hvelzfv44aMSmWaz8bS8srq2npuI7+5tb2zW9jbb8sgEpi0cMACYbtIEkY5aSmqGLFDQZDvMtJxx5ep37knQtKA36pJSHo+GnLqUYyURv2C3SzZ0BmSO+h4AuH4hCdxPSlDh/2g2PGRGrkuvEpKdjnRLdV6As8zl1dr8ygFSb9QNCvmtOCisDJRBFk1+4UnZxDgyCdcYYak7FpmqHoxEopiRpK8E0kSIjxGQ9LVkiOfyF48TSCBx5oMoBcI/biCU/p7Ika+lBPf1Z3pMXLeS+F/XjdS3lkvpjyMFOF4tsiLGFQBTOOEAyoIVmyiBcKC6r9CPEI6BaVDz+sQrPmTF0W7VrHMinVjFhsXWRw5cAiOQAlY4BQ0wDVoghbA4AG8gDfwbjwar8aH8TlrXTKymQPwp4yvbzVMrkI=</latexit><latexit sha1_base64="HcuvOFcpjVlkBhuZxxBdEiP5M9o=">AAACQXicbZDNS8MwGMZTP+f8mnr0EhzCdtnaOdCLMBTB4wS3FdYx0izdwtK0JqkwSv81L/4H3rx78aCIVy+mWxHdfCHw8Hvelzfv44aMSmWaz8bS8srq2npuI7+5tb2zW9jbb8sgEpi0cMACYbtIEkY5aSmqGLFDQZDvMtJxx5ep37knQtKA36pJSHo+GnLqUYyURv2C3SzZ0BmSO+h4AuH4hCdxPSlDh/2g2PGRGrkuvEpKdjnRLdV6As8zl1dr8ygFSb9QNCvmtOCisDJRBFk1+4UnZxDgyCdcYYak7FpmqHoxEopiRpK8E0kSIjxGQ9LVkiOfyF48TSCBx5oMoBcI/biCU/p7Ika+lBPf1Z3pMXLeS+F/XjdS3lkvpjyMFOF4tsiLGFQBTOOEAyoIVmyiBcKC6r9CPEI6BaVDz+sQrPmTF0W7VrHMinVTLzYusjhy4BAcgRKwwClogGvQBC2AwQN4AW/g3Xg0Xo0P43PWumRkMwfgTxlf3zaMrkY=</latexit><latexit sha1_base64="HcuvOFcpjVlkBhuZxxBdEiP5M9o=">AAACQXicbZDNS8MwGMZTP+f8mnr0EhzCdtnaOdCLMBTB4wS3FdYx0izdwtK0JqkwSv81L/4H3rx78aCIVy+mWxHdfCHw8Hvelzfv44aMSmWaz8bS8srq2npuI7+5tb2zW9jbb8sgEpi0cMACYbtIEkY5aSmqGLFDQZDvMtJxx5ep37knQtKA36pJSHo+GnLqUYyURv2C3SzZ0BmSO+h4AuH4hCdxPSlDh/2g2PGRGrkuvEpKdjnRLdV6As8zl1dr8ygFSb9QNCvmtOCisDJRBFk1+4UnZxDgyCdcYYak7FpmqHoxEopiRpK8E0kSIjxGQ9LVkiOfyF48TSCBx5oMoBcI/biCU/p7Ika+lBPf1Z3pMXLeS+F/XjdS3lkvpjyMFOF4tsiLGFQBTOOEAyoIVmyiBcKC6r9CPEI6BaVDz+sQrPmTF0W7VrHMinVTLzYusjhy4BAcgRKwwClogGvQBC2AwQN4AW/g3Xg0Xo0P43PWumRkMwfgTxlf3zaMrkY=</latexit><latexit sha1_base64="HcuvOFcpjVlkBhuZxxBdEiP5M9o=">AAACQXicbZDNS8MwGMZTP+f8mnr0EhzCdtnaOdCLMBTB4wS3FdYx0izdwtK0JqkwSv81L/4H3rx78aCIVy+mWxHdfCHw8Hvelzfv44aMSmWaz8bS8srq2npuI7+5tb2zW9jbb8sgEpi0cMACYbtIEkY5aSmqGLFDQZDvMtJxx5ep37knQtKA36pJSHo+GnLqUYyURv2C3SzZ0BmSO+h4AuH4hCdxPSlDh/2g2PGRGrkuvEpKdjnRLdV6As8zl1dr8ygFSb9QNCvmtOCisDJRBFk1+4UnZxDgyCdcYYak7FpmqHoxEopiRpK8E0kSIjxGQ9LVkiOfyF48TSCBx5oMoBcI/biCU/p7Ika+lBPf1Z3pMXLeS+F/XjdS3lkvpjyMFOF4tsiLGFQBTOOEAyoIVmyiBcKC6r9CPEI6BaVDz+sQrPmTF0W7VrHMinVTLzYusjhy4BAcgRKwwClogGvQBC2AwQN4AW/g3Xg0Xo0P43PWumRkMwfgTxlf3zaMrkY=</latexit><latexit sha1_base64="HcuvOFcpjVlkBhuZxxBdEiP5M9o=">AAACQXicbZDNS8MwGMZTP+f8mnr0EhzCdtnaOdCLMBTB4wS3FdYx0izdwtK0JqkwSv81L/4H3rx78aCIVy+mWxHdfCHw8Hvelzfv44aMSmWaz8bS8srq2npuI7+5tb2zW9jbb8sgEpi0cMACYbtIEkY5aSmqGLFDQZDvMtJxx5ep37knQtKA36pJSHo+GnLqUYyURv2C3SzZ0BmSO+h4AuH4hCdxPSlDh/2g2PGRGrkuvEpKdjnRLdV6As8zl1dr8ygFSb9QNCvmtOCisDJRBFk1+4UnZxDgyCdcYYak7FpmqHoxEopiRpK8E0kSIjxGQ9LVkiOfyF48TSCBx5oMoBcI/biCU/p7Ika+lBPf1Z3pMXLeS+F/XjdS3lkvpjyMFOF4tsiLGFQBTOOEAyoIVmyiBcKC6r9CPEI6BaVDz+sQrPmTF0W7VrHMinVTLzYusjhy4BAcgRKwwClogGvQBC2AwQN4AW/g3Xg0Xo0P43PWumRkMwfgTxlf3zaMrkY=</latexit><latexit sha1_base64="HcuvOFcpjVlkBhuZxxBdEiP5M9o=">AAACQXicbZDNS8MwGMZTP+f8mnr0EhzCdtnaOdCLMBTB4wS3FdYx0izdwtK0JqkwSv81L/4H3rx78aCIVy+mWxHdfCHw8Hvelzfv44aMSmWaz8bS8srq2npuI7+5tb2zW9jbb8sgEpi0cMACYbtIEkY5aSmqGLFDQZDvMtJxx5ep37knQtKA36pJSHo+GnLqUYyURv2C3SzZ0BmSO+h4AuH4hCdxPSlDh/2g2PGRGrkuvEpKdjnRLdV6As8zl1dr8ygFSb9QNCvmtOCisDJRBFk1+4UnZxDgyCdcYYak7FpmqHoxEopiRpK8E0kSIjxGQ9LVkiOfyF48TSCBx5oMoBcI/biCU/p7Ika+lBPf1Z3pMXLeS+F/XjdS3lkvpjyMFOF4tsiLGFQBTOOEAyoIVmyiBcKC6r9CPEI6BaVDz+sQrPmTF0W7VrHMinVTLzYusjhy4BAcgRKwwClogGvQBC2AwQN4AW/g3Xg0Xo0P43PWumRkMwfgTxlf3zaMrkY=</latexit><latexit sha1_base64="HcuvOFcpjVlkBhuZxxBdEiP5M9o=">AAACQXicbZDNS8MwGMZTP+f8mnr0EhzCdtnaOdCLMBTB4wS3FdYx0izdwtK0JqkwSv81L/4H3rx78aCIVy+mWxHdfCHw8Hvelzfv44aMSmWaz8bS8srq2npuI7+5tb2zW9jbb8sgEpi0cMACYbtIEkY5aSmqGLFDQZDvMtJxx5ep37knQtKA36pJSHo+GnLqUYyURv2C3SzZ0BmSO+h4AuH4hCdxPSlDh/2g2PGRGrkuvEpKdjnRLdV6As8zl1dr8ygFSb9QNCvmtOCisDJRBFk1+4UnZxDgyCdcYYak7FpmqHoxEopiRpK8E0kSIjxGQ9LVkiOfyF48TSCBx5oMoBcI/biCU/p7Ika+lBPf1Z3pMXLeS+F/XjdS3lkvpjyMFOF4tsiLGFQBTOOEAyoIVmyiBcKC6r9CPEI6BaVDz+sQrPmTF0W7VrHMinVTLzYusjhy4BAcgRKwwClogGvQBC2AwQN4AW/g3Xg0Xo0P43PWumRkMwfgTxlf3zaMrkY=</latexit>
❖ For distributions encountered in practice, Markov’s inequality gives a very loose bound.❖ Why?
Chebyshev’s Inequality
❖ Let X be a random variable. For every real number r>0,
P (|X � E(X)| � a) V (X)
a2<latexit sha1_base64="EmG2ec8vSIpiqKdxp47b53bNDhY=">AAACHXicbVDNS8MwHE3n15xfVY9egkPYDo52DPQ4FMHjBLcV1jrSLN3C0g+TVBhd/xEv/itePCjiwYv435huPejmg5DHe78fyXtuxKiQhvGtFVZW19Y3ipulre2d3T19/6Ajwphj0sYhC7nlIkEYDUhbUsmIFXGCfJeRrju+zPzuA+GChsGtnETE8dEwoB7FSCqprzdalakFT2Fi+0iOXBdepRWrOoX2kNxDVIU2U7ftcYSTjjLSBN3V075eNmrGDHCZmDkpgxytvv5pD0Ic+ySQmCEheqYRSSdBXFLMSFqyY0EihMdoSHqKBsgnwklm6VJ4opQB9EKuTiDhTP29kSBfiInvqsksg1j0MvE/rxdL79xJaBDFkgR4/pAXMyhDmFUFB5QTLNlEEYQ5VX+FeIRUFVIVWlIlmIuRl0mnXjONmnnTKDcv8jqK4AgcgwowwRlogmvQAm2AwSN4Bq/gTXvSXrR37WM+WtDynUPwB9rXD6nWn9E=</latexit><latexit sha1_base64="EmG2ec8vSIpiqKdxp47b53bNDhY=">AAACHXicbVDNS8MwHE3n15xfVY9egkPYDo52DPQ4FMHjBLcV1jrSLN3C0g+TVBhd/xEv/itePCjiwYv435huPejmg5DHe78fyXtuxKiQhvGtFVZW19Y3ipulre2d3T19/6Ajwphj0sYhC7nlIkEYDUhbUsmIFXGCfJeRrju+zPzuA+GChsGtnETE8dEwoB7FSCqprzdalakFT2Fi+0iOXBdepRWrOoX2kNxDVIU2U7ftcYSTjjLSBN3V075eNmrGDHCZmDkpgxytvv5pD0Ic+ySQmCEheqYRSSdBXFLMSFqyY0EihMdoSHqKBsgnwklm6VJ4opQB9EKuTiDhTP29kSBfiInvqsksg1j0MvE/rxdL79xJaBDFkgR4/pAXMyhDmFUFB5QTLNlEEYQ5VX+FeIRUFVIVWlIlmIuRl0mnXjONmnnTKDcv8jqK4AgcgwowwRlogmvQAm2AwSN4Bq/gTXvSXrR37WM+WtDynUPwB9rXD6nWn9E=</latexit><latexit sha1_base64="EmG2ec8vSIpiqKdxp47b53bNDhY=">AAACHXicbVDNS8MwHE3n15xfVY9egkPYDo52DPQ4FMHjBLcV1jrSLN3C0g+TVBhd/xEv/itePCjiwYv435huPejmg5DHe78fyXtuxKiQhvGtFVZW19Y3ipulre2d3T19/6Ajwphj0sYhC7nlIkEYDUhbUsmIFXGCfJeRrju+zPzuA+GChsGtnETE8dEwoB7FSCqprzdalakFT2Fi+0iOXBdepRWrOoX2kNxDVIU2U7ftcYSTjjLSBN3V075eNmrGDHCZmDkpgxytvv5pD0Ic+ySQmCEheqYRSSdBXFLMSFqyY0EihMdoSHqKBsgnwklm6VJ4opQB9EKuTiDhTP29kSBfiInvqsksg1j0MvE/rxdL79xJaBDFkgR4/pAXMyhDmFUFB5QTLNlEEYQ5VX+FeIRUFVIVWlIlmIuRl0mnXjONmnnTKDcv8jqK4AgcgwowwRlogmvQAm2AwSN4Bq/gTXvSXrR37WM+WtDynUPwB9rXD6nWn9E=</latexit><latexit sha1_base64="EmG2ec8vSIpiqKdxp47b53bNDhY=">AAACHXicbVDNS8MwHE3n15xfVY9egkPYDo52DPQ4FMHjBLcV1jrSLN3C0g+TVBhd/xEv/itePCjiwYv435huPejmg5DHe78fyXtuxKiQhvGtFVZW19Y3ipulre2d3T19/6Ajwphj0sYhC7nlIkEYDUhbUsmIFXGCfJeRrju+zPzuA+GChsGtnETE8dEwoB7FSCqprzdalakFT2Fi+0iOXBdepRWrOoX2kNxDVIU2U7ftcYSTjjLSBN3V075eNmrGDHCZmDkpgxytvv5pD0Ic+ySQmCEheqYRSSdBXFLMSFqyY0EihMdoSHqKBsgnwklm6VJ4opQB9EKuTiDhTP29kSBfiInvqsksg1j0MvE/rxdL79xJaBDFkgR4/pAXMyhDmFUFB5QTLNlEEYQ5VX+FeIRUFVIVWlIlmIuRl0mnXjONmnnTKDcv8jqK4AgcgwowwRlogmvQAm2AwSN4Bq/gTXvSXrR37WM+WtDynUPwB9rXD6nWn9E=</latexit>
Chebyshev’s Inequality
❖ Let X be a random variable. For every real number r>0,
How likely is it that RV X takes a valuethat’s at least distance a from its expected value?
P (|X � E(X)| � a) V (X)
a2<latexit sha1_base64="EmG2ec8vSIpiqKdxp47b53bNDhY=">AAACHXicbVDNS8MwHE3n15xfVY9egkPYDo52DPQ4FMHjBLcV1jrSLN3C0g+TVBhd/xEv/itePCjiwYv435huPejmg5DHe78fyXtuxKiQhvGtFVZW19Y3ipulre2d3T19/6Ajwphj0sYhC7nlIkEYDUhbUsmIFXGCfJeRrju+zPzuA+GChsGtnETE8dEwoB7FSCqprzdalakFT2Fi+0iOXBdepRWrOoX2kNxDVIU2U7ftcYSTjjLSBN3V075eNmrGDHCZmDkpgxytvv5pD0Ic+ySQmCEheqYRSSdBXFLMSFqyY0EihMdoSHqKBsgnwklm6VJ4opQB9EKuTiDhTP29kSBfiInvqsksg1j0MvE/rxdL79xJaBDFkgR4/pAXMyhDmFUFB5QTLNlEEYQ5VX+FeIRUFVIVWlIlmIuRl0mnXjONmnnTKDcv8jqK4AgcgwowwRlogmvQAm2AwSN4Bq/gTXvSXrR37WM+WtDynUPwB9rXD6nWn9E=</latexit><latexit sha1_base64="EmG2ec8vSIpiqKdxp47b53bNDhY=">AAACHXicbVDNS8MwHE3n15xfVY9egkPYDo52DPQ4FMHjBLcV1jrSLN3C0g+TVBhd/xEv/itePCjiwYv435huPejmg5DHe78fyXtuxKiQhvGtFVZW19Y3ipulre2d3T19/6Ajwphj0sYhC7nlIkEYDUhbUsmIFXGCfJeRrju+zPzuA+GChsGtnETE8dEwoB7FSCqprzdalakFT2Fi+0iOXBdepRWrOoX2kNxDVIU2U7ftcYSTjjLSBN3V075eNmrGDHCZmDkpgxytvv5pD0Ic+ySQmCEheqYRSSdBXFLMSFqyY0EihMdoSHqKBsgnwklm6VJ4opQB9EKuTiDhTP29kSBfiInvqsksg1j0MvE/rxdL79xJaBDFkgR4/pAXMyhDmFUFB5QTLNlEEYQ5VX+FeIRUFVIVWlIlmIuRl0mnXjONmnnTKDcv8jqK4AgcgwowwRlogmvQAm2AwSN4Bq/gTXvSXrR37WM+WtDynUPwB9rXD6nWn9E=</latexit><latexit sha1_base64="EmG2ec8vSIpiqKdxp47b53bNDhY=">AAACHXicbVDNS8MwHE3n15xfVY9egkPYDo52DPQ4FMHjBLcV1jrSLN3C0g+TVBhd/xEv/itePCjiwYv435huPejmg5DHe78fyXtuxKiQhvGtFVZW19Y3ipulre2d3T19/6Ajwphj0sYhC7nlIkEYDUhbUsmIFXGCfJeRrju+zPzuA+GChsGtnETE8dEwoB7FSCqprzdalakFT2Fi+0iOXBdepRWrOoX2kNxDVIU2U7ftcYSTjjLSBN3V075eNmrGDHCZmDkpgxytvv5pD0Ic+ySQmCEheqYRSSdBXFLMSFqyY0EihMdoSHqKBsgnwklm6VJ4opQB9EKuTiDhTP29kSBfiInvqsksg1j0MvE/rxdL79xJaBDFkgR4/pAXMyhDmFUFB5QTLNlEEYQ5VX+FeIRUFVIVWlIlmIuRl0mnXjONmnnTKDcv8jqK4AgcgwowwRlogmvQAm2AwSN4Bq/gTXvSXrR37WM+WtDynUPwB9rXD6nWn9E=</latexit><latexit sha1_base64="EmG2ec8vSIpiqKdxp47b53bNDhY=">AAACHXicbVDNS8MwHE3n15xfVY9egkPYDo52DPQ4FMHjBLcV1jrSLN3C0g+TVBhd/xEv/itePCjiwYv435huPejmg5DHe78fyXtuxKiQhvGtFVZW19Y3ipulre2d3T19/6Ajwphj0sYhC7nlIkEYDUhbUsmIFXGCfJeRrju+zPzuA+GChsGtnETE8dEwoB7FSCqprzdalakFT2Fi+0iOXBdepRWrOoX2kNxDVIU2U7ftcYSTjjLSBN3V075eNmrGDHCZmDkpgxytvv5pD0Ic+ySQmCEheqYRSSdBXFLMSFqyY0EihMdoSHqKBsgnwklm6VJ4opQB9EKuTiDhTP29kSBfiInvqsksg1j0MvE/rxdL79xJaBDFkgR4/pAXMyhDmFUFB5QTLNlEEYQ5VX+FeIRUFVIVWlIlmIuRl0mnXjONmnnTKDcv8jqK4AgcgwowwRlogmvQAm2AwSN4Bq/gTXvSXrR37WM+WtDynUPwB9rXD6nWn9E=</latexit>
Chebyshev’s Inequality: Proof
Chebyshev’s Inequality: ProofObserve that
P (|X � E(X)| � r) = P ((X � E(X))2 � r2)
Chebyshev’s Inequality: ProofObserve that
P (|X � E(X)| � r) = P ((X � E(X))2 � r2)
Applying Markov’s inequality, we get
P ((X � E(X))2 � r2) E((X � E(X))2)
r2=
V (X)
r2
Markov vs Chebyshev
P (X � kµ) 1
k<latexit sha1_base64="Dt7dwsm5ITLnxO4mtUy/zMlL8So=">AAACCHicdVDLSgMxFM34rPU16tKFwSLUTUl81l3RjcsK9gGdoWTSTBuaeZhkhDJ06cZfceNCEbd+gjv/xnQ6gooeuHA4517uvceLBVcaoQ9rZnZufmGxsFRcXlldW7c3NpsqSiRlDRqJSLY9opjgIWtorgVrx5KRwBOs5Q0vJn7rlknFo/Baj2LmBqQfcp9Too3UtXfq5TZ0+uwGDp0g2YeOMNTxJaEpHqfDcdcuocoxwmcnCKIKypCRKj7EEOdKCeSod+13pxfRJGChpoIo1cEo1m5KpOZUsHHRSRSLCR2SPusYGpKAKTfNHhnDPaP0oB9JU6GGmfp9IiWBUqPAM50B0QP125uIf3mdRPtVN+VhnGgW0ukiPxFQR3CSCuxxyagWI0MIldzcCumAmBS0ya5oQvj6FP5PmgcVjCr46qhUO8/jKIBtsAvKAINTUAOXoA4agII78ACewLN1bz1aL9brtHXGyme2wA9Yb59AVZjg</latexit><latexit sha1_base64="Dt7dwsm5ITLnxO4mtUy/zMlL8So=">AAACCHicdVDLSgMxFM34rPU16tKFwSLUTUl81l3RjcsK9gGdoWTSTBuaeZhkhDJ06cZfceNCEbd+gjv/xnQ6gooeuHA4517uvceLBVcaoQ9rZnZufmGxsFRcXlldW7c3NpsqSiRlDRqJSLY9opjgIWtorgVrx5KRwBOs5Q0vJn7rlknFo/Baj2LmBqQfcp9Too3UtXfq5TZ0+uwGDp0g2YeOMNTxJaEpHqfDcdcuocoxwmcnCKIKypCRKj7EEOdKCeSod+13pxfRJGChpoIo1cEo1m5KpOZUsHHRSRSLCR2SPusYGpKAKTfNHhnDPaP0oB9JU6GGmfp9IiWBUqPAM50B0QP125uIf3mdRPtVN+VhnGgW0ukiPxFQR3CSCuxxyagWI0MIldzcCumAmBS0ya5oQvj6FP5PmgcVjCr46qhUO8/jKIBtsAvKAINTUAOXoA4agII78ACewLN1bz1aL9brtHXGyme2wA9Yb59AVZjg</latexit><latexit sha1_base64="Dt7dwsm5ITLnxO4mtUy/zMlL8So=">AAACCHicdVDLSgMxFM34rPU16tKFwSLUTUl81l3RjcsK9gGdoWTSTBuaeZhkhDJ06cZfceNCEbd+gjv/xnQ6gooeuHA4517uvceLBVcaoQ9rZnZufmGxsFRcXlldW7c3NpsqSiRlDRqJSLY9opjgIWtorgVrx5KRwBOs5Q0vJn7rlknFo/Baj2LmBqQfcp9Too3UtXfq5TZ0+uwGDp0g2YeOMNTxJaEpHqfDcdcuocoxwmcnCKIKypCRKj7EEOdKCeSod+13pxfRJGChpoIo1cEo1m5KpOZUsHHRSRSLCR2SPusYGpKAKTfNHhnDPaP0oB9JU6GGmfp9IiWBUqPAM50B0QP125uIf3mdRPtVN+VhnGgW0ukiPxFQR3CSCuxxyagWI0MIldzcCumAmBS0ya5oQvj6FP5PmgcVjCr46qhUO8/jKIBtsAvKAINTUAOXoA4agII78ACewLN1bz1aL9brtHXGyme2wA9Yb59AVZjg</latexit><latexit sha1_base64="Dt7dwsm5ITLnxO4mtUy/zMlL8So=">AAACCHicdVDLSgMxFM34rPU16tKFwSLUTUl81l3RjcsK9gGdoWTSTBuaeZhkhDJ06cZfceNCEbd+gjv/xnQ6gooeuHA4517uvceLBVcaoQ9rZnZufmGxsFRcXlldW7c3NpsqSiRlDRqJSLY9opjgIWtorgVrx5KRwBOs5Q0vJn7rlknFo/Baj2LmBqQfcp9Too3UtXfq5TZ0+uwGDp0g2YeOMNTxJaEpHqfDcdcuocoxwmcnCKIKypCRKj7EEOdKCeSod+13pxfRJGChpoIo1cEo1m5KpOZUsHHRSRSLCR2SPusYGpKAKTfNHhnDPaP0oB9JU6GGmfp9IiWBUqPAM50B0QP125uIf3mdRPtVN+VhnGgW0ukiPxFQR3CSCuxxyagWI0MIldzcCumAmBS0ya5oQvj6FP5PmgcVjCr46qhUO8/jKIBtsAvKAINTUAOXoA4agII78ACewLN1bz1aL9brtHXGyme2wA9Yb59AVZjg</latexit>
vsP (|X � µ| � k�) 1
k2<latexit sha1_base64="dz4UVgVlLMEUmb1W6XtV9Jxh6mE=">AAACE3icdVDLSgMxFM3UV62vqks3wSJUwTIpPndFNy4r2Ad0xpJJM9PQZGZMMkKZ9h/c+CtuXCji1o07/8a0VqiiBy6cnHMvufd4MWdK2/aHlZmZnZtfyC7mlpZXVtfy6xt1FSWS0BqJeCSbHlaUs5DWNNOcNmNJsfA4bXi985HfuKVSsSi80v2YugIHIfMZwdpI7fxetTho7jsiGUAnoDew5ygWCLwLHW5eji8xSdEw7V2Xh+18wS7ZY8Apcmij0yME0UQpgAmq7fy704lIImioCcdKtZAdazfFUjPC6TDnJIrGmPRwQFuGhlhQ5abjm4Zwxygd6EfSVKjhWJ2eSLFQqi880ymw7qrf3kj8y2sl2j9xUxbGiaYh+frITzjUERwFBDtMUqJ53xBMJDO7QtLFJgdtYsyZEL4vhf+TermE7BK6PChUziZxZMEW2AZFgMAxqIALUAU1QMAdeABP4Nm6tx6tF+v1qzVjTWY2wQ9Yb5+g1p1k</latexit><latexit sha1_base64="dz4UVgVlLMEUmb1W6XtV9Jxh6mE=">AAACE3icdVDLSgMxFM3UV62vqks3wSJUwTIpPndFNy4r2Ad0xpJJM9PQZGZMMkKZ9h/c+CtuXCji1o07/8a0VqiiBy6cnHMvufd4MWdK2/aHlZmZnZtfyC7mlpZXVtfy6xt1FSWS0BqJeCSbHlaUs5DWNNOcNmNJsfA4bXi985HfuKVSsSi80v2YugIHIfMZwdpI7fxetTho7jsiGUAnoDew5ygWCLwLHW5eji8xSdEw7V2Xh+18wS7ZY8Apcmij0yME0UQpgAmq7fy704lIImioCcdKtZAdazfFUjPC6TDnJIrGmPRwQFuGhlhQ5abjm4Zwxygd6EfSVKjhWJ2eSLFQqi880ymw7qrf3kj8y2sl2j9xUxbGiaYh+frITzjUERwFBDtMUqJ53xBMJDO7QtLFJgdtYsyZEL4vhf+TermE7BK6PChUziZxZMEW2AZFgMAxqIALUAU1QMAdeABP4Nm6tx6tF+v1qzVjTWY2wQ9Yb5+g1p1k</latexit><latexit sha1_base64="dz4UVgVlLMEUmb1W6XtV9Jxh6mE=">AAACE3icdVDLSgMxFM3UV62vqks3wSJUwTIpPndFNy4r2Ad0xpJJM9PQZGZMMkKZ9h/c+CtuXCji1o07/8a0VqiiBy6cnHMvufd4MWdK2/aHlZmZnZtfyC7mlpZXVtfy6xt1FSWS0BqJeCSbHlaUs5DWNNOcNmNJsfA4bXi985HfuKVSsSi80v2YugIHIfMZwdpI7fxetTho7jsiGUAnoDew5ygWCLwLHW5eji8xSdEw7V2Xh+18wS7ZY8Apcmij0yME0UQpgAmq7fy704lIImioCcdKtZAdazfFUjPC6TDnJIrGmPRwQFuGhlhQ5abjm4Zwxygd6EfSVKjhWJ2eSLFQqi880ymw7qrf3kj8y2sl2j9xUxbGiaYh+frITzjUERwFBDtMUqJ53xBMJDO7QtLFJgdtYsyZEL4vhf+TermE7BK6PChUziZxZMEW2AZFgMAxqIALUAU1QMAdeABP4Nm6tx6tF+v1qzVjTWY2wQ9Yb5+g1p1k</latexit><latexit sha1_base64="dz4UVgVlLMEUmb1W6XtV9Jxh6mE=">AAACE3icdVDLSgMxFM3UV62vqks3wSJUwTIpPndFNy4r2Ad0xpJJM9PQZGZMMkKZ9h/c+CtuXCji1o07/8a0VqiiBy6cnHMvufd4MWdK2/aHlZmZnZtfyC7mlpZXVtfy6xt1FSWS0BqJeCSbHlaUs5DWNNOcNmNJsfA4bXi985HfuKVSsSi80v2YugIHIfMZwdpI7fxetTho7jsiGUAnoDew5ygWCLwLHW5eji8xSdEw7V2Xh+18wS7ZY8Apcmij0yME0UQpgAmq7fy704lIImioCcdKtZAdazfFUjPC6TDnJIrGmPRwQFuGhlhQ5abjm4Zwxygd6EfSVKjhWJ2eSLFQqi880ymw7qrf3kj8y2sl2j9xUxbGiaYh+frITzjUERwFBDtMUqJ53xBMJDO7QtLFJgdtYsyZEL4vhf+TermE7BK6PChUziZxZMEW2AZFgMAxqIALUAU1QMAdeABP4Nm6tx6tF+v1qzVjTWY2wQ9Yb5+g1p1k</latexit>
Chebyshev’s Inequality: An Example❖ Assume we have a distribution whose mean is 80 and
standard deviation is 10. What is a lower bound on the percentage of values that fall between 60 and 100 (exclusively) in this distribution?
Chebyshev’s Inequality: An Example❖ Assume we have a distribution whose mean is 80 and
standard deviation is 10. What is a lower bound on the percentage of values that fall between 60 and 100 (exclusively) in this distribution?
p(|X(s)� E(X)| � r) V
r2
Chebyshev’s Inequality: An Example❖ Assume we have a distribution whose mean is 80 and
standard deviation is 10. What is a lower bound on the percentage of values that fall between 60 and 100 (exclusively) in this distribution?
p(|X(s)� E(X)| � r) V
r2
E(X) = 80 V = 100 r = 20
Chebyshev’s Inequality: An Example❖ Assume we have a distribution whose mean is 80 and
standard deviation is 10. What is a lower bound on the percentage of values that fall between 60 and 100 (exclusively) in this distribution?
p(|X(s)� E(X)| � r) V
r2
E(X) = 80 V = 100 r = 20
p(|X(s)� 80| � 20) 1
4
Chebyshev’s Inequality: An Example❖ Assume we have a distribution whose mean is 80 and
standard deviation is 10. What is a lower bound on the percentage of values that fall between 60 and 100 (exclusively) in this distribution?
p(|X(s)� E(X)| � r) V
r2
E(X) = 80 V = 100 r = 20
p(|X(s)� 80| � 20) 1
4
) lower bound is 75%
Illustration: Estimating π Using the Monte Carlo Method
❖ Here’s a simple algorithm for estimating π:
❖ Throw darts at a square whose area is 1, inside which there’s a circle whose radius is 1/2.
❖ The probability that it lands inside the circle equals the ratio of the circle area to the square area (π/4). Therefore, calculate the proportion of times that the dart landed inside the circle and multiply it by 4.
r=1/2
(0.5,0.5)
Illustration: Estimating π Using the Monte Carlo Method
r=1/2
(0.5,0.5)
COMP 182: Algorithmic Thinking
Monte Carlo Estimation of ⇡
Luay Nakhleh
April 16, 2018
Algorithm 1: MonteCarlo ⇡Estimation.
Input: n 2 N.
Output: Estimate ⇡̂ of ⇡.
for i = 1 to n do
a random(0, 1); // random number in [0, 1]b random(0, 1); // random number in [0, 1]Xi 0;
ifp
(a� 0.5)2 + (b� 0.5)2 0.5 then
Xi 1; // the dart landed inside/on the circle
⇡̂ 4 · (Pn
i=1 Xi)/n;
return ⇡̂;
1
Illustration: Estimating π Using the Monte Carlo Method
❖ Let Xi be the random variable that denotes whether the i-th dart landed inside the circle (1 if it did, and 0 otherwise).
❖ Then, ⇡̂(n) = 4
Pni=1 Xi
n
Illustration: Estimating π Using the Monte Carlo Method
❖ Let Xi be the random variable that denotes whether the i-th dart landed inside the circle (1 if it did, and 0 otherwise).
❖ Then, ⇡̂(n) = 4
Pni=1 Xi
nE(Xi) =
⇡
41 + (1� ⇡
4)0 =
⇡
4
V (Xi) =⇡
4(1� ⇡
4)
Illustration: Estimating π Using the Monte Carlo Method
❖ Let Xi be the random variable that denotes whether the i-th dart landed inside the circle (1 if it did, and 0 otherwise).
❖ Then, ⇡̂(n) = 4
Pni=1 Xi
nE(Xi) =
⇡
41 + (1� ⇡
4)0 =
⇡
4
V (Xi) =⇡
4(1� ⇡
4)
Illustration: Estimating π Using the Monte Carlo Method
❖ Let Xi be the random variable that denotes whether the i-th dart landed inside the circle (1 if it did, and 0 otherwise).
❖ Then, ⇡̂(n) = 4
Pni=1 Xi
nE(Xi) =
⇡
41 + (1� ⇡
4)0 =
⇡
4
V (Xi) =⇡
4(1� ⇡
4)
V (⇡̂) = V (4
n
nX
i=1
Xi) =16
n2
nX
i=1
V (Xi) =⇡(4� ⇡)
n
Illustration: Estimating π Using the Monte Carlo Method
❖ The question of interest is: How big should n be for us to get a good estimate?
Illustration: Estimating π Using the Monte Carlo Method
❖ In a probabilistic setting, the question can be asked as:
❖ What should the value of n be so that the estimation error of π is within δ with probability at least ε?
❖ (of course, we want δ to be very small and ε to be as close to 1 as possible. For example, δ=0.001 and ε=0.95)
Illustration: Estimating π Using the Monte Carlo Method
❖ In other words, we are interested in the value of n that yields
p(|⇡̂(n)� ⇡| < �) > "
Illustration: Estimating π Using the Monte Carlo Method
❖ For δ=0.001 and ε=0.95, we seek n such that
Illustration: Estimating π Using the Monte Carlo Method
❖ For δ=0.001 and ε=0.95, we seek n such that
r V/r2Chebyshev’s inequality:
Illustration: Estimating π Using the Monte Carlo Method
❖ For δ=0.001 and ε=0.95, we seek n such that
r V/r2Chebyshev’s inequality:
Illustration: Estimating π Using the Monte Carlo Method
❖ For δ=0.001 and ε=0.95, we seek n such that
r V/r2Chebyshev’s inequality:
So, we would like n such that ⇡(4� ⇡)
n(0.001)2 0.05
Illustration: Estimating π Using the Monte Carlo Method
⇡(4� ⇡)
n(0.001)2 0.05
Illustration: Estimating π Using the Monte Carlo Method
⇡(4� ⇡)
n(0.001)2 0.05
⇡(4� ⇡) 4
Illustration: Estimating π Using the Monte Carlo Method
⇡(4� ⇡)
n(0.001)2 0.05
⇡(4� ⇡) 4
) ⇡(4� ⇡)
n(0.001)2 4
n(0.001)2 0.05
Illustration: Estimating π Using the Monte Carlo Method
⇡(4� ⇡)
n(0.001)2 0.05
⇡(4� ⇡) 4
) ⇡(4� ⇡)
n(0.001)2 4
n(0.001)2 0.05
) n � 80, 000, 000
A Corollary of Chebyshev’s Inequality
❖ Let X1,X2,…,Xn be independent random variables with
E(Xi) = µi<latexit sha1_base64="7/IG0VTfjvOMvDzaktxkxgmGyXo=">AAACAXicbVDLSsNAFJ3UV62vqBvBzWAR6qYkIuhGKIrgsoJ9QBPCZDpph85MwsxEKKFu/BU3LhRx61+482+ctFlo64ELh3Pu5d57woRRpR3n2yotLa+srpXXKxubW9s79u5eW8WpxKSFYxbLbogUYVSQlqaakW4iCeIhI51wdJ37nQciFY3FvR4nxOdoIGhEMdJGCuyDzONID8MQ3kxq3YCewEvo8TSggV116s4UcJG4BamCAs3A/vL6MU45ERozpFTPdRLtZ0hqihmZVLxUkQThERqQnqECcaL8bPrBBB4bpQ+jWJoSGk7V3xMZ4kqNeWg683PVvJeL/3m9VEcXfkZFkmoi8GxRlDKoY5jHAftUEqzZ2BCEJTW3QjxEEmFtQquYENz5lxdJ+7TuOnX37qzauCriKINDcARqwAXnoAFuQRO0AAaP4Bm8gjfryXqx3q2PWWvJKmb2wR9Ynz+mw5W2</latexit><latexit sha1_base64="7/IG0VTfjvOMvDzaktxkxgmGyXo=">AAACAXicbVDLSsNAFJ3UV62vqBvBzWAR6qYkIuhGKIrgsoJ9QBPCZDpph85MwsxEKKFu/BU3LhRx61+482+ctFlo64ELh3Pu5d57woRRpR3n2yotLa+srpXXKxubW9s79u5eW8WpxKSFYxbLbogUYVSQlqaakW4iCeIhI51wdJ37nQciFY3FvR4nxOdoIGhEMdJGCuyDzONID8MQ3kxq3YCewEvo8TSggV116s4UcJG4BamCAs3A/vL6MU45ERozpFTPdRLtZ0hqihmZVLxUkQThERqQnqECcaL8bPrBBB4bpQ+jWJoSGk7V3xMZ4kqNeWg683PVvJeL/3m9VEcXfkZFkmoi8GxRlDKoY5jHAftUEqzZ2BCEJTW3QjxEEmFtQquYENz5lxdJ+7TuOnX37qzauCriKINDcARqwAXnoAFuQRO0AAaP4Bm8gjfryXqx3q2PWWvJKmb2wR9Ynz+mw5W2</latexit><latexit sha1_base64="7/IG0VTfjvOMvDzaktxkxgmGyXo=">AAACAXicbVDLSsNAFJ3UV62vqBvBzWAR6qYkIuhGKIrgsoJ9QBPCZDpph85MwsxEKKFu/BU3LhRx61+482+ctFlo64ELh3Pu5d57woRRpR3n2yotLa+srpXXKxubW9s79u5eW8WpxKSFYxbLbogUYVSQlqaakW4iCeIhI51wdJ37nQciFY3FvR4nxOdoIGhEMdJGCuyDzONID8MQ3kxq3YCewEvo8TSggV116s4UcJG4BamCAs3A/vL6MU45ERozpFTPdRLtZ0hqihmZVLxUkQThERqQnqECcaL8bPrBBB4bpQ+jWJoSGk7V3xMZ4kqNeWg683PVvJeL/3m9VEcXfkZFkmoi8GxRlDKoY5jHAftUEqzZ2BCEJTW3QjxEEmFtQquYENz5lxdJ+7TuOnX37qzauCriKINDcARqwAXnoAFuQRO0AAaP4Bm8gjfryXqx3q2PWWvJKmb2wR9Ynz+mw5W2</latexit><latexit sha1_base64="7/IG0VTfjvOMvDzaktxkxgmGyXo=">AAACAXicbVDLSsNAFJ3UV62vqBvBzWAR6qYkIuhGKIrgsoJ9QBPCZDpph85MwsxEKKFu/BU3LhRx61+482+ctFlo64ELh3Pu5d57woRRpR3n2yotLa+srpXXKxubW9s79u5eW8WpxKSFYxbLbogUYVSQlqaakW4iCeIhI51wdJ37nQciFY3FvR4nxOdoIGhEMdJGCuyDzONID8MQ3kxq3YCewEvo8TSggV116s4UcJG4BamCAs3A/vL6MU45ERozpFTPdRLtZ0hqihmZVLxUkQThERqQnqECcaL8bPrBBB4bpQ+jWJoSGk7V3xMZ4kqNeWg683PVvJeL/3m9VEcXfkZFkmoi8GxRlDKoY5jHAftUEqzZ2BCEJTW3QjxEEmFtQquYENz5lxdJ+7TuOnX37qzauCriKINDcARqwAXnoAFuQRO0AAaP4Bm8gjfryXqx3q2PWWvJKmb2wR9Ynz+mw5W2</latexit>
Then, for any a>0:
and V (Xi) = �2i
<latexit sha1_base64="Ng+7wxss3urtRwLzwgf1CfQ2G6A=">AAAB/HicbVBNS8NAEJ3Ur1q/oj16WSxCvZSkCHoRil48VrAf0Maw2W7bpZtN2N0IIdS/4sWDIl79Id78N27bHLT1wcDjvRlm5gUxZ0o7zrdVWFvf2Nwqbpd2dvf2D+zDo7aKEkloi0Q8kt0AK8qZoC3NNKfdWFIcBpx2gsnNzO88UqlYJO51GlMvxCPBhoxgbSTfLrerXZ+doSvUV2wUYp891H274tScOdAqcXNSgRxN3/7qDyKShFRowrFSPdeJtZdhqRnhdFrqJ4rGmEzwiPYMFTikysvmx0/RqVEGaBhJU0Kjufp7IsOhUmkYmM4Q67Fa9mbif14v0cNLL2MiTjQVZLFomHCkIzRLAg2YpETz1BBMJDO3IjLGEhNt8iqZENzll1dJu15znZp7d15pXOdxFOEYTqAKLlxAA26hCS0gkMIzvMKb9WS9WO/Wx6K1YOUzZfgD6/MHmyKTcg==</latexit><latexit sha1_base64="Ng+7wxss3urtRwLzwgf1CfQ2G6A=">AAAB/HicbVBNS8NAEJ3Ur1q/oj16WSxCvZSkCHoRil48VrAf0Maw2W7bpZtN2N0IIdS/4sWDIl79Id78N27bHLT1wcDjvRlm5gUxZ0o7zrdVWFvf2Nwqbpd2dvf2D+zDo7aKEkloi0Q8kt0AK8qZoC3NNKfdWFIcBpx2gsnNzO88UqlYJO51GlMvxCPBhoxgbSTfLrerXZ+doSvUV2wUYp891H274tScOdAqcXNSgRxN3/7qDyKShFRowrFSPdeJtZdhqRnhdFrqJ4rGmEzwiPYMFTikysvmx0/RqVEGaBhJU0Kjufp7IsOhUmkYmM4Q67Fa9mbif14v0cNLL2MiTjQVZLFomHCkIzRLAg2YpETz1BBMJDO3IjLGEhNt8iqZENzll1dJu15znZp7d15pXOdxFOEYTqAKLlxAA26hCS0gkMIzvMKb9WS9WO/Wx6K1YOUzZfgD6/MHmyKTcg==</latexit><latexit sha1_base64="Ng+7wxss3urtRwLzwgf1CfQ2G6A=">AAAB/HicbVBNS8NAEJ3Ur1q/oj16WSxCvZSkCHoRil48VrAf0Maw2W7bpZtN2N0IIdS/4sWDIl79Id78N27bHLT1wcDjvRlm5gUxZ0o7zrdVWFvf2Nwqbpd2dvf2D+zDo7aKEkloi0Q8kt0AK8qZoC3NNKfdWFIcBpx2gsnNzO88UqlYJO51GlMvxCPBhoxgbSTfLrerXZ+doSvUV2wUYp891H274tScOdAqcXNSgRxN3/7qDyKShFRowrFSPdeJtZdhqRnhdFrqJ4rGmEzwiPYMFTikysvmx0/RqVEGaBhJU0Kjufp7IsOhUmkYmM4Q67Fa9mbif14v0cNLL2MiTjQVZLFomHCkIzRLAg2YpETz1BBMJDO3IjLGEhNt8iqZENzll1dJu15znZp7d15pXOdxFOEYTqAKLlxAA26hCS0gkMIzvMKb9WS9WO/Wx6K1YOUzZfgD6/MHmyKTcg==</latexit><latexit sha1_base64="Ng+7wxss3urtRwLzwgf1CfQ2G6A=">AAAB/HicbVBNS8NAEJ3Ur1q/oj16WSxCvZSkCHoRil48VrAf0Maw2W7bpZtN2N0IIdS/4sWDIl79Id78N27bHLT1wcDjvRlm5gUxZ0o7zrdVWFvf2Nwqbpd2dvf2D+zDo7aKEkloi0Q8kt0AK8qZoC3NNKfdWFIcBpx2gsnNzO88UqlYJO51GlMvxCPBhoxgbSTfLrerXZ+doSvUV2wUYp891H274tScOdAqcXNSgRxN3/7qDyKShFRowrFSPdeJtZdhqRnhdFrqJ4rGmEzwiPYMFTikysvmx0/RqVEGaBhJU0Kjufp7IsOhUmkYmM4Q67Fa9mbif14v0cNLL2MiTjQVZLFomHCkIzRLAg2YpETz1BBMJDO3IjLGEhNt8iqZENzll1dJu15znZp7d15pXOdxFOEYTqAKLlxAA26hCS0gkMIzvMKb9WS9WO/Wx6K1YOUzZfgD6/MHmyKTcg==</latexit>
P
�����
nX
i=1
Xi �nX
i=1
µi
����� � a
!Pn
i=1 �2i
a2<latexit sha1_base64="U6RSeLtoQEK/q0r1Qt3/RcXczuI=">AAACWHicbVFNaxsxENVukyZxv5z02IuoKaSHmt0QaC+BkFxydKFODJazaOXZ9RBJu5FmC2brP1nIofkrvVT+OOSjAxKP9+aNpKe81ugpSf5E8Yut7Zc7u3udV6/fvH3X3T+49FXjFAxVpSs3yqUHjRaGhKRhVDuQJtdwld+cL/Wrn+A8VvYHzWuYGFlaLFBJClTWrQZCQ0GH4gzL8pfwjclaPEkX13aUIf/CHzJcmCbDdScXJdxyyYXDckafeZhyy0XhpGofWzyWRmZ4fbRoZdiybi/pJ6viz0G6AT22qUHW/S2mlWoMWFJaej9Ok5omrXSESsOiIxoPtVQ3soRxgFYa8JN2FcyCfwrMlBeVC8sSX7EPHa003s9NHjqNpJl/qi3J/2njhopvkxZt3RBYtT6oaDSnii9T5lN0oEjPA5DKYbgrVzMZ0qHwF50QQvr0yc/B5VE/Tfrp9+Pe6dkmjl32gX1khyxlX9kpu2ADNmSK3bG/0Va0Hd3HLN6J99atcbTxvGePKj74B++is3U=</latexit><latexit sha1_base64="U6RSeLtoQEK/q0r1Qt3/RcXczuI=">AAACWHicbVFNaxsxENVukyZxv5z02IuoKaSHmt0QaC+BkFxydKFODJazaOXZ9RBJu5FmC2brP1nIofkrvVT+OOSjAxKP9+aNpKe81ugpSf5E8Yut7Zc7u3udV6/fvH3X3T+49FXjFAxVpSs3yqUHjRaGhKRhVDuQJtdwld+cL/Wrn+A8VvYHzWuYGFlaLFBJClTWrQZCQ0GH4gzL8pfwjclaPEkX13aUIf/CHzJcmCbDdScXJdxyyYXDckafeZhyy0XhpGofWzyWRmZ4fbRoZdiybi/pJ6viz0G6AT22qUHW/S2mlWoMWFJaej9Ok5omrXSESsOiIxoPtVQ3soRxgFYa8JN2FcyCfwrMlBeVC8sSX7EPHa003s9NHjqNpJl/qi3J/2njhopvkxZt3RBYtT6oaDSnii9T5lN0oEjPA5DKYbgrVzMZ0qHwF50QQvr0yc/B5VE/Tfrp9+Pe6dkmjl32gX1khyxlX9kpu2ADNmSK3bG/0Va0Hd3HLN6J99atcbTxvGePKj74B++is3U=</latexit><latexit sha1_base64="U6RSeLtoQEK/q0r1Qt3/RcXczuI=">AAACWHicbVFNaxsxENVukyZxv5z02IuoKaSHmt0QaC+BkFxydKFODJazaOXZ9RBJu5FmC2brP1nIofkrvVT+OOSjAxKP9+aNpKe81ugpSf5E8Yut7Zc7u3udV6/fvH3X3T+49FXjFAxVpSs3yqUHjRaGhKRhVDuQJtdwld+cL/Wrn+A8VvYHzWuYGFlaLFBJClTWrQZCQ0GH4gzL8pfwjclaPEkX13aUIf/CHzJcmCbDdScXJdxyyYXDckafeZhyy0XhpGofWzyWRmZ4fbRoZdiybi/pJ6viz0G6AT22qUHW/S2mlWoMWFJaej9Ok5omrXSESsOiIxoPtVQ3soRxgFYa8JN2FcyCfwrMlBeVC8sSX7EPHa003s9NHjqNpJl/qi3J/2njhopvkxZt3RBYtT6oaDSnii9T5lN0oEjPA5DKYbgrVzMZ0qHwF50QQvr0yc/B5VE/Tfrp9+Pe6dkmjl32gX1khyxlX9kpu2ADNmSK3bG/0Va0Hd3HLN6J99atcbTxvGePKj74B++is3U=</latexit><latexit sha1_base64="U6RSeLtoQEK/q0r1Qt3/RcXczuI=">AAACWHicbVFNaxsxENVukyZxv5z02IuoKaSHmt0QaC+BkFxydKFODJazaOXZ9RBJu5FmC2brP1nIofkrvVT+OOSjAxKP9+aNpKe81ugpSf5E8Yut7Zc7u3udV6/fvH3X3T+49FXjFAxVpSs3yqUHjRaGhKRhVDuQJtdwld+cL/Wrn+A8VvYHzWuYGFlaLFBJClTWrQZCQ0GH4gzL8pfwjclaPEkX13aUIf/CHzJcmCbDdScXJdxyyYXDckafeZhyy0XhpGofWzyWRmZ4fbRoZdiybi/pJ6viz0G6AT22qUHW/S2mlWoMWFJaej9Ok5omrXSESsOiIxoPtVQ3soRxgFYa8JN2FcyCfwrMlBeVC8sSX7EPHa003s9NHjqNpJl/qi3J/2njhopvkxZt3RBYtT6oaDSnii9T5lN0oEjPA5DKYbgrVzMZ0qHwF50QQvr0yc/B5VE/Tfrp9+Pe6dkmjl32gX1khyxlX9kpu2ADNmSK3bG/0Va0Hd3HLN6J99atcbTxvGePKj74B++is3U=</latexit>
The Law of Large Numbers
❖ Let X1,X2,…,Xn be independently and identically distributed (i.i.d.) random variables, where the (unknown) expected value μ is the same for all variables (that is, ) and their variance is finite. Then, for any ε>0, we have
E(Xi) = µ
P
�����
1
n
nX
i=1
Xi
!� µ
����� � "
!n!1����! 0
Questions?