new random variables and distributions · 2020. 9. 3. · probability and random variables discrete...

03 - RandomVariables

Random Variables

Probability andRandom Variables

Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs

Multiple RandomVariables

Independence

Covariance

References

References

Random Variables and Distributions

Brian Vegetabile

2017 Statistics BootcampDepartment of Statistics

University of California, Irvine

September 15th, 2016

1 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

A Few Definitions for Random Variables

Definition (Random Variable [Casella and Berger, 2002])

A random variable is a function from a sample space intothe real numbers.

2 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References


Definition (Random Variable [Schervish, 2014])

Consider an experiment for which the sample space isdenoted by S. A real-valued function that is defined on thespace S is called a random variable. In other words, in aparticular experiment a random variable X would be somefunction that assigns a real number X(s) to each possibleoutcome s ∈ S.

3 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References


Definition (Random Variable [Dudewicz and Mishra,1988])

A random variable X is a real-valued function with domainΩ [i.e. for each ω ∈ Ω, X(ω) ∈ R = y : −∞ < y < +∞].An n-dimensional random variable (or n-dimensionalrandom vector, or vector random variable),X = (X1, . . . , Xn) is a function with domain Ω and range inEuclidean n-space Rn. [i.e. for each ω ∈ Ω,X(ω) ∈ Rn =(y1, . . . , yn) : −∞ < yi < +∞, (1 ≤ i ≤ n)]

4 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Example - Random Variables

I We can consider a random variable X which is the sumobtained from rolling two fair dice.

I Each die is capable of taking on the valuesRolli = 1, 2, 3, 4, 5, 6, thus the sample space Ω is

Ω =

(1, 1) (1, 2) (1, 3) (1, 4) (1, 5) (1, 6)(2, 1) (2, 2) (2, 3) (2, 4) (2, 5) (2, 6)

......

......

......

(6, 1) (6, 2) (6, 3) (6, 4) (6, 5) (6, 6)

I And thus we can map values from the sample space

into the reals. For example,

X((1, 6)) = 7

5 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Idea of “Induced Probabilities” I

I We now show how probabilities can be created forrandom variables, the following is from [Casella andBerger, 2002]

I This definition will be investigated more in higher levelprobability and statistics courses

6 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Idea of “Induced Probabilities” II

I From Casella and Berger, consider the sample space Ω,with probability function P and we define a randomvariable X with range X .

I We can define a probability function PX on X in thefollowing way.

I We will observe X = xi, xi ∈ X if and only if theoutcome of the random experiment is an ωj ∈ Ω, suchthat X(ωj) = xi.

I Thus,

PX(X = xi) = P (ωj ∈ Ω : X(ωj) = xi)

7 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Example - ‘Induced’ Probabilities I

I We can consider a random variable X which is the sumobtained from rolling two fair dice.

I The sample space for the experiment isΩ = (i, j) : i, j = 1, 2, 3, 4, 5, 6.

I We have a probability function P , which describes thechance that each outcome in Ω is possible. That is

P ((i, j)) =1

36for any combination of i, j

since we are considering fair dice.

8 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Example - ‘Induced’ Probabilities II

I Consider the random variable X, such that for eachω ∈ Ω,

X(ω) = X((i, j)) = i+ j

Therefore X can take on the values2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12.

I Now using this fact, we can create probabilities for therandom variable X. For example, we can find theprobability that X = 4.

PX(X = 4) = P (ωj ∈ Ω : X(ωj) = 4)= P ((1, 3), (2, 2), (3, 1))= P ((1, 3)) + P ((2, 2)) + P ((3, 1))

=3

36=

1

12

9 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Example - ‘Induced’ Probabilities III

I Therefore using the original probability function on thesample space we can find probabilities for the randomvariable X.

I PX is an induced probability function on X .

10 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Discrete Random Variables

I Random variables which can only take on at most acountable number of possible values are considered tobe discrete random variables.

I Each one of these possible values will be assigned somenon-negative probability, therefore we can intuitively asplacing ‘mass’ of probability at each possible value.

11 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Discrete Random Variables

Definition (probability mass function)

If a random variable X is a discrete distribution (that is ittakes on only a countable number of different values) thenthe probability mass function of X is defined as the functionf such that for every real number x

f(x) = PX(X = x) for all x

Since f is a probability function it follows that f(x) ≥ 0 forall x and ∑

all x

f(x) = 1

12 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Example - Probability Mass Function for RollingTwo Dice

I Again consider the random variable X which is the sumobtained from rolling two fair dice.

I Clearly the random variable takes on a countablenumber of different values

I Thus we can construct the probability mass function forthis random variable.

2 4 6 8 10 12

0.00

0.05

0.10

0.15

0.20

Probability Mass Function − Rolling Two Dice

Experimental Outcome

Pro

babi

lity

Mas

s F

unct

ion

0.028

0.056

0.083

0.111

0.139

0.167

0.139

0.111

0.083

0.056

0.028

13 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Continuous Random Variables I

I Continuous random variables are concerned withprobability on intervals.

I From Degroot/Schervisch, a random variable X has acontinuous distribution, or is a continuous randomvariable, if there exists a non-negative function f ,defined on the real line, such that for every subset A ofthe real line, the probability that X takes a value in Ais the integral over the set A.

PX(X ∈ A) =

∫x∈A

f(x)dx

14 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Continuous Random Variables II

I When dealing with intervals (a, b] this becomes

PX(a < X ≤ b) =

∫ b

af(x)dx

I The function f is called the probability density functionof the random variable X. Again f(x) ≥ 0 for all x and∫ ∞

−∞f(x)dx = 1

I Additionally recall from calculus that∫ aa f(x)dx = 0 for

any x. This further implies that

PX(X = x) = 0

for a continuous random variable.

15 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Example - Continuous Random Variable I

I Consider a random variable that measures the timebetween events occurring in an interval of time.

I Notice this is a measure of time and therefore we willonly expect probability density on the non-negativereals.

I Such a random variable is called an exponential randomvariable and has the following density

f(x|λ) =

1λ exp

(−xλ

)if x ≥ 0

0 otherwise

16 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Example - Continuous Random Variable II

0 1 2 3 4 5

0.0

0.5

1.0

1.5

2.0

Various Exponential Distributions

x

Pro

babi

lity

Den

sity

λ = 0.5λ = 1λ = 3λ = 5

17 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

A Quick Aside:Observable Data vs. Unobservable Parameters

I Before continuing this is a good time to point out afundamental issue in statistics

I In our dice example, we are able to make assumptions,based on geometry or other things, which allow us toapriori know the probability distribution for eachexperimental outcome.

I In the continuous distribution example, notice theparameter λ in the distribution.

I It is often impossible to know these parametersbeforehand and thus our goal eventually will be toestimate these parameters.

I Often we have a set of “observed data” and we want tohelp us learn about the “unobservable parameters”

18 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Cumulative Distribution Functions I

I Once we understand the probability functions forrandom variables we can begin to talk about otherfunctions of, and properties of, the random variables.

I The first major function that is considered for everyrandom variable is its cumulative distribution. This isone of the first places that integration will come intoplay.

19 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Cumulative Distribution Functions II

Definition (Cumulative Distribution Function)

The distribution function FX of a random variable X is afunction defined for each real number x as follows:

F (x) = PX(X ≤ x) for all x

I Additionally based on our previous definitions of thedensity function this is equivalent to

F (x) = PX(X ≤ x) =

∫ x

−∞f(x)dx

20 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Relationship between Mass/Density FunctionsAnd Distribution Functions

I From the Fundamental Theorem of Calculus if therandom variable is continuous we have that

F ′(x) =d

dxF (x) = f(x)

21 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Properties of the CDF

i) The function FX(x) is nondecreasing as x increases;that is, if x1 < x2, then FX(x1) ≤ FX(x2).

ii) limx→−∞ FX(x) = 0 and limx→+∞ FX(x) = 1

iii) FX(x) is right continuous.

22 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Example - Discrete Random Variable CDF I

I Consider the function

f(x) =

0.1 if x = 00.2 if x = 10.7 if x = 20 otherwise

I Clearly this is a probability mass function sincef(x) ≥ 0 and

∑2i=0 f(x) = 1

I To find the distribution function must have a rightcontinuous function where we find P (X ≤ x) for all x

23 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Example - Discrete Random Variable CDF II

I Therefore, by the mass function we have thatP (X < 0) = 0, and we see that P (X ≤ 0) = 0.1.

I Now since the next ‘mass jump’ is at x = 1 thefunction remains constant up until that point and forexample P (X ≤ 0.5) = 0.1

I Once x = 1 though we add mass and haveP (X ≤ 1) = 0.3 and finally P (X ≤ 2) = 1.

I We can see how this is a ‘cumulative’ function.

24 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Example - Discrete Random Variable CDFI Continuing with our rolling two dice example we can

create the CDF of this distribution by consideredP (X ≤ x) for all x.

I Therefore we obtain the following function,

2 4 6 8 10 12

0.0

0.2

0.4

0.6

0.8

1.0

Cumulative Distribution Function − Rolling Two Dice


Cum

ulat

ive

Dis

trib

utio

n F

unct

ion

0.0280.083

0.167

0.278

0.417

0.583

0.722

0.8330.917

0.972

25 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Example - Continuous Random Variable CDF I

I For the exponential distribution we can integrate theprobability density function to obtain the cumulativedistribution function

I Recall f(x|λ) = 1λ exp

(−xλ

)and F (x) =

∫ x−∞ f(x)dx.

26 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Example - Continuous Random Variable CDF II

I Thus

F (x) =

∫ x

−∞f(x)dx) =

∫ x

0

1

λexp

(−xλ

)dx

= − exp(−xλ

)∣∣∣x0

= − exp(−xλ

)+ 1

= 1− exp(−xλ

)I We can use the CDF is this case to find the probability

that the time between events is less than some value(or greater than).

I This can inform us about rare event times.

27 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Example - Continuous Random Variable CDF III

0 2 4 6 8 10

0.0

0.2

0.4

0.6

0.8

1.0

Various Exponential CDFs

x

Cum

ulat

ive

Dis

trib

utio

n F

unct

ions

λ = 0.5λ = 1λ = 3λ = 5

28 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Expectations of Random Variables

I The distribution of a random variable X contains all ofthe probabilistic information about X.

I Often we summarize the random variable by certainmeasures, namely expectations, variances, means,modes, etc.

I We introduce the first such summary, the expectationof the random variable

29 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Defining the Expected Value

I This value is sometimes referred to as the mean of therandom variable.

Definition (Expected Value)

The expected value of a random X, denoted E(X) isdefined

E(X) =

∫ ∞−∞

xf(x)dx

For a discrete random variable this would become

E(X) =∑all x

xf(x)

30 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Example - Discrete Random Variable I

I We continue investigating our created randomvariables.

I As an intuition the expected value can be thought of asa “balancing point” of the distribution.

2 4 6 8 10 12

0.00

0.05

0.10

0.15

0.20

Probability Mass Function − Rolling Two Dice


Pro

babi

lity

Mas

s F

unct

ion

0.028

0.056

0.083

0.111

0.139

0.167

0.139

0.111

0.083

0.056

0.028

31 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Example - Discrete Random Variable II

I Performing our calculation

E(X) =∑x

xf(x) = 2× 0.028 + 3× 0.056 + . . .

= 7

I One thing worth noting is that It is NOT where themost mass is, as we will see in the continuous example.

32 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Example Expected Value of a ContinuousRandom Variable I

I For an exponential random variable, this can bethought of as the expected time between two eventsoccurring.

I Calculating...

E(X) =

∫ ∞−∞

xf(x)dx =

∫ ∞0

x

λexp

(−xλ

)

33 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Example Expected Value of a ContinuousRandom Variable II

I We can do this by integration by parts, where‘∫udv = uv −

∫vdu′ with the following

u =x

λdv = exp

(−xλ

)du =

1

λv = −λ exp

(−xλ

)

34 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Example Expected Value of a ContinuousRandom Variable III

I Thus,

E(X) =

∫ ∞0

x

λexp

(−xλ

)= −x exp

(−xλ

)∣∣∣∞0

+

∫ ∞0

exp(−xλ

)dx

= −λ exp(−xλ

)∣∣∣∞0

= λ

I As we saw from our earlier example, the maximum ofthe distribution occurs at 0, which does not correspondto the expected value of this distribution.

35 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

An Important Property of Expectations I

I We now present a useful relationship for expectations

I Claim: Y = aX + b, then E(Y ) = aE(X) + b.

36 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

An Important Property of Expectations II

I Claim: Y = aX + b, then E(Y ) = aE(X) + b.

I Proof:

E(Y ) = E(aX + b) =

∫(ax+ b)f(x)dx

= a

∫xf(x)dx+ b

∫f(x)dx

= aE(X) + b

37 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Generalizing Expectations of Random Variables

I We are not restricted to only finding the expected valuefor the random variable X, we can also find theexpected value of functions of the random variable X,that is some g(X).

Definition (Law of The Unconcious Statistician)

The expected value of a function of the random value X,g(X), denoted E(g(X)) is defined

E(g(X)) =

∫ ∞−∞

g(x)f(x)dx

For a discrete random variable this would become

E(g(X)) =∑all x

g(x)f(x)

38 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Variance of Random Variables I

I One function that is ultimately of interest is thevariance of a random variable

I The variance allows us to quantify the variability or thespread of a distribution.

Definition (Variance)

The variance of a random variable is defined as follows

V ar(X) = E[(X − E(X))2]

Often the expected value is defined as E(X) = µ andtherefore this is often written in textbooks as

V ar(X) = E[(X − µ)2]

39 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Variance of Random Variables II

The equation for variance can be simplified to the followinguseful relationship.

I Claim : V ar(X) = E(X2)− (E(X))2

40 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Variance of Random Variables III

I Proof:

V ar(X) = E[(X − µ)2]

= E[X2 − 2µX + µ2]

=

∫(x2 − 2µx+ µ2)f(x)dx

=

∫x2f(x)dx−

∫2µxf(x)dx+

∫µ2f(x)dx)

= E(X2)− 2µE(X) + µ2

= E(X2)− µ2

= E(X2)− (E(X))2

41 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

A Simple Property of Variance I

I Claim: V ar(aX + b) = a2V ar(X)

42 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

A Simple Property of Variance II

I Claim: V ar(aX + b) = a2V ar(X)

I Proof: We know E(aX + b) = aE(X) + b, thus

V ar(aX + b) = E((aX + b− E(aX + b))2)

= E((aX + b− aE(X) + b)2)

= E(a2(X − E(X))2

= a2E((X − E(X))2) = a2V ar(X)

43 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Moments of Random Variables

I Related to both the mean and variance of a randomvariable are it’s moments.

I Moments will become useful for simple estimationstrategies

Definition (Moment of a Random Variable)

For each random variable X and every positive integer k,E(Xk) is called the kth moment of X.

I We can also talk about Central Moments

Definition (Central Moment of a Random Variable)

Suppose that X is a RV such that E(X) = µ, then forevery positive integer k, E((X − µ)k) is called the kth

central moment of X.

44 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Moment Generating Functions

I Finally, one of the most important properties of arandom variable is its moment generating function.

I If two random variables have moment generatingfunctions that exist and are equal, then they have thesame distribution and similarly the converse is true.

I MGFs will be useful for showing convergence of randomvariables, they’re used tor finding the distribution of asum of independent and identically distributed randomvariables, and they play important role in the CentralLimit Theorem.

45 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Moment Generating Functions

Definition (Moment Generating Functions)

The moment-generating function of a random variable X isdefined for every real number t by MX(t) = E(etX).

I MGF’s can be used to ‘generate’ the moments of therandom variable in the following way:

EXn = MX(n)(0) ≡ dn

dtnMX(t)

∣∣∣∣t=0

46 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

MGF for the Exponential Distribution I

I Want to find E(etX)

E(etX) =

∫ ∞0

exp(tx)1

λexp

(−xλ

)dx

47 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

MGF for the Exponential Distribution II

E(etX) =

∫ ∞0

exp(tx)1

λexp

(−xλ

)dx

=

∫ ∞0

1

λexp

(tx− x

λ

)dx

=

∫ ∞0

1

λexp

(−x1− λt

λ

)dx

= − 1

λ

λ

1− λtexp

(−x1− λt

λ

)∣∣∣∣∞0

= 0 +1

1− λt

=1

1− λt

48 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Extending to Multiple Random Variables I

I Clearly, we will want to extend these concepts tomultiple random variables.

I For example, we may like to consider the jointdistribution for two random variables X and Y .

I Therefore we require a function f such that such thatfor every subset A of the sample space, the probabilitythat X and Y , the pair (x, y), takes a value in A, isthe integral over the set A.

P (X,Y ∈ A) =

∫∫Af(x, y)dxdy

49 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Extending to Multiple Random Variables II

I When we are considering two random variables, this isconsidered a bivariate distribution.

I When we consider more than two random variables thisis considered a multivariate distribution.

P (X1, X2, . . . Xn ∈ A)

=

∫. . .

∫∫Af(x1, x2, . . . xn)dx1dx2 . . . dxn

I Multivariate random variables have correspondingdistribution functions, expectations, moments, etc.

50 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Marginal Distributions

I We now consider properties of joint distributionfunctions, which we will illustrate mainly using bivariatedistributions.

I First we consider finding the marginal distribution ofrandom variables.

I Consider two random variables X and Y and their jointdistribution f(x, y).

I The marginal distribution of X and Y are given asfollows

fX(x) =

∫y∈R

f(x, y)dy fY (y) =

∫x∈R

f(x, y)dx

51 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Example - Joint Distribution I

I Consider the density

f(x, y) = x2(1 + y) +3

2y2 for 0 ≤ x ≤ 1 and 0 ≤ y ≤ 1

I We first ask is this a proper density?I Need f(x, y) ≥ 0 for any (x, y) ∈ Ω = [0, 1]× [0, 1]I Need

∫∫f(x, y)dxdy = 1

52 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Example - Joint Distribution II

I Taking derivatives(d

dxf(x, y),

d

dyf(x, y)

)=(2x(1 + y), x2 + 3y

)We see that the extreme values can be found at (0, 0)which occurs at a corner.

53 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Example - Joint Distribution III

I Additionally, plotting we see that the distribution isstrictly increasing

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

0.5

1

1.5 2 2.5 3

54 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Example - Joint Distribution IV

I Now interested in∫∫

f(x, y)dxdy = 1

∫ 1

0

∫ 1

0x2(1 + y) +

3

2y2dydx =

∫ 1

0x2(y +

y2

2

)+

3

2

y3

3

∣∣∣∣10

dx

=

∫ 1

0

3

2x2 +

1

2dx

=3

2

x3

3+

1

2x

∣∣∣∣10

=1

2+

1

2= 1

55 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Example - Joint Distribution V

I Now let us find the marginal distributions

f(y) =

∫ 1

0x2(1 + y) +

3

2y2dx

=x3

3(1 + y) +

3

2y2x

∣∣∣∣10

=1

3(1 + y) +

3

2y2

f(x) =

∫ 1

0x2(1 + y) +

3

2y2dy

= x2(y +

y2

2

)+y3

2

∣∣∣∣10

=3

2x2 +

1

2

56 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Conditioning on Random Variables

I Once we have a way to calculate the marginaldistribution of a random variable, we can begin to thinkof conditioning on random variables.

I This is similar to conditioning in probability that welearned about.

Definition (Conditional Density Function)

Consider two random variables X and Y , we define theconditional density of X given Y = y denoted X|Y = y tobe

fX|Y=y(x|y) =f(x, y)

fY (y)

I Conditional distributions will become increasinglyimportant in Bayesian statistics

57 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Independence between Random Variables

I Consider random variables X1, X2, . . . , Xn, we say therandom variables are independent if

f(x1, x2, . . . xn) = fX1(x1)fX2(x2) . . . fXn(xn)

I That is random variables are independent if we canfactorize the joint distribution into a product of theindividual marginal distributions.

I If each fXi(·) is the same, we say the they areidentically distributed.

I Independence and identical distributions will come intouse for describing random samples

58 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Expectations of Joint Distributions

I Similar to the univariate case, we can discussexpectations in terms of the joint distribution.

I For a bivariate distribution we can obtain the followingrelationships

E(X) =

∫ ∫xf(x, y)dxdy =

∫xf(x)dx

E(Y ) =

∫ ∫yf(x, y)dxdy =

∫yf(y)dy

E(XY ) =

∫ ∫xyf(x, y)dxdy

I Additionally, if X and Y are independent we can seethat E(XY ) = E(X)E(Y )

59 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Covariance & Correlation between RandomVariables

I Summaries of the random variables allow us tounderstand how they variables behave (expectations),but additionally we would like to know how theirbehavior is related

I This is especially true if the random variables are notindependent.

I Both the covariance and correlation between randomvariables attempts to measure the dependence betweenrandom variables

60 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Defining Covariance between Random Variables I

I Covariance allows us to begin to talk about thetendency of two variables to vary together rather thanindependently.

I If greater values of one random variable correspond togreater values of the other random variable (similar forsmaller values), then the covariance will be positive.

I If greater values of one random variable correspond tolesser values of another random variable, then thecovariance will be negative.

I The magnitude of covariance is often uninterpretable

61 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Defining Covariance between Random VariablesII

Definition

Covariance Consider the random variables X and Y and letE(X) = µX and E(Y ) = µY we define the covariancebetween X and Y to be

Cov(X,Y ) = E((X − µX)(Y − µY ))

I Similar to the univariate case, it can be shown that theabove reduces to

Cov(X,Y ) = E(XY )− E(X)E(Y )

which is much easier to work with in application.

62 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Correlation between Random Variables I

I As stated in the previous slides, the magnitude ofcovariance is not helpful in understanding how torandom variables values are related.

I This is due to the fact that covariance is sensitive tothe relative magnitudes of X and Y .

I To remedy this, the notion of correlation is not drivenby arbitrary changes in the scales of the randomvariables.

63 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

Correlation between Random Variables II

Definition (Correlation)

Consider the random variables X and Y , with finite positivevariances V ar(X) and V ar(Y ), then the correlationbetween X and Y is given as follows

ρ(X,Y ) =Cov(X,Y )

V ar(X)V ar(Y )

I It can be shown that the correlation is restrictedbetween −1 and 1 thus

−1 ≤ ρ(X,Y ) ≤ 1

64 / 65


Random Variables


Discrete RVs

Continuous RVs

CDFs

Expectations

Moments

MGFs


Independence

Covariance

References

References

References

George Casella and Roger L Berger. Statistical inference.Second edition, 2002.

EJ Dudewicz and SN Mishra. Modern mathematical statis-tics. john wilsey & sons. Inc., West Sussex, 1988.

Mark J Schervish. Probability and Statistics. 2014.

65 / 65

new random variables and distributions · 2020. 9. 3. · probability and random variables discrete...

Documents