Introduction to Probability and Statistics

Probability & Statistics for Engineers & Scientists, 9th Ed.

2010

Handout #2

Instructor: Lingzhou Xue

TA: Daniel Eck

1

Chapter 3

Random Variables

and

Probability Distributions

2

• Concept of a Random Variable

• Discrete Probability Distributions

• Continuous Probability Distributions

• Joint Probability Distribution

3

3.1 Concept of a Random Variables

4

Random variable

A random variable is a numerical description of the outcome

of an experiment. Each experimental outcome gets assigned

a numerical value. In fact, most of the time the experimental

outcome is a number so we just use that number as the number

we assign.

Example 1

Roll a dice. Suppose X = i if the outcome of the throw is

number i. That is, X = 1 represents the ”1” showing up. Then

X is a random variable.

5

Random variable

A random variable is a function that associates a real number

with each element in the sample space. So there is a function

such that

X : S → R

This statement is not 100% mathematically accurate, because

what a random variable does is to take subsets of S and maps

them to the real line R, but for us, this definition will suffice.

Example 2

Roll a dice. Suppose X = 1 if the outcome of the throw is an

even number, i.e., one of = {2, 4, 6}; and X = 0 otherwise.

Then X is a random variable.6

Random variables are interesting because probability calculations

are streamlined with these. For each random variable, we have to

know the range of values that it can take on the real line, and the

probability to be assigned to every point in this range. The way

these probabilities stated is called a probability distribution of a

random variable. Random variables can be discrete if the range

of values taken is a subset of integers, or it can be continuous.

7

Discrete Sample Space

If a sample space contain a finite number of a possibilities or

an unending sequence with as many elements as there are whole

numbers, it is called a discrete sample space.

Continuous Sample Space

If a sample space contain a infinite number of a possibilities

equal to the number of points on a line segment, it is called a

continuous sample space.

8

Discrete Random Variable

A random variable is called a discrete random variable if its

set of possible outcomes is countable.

Continuous Random Variable

When a random variable can take on values on a continuous

scale, it is called a continuous random variable.

9

Example 3: Discrete

Two balls are drawn in succession without replacement from anurn containing 4 red balls and 3 black balls. The possible out-comes and the values y of the random variable Y , where Y is thenumber of red balls are

Sample Space yRR 2RB 1BR 1BB 0

Example 4: Continuous

Let X be the random variable defined by the waiting time, inminute, for a bus at a specific bus stop. The random variable X

takes on all values x for which x ≥ 0.

10

3.2 Discrete Probability Distributions

11

Frequently, it is convenient to represent all the probabilities of

a random variable X by a formula. Such a formula would nec-

essarily be a function of the numerical values f(x) = P (X = x);

for instance, f(3) = P (X = 3).

Probability Function

For random variable X of the discrete type, the probability P (X =

x) is frequently denoted by f(x), and this function f(x) is called

the probability function, probability mass function or prob-

ability distribution.

12

Example 5

In the case of tossing a coin twice, the random variable X, repre-

sents the number of heads. The possible value x of X and their

probabilities are

x 0 1 2

f(x) = P (X = x) 14

24

14

13

Definition

The probability function, probability mass function or prob-

ability distribution, f(x), of a discrete random variable X if,

for each possible outcomes x, it satisfies the following properties:

1. f(x) ≥ 0,

2.∑

x f(x) = 1,

3. P (X = x) = f(x).

14

Example 6

Determine the value c so that each of the following functions

can serve as a probability distribution of the random variable X:

1. f(x) = c(x2 + 1), for x = 1, 3, 5.

2. f(x) = c(5− x2), for 0 ≤ x ≤ 2.

15

Solution:

16

Example 7

Let a random experiment be the cast of a pair dice, each having

six faces, and let the random variable X denote the sum of the

dice. Determine the probability function f(x) of X.

17

Solution:

18

Example 8

A shipment of 8 similar microcomputers to a retail outlet con-

tains 3 that are defective. If a school makes a random purchase

of 2 of these computers, find the probability distribution for the

number of defectives.

Solution:

Let X be a random variable whose values x are the possible num-

bers of defective computers purchased by the school. Then x

can be any of the numbers 0, 1, and 2. Now,

19

f(0) = P (X = 0) =

(30

)(52

)(

82

) =10

28,

f(1) = P (X = 1) =

(31

)(51

)(

82

) =15

28,

f(2) = P (X = 2) =

(32

)(50

)(

82

) =3

28.

Thus the probability distribution of X is

x 0 1 2f(x) = P (X = x) 10

281528

328

Goal

Wish to compute the probability that the observed value of a

random variable X will be less than or equal to some real

number x.

Cumulative Distribution Function: Discrete

The cumulative distribution function (c.d.f.) F (x) of a dis-

crete random variable X with probability distribution f(x) is

F (x) = P (X ≤ x) =∑t≤x

f(t), for −∞ < x <∞.

20

Example 5 Cont’d

In the case of tossing a coin twice, the random variable X repre-

sents the number of heads. The possible value x of X and their

probabilities are

x 0 1 2

f(x) = P (X = x) 14

24

14

Find the cumulative distribution function of the random variableX in Example 5.

F (0) = P (X ≤ 0) = f(0) =1

4,

F (1) = P (X ≤ 1) = f(0) + f(1) =1

4+

2

4=

3

4,

F (2) = P (X ≤ 2) = f(0) + f(1) + f(2) =1

4+

2

4+

1

4= 1.

21

F (x) =

0, for x < 0;14, for 0 ≤ x < 1;34, for 1 ≤ x < 2;1, for x ≥ 2.

f(2) = F (2)− F (1) = 1−3

4=

1

4.

22

23

Example 8 Cont’d

Find the cumulative distribution function of the random variable

X in Example 8. Then using F (x), verify that f(1) = 1528.

F (x) =

0, x < 0;1028, 0 ≤ x < 1;2528, 1 ≤ x < 2;1, x ≥ 2.

24

f(1) = P (X = 1) = F (1)− F (0) =15

28

Example 9: Finding Probability Using CDF

The cumulative distribution function of X is given by

F (x) =

0, x < 1;15, 1 ≤ x < 5;13, 5 ≤ x < 7;35, 7 ≤ x < 10;1, x ≥ 10.

Find

1. P (X = 6);

2. P (X > 7);

3. P (2.5 < X < 9.2).

25

26

Solution:

f(x) = P (X = x) =

15, if x = 1;2

15, if x = 5;4

15, if x = 7;25, if x = 10.

1. P (X = 6) = 0.

2. P (X > 7) = 1− P (X ≤ 7) = 1− F (7) = 25.

3. P (2.5 < X < 9.2) = P (X = 5) + P (X = 7) = 25.

27

Definition of the Integral as an Anti-Derivative

If the derivative of F (x) is f(x), then we say that an indefinite

integral of f(x) with respect to x is F (x).

d

dx[F (x)] = f(x)⇒

∫f(x) = F (x) + c,

where c is the arbitrary constant.

Example 10

d

dxx4 = 4x3, so

∫4x3 = x4 + c

28

Example 11

Let f(x) be the derivative of F (x), what is the derivative of the

following integral ∫ x

af(u)du

Solution: ∫ x

af(u)du = F (x)− F (a)

⇒d

dx[∫ x

af(u)du] =

d

dxF (x)−

d

dxF (a) = f(x)

29

Indefinite Integral Formulas

1.∫

cf(x)dx = c∫

f(x)dx + constant.

2.∫

[af(x) + bg(x)] dx = a∫

f(x)dx + b∫

g(x)dx + constant.

3.∫

xndu = 1n+1xn+1 + constant, if n 6= −1.

4.∫ 1

x = ln |x|+ constant.

5.∫

exdu = ex + constant.

30

3.3 Continuous Probability Distributions

31

Definition

The function f(x) is a probability density function (p.d.f.) for

the continuous random variable X, defined over the set of real

number R, if

1. f(x) ≥ 0, for all x ∈ R.

2.∫∞−∞ f(x) = 1,

3. P (a < X < b) =∫ ba f(x)dx.

32

Cumulative Distribution Function: Continuous

The cumulative distribution function (c.d.f.) F (x) of a ran-

dom variable X with probability distribution f(x) is

F (x) = P (X ≤ x) =∫ x

−∞f(t)dt, −∞ < x <∞.

As an immediate consequence in the previous definition one can

write the two results,

P (a < X < b) = F (b)− F (a), and f(x) =dF (x)

dx,

if the derivative exists.

33

Example 12

Suppose that the error in the reaction temperature, in ◦C, for acontrolled laboratory experiment is a continuous random variableX having the probability density function

f(x) =

{x2

3 , −1 < x < 2;0, elsewhere.

1. Verify f(·) is a probability density function for continuousrandom variable X

2. Find P (0 < X ≤ 1).

3. Find the cumulative distribution function, F (x), of X anduse it to evaluate P (0 < X ≤ 1).

34

Solution:

1. ∫ ∞−∞

f(x)dx =∫ 2

−1

x2

3dx =

x3

9|2−1 =

8

9+

1

9= 1.

2. For −1 < x < 2,

F (x) =∫ x

−∞f(t)dt =

∫ x

−1

t2

3dt =

t3

9|x−1 =

x3 + 1

9.

Therefore,

F (x) =

0, x < −1;x3+1

9 , −1 ≤ x < 2;1, x ≥ 2.

3.

P (0 < X ≤ 1) = F (1)− F (0) =2

9−

1

9.

A continuous random variable has a probability of zero ofassuming exactly any of its values. (It is because

∫ aa f(x)dx =

0.) So, when X is continuous

P (a < X ≤ b) = P (a < X < b) + P (X = b) = P (a < X < b).

That is, it doesn’t matter whether we include an endpoint of theinterval or not. This is not true, though, when X is discrete.

35

Example 13

The Department of Energy (DOE) puts projects out on bid and

generally estimates what a reasonable bid should be. Call the

estimate b. The DOE has determined that the density function

of winning (low) bid is

f(y) =

{58b,

2b5 < y < 2b;

0, elsewhere.

Find F (y) and use it to determine the probability that the winning

bid is less than the DOE’s preliminary estimate b.

36

Solution:

For 2b5 < y < 2b

F (y) =∫ y

2b/5

5

8bdt =

5t

8b|y2b/5 =

5y

8b−

1

4.

Thus

F (y) =

0, if y < 2b

5 ;5y8b −

14, if 2b

5 ≤ y < 2b;1, if y ≥ 2b.

To determine the probability that the winning bid is less than

the preliminary bid estimate b, we have

P (Y ≤ b) = F (b) =5

8−

1

4=

3

8.

37

Example 14: Challenging Problem

1. Sketch the graph of the function

f(x) =

{12 −

14|x− 3|, if 1 ≤ x ≤ 5;

0, otherwise,

and show that it is the probability density function of a ran-

dom variable X.

2. Find the cumulative distribution function, F , of X.

38

Solution:

1.

f(x) = P (X = x) =

0, if x < 1;12 + 1

4(x− 3), if 1 ≤ x < 3;12 −

14(x− 3), if 3 ≤ x < 5;

0, if x ≥ 5.

39

2. To calculate the c.d.f. of X, we use the formula F (x) =∫ x−∞ f(t)dt.

For x < 1

F (x) =∫ x

−∞f(t)dt =

∫ x

−∞0dt = 0.

For 1 ≤ x < 3

F (x) =∫ x

−∞f(t)dt =

∫ x

1

[1

2+

1

4(t− 3)

]dt =

1

8x2 −

1

4x +

1

8.

For 3 ≤ x < 5

F (x) =∫ x

−∞f(t)dt

=∫ 3

1

[1

2+

1

4(t− 3)

]dt +

∫ x

3

[1

2−

1

4(t− 3)

]dt

=1

2+(−

1

8x2 +

5

4x−

21

8

)= −

1

8x2 +

5

4x−

17

8.

For x ≥ 5

F (x) =∫ x

−∞f(t)dt

=∫ 3

1

1

2+

1

4(t− 3)dt +

∫ 5

3

1

2+

1

4(t− 3)dt

= 1.

F (x) =

0, if x < 1;18x2 − 1

4x + 18, if 1 ≤ x < 3;

−18x2 + 5

4x− 178 , if 3 ≤ x < 5;

1, if x ≥ 5.

3.4 Joint Probability Distributions

40

Joint Probability Distribution

If X and Y are two random variables, the probability distribu-

tion for their simultaneous occurrence can be represented by a

function with values f(x, y) for any pair of values (x, y). It is

customary to refer to this function as the joint probability dis-

tribution of X and Y .

41

Definition: Discrete

The probability function f(x, y) of a joint probability distribu-

tion of the discrete random variables X and Y if

1. f(x, y) ≥ 0, for all (x, y),

2.∑

x∑

y f(x, y) = 1,

3. P (X = x, Y = y) = f(x, y).

For any region A in xy plane, P [(X, Y ) ∈ A] =∑∑

A f(x, y)

42

Example 15

Two balls are selected from a box that contains 3 blue balls, 2

red balls, and 3 green balls. If X is the number of blue balls and

Y is the number of red balls selected, find

1. the joint probability function f(x, y),

2. P [(X, Y ) ∈ A], where A is the region {(x, y)|x + y ≤ 1}.

43

Solution:

1. The possible pairs of values (x, y) are (0, 0), (0, 1), (1, 0),(1, 1), (0, 2), and (2, 0). Now

f(1, 1) =6

28:

represents the probability that a red and a green ball are slected.

• The total number of equally likely ways of selecting 2 balls

from the 8 is

(82

)= 28.

• The number of ways of selecting 1 red and 1 green ball is(21

)(31

)(30

)= 6

The joint probability distribution of (X, Y ) can be represented

44

by the formula

f(x, y) = P (X = x, Y = y) =

(3x

)(2y

)(3

2− x− y

)(

82

)for x = 0, 1, 2; y = 0, 1, 2; and 0 ≤ x + y ≤ 2.

xf(x, y) 0 1 2 h(y)

y0 3

289

283

281528

1 628

628 0 12

282 1

28 0 0 128

g(x) 1028

1528

328 1

2.

P [(X, Y ) ∈ A] = P (X+Y ≤ 1) = f(0, 0)+f(1, 0)+f(0, 1) =9

14

Definition: Continuous

The probability function f(x, y) of a joint probability distribu-

tion of the continuous random variables X and Y if

1. f(x, y) ≥ 0, for all (x, y),

2.∫∞−∞

∫∞−∞ f(x, y) = 1,

3. P [(X, Y ) ∈ A] =∫ ∫

A f(x, y)dxdy, for any region A in xy plane.

45

Example 16

A candy company distributes boxes of chocolates with a mixture

of creams, toffees, and nuts coated in both light and dark choco-

late. For a randomly selected box, let X and Y , respectively, be

the proportions of the light and dark chocolates that are creams

and suppose that the joint density function is

f(x, y) =

{25(2x + 3y), 0 ≤ x ≤ 1, 0 ≤ y ≤ 1;0, elsewhere.

1. Verify f(x, y) is a joint density function.

2. Find P [(X, Y ) ∈ A], where A = {(x, y)|0 < x < 12, 1

4 < y < 12}.

46

Solution:

1. ∫ ∞−∞

∫ ∞−∞

f(x, y)dxdy =∫ 1

0

∫ 1

0

2

5(2x + 3y)dxdy

=∫ 1

0

2

5(x2 + 3yx)|x=1

x=0dy

=∫ 1

0

2

5(1 + 3y)dy

=2

5(y +

3

2y2)|y=1

y=0

=1

2.

P [(X, Y ) ∈ A] = P (0 < X <1

2,1

4< Y <

1

2) =

13

160.

47

Marginal Distribution

Given two jointly distributed random variables X and Y , the

marginal distribution of X is simply the probability distribution

of X ignoring information about Y .

The marginal distributions of X alone and of Y alone are

Discrete case:

g(x) =∑y

f(x, y) and h(y) =∑x

f(x, y),

Continuous case:

g(x) =∫ ∞−∞

f(x, y)dy and h(y) =∫ ∞−∞

f(x, y)dx,

48

Example 14 Cont’d

x 0 1 2

g(x) 1028

1528

328

y 0 1 2

h(y) 1528

1228

128

Solution:

g(0) =∑y

f(0, y) = f(0, 0) + f(0, 1) + f(0, 2) =5

14,

g(1) =∑y

f(1, y) = f(1, 0) + f(1, 1) + f(1, 2) =15

28,

g(2) =∑y

f(2, y) = f(2, 0) + f(2, 1) + f(2, 2) =3

28.

49

Example 16 Cont’d

g(x) =∫ ∞−∞

f(x, y)dy =∫ 1

0

2

5(2x + 3y)dy =

{4x+3

5 , 0 ≤ x ≤ 1;0, elsewhere.

h(y) =∫ ∞−∞

f(x, y)dx =∫ 1

0

2

5(2x + 3y)dx =

{2+6y

5 , 0 ≤ y ≤ 1;0, elsewhere.

The marginal distributions g(x) and h(y) are indeed the proba-

bility distributions of the individual variables X and Y alone.∫ ∞−∞

g(x) =∫ ∞−∞

∫ ∞−∞

f(x, y)dydx = 1,

and

P (a < X < b) = P (a < X < b,−∞ < Y <∞)

=∫ b

a

∫ ∞−∞

f(x, y)dydx =∫ b

ag(x)dx

50

Conditional Distribution

Let X and Y be two random variables, discrete or continuous.

Suppose f(x, y) is the joint probability density function of X and

Y , and g(x) and h(y) are the marginal distributions for X and

Y , respectively. The conditional distribution of the random

variable Y given that X = x is

f(y|x) =f(x, y)

g(x), g(x) > 0,

Similarly the conditional distribution of the random variable X

given that Y = y is

f(x|y) =f(x, y)

h(y), h(y) > 0.

51

Conditional Distribution

If we wish to find the probability that the discrete random variable

X fall between a and b when it known that the discrete variable

Y = y, we evaluate

P (a < X < b|Y = y) =∑

a<x<b

f(x|y),

where the summation extends over all values of X between a and

b.

When X and Y are continuous, we evaluate

P (a < X < b|Y = y) =∫ b

af(x|y)dx.

52

Example 15 Cont’d

Find the conditional distribution of X, given that Y = 1, and use

it to determine P (X = 0|Y = 1).

Solution:

h(1) =2∑

x=0

f(x, 1) =3

14+

3

14+ 0 =

6

14.

f(x|1) =f(x, y)

h(1)=

7

3f(x, 1), for x = 0, 1, 2.

Therefore, if it is know that 1 of the 2 balls selected is red, we

have the probability equal to 12 that the other ball is not blue.

x 0 1 2

f(x|1) 12

12 0

53

Example 17

Given the joint density function

f(x, y) =

{x(1+3y2)

4 , 0 < x < 2, 0 < y < 1;0, elsewhere.

find g(x), h(y), f(x|y), and evaluate P (14 < X < 1

2|Y = 13).

Solution:

• g(x) =∫∞−∞ f(x, y)dy =

∫ 20

x(1+3y2)4 dy = x

2, 0 < x < 2.

• h(y) =∫∞−∞ f(x, y)dx =

∫ 10

x(1+3y2)4 dx = 1+3y2

2 , 0 < y < 1.

• f(x|y) = f(x,y)h(y) = x(1+3y2)/4

1+3y2/2= x

2, 0 < x < 2.

54

• P (14 < X < 1

2|Y = 13) =

∫ 1/21/4

x2dx = 3

64

Statistical Independent

Let X and Y be two random variables with joint probability dis-

tribution f(x, y) and marginal distributions g(x) and h(y), respec-

tively. The random variables X and Y are said to be statistically

independent if and only if

f(x, y) = g(x)h(y)

for all (x, y) within their range.

55

Example 15 Cont’d

• f(0, 1) = 628.

• g(0) =∑2

y=0 f(0, y) = 1028.

• h(1) =∑2

x=0 f(x, 1) = 1228.

• Clearly, f(0, 1) 6= g(0)h(1).

Therefore, X and Y are not statistically independent.

56

Example 18

Let X and Y have joint probability density function (p.d.f.)

f(x, y) =

{3x2y32 , if 0 < x < 2 and 1 < y < 3;

0, otherwise.

Find the marginal probability density functions of X and Y , re-

spectively, and determine if X and Y are independent ?

Solution:

g(x) =∫ 3

1

3x2y

32dy =

3x2

8, 0 < x < 2;

h(y) =∫ 2

0

3x2y

32dy =

y

4, 1 < y < 3.

Since f(x, y) = g(x)h(y), they are independent.

57

Example 19

Consider the following joint probability density function of the

random variables X and Y

f(x, y) =

{3x−y

18 , 1 < x < 4, 1 < y < 2;0, elsewhere.

1. Find the marginal density functions of X and Y .

2. Are X and Y are independent?

3. Find P (X > 2|Y = 2).

58

Solution:

1.

g(x) =∫ 2

1f(x, y)dy =

1

6(x−

1

2) 1 < x < 4;

h(y) =∫ 4

1f(x, y)dx =

1

6(15

2− y) 1 < y < 2.

2. Since f(x, y) 6= g(x)h(y), they are not independent.

3.

P (X > 2|Y = 2) =∫ 4

2

f(x, 2)

h(2)dx =

12

33.

59

Statistical Independent: General Case

Let X1, X2, . . . , Xn be random variables, discrete or continuous,

with joint probability distribution f(x1, x2, . . . , xn) and marginal

distributions f1(x1), f2(x2), . . . , f(xn), respectively. The random

variables X1, X2, . . ., Xn are said to be statistically independent

if and only if

f(x, y) = f1(x1)f2(x2) · · · fn(xn)

for all (x1, x2, . . . , xn) within their range.

60

Example 20

Suppose that the shelf life, in years, of a certain perishable food

product packaged in cardboard containers is a random variable

whose probability density function is given by

f(x) =

{e−x, x > 0;0, otherwise.

Let X1, X2 and X3 represent the shelf lives for three of these

containers selected independently and find

P (X1 < 2, 1 < X2 < 3, X3 > 2).

61

Solution:

Since the containers were selected independently, we can assume

that the random variables X1, X2 and X3 are statistically inde-

pendent, having the joint probability density

f(x1, x2, x3) =

{f(x1)f(x2)f(x3) = e−x1−x2−x3, x1 > 0, x2 > 0, x3 > 0;0, elsewhere.

Hence,

P (X1 < 2, 1 < X2 < 3, X3 > 2) =∫ ∞

2

∫ 3

1

∫ 2

0e−x1−x2−x3dx1dx2dx3

= (1− e−2)(e−1 − e−3)e−2

= 0.0372.

62