ch.3 random variables (불규칙변수ocw.snu.ac.kr/sites/default/files/note/1787.pdf ·...

확률 변수 및 확률과정의 기초

Ch.3 Random Variables (불규칙변수)

3.1 The Notion of a Random VariableS X(ζ) = x

real linex

S X(ζ) x

ζ •

Random variable X a function that assigns a real number X(ζ)

xSX

Random variable X = a function that assigns a real number X(ζ) to each outcome ζ in the sample space of a random experimentSpecification of a measurement on the outcome of a randomSpecification of a measurement on the outcome of a random experiment

Define a function on the sample space, i.e., a random variable

1


S = the domain of the random variableSX = the range of the random variable

Ex.3.1:Afte th ee time of oin tossing the seq en e of heads andAfter three time of coin tossing, the sequence of heads and tails is noted.

∴ S = {HHH, HHT, HTH, HTT, THH, THT, TTH, TTT}{ , , , , , , , }

ζ = outcome

X = the number of heads in three coin tossesSX = {0, 1, 2, 3} a new sample space

2

X { , , , } p p


ζ : HHH HHT HTH THH HTT THT TTH TTTX(ζ) : 3 2 2 2 1 1 1 0

cf) If ζ is already a numerical value, the outcome can be a random variable defined by identity function X(ζ) ζrandom variable defined by identity function, X(ζ) = ζ

Function is fixed and deterministicFunction is fixed and deterministicRandomness is due to the experiment outcomes ζSample space S, random outcome ζ and the event ASample space S, random outcome ζ and the event ASample space SX,, random variable X(ζ) and the event B in SX,

A = {ζ : X(ζ) in B}

3

{ζ (ζ) }P[B] = P[A] = P[{ζ : X(ζ) in B}]


3.2. Cumulative Distribution Function (cdf)

The cdf of a random variable X : the probability of the event {X ≤ x}{ }FX(x) = P[X ≤ x] for -∞ < x < ∞

a numerical variable (not random)

The probability that X takes on a value in the set (-∞, x]

a numerical variable (not random)a random variable

p obab y a a o a a u ( , ]The probability of the event {ζ : X(ζ ) ≤ x}

Note“The event {X ≤ x} and P[X ≤ x] vary with x” or

FX(x) : a function of the variable x.

4


Events of interest when dealing with numbersIntervals of the real line, their complements, unions, , p , ,intersections

Properties of the CDF0 ≤ FX(x) ≤ 1

cf) Axiom I : 0 ≤ P[A] and Corollary 2: P[A] ≤ 1

1)(lim =∞→

xFXx

cf) The event {x < ∞} is the entire sample space, then Axiom II : P[S] = 1

5


0)(lim =−∞→

xFXx

cf) The event {x ≤ －∞} is the empty set, thenCorollary 3: P[Φ]=0

FX(x) is a nondecreasing function of x,i.e., if a < b, then FX(a) ≤ FX(b) 단조증가함수, , X( ) X( )

cf) Corollary 7: the event {x ≤ a} ⊂ the event {x ≤ b}, thenP[x ≤ a] ≤ P[x ≤ b]

FX(x) is continuous from the right for h > 0 )()(lim)( ++ bFhbFbF

6

for h > 0, )()(lim)(0→

=+= bFhbFbF XXhX


P[a < X ≤ b]= FX(b)－FX(a)cf) {X ≤ a} ∪ {a < X ≤ b} = {X ≤ b}, then by Axiom III

P[X ≤ a] + P[a < X ≤ b] = P[X ≤ b]

P[X = b] = FX(b)－FX(b－)P[X b] FX(b) FX(b )cf) let a = b－ε and ε > 0

P[b－ε < X ≤ b] = FX(b)－FX(b－ε)

Note : If the CDF is continuous at a point b, then the event {X = b}

)()(][][lim0

−

→−===≤<− bFbFbXPbXbP XXε

ε

Note : If the CDF is continuous at a point b, then the event {X b}has probability zero,

i.e., P[X = b] = 0 if CDF is continuous at a point b.

7


Note : {a ≤ X ≤ b} = {X = a} ∪ {a < X ≤ b}∴ P[a ≤ X ≤ b] = FX(a)－FX(a－) + FX(b)－FX(a) = FX(b)－FX(a－)

Note : If the CDF is continuous at endpoints, i e at the points X = a and X = b theni.e., at the points X = a and X = b, then

P[a < X < b] = P[a ≤ X < b] = P[a < X ≤ b] = P[a ≤ X ≤ b]

P[X > x] = 1－FX(x)

8


Ex 3.4:The cdf is continuous from the right and equal to ½ at the g qpoint x = 1.P[X = 1] = 3/8, the magnitude of the jump at the point x = 1.cf) For a small positive number δ,

21)1(,

21)1(,

81)1( =+==− δδ XXX FFF

83)1()1(]1[

228

=−−= δXX FFP⎧ < 00 x

cf) In terms of unit step function

)3(1)2(3)1(3)(1)(F

⎩⎨⎧

≥<

=0100

)(xx

xu

9

)3(8

)2(8

)1(8

)(8

)( −+−+−+= xuxuxuxuxFX


Three Types of Random Variables

Discrete Random Variablecdf is a right-continuous, staircase function of x with jumps at a g , j pcountable set of points x0, x1, x2,…SX = {x0, x1, x2,…}probability mass function (pmf) of X : the set of probabilities pX(xk) = P[X = xk] of the elements in SX

cdf of a discrete random variablecdf of a discrete random variable

)()()( kkXX xxuxpxF −=∑where pX(xk) = P[X = xk] : the magnitude of jumps in the cdf

k

10


Continuous Random VariableFX(x) is continuous everywhere and sufficiently smooth.X( ) y y

∫ ∞−=

x

X dttfxF )()(

where f(t) is a nonnegative function

P[X = x] = 0

11


Random Variable of Mixed Typecdf has jumps on a countable set of points x0, x1, x2,…j p p 0, 1, 2,

cdf increases continuously over at least one interval of values of yx.

FX(x) = pF1(x) + (1－p)F2(x)where 0<p<1, and F1(x) is the cdf of a discrete random variable and F2(x) is the cdf of a continuous random variable

12


3.3. The Probability Density Function (pdf)

The probability density function of X (pdf) : fX(x)xdFxf X )()(

Note : The pdf represents the density of probability at

dxxf X )( =

Note : The pdf represents the density of probability at the point x.

)()(][ xFhxFhxXxP XX −+=+≤<)()(

)()(][

hh

xFhxF XX

XX

⋅−+

=

)()()()(li

. )(

fFxFhxFhhxF

XX

X

′−+

′≅

where

interval smallfor very

13

)()()()(lim 0

xfxFh XX

XX

h=′=

→where


1 fX(x) ≥ 0cf) and FX(x) is a nondecreasing functionxdFxf X

X)()( =) X( ) g

dxxf X )(

dxxfdxxXxP

X )(][

≅+≤<

The probabilities of eventsf th f “ f ll i ll

f X )(

of the form. “X falls in a smallInterval of width dx about the point x”

14


b2

cf) The probability of an interval [a, b]dxxfbXaP

b

a X∫=≤≤ )(][) p y [ , ]

The area under fX(x) in that interval

dxxfbXaPb

∫≤≤ )(][ dxxfbXaPa X∫=≤≤ )(][

15


3Note : The pdf completely specifies the behavior of continuous

dttfxFx

XX ∫ ∞−= )()(

p p y prandom variables

∫∞

4 : normalization condition for pdf’sNote : a valid pdf can be formed from any nonnegative, i i i f i h h fi i i l

dttf X∫∞

∞−= )(1

piecewise continuous function g(x), that has a finite integral

cdxxg )( ∞<=∫∞

∞

cxgxf X)()( =⇒

∫ ∞−

16

c


Ex. 3.8 The pdf of the samples of the amplitude of speech waveforms.p p p p

∞<<∞−= − xcexf xX )( α

1 c = ? (using the normalization condition)

2α∫∞ cx 21

αα

α == ∫ ∞−

− cdxce x

2 α

=∴ c

17


α2

v

v

x dxevXP αα−

−

⎞⎛

=< ∫2][

v xdxe αα −⎟⎠⎞

⎜⎝⎛= ∫2

2 0

ve α−−=1

18


pdf of Discrete Random Variables

cf) The derivative of the cdf does not exist at the discontinuities.

cf) The relation between u(x) and δ(x)cf) The relation between u(x) and δ(x)Unit step function

⎨⎧ < 0 0

)(x

⎩⎨⎧

≥=

0 1)(

xxu

Delta function (definition of the delta function)dttxu

x

∫ ∞= )()( δ

19

∫ ∞−


The cdf of a discrete random variable)()()( kkXX xxuxpxF −=∑

][)( kkX

k

xXPxp ==

∑ where

Generalized definition of the pdf fX(x)

= ∫ )()(x

XX dttfxF

−=−⇒ ∫∫

∞

∞−

∞−

)()(

)()( XX

bxubx

f

δ

−=∴ ∑ )()()( kk

kXX xxxpxf δ

20

⇒ The pdf for a discrete random variable


Ex.3.9X : the number of heads in three coin tosses.SX = {0, 1, 2, 3}Find the pdf of XFind P[1 < X ≤ 2] and P[2 ≤ X < 3] by integrating the pdf.

sol) )3(81)2(

83)1(

83)(

81)( −+−+−+= xuxuxuxuxFX①

8888

)3(81)2(

83)1(

83)(

81)( −+−+−+= xxxxxf X δδδδ②

③[ ]

[ ]

2

1

3

31 2 ( )83

XP X f x dx+

+

−

< ≤ = =∫

∫21

[ ]3

2

32 3 ( )8XP X f x dx

−≤ < = =∫


Conditional cdf’s and pdf’s

Definition of conditional cdf of X given Ag

0][ if ][

]}[{)|( >≤

= APAP

AxXPAxFX∩

conditional pdf of X given A

)|()|( AxFdAxf XX =

Ex. 3.10 l f f h d bl df

)|()|(dx

f XX

1 life-time of a machine : random variable X → cdf FX(x)2 Find the conditional cdf and pdf given the event A = {X > t}

(i e “machine is still o king at time t” )

22

(i.e., “machine is still working at time t” )


sol) 1 The conditional cdf( ) ]|[| tXxXPtXxFX >≤=>

][}]{}[{

tXPtXxXP

>>≤

=∩

}{}{}{ }{}{

xXttXxXtxtXxXtx

≤<=>≤≥=>≤<

∩∩ φif

)()( 0

)|( ⎪⎨

⎧−

≤=>∴ tFxF

txtXxF

}{}{}{

cf) continuous at x = t

)(1)()()|(

⎪⎩

⎨ >−−=>∴

txtF

tFxFtXxF

X

XXX

23

cf) continuous at x = t


The conditional pdf

xfXf X ≥)()|( txtF

ftXxfX

XX ≥

−=>

)(1)()|(

H W P 2 4 9 13 15 16 17 23 26 27H.W. P 2, 4, 9, 13, 15, 16, 17, 23, 26, 27

24


3.4. Some Important Random Variables

Discrete Random Variables

Bernoulli Random Variable

Binomial Random Variable

Geometric Random Variable

Negative Binomial Random Variable

P i R d V i bl

25

Poisson Random Variable


C ti R d V i blContinuous Random Variable

Uniform Random Variable

Exponential Random Variable

Ga ssian Random Va iableGaussian Random Variable

m-Erlang Random Variable

Chi-Square Random Variable

Rayleigh Random Variableg

Cauchy Random Variable

26

Laplcian Random Variable


Discrete Random Variables

- Counting is involvedBernoulli Random Variable

⎩⎨⎧

=A

AI A 1

0)(

inifinnot if

ζζ

ζ

I → Random Variable

Indicator function for A

⎩

IA → Random VariableSX = {0, 1}pmf (Probability Mass Function)p ( y )pI(0) = 1－p and pI(1) = pwhere P[A] = p → cf) tossing a biased coin

27

→ IA : Bernoulli random variable


Binomial Random Variable

n times of independent trialsIj : The indicator function for the event A in the j th trialj

X = I1 + I2 + … + In ( cf) Ij = 0 or 1 )pmf of X n kk⎟

⎞⎜⎛

,n,kppkn

kXP knk …0 )1(][ =−⎟⎟⎠

⎞⎜⎜⎝

⎛== − for

X : binomial random variableNote : From the pdf, P[X = k] is maximum at kmax = [(n+1)p],where [x] is the largest integer that is smaller than or equal to xNote : If (n+1)p is an integer, then the maximum is achieved at k and k 1 (P b 33)

28

kmax and kmax－1 (Prob. 33)


Geometric Random Variable

The number M of independent Bernoulli trials until the first occurrence of a successSM = {1, 2, …}pmf of Mpmf of M

P[M = k] = (1－p)k－1p k = 1, 2, …where p = P[A] is the probability of “success” in each p [ ] p obab y o u aBernoulli trial

From the pdfFrom the pdfP[M = k] decays geometrically with k.

cf) q(k－1)p

29

) q p


cdfk

kkj qqppqkMP −=

−⋅==≤ ∑ − 11][ 1

cf) Geometric Series

jq

qppq

−∑= 1

][1

(1 )na r1 (1 )1

n a ra ar arr

− −+ + + =

−

Memoryless Property

[ ] ][],[| jkMPjMjkMPjMjkMP +≥=

>+≥=>+≥[ ]

][][][

|

kMPjMPjMP

jMjkMP

≥=>

=>

=>+≥

30

][


cf)

1][

][

11

1

1

kMP

qjkMP

kk

j

jk=+≥

−−

−

−+

∑][1][

1][ 1

1

1

kMPkMP

qpqkMP k

j

j

<−=≥

−==<=∑

][

][][

1

1

qjkMPq

jk

k

+≥

=−+

−

][]1[][ 1 kMPq

qq

jMPjkMP k

j ≥===+≥+≥

∴ −

31


The only memoryless discrete random variablecf) memoryless : Each time a failure occurs, the system forgets ) y , y gand begins anew as if it were performing the first trial

e.g. The memoryless property1 ][]|[ >≥=>+≥ j, kkMPjMjkMP allfor

32


Poisson Random Variable

Counting the number of occurrences of an event in a certain time period or in a certain region in space.p g p

where the events occur completely at random in time or spaceCounts of emissions from radioactive substancesCounts of demands for call connectionCounts of defects in a semiconductor chip

pmf for the Poisson random variable

,,2,1,0 ][ …=== − kekNPk

αα

where α is the average number of event occurrences in a ifi d ti i t l i i

,,,,0!

][ …kek

kN

33

specified time interval or region in space.


Note for α < 1, P[N = k] is maximum at k = 0for α > 1, P[N = k] is maximum at k = [α]if α is a positive integer, kmax= α and α－1

Note 1=⋅== −∞

−−∞

∑∑ αααα αα eeeekk

f)

1!! 00 ==

∑∑ eek

eek kk

0lim =kα

k! kcf)even for very large α

0!

lim =∞→ kk

k!

αk

αk

k!

34

k k


Approximation of the binomial probabilities with veryApproximation of the binomial probabilities with very large n and small p.

k⎞⎛ ( ) npek

ppkn

pk

knkk =≅−⎟⎟

⎠

⎞⎜⎜⎝

⎛= −− αα α

!1 where

cf) +−

−++−+−=−

−−

)!1()1(

!3!21

11

32

ne

nn ααααα

2

011 1 ( 1)2!

n

p n n nn n nα α α⎡ ⎤⎛ ⎞ ⎛ ⎞= − = − ⋅ + − ⋅ − +⎢ ⎥⎜ ⎟ ⎜ ⎟

⎝ ⎠ ⎝ ⎠⎢ ⎥⎣ ⎦211

2!e αα α −

⎣ ⎦

→ − + + =

35

1 ( ) (1 / )( 1)(1 ) ( 1)(1 / ) 1

k

k

p n k p k np k p k n k

α αα

+ − −= = →

+ − + − +


′′′ )()( ffcf) −−′′

+−′

+=

− )(

)(!2

)()(!1

)()()(

)1(

2

f

axafaxafafxf

n

+−−

+ −)()!1(

)(

2

1)1(

hh

axn

af nn

+′′+′+=+ )(!2

)(!1

)()(2

afhafhafhaf

36


Ex. 3.12Rate of requests for telephone connections = λ (calls/sec)q p ( / )

The number of requests in a time period → Poisson random variable

Fi d th b bilit f ll t i dFind the probability for no call requests in t seconds

Find the probability for or more requestsFind the probability for n or more requests

37


sol)Average number of requests in t seconds α = λt

tt eettNP λλλ −− ===∴!0)(]0)([

0

[ ]t

n ktntNPntNP

λλ−

∑

<−=≥1 )(

])([1)(

t

k

ekt λλ −

=∑−=

0 !)(1

38


Continuous Random Variables

Easier to handle analyticallyThe limiting form of many discrete random variablesThe limiting form of many discrete random variables→ continuous random variable

Uniform Random VariableUniform Random VariableAll values in an interval of the real line are equally likely to occur

39


Exponential Random Variable

The time between occurrence of eventsThe lifetime of devices and systemsThe lifetime of devices and systemspdf and cdf

⎧⎧ 0000

⎩⎨⎧

≥−

<=

⎩⎨⎧

≥

<=

−− 0 10 0

)( 0 0 0

)(xex

xFxex

xfxXxX λλλ

fX(x) FX(x)λe-λx 1－e-λx

40

x x


λ = the rate at which events occurcf) the probability of an event occurring by time x increases as

hthe rate λ increases.Limiting form of the geometric random variableAn interval of duration T → Subintervals of length TAn interval of duration T → Subintervals of length cf)

∴ discrete model → continuous model0, →∞→Tn

n

The sequence of subintervals → a sequence of independent

,n

αBernoulli trials with

where α = the average number of events per T seconds

np α=

41

where α = the average number of events per T seconds.


The number of subintervals until the occurrence of an event → a geometric random variable M.

TThe time until the occurrence of the first event

nTMX ⋅=

[ ] tnMPtXP ⎥⎦⎤

⎢⎣⎡ ⋅>=>[ ]

tnMP

T

⎥⎦⎤

⎢⎣⎡ ⋅≤−=

⎥⎦⎢⎣

1

Ttn

p

T⋅

⎬⎫

⎨⎧

−−−=

⎥⎦⎢⎣

)1(11

Ttn

p

p

⋅

⎭⎬

⎩⎨

)1(

)1(11

42

Tp−= )1(


⎫⎧t

∞→→⎪⎭

⎪⎬⎫

⎪⎩

⎪⎨⎧

⎟⎠⎞

⎜⎝⎛ −= − ne

nTtTn

)(1 asαα

⎭⎩t

TX etXPxF

α−

−=≤= 1][)(

the exponential random variablecdf with

Tαλ =

Note : Poisson random variableP[N(t) 0] λt → No calls for t seconds

T

P[N(t) = 0] = e-λt → No calls for t seconds→ an interval between any two calls > t seconds

43


Meanwhile

secondsthatyprobabilit][ tXtXPt

T >>−α

secondsthaty probabilit : ][ tXetXP T >=>

In conclusion, for Poisson random variable. the time between events is an exponentially distributed random

i bl i hα

variable with

Note : The exponential random variable is the only continuous

Tαλ =

Note : The exponential random variable is the only continuous random variable that satisfies the memoryless property.

44


Gaussian(Normal) Random Variable

A random variable consisting of the sum of a large number of “small” random variables approaches the ppGaussian random variable : the central limit theorem

The pdf for the Gaussian random variable Xmx1 22 2)( σ

Where m and σ are the mean and standard deviation of X

∞<<∞−= −− xexf mxX

21)( 2)( σ

σπWhere m and σ are the mean and standard deviation of X.

45


cdf ( ) ′=≤= ∫ ∞−

−′−

σπσ xdexXPxF

x mxX

2)( 22

21][

= ∫−

∞−

−

σπσ dte

mx t)(2

2

21

⎟⎠⎞

⎜⎝⎛ −

Φ=

∫ ∞σπmx

2

⎠⎝ σ

dtexx t∫ −=Φ 221)( where

cf) Φ(x) is the cdf of a Gaussian random variable with m = 0 and σ = 1

∫ ∞−2)(

π

46

m 0 and σ 1


12x

Ex. 3.14: Show that 121 2 =∫

∞

∞−

−dxe

x

π

σ=1

σ=1

m m=0

47


2⎡ ⎤2 2 2

22 2 21 1

22x x ye dx e dx e dy

ππ∞ ∞ ∞− − −

−∞ −∞ −∞

⎡ ⎤ =⎢ ⎥⎣ ⎦∫ ∫ ∫2 2( ) 21

2x ye dxdy

π∞ ∞ − +

−∞ −∞= ∫ ∫

22 2

0 0

12

re rdrdπ

θπ

∞ −= ∫ ∫1=

11 22

∫∞ − dx 1

2 2 =∴ ∫ ∞−

dxe x

π

∫∞ 2

f)

48

∫ ∞−

− = πdxe x2

cf)


Q-function( ) xxQ Φ−= )(1( )

dte

xxQ

x

t∫∞ −=

Φ

22

21

)(1

πthe probability of the “tail” of the pdf

x∫2π

)(1)(,21)0( −=−=→ xQxQQ

11)(

2

2

2

2

⎥⎥⎤

⎢⎢⎡

≈→ −exQ x

π

π

2,1

2)1()(

2

==

⎥⎥⎦⎢

⎢⎣ ++−

ba

bxaxaQ

where

49

ππ

2 , ba where


Note : Table 3.4The value of x for which Q(x) = 10－k

where k = 1, 2, …,10k = 1 : the probability of tale 0.1 or k = 2: the probability of tale 0.01

0.1 or 0.01

-x 0 "Zero mean

Gaussian

50


Gamma Random Variable

pdf

∞<<−−

xexxfx

0)()(1λλ λα

where two parameters α and λ are positive numbers and

∞<<Γ

= xxf X 0 )(

)(α

p pΓ(z) is the gamma function.

forcf) zdxexz xz 0)( 1 >=Γ −∞ −∫ for cf) zdxexz

21

0 )(0

=⎟⎠⎞

⎜⎝⎛Γ

>=Γ ∫π

i ttiffor zzzz

!)1(0 )()1(

2

Γ>Γ=+Γ

⎠⎝

51

integerenonnegativa for mmm !)1( =+Γ


Many random variables are special cases of the gamma random variable

The exponential random variableThe exponential random variableSpecial case of the gamma r.v. with α = 1

52


Chi-Square Random Variableintegerpositiveaiswhere kk,1

== αλ

12

21k

xx −−⎛ ⎞

⎜ ⎟

integerpositiveais where k2

,2

αλ

( )2

2 2X

ef x

k

⎛ ⎞⎜ ⎟⎝ ⎠=

⎛ ⎞Γ⎜ ⎟

22 2

2k x

x e−⎛ ⎞

−⎜ ⎟⎝ ⎠

Γ⎜ ⎟⎝ ⎠

2 2

222

kx e

k

⎝ ⎠

=⎛ ⎞Γ⎜ ⎟⎝ ⎠

53

2⎝ ⎠


m-Erlang Random Variableα = m )( 1−− xe mx λλ λ

0)!1()(

>−

= xm

xef Xλλ

The time Sm that elapses until the occurrence of the mth event. : m-Erlang r.v.Sm = X1 + X2 +…+ XmX1, X2, , Xm : the times between events

→ exponential random variables→ exponential random variablesSm≤ t if and only if m or more events occur in t seconds → N(t) ≥ m → mth event occurred by time t

54

N(t) ≥ m m event occurred by time t


( )])([1])([

][

mtNPmtNP

tSPtF mSm

<−=≥=

≤=

where N(t) is the Poisson random variable for the number of events in t seconds.

])([])([

cf) α = λt

dtdF

fk

ettF m

mm

SS

m

k

tk

S =−= ∑−

=

−

,!

)(1)(1

0

λλ

So, Sm is an m-Erlang random variable!

55

ch.3 random variables (불규칙변수ocw.snu.ac.kr/sites/default/files/note/1787.pdf ·...

Documents