eee 461 1 probability and random variables huseyin bilgekul eee 461 communication systems ii...
TRANSCRIPT
EEE 461 1
Probability and Random Probability and Random VariablesVariables
Huseyin BilgekulEEE 461 Communication Systems II
Department of Electrical and Electronic Engineering Eastern Mediterranean University
Why Probability in Communications Probability Random Variables Probability Density Functions Cumulative Distribution Functions
EEE 461 2
Why probability in Why probability in Communications?Communications?
• Modeling effects of noise– quantization – Channel– Thermal
• What happens when noise and signal are filtered, mixed, etc?
• Making the “best” decision at the receiver
EEE 461 3
SignalsSignals• Two types of signals
– Deterministic – know everything with complete certainty
– Random – highly uncertain, perturbed with noise
• Which contains the most information? Information content is determined from the amount of uncertainty and unpredictability. There is no information in deterministic signals
Information = Uncertainty
x(t) y(t)
(t)
0 100 200 300 400 500 600 700 800 900 1000-2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5y[n]x[n]
Let x(t) be a radio broadcast. How useful is it if x(t) is known? Noise is ubiquitous.
EEE 461 4
Need for Probabilistic Need for Probabilistic AnalysisAnalysis
• Consider a server process – e.g. internet packet switcher, HDTV frame decoder, bank teller line,
instant messenger video display, IP phone, multitasking operating system, hard disk drive controller, etc., etc.
Rejected customer, Queue full
Customers arrive at random times
Queue,Length L
Server:1 customer
per seconds
Satisfied customer
EEE 461 5
Probability DefinitionsProbability Definitions
• Random Experiment – outcome cannot be precisely predicted due to complexity
• Outcomes – results of random experiment• Events – sets of outcomes that meet a criteria,
roll of a die greater than 4• Sample Space – set of all possible outcomes,
EE (sometimes called the Universal Set)
EEE 461 6
ExampleExample• B={, , }• Complement
– BC={, , }• Union
• Intersection
• Null Set (), empty set
1
2
3
4 6
5
EEAo
B
Ae
1 3 4 5 6, , , ,oA B
5o oA B A B
EEE 461 7
Relative FrequencyRelative Frequency• nA – number of elements in a set, e.g. the
number of times an event occurs in N trials
• Probability is related to the relative frequency
• For N small, fraction varies a lot; usually gets better as N increases
Relative Frequency
lim Probability
0 1
0 Never Occurs
1 Always Occurs
A
A
n
nf A
nn
P An
P A
P A
P A
EEE 461 8
Joint ProbabilityJoint Probability• Some events occur together
– Sum of two dice is 6– Chance of drawing a pair of jacks
• Events can be – mutually exclusive (no intersection) – tossing a coin– Intersect and have common elements
• The probability of a JOINT EVENT, AB, is
lim Joint Probability
Let then
AB
n
nP AB
n
E A B
P E P A B P A P B P AB
EEE 461 9
Bayes Theorem and Independent Bayes Theorem and Independent EventsEvents
/ = / Bayes Theorem
/ Probability that A occurs given that B has occured
/ Probability that B occurs given that A has occured
P AB P A P B A P B P A B
P A B
P B A
1 2
1 2 1 2
Two events are INDEPENDENT if
/ =
/ =
If a set of events , , ...... are INDEPENDENT
, , ...... = ..... n
n n
P A B P A
P B A P B
A A A
P A A A P A P A P A
EEE 461 10
Axioms of ProbabilityAxioms of Probability
• Probability theory is based on 3axioms– P(A) >0– P(E) = 1– P(A+B) = P(A) + P(B) If P(AB) =
EEE 461 11
Random VariablesRandom Variables
• Definition: A real-valued random variable (RV) is a real-valued function defined on the events of the probability system
Event RV
Value
P(x)
A 3 0.2
B -2 0.5
C 0 0.1
D -1 0.2
E
A
DC
BP(x)
x3-1 0-2
0.5
1
EEE 461 12
Cumulative Density FunctionCumulative Density Function
• The cumulative density function (CDF) of the RV, x, is given by Fx(a)=Px(x<a)
Fx(a)
a3-1 0-2
0.5
1
P(x)
x3-1 0-2
0.51
0.2 0.20.1
EEE 461 13
Probability Density FunctionProbability Density Function
• The probability density function(PDF) of the RV x is given by f(x)
• Shows how probability is distributed across the axis
x xx
a x a x
dF a dP x af x
da da
fx(x)
x3-1 0-2
0.51
0.2 0.20.1
EEE 461 14
Types of DistributionsTypes of Distributions• Discrete-M discrete values at x1, x2, x3,. . . , xm
• Continuous- Can take on any value in an defined interval
fx(x)
x10-1
0.5
1
Fx(a)
x10-1
0.5
1
Fx(a)
a3-1 0-2
0.5
1
fx(x)
x3-1 0-2
0.51
0.2 0.20.1DISCRETE
Continuous
EEE 461 15
Properties of CDF’sProperties of CDF’s• Fx(a) is a non decreasing function• 0 < Fx(a) < 1• Fx(-infinity) = 0• Fx(infinity) = 1• F(a) is right-hand continuous
x x0
lima
F a f x dx
x x0
limF a F a
EEE 461 16
PDF PropertiesPDF Properties
• fx(x) is nonnegative, fx(x) > 0
• The total probability adds up to one
x 1xf x dx F
fx(x)PDF
10-1
2
Fx(a)CDF
1-1
1
EEE 461 17
Calculating ProbabilityCalculating Probability
• To calculate the probability for a range of values
x x x
x x
x0 lim
b
a
P a x b P x b P x a
F b F a
f x dx
fx(x)
10-1
2
1-1ba ba
F(b)F(a)
AREA= F(b)- F(a)
EEE 461 18
Discrete Random VariablesDiscrete Random Variables
1
1
If x is discretely distributed and represents a discrete event
( ) ( ) ( - )
F( ) ( )
i
M
i ii
L
ii
x
f x P x x x
a P x
• Summations are used instead of integrals for discrete RV.• Discrete events are represented by using DELTA
functions.
EEE 461 19
PDF and CDF of a Triangular PDF and CDF of a Triangular WaveWave
-A
A
• Calculate Probability that the amplitude of a triangle wave is greater than 1 Volt, if A=2.
• Sweep a narrow window across the waveform and measure the relative frequency of occurrence of different voltages.
s(t)fx(x)
A
-A
fx(x)
0
1/2A
A-A
EEE 461 20
PDF and CDF of a Triangular PDF and CDF of a Triangular Wave Wave
fV(v)
0
• Calculate Probability that the amplitude of a triangle wave is greater than 1 Volt, if A=2.
1/4
2-2 1
2
V 1 1
V V V
1 11
4 43 1
1 1 14 4
VP v f v dv dv
P v F F
FV(v)
0
3/4
2-2 1
1
EEE 461 21
fV(v)
0
• Calculate Probability that the amplitude of a triangle wave is in the range [0.5,1] v, if A=2.
1/4
2-2 1
1 1
V 0.5 0.5
V V V
1 10.5 1
4 81
0.5 1 1 0.58
VP v f v dv dv
P v F F
FV(v)
0
3/4
2-2 1
1
5/8
PDF and CDF of a Triangular PDF and CDF of a Triangular WaveWave
EEE 461 22
PDF and CDF of a Square PDF and CDF of a Square WaveWave
-A
A
• Calculate Probability that the amplitude of a square wave is at +A.
• Sketch PDF and CDF
s(t)fx(x)
0 A-A
EEE 461 23
PDF and CDF of a Square PDF and CDF of a Square WaveWave
• Calculate Probability that the amplitude of a square wave is at +A. 1/4
• Sketch PDF and CDF
-A
A
s(t) fx(x)
0 A-A
Fx(x)
0 A-A
EEE 461 24
Ensemble AveragesEnsemble Averages• The expected value (or ensemble average) of
y=h(x) is:
x
x
x
[ ] [ ]
For Discrete distributions
[ ( )] i ii
y E y h x f x dx
f x dx
y h x E y h x f x
EEE 461 25
MomentsMoments• The r th moment of RV x about x=xo is
x
o
x
2
2 2 2x
( ) ( )
MEAN is the first moment taken about x =0
VARIANCE is the second moment around the mean
( ) ( )
STANDARD DEVIATION - the second moment around
r ro ox x x x f x dx
m x x f x dx
x x x x f x dx
2
2 2x
2 2
mean
( )
( )
x x f x dx
x x