lecture 7 channel capacity
TRANSCRIPT
-
8/8/2019 Lecture 7 Channel Capacity
1/35
Outline
Transmitters (Chapters 3 and 4, Source Coding andModulation) (week 1 and 2)
Receivers (Chapter 5) (week 3 and 4)Received Signal Synchronization(Chapter 6) (week 5)Channel Capacity (Chapter 7) (week 6)Error Correction Codes (Chapter 8) (week 7 and 8)Equalization (Bandwidth Constrained Channels) (Chapter 10) (week 9)Adaptive Equalization (Chapter 11) (week 10 and 11)Spread Spectrum (Chapter 13) (week 12)Fading and multi path (Chapter 14) (week 12)
-
8/8/2019 Lecture 7 Channel Capacity
2/35
Channel Capacity (Chapter 7) (week 6)
Discrete Memoryless ChannelsRandom Codes
Block CodesTrellis Codes
-
8/8/2019 Lecture 7 Channel Capacity
3/35
Channel Models
Discrete Memoryless Channel Discrete-discrete
Binary channel, M-ary channel
Discrete-continuousM-ary channel with soft-decision (analog)
Continuous-continuous
Modulated waveform channels (QAM)
-
8/8/2019 Lecture 7 Channel Capacity
4/35
Discrete Memoryless Channel
Discrete-discrete Binary channel, M-ary channel
-
!!!
!
11
11
...
....
.)|(..
...)|(
Qq
ji ji
p
p x y P
x X yY P
P
Probability transition matrix
-
8/8/2019 Lecture 7 Channel Capacity
5/35
Discrete Memoryless Channel
Discrete-continuous M-ary channel with soft-decision (analog)
output x
0 x1 x2..
. xq-1
y
-
)(
)( 1
k x y p
x y pP
22 2/)(
2
1)|( W
TW
k x yk
e x y p !AWGN
-
8/8/2019 Lecture 7 Channel Capacity
6/35
Discrete Memoryless Channel
Continuous-continuous Modulated waveform channels (QAM) Assume Band limited waveforms, bandwidth = W
Sampling at Nyquist = 2W sample/s
Then over interval of N = 2WT samples use anorthogonal function expansion:
)(
)(
1
t f x
t x N
iii
!
!
)(
)(
1
t f n
t n N
i
ii!
!
!
! N
iii t f y
t y
1
)()(
-
8/8/2019 Lecture 7 Channel Capacity
7/35
Discrete Memoryless Channel
Continuous-continuous Using orthogonal function expansion:
)(
)(
1
t f x
t x N
iii
!
!
)(
)(
1
t f n
t n N
iii
!! _ a
? A_ a
? A
!
!
!
!
!
!
!
!
N
iiii
N
ii
T
i
N
ii
T
i
N
iii
t f n x
t f dt t f t nt x
t f dt t f t y
t f y
t y
1
10
*
10
*
1
)(
)()()()(
)()()(
)(
)(
-
8/8/2019 Lecture 7 Channel Capacity
8/35
Discrete Memoryless Channel
Continuous-continuous Using orthogonal function expansion get an
equivalent discrete time channel:
N x
x
.
.
.
.1
N y
y
y
.
.
.2
1
22 2/)(
2
1)( iii x y
i
iie x y p
W
TW !
111 n x y ! Gaussian noise
-
8/8/2019 Lecture 7 Channel Capacity
9/35
Capacity of binary symmetric channel
BSC
-!
p p
p p
1
1P
}1,0{ }1,0{ !! Y
0 0
11
X Y p1
p1
p p
-
8/8/2019 Lecture 7 Channel Capacity
10/35
Capacity of binary symmetric channel
Average Mutual Information 0 0
11
X Y p1
p1
p p
)1()1()0(
1log)1)(1(
)1()0()1(log)1(
)1()1()0(log)0(
)1()0()1(
1log)1)(0(
)1()11(
log)11()1(
)0()10(
log)10()1()1(
)01(log)01()0(
)0()00(
log)00()0();(
!!!
!!!
!!!
!!!!
!!!!!!
!!!!!!
!
!!!!!
!!!!!!!
X P p X p P
p p X P
X p P X P p p
p X P
X P p X p P p
p X P
X p P X P p
p p X P
Y P X Y P X Y P X P
Y P X Y P X Y P X P
Y P
X Y P X Y P X P
Y P X Y P
X Y P X P Y X I
.
.
.
.
.
.
-
8/8/2019 Lecture 7 Channel Capacity
11/35
Capacity of binary symmetric channel
Channel Capacity is Maximum Information earlier showed:
0 0
11
X Y p1
p1
p p
p p p p
X P p X p P p
p X P
X p P X P p p
p X P
X P p X p P p
p X P
X p P X P p p p X P Y X I
X P X P
2log)1(2log)1(
)1()1()0(1
log)1)(1(
)1()0()1(log)1(
)1()1()0(log)0(
)1()0()1(1log)1)(0());(max(
21)0()1(
!
!!!
!!!
!!!
- !!!!!
!!!!
.
.
.
21
)0()1());(max( !!!! X P X P Y X I
-
8/8/2019 Lecture 7 Channel Capacity
12/35
-
8/8/2019 Lecture 7 Channel Capacity
13/35
Capacity of binary symmetric channel
Effect of SNR on Capacity Binary PAM signal (digital signal amplitude 2 A )
0 0
11
X Y p1
p1
p p
)(t g
)(t g )(t g )(t g
)(t g A
A
02
02
/)(
02
/)(
01
1)|(
1)|(
N r
N r
b
b
e N
sr p
e N
sr p
I
I
T
T
!
!AGWN
-
8/8/2019 Lecture 7 Channel Capacity
14/35
Capacity of binary symmetric channel
Effect of SNR Binary PAM signal (digital signal amplitude 2 A )
0 0
11
X Y p1
p1
p p
)(t g
)(t g )(t g )(t g
)(t g A
A
)(2
1)(
20
0 /)(
01
02
se P N
dr e N
se P
b
N r b
!
!
! gI
I
T
0
1 s
-
8/8/2019 Lecture 7 Channel Capacity
15/35
Capacity of binary symmetric channel
Effect of SNR Binary PAM signal (digital signal amplitude 2 A )
0 0
11
X Y p1
p1
p p
!
!!
!
!
0
0
221
121
21
2
22
2
)|()|(
N AQ
QS NRQ
N Q
se P se P P
bb
b
b
K
I
-
8/8/2019 Lecture 7 Channel Capacity
16/35
Capacity of binary symmetric channel
Effect of SNR Binary PAM signal (digital signal amplitude 2 A )
0 0
11
X Y p1
p1
p p
!
!
!
!
!!
noiserms22
422
21
2
2 noiserms
21
21
0
0
0
AQ
AQ
N AQ
N AQ P
N
b
W
W Not sure about thisDoes it depend on bandwidth?
-
8/8/2019 Lecture 7 Channel Capacity
17/35
Capacity of binary symmetric channel
Effect of SNR Binary PAM signal (digital signal amplitude 2 A )
0 0
11
X Y p1
p1
p p
!
!!
noiserms2
mplitudeer c
21
noiserms
mplitude
21
21Q p P b
p p p p 2log)1(2log)1(!
-
8/8/2019 Lecture 7 Channel Capacity
18/35
Capacity of binary symmetric channel
Effect of SNR Binary PAM signal (digital signal amplitude 2 A )
0 0
11
X Y p1
p1
p p
SNR Pb1 0.3085382 0.1586553 0.0668074 0.022755 0.006216 0.001357 0.0002338 3.17E-059 3.4E-06
10 2.87E-0711 1.9E-0812 9.87E-1013 4.02E-1114 1.28E-1215 3.19E-1416 6.11E-16
Pb (BER) vs SNR for binary channel
1.40E+01,
1.28E-12
1.20E+01,9.87E-10
1E-16
1E-15
1E-14
1E-131E-12
1E-11
1E-10
1E-09
1E-08
1E-07
1E-06
1E-05
0.0001
0.001
0.01
0.1
1
0 1 2 3 4 5 6 7 8 9 1 0 11 12 13 14 15 16 17
SNR = A/rms noise
B E R
!
noiserms2
mplitudeer c
21
21
b P
-
8/8/2019 Lecture 7 Channel Capacity
19/35
Capacity of binary symmetric channel
Effect of SNR Binary PAM signal (digital signal amplitude 2 A )
0 0
11
X Y p1
p1
p p
p p p p 2log)1(2log)1(C 22!S
Pb C0 0.5 01 0.308538 0.1085222 0.158655 0.368917
3 0.066807 0.6461064 0.02275 0.8433855 0.00621 0.9455446 0.00135 0.9851857 0.000233 0.9968578 3.17E-05 0.9994819 3.4E-06 0.999933
10 2.87E-07 0.99999311 1.9E-08 0.99999912 9.87E-10 113 4.02E-11 114 1.28E-12 1
15 3.19E-14 116 6.11E-16 1
C apacity and E vs S for binary channel
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
S
/r s noise
E
!
noiserms2
mplitudeer c
21
21
b P
At capacity SNR = 7,so waste lots of SNR to get low BER!!!
-
8/8/2019 Lecture 7 Channel Capacity
20/35
Capacity of binary symmetric channel
Effect of SNR b Binary PAM signal (digital signal amplitude 2 A )
0 0
11
X Y p1
p1
p p
p p p p 2log)1(2log)1(C 22!S NR (dB)
-
! "
"
#
- # $
" !
#
%
- # !
# #
%
$ "
- #
$ $ " !
%
"
#
- #
!
#
% !
#
"
- #
!
$
"
-$
$ !
#
%
#
% % !
#
-!
" "
! %
-
#
$ !
# #
! "
-
#
! %
"
$ ! %
! % "
% !
! " !
#
#
%
#
" %
!
$ $
" % %
$
#
"
#
" "
! !
#
$
& -!
" " " " %
#
"
# & -"
#
#
!
$
# & - #
#
Ca p aci t ' C a nd B ( R vsSNR fo ) b in ary c h a nn 0 1
- 2 3 4
2
2 3 4
2 3
5
2 3 6
2 3 7
8
8 3 4
- 4 2 - 8 4 -5 5
8 4
S NR p 9 r b it (dB)
C a p a c
i t y
C a
n d B
@
bbb Q p P
K
K
erfc21
2
!!!
-
8/8/2019 Lecture 7 Channel Capacity
21/35
Channel Capacity of DiscreteMemoryless Channel
Discrete-discrete Binary channel, M-ary channel
-
!
!!
!
11
11
...
....
.)|(..
...)|(
Qq
ji ji
p
p x y P
x X yY P
P
Probability transition matrix
-
8/8/2019 Lecture 7 Channel Capacity
22/35
Channel Capacity of DiscreteMemoryless Channel
Average Mutual Information
! !
! !
|
!
!!!!!!
1
0
1
1
1
0
1
1
)(
)|(log)|()(
)(
)|(log)|()();(
q
j
Q
i j
ji ji j
q
j
Q
i j
ji
ji j
y P
x y P x y P x P
yY P
x X yY P x X yY P x X P Y X I
-
8/8/2019 Lecture 7 Channel Capacity
23/35
-
8/8/2019 Lecture 7 Channel Capacity
24/35
-
8/8/2019 Lecture 7 Channel Capacity
25/35
Channel Capacity DiscreteMemoryless Channel
Discrete-continuousChannel Capacity with AWGN
x0 x1 x2... xq-1
y
22
2/)(
21)|( W TW
k x yk e x y p !
!!!
!g
g
!
1
01
0
2/)(
2/)(
2/)(
)( 22
22
22
21)(
21
log2
1)(max
q
iq
i
x yi
x y
x yi
x P
d ye x X P
e
e x X P i
i
i
i W
W
W
TW
TW
TW
-
8/8/2019 Lecture 7 Channel Capacity
26/35
Channel Capacity DiscreteMemoryless Channel
Binary Symmetric PAM-continuousMaximum Information when:
x0 x1 x2... xq-1
y
21
)()( !!!! A X P A X P
-
!
A
A
g
g
d yeee
e
d yeee
eC
y
A A
A
y
A A
A
22
2222
22
22
2222
22
2/
2/2/
2/2
2/
2/2/
2/2
21
2log
2log
2
1
W
W W
W
W
W W
W
TW
-
8/8/2019 Lecture 7 Channel Capacity
27/35
Channel Capacity DiscreteMemoryless Channel
Binary Symmetric PAM-continuousMaximum Information when:
-
!
g
g
g
g
d yeee
e
d yeee
eC
y
A A
A
y
A A
A
22
2222
22
22
2222
22
2/
2/2/
2/2
2/
2/2/
2/2
21
2log
2log
2
1
W
W W
W
W
W W
W
TW
-
8/8/2019 Lecture 7 Channel Capacity
28/35
Channel Capacity DiscreteMemoryless Channel
Binary Symmetric PAM-continuousVersus Binary Symmetric discreteS NR B (dB) C B D
-E F F
G
H H I P
Q R F
G
F F R
S
H I
- S T F
G
H E R I H Q F
G
F
S
H H U E
- S Q F
G
H
S S
I E U
F
G
F E E T F R
- S H F
G
I T T R F Q F
G
F I U
R
S
- S E F
G
I Q
S
E F P F
G
F U Q I
S
R
- S F F
G
I E P
I Q F
G
F T P P
R I
-T F
G
E T Q P
S
U F
G S
I U U Q
S
-Q F
G
E I R E E R F
G
E F Q E H U
-H F
G S
T Q
S S
H F
G
I F Q P E R
-E F
G S
I F Q H U
F
G
H H F P
R P
F F
G
F P T Q U F
G
Q F E U R P
E F
G
F I P U F Q F
G
P Q R E Q
S
H F
G
F
S
E U
F
S
F
G
R F I F U
Q F
G
F F E I T T F
G
R P U P U P
T F
G
F F F
S
R
S
F
G
R R P
I Q Q
S
F I
G
T P
V -F Q F
G
R R R R E U
S
E R
G
F
S V -F R
S
S
H Q
G
T
S V - S I
S
Ca p aci ty C a nd B W R vsSNR fo r b in ary c h a nn X Y
- ` a b
`
` a b
` a
c
` a d
` a e
f
f a b
- b ` - f b -c c
f b
S NR p g r b it (dB)
C a p a c
i t y
C a
n d B
h
-
8/8/2019 Lecture 7 Channel Capacity
29/35
Discrete Memoryless Channel
Continuous-continuous Modulated waveform channels (QAM) Assume Band limited waveforms, bandwidth = W
Sampling at Nyquist = 2W sample/s
Then over interval of N = 2WT samples use anorthogonal function expansion:
)(
)(
1
t f x
t x N
iii
!
!
)(
)(
1
t f n
t n N
iii
!!
!
! N
iii t f y
t y
1
)()(
-
8/8/2019 Lecture 7 Channel Capacity
30/35
Discrete Memoryless Channel
Continuous-continuous Using orthogonal function expansion get an
equivalent discrete time channel:
N x
x
.
.
.
.1
N y
y
y
.
.
.2
1
22 2/)(
2
1)( iii x y
i
iie x y p
W
TW !
111 n x y ! Gaussian noise
-
8/8/2019 Lecture 7 Channel Capacity
31/35
Discrete Memoryless Channel
Continuous-continuousCapacity is (Shannon)
)(
)(
1
t f x
t xM
iii
!
!
)(
)(
1
t f n
t n M
iii
!!
!
! M
iii t f y
t y
1
)(
)(
);(1maxlim)(
Y X I T
C x pT gp
!
ii
N
i i
ii jii
N N N
N N N N N N N
d x yd y p
x y p x p x y p
d d p
p p p I
WT N
N N
!
g
g
g
g!
!
!
1 )()(
log)()(
)()(log)()();(
2
XY
yxy
xyxxyYX ..
22 2/)(
21)|( iii x y
i
iie x y p
W
TW
!
-
8/8/2019 Lecture 7 Channel Capacity
32/35
Discrete Memoryless Channel
Continuous-continuousMaximum Information when:
!
!
!
!
0
2
0
2
1 0
2
21
)(
21log
21log
21
21log);(max
N W T
N N
N I
x
x
N
i
x N N
x p
W
W
W YX
222/
21)( xi x
x
i e x pW
TW ! Statistically independentzero mean Gaussian inputs
then
-
8/8/2019 Lecture 7 Channel Capacity
33/35
-
8/8/2019 Lecture 7 Channel Capacity
34/35
Discrete Memoryless Channel
Continuous-continuousThus Capacity is:
)(
)(
1
t f x
t xM
iii
!
!
)(
)(
1
t f n
t n M
iii
!!
!
! M
iii t f y
t y
1
)(
)(
!
!
!
gp
gp
0
0
2
)(
1log
21loglim
);(1
maxlim
W N
P W
N
W
I T
C
a v
x
T
N N x pT
W
YX
22 2/)(
21)|( iii x y
i
iie x y p
W
TW
!
-
8/8/2019 Lecture 7 Channel Capacity
35/35
Discrete Memoryless Channel
Continuous-continuousThus Normalized Capacity is:
)(
)(
1
t f x
t xM
iii
!
!
)(
)(
1
t f n
t n M
iii
!!
!
! M
iii t f y
t y
1
)(
)(
W C
N
WN
C
C P WN P
W
C
W C b
b
bavav
/
12
1log
but,1log
/
0
0
2
02
!
!
!
!
I
I I
22 2/)(
21)|( iii x y
i
iie x y p
W
TW
!
etab/No (d C/W-1.44036 0.1-1. 36402 0.15-1.24869 0.225-1.07386 0.3375
-0.8075 0.50625-0.39875 0.759375
0.234937 1.1390631.230848 1.7085942.822545 2.562891
5.41099 3.8443369.669259 5.76650416.65749 8.64975627.92605 12.9746345.69444 19.4619573.22669 29.19293115.4055 43.78939179.5542 65.68408 0.1
1
10
-10 0 10 20 30