wavelet-based denoising using hidden markov models
Post on 05-Jan-2016
37 Views
Preview:
DESCRIPTION
TRANSCRIPT
Wavelet-Based Denoising Using Hidden Markov Models
M. Jaber Borran and Robert D. Nowak
Rice University
Some properties of DWT
• Primary– Locality Match more signals– Multiresolution
– Compression Sparse DWT’s
• Secondary– Clustering Dependency within scale
– Persistence Dependency across scale
Probabilistic Model for an Individual Wavelet Coefficient
• Compression many small coefficients
few large coefficients
S
W
pS(1)
fW|S(w|1)
pS(2)
fW|S(w|2)
fW (w)
Probabilistic Model for a Wavelet Transform
Ignoring the dependencies
Independent Mixture (IM) Model
Clustering
Hidden Markov Chain Model
Persistence
Hidden Markov Tree Model
t
f
Parameters of HMT Model
• pmf of the root node
• transition probability
• (parameters of the) conditional pdfs
e.g. if Gaussian Mixture is used
)(1
mpS
rmii
,)(,
)|()( |, mwfwfii SWmi
2,, and mimi
: Model Parameter Vector
Dependency between Signs of Wavelet Coefficients
SignalWavelet
0T
T
w1T
w1
0T/2
TT/2
T/2
w2
T/2
w2
T
T/2 w2
T/2 w2
T
New Probabilistic Model for Individual Wavelet Coefficients
S
W
pS(1)
fW|S(w|1)
pS(2)
fW|S(w|2)
fW (w)pS(3)
fW|S(w|3)
pS(4)
fW|S(w|4)
• Use one-sided functions as conditional probability densities
Proposed Mixture PDF
• Use exponential distributions as components of the mixture distribution
0 0
0 )|(
,
,|
w
weswf
wsi
SW
si
ii
0 0
0 )|(
,
,|
w
weswf
wsi
SW
si
ii
m even
m odd
PDF of the Noisy Wavelet Coefficients
yQesyf si
y
siSY
sisi
ii ,2
1
,|
22,,
)|( m even
),0(~ where, 2Nnnwy
yQesyf si
y
siSY
sisi
ii ,2
1
,|
22,,
)|(
Wavelet transform is orthonormal, therefore if the additive noise is white and zero-mean Gaussian process with variance then we have
Noisy wavelet coefficient,
m odd
Training the HMT Model
• y: Observed noisy wavelet coefficients
• s: Vector of hidden states• Model parameter vector
Maximum likelihood parameter estimation:
)|( maximize θyYθ
f
Intractable, because s is unobserved (hidden).
Model Training Using Expectation Maximization Algorithm
• and then,
N
i
ssiiS
N
iiiSY
ii
iispfsyff
ffff
2
,)(,1
1|
)(
1)()|( ,)|(),|(
)|(),|()|,()|(
θsθsy
θsθsyθsyθx
SY
SYYSX
• Define the set of complete data, x = (y,s)
ll fU θyθxθθ XSθ
,|)|(logE),( maximize
EM Algorithm (continued)
Nii
ii
N
M
N
iiiSY
N
i
ssiiS
l
M
ll
syfspf
ffU
,...,1 1|
2
,)(,1|
,...,1|
)|(loglog)(log),|(
)|,(log),|(),(
)(
1
sYS
sYSYS
θys
θsyθysθθ
• State a posteriori probabilities are calculated using Upward-Downward algorithm• Root state a priori pmf and the state transition probabilities are calculated using Lagrange multipliers for maximizing U.• Parameters of the conditional pdf may be calculated analytically or numerically, to maximize the function U.
Denoising
yQ
e
syfswyfswfsywf
wywy
SYWSYSWYSW
2
222
2
)()(
2
1
||||
2
1
)|(),|()|(),|(
• MAP estimate:
2| ),|(maxargˆ ysywfw YSW
ws
s
sYSWSs
ww
sywfsps
ˆ
|
ˆˆ
),|ˆ()(maxargˆ
Denoising (continued)
• Conditional Mean estimate:
s
M
sS wspw ˆ)(ˆ
1
)(2
,|Eˆ 22
12
yy
Q
esyww
y
s
0 0.5 1-100
0
100
200
Orig
inal
0 0.5 1-100
0
100
200
Noi
sy
0 0.5 1-100
0
100
200
4 G
auss
ian,
Haa
r
0 0.5 1-100
0
100
200
4 G
auss
ian,
D8
0 0.5 1-100
0
100
200
4 E
xpon
entia
l, H
aar
0 0.5 1-100
0
100
200
4 E
xpon
entia
l, D
8
Init. MSE = 24.639723 4 mix, Haar 4 mix, D8
Gaussian Mixture 3.078267 7.020152Exponential Mixture 2.326472 7.030970
0 0.5 1-20
0
20
Orig
inal
0 0.5 1-20
0
20
Noi
sy
0 0.5 1-20
0
20
2 G
auss
ian,
D8
0 0.5 1-20
0
20
4 G
auss
ian,
D8
0 0.5 1-20
0
20
2 E
xpon
entia
l, D
8
0 0.5 1-20
0
20
4 E
xpon
entia
l, D
8
Init. MSE = 2.429741 2 mix, D8 4 mix, D8
Gaussian Mixture 0.471568 0.417795Exponential Mixture 0.426488 0.397808
0 0.5 1-200
-100
0
100
Orig
inal
0 0.5 1-200
0
200
Noi
sy
0 0.5 1-200
0
200
2 G
auss
ian,
D4
0 0.5 1-200
-100
0
100
4 G
auss
ian,
D8
0 0.5 1-200
-100
0
100
2 E
xpon
entia
l, D
4
0 0.5 1-200
-100
0
100
4 E
xpon
entia
l, D
8
Init. MSE = 92.907059 2 mix, D4 4 mix, D8
Gaussian Mixture 8.442306 7.873508Exponential Mixture 8.394187 7.862579
Conclusion
• We observed a high correlation between the signs of the wavelet coefficients in adjacent scales.
• We used one-sided distributions as mixture components for individual wavelet coefficients.
• We used hidden Markov tree model to capture the dependencies.
• The proposed method achieves better MSE in denoising and the denoised signals are much smoother.
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-100
-50
0
50
100
150Noisy Signal
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-60
-40
-20
0
20
40
60
80
100
120
140Denoised Using 4 Gaussian Mixture and Haar Wavelet
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-60
-40
-20
0
20
40
60
80
100
120
140Denoised Using 4 Exponential Mixture and Haar Wavelet
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-15
-10
-5
0
5
10
15
20Noisy Signal
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-15
-10
-5
0
5
10
15Denoised Using 4 Gaussian Mixture and Daubechy Length 8
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-15
-10
-5
0
5
10
15Denoised Using 4 Exponential Mixture and Daubechy Length 8
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-200
-150
-100
-50
0
50
100
150Noisy Signal
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-150
-100
-50
0
50
100Denoised Using 4 Gaussian Mixture and Daubechy Length 8
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-150
-100
-50
0
50
100Denoised Using 4 Exponential Mixture and Daubechy Length 8
top related