designs of orthogonal filter banks and orthogonal cosine ...jyan/publications/thesis slides.pdf ·...
TRANSCRIPT
Designs of Orthogonal Filter Banks andOrthogonal Cosine-Modulated Filter Banks
Jie Yan
Department of Electrical and Computer EngineeringUniversity of Victoria
April 16, 2010
1 / 45
OUTLINE
1 INTRODUCTION
2 LS DESIGN OF ORTHOGONAL FILTER BANKS ANDWAVELETS
3 MIMINAX DESIGN OF ORTHOGONAL FILTER BANKS ANDWAVELETS
4 DESIGN OF ORTHOGONAL COSINE-MODULATED FILTERBANKS
5 CONCLUSIONS AND FUTURE RESEARCH
2 / 45
1. INTRODUCTION
A two-channel conjugate quadrature (CQ) filter bank
H1(z) = −z−(N−1)H0(−z−1)G0(z) = H1(−z)G1(z) = −H0(−z)
where H0(z) =∑N−1
n=0 hnz−n
0 ( )H z
1( )H z
2
2
2
2
0 ( )G z
1( )G z}Analysis Filter Bank
}Synthesis Filter Bank
3 / 45
Two-Channel Orthogonal Filter Banks
Perfect reconstruction (PR) condition
N−1−2m∑n=0
hn ⋅ hn+2m = �m for m = 0, 1, ..., (N − 2)/2
Vanishing moment (VM) requirement:
A CQ filter has L vanishing moments if
N−1∑n=0
(−1)n ⋅ nl ⋅ hn = 0 for l = 0, 1, ...,L− 1
4 / 45
Two-Channel Orthogonal Filter Banks Cont’d
A least squares (LS) design of CQ lowpass filter H0(z) having LVMs
minimize∫ �
!a
∣H0(ej!)∣2d!
subject to: PR condition and VM requirement
The LS problem above can be expressed as
minimize hTQh
subject to:N−1−2m∑
n=0
hn ⋅ hn+2m = �m for m = 0, 1, ..., (N − 2)/2
N−1∑n=0
(−1)n ⋅ nl ⋅ hn = 0 for l = 0, 1, ...,L− 1
5 / 45
Two-Channel Orthogonal Filter Banks Cont’d
A minimax design minimizes the maximum instantaneous powerof H0(z) over its stopband
minimize maximize!a≤!≤�
∣H0(ej!)∣
subject to: PR condition and VM requirement
The minimax problem can be further cast as
minimize �
subject to: ∥T(!) ⋅ h∥ ≤ � for ! ∈ ΩN−1−2m∑
n=0
hn ⋅ hn+2m = �m for m = 0, 1, ..., (N − 2)/2
N−1∑n=0
(−1)n ⋅ nl ⋅ hn = 0 for l = 0, 1, ...,L− 1
6 / 45
Orthogonal Cosine-Modulated Filter Banks
An orthogonal cosine-modulated (OCM) filter bank
hk(n) = 2h(n) cos[�
M
(k +
12
)(n− D
2
)+ (−1)k�
4
]fk(n) = 2h(n) cos
[�
M
(k +
12
)(n− D
2
)− (−1)k�
4
]for 0 ≤ k ≤ M − 1 and 0 ≤ n ≤ N − 1
x(n) y(n)H0(z)
HM-1(z)
M M
M M
M M
. . .
. . .
H1(z)
F0(z)
FM-1(z)
F1(z)
+
} }
7 / 45
Orthogonal Cosine-Modulated Filter Banks Cont’d
An M-channel OCM filter bank is uniquely characterized by itsprototype filter (PF)The design of the PF of an OCM filter bank can be formulated as
minimize∫ �
!s
∣H0(ej!)∣2d!
subject to: PR condition
As the PF has linear phase, h is symmetrical. The design problemcan be reduced to
minimize e2(h) = hT Phsubject to: al,n(h) = hTQl,nh− cn = 0
for 0 ≤ n ≤ m− 1 and 0 ≤ l ≤ M/2− 1
where the design variables are reduced by half toh = [h0 h1 ⋅ ⋅ ⋅ hN/2−1]T .
8 / 45
Overview and Contribution of the Thesis
Overview
We have formulated three nonconvex optimization problems
LS design of CQ filter banks
Minimax design of CQ filter banks
Design of OCM filter banks
Contribution of the thesis
Several improved local design methods for the three problems
Several strategies proposed for potentially GLOBAL solutions ofthe three problems
9 / 45
Global Design Method at a Glance
Multiple local solutions exist for a nonconvex problem
Algorithms in finding a locally optimal solution are available
Start the local design algorithm from a good initial point
How do we secure such a good initial point?
10 / 45
2. LS DESIGN OF ORTHOGONAL FILTER BANKS AND WAVELETS
A least squares (LS) design of a conjugate quadrature (CQ) filter
of length-N with L vanishing moments (VMs) can be cast as
minimize hTQh
subject to:N−1−2m∑
n=0
hn ⋅ hn+2m = �m for m = 0, 1, ..., (N − 2)/2
N−1∑n=0
(−1)n ⋅ nl ⋅ hn = 0 for l = 0, 1, ...,L− 1
11 / 45
Local LS Design of CQ Filter Banks
An effective direct design method is recently proposed by
W.-S. Lu and T. Hinamoto
Based on the direct design technique, we develop two local
methods
Sequential convex-programming (SCP) method
Sequential quadratic-programming (SQP) method
Both methods produce improved local designs than the directmethod
12 / 45
Local LS Design of CQ Filter Banks Cont’d
Sequential Convex-Programming Method
Suppose we are in the kth iteration to compute �h so that hk+1 = hk + �h
reduces the filter’s stopband energy and better satisfies the constraints, then
hTk+1Qhk+1 = �T
h Q�h + 2�Th Qhk + hT
k Qhk
N−1∑n=0
(−1)n ⋅ nl ⋅ (�h)n = −N−1∑n=0
(−1)n ⋅ nl ⋅ (hk)n
N−1−2m∑n=0
(hk)n(�h)n+2m +
N−1−2m∑n=0
(hk)n+2m(�h)n
≈ �m −N−1−2m∑
n=0
(hk)n(hk)n+2m
13 / 45
Local LS Design of CQ Filter Banks Cont’d
With h bounded to be small, the kth iteration assumes the form
minimize �Th Q�h + �T
h gk
subject to: Ak�h = −ak
C�h ≤ b
By using SVD to remove the equality constraint, the problem is reduced to
minimize xTQx + xT gk
subject to: Cx ≤ b
We modify the problem to make it always feasible as
minimize xTQx + xT gk
subject to: Fx ≤ a
which is a convex QP problem.
14 / 45
Local LS Design of CQ Filter Banks Cont’d
Sequential Quadratic-Programming Method
The design problem is a general nonlinear optimization problem
minimize f (h)
subject to: ai(h) = 0 for i = 1, 2, ..., p
By using the first-order necessary conditions of a local minimizer,
the problem can be reduced to
minimize12�T
h Wk�h + �Th gk
subject to: Ak�h = −ak
∣∣�h∣∣ is small
15 / 45
Local LS Design of CQ Filter Banks Cont’d
where
Wk = ∇2hf (hk)−
p∑i=1
(�k)i∇2hai(hk) (13a)
Ak =[∇ha1(hk) ∇ha2(hk) ⋅ ⋅ ⋅ ∇hap(hk)
]T(13b)
gk = ∇hf (hk) (13c)
ak =[
a1(hk) a2(hk) ⋅ ⋅ ⋅ ap(hk)]T (13d)
By removing the equality constraint using the SVD or QR decomposition, theproblem assumes the form of a QP problem. Once the minimizer �∗h is found,the next iterate is set to
hk+1 = hk + �∗h , �k+1 = (AkATk )−1Ak(Wk�
∗h + gk)
16 / 45
Global LS Design of Low-Order CQ Filter Banks
The LS design problem is a polynomial optimization problem
(POP)
Two recent breakthroughs in solving POPs
Global solutions of POPs are made available by Lasserre’s
method
Sparse SDP relaxation is proposed for global solutions of
POPs of relatively larger scales
MATLAB toolbox SparsePOP and GloptiPoly can be used to
find global solutions of POPs, but only for POPs of limited sizes
17 / 45
Global LS Design of Low-Order CQ Filter Banks Cont’d
Example: Design a globally optimal LS CQ filter with N = 6, L = 2 and!a = 0.56�
MATLAB toolbox GloptiPoly and SparsePOP are utilized toproduce the globally optimal solution
h(6,2)LS =
⎡⎢⎢⎢⎢⎢⎢⎣
0.332680987886290.806895914548490.45986215652386−0.13501431772967−0.08543638600240
0.03522516035714
⎤⎥⎥⎥⎥⎥⎥⎦However, GloptiPoly and SparsePOP fail to work as long as thefilter length N is greater than or equal to 18
18 / 45
Global LS Design of High-Order CQ Filter Banks
A common pattern shared among globally optimal low-order impulseresponses.
0 0.2 0.4 0.6 0.8 1
−0.2
−0.1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
N = 6, L = 2N = 8, L = 2N = 10, L = 2
19 / 45
Global LS Design of High-Order CQ Filter Banks Cont’d
h6: Globally optimal impulse response when N = 6hzp
8 : Impulse response generated by zero-padding h6h8: Globally optimal impulse response when N = 8
0 0.2 0.4 0.6 0.8 1
−0.2
−0.1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
h6 (N=6, L=2)
h8zp
h8 (N=8, L=2)
Generate initial point by zero-padding!20 / 45
Global LS Design of High-Order CQ Filter Banks Cont’d
Global design strategy in brief:
1 Design a globally optimal CQ filter of short length, say 4, using
e.g. GloptiPoly
2 Generating an impulse response for higher order design by
zero-padding
3 Apply the SCP or SQP method with the zero-padded impulse
response as the initial point to obtain the optimal impulse
response of higher order
4 Follow this concept in an iterative way, until desired filter length
is reached
21 / 45
Global LS Design of High-Order CQ Filter Banks Cont’d
The designs obtained are quite likely to be globally optimal because:1 Zero-padded initial point sufficiently close to the global minimizer.2 The local design methods are known to converge to a nearby
minimizer.
22 / 45
Design Examples
Potentially globally optimal design of an LS CQ filter with N = 96, L = 3and ! = 0.56�
0 0.2 0.4 0.6 0.8 1−120
−100
−80
−60
−40
−20
0
Normalized frequency
23 / 45
Design Examples Cont’d
ComparisonsGlobal design
Global design based on SCPEnergy in stopband 1.18103e-9
Largest eq. error 1e-14
Local design
Local design based on SCPEnergy in stopband 3.15564e-9
Largest eq. error 1e-14
24 / 45
Design Examples Cont’d
Zero-pole plots
−1 −0.5 0 0.5 1
−1
−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
0.8
1
4 95
Real Part
Imag
inar
y Pa
rt
−1.5 −1 −0.5 0 0.5 1 1.5
−1
−0.5
0
0.5
1
3 95
Real PartIm
agin
ary
Part
Global design Local design
The globally optimal LS CQ filter possesses minimum phase
25 / 45
3. MIMINAX DESIGN OF ORTHOGONAL FILTER BANKS ANDWAVELETS
A minimax design of a conjugate quadrature (CQ) filter of length-N
with L vanishing moments (VMs) can be cast as
minimize �
subject to: ∥T(!) ⋅ h∥ ≤ � for ! ∈ ΩN−1−2m∑
n=0
hn ⋅ hn+2m = �m for m = 0, 1, ..., (N − 2)/2
N−1∑n=0
(−1)n ⋅ nl ⋅ hn = 0 for l = 0, 1, ...,L− 1
26 / 45
Local Minimax Design of CQ Filter Banks
Like the LS design, an effective direct design method is recently
proposed by W.-S. Lu and T. Hinamoto
Based on the direct design technique, we develop an improved
method named the SCP-GN method
The SCP-GN method can achieve convergence at a small
tolerance ", by implementing two techniques
1 Constructing Ω by locating magnitude-response peaks
2 A Gauss-Newton method with adaptively controlled weights
27 / 45
Local Minimax Design of CQ Filter Banks Cont’d
In the kth iteration, the problem assumes the form
minimize �
subject to: ∥T(!)(hk + �h)∥ ≤ � for ! ∈ Ω
Ak�h = −ak
C�h ≤ b
By using SVD of matrix Ak to remove the equality constraints, the problem can bereduced to
minimize �
subject to: ∥Tk(!)x + ek(!)∥ ≤ � for ! ∈ Ω
Cx ≤ b
As a technical remedy to make the above problem to be always feasible, we modifythe problem as
minimize �
subject to: ∥Tk(!)x + ek(!)∥ ≤ � for ! ∈ Ω
Fx ≤ a
which is a second-order cone programming (SOCP) problem for which efficientsolvers such as SeDuMi exist. 28 / 45
Global Minimax Design of Low-Order CQ Filter Banks
Example: Design a globally optimal minimax CQ filter with N = 4,
L = 1 and !a = 0.56�
Since the Minimax design problem is a POP, GloptiPoly and
SparsePOP can be used to produce the globally optimal solution
h(4,1)minimax =
⎡⎢⎢⎣0.482962821735310.836516231382340.22414405492402−0.12940935473280
⎤⎥⎥⎦However, GloptiPoly and SparsePOP fail to work as long as the
filter length N is greater than or equal to 6
29 / 45
Global Minimax Design of High-Order CQ Filter Banks
Method 1Globally optimal minimax impulse responses appear to exhibit a
pattern similar to that in the LS caseThus, we proposed method 1 in spirit similar to that utilized in the
global LS designs by passing the zero-padded impulse response
as the initial point for the SCP-GN local method in each round ofiteration
Method 2We simply pass the impulse response of the globally optimal LS
filter as an initial point for the SCP-GN method to design an
optimal minimax filter with the same design specifications
30 / 45
Design Examples
Potentially globally optimal design of a minimax CQ filter with N = 96,L = 3 and ! = 0.56�
0 0.2 0.4 0.6 0.8 1
−100
−80
−60
−40
−20
0
Normalized frequency
31 / 45
Design Examples Cont’d
ComparisonsGlobal design
Global design based on Method 1Maximum instantaneous
energy in stopband 6.75750e-9Largest eq. error <1e-15
Local design
Local design based on SCP-GNMaximum instantaneous
energy in stopband 1.81165e-8Largest eq. error 2.9e-14
32 / 45
Design Examples Cont’d
Zero-pole plots
−1 −0.5 0 0.5 1
−1
−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
0.8
1
4 95
Real Part
Imag
inar
y Pa
rt
−1 −0.5 0 0.5 1 1.5
−1
−0.5
0
0.5
1
4 95
Real Part
Imag
inar
y Pa
rt
Global design Local design
The globally optimal minimax CQ filter possesses minimum phase
33 / 45
4. DESIGN OF ORTHOGONAL COSINE-MODULATED FILTERBANKS
We have formulated the design of the prototype filter (PF) of an
orthogonal cosine-modulated (OCM) filter bank as
minimize e2(h) = hT Phsubject to: al,n(h) = hTQl,nh− cn = 0
for 0 ≤ n ≤ m− 1 and 0 ≤ l ≤ M/2− 1
34 / 45
Local Design of OCM Filter Banks
We improved an effective direct design method proposed by
W.-S. Lu, T. Saramäki and R. Bregovic
Gauss-Newton method with adaptively controlled weights was
applied for the algorithm to converge to a highly accurate solution
35 / 45
Local Design of OCM Filter Banks Cont’d
Suppose we are in the kth iteration to compute � so that hk+1 = hk + �reduces the PF’s stopband energy and better satisfies the PR conditions.Then,
hTk+1Phk+1 = �T P� + 2�T Phk + hT
k Phk
al,n(hk + �) ≈ al,n(hk) + gTl,n(hk)� = 0
for 0 ≤ n ≤ m− 1 and 0 ≤ l ≤ M/2− 1
And the kth iteration assumes the form
minimize �T P� + �Tbk
subject to: Gk� = −ak
∥�∥ is small
36 / 45
Local Design of OCM Filter Banks Cont’d
The equality constraint can be eliminated via SVD of Gk = UΣV as
� = Ve� + �s (18)
Thus, the problem can be cast as
minimize �T Pk� + �T bk
subject to: ∣∣�∣∣ is small
The Gauss-Newton technique with adaptively controlled weights is used as
a post-processing step to achieve convergence at a small tolerance.
37 / 45
Global Design of Low-Order OCM Filter Banks
Example: Design a globally optimal OCM filter bank with M = 2, m = 1and � = 1
GloptiPoly and SparsePOP can be used to produce the globallyoptimal solution
h(2,1) =
⎡⎢⎢⎣0.2359234169663530.4408402673665810.4408402673665810.235923416966353
⎤⎥⎥⎦The software was found to work only for the following cases:a) M = 2, 1 ≤ m ≤ 5;b) M = 4, 1 ≤ m ≤ 3;c) M = 6, m = 1;d) M = 8, m = 1.
38 / 45
Global Design of High-Order OCM Filter Banks
Two observations:1 For a fixed M, the impulse responses with different m exhibit a
similar pattern and are close to each other2 For m = 1, the impulse responses with different M also exhibit a
similar shape.
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−0.05
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
m=1,M=2m=2,M=2m=3,M=2m=4,M=2m=1,M=4m=2,M=4
39 / 45
Global Design of High-Order OCM Filter Banks Cont’d
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−0.05
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
m=1,M=4
h0zp
m=2,M=4
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
m=1,M=2
h0int
m=1,M=4
Effect of zero-padding when M = 4 Effect of linear interpolation when m = 1
40 / 45
Global Design of High-Order OCM Filter Banks Cont’d
An improvement in initial point when m = 1, by downshifting hint0 by a
constant value d computed using the Gauss-Newton method withadaptively controlled weights.
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
h0int
h0
h
d
41 / 45
Global Design of High-Order OCM Filter Banks Cont’d
An order-recursive algorithm in brief
1 Obtaining a low-order global design;
2 Using zero-padding/linear interpolation in conjuction of the G-N
method with adaptively controlled weights of the impulse response
to produce a desirable initial point for PF of slightly increased
order, and carrying out the design by a locally optimal method;
3 Repeating step 2 until the filter order reaches the targeted value.
42 / 45
Design Examples
Design of an OCM filter bank with m = 20, M = 4 and � = 1. Shownbelow are the impulse responses of the PF from global and localdesign, respectively.
0 0.2 0.4 0.6 0.8 1−160
−140
−120
−100
−80
−60
−40
−20
0
20
Normalized frequency0 0.2 0.4 0.6 0.8 1
−200
−180
−160
−140
−120
−100
−80
−60
−40
−20
0
20
Normalized frequency
Global design Local design
43 / 45
Design Examples Cont’d
Performance comparison for OCM filter banks with m = 20, M = 4
and � = 1
Global design Local designEnergy in stopband 8.226e-13 6.585e-10
Largest eq. error 1.839e-15 2.297e-10
By comparing the OCM filter banks reported in the literature, the
OCM filter bank designed using our proposed algorithm offers the
BEST performance, because it is a globally optimal design.
44 / 45
5. CONCLUSIONS AND FUTURE RESEARCH
We have investigated three design problems,
1 LS design of orthogonal filter banks and wavelets
2 Minimax design of orthogonal filter banks and wavelet
3 Design of OCM filter banks
Improved local design methods for the three problems
Several strategies proposed for GLOBAL designs of the
three design scenarios
Future research
Theoretical proof of the global optimality of our proposed method
45 / 45