with applications in image processing
DESCRIPTION
With Applications in Image Processing. The Story of Wavelets. Polikar, (C) 2007. Guest Lecture Dr. Robi Polikar. Outline. Review of Fourier based methods Analyzing non-stationary signals Short-time Fourier transform Continuous wavelet transform Discrete wavelet transform Applications - PowerPoint PPT PresentationTRANSCRIPT
With Applications in With Applications in Image ProcessingImage Processing
Guest LectureDr. Robi Polikar
OutlineOutline
Review of Fourier based methods Analyzing non-stationary signals
Short-time Fourier transform Continuous wavelet transform Discrete wavelet transform Applications
Discontinuity detectionSignal denoisingCompression
2-D wavelet transformImage analysis and compression
What is a TransformWhat is a Transformand Why Do we Need One ?and Why Do we Need One ?
Transform: A mathematical operation that takes a function or sequence and maps it into another one
Transforms are good things because… The transform of a function may give additional /hidden
information about the original function, which may not be available /obvious otherwise
The transform of an equation may be easier to solve than the original equation (recall Laplace transforms for “Defeques”)
The transform of a function/sequence may require less storage, hence provide data compression / reduction
An operation may be easier to apply on the transformed function, rather than the original function (recall convolution)
Properties of TransformsProperties of Transforms
Most useful transforms are: Linear: we can pull out constants, and apply
superposition
One-to-one: different functions have different transforms Invertible:for each transform T, there is an inverse
transform T-1 using which the original function f can be recovered (kind of – sort of the undo button…)
Continuous transform: map functions to functions Discrete transform: map sequences to sequences Semi-discrete transform: relate functions with sequences
Tf F T-1 f
( ) ( ) ( )T f g T f T f
What Does a Transform What Does a Transform Look Like…?Look Like…?
Complex function representation through simple building blocks
Compressed representation through using only a few blocks (called basis functions / kernels)
Sinusoids as building blocks: Fourier transform Frequency domain representation of the
function
)(),()(
s)(continuou )(),()(
(discrete)
dxxfxKF
dFxKxf
fKFKFf
ionBasisFunctweightFunctionComplex
jjijiij
ii
ii
i
Fourier series
Continuous Fourier transform
Discrete Fourier transform
Laplace transform
Z-transform
What Transforms Are Available?What Transforms Are Available?
1
0
/2/2 ][21)( )(][
N
i
NktjNktj ekFtfdtetfkF
deFtfdtetfF tjtj )(21)( )()(
1
0
/21
0
/2 ][21 ][][
N
k
NkjN
n
Nkj ekFf[n]enfkF
dsesFtfdtetfsF stst )(21)( )()(
1
0
1
0][
21 ][][
N
z
znN
n
zn ezFf[n]enfzF
How Does FT Work Anyway?How Does FT Work Anyway?
Recall that FT uses complex exponentials (sinusoids) as building blocks.
For each frequency of complex exponential, the sinusoid at that frequency is compared to the signal.
If the signal consists of that frequency, the correlation is high large FT coefficients.
If the signal does not have any spectral component at a frequency, the correlation at that frequency is low / zero, small / zero FT coefficient.
deFtfdtetfF tjtj )(21)( )()(
tjte tj sincos
FT At WorkFT At Work
)52cos()(1 ttx
)252cos()(2 ttx
)502cos()(3 ttx
FT At WorkFT At Work
)(1 X
)(2 X
)(3 X
F)(1 tx
F)(2 tx
F)(3 tx
FT At WorkFT At Work
)502cos()252cos(
)52cos()(4
tt
ttx
)(4 XF)(4 tx
Stationary and Stationary and Non-stationary SignalsNon-stationary Signals
FT identifies all spectral components present in the signal, however it does not provide any information regarding the temporal (time) localization of these components. Why?
Stationary signals consist of spectral components that do not change in time all spectral components exist at all timesno need to know any time informationFT works well for stationary signals
However, non-stationary signals consists of time varying spectral componentsHow do we find out which spectral component appears
when?FT only provides what spectral components exist , not
where in time they are located.Need some other ways to determine time localization of
spectral components
Stationary and Stationary and Non-stationary SignalsNon-stationary Signals
Stationary signals’ spectral characteristics do not change with time
Non-stationary signals have time varying spectra
)502cos()252cos(
)52cos()(4
tt
ttx
][)( 3215 xxxtx
Concatenation
Non-stationary Signals Non-stationary Signals
5 Hz 20 Hz 50 Hz
Perfect knowledge of what frequencies exist, but no information about where these frequencies are located in time
FT ShortcomingsFT Shortcomings
Complex exponentials stretch out to infinity in timeThey analyze the signal globally, not locallyHence, FT can only tell what frequencies exist
in the entire signal, but cannot tell, at what time instances these frequencies occur
In order to obtain time localization of the spectral components, the signal need to be analyzed locally
HOW ?
Short Time Short Time Fourier Transform (STFT)Fourier Transform (STFT)
1. Choose a window function of finite length2. Put the window on top of the signal at t=03. Truncate the signal using this window4. Compute the FT of the truncated signal, save.5. Slide the window to the right by a small amount6. Go to step 3, until window reaches the end of the signal For each time location where the window is centered, we
obtain a different FT Hence, each FT provides the spectral
information of a separate time-slice of the signal, providing simultaneous time and frequency information
300 Hz 200 Hz 100Hz 50Hz
STFT ExampleSTFT Example
2/2
)( atet
STFT ExampleSTFT Example
a=0.01
STFT ExampleSTFT Example
a=0.001
STFT ExampleSTFT Example
a=0.0001
STFT ExampleSTFT Example
a=0.00001
STFT ExampleSTFT Example
Time – Frequency Time – Frequency ResolutionResolution
Time – frequency resolution problem with STFTAnalysis window dictates both time and
frequency resolutions, once and for allNarrow window Good time resolutionNarrow band (wide window) Good
frequency resolution When do we need good time resolution, when do we
need good frequency resolution?
Heisenberg PrincipleHeisenberg Principle
41
ft
Time resolution: How well two spikes in time can be separated from each other in the transform domain
Frequency resolution: How well two spectral components can be separated from each other in the transform domain
Both time and frequency resolutions cannot be arbitrarily high!!! We cannot precisely know at what time instance a frequency component is located. We can only know what interval of frequencies are present in which time intervals
The Wavelet TransformThe Wavelet Transform
Overcomes the preset resolution problem of the STFT by using a variable length window
Analysis windows of different lengths are used for different frequencies:Analysis of high frequencies Use narrower
windows for better time resolutionAnalysis of low frequencies Use wider windows
for better frequency resolution This works well, if the signal to be analyzed mainly consists of slowly varying
characteristics with occasional short high frequency bursts. Heisenberg principle still holds!!! The function used to window the signal is called the wavelet
1:44,500,000 1:2,500,000
1:62,5001:375,500
frequencyscale 1
Continuous Wavelet TransformContinuous Wavelet Transform
dtabttx
abaWbaCWTx )(1),(),()(
Normalization factor
Mother wavelet translation
Scaling:Changes the support of the wavelet based on the scale (frequency)
CWT of x(t) at scalea and translation bNote: low scale high frequency
Why Wavelet?Why Wavelet?
We require that the wavelet functions, at a minimum, satisfy the following:
0)( dtt
dtt 2)(
Wave…
…let
Time-Frequency GridTime-Frequency Gridin CWTin CWT
Time
Scal
e
Δf/2
Δf
2Δf
4Δf
Δt
2Δt
4Δt
Δt/2
Δt/4a=1/4
a=1/2
a=1
a=2
Δt * Δf ≥ ½
Matlab Demos on CWTMatlab Demos on CWT
Discretization of Discretization of Time & Scale ParametersTime & Scale Parameters
Recall that, if we use orthonormal basis functions as our mother wavelets, then we can reconstruct the original signal by
where W(a,b) is the CWT of x(t) Q: Can we discretize the mother wavelet a,b(t) in such a way that
a finite number of such discrete wavelets can still form an orthonormal basis (which in turn allows us to reconstruct the original signal)? If yes, how often do we need to sample the translation and scale parameters to be able to reconstruct the signal?
A: Yes, but it depends on the choice of the wavelet!
)(),,()( , tbaWtx ba
Dyadic GridDyadic Grid We do not have to use a uniform sampling rate for the translation
parameters, since we do not need as high time sampling rate when the scale is high (low frequency).
Let’s consider the following sampling grid:
log a
b where a is sampled on a log scale, and b is sampled at a higher rate when a is small,that is,
where a0 and b0 are constants,and j,k are integers.
00
0
bakb
aaj
j
An example on MRAAn example on MRA
dftfk
kjj
j
j
)(21)(
)1(2
2
Zkjktk jjj
,222
0)(lim
)()(lim
tf
tftf
jj
jj
Let’s suppose we wish to approximate the function f(t) using a simpler function that has constant values over set intervals of 2j, such that the error ||f(t)-fj(t)|| is minimum for any j. The best approx. would be the average of the function over the interval length. In general, then…
j=
j=-
j=-2 j=-1
j=0 j=1
j=2 j=4
j=5
f(t)
f0(t)
f2(t)
An Example (cont.)An Example (cont.) As we go from a value of j to a higher value, we obtain a coarser
approximation of the function, hence some detail is lost. Let’s denote the detail lost at each approximation level with a
function g(t):
)()()()( 11 tftftftf jj
)()()( 1 tftftg jjj
Detail functionApproximation function
)()()(
)()()(
010
121
tftftg
tftftg jjj
jj tgtf )()( That is, the original function
can be reconstructed simply by adding all the details (!)
)0( f
Scaling & Wavelet FunctionsScaling & Wavelet Functions
dtttxkja kj )()(),( ,
Approximation coefficientsat scale j
kkjjj tkjatxtf )(),()()( ,
Smoothed, scaling-function-dependentapproximation of x(t) at scale j)2(2)( 2/
, ktt jjkj
)()( ,, tdtgk
kjkjj
kj
kj
d
dtttxbaW
,
, )()(),(
ktt jjkj 22)( 2/
, Detail coefficients
at scale j
Wavelet-function-dependentdetail of x(t) at scale j
Furthermore:
jj tgtf )()( & )()( ,, tdtg
kkjkjj
j kkj tkjdtxtf )(),()()( , Wavelet
Reconstruction
The scaling function
The wavelet function
Putting ItPutting ItAll Together (cont.)All Together (cont.)
We can start reconstruction from any level (scale) in between:
0
0 0, , ,( ) ( ) ( , ) ( )j
j k j k j kk j k
f t x t a d j k t
Approximation coefficientsat some arbitrary scale j0
Smoothed, scaling-function-dependentapproximation of x(t) at the scale j0
Detail coefficientsat scale j0 and below
Wavelet-function-dependentdetails of x(t) at scales j0 and below
0 0j jf t x t 0j
g t
0
0 0
j
j jj
x t x t g t
The essence of wavelet transform: A signal can be obtained from (or decomposed into) an approximation at a specific level, and sum of details lost up to that level.
Discrete Wavelet TransformDiscrete Wavelet TransformImplementationImplementation
G
H
2
2 G
H
2
2
2
2
G
H
+
2
2
G
H
+
x[n]x[n]
Decomposition Reconstruction
~
~ ~
~
n
high kngnxky ]2[][][~
n
low knhnxky ]2[][][~
k
high kngky ]2[][
k
high kngky ]2[][
2-level DWT decomposition. The decomposition can be continues as 2-level DWT decomposition. The decomposition can be continues as long as there are enough samples for down-sampling.long as there are enough samples for down-sampling.
G
H
Half band high pass filter
Half band low pass filter
2
2
Down-sampling
Up-sampling
DWT - DemystifiedDWT - DemystifiedLength: 512B: 0 ~
g[n] h[n]
g[n] h[n]
g[n] h[n]
2
d1: Level 1 DWTCoeff.
Length: 256B: 0 ~ /2 HzLength: 256
B: /2 ~ Hz
Length: 128B: 0 ~ /4 HzLength: 128
B: /4 ~ /2 Hz
d2: Level 2 DWTCoeff.
d3: Level 3 DWTCoeff.
…a3….
Length: 64B: 0 ~ /8 HzLength: 64
B: /8 ~ /4 Hz
2
2 2
22
|H(jw)|
w/2-/2
|G(jw)|
w- /2-/2a2
a1
Level 3 approximation
Coefficients
Matlab ImplementationMatlab ImplementationCommand LineCommand Line
Single level decomposition and reconstruction [cA1 cD1]=dwt(x, ‘db4’) single level decomposition x=idwt(cA1, cD1, ‘db4’):3 single level inverse transform
Direct reconstruction of approximation and detail signals D1=upcoef(‘d’, cD1, ‘db4’, 1, N): one-level reconstruction of detail
signal from detail coefficient A1=upcoef(‘a’, cA1, ‘db4’, 1, N): one level reconstruction of
approximation signal from approximation coefficients Multi-level decomposition & reconstruction
[C L]=wavedec(x, K, ‘db4’): k-level decomposition A0=waverec(C, L, ‘db4’): reconstruct the signal
Extracting coefficients cA3=appcoef(C, L, ‘db4’, 3); cD5=detcoef(C, L, 5);
Single branch reconstruction A3=wrcoef(‘a’, C, L, ‘db4’, 3); Reconstruct level 3 approx. signal D2=wrcoef(‘d’, C, L, ‘db4’, 2); Reconstruct level 2 detail signal
A3 D3 D2 D1
L=[lA3 lD3 lD2 lD1 lsignal]
lA3 lD3 lD2 lD1
C=
Quadrature Mirror FiltersQuadrature Mirror Filters
It can be shown that
that is, h[] and g[] filters are related to each other: in fact, that is, h[] and g[] are mirrors of each
other, with every other coefficient negated. Such filters are called quadrature mirror filters. For example, Daubechies wavelets with 4 vanishing moments…..
1)()(1)()(2~2~22 jGjHjGjH
][)1(]1[ ngnLh n
DB-4 WaveletsDB-4 Wavelets
h = -0.0106 0.0329 0.0308 -0.1870 -0.0280 0.6309 0.7148 0.2304
g = -0.0106 -0.0329 0.0308 0.1870 -0.0280 -0.6309 0.7148 -0.2304
h = 0.2304 0.7148 0.6309 -0.0280 -0.1870 0.0308 0.0329 -0.0106
g = -0.0106 -0.0329 0.0308 0.1870 -0.0280 -0.6309 0.7148 -0.2304
~~
][][ ],[][
][)1(]1[~~
ngngnhnh
ngnLh n
L: filter length (8, in this case)
Implementation of DWT on Implementation of DWT on MATLABMATLAB
Load signal
Choose waveletand numberof levels
Hit Analyze button
Level 1 coeff.Highest freq.
Approx. coef. at level 5
s=a5+d5+…+d1
(Wavedemo_signal1)
Details vs. ApproximationsDetails vs. Approximations
ApplicationsApplications
Detect discontinuities
ApplicationsApplications
Detect hiddendiscontinuities
ApplicationsApplications
Simpledenoising
CompressionCompression
DWT is commonly used for compression, since most DWT are very small, can be zeroed-out!
CompressionCompression
1D-DWT1D-DWT2D-DWT2D-DWT
Recall fundamental concepts of 1D-DWTProvides time-scale (frequency) representation of
non-stationary signalsBased on multiresolution approximation (MRA)
• Approximate a function at various resolutions using a scaling function, (t) • Keep track of details lost using wavelet functions, (t)• Reconstruct the original signal by adding approximation and detail coeff.
Implemented by using a series of lowpass and highpass filters
• Lowpass filters are associated with the scaling function and provide approximation
• Highpass filters are associated with the wavelet function and provide detail lost in approximating the signal
2-D DWT2-D DWT
How do we generalize these concepts to 2D? 2D functions images f(x,y) I[m,n] intensity function What does it mean to take 2D-DWT of an image? How do we
interpret?How can we represent an image as a function?How do we define low frequency / high frequency in
an image?How to we compute it?
Why would we want to take 2D-DWT of an image anyway?CompressionDenoisingFeature extraction
2D Scaling/Wavelet Functions2D Scaling/Wavelet Functions
We start by defining a two-dimensional scaling and wavelet functions:
If (t) is orthogonal to its own translates,
is also orthogonal to its own translates. Then, if fo(x,y) is the projection of f(x,y) on the space Vo generated by s(x,y):
)()(),( yxyxs
Zlklykxs ,:),(
),(),,(),(
),(),(),(
jyixsyxfjia
jyixsjiayxf
o
i joo
)()(),( yxyxs
2D-DWT2D-DWT
Just like in 1D we generated an approximation of the 2D function f(x,y). Now, how do we compute the detail lost in approximating this function?
Unlike 1D case there will be three functions representing the details lost:Details lost along the horizontal directionDetails lost along the vertical directionDetails lost along the diagonal direction
1D Two sets of coeff.; a(k,n) & d (k,n) 2D Four sets of coefficients: a(k,n), b(k, n), c(k, n) & d(k,n)
Four Faces of 2D-DWTFour Faces of 2D-DWT
One level of 2D DWT reconstruction:
)()(),()()(),()()(),(
)()(),(),(1
nxnxpndnxnxpncnxnxpnb
nxnxpnayxf
o
o
o
n po
Approximation coefficients
Detail coefficients along the horizontal directionDetail coefficients along the vertical directionDetail coefficients along the diagonal direction
Implementation of 2D-DWTImplementation of 2D-DWT
INPUTIMAGE…
…
……
ROW
S
COLUMNSH~ 2 1
G~ 2 1
H~ 1 2
G~ 1 2
H~ 1 2
G~ 1 2
ROWS
ROWS
COLUMNS
COLUMNS
COLUMNS
LL
LH
HL
HH
)(1
hkD
)(1
vkD
)(1
dkD
1kA
INPUTIMAGE
LL LH
HL HH
LLLH
HL HH
LHH
LLH
LHL
LLLLH
HL HH
LHH
LLH
LHL
Up and Down … Up and DownUp and Down … Up and Down
2 1Downsample columns along the rows: For each row, keep the even indexed columns, discard the odd indexed columns
1 2Downsample columns along the rows: For each column, keep the even indexed rows, discard the odd indexed rows
2 1
1 2
Upsample columns along the rows: For each row, insert zeros at between every other sample (column)
Upsample rows along the columns: For each column, insert zeros at between every other sample (row)
Implementing 2D-DWTImplementing 2D-DWTDecomposition
ROW i
COLU
MN
j
ReconstructionReconstruction
)(1
hkD
)(1
vkD
)(1
dkD
1kA 1 2
1 2
1 2
1 2
H
G
H
G
2 1
2 1
H
G
ORIGINALIMAGE
LL
LH
HL
HH
2-D DWT ON MATLAB2-D DWT ON MATLAB
Load Image
(must be.mat file)
Choosewavelet type
HitAnalyze
Choosedisplayoptions
2D Wavelets2D Wavelets
Other uses for Wavelets in image processing (as seen in later slides)
1) Image Compression
The Discrete wavelet transform decomposes an image into a set of successfully smaller orthogonal images. Often it is possible to coarsely quantize or eliminate low valued coefficients without sacrificing the integrity of the image
5MB2.5MB
5MB
2D Wavelets2D Wavelets
2) Image Enhancement
Since the DWT decomposes the image into components of different size, position and orientation, you can alter the coefficients before reconstruction so that you can attenuate certain characteristics of the image.
2D Wavelets2D Wavelets
3) Image Fusion
Image fusion combines 2 or more registered images of the same object into a single object, that in most cases is better than the original. This is helpful in the medical imaging field where multiple images from different machines. The images are combined in the transform domain by taking the highest amplitude coefficient then performing the inverse on the new fused image.
Run Image Fusion GUI
Image DenoisingImage Denoising
Project Idea Project Idea
Implement JPEG 2000A much improved form of JPEG compression
that primarily uses the wavelet based compression idea we just discussed.