thomas abraham, phd · 1) take a guess at the object 2) convolve guess (object) with the psf. if...
Post on 27-Jun-2020
2 Views
Preview:
TRANSCRIPT
Thomas Abraham, PhD(tabraham1@hmc.psu.edu)
• Deconvolution, also termed as “Restoration” or “Deblurring” is an image processing technique used in a wide variety of fields from 1D spectroscopy to 3D biomedical imaging.
• It has been rigorously applied in the field of astronomy, because so much expense and time are spent in acquiring astronomical images.
What is Deconvolution?
Raw Image Deconvolved
• An image is produced by an instrument. It is not the same as the object but is some “photocopy” of the object.
What makes an image?
Convolution
• The PSF describes what the instrument does to the object to produce an image.
• In mathematical terms, the image is the object convolved by the ‘point spread function’ (PSF).
Image = Object PSF
• Every lens element in the optical path alters the object (specimen) in some way.
• We call this the point spread function (PSF).
• The PSF of the system is a convolution of all of the PSF’s of all the lens elements
and apertures in the optical path.
Point Spread Function {PSF}
• The image of that macromolecular object can be considered as the superposition of large number of points convolved with the PSF.
• Likewise, a macromolecular object can be considered as a large number of points arranged in 3D space.
• For instance, an image of point source is not a point but a point spread in 3D space.
Point Spread Function {PSF}
Point Source
Image
Image(x,y,z) = ∫∫∫Object(x’,y’,z’)psf(x-x’,y-y’,z-z’)dx’dy’dz’
Convolution Theorem
In mathematical terms, for each point, take the object and multiply it by the entire PSF centered at that point in the image and sum the results of the multiplication.
Convolution Theorem
Whatever we record as the image intensity is the convoluted product of the PSF and the object intensity.
Restoring the object intensity from the smear of PSF by some mathematical operations
Image = Object PSF
Image(x,y,z) = ∫∫∫Object(x’,y’,z’)psf(x-x’,y-y’,z-z’)dx’dy’dz’
Deconvolution
The Fourier Transform
I(x) ai(coskix) ibi(sinkix)i
Amplitudes Phase
From spatial domain to Frequency Domain
•Any image can be decomposed into a series of sine and cosine wave functions.
Low frequency(Coarse structural details)
High frequency (Fine structural details)
Fourier Transform of the elastin collagen fiber bundle in lung alveoli
FT
Closely related to the diffraction pattern of the object
Spatial Domain Frequency Domain
PSD
Fourier Transform, Examples
FT
FT
Dude (Big Labowski)
Spatial Domain Frequency Domain
Bad looking Vs Good Looking
Fourier Transform, Examples
FT
FT
Spatial Domain Frequency Domain
Unidirectional Objects Vs Randomly Oriented ObjectsFrequency distribution
is orthogonal
Fourier Transform, Examples
FFT
FFT
Spatial Domain Frequency Domain
Fine Vs Fuzzy Objects
Inverse Fourier Transform of the FourierTransform Returns the original Image
F -1
A powerful mathematical tool for image processing
=
Deconvolution using Fourier Transformation
It has been proved that the Fourier transform of the image is just the Fourier transform of object times the Fourier transform
of the PSF in frequency domain
So, to obtain the object, we simply divide Fourier transform of Image by the Fourier transform of the PSF
F {Image} = F {Object PSF}
F {Object} = F {Image} / F {PSF}
This is known as Optical Transfer Function or OTF or CTF
F {Image} = F {Object} X F {PSF}
By applying an inverse Fourier transform, We can restore object
(specimen)
Transforms back to spatial domain
Object (or sample) = F {Object} -1
Step 1: Acquire the image and obtain its Fourier transform
Convoluted Image{Object
Fourier Transform of Image ⊗ PSF}
Steps in Deconvolution
Step 2: Generate PSF and obtain its Fourier transform
Steps in Deconvolution
Point spread function Fourier transform of PSF
Step 3: Divide and inverse Fourier transform
F -1
Steps in Deconvolution
Object
FT
FT
Raw Image
Deconvolved Image( Restored Object)
F {Object} = F{Image} / F{PSF}
OTF or CTF
Object (or sample) = F.T. {Object} -1
Works perfectly if there is no noise in the image. But there is always noise
Various Sources of NOISEPhoton (or shot) noise: Photon detection is a quantum mechanical event. So, it is subjected to random fluctuations (or random noise) which is governed by Poisson statistics.
Electronic (thermal) noise: The light is detected electronically and all detectors have thermal noise.
Bit noise: Digital images have noise due to the number of bits (~8-14) used during A/D conversion.
Processing noise: The computer is operating on digital data with limited precision.
Effect of noise in data
More Noise
Relatively Noise FreeLess Noise
Each spatial frequency has an associated noise component resulting from various sources of the noise, and at high spatial frequencies (corresponds fine structural details) this can be very large.
Non-iterative Deconvolution
So simply dividing F{Image} by OTF will magnify noise components - this obscures real structural details.
Alternate approach is to modify OTF (or CTF) on the basis of local signal levels –Wiener filter.
F {Object} = F{Image} / F{PSF}
OTF or CTF
Deconvolution using Wiener filter
• Here an extra noise dependent factor (K) is added to the OTF values, so that in noisy regions the divisor is larger, and the noisy components are not boosted.
• One major drawback here is that we don’t actually know what is real signal and what is noise.
• This often results in rejecting genuinely real spatial information. Consequently the reconstructed image can be of lower resolution.
Deconvolution by Iteration1) Take a guess at the object2) Convolve guess (object) with the PSF. If PSF is unknown, guess the PSF too.3) If blurred guess = image then stop iteration, otherwise refine guess.
This is also known as maximum likelihood approach.
Richardson-Lucy algorithm (which includes Poisson Noise model) :
xy plane
xz plane
A
F
CB
D E
Abraham et al. (2010) Micron
xy plane
xz plane
A CB
Abraham et al. (2010) Micron
A B
C D
Abraham et al. (2010) Micron
• Imaging is a convolution of the object by the microscope’s PSF and an image is a convoluted product of the object and PSF.
• Deconvolution attempts to restore the various spatial frequency components to something nearer what they were in the original object before being attenuated by the optical path.
Summary
Recommended Readings:
Cannell, et al. (2006) Image Enhancement by Deconvolution, In Handbook of Biological Confocal Microscopy, Third Edition, Pawley, J. B, Ed., Springer Science + Business Media, LLC, New York
Shaw, P. (1994) Deconvolution in 3-D optical microscopy, HistochemicalJournal 26, 687-694.
Agard, D. A and Sedat, J. W. (1983) Three-dimensional architecture of a polythene nucleus. Nature 302, 676-81.
Sindelar, C.V and Grigorieff,N. (2011) An adaptation of the Wiener filter suitable for analyzing images of isolated single particles, J. Struct. Biol. 176, 60-74.
Sigworth, et al., (2010) An Introduction to Maximum-Likelihood Methods in Cryo-EM, Methods in Enzymology, Volume 482, 263-294
top related