3 image enhancement in the spatial domain.ppt [相容模 …140.115.154.40/vclab/teacher/2012dip/3...
TRANSCRIPT
Image Enhancement in the Image Enhancement in the Spatial DomainSpatial Domain
Angela Chih Wei Tang (唐 之 瑋)Angela Chih-Wei Tang (唐 之 瑋)Department of Communication Engineering
National Central UniversityNational Central UniversityJhongLi, Taiwan
2012 Spring
Outline
Gray level transformations Histogram processings og a p ocess g Enhancement using arithmetic operations
S ti l filt i Spatial filtering
C.E., NCU, Taiwan Angela Chih-Wei Tang, 2012 2
Background
)],([),( yxfTyxg Spatial domain methods
operate directly on the pixelsT t i hb h d f ( ) T operates over some neighborhood of (x,y) neighborhood shape: square & rectangular arrays are
the most predominant due to the ease of implementation mask processing/filtering masks/filters/kernels/templates/windowsmasks/filters/kernels/templates/windows e.g., image sharpening
the center of the window f i l t i lmoves from pixel to pixel
the simplest form: gray-level transformationg y s=T(r)
T can operate on a set of input images e g sum of input images for e.g., sum of input images for
noise reduction3C.E., NCU, Taiwan Angela Chih-Wei Tang, 2012
Point Processing Examples
contrast stretching: produce an image of higher contrast contrast stretching: produce an image of higher contrast than the original darken the levels below m & brighten the levels above m
the limiting case of contrast stretching: thresholding function
Contrast Stretching Thresholding Function
C.E., NCU, Taiwan Angela Chih-Wei Tang, 2012 4
Gray Level Transformations –I N tiImage Negatives
Re e se the intensit le els rLs 1 Reverse the intensity levels Enhance white/gray detail embedded in dark regions of an
image, especially when the black areas are dominant in size
rLs 1
g , p y
Original image Negative image6C.E., NCU, Taiwan Angela Chih-Wei Tang, 2012
Gray Level Transformations –L T f tiLog Transformations
s=clog(1+r), c: constant, r≥0 Map a narrow range of low gray-level values in the input
image into a wider range of output levels Compress the higher-level values Compress the higher level values An example: Fourier spectrum of wide range ( )
a significant degree of detail would lost in the display
610~0
Display in a 8 b8-bit display
Fourier spectrum( )
After log transformation(c=1 s:0~6.2)6105.1~0 7C.E., NCU, Taiwan 7
Gray Level Transformations –P L T f tiPower-Law Transformations
c & : positive crs , c & : positive constants fractional : map a
narrow range of dark
crs
narrow range of dark input values into a wider range of output valuesvalues
a family of possible transformation curves: varying varying
A variety of devices used for image capture,
i ti & di l printing & display respond according to a power law
d i d d device-dependent value of gamma
8C.E., NCU, Taiwan Angela Chih-Wei Tang, 2012
Example #1 of Power-Law Transformations Gamma CorrectionTransformations – Gamma Correction Correct the power-law response phenomena
an example: The intensity-to-voltage response of CRT an example: The intensity to voltage response of CRT devices is a power function (exponents : 1.8~2.5) darker than intended
5.2
4.05.2/1
9C.E., NCU, Taiwan
Example #2 of Power-Law Transformations Contrast ManipulationTransformations – Contrast Manipulation
Original image c=1, =0.6
c=1, =0.4 c=1, =0.3
Magnetic resonance (MR) image of a fractured human spine10
Example #3 of Power-Law Transformations Contrast ManipulationTransformations – Contrast Manipulation
Original image c=1, =3.0
c=1, =4.0 c=1, =5.0
A Washed-out appearance: a compression of gray levels is desirable11
Piecewise-Linear Transformation F nctionsFunctions
Advantage: arbitrarily Advantage: arbitrarily complex
Disadvantage: the specification requires specification requires considerably more user inputs
Example: contrast-stretching transformation increase the dynamic increase the dynamic
range of the gray levels of the input image
r1=r2=m (Thresholding function)
(r1,s1)=(rmin,0)(r2,s2)= (rmax,L-1) 12
Gray-Level Slicing
Highlight a specific range f l l i ft of gray levels is often
desired Various ways to y
accomplish this highlight some range
and reduce all others to and reduce all others to a constant level
highlight some range but preserve all other levels
O i i l i T f d Original image Transformed image by (a)
13C.E., NCU, Taiwan
Bit-plane Slicingp g
Instead of highlighting More
significantMore
significant Instead of highlighting
gray-level ranges, highlighting the contribution made to
gg
contribution made to total image appearance by specific bits
Higher-order bits: the majority of the visually significant datag
Lower-order bits: subtle detailsObt i bit l i Obtain bit-plane images thresholding gray-level
transformation function
14C.E., NCU, Taiwan Angela Chih-Wei Tang, 2012
OutlineOutline
Gray level transformationsHi t i Histogram processing
Enhancement using arithmetic operations Spatial filtering
C.E., NCU, Taiwan Angela Chih-Wei Tang, 2012 15
Histogram Processing
Hi t A l t H )( Histogram: A plot vs. : kth gray level : number of pixels in the
kr kk nrH )(krkn p
image having gray level k=0,1,..,L-1
Normalized histogram
kkr
nnrp kk /)( Normalized histogram Purposes
image enhancementimage compression
nnrp kk /)(
image compression segmentation
Simple to calculate economic hardware implementation proper for real-time image
processing
C.E., NCU, Taiwan Angela Chih-Wei Tang, 2012 16
Histogram Equalization/Histogram Linearization (I)
Having gray-level values automatically cover the entire gray scale controlling the probability density function (pdf) of its controlling the probability density function (pdf) of its
gray levels via the transformation function T(r), where
k k
jjrkk Lk
nrprTs 1,...,2,1,0,)()(
Continuous transformation S=T(r): assume T(r) satisfies (a.1) singled-valued: guarantee that the inverse
j j
j n0 0
transformation will exist (a.2) monotonically increasing: preserves the increasing
order from black to white in the output imagep g not monotonically increasing: at least a section of the
intensity range being inverted (b) : the output gray levels will be in the 10for10 rT (b) : the output gray levels will be in the
same range as the input levels10for 10 rTr
17C.E., NCU, Taiwan Angela Chih-Wei Tang, 2012
Histogram Equalization (II)g q ( )
According to an elementary probability theory, if and T(r) are known and satisfies condition (a), then the
b b l d f f h f d bl)(
)(rpr)(1 sT
probability density function of the transformed variable s can be obtained by
dr
)(sps
dsdrrpsp rs )()(
The probability density function of the transformed variable s is determined by the gray-level pdf of the input image and by the chosen transformation functionp g y
C.E., NCU, Taiwan Angela Chih-Wei Tang, 2012 18
Histogram Equalization (III)g q ( ) Consider the transformation function
r
r dwwprTs0
)()(
)( drdTds r
)(])([)()(
0rpdwwp
drd
rdrdT
drds
rr
r
1s01]1[1)()(
rpsp 1s0,1]1[)(
)()()(
)(1
1
sTrsTrr
rs rprpsp
1. Performing the transformation gyields a random variable scharacterized by a uniform probability density function.probability density function.
2. T(r) depends on , but the resulting always is uniform, independent of
)(rpr)(sps
)(rp
C.E., NCU, Taiwan Angela Chih-Wei Tang, 2012 19
independent of .)(rpr
Results of Histogram
Equalization (1/2)Equalization (1/2)
Transformation functions
20C.E., NCU, Taiwan Angela Chih-Wei Tang, 2012
Histogram Matching (Hi t S ifi ti ) (1/2)(Histogram Specification) (1/2)
Motivation sometimes, it is useful to specify the shape of the
histogram of the processed image instead of uniform histogramhistogram
the original image & histogram specification are given
C ti l l & Histogram
l
r
r dwwprTs0
)()(z
)()( zTzG
Obtain T(r)Give input image r,
Continuous gray level r & z Equalization
sdttpzGz
z 0 )()(, define
)]([)( 11 rTGsGz
)()(Obtain G(z))(zpzGive
)]([)(
Discrete gray level
1210)(1 LksGz kk 1,...,2,1,0),( LksGz kk
22C.E., NCU, Taiwan Angela Chih-Wei Tang, 2012
Histogram Matching (2/2)
Inverse mapping from sto the corresponding z
C.E., NCU, Taiwan Angela Chih-Wei Tang, 2012 23
Local Enhancement Motivation
to enhance details over small areas in an imageth t ti f l b l t f ti d t the computation of a global transformation does not guarantee the desired local enhancement
Solution define a square/rectangular neighborhood move the center of this area from pixel to pixel histogram equalization in each neighborhood region
OriginalImage
After GlobalEnhancement
After LocalEnhancement 24
Use of Histogram Statistics for Image g gEnhancement
P id i l hil 1L
nth moment of r
Provide simple while powerful image enhancement
1
0)()()(
L
ii
nin rpmrr
Histogram statistics global mean
f
1
0)(
L
iii rprm
a measure of average gray level in an image
0i
1
0
22 )()()(
L
iii rpmrr
global variance a measure of average
contrast
xy
xySts
tstss rprrm),(
,, )()(
contrast local mean local variance
xy
xyxySts
tsStsS rpmr),(
,2
,2 )(][
C.E., NCU, Taiwan Angela Chih-Wei Tang, 2012 25
OutlineOutline
Gray level transformationsHi t i Histogram processing
Enhancement using arithmetic operations Spatial filtering
C.E., NCU, Taiwan Angela Chih-Wei Tang, 2012 26
Enhancement Using Arithmetic O ti I S bt tiOperations – Image Subtraction
Original imageOriginal image
Zero the 4 lower-order
bit plane
Zero the 4 lower-order
bit plane
The enhancement
Original imageOriginal image bit planebit plane
),(),(),( yxhyxfyxg
of the differences between images to bring out more to bring out more
details, contrast stretching can be performedperformed histogram
equalization power-law
transformation
27C.E., NCU, Taiwan
Image Subtraction –Mask Mode RadiographMask Mode Radiography
The mask is an X-ray image of a region of a patient’s body The image to be subtracted is taken after injection of a The image to be subtracted is taken after injection of a
contrast medium into the bloodstream The bright arterial paths carrying the medium are
k bl h dunmistakably enhanced The overall background is much darken than the mask image
Mask imageMask image After inject of a contrast medium into the bloodstreamAfter inject of a contrast medium into the bloodstream28C.E., NCU, Taiwan 28Angela Chih-Wei Tang, 2012
The Range of the Image Values After Image Subtraction
F 8 bi i 255 255 For 8-bit image: -255~255 Two principle ways to scale a different image
add 255 to every pixel & divide by 2 add 255 to every pixel & divide by 2 limitation: the full range of the display may not be
utilized the value of the minimum difference is obtained & its
negative added to all the pixels in the difference image scale to the interval [0 255] scale to the interval [0, 255] more complex
C.E., NCU, Taiwan Angela Chih-Wei Tang, 2012 29
Image Averagingg g g
N i i )()()( f Noisy image assumption: at every pair of coordinates (x,y), the noise
is uncorrelated & has zero average value
),(),(),( yxyxfyxg
s u co e a ed & as e o a e age a ue uncorrelated: covariance
if an image is formed by averaging K different nois images
0)])([( jjii mxmxE),( yxg
noisy images
K
ii yxg
Kyxg
1),(1),(
),(),( yxfyxgE 2
),(2
),(1
yxyxg K
as K increases: the variability (noise) of the pixel values at each location (x y) decreases
),(),( yxyxg K
C.E., NCU, Taiwan Angela Chih-Wei Tang, 2012 30
at each location (x,y) decreases
An Example of I A iImage Averaging An important application of
image averaging: astronomy motivation: imaging with motivation: imaging with
very low light levels is routine, causing sensor noise frequently to render noise frequently to render single images virtually useless for analysis
solution: image averaging
31C.E., NCU, Taiwan
An Example of Image Averaging –Difference ImagesDifference Images
Th ti l l The vertical scale: number of pixels
The horizontal scale: gray l l [0 255]
]106.2,0[ 4
level [0, 255] The mean & standard
deviation of the difference images decrease as K increases the average image g g
should approach the original as K increases
The effect of a decreasing mean: the difference images become darker as Kincreases
32Angela Chih-Wei Tang, 2012
OutlineOutline
Gray level transformationsHi t i Histogram processing
Enhancement using arithmetic operations Spatial filtering
C.E., NCU, Taiwan Angela Chih-Wei Tang, 2012 33
Basics of Spatial Filtering (1/2)
The concept of filtering f th f th comes from the use of the
Fourier transform for signal processing in the frequency domainfrequency domain.
At each point (x,y), the response of the filter at th t i t i l l t d that point is calculated using a predefined relationship.
l f l h Linear spatial filtering: the response is the sum of products
3434
Basics of Spatial Filtering (2/2)p g ( )
Linear spatial filtering often is referred to as “convolving a mask with an image.” filtering mask / convolution mask / convolution kernel
Nonlinear spatial filtering Nonlinear spatial filtering the filtering operation is based conditionally on the
values of the pixels in the neighborhood under consideration, but not explicitly use coefficients in the sum-of-products manner
e.g., median filter for noise reduction: compute the e.g., median filter for noise reduction: compute the median gray-level value in the neighborhood
C.E., NCU, Taiwan Angela Chih-Wei Tang, 2012 35
What Happens When the Center of the Filter A h th B d f th I ?Approaches the Border of the Image ?
One or more rows/columns of the mask will be located One or more rows/columns of the mask will be located outside the image plane
Solution limit the excursions of the center of the mask to be at a
distance no less than (n-1)/2 pixels from the border the resulting filtered image is smaller than the original the resulting filtered image is smaller than the original
partial filter mask: filter all pixels only with the section of the mask that is fully contained in the image
Padding add rows & columns of 0’s (or other constant gray
level) or replicate rows/columnslevel), or replicate rows/columns the padding is stripped off at the end of the process side effect: an effect near the edges that becomes
more prevalent as the size of the mask increases36C.E., NCU, Taiwan Angela Chih-Wei Tang, 2012 36
Smoothing Linear Filters (1/2)g ( )
A i fil /l fil l h l f Averaging filters/low pass filters: replace the value of every pixel in an image by the average of the gray levels of the pixels contained in the neighborhood of the filter mask
Reduce sharp transitions in gray levels side effect: blur edgesR d i l t d t il i i d t Reduce irrelevant detail in an image and get a gross representation of objects of interest irrelevant: pixel regions that are small with respect to p g p
the size of the filter mask Applications
i d ti noise reduction smooth false contouring
C.E., NCU, Taiwan Angela Chih-Wei Tang, 2012 37
Smoothing Linear gFilters (2/2)
Computationally efficient: instead of being 1/9 (1/16), the coefficients of the filter are all 1’s
Box filter: a spatial averaging filter in which all coefficients are equalequal
Weighted average: give more importance (weight) to some pixels reduce blurring in the smoothing process
the pixel at the center of the mask is multiplied by a higher value than any otherthe other pixels are inversely weighted as a function of their the other pixels are inversely weighted as a function of their distance from the center of the mask
The attractive feature of “16”: integer power of 2 for computer implementation
38C.E., NCU, Taiwan Angela Chih-Wei Tang, 2012 38
Examples of Smoothing Li Filt (1/2)Linear Filters (1/2)
n=3: general slight blurring through the n=3: general slight blurring through the entire image, but details are of approximately the same size as the filter mask are affected considerably filter mask are affected considerably more
n=9: the significant further smoothing of the noisy rectangles
n=15 & 35: generally eliminate small objects from an imageobjects from an image
n=35: black order due to zero padding
39
Examples of Smoothing Linear Filters (2/2) Goal: get a gross representation of objects of interest, such
that the intensity of smaller objects blends with the background and larger objects become bloblike the size of the maskSmoothing linear filtering + thresholding to eliminate Smoothing linear filtering + thresholding to eliminate objects based on their intensity
40
Order-Statistics Filters
N li ti l filt h i b d Nonlinear spatial filters whose response is based on ordering (ranking) the pixels in the area to be filtered
Examples: max filter, min filter & median filter Median filter
replace the value of a pixel by the median of the gray levels in the neighborhood of that pixelg p
force points with distinct gray levels to be more like their neighbors
for 3x3 neighborhood for 3x3 neighborhood(10,20,20,20,15,20,20,25,100) -> median is 20
excellent noise-reduction capabilities, with considerably l bl i th li thi filt f i il iless blurring than linear smoothing filters of similar size
particular effective for impulse noise (salt-and-pepper noise
C.E., NCU, Taiwan Angela Chih-Wei Tang, 2012 41
An Example of Applying Median Filtersp pp y g
The image processed with the averaging filter has less The image processed with the averaging filter has less visible noise, but the price paid is significant blurring.
C.E., NCU, Taiwan Angela Chih-Wei Tang, 2012 42
Foundations of Sharpening Spatial Filters (1/2)
Highlight fine detail in an image or to enhance detail that has been blurred
Averaging is analogous to integration, and sharpening can be accomplished by spatial differentiation
For our use what we require for a first derivative are For our use, what we require for a first derivative are zero in flat areas nonzero at the onset of a gray-level step or ramp nonzero along ramps
For our use, what we require for a second derivative zero in flat areas zero in flat areas nonzero at the onset and end of a gray-level step or ramp zero along ramps of constant slope
C.E., NCU, Taiwan Angela Chih-Wei Tang, 2012 43
Foundations of Sharpening Spatial p g pFilters (2/2)
Th fi d d i i f 1 D f i f( ) The first-order derivative of a 1-D function f(x)
)()1( xfxfxf
The second-order derivative of f(x)x
)(2)1()1(2
ffff
The two definitions satisfy the conditions stated previously
)(2)1()1(2 xfxfxfxf
C.E., NCU, Taiwan Angela Chih-Wei Tang, 2012 44
An Example of Applying Sharpening S ti l Filt (2/3)Spatial Filters (2/3)
Ramp: the first-order derivative is nonzero along the entire ramp, while the second-order derivative is nonzero only at the onset and end of the ramp. first-order derivatives produce thick edges and second- first order derivatives produce thick edges and second
order derivatives, much finer ones Isolated noise point: the response at and around the point is
h t f th d th f th fi t d much stronger for the second- than for the first-order derivative. second-order derivative enhances much more than a first-
order derivative does. Thin line: essentially the same difference between the two
derivativesderivatives Gray-level step: the response of the two derivatives is the
same double-edge effect: The second derivative has a transition
from positive back to negative. 46
Foundation of Sharpening Spatial Filters (3/3)
l h d d b In most applications, the second derivative is better suited than the first derivative for image enhancement.
The principle use of first derivatives in image The principle use of first derivatives in image processing: edge extraction.
C.E., NCU, Taiwan Angela Chih-Wei Tang, 2012 47
Second Derivatives for E h t Th L l iEnhancement – The Laplacian
Isotropic filters Isotropic filters rotation invariant: rotating the image and applying the
filter gives the same result as applying the filter to the f d h h limage first and then rotating the result
The simplest isotropic derivative operator: Laplacianff 22
2
Laplacian is a linear operator : because derivatives of any order yf
xff 22
2
are linear operator
),(2),1(),1(2
2
yxfyxfyxfxf
),(2)1,()1,(2
2
yxfyxfyxfxfx
),(4)]1,()1,(),1(),1([2 yxfyxfyxfyxfyxff 48
Filter Masks – The Laplacian (1/2)Isotropic for
increments of 45 deg.Isotropic for
increments of 90 deg.
4949
Filter Masks – The Laplacian (2/2)p ( )
Th L l i i d i i The Laplacian is a derivative operator highlights gray-level discontinuities in an image &
deemphasizes regions with slowly varying gray levelsdee p as es eg o s s o y a y g g ay e e s Being used frequently for sharpening digital images
The Lapalcian for image enhancement
)()(),(),(),( 2
2
ffyxfyxfyxg
If the center coefficient of theLaplacian mask is negativeIf the center coefficient of the ),(),( 2 yxfyxf If the center coefficient of the Lapalcian mask is positive
C.E., NCU, Taiwan Angela Chih-Wei Tang, 2012 50
Examples of the Laplacian Enhanced Image (1/2)Enhanced Image (1/2)
Notice that the Laplacian o a ap a aimage may contain both positive and negative valuesvalues scaling is desired
Adding the original image to the Laplacian image small details were small details were
enhanced the background
t lit i bl tonality is reasonably preserved
51C.E., NCU, Taiwan 51
Examples of the Laplacian Enhanced Image (2/2)Enhanced Image (2/2)
The results obtainable with the
k i i mask containing the diagonal terms usually are a little sharper than those obtained with the more basic mask
52C.E., NCU, Taiwan Angela Chih-Wei Tang, 2012
First Derivatives for Enhancement –The Gradient (1/2) First derivatives in image processing are implemented using
the magnitude of the gradient gradient is defined as the 2-D column vector gradient is defined as the 2-D column vector
xf
Gxf
the magnitude of this vector (gradient)
yfx
Gyf
the magnitude of this vector (gradient)2/122
2/122 ][)f(
yf
xfGGmagf yx
nonlinear operator isotropic
yx
C.E., NCU, Taiwan Angela Chih-Wei Tang, 2012 53
isotropic
First Derivatives for Enhancement –The Gradient (2/2) Problem: the computational burden of implementing the Problem: the computational burden of implementing the
gradient The approximation of the magnitude of the gradient
the isotropic properties of the digital gradient are preserved
yx GG f
the isotropic properties of the digital gradient are preserved only for a limited number of rotational increments (90 deg.)
Digital approximation of the gradient
5658 , zzGzzG yx
6859 zzGzzG 1
2 6859 , zzGzzG yx2
54C.E., NCU, Taiwan Angela Chih-Wei Tang, 2012
A 3x3 Region of an Image & Masks for Comp ting the Gradient
R b di
for Computing the Gradient Roberts cross-gradient
operators
6859 zzzzf (From )26859 zzzzf (From )2
Since even size are awkward toimplement 3x3 Sobel Operator
)2()2(
)2()2( 321987
zzzzzz
zzzzzzf
implement 3x3 Sobel Operator
)2()2( 741963 zzzzzz
derivative in x derivative in y55C.E., NCU, Taiwan Angela Chih-Wei Tang, 2012
An Example of Using Sobel Gradientp g An optical image of a contact lens illuminated by a lighting
t t hi hli ht i f tiarrangement to highlight imperfections Sobel gradient: the edge defects are quite visible, and the
slowly varying shades of gray are eliminated.y y g g y simplify the computational task required for automated
inspection.
C.E., NCU, Taiwan Angela Chih-Wei Tang, 2012 56