istanbul technical university faculty of aeronautics & astronautics department of meteorological...

ISTANBUL TECHNICAL UNIVERSITY FACULTY OF AERONAUTICS & ASTRONAUTICS

DEPARTMENT OF METEOROLOGICAL ENGINEERING

Digital Image ProcessingDigital Image Processing

SEZEL KARAYUSUFOĞLUSEZEL KARAYUSUFOĞLU110030226110030226

. There are several advantages connected with using digital images:- Digital softcopy workstations running on standard computers can be used rather than expensive plotting devices requiring special hardware, - Data transfer is simpler with regard to plotting results because they can be post-processed on the same computer, - Digital image processing techniques can be used for image enhancement,- Digital image processing techniques render possible the automation of photogrammetric measurement tasks (=> Research Goals). However, there are problems connected with image acquisition. For many applications the format of CCD-sensors which are commercially available is too small or the sensors are too expensive. In this case or if high precision is required and metric cameras have to be used, one will first get analogue photographs. Those images have to be scanned offline using photogrammetric scanners, which turns out to be a very time-consuming process. In order to achieve precisions comparable to those achieved in analogue photogrammetry, point localization has to be done with an accuracy better than 1 pixel (Subpixel Accuracy).

DIGITAL IMAGE PROCESSINGDIGITAL IMAGE PROCESSING

Digital Photogrammetry-Data AquisatationDigital Photogrammetry-Data Aquisatation

CAMERA

CCD

CAMERA

Photo Labor.

DIGITISER

DIGITAL

IMAGE

DIGITAL PHOTOGRAMETRY

ÜNIT

CAD SYSTEM

Digital Photogrammetry - Digital Photogrammetry - SCANNINGSCANNING

Scanning of a one-dimensional function

The original image function f(x) is a continuous one regarding location and intensity.

1.Scanning: Discretization of the location by defining picture elements (pixel)

2. Quantization:Discretization of the intensity; the intensity range IG is part of the natural numbers including zero (e.g. IG = 0, 1, 2, … 255 = 8 bit quantization)

The result of this procedures is a digitized (“digital”) image.

The sampling procedure can be interpreted as convolution of the image function f(x,y) with the response function r(x,y) of the sensor area Δx Δy.

1 |x| ≤ Δx/2 |y| ≤ Δy/2r(x,y) = 0 |x| > Δx/2 |y| > Δy/2

Two-dimensional convolution S(x,y) = f(x,y) * r(x,y) = f(m,n) g(x-m, y-n)dm dn

Discrete two-dimensional convolution

S(x,y) = The sampling theorem states, that the spatial frequencies fx and fy, which can be

recovered from the data, are a function of the sampling intervals Δx and Δy.

1

0

1

1

),(),(M

m

N

n

nymxgnmf

Digital Digital SCANNINGSCANNING

IMAGE CORRELATIONIMAGE CORRELATION

The aim of image correlation is to determine identical (homoloque) points in digital image data. For this purpose a reference matrix A is defined in the first image and a (larger) search matrix B in the second one. Then the correlation coefficients r are calculated for each possible position (n,m) of the reference matrix within the search matrix.

A Referencematrix B Search matrix

The maximum of r defines the most probable position of A within B.

n

i

n

iii

n

i

n

iii

n

i

n

ii

n

iiii

yn

yxn

x

yxn

yx

r

1

2

1

2

1

2

1

2

1 11

11

1

Least-squares method of image correlation

1. Definition of two submatrices g1, g2

2. Approximate values Image coordinate differences ∆x, ∆y (<2 pixel)

Parameters of the affine transformation r0 = 0

r1 = 1

Number of iterations k

3. For each pixel

yaxaax 210 yaxaax 210

Radiometric adjustment 2102 grrg


5. Determination of intensity gradients out of g2y

g

x

g

,

6. Establishing the error equations

1012210210 grrgyby

gxb

y

gb

y

gya

x

gxa

x

ga

x

gv i

7. Esstablishment and solution of normal equations N= ATA Solution yields the unknowns

10

210

210

,

,,

,,

rr

bbb

aaa

8. Correction of approximate values

00

10

001

0

bbb

aaakk

kk

11

11

111

1

bbb

aaakk

kk

22

12

221

2

bbb

aaakk

kk

11

11

001

0

rrr

rrrkk

kk


10. Standard error of unit weight

80 n

vvm

Number of pixels n

Standard error of unknowns

xxx Qmm 0


Image ProcessingImage Processing

Image processing topics are related to applications directly performed by human vision. The main subjects of image processing are image restoration and image enhancement. Image restoration topics contain the techniques, which are used to repair a deformed image by using known or predicted statistical or geometrical degradation models.

Image restoration is used in the applications, which require quality images such as in photogrammetry, in publication, in photography etc. Repair of the distorted images and filtering of the noises are example of restoration applications.

Image enhancement topics are usually used to have better appearance of images to human vision. For example, contrast stretching is an enhancement application. There is no general enhancement techniques, which are used to enhance every kind of images, without any modification. For example, if an enhancement technique, which is applied to a satellite image, is applied to a medical image, the desired result may not be obtained

SAMPLING of DIGITAL IMAGESSAMPLING of DIGITAL IMAGES

Conversion process of video signals to digital images is called digitization. To process an image by a computer, it should be converted to digital form. Because video signals are continuous or analog functions. A typical video signal may be considered as having windows or frames which contain video information. Every window or frame comprises fully one screen visual information. Every frame is subdivided to fields and fields contain line information. In fig-1, one frame video information has shown.

In fig-1, solid lines show fields and dashed lines correspond line information.

Figure-1: Frames and lines

one line information

one pixel

one line

time

voltage

In fig-2 a video signal corresponds to a line video signal has shown. In fig-2 if it is noticed that, at the beginning and end of the signal, impulses are seen. These indicate the imaging system hardware, a new line data is sent. Analog video signals are converted to digital form by sampling with equal time intervals.



In fig-3 an analog signal and its sampling have been shown. As seen in fig-3, a line video signal is sampled in equal time intervals and corresponding voltage values are recorded as integers. These integers correspond to pixel brightness at t time or t point. After a frame had been digitized, this frame can be stored and processed as digital image on computers. Digitized image may be considered as two dimensional array (matrix). Every element of the array corresponds to one pixel in the image. In general f(x,y) notation is used for digital images. Here f(x,y) is the brightness value of (x,y) point and x, y are row and column number of image respectively

one line information

impulse signals

Figure-3: Sampling of analog signal (digitization)


High level

Low level

pixel

neighboring, segmentation

spectral segmentation,edge/line

featureobject

raw pixel data

preprocessing

transformations, imagesegmentation, edge detection

feature extraction,pattern recognition

Figure-4: Image Operations

Figure-4: Image Operations

After an image had been digitized, image data can be processed with a software. Processing operations can be shown hierarchically in figure-4

In the lowest level of the pyramid, raw image data is seen. At this level, some preprocessing operations can be performed directly to pixels. At the second level, by using pixel neighborhood, preprocessing operations may be done. And at the higher levels, more complex operations are performed. For example edge extraction and object recognition etc.

RESOLUTION of IMAGESRESOLUTION of IMAGES

Resolution may be described as measure of the power that how two neighbor pixels are separated. If two adjacent pixels are seen as one pixel, they cannot be distinguished from each other. In other words, if two adjacent pixels belong to two different objects, and if they seem as one pixel, then these two different objects will be seem as one. And this result causes mistakes in some applications. One can say that, the higher resolution, more much object details on an image.

Spatial Frequency Resolution

In fig-5, to express these spatial frequencies, a square wave has been shown. Frequency shows that how often the signal changes in the space. The signal in the figure has two values, zero and maximum.

0

max

space

brightness

Figure-5: a.) Square wave used to form 5.b. image, b.) Low frequency image (f=2).


If the square signal in fig-5 is considered to be used to represent one line of an image, and if it repeats as same down to whole image rows, then an image is obtained which is formed by vertical strips as in fig-5.b. If the frequency of the wave is increased, vertical strips get close to each other and they also get narrow. In this case, whole image will have more details. But in this case, if someone looks at the image from a far distance, that eye very hard may resolve time vertical strips and in a limit distance they will not be distinguished. If so, one can say that, spatial frequency is also related to the distance of the location of an observer from the image. To define this relation (relation between spatial frequency and observation distance), a sight angle and vertical period is needed. Here the period is exact change of a signal. For example a period for a square wave corresponds to one minimum and one maximum point. As seen, to define a period, at least two pixels are needed. One for maximum, and one for minimum point. When more much periods are used, degrees correspond to sight angle i.e., when some get away image, to provide same resolution, proportional to distance, wider periods should be used. With this thought, a metric measure of resolution is obtained for the human vision system.

REPRESENTATION of IMAGES on COMPUTERSREPRESENTATION of IMAGES on COMPUTERS

Images are considered as 2-D functions and shown with f(x,y) notation. Brightness of a pixel at (x,y) location corresponds to f(x,y) data. In linear algebra 2-D arrays contain f(x,y) data and 2-D image data f(x,y) is expressed in matrix form. One row or column of a matrix is called vector. The images may be binary, gray-level or multi color. 4.1. Binary ImagesBinary images are the simplest form. In these images, pixels may only have two different values, for example 0=black, 1=white. Binary images are often used to represent printed documents or to represent the basic shapes of interested objects. To transmit printed documents by FAX, generally binary images are used. This images may also be used for optical character recognition. To have a binary image, commonly thresholding techniques are used. For example if a gray-level image is thresholded at a chosen level, the values equal or exceeds the threshold are assigned 1 and the lower values are assigned zero. Thus the final thresholded image is binary.


Gray-Level ImagesGray level images bear tone information. Bit number used to represent a pixel

corresponds to a different brightness value. For example 8 bop data corresponds 256 different gray level values between 0-256.

Color ImagesColor images may be considered as a collection of three different gray-level bands. In the

color images, real information belongs to three separate spectral bands are stored. The most common type of color images is RGB images. If each color band requires 8-bit, that time an RGB image is formed with 24 bits. In fig-6, typical RGB image structure has shown.

fR (x,y) fB (x,y)fG (x,y)Blue

RedGreen

RGB color vector

6.a. Typical RGB images may be considered as 3 different images fR(x,y), fG(x,y) fB(x,y)

6.a. Typical RGB images may be considered as 3 different images fR(x,y), fG(x,y)

fB(x,y)


In many applications, RGB color information is transformed to a mathematical space and it is converted to brightness value to be used. After this operation the image contains 1-D brightness information or 2-D color space. 2-D color spaces don’t contain brightness values. In stead of this it contains typical color information related to different color amounts. Transformation of color images in to mathematical spaces mentioned above, provides to be described the colors easily by humans

Another color transformation based on geometrical coordinates is spherical transform. With this transformation brightness values are obtained from color information. Spherical coordinate transform can be used in color segmentation algorithms. The relations between RGB components and spherical coordinates may be expressed by the following equations;

Here L is length of RGB vector. A is the angle between B axis and RG plane and B is the angle between R and G axis. Here L bears brightness information while A and B angles bear color information. In fig-7, HSL space has shown.

)L.sin(L

RcosL

L

BcosL

BGRL

A

1B

1A

222


Black

White

Green

Red Blue

saturation

HueFigure-7: HSL color space

RGB color cubeR

G

BL

S

H

Nearby in fig-8, RGB to HSL transformation has shown.


One problem related to the color spaces described above is that they are not perceptually uniform. This means that a color described in a space may be percepted as a different color in the other space even if they are at the same location. So, there is no metric criterion to measure the resemblance of the colors in terms of human perception. In computer vision to recognize objects, it’s necessary to compare the colors. And to do this we should find a method for this comparison. For this purpose in computer vision systems uniform color spaces may be used.

Color and human vision perception subjects have been studied extensively in many scientific researches. Especially, Commission Internationale de L’Eclariage (CIE) has defined some color standards on this subject. One of these standards is chromatic coordinates. Chromatic coordinates for RGB color space is defined as;

BGR

Gg

BGR

Bb

BGR

Rr

IMAGE ANALYSIS

Image analysis is used in computer imaging, to find how to perform processing operations in an image. These analysis techniques constitute a part of wide range image processing topics. Usually, image analysis is performed with iterative steps, and these are used to find the answers of some special questions. For example, the questions; Is color information needed?, Is it necessary to transform image to frequency domain?, Is segmentation required for extraction of information about objects in the image?, What are the important details?, are answered with image analysis.Image analysis process may be subdivided into three groups as in fig-9; preprocessing, data reduction and feature analysis. Preprocessing operations are used to filter noises, and to eliminate unnecessary or unwanted details or data. Furthermore, preprocessing contains gray-level or spatial quantisation processes. Such as reducing number of bits per pixel or the image size. Second step is data reduction. In this step, the data is reduced either in spatial domain or transformed to frequency domain, and next feature analysis step is performed

In computer vision, final production is commonly high level analysis or processing information. For example, the geometrical or other spatial information to move a robot arm, is higher level information obtained after higher level analysis. In medical imaging, the information describes a tumor is also an example of high level information, and these information is obtained with image analysis.In image processing applications, image analysis methods are also too useful to decide that which processing techniques should be used to reach aimed purpose. For example, in image restoration, finding the degrading model to repair image is defined by image analysis.

inputimage

preprocessing

datareduction

featureanalysis

IMAGE ANALYSIS

Figure- 9: Image Analysis Operations.

IMAGE ANALYSIS

The mathematical models for image analysis are constructed either in pixel geometry (spatial domain) or in frequency domain (or spectral domain), (fig-10).

inputºimage

preprocessing

FrequencyDomain

FeatureAnalysis

PixelDomain

Figure-10: Image analysis domains

SIMPLE ALGEBRAIC OPERATIONS in IMAGES

Algebraic operations used for images are commonly viewed in two groups; mathematical and logical operations. Image adding, subtracting, dividing and multiplying operations constitute mathematical processings and “AND, OR, NOT” etc. operations forms logical operations.a.) Image Addition To perform mathematical operations on two images, both in two images corresponding pixels are considered. If we assume that f3 is result image of addition of f1 and f2 images, f3 is obtained by adding corresponding pixel values of f1 and f2. This may be shown as f3(x,y) = f1(x,y) + f2(x,y). In fig-11, a and b images have been added and final image 11-c has obtained.

Figure-11:

Image addition


b.) Image SubtractionImage subtraction operation is commonly used

in motion images to find image differences, (fig-12).

Fig-12 image has obtained by subtracting 11-a from 11-b.

Figure-12: Subtracted image

c.) Image Multiplication and DivisionOriginal image is either multiplied or divided by a constant number. After multiplication, lighter image is obtained and darker image after division, (fig-13).


Figure-13: a.) Original image, b.) Original image*2 c.) Original image /2

d.)Logical OperationsLogical operations are bitwise operations AND, OR and NOT. With their different combinations XOR, NOR and NAND operations may also be performed easily.AND and OR operations are used to combine information of two images. They have special effects but they’re too useful in image processing. In fig-14 some logical operations is shown.


Figure-14: Logical image operations.

In fig-14-a, a 128 x 128 image area formed by wholly white pixels is used as mask, then this mask is used to perform AND operation with the image in 13-a, and the final AND image has obtained in, 14-b. In 14-c, not image of 13-a has shown.

EDGE / LINE DETECTION

For detection of edge and lines in images, various detection operators are used. Most of these are applied with convolution masks and most of these are based on differential operations. Differentials for edge/line detection are used to define color or brightness changes of pixels and their change directions. If there is an abrupt change of brightness in a short interval in an image, it means that in that interval there is an edge with great probability. Some edge/line detection operators also contain dimension information in spite of brightness.

Edge/line detection operators are used as primary step to define objects. Potential edges of complex objects are fixed with edge operators as first step. During this process, direction of each possible edges are defined too. In the second step, using both brightness and dimension information joins these potential edges. Thus the final edges of objects are obtained.

Many of the edge operators are negatively affected from noises in noisy images. So, before edge detection noises should be filtered as possible. Instead of this, frequency domain filters may also be used for edge detection. But here spatial domain methods are being considered.


The size of operator mask and operator coefficients should be chosen carefully not to be effected of noises. Otherwise, noise points may be accepted as an edge point. If the mask size is chosen greater, that time operator is more effected from noises.

The basic idea of edge detection operators is based on comparison of the brightness values of pixels with their neighbors. For example, if a pixel’s value is the same as its neighbors or too close, then with a great probability this pixel will not lie on an edge. If there is difference between pixel and its neighbors then it may be thought that this pixel may lie on an edge. An ideal edge appears with an abrupt gray-level change. But in practice, instead of such abrupt change, smooth changes occur. In fig-15, an ideal and corresponding practical situation of gray-level changes of an edge has shown.

Brightness Brightness

Spatial coordinates Spatial coordinates

a. Ideal edge b.) Real edge

Figure-15: Ideal and real (practical edge)


a.)Sobel OperatorsSobel edge detection masks detect vertical and horizontal edges separately. And these directional edges are combined finally. Coefficients of 3 x 3 sobel masks are;

horizontal mask vertical mask

-1

0

1

-2

0

2

-1

0

1

-1

-2

-1

0

0

0

1

2

1

These two masks are convoluted with image separately. The magnitude and direction of an edge is calculated by using convolution results of two masks. These are;

In fig-16, effect of sobel operators has shown.

S = tan12 -1S S S2

21 2 /

S=

16.a.Effect of horizontal Sobel mask

16.b. Effect of vertical Sobel mask

16.c. Sobel edge magnitude

In fig-16, the results of sobel masks applied to original image in fig-13 has shown. 16-a is the result of horizontal sobel mask, 16-b is vertical sobel mask effect and 16-c is magnitude of sobel masks.



Prewitt Operators

Prewitt operators are similar to sobel, but the mask coefficients are different.

horizontal mask vertical mask-101

-101

-101

111

000

111

Magnitude and direction of an edge is found by;

P = tan12 -1P P P2

21 2 /

S=.

17.a. Effect of horizontal Prewitt mask

17.b. Effect of

vertical Prewitt mask

Laplace OperatorsLaplace operators are used as the ones described above. Laplace operators are symmetric. I.e all changes in all directions effect the result. Sign of the result defines the direction of gray-level change. Below laplacian masks has shown.


0 -1 0 1 -2 1

-1 5 -1 -2 5 -2

0 -1 0 1 -2 1

In fig-20, original image in 20-a has been projected on to l1 and l2 laplacian masks. Magnitude of edges has shown in 20-d.

Figure-20: a.)Original image, b.) Effect of l1 mask

c.) Effect of l2 mask,

d.) Edge magnitude


Hough Transformation

Hough Transformation is used to define known geometrical shapes such as line, circle, ellipse etc. With the edge operators, only the pixels, which are, potentially edge pixels are found. However a line is formed with the neighbor (connected) pixels arranged at the same direction. In other words, lines should express the potential edge pixels lie on the same direction. To do this, Hough transformation may be used.


IMAGE FILTERING With SPATIAL MASKS

Spatial filters are used effectively to filter noises. These filters may be effective from 3 x 3 neighbourhood to 11 x 11. Spatial filters are subdivided in two groups; ordered filters and average filters. In ordered filters, gray-level values in the chosen neighbourhood are ordered ascending manner. From this order, the pixel which has the minimum probability to be noise is chosen and center pixel is replaced with this chosen value. In average filters, according to a defined average criterion, the average value of the neighbouring pixels are calculated and this value is put to the center pixel location. Ordered filters are usually used to filter salt and pepper noises, negative exponential noises and Rayleigh distributed noises. Average filters are used to remove Gauss distributed noises or uniform distributed noises.

The most effective ordered filter is median filter. With this filter, gray-level values in a chosen neighbourhood are sorted in the ascending manner. Center pixel of the mask is replaced with the median value of the ordered gray-levels. With sorting operation, noise valued pixels will be either on the head of the array or in the end of the array with a great probability. And the median value will not be a noise point with a great probability. If there is a noise point in the center of the mask and if it’s replaced with median value, then the noise could be filtered out. In fig-23 the effect of median filter has shown.

a. noisy image

b. median filtered image with 3*3 mask

Fig 23-a image is obtained by adding 5000 salt and pepper noise to 16-a image. And this noise image is filtered with a 3 x 3 median filter mask. Result image has shown in 23-b.Some other types of ordered filters are also available. For example in some ordered filters, the average value of max. and min. values is replaced with center pixels.


In average filters, the center pixel is replaced with an average value of neighboring pixel. In fig-24, an arithmetic average filter has applied to original image, and the result has shown in 24-b.

a. Original image b . Effect of average (mean) filterFigure-24: Average filter.


HISTOGRAM ANALYSIS TECHNIQUES

Global Brightness Changes

In order to change brightness of an image, image histogram should be changed. To do this, either the number of gray levels of an image or the gray level values should be changed. The simplest brightness (contrast) changing method is to slide the original image histogram to right or left. This operation is called in some papers as histogram slicing. To slide the histogram to the right, a constant should be added to all gray levels, and to slide it to left the inverse operation is needed. Image multiplation operation previously explained in the previous chapter may also be used for histogram slicing.

Another contrast changing method is histogram stretching and compressing. In this method either the gray level values are separated from each other when they’re

assembled together, or more separated gray level values are combined

In fig-25, an original aerial photo and its histogram has shown. In fig-26, this original image has remapped by bunching gray levels with 10 intervals. Final image in fig-26 has 25 gray levels, while original has 256

Figure-25: Original aerial photo image with 256 gray-levels and its histogram.


Figure-26: Bunched (histogram compressed) image with bunchsize=10 and its histogram



Thresholding

This technique is used to remove the gray-level trends in an image, to make the gray-level regions more discrete. This technique is generally used to transform the original image to a binary form. For example, a threshold value say “k” is chosen. The gray-levels greater than k is set to maximum level, and levels less than k are set to minimum level of image. This technique is generally used for processing scanned text images. The same technique may be used with more threshold values for the same image also.

Original image in fig- 25 is thresholded at the gray-level 155. The levels greater than or equal 155, are set to value 255 which is the maximum of range, and the levels less than 155 are set to minimum range value zero. The result thresholded image has shown in Figure 27.

Figure-27: Thresholded binary image, ( threshold value T= 155.)



Histogram EqualizationIn the original image histogram in fig-25, the gray-levels of the image are too close to each other. And this image’s density function is not uniform or near uniform. This means that the probabilities of the gray-levels are not equal. An ideal image’s density function should be uniform, i.e. the probabilities of the gray-levels should be equal to each other. If we transform the original histogram to a uniform histogram, new image will be close to ideal. Such a transformation is called histogram equalization. If the original image histogram is transformed to another histogram by using its own distribution function as transformation function, the result image histogram is equalized, (fig-28),

Figure-28: Histogram equalized image transformed by its original distribution function

istanbul technical university faculty of aeronautics & astronautics department of meteorological...

Documents