Image Processing: Assignment 1Basics of convolution and Fourier

transform

Date: 18.02.2011

Name: Laura Stoilescu

Address: Calslaan 28-5, Zip 7522MC, Enschede

Email: l.i.stoilescu@student.utwente.nl

Student Number: S1070053

Educational Program: Electrical Engineering- Telecommunication Networks

Given the distorted image below, find the filter for removing the distortion.

Figure 1: Initial, distorted image

1. Create an m-file that would: Read and display the given image;

im =im2double(imread('C:\Users\Laura\Desktop\dbb_exercises\car_dis.png'));figure;imshow(im, 'Border', 'tight');

Calculate the log-amplitude spectrum using the Fourier transform in a controlled manner.IM=fft2(im);spec=fftshift(log(c+abs(IM)));

figure;imshow(spec,[], 'Border', 'tight');title('Amplitude spectrum');

2. Explain the obtained results

The frequency spectrum of an image can be generated using a Fourier transform. The resulting values are usually presented as amplitude and phase, both plotted versus frequency but the phase spectrum lacks structure thus being less relevant. Since the image is not considered a continuous one (256x256 pixels), a discrete transform will be applied. The picture is considered to be a two-dimensional matrix so the function to be applied will not be fft but fft2 since fft2 returns the two-dimensional Discrete Fourier transform of the image. Because we are referring to the amplitude spectrum, we specify the absolute value of the spectrum (abs).

Still, by imposing these restrictions only to the logarithmic representation, the image obtained doesn’t look as we might expect. Since the wave fronts in Figure 1 are thin and dense we would guess the presence of a harmonic function on a relatively large frequency with (almost) vertical orientation. This would translate in the spectrum as two well defined clusters on the vertical axis containing the origin, situated at a relatively large distance from each other. Instead we get as predominant components two points on each lateral side of the image.

Figure 2: Log- amplitude spectrum without components shifting Figure 3: Log- amplitude spectrum of initial image

This image (depicted in Error: Reference source not found) is the first quadrant of the amplitude- frequency plane and, since the sampling determines spectrum repetition on all the multiples of the sampling frequency (the aliasing effect), it contains all the wanted components only shifted around the lateral edges. To solve this issue we can apply fftshift which compensates for this effect of fft and fft2.

The final image is depicted in Figure 3 with an increased contrast by adding a constant c and, thus, increasing the range of possible values to be taken by each pixel.

3. Extend the code with an average rectangular point spread function selecting the optimal vertical and horizontal limits. Show and interpret the results

hsize=[7 1];psf1=fspecial('average',hsize);imconv=imfilter(im,psf1);repli = imfilter(im,psf1,'replicate');

figure;imshow (psf1,[], 'Border', 'tight');title ('PSF');

figure; subplot(1,2,1);imshow(imconv,[], 'Border', 'tight'); title('Convoluted image');subplot (1,2,2);imshow(repli,[], 'Border', 'tight'); title('Replicate');

At a close look, the distortion pattern is almost1 parallel to the horizontal direction. To compensate for it, we should apply the average filter on the vertical axis after finding the signal’s periodicity. A rough approximation leads to a period of 7 pixels on the vertical so the dimensions of the point spread function (PSF) are seven rows and one column ([7 1]).

1 This is an approximation. Actually the propagation direction of the harmonic function representation in the spatial domains corresponds to the line connecting the energy points in the frequency spectrum but both are different from the vertical axis.

The low pass filter actually replaces each pixel with the weighted sum of its vertical neighbors. The weights are equal to 1/N where N is the number of neighboring pixels to average and the current pixel is also included in the sum.

Figure 4: Finding the distortion period; the brightest line is used as a marker of a new sequence on the vertical axis

The PSF is a combination between Cumulative Moving Average- CMA (or running average) and Weighted Moving Average-WMA. The CMA influence is found in the fact that an already averaged pixel will take part in the average for the next 6 pixels down its vertical axis. On the other hand, the next pixels will not be influenced more by the first one, each of them having equal weights.

Since the pattern is not absolutely horizontal, the filter will not solve the problem completely but the results are satisfying and considering the reduced dimensions of the PSF matrix, the convolution method is appropriate.

There is still the edges problem. The convoluted image is not correctly defined for the extreme sides because the external pixels are considered to be null, therefore the edges are fading to black. To compensate for this issue, I picked the replicate option of imshow function because, visually at least, had the best results.

Figure 5: The result of convolving the original image with the average filter

4. Find the correct optical transfer function (OTF) of the given PSF

otf1=psf2otf(psf1,size(im));

OTF1=log(c+abs(fftshift(otf1))); figure;imshow(OTF1,[], 'Border', 'tight');title ('OTF');

Figure 6: Optical Transfer Function

Any image can be represented in the frequency domain as an (almost infinite) sum of simple oscillating functions so the frequency range is practically infinite but as it grows, the intensity in the amplitude spectrum drops. This is the reason why the limits for the OTF matrix were chosen to be the original image’s dimension.

The frequency response of the PSF takes the form of a sinc function. In the graphical representations we should see horizontal stripes fading to the upper and lower edges of the picture. The dark stripes represent the blocked frequency bands (neighboring areas to the zero crossings of the transfer function in absolute value) and, if overlapped, the two clusters found in the initial spectrum should be suppressed.

Because the ideal filter would not block entire bands but just the areas corresponding to the clusters, in our case we also get some other wanted frequencies eliminated, such as components on the vertical direction.

Figure 7: 1) The filter spectrum with the dark shades as blocked bands; 2) Initial image spectrum emphasizes the clusters symmetric with respect to the origin and 3) the filtered image where we can see how the lateral peaks turned dark blue, therefore being filtered

5. Explain why the selected sizes of the PSF are best? You can approach this explanation from the spatial domain, but also from the frequency domain by looking at the Fourier transforms and the OTF.

From exercise 3 we see the chosen dimensions were the minimum for good distortion suppression because a small group of pixels is preferred to a large kernel. Averaging over a greater area gets a more severe smoothing, blurring the image even more. The effects of applying several times a small kernel aren’t equal to applying once a large one.

Also, as discussed at exercise 4, the filter already blocks other components besides from the unwanted ones. A solution is applying a band-reject filter.

Filtering in the frequency domain blocks the unwanted frequency components and leaves all the other unaffected.

6. Identify the frequencies associated to the cluster we want to filter out.

Since the spectrum is symmetric with respect to the origin which, in the shifted version, corresponds to the center of the image. In order to find the coordinates of the clusters, we only need to get the ones of one cluster and the compute the symmetric with respect to C (128.128).We expect the values to be close to each other since the clusters are very well defined.

Using the data cursor tool, we get the [128,133] interval of variation on the horizontal axis and on the vertical axis: [89, 93] and [164:168] respectively. Since the transfer function should be symmetrical, we round these intervals to [124:132] for the x direction, [87:92] and [164:169] for the y direction.

Figure 8: The cluster coordinates

7. Create the H matrix of a filter that would block the frequency clusters of the distortion

H=ones(256,256);H(124:132, 87:92)=0;H(124:132, 164:169)=0;H1=im2double(fftshift(H)); figure;imshow(fftshift(H1),[]);title('Frequency Filter');

Figure 9: Frequency LPF determined by the computed intervals of variations on the axes

8. Apply the filter and verify the result computing the inverse Fourier Transform. Check for the imaginary parts of the filtered image.

IM=fft2(im);OTF2=im2double(H1.*IM);

figure;imshow(log(c+abs(fftshift(OTF2))),[]); psf2=ifft2(OTF2); figure; imshow(real(psf2),[]);

The round-off errors resulted from the fft operations and propagate and can be estimated by using:

er=ifft2(fft2(im))

The values found are in the range [0.0471, 0.9882]. The filtered image has the spectrum shown in Figure 10 and filters some of the distortions but there are still unfiltered components. Verifying the imaginary part of the reconstructed image, we find it to be far from null. Searching for these components in both “symmetrical” and “nonsymmetrical” psf given by the inverse Fourier Transform, we get to the same result different from zero (0.0408). Even if the rectangles were chosen to be symmetrical, the picture is 256x256 so the central pixel has practically the coordinates on both axes between pixels 127 and 128.

max1=max(imag(psf2));

Figure 10: Rectangular Low Pass Filter in frequency domain

This concludes that the chosen filter is still far from optimal. One solution would be increasing the rejected band by increasing the area with zero elements in the H matrix. Doing so, improved rejection is expected but the image should be heavily blurred.

Figure 11: An LPF with a wider reject band and the resulting image

Also, the imaginary component of the resulting reconstructed image has now dropped at a value of 0.0148 but this is still an unsatisfactory result. One solution is creating a filter that would not attenuate some of the wanted components, such as the corners of the rectangles.

Having the coordinates below for the new H matrix, we have the results depicted in Figure 12

H=ones(256,256);H(90:93,120:140)=0;H(84:100,126:134)=0;H(158:174,124:132)=0;H(165:168,118:138 )=0;

Figure 12: Non-rectangular filter and the resulting image

As another improvement of the cross low pass filter, the imaginary part of the reconstructed image is now equal to zero so the H array respects the symmetry requirements of a low pass filter transfer function.

9. Interpret the results

Eliminating a distortion element in an image can be filtered either in time or frequency domain. When the point spread function has reduced dimension, is more effective to run a direct filtering, by convoluting the image with the psf. The advantage of the direct filtering is the time needed for all the operations.

Comparing the two methods, we got for the direct approach a time of execution of 0.2052 seconds while for switching to frequency domain we would need 0.3386 seconds. The choice of the right method depends on the interested aspect of the program: accuracy or speed of execution.

In both cases the distortion was not completely filtered due to the approximation made when considering the stripes parallel to the horizontal. An improvement that could be brought is rotating the image so that the lines would be indeed parallel to the x axis, applying the filter and then rotating back to the original position.

Low pass filtering, as well as band-reject filtering, suppresses a (semi-infinite2) band of high frequencies thus, inevitably, reducing the detail of the resulting image on the vertical axis. Also, at the boundaries the distortion will still be present since the Fourier transform considers the image being circular.

2 In the low pass filter case

APPENDIXclcclose allclear all c=1e1; im=im2double(imread('C:\Users\Laura\Desktop\dbb_exercises\car_dis.png')); figure;imshow(im, 'Border', 'tight'); IM=fft2(im);spec=fftshift(log(c+abs(IM))); figure;imshow(spec,[], 'Border', 'tight');title('Amplitude spectrum'); tichsize=[7 1];psf1=fspecial('average',hsize);imconv=imfilter(im,psf1); figure;imshow (psf1,[], 'Border', 'tight');title ('PSF');repli = imfilter(im,psf1,'replicate'); figure; subplot(1,2,1);imshow(imconv,[], 'Border', 'tight'); title('Convoluted image'); subplot(1,2,2);imshow(repli,[], 'Border', 'tight'); title('Replicate');timert=toc; otf1=psf2otf(psf1,size(im));OTF1=log(c+abs(fftshift(otf1))); figure;imshow(OTF1,[], 'Border', 'tight');title ('OTF'); H=ones(256,256);H(90:93,120:140)=0;H(84:100,126:134)=0;H(158:174,124:132)=0;H(165:168,118:138 )=0;H1=im2double(fftshift(H)); figure;imshow(fftshift(H1),[], 'Border', 'tight');title('Frequency Filter'); tic OTF2=H1.*IM; figure;imshow(log(c+abs(fftshift(OTF2))),[]);timerf=toc; psf2=ifft2(OTF2); figure;imshow(log(c+abs(fftshift(OTF2))),[]); imshow(real(psf2),[]); PSF = abs(otf2psf(H1,size(im)));OTF = psf2otf(PSF); figure; subplot(1,2,1);mesh(PSF); title('PSF'); subplot(1,2,2);mesh(fftshift(OTF)); title('OTF');