how to deblur the image

8/12/2019 how to deblur the image

1/28

All contents are Copyright 2013 DeNies Video Software. All rights reserved 1

Deblurring: The Details MatterDecember, 2013

Mark DeNies

DeNies Video Software

Abstract

In recent years there have been majoradvancements in solving the ill-posed blind

deconvolution problem, that is: deblurring images

where both the true image and the blur areunknown. While in the general case the unknown

blur varies across the image, this paper deals only

with the shift-invariant blur, i.e., where the blurremains constant throughout the image. Thispaper examines the deblurring method used by

Levin et al. [8]1and the variant of Babacan et al.

[5]2 and finds that there are subtle changes in

implementation that individually make smallimprovements, but collectively result in a

significant advancement over the prior art.

The improvements can be grouped in three

categories: those that improve the quality of the

result, those that improve the flexibility of thealgorithms, and those that diminish the length of

time to solve the problem.

Nomenclature

The usual description of the blind deconvolution

problem is expressed as follows:

y= kx+ n (1)

whereyis the observed blurry and noisy image, kis the kernel or Point Spread Function (PSF), xis

the unknown true (or sharp) image, and n is

additive noise. This equation is applicable in the

1For the remainder of this paper Levin et al. [8] will be

simplified to Levin.

special case where the PSF is uniform throughout

the image.

The PSF is normally describe as an image thatrepresents how the light that should have fallen on

the center pixel was somehow diverted to other

pixels in the neighborhood while the shutter was

open. This can occur because of a combination offactors: an imperfect lens, an out of focus image,

and/or motion of the camera or the subject. The

known blurred image is the result of convolvingthe unknown true image with the unknown PSF

combined with noise, as described in equation (1).

Other terms used in this paper are MAPx whichis a maximum a-posteriori algorithm used to solve

for the true image, given a known blurred image

and a fixed PSF. Likewise, MAPk is amaximum a-posteriori algorithm which solves for

a PSF given a fixed true image and a known

blurred image.

Overview

Much of the recent research in blinddeconvolution began with the work of Fergus et

al. [3]. Fergus made the observation that natural

images have a unique distribution of first

derivatives, which can be approximated as amixture of Gaussian distributions, and used as a

prior to restrict the solution. Levin et al. [4]provides an analysis of why this technique was sosuccessful. A number of others have built on these

observations, such as Levin and Babacan. Yet

other researchers have pursued different paths,concentrating on refinements of MAP (Cho [12]),

2For the remainder of this paper, Babacan et al. [5] will

be simplified to Babacan.


2/28


3/28


Description of the Algorithm

In equation (1) k,xand nare all unknown. Thus,

equation (1) is a highly ill-posed problem with aninfinite number of solutions. In order to solve this

equation, additional constraints must be imposed.In the Bayesian framework, priors that model the

statistics of natural images are used.

A characteristic of natural images is that the first

derivatives of the image are sparse, with mostvalues being close to zero and few large values at

sharp edges. The distribution is super-Gaussian

and can be modelled as a mixture of Gaussian

(MOG) distributions [3, 5, 8] or as a hyper-Laplacian [9].

Figure 1: distribution of gradients. The verticalaxis is the fraction of pixels with the absolute

difference from its neighbors on the horizontal

axis.

Blue: blurred imageGreen: deblurred image

Red: true image

Magenta: Mixture of Gaussians

This article follows the successful strategy used in

Babacan and Levin, and provides a summary ofthe algorithm. See their work listed in the

references for a full description of the algorithm.

Starting with a low resolution version of the

blurred image where the blur is likely to be very

small, and working up to the full-sized image, the

PSF and deblurred image is found at each level by

alternately solving for the true image given a fixedPSF and blurred image, and solving for the PSF

given a fixed true image and the blurred image.

Formally:

x= arg max p(x|y, k) (2)

k= arg max p(k|x,y) (3)

When solving (2), this paper solves for the mean

image of p(x|y, k) and calculates the covariance

matrix Cxaround it.

The solution to (3) is:

k = arg min kTAkk2bkTk (4)

Given a PSF of size m by n, M = m*n and Akis a

matrix of size M by M. It contains the covariance

of all m by n windows in x, and bk is thecorrelation of x with y.

C(i,i) = diag Cx (5)p

Ak(i,j) = x(p+i)x(p+j) + C(p+i,p+j) (6)p

bk(i) = M x(n+i)y(n) (7)p

Where i,j are indexes in the PSF (i.e., 1M), and

p sums the correlation across all image pixels.

Methodology

Much of the methodology used in this work wascreated to minimize image edge effects. These

can occur in all stages of working on the

deblurring problem: when creating a syntheticblur, when making calculations throughout the

solution of the deconvolution problem, and lastly

when measuring the fidelity of the result.


4/28


Most researchers use synthetically blurred images

to test their algorithms, and this paper uses them

as well. This allows the resulting image to benumerically compared with the original using an

objective measurement such as PSNR (Peak

Signal to Noise Ratio), SSIM (StructuralSimilarity), SSD (Sum-Square-Distances) or errorratio [4].

The first concern is the method used to construct

a realistic blurred image. The method used in thiswork is to center the PSF, pad the image on all

sides by half the size of the PSF, edge taper the

image, and only then apply the blur. The

centering of the PSF prevents the process of

blurring from also causing a shift. Since the

deblurring algorithm solves for a true imagewhich is equally padded on all sides by half the

size of the blur, it is implicitly assuming a

centered PSF. Padding and edge tapering arecommon techniques used to minimize the edgeeffects. In Table 1 below, the effects of these

steps can be seen. The deblurred image created

from the un-centered PSF is shifted down and

slightly to the left compared to the original image.The PSNR is improved due to the improved

symmetry of the blurred image.

Table 1: Effect of centering PSF before creating a blurred image (using Levins code)

Original Image Blurred image Deblurred ImageUn-centered

PSF

PSNR: 28.11 SSIM=0.885Centered

PSF

PSNR: 28.90 SSIM: 0.896


5/28


In measuring the fidelity of the image (PSNR and

SSIM), this work employs two techniques: First,

only the center of the deblurred image iscompared to the reference image, since the edge

contains the effects of the padded area around the

blurred image. Secondly, to allow for small shiftsin the result, the image is matched to sub-pixelresolution. An interesting relationship is seen in

the following table, showing how the PSNR of the

first deblurred image varies with the amount of

sub-pixel resolution:

Table 2: Effect of Pixel resolution on PSNR

Pixelresolution

PSNR

1 33.881

0.5 33.8810.25 34.588

0.125 34.651

0.0625 34.8079

0.03125 34.8079

0.015625 34.8083

0.0078125 34.8104

Sub-pixel resolution matching can result in up toa difference in measured PSNR of almost 1.0 over

a match to the nearest pixel for the same pair of

images. In this paper, the results will be shownfor a pixel matching resolution of 0.0078125pixels, and the search area will be 4 pixels in each

direction.

Another concern is whether the true image is

actually free of blur. In the case of the test set used

here (from Levin), it can easily be shown that theoriginal image contains a small blur. Using the

deblurring algorithm of Levin on their first test

image the following images result:

Figure 2: Original image

Figure 3: Deblurred Original image and the

discovered PSF (zoom in to see the small PSF):

The image of Figure 3 is noticeably sharper than

the original image of Figure 2. The PSNR

between the two is 33.9.

When the algorithm in this paper is used to deblur

the first image in the test suite of Levin (the first

blurred image in the Appendix), the followingimage is obtained:


6/28


Figure 4: Deblurred Original Image

PSNR to Fig 2: 30.35 PSNR to Fig 3: 31.47

This image is closer to Figure 3, which indicatesthat the deblurring algorithm is removing most of

the blur introduced by the synthetic blur as well as

some of the latent blur in the original image. For

this reason, when results are reported in thispaper, the True image will be the deblurred

original image.

Yet another consideration is to be mindful of theedge effects in the deblurring in all of the

algorithms used in blind deconvolution. This

subject will come up often in the next twosections.

Details of the Implementation

This work uses a multilevel solver in order toquickly determine the support and shape of the

PSF. Within each level the noise parameter is

reduced on each iteration and the algorithmalternates between a Bayesian MAPx solver to

solve for the true image and a modified MAPk

solver from Levin to solve for the PSF.

The MAPxsolver used in this work, unlike that of

other researchers, solves in the image space using

five derivatives. As is typical, a conjugate

gradient algorithm is used. This conjugategradient algorithm is initialized with the previous

true image, and a novel stopping criteria is

introduced. Simultaneous with solving for the

true image, it calculates the covariances of the trueimages x and y gradients for use in the MAPk

solver.

The MAPksolver is the same as the one used byLevin and Babacan. However, only the interior of

the blurred and true images are used to set up the

quadratic problem to be solved. This excludes the

problematic edge pixels from participating in thesolution. Because of this, a somewhat lower

sparsity weight is used than in Babacan, which in

turn is significantly lower than in Levin. This willbe discuss further in the section on Quality.

Between each level, the PSF is resized, trimmedand centered. In the default case, the true andblurred images are doubled in area (1.414 in each

dimension). But, typically the new PSFs

dimensions will not be an exact multiple of 1.414.

Centering the PSF to a sub-pixel resolutioncorrects for any drift at the current level. To

accomplish both objectives, an intermediate PSF

is created which is a multiple of the destinationPSF, the PSF is scaled, centered, and then down-

sampled. For example, if the original PSF is 7x9,

and the desired PSF for the next level is 11x13,the 7x9 PSF would first be resized to 99x127 (the

closest integers to 14.14 times 7x9). The 99x127

PSF is placed in the center of a second 110x130

PSF, and then centered to the nearest pixel. The110x130 PSF is then down-sampled by a factor of

10 to create an 11x13 pixel PSF accurately

centered to within 1/10 of a pixel.

Similarly, the deblurred image is resized for the

next level, taking care to separately resize the

core image and the padded border. Morespecifically, the padded border is removed, and

the resultant core is resized by 1.414 in each

dimension. The padded border is separated into

the 4 sides and 4 corners. The sides are resized byratio of the PSF sizes in the dimension toward the

edge and 1.414 in the other. The corners are

resized by the ratio of the PSF sizes.


7/28


In the Matlab source code associated with this

paper, there are a number of parameters that areset in the top level which control various aspects

of the solver. These include:

Blind noise level: the noise level to beused in the solution of the multilevelprocess of finding the PSF. Default is .01.

The non-blind noise level to be used in the

final deconvolution of the image. Default:

.007. The amount of image reduction at each

level. Default: .7071 (each image is half

size). The number of iterations in each level.

Default: 15

A threshold value used to trim tiny PSFvalues when resizing the PSF. Default:.005 (pixels less than this are set to zero).

The noise step size. Default: .88

The conjugate gradient stopping criteria.

Default: stop when .00075 of the pixelschange less than .01 from the prior

iteration.

The matlab implementation may be foundhere.

Quality Improvements

Improvements to the quality of the solutiondiscovered in this work include employing a

novel stopping criteria in the conjugate gradient

algorithm, using only the interior pixels of the trueimage to solve for the PSF, and solving for the

true image in image space rather than the more

common solution in filter (derivative) space.

In solving for the MAPx (the true image), care

must be taken to bring out the detail, and yet avoidmagnifying the noise. A characteristic of theconjugate gradient algorithm is that it spends the

initial iterations enhancing the detail, and then

later as it continues to change the image, it alsoenhances the noise. So, there is a balancing act

between the number of iterations in the conjugate

gradient solver and the specified noise level.

Using a fixed value of the residual as a stopping

criterion is not altogether effective, which is why

many researchers use a fixed number of iterationsand rely on the noise parameter to control the

enhancement of noise. To be safe, these

researchers often use a larger noise parameter. Toaddress this problem, this work introduces astopping criterion which appears to be effective in

exiting the conjugate gradient algorithm once the

image details have first appeared. Doing so has

the added benefit of significantly reducing theiteration count and allowing smaller noise

parameters to be used. The stopping criteria is

controlled by two parameters: MaxChange andMaxPCT. Only the center pixels in the image are

examined: those that are a distance of at least half

the PSF size from the edge. When less thanMaxPCT of these pixels change more thanMaxChange from the prior iteration, the

conjugate gradient loop will exit. The default

values are MaxChange = .00075, and MaxPCT =

.01 (1 percent).

Levin shows that a MAPk solution produces

superior results. This technique was also used inthe work by Babacan. The difference between

these two implementations is that in Levin, the

padded true image and its covariances are usedalong with the smaller blurred image to create the

coefficients in the quadratic equation to be solved,

and a larger value (.01) is used for the sparsity

parameter (k_prior_ivar). Babacan uses a trueimage and covariance matrix the same size as the

blurred image and a smaller sparsity paramenter

(.001). Using the padded true image introducesinformation from the problematic edges of the

true image. This introduces noise into the PSF,

which is partially removed by a larger sparsity

parameter. This papers implementation goes astep further than Babacan: the MAPkmatrix and

vector are built from the interiors of the true image

and the blurred image. This allows a smaller

sparsity parmeter to be used (.0005) which issmaller than that used by Babacan (.001). To

illustrate the efficacy of these changes, Levins

original code was modified to use the smaller
http://www.deniesvideo.com/DeblurCode.ziphttp://www.deniesvideo.com/DeblurCode.ziphttp://www.deniesvideo.com/DeblurCode.ziphttp://www.deniesvideo.com/DeblurCode.ziphttp://www.deniesvideo.com/DeblurCode.zip


8/28


prior_ivar_k, and the smaller interior of the true

and blurred images. This results in a significant

improvement in detail as shown in Table 3.

Table 3: Effect of using different sized areas in calculating the MAPkmatrix and vector

Blurred Image Levin Babacan Modified Levin Proposed

PSF

PSNR 26.30 28.72 29.10 31.47

PSF

PSNR 23.74 24.64 26.66 27.56

These results show the effects: the original Levin

code, by using the larger areas (and including thetrue images boundary), has more noise in thePSF, even though it uses the larger sparsity

parameter. Babacans results show the smallest

amount of noise in the PSF, but the PSF is actually

too sparse. The modified Levin example doesbetteralthough there is still noise in the PSF, it

is small and the PSF is sharper. The proposed

algorithm has some noise, which is between that

of Babacan and the modified Levin, but it has asharp PSF. This improvement is the result of theother modifications, such as a more complex

resizing of the PSF and estimated true image

between levels, and always using the prior

iterations result when solving for the currentiterations true image. Also, unlike Levin and

Babacan, the true image is solved in image space,


9/28


and like them, the PSF is solved in derivative (or

filter) space.

Time/flexibility improvements

Several techniques were used to significantly

reduce the execution time of the software:1. Theprior iterations true image is used as

the conjugate gradients starting image.

2. The PSF and true image are accuratelyresized between levels.

3. Further speed improvements are

incorporated into the calculation of the

MAPkarrays.

The first is to use the deblurred image from theprevious iteration as the starting image for thecurrent iteration. When this is done, the conjugate

gradient function starts with an estimate that is

closer to the final solution, and therefore will

converge faster. This assumes that the stoppingcriteria works well. For the initial iteration at each

level, the deblurred image from the prior level is

used. In the MAPxsolver the conjugate gradient

solver is called three times. Most of the iterationsoccur in the first call.

Using the first image of Levins test suite as an

example, if the conjugate gradient solver isinitialized with a constant image (e.g., gray), it

typically takes 150 iterations for the three

conjugate gradient solutions. Initializing with theblurred image cuts the number of iterations to 85.

Initializing with the prior deblurred image reduces

it further to 60. This is a significant time savings,

allowing the slower solution in image space to becompetitive with solving in filter (derivative)

space. In the filter space solver of Levin, themaximum number of iterations (15) is reachedabout 80% of the time, and since there are two

derivatives solved separately on each iteration,

this results in approximately 80 conjugategradient iterations per pass.

For this to be effective for the first iteration at

each level, the deblurred image from the prior

level must be properly resized. The deblurredimage consists of the center core surrounded by

padding. The core scales by the same factor as the

blurred image between each level. But thepadding scales according to the ratios of the PSFsizes. This might seem like a small difference, but

it can significantly increase the number of

iterations if the resizing is not performed properly.

So, the true image is resized in 9 pieces: the core,the four sides, and the four corners.

The PSF is resized as described in the sectionDetails of the Implementation.

Another, rather mechanical time improvement isto further enhance the already efficient correlationfunction, getCorAbDiagCov() of Levin by

unrolling the loops. This saves 30% of the time

spent by getCorAbDiagCov(), and reduces the

execution time overall by 7%.

Multilevel Solver

Fergus [3] introduced a multilevel solver as wellas the use of the distribution of image gradients to

solve the blind deconvolution problem. Manyresearchers have adopted this technique in their

work. The idea is that solving the low resolutionimages will find the gross details of the PSF, and

these will be refined at higher resolution images.

At each level, a number of iterations (10 or more)will be made to further bring out detail as the

algorithm alternates between solving for the

image (typically in gradient space) and solving for

the PSF, gradually reducing the noise estimate tothe final estimated noise.

Babacan utilizes this technique and convenientlyproduces an image showing the progression of

refinement of the PSF. The two figures below

show the progress in the final two levels.


10/28

All contents are Copyright 2013 DeNies Video Software. All rights reserved10

Figure 5: Next to last level

Figure 6: Last level

As can be seen in Figures 5 and 6, although thefinal PSF from the first image is resized and used

in the starting iteration of the next level, it is

destroyed by the MAPk solver, and the detailed

PSF is then re-discovered. Why? The answer isthe high noise estimates which control how much

of the image is to be considered as noise (and

thrown away), leaving only the coarse imagedetails for the MAPksolver to discover.

The loss of PSF detail in the first iteration

suggests two possibilities: could the lower levels

be eliminated, and only the full resolution imagebe solved? Or could the initial noise level be

reduced when iterating at each level?

The answer to the first possibility is yes. The

initial levels are wasted. Usings Levins code

with the first image in her test suite, and using anidentity starting PSF, Table 4 shows the resultswith and without the initial four levels:

Table 4: Levins algorithm with and without using a multilevel solver

5 levels Last level only

DeblurredImage

PSF

PSNR: 26.31 SSIM: .806 Time: 389s PSNR: 26.45 SSIM: .810 Time: 257s


11/28


Table 5: Levins algorithm with fewer noise reduction iterations

Iterations: 11 9 7 5

DeblurredImage

PSFon each

iteration

of final

level

PSNR: 26.31 SSIM:

.806 Time: 389s

PSNR: 26.34 SSIM:

.808 Time: 306s

PSNR: 25.95 SSIM:

.800 Time: 249s

PSNR: 25.68 SSIM:

.794 Time: 184s

As for the second possibility: what happens if

each level starts with a lower noise level and the

number of steps at each level is reduced? For theexample shown in Figures 5 and 6, if the first five

iterations are eliminated, the starting PSF of

Figure 6 would be closest to the final PSF of

Figure 5. This sounds appealing, since much ofthe PSF is known from the prior levels, and only

the details remain to be discovered. An additional

observation is that the low resolution images have

inherently less noise than the full sized image much of the noise is canceled out by averaging

pixel values in the process of reducing the image.

However, as shown in Table 5, anything below 9-11 iterations gives increasingly poor results. It

appears that the PSF becomes overly sharp with a

reduced size image.

So is there any reason to use the multilevel solver?

There are, in fact, two reasons. The first is when

solving in image space. This will be shown in thenext section. The second is that using a multilevel

solver allows the software to determine the

support for the PSF in a time-efficient manner,

allowing the final step to be performed with an

estimate of the PSF size which is appropriate(over-estimating the support is costly in execution

time).

Image space vs. Gradient Space

Levin provides a number of solvers with which toexperiment. In general their tests show that

solving the problem in gradient space works best.

Not only are the results more accurate, but the

MAPxgradient solver can be terminated after 15iterations. By contrast the MAPx image space

solver must iterate 50 times. It should also be

noted that Levin set the number of iterations for

the final image space solver to 70. Deblurring the

first test image (image1/kernel1) and the twelfthimage) image2/kernel4 with the gradient space

deblur algorithm and image space algorithmproduces the following results:


12/28


Table 6: Progression of PSF at the final level using various algorithms

MAPx/MAPk

Deblurred Image PSF for eachiteration

Deblurred Image PSF for eachiteration

Gradient

Space/Gradient

Space

(Levin)

PSNR = 26.31 PSNR = 23.74

ImageSpace/

Image

Space(Levin)

PSNR = 25.16 PSNR = 21.99

ImageSpace/

Gradient

Space(this

work)

PSNR = 31.47 PSNR = 27.56

First, it is apparent that solving in gradient space

is superior to solving in image space, as Levin

observed. However, the progression of PSFsolution with noise is much slower in the image

space solver. In fact, using the multilevel solver

is absolutely required when solving in image

space!

In this work, it was decided to use an image space

MAPxsolver inside of each levels iteration loop.


13/28


The reason for this was that other researchers

(Levin and Babacan among them) preferred to use

the image space solver for the final deblur, usingfive derivatives for more accuracy. Since this

results in a more accurate true image at each

iteration, it was felt that a more accurate trueimage should also result in more accuracy in thesolved PSF. But, solving in image space is

slower. In this work, the slowness was partially

rectified by initializing the MAPx image space

solver with the prior iterations solution, thuseliminating the first set of conjugate gradient

iterations. Then by adding a stopping criteria,

which measures the rate of change in the interiorof the true image, the MAPx solver exits well

before the maximum number of iterations is

reached.

As shown by the results in the Appendix, this

combination results in a superior solution. It

seems that when solving for the true image in the

filter space, the solution converges faster, but thatsolving in the image space, a better image is

obtained. This only works if it is coupled with the

superior derivative space solver for the PSF.Secondly, there is little carry-over of information

from one level to the next in the filter space solver,

since its solutions during the initial iterations ofeach level results in data which the MAPksolver

uses to find a very Gaussian-like PSF (as

mentioned in the section Multilevel Solver).

Results on Synthetic Images

For comparison this paper uses the test suite of

Levin. The Appendix shows the results for all 32

images in the suite using the followingalgorithms: the original Levin code, Levin

modified with the changes described in the

Quality Improvements section, Babacan, and this

implementation.

The Appendix shows a steady improvement in

quality in the order: Levin Original, Babacan,

Modified Levin, and this papersimplementation.

A surprise result was that there is a significant

time improvement for the Modified Levin test

over the Levin Originaltest.

Results on Real Images

This section compares the techniques of Levin,Babacan, Shan, and this work. The test images

were taken from Krishnan et al. [9] and Cho [12].Default control parameters were used for all tests.

Table 8 shows that, overall, Levins original

technique produces the images with the fewest

artifacts, although the images are still a bit blurry(e.g. the Philipson image 3 is barely readable).

The modified Levin code, which did much better

on Levins test suite in the Appendix, fared poorly

here. Babacans results are intermediate betweenthe two. The results for Shan et al. [7] take less

time (the algorithm is released as an executable,not as matlab code) and does well on these

examples. The proposed algorithm createssharper images (except for the sailboat), but has

noticeable artifacts.

Better results could have been found by

experimenting with the parameters, which each

technique allows.


14/28


Table 7: Real images (zoom in to see details). Note: Levin and Babacan were modified to deblur color

images.

Blurred

Levin

Time: 12021s Time: 5427s Time: 5009s Time: 4864s

Levin

Modified

Time: 5677s Time: 4212s Time: 4267s Time: 3973

Babacan



15/28


Shan


ShanPSF

Proposed


Table 8 shows the results of deblurring the

sailboat and flowers images with this papersalgorithm using default parameters, and an

alternate set of parameters. Unfortunately, this is

a tedious process, and relies on the subjectivejudgment of the person deciding what is best.

In this case, four to six different trials were made

with various combinations of higher sparsity andhigher noise settings before choosing the ones in

the second and fourth panels of Table 8.

Essentially sharpness was traded for fewer echoartifacts in these images.


16/28


Table 8: Sailboat and flowers with alternate parameters

Default

settings

Sparsity .002

Blind noise .04Non-blind Noise .02

TrimPSFPct .02

Default

settings

Sparsity .005

Blind Noise .014Non-blind Noise .012

Time: 6240s Time:1334s

Time: 4509 Time: 3920s

What doesnt work as well

Although the technique presented here performswell, there are blurred images for which the

algorithm fails. The first example is when theoriginal Levin images are deblurred to find a

True image. The first three images in the

Appendix, when deblurred, find reasonable, small

blurs. But the fourth image has a significantdiagonal tail when deblurred. Not so

coincidently, this tail is in the same direction as

the stripes on the childs shirt. In many of the

deblurred fourth images in the Appendix, this

same diagonal tail appears. In fact, this same tailappears when the eight blurred images are

deblurred regardless of the deblurring algorithmsthat were used, including: Shan et al., Cho, and

others.

As a second example of this effect, a non-blurry

image with a strong diagonal pattern in a portionof the image was used (in this case, a fence). The

entire image was deblurred, then only the portion

without the diagonal feature was deblurred. The

results appear in Table 9.


17/28


Table 9: Effect of strong features on the discovered PSF.

Original Deblurred PSF

Zoom in to see that the PSF for the first image has a noticeable diagonal.

The PSF for the image with the fence has a faint

tail in the same direction as the fence.

Presumably this will also occur for more complex

correlation patterns. It is even possible thathidden correlation patterns exist in the other three

images in the test suite that account for some of

the noise which is observed in the PSFs.

A method to eliminate this effect is left to further

research.

Conclusion

This paper has demonstrated that multiple small

changes in the implementation of a deblurring

algorithm can combine to result in significant

improvements in the quality of the results and/orthe time required to achieve a solution.

Overall, the enhancements in this paper result in

an average quality improvement as measured byPSNR from 26.26 for Levin, 27.12 for Babacan,

to 29.20 for this papers implementation.


18/28


Future work

A natural follow-on to this work is to investigate

how to overcome the false tail which wasdiscussed in the section What doesnt work

well.

An unrelated observation is that some PSFs resultin a deblurred image which is still noticeably

blurry, no matter whose algorithm is used. A

Gaussian blur is one of these. Why this occurs hasnot been addressed in the research as far as this

author could find.

As was noted in the section Results on RealImages, the default parameters which control the

deblurring process are, at best, a compromise.

Better results for an individual image are often

found with different settings for the parameters.This could be automated, but an algorithmic test

of the goodness of the image is needed. There

has been some research into this problem(especially in estimation of noise), but this authorfound the methods for estimating noise gave poor

results for the images used in this paper.

Another potential area of investigation is to applythe techniques used here to the general shift-

variant deconvolution problem.


19/28


Appendix

This appendix compares several algorithms using Levins test suite. This suite consists of four imagesblurred with eight PSFs as described in the Methodology section. In this work, the original image is also

deblurred to discover a True image. The tests labeled Levin Original and Babacan were run using

the original codes settings, but Babacans was modifiedto use Levins equivalent optimized correlationcode to calculate the kernel at each step, since Babacans method ran out of memory on 32 bit machines forthe larger kernels.

Modified Levinis the original Levin code modified to use only the interiors of the true and blurred imagesin getCorAbDiag, and the k_prior_ivar (kernel sparsity constant) changed from .01 to .001, and w0 in

deconvSPS.m (controls the amount of detail/noise) changed from .1 to .01.

The following PSFs were applied to the Original image after being centered. The TruePSF will be acombination of this PSF and the small blur inherent in the Original image.

PSFs:

Image 1

Original True


20/28


Blurred Levin Original Modified Levin Babacan Proposed

PSNR: 26.31 SSIM: .806 398s PSNR: 29.10 SSIM: .901 320s PSNR: 28.72 SSIM: 0.91 547s PSNR: 31.47 SSIM: .943 307s


PSNR: 27.05 SSIM:.835 286s PSNR: 29.89 SSIM: .910 262s PSNR: 29.90 SSIM: 0.93 379s PSNR: 31.93 SSIM: .946 324s

PSNR: 23.97 SSIM:.741 889s PSNR: 26.75 SSIM: .838 581s PSNR: 26.65 SSIM: 0.87 1259s PSNR: 26.68 SSIM:.881 685s



21/28






22/28


Image 2Original True

Blurred Levin Modified Levin Babacan Proposed

PSNR: 24.50 SSIM:.763 379s PSNR: 27.02 SSIM: .863 307s PSNR: 25.29 SSIM: 0.83 532s PSNR: 28.61 SSIM: .908 426s




23/28




PSNR: 26.09 SSIM: .829 537s PSNR: 30.06 SSIM:.917 383s PSNR: 27.51 SSIM: 0.91 762s PSNR: 28.91 SSIM: .920 514s




24/28


Image 3

Original True






25/28






PSNR: 27.66 SSIM: .869 643s PSNR: 31.75 SSIM: .935 429s PSNR: 28.67 SSIM:0.90 886s PSNR: 24.85 SSIM: .805 498s


26/28


Image 4

Original True


PSNR: 25.52 SSIM: .793 379s PSNR: 26.38 SSIM:.842 301s PSNR: 25.55 SSIM:.82 495s PSNR: 26.54 SSIM: .863 692s

PSNR: 24.00 SSIM: .756 332s PSNR: 24.42 SSIM:.784 268s PSNR: 21.95 SSIM: 0.66 429s PSNR: 24.88 SSIM: .819 632s



27/28





PSNR: 24.62 SSIM: .799 530s PSNR: 24.30 SSIM: .798 380s PSNR: 24.21 SSIM: .82 725s PSNR: 25.35 SSIM: .862 807s


Averages for all 32 images: PSNR: 26.26 SSIM: .822 478s PSNR: 28.37 SSIM: .876 352s PSNR: 27.12 SSIM: .86 657s PSNR: 29.20 SSIM: .899 569s


28/28

References

1. William Hadley Richardson. Bayesian-Based Iterative Method of Image

Restoration, (1972).

http://www.opticsinfobase.org/abstract.cfm?id=54565.

2. L. B. Lucy. "An iterative technique for the

rectification of observed distributions", 1974.

Astronomical Journal79(6): 745754.

Bibcode:1974AJ.....79..745L.

doi:10.1086/111605.

3. Rob Fergus, Barun Singh, Aaron Hertzmann,

Sam T. Roweis, William T. Freeman.

Removing Camera Shake from a SinglePhotograph, 2006.http://cs.nyu.edu/~fergus/papers/deblur_ferg

us.pdf

4. Anat Levin, Yair Weiss, Fredo Durand,

William T. Freeman. Understanding and

evaluating blind deconvolution algorithms.Computer Vision and Pattern Recognition,

2009.

http://www.wisdom.weizmann.ac.il/~levina/

papers/deconvLevinEtalCVPR09.pdf

5. S. Derin Babacan, Rafael Molina, Minh N.

Do, Aggelos K. Katsaggelos. BayesianBlind Deconvolution With General Sparse

Image Priors, 2012.

http://www.dbabacan.info/BDGSP.php

6. Deepa Kundur. A Novel Blind

Deconvolution Scheme for Image

Restoration Using Recursive Filtering,

1998.http://ik.itba.edu.ar/~jacoby/Image%20Proce

ssing/Teoria/Clase7/Papers/a%20novel%20bl

ind%20deconvolution%20scheme%20for%2

0ir%20using%20recursive%20fi.pdf

7. Qi Shan, Jiaya Jia, Aseem Agarwala. High-

quality Motion Deblurring from a Single

Image, 2008.http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.218.6835&rep=rep1&type=pdf

8. Anat Levin, Yair Weiss, Fedo Durand,

William T. Freeman. Efficient MarginalLikelihood Optimization in Blind

Deconvolution, 2011.

http://www.wisdom.weizmann.ac.il/~levina/papers/deconvLevinEtalCVPR11.pdf

9.

Dilip Krishnan, Rob Fergus. Fast ImageDeconvolution using Hyper-LaplacianPriors, 2009.

http://cs.nyu.edu/~dilip/research/papers/fid_n

ips09.pdf

10.Anat Levin, Rob Fergus, Fredo Durand,

William T. Freeman. Deconvolution using

natural image priors MIT 2007.http://groups.csail.mit.edu/graphics/CodedAp

erture/SparseDeconv-LevinEtAl07.pdf

11.Neel Joshi, Richard Szeliski, David J.

Kriegman. PSF Estimation using Sharp

Edge Prediction, 2008.

http://vision.ucsd.edu/kriegman-grp/research/psf_estimation/psf_estimation.p

df

12.Taeg Sang Cho. Removing motion blur

from photographs. PhD thesis, MIT 2010.

http://people.csail.mit.edu/taegsang/Docume

nts/PhDThesis_TaegSangCho.pdf
http://www.opticsinfobase.org/abstract.cfm?id=54565http://www.opticsinfobase.org/abstract.cfm?id=54565http://www.opticsinfobase.org/abstract.cfm?id=54565http://en.wikipedia.org/wiki/Bibcodehttp://adsabs.harvard.edu/abs/1974AJ.....79..745Lhttp://adsabs.harvard.edu/abs/1974AJ.....79..745Lhttp://adsabs.harvard.edu/abs/1974AJ.....79..745Lhttp://en.wikipedia.org/wiki/Digital_object_identifierhttp://dx.doi.org/10.1086%2F111605http://dx.doi.org/10.1086%2F111605http://dx.doi.org/10.1086%2F111605http://cs.nyu.edu/~fergus/papers/deblur_fergus.pdfhttp://cs.nyu.edu/~fergus/papers/deblur_fergus.pdfhttp://cs.nyu.edu/~fergus/papers/deblur_fergus.pdfhttp://www.wisdom.weizmann.ac.il/~levina/papers/deconvLevinEtalCVPR09.pdfhttp://www.wisdom.weizmann.ac.il/~levina/papers/deconvLevinEtalCVPR09.pdfhttp://www.wisdom.weizmann.ac.il/~levina/papers/deconvLevinEtalCVPR09.pdfhttp://www.dbabacan.info/BDGSP.phphttp://www.dbabacan.info/BDGSP.phphttp://ik.itba.edu.ar/~jacoby/Image%20Processing/Teoria/Clase7/Papers/a%20novel%20blind%20deconvolution%20scheme%20for%20ir%20using%20recursive%20fi.pdfhttp://ik.itba.edu.ar/~jacoby/Image%20Processing/Teoria/Clase7/Papers/a%20novel%20blind%20deconvolution%20scheme%20for%20ir%20using%20recursive%20fi.pdfhttp://ik.itba.edu.ar/~jacoby/Image%20Processing/Teoria/Clase7/Papers/a%20novel%20blind%20deconvolution%20scheme%20for%20ir%20using%20recursive%20fi.pdfhttp://ik.itba.edu.ar/~jacoby/Image%20Processing/Teoria/Clase7/Papers/a%20novel%20blind%20deconvolution%20scheme%20for%20ir%20using%20recursive%20fi.pdfhttp://ik.itba.edu.ar/~jacoby/Image%20Processing/Teoria/Clase7/Papers/a%20novel%20blind%20deconvolution%20scheme%20for%20ir%20using%20recursive%20fi.pdfhttp://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.218.6835&rep=rep1&type=pdfhttp://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.218.6835&rep=rep1&type=pdfhttp://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.218.6835&rep=rep1&type=pdfhttp://www.wisdom.weizmann.ac.il/~levina/papers/deconvLevinEtalCVPR11.pdfhttp://www.wisdom.weizmann.ac.il/~levina/papers/deconvLevinEtalCVPR11.pdfhttp://www.wisdom.weizmann.ac.il/~levina/papers/deconvLevinEtalCVPR11.pdfhttp://cs.nyu.edu/~dilip/research/papers/fid_nips09.pdfhttp://cs.nyu.edu/~dilip/research/papers/fid_nips09.pdfhttp://cs.nyu.edu/~dilip/research/papers/fid_nips09.pdfhttp://groups.csail.mit.edu/graphics/CodedAperture/SparseDeconv-LevinEtAl07.pdfhttp://groups.csail.mit.edu/graphics/CodedAperture/SparseDeconv-LevinEtAl07.pdfhttp://groups.csail.mit.edu/graphics/CodedAperture/SparseDeconv-LevinEtAl07.pdfhttp://vision.ucsd.edu/kriegman-grp/research/psf_estimation/psf_estimation.pdfhttp://vision.ucsd.edu/kriegman-grp/research/psf_estimation/psf_estimation.pdfhttp://vision.ucsd.edu/kriegman-grp/research/psf_estimation/psf_estimation.pdfhttp://vision.ucsd.edu/kriegman-grp/research/psf_estimation/psf_estimation.pdfhttp://people.csail.mit.edu/taegsang/Documents/PhDThesis_TaegSangCho.pdfhttp://people.csail.mit.edu/taegsang/Documents/PhDThesis_TaegSangCho.pdfhttp://people.csail.mit.edu/taegsang/Documents/PhDThesis_TaegSangCho.pdfhttp://people.csail.mit.edu/taegsang/Documents/PhDThesis_TaegSangCho.pdfhttp://people.csail.mit.edu/taegsang/Documents/PhDThesis_TaegSangCho.pdfhttp://vision.ucsd.edu/kriegman-grp/research/psf_estimation/psf_estimation.pdfhttp://vision.ucsd.edu/kriegman-grp/research/psf_estimation/psf_estimation.pdfhttp://vision.ucsd.edu/kriegman-grp/research/psf_estimation/psf_estimation.pdfhttp://groups.csail.mit.edu/graphics/CodedAperture/SparseDeconv-LevinEtAl07.pdfhttp://groups.csail.mit.edu/graphics/CodedAperture/SparseDeconv-LevinEtAl07.pdfhttp://cs.nyu.edu/~dilip/research/papers/fid_nips09.pdfhttp://cs.nyu.edu/~dilip/research/papers/fid_nips09.pdfhttp://www.wisdom.weizmann.ac.il/~levina/papers/deconvLevinEtalCVPR11.pdfhttp://www.wisdom.weizmann.ac.il/~levina/papers/deconvLevinEtalCVPR11.pdfhttp://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.218.6835&rep=rep1&type=pdfhttp://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.218.6835&rep=rep1&type=pdfhttp://ik.itba.edu.ar/~jacoby/Image%20Processing/Teoria/Clase7/Papers/a%20novel%20blind%20deconvolution%20scheme%20for%20ir%20using%20recursive%20fi.pdfhttp://ik.itba.edu.ar/~jacoby/Image%20Processing/Teoria/Clase7/Papers/a%20novel%20blind%20deconvolution%20scheme%20for%20ir%20using%20recursive%20fi.pdfhttp://ik.itba.edu.ar/~jacoby/Image%20Processing/Teoria/Clase7/Papers/a%20novel%20blind%20deconvolution%20scheme%20for%20ir%20using%20recursive%20fi.pdfhttp://ik.itba.edu.ar/~jacoby/Image%20Processing/Teoria/Clase7/Papers/a%20novel%20blind%20deconvolution%20scheme%20for%20ir%20using%20recursive%20fi.pdfhttp://www.dbabacan.info/BDGSP.phphttp://www.wisdom.weizmann.ac.il/~levina/papers/deconvLevinEtalCVPR09.pdfhttp://www.wisdom.weizmann.ac.il/~levina/papers/deconvLevinEtalCVPR09.pdfhttp://cs.nyu.edu/~fergus/papers/deblur_fergus.pdfhttp://cs.nyu.edu/~fergus/papers/deblur_fergus.pdfhttp://dx.doi.org/10.1086%2F111605http://en.wikipedia.org/wiki/Digital_object_identifierhttp://adsabs.harvard.edu/abs/1974AJ.....79..745Lhttp://en.wikipedia.org/wiki/Bibcodehttp://www.opticsinfobase.org/abstract.cfm?id=54565http://www.opticsinfobase.org/abstract.cfm?id=54565