mathematical beer goggles or the mathematics of image ...mamamf/talks/pss08.pdfthe image deblurring...
TRANSCRIPT
![Page 1: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/1.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Mathematical Beer Gogglesor
The Mathematics of Image Processing
Melina Freitag
Department of Mathematical SciencesUniversity of Bath
Postgraduate Seminar SeriesUniversity of Bath12th February 2008
![Page 2: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/2.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
1 Motivation
2 How images become numbers
3 Compressing images
4 The image deblurring problem
5 Blurring and Deblurring imagesThe blurring functionDeblurring
![Page 3: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/3.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Outline
1 Motivation
2 How images become numbers
3 Compressing images
4 The image deblurring problem
5 Blurring and Deblurring imagesThe blurring functionDeblurring
![Page 4: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/4.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Outline
1 Motivation
2 How images become numbers
3 Compressing images
4 The image deblurring problem
5 Blurring and Deblurring imagesThe blurring functionDeblurring
![Page 5: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/5.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Grayscale images
One dimensional matrix
X =
0 0 0 0 0 0 0 0 00 8 8 0 4 4 0 2 00 8 8 0 4 4 0 2 00 8 8 0 4 4 0 2 00 8 8 0 4 4 0 2 00 0 0 0 0 0 0 0 0
imagesc(X), colormap(gray)
![Page 6: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/6.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
MATLAB image
![Page 7: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/7.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
RGB images
Three dimensional matrix
X(:, :, 1) =
0 0 0 0 0 0 0 0 00 1 1 0 1 1 0 0 00 1 1 0 1 1 0 0 00 0 0 0 0 0 0 0 0
X(:, :, 2) =
0 0 0 0 0 0 0 0 00 0 0 0 1 1 0 0 00 0 0 0 1 1 0 0 00 0 0 0 0 0 0 0 0
X(:, :, 3) =
0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 1 00 0 0 0 0 0 0 1 00 0 0 0 0 0 0 0 0
imagesc(X)
![Page 8: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/8.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
MATLAB image
X = imread(’pic.jpg’), imwrite(X,’pic.jpg’)
![Page 9: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/9.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Outline
1 Motivation
2 How images become numbers
3 Compressing images
4 The image deblurring problem
5 Blurring and Deblurring imagesThe blurring functionDeblurring
![Page 10: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/10.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
The singular value decomposition (SVD)
Existence and Uniqueness
Let X ∈ Cm,n, m ≥ n Then
X
[v1|v2| . . . |vn
]=[u1|u2| . . . |um
]
σ1
σ2
. . .
σn
0
orX = UΣV T ,
where UT U = I, with columns of U called left singular vectorsand V T V = I with right singular vectors as columns of V andΣ = diag(σ1, . . . , σn) called singular values ordered such thatσ1 ≥ σ2 ≥ . . . ≥ σn ≥ 0.
![Page 11: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/11.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Low-rank approximations
Theorem (The rank of a matrix)
The rank of X is r, the number of nonzero singular values in
X = UΣV T = U
σ1
σ2
. . .
σr
0 · · ·
0. . .
V T .
![Page 12: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/12.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Low-rank approximations
Theorem (The rank of a matrix)
The rank of X is r, the number of nonzero singular values in
X = UΣV T = U
σ1
σ2
. . .
σr
0 · · ·
0. . .
V T .
Theorem (Another representation)
X is the sum of r rank-one matrices
X =r∑
j=1
σjujvTj .
![Page 13: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/13.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Low-rank approximations
Theorem
For any ν with 0 ≤ ν ≤ r, define
Xν =ν∑
j=1
σjujvTj ,
Then
‖X − Xν‖2 = infB∈Cm,n, rank(B)≤ν
‖X − B‖2 = σν+1.
![Page 14: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/14.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Low-rank approximations
Proof (m = n)
‖X − Xν‖2 = ‖
r∑
j=ν+1
σjujvTj ‖2 = ‖U
0σν+1
. . .
σn
V T ‖2
= σν+1
Remains to show that there is no closer rank ν matrix to X .
![Page 15: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/15.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Low-rank approximations
Proof (m = n)
‖X − Xν‖2 = ‖
r∑
j=ν+1
σjujvTj ‖2 = ‖U
0σν+1
. . .
σn
V T ‖2
= σν+1
Remains to show that there is no closer rank ν matrix to X .
Let B have rank ν, null space of (B) has dimension n − ν
{v1, . . . , vν+1} has dimension ν + 1
Let h be a unit vector in their intersection:
‖X − B‖2 ≥ ‖(X − B)h‖2 = ‖Xh‖2 = ‖UΣV T h‖2
= ‖Σ(V T h)‖2 ≥ σ2ν+1‖V
T h‖2 ≥ σ2ν+1.
![Page 16: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/16.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Example m = 604, n = 453, m ∗ n = 273612
Compression ratio
c :=(m + n)ν
mn
![Page 17: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/17.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Rank-1 approximation m = 604, n = 453,
m + n = 1057
Compression ratioc := 3.8631e− 03
![Page 18: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/18.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Rank-2 approximation
Compression ratioc := 7.7263e− 03
![Page 19: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/19.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Rank-3 approximation
Compression ratioc := 1.1589e− 02
![Page 20: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/20.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Rank-4 approximation
Compression ratioc := 1.5453e− 02
![Page 21: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/21.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Rank-5 approximation
Compression ratioc := 1.9316e− 02
![Page 22: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/22.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Rank-10 approximation
Compression ratioc := 3.8631e− 02
![Page 23: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/23.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Rank-20 approximation
Compression ratioc := 7.7263e− 02
![Page 24: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/24.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Rank-30 approximation
Compression ratioc := 0.11589
![Page 25: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/25.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Rank-40 approximation
Compression ratioc := 0.15453
![Page 26: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/26.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Rank-60 approximation
Compression ratioc := 0.23179
![Page 27: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/27.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Rank-80 approximation
Compression ratioc := 0.30905
![Page 28: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/28.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Outline
1 Motivation
2 How images become numbers
3 Compressing images
4 The image deblurring problem
5 Blurring and Deblurring imagesThe blurring functionDeblurring
![Page 29: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/29.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Blurred and exact images
Let X be the exact image
Let B be the blurred image
![Page 30: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/30.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Blurred and exact images
Let X be the exact image
Let B be the blurred image
If the blurring of the columns is independent of the blurringin the rows then
AcXATr = B, Ac ∈ R
m,m, Ar ∈ Rn,n
![Page 31: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/31.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Blurred and exact images
Let X be the exact image
Let B be the blurred image
If the blurring of the columns is independent of the blurringin the rows then
AcXATr = B, Ac ∈ R
m,m, Ar ∈ Rn,n
First attempt at deblurring
XNaive = A−1c BA−T
r .
![Page 32: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/32.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
First attempt at deblurring
![Page 33: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/33.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
XNaive = A−1c BA
−Tr
![Page 34: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/34.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
What is the problem?
A noisy blurred image
B = Bexact + E = AcXATr + E
and thereforeXNaive = X + A−1
c EA−Tr .
![Page 35: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/35.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
What is the problem?
A noisy blurred image
B = Bexact + E = AcXATr + E
and thereforeXNaive = X + A−1
c EA−Tr .
Error
The naive solution satisfies
‖XNaive − X‖F
‖X‖F
≤ cond(Ac)cond(Ar)‖E‖F
‖B‖F
.
![Page 36: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/36.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Deblurring using a general model
The blurring process as a linear model
We assume the blurring process is linear, i.e.
x = vec(X) =
x1
...xN
∈ R
N , b = vec(B) =
b1
...bN
∈ R
N
N = m ∗ n are related by the linear model
Ax = b
![Page 37: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/37.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Deblurring using a general model
The blurring process as a linear model
We assume the blurring process is linear, i.e.
x = vec(X) =
x1
...xN
∈ R
N , b = vec(B) =
b1
...bN
∈ R
N
N = m ∗ n are related by the linear model
Ax = b
b = bexact + e
xNaive = A−1b = A−1bexact + A−1e = x + A−1e
![Page 38: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/38.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Separable two dimensional blurs
The Kronecker product
If horizontal and vertical flow can be separated then
Ax = b ⇔ Avec(X) = vec(B) = vec(AcXATr )
(Ar ⊗ Ac)vec(X) = vec(AcXATr ),
where
A = Ar ⊗ Ac =
ar11Ac . . . ar
1nAc
......
...ar
n1Ac . . . arnnAc
,
(UrΣrVTr ) ⊗ (UcΣcV
Tc ) = (Ur ⊗ Uc)(Σr ⊗ Σc)(Vr ⊗ Vc)
T .
![Page 39: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/39.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
xNaive = x + A−1e
![Page 40: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/40.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Outline
1 Motivation
2 How images become numbers
3 Compressing images
4 The image deblurring problem
5 Blurring and Deblurring imagesThe blurring functionDeblurring
![Page 41: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/41.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Taking bad pictures
Sources of bad pictures
defocus the camera lens (limitations in the optical system)
motion blur
air turbulence
atmospheric blurring
![Page 42: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/42.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Taking bad pictures
Sources of bad pictures
defocus the camera lens (limitations in the optical system)
motion blur
air turbulence
atmospheric blurring
Noise E
background photons from both natural or artificial sources
signal represented by finite number of bits (quantisationerror)
![Page 43: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/43.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Modelling the blurring matrix A
Single bright pixel
x = ei ⇒ Aei = column i of A
Figure: Point source(single bright pixel)
Figure: Point spreadfunction (PSF)
![Page 44: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/44.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Modelling the blurring matrix A
Figure: Motion blur Figure: Out-of-focus blur
pij =
{1/(πr)2 if (i − k)2 + (j − l)2 ≤ r2
0 otherwise
![Page 45: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/45.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Modelling the blurring matrix A
Figure: Atmospheric turbulence blur
pij = exp
(
−1
2
[i − kj − l
]T [s21 ρ2
ρ2 s22
]−1 [i − kj − l
]T)
pij =
(
1 +
[i − kj − l
]T [s21 ρ2
ρ2 s22
]−1 [i − kj − l
]T)−β
![Page 46: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/46.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Boundary conditions and structured matrix
computations
Boundary conditions
Zero boundary conditions
Periodic boundary conditions
Reflexive boundary conditions
The matrix A which is obtained from P by convolution becomes
Block Toeplitz matrix
Block Circulant matrix
Sum of Block Toeplitz and Block Hankel and Block Toeplitzplus Hankel matrices
![Page 47: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/47.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Spectral filtering
The SVD again
With
A = UΣV T =[u1|u2| . . . |uN
]
σ1
σ2
. . .
σN
vT1...
vTN
we have
xNaive = A−1b = V Σ−1UT b =
N∑
i=1
uTi b
σi
vi
XNaive =
N∑
i=1
uTi b
σi
Vi =
N∑
i=1
uTi bexact
σi
Vi +
N∑
i=1
uTi e
σi
Vi
![Page 48: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/48.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Spectral filtering
Behaviour of singular values
σi → 0 as i grows
the more “blurry” the function, the faster the decay rate
cond(A) = σ1/σN
![Page 49: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/49.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Spectral filtering
Behaviour of singular values
σi → 0 as i grows
the more “blurry” the function, the faster the decay rate
cond(A) = σ1/σN
The regularised solution
Introduce filter factors Φi
xNaive =N∑
i=1
Φi
uTi b
σi
vi
![Page 50: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/50.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Two methods
TSVD
Φi =
{1 i = 1, . . . , k0 i = k + 1, . . . , N
![Page 51: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/51.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Two methods
TSVD
Φi =
{1 i = 1, . . . , k0 i = k + 1, . . . , N
Tikhonov regularisation
Φi =σ2
i
σ2i + α2
where α > 0 is a regularisation parameter, This choice of filterfactor yields solution to the minimisation problem
minx
{‖b − Ax‖22 + α2‖x‖2
2}.
![Page 52: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/52.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Regularisation and perturbation errors
Regularised solution
xfilt = V ΦΣ−1UT b
= V ΦΣ−1UT Axexact + V ΦΣ−1UT e
= V ΦV T xexact + V ΦΣ−1UT e
xexact − xfilt = (I − V ΦV T )xexact︸ ︷︷ ︸
Regularisation error
− V ΦΣ−1UT e︸ ︷︷ ︸
Perturbation error
![Page 53: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/53.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Regularisation and perturbation errors
Regularised solution
xfilt = V ΦΣ−1UT b
= V ΦΣ−1UT Axexact + V ΦΣ−1UT e
= V ΦV T xexact + V ΦΣ−1UT e
xexact − xfilt = (I − V ΦV T )xexact︸ ︷︷ ︸
Regularisation error
− V ΦΣ−1UT e︸ ︷︷ ︸
Perturbation error
Oversmoothing and undersmoothing
small regularisation error, large perturbation error leads toundersmoothed solution
large regularisation error, small perturbation error leads tooversmoothed solution
![Page 54: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/54.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Regularisation and perturbation errors
Parameter choice methods
Discrepancy Principle
Generalised Cross Validation
L-Curve Criterion
![Page 55: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/55.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
A second attempt at deblurring
![Page 56: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/56.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
XNaive = A−1c BA
−Tr
![Page 57: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/57.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Filtered solution using TSVD
Figure: k = 4801,N = 83000
Figure: k = 6630,N = 238650s
![Page 58: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/58.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
Tikhonov regularisation
Figure: α = 0.0276
Figure: α = 0.0137
![Page 59: Mathematical Beer Goggles or The Mathematics of Image ...mamamf/talks/pss08.pdfThe image deblurring problem Blurring and Deblurring images The blurring function Deblurring Mathematical](https://reader033.vdocuments.site/reader033/viewer/2022042103/5e814e5faaf42d6f9c41ae05/html5/thumbnails/59.jpg)
Deblurring
images
Melina
Freitag
Outline
Motivation
How images
become
numbers
Compressing
images
The image
deblurring
problem
Blurring and
Deblurring
images
The blurringfunction
Deblurring
P. C. Hansen, J. G. Nagy, and D. P. O’Leary,Deblurring Images - Matrices, Spectra and Filtering, SIAM,Philadelphia, 1st ed., 2006.