compressive sensing - examples in image compressionw3.impa.br/~aschulz/cs/lecture4.pdf · (a) dct 0...
TRANSCRIPT
![Page 1: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/1.jpg)
Compressive Sensing
1
Compressive SensingExamples in Image Compression
Lecture 4 , July 30, 2009
Luiz VelhoEduardo A. B. da SilvaAdriana Schulz
![Page 2: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/2.jpg)
Compressive Sensing
2
Today’s Lecture
• Discuss applications of CS in image compression• Evaluate CS efficiency• Review important definitions and theorems
![Page 3: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/3.jpg)
Compressive Sensing
3
Software
• Matlab functions for CS imaging• Optimization algorithms• Data• Demos
http://www.impa.br/∼aschulz/CS/course.html
![Page 4: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/4.jpg)
Compressive Sensing
4
Experimental Setup• 4 images of size 256× 256
(a) Phantom (b) Lena
(c) Camera man (d) Text
![Page 5: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/5.jpg)
Compressive Sensing
5
Frequency Distributions
50 100 150 200 250
50
100
150
200
250
(a) Phantom
50 100 150 200 250
50
100
150
200
250
(b) Lena
50 100 150 200 250
50
100
150
200
250
(c) Camera man
50 100 150 200 250
50
100
150
200
250
(d) Text
![Page 6: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/6.jpg)
Compressive Sensing
6
Acquisition
• Images are stored as a matrix of pixels
![Page 7: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/7.jpg)
Compressive Sensing
7
Noiselets
• RIP• orthogonal• self-adjoint
Φ =12·
1 −1 1 1
−1 1 1 11 1 −1 11 1 1 −1
![Page 8: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/8.jpg)
Compressive Sensing
8
Sparsity
50
100
150
200
250
![Page 9: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/9.jpg)
Compressive Sensing
9
Reconstruction
Acquisition:y = ΦΩx
Reconstruction:y = ΘΩs
s = mins ‖s‖l1 s. t.
y = Θs‖y −Θs‖l2 ≤ ε
x = Ψ∗s
![Page 10: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/10.jpg)
Compressive Sensing
9
Reconstruction
Acquisition:y = ΦΩx
Reconstruction:y = ΘΩs
s = mins ‖s‖l1 s. t.
y = Θs‖y −Θs‖l2 ≤ ε
x = Ψ∗s
![Page 11: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/11.jpg)
Compressive Sensing
9
Reconstruction
Acquisition:y = ΦΩx
Reconstruction:y = ΘΩs
s = mins ‖s‖l1 s. t.
y = Θs‖y −Θs‖l2 ≤ ε
x = Ψ∗s
![Page 12: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/12.jpg)
Compressive Sensing
9
Reconstruction
Acquisition:y = ΦΩx
Reconstruction:y = ΘΩs
s = mins ‖s‖l1 s. t.
y = Θs‖y −Θs‖l2 ≤ ε
x = Ψ∗s
![Page 13: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/13.jpg)
Compressive Sensing
10
Basic CS• Images are sparse in the Ψ domain• No noise is added
Ordinary images are only approximately sparse :
50
100
150
200
250
(a)
50
100
150
200
250
(b)
Figure: Different visualizations of Lena’s DCT.
![Page 14: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/14.jpg)
Compressive Sensing
10
Basic CS• Images are sparse in the Ψ domain• No noise is added
Ordinary images are only approximately sparse :
50
100
150
200
250
(a)
50
100
150
200
250
(b)
Figure: Different visualizations of Lena’s DCT.
![Page 15: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/15.jpg)
Compressive Sensing
11
Forcing SparsityDCT Example:
(a) Original Image (b) Original DCT (c) Original DCT
(d) Result Image (e) Result DCT (f) Result DCT
![Page 16: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/16.jpg)
Compressive Sensing
12
First Experiment
• x Ψ−→ sforce sparsity−−−−−−−→ sS
Ψ∗
−−→ xS
• Acquisition: y = ΦΩxS
• Reconstruction of x via l1 minimization
• Compare x and xS using PSNR (Peak Signal to Noise Ratio)
![Page 17: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/17.jpg)
Compressive Sensing
13
First Result
10 20 30 40 50 6020
30
40
50
60
70
80
Measurements( × 103)
PS
NR
Figure: Basic CS experiment: 10k-sparse Lena.
Theorem
Reconstruction is exact if
M & S · µ2(Φ,Ψ) · log N
![Page 18: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/18.jpg)
Compressive Sensing
14
Varying S
10 20 30 40 50 6020
25
30
35
40
45
50
55
60
65
70
Measurements( × 103)
PS
NR
3.5k−Sparse6k−Sparse10k−Sparse14k−Sparse
![Page 19: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/19.jpg)
Compressive Sensing
15
DCT Linear Compression
We measure M N coefficients on the upper-left corner
![Page 20: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/20.jpg)
Compressive Sensing
15
DCT Linear Compression
We measure M N coefficients on the upper-left corner
![Page 21: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/21.jpg)
Compressive Sensing
15
DCT Linear Compression
We measure M N coefficients on the upper-left corner
![Page 22: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/22.jpg)
Compressive Sensing
15
DCT Linear Compression
We measure M N coefficients on the upper-left corner
![Page 23: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/23.jpg)
Compressive Sensing
16
DCT Linear Compression
Reconstruction: x = Ψ∗s
![Page 24: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/24.jpg)
Compressive Sensing
17
DCT Linear Compression
Problem: we don’t measure all the significant coefficients andmeasure some zero coefficients!
![Page 25: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/25.jpg)
Compressive Sensing
17
DCT Linear Compression
Problem: we don’t measure all the significant coefficients andmeasure some zero coefficients!
![Page 26: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/26.jpg)
Compressive Sensing
18
CS: Linear + Noiselets Measurements
Reconstruction: s = mins ‖s‖l1 s. t. Φ′
ΩΨ∗s = yx = Ψ∗s
![Page 27: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/27.jpg)
Compressive Sensing
19
Results: CS × Linear Compression10 20 30 40 50 60
[DCT−l1−N] [DCT−l
2−L] [DCT−l
1−LN]
10 20 30 40 50 6020
30
40
50
60
70
80
Measurements( × 103)
PS
NR
(a) 3.5K-Sparse
10 20 30 40 50 6020
30
40
50
60
70
80
Measurements( × 103)
PS
NR
(b) 6K-Sparse
10 20 30 40 50 6020
30
40
50
60
70
80
Measurements( × 103)
PS
NR
(c) 10K-Sparse
10 20 30 40 50 6020
30
40
50
60
70
80
Measurements( × 103)
PS
NR
(d) 14K-Sparse
![Page 28: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/28.jpg)
Compressive Sensing
20
Coherence
Before Boundary: best to use linear compressionAfter Boundary: best to use CS
10 20 30 40 50 6020
30
40
50
60
70
80
90
100
110
120
Measurements( × 103)
PS
NR
DCT−l1−N
DCT−l1−LN
![Page 29: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/29.jpg)
Compressive Sensing
21
Different ImagesWhat happens when we consider the other images?
10 20 30 40 50 6020
30
40
50
60
70
80
Measurements( × 103)
PS
NR
(e) Linear Compression
10 20 30 40 50 6020
30
40
50
60
70
80
Measurements( × 103)
PS
NR
(f) Compressive Sensing
(a) Phantom (b) Lena (c) Camera man (d) Text
![Page 30: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/30.jpg)
Compressive Sensing
21
Different ImagesWhat happens when we consider the other images?
10 20 30 40 50 6020
30
40
50
60
70
80
Measurements( × 103)
PS
NR
(e) Linear Compression
10 20 30 40 50 6020
30
40
50
60
70
80
Measurements( × 103)
PS
NR
(f) Compressive Sensing
0 50 100 150 200 250
0
50
100
150
200
250
(a) Phantom
0 50 100 150 200 250
0
50
100
150
200
250
(b) Lena
0 50 100 150 200 250
0
50
100
150
200
250
(c) Camera man
0 50 100 150 200 250
0
50
100
150
200
250
(d) Text
![Page 31: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/31.jpg)
Compressive Sensing
21
Different ImagesWhat happens when we consider the other images?
20 40 60 phantom lena cameraman text
10 20 30 40 50 6020
30
40
50
60
70
80P
SN
R
Measurements( × 103)
(a) Linear Compression
10 20 30 40 50 6010
20
30
40
50
60
70
80
PS
NR
Measurements( × 103)
(b) Compressive Sensing
0 50 100 150 200 250
0
50
100
150
200
250
(a) Phantom
0 50 100 150 200 250
0
50
100
150
200
250
(b) Lena
0 50 100 150 200 250
0
50
100
150
200
250
(c) Camera man
0 50 100 150 200 250
0
50
100
150
200
250
(d) Text
![Page 32: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/32.jpg)
Compressive Sensing
22
Different Sparsity DomainsWhat happens when we consider different sparsity domains?
0 50 100 150 200 250
0
50
100
150
200
250
nz = 10000
(a) DCT
0 50 100 150 200 250
0
50
100
150
200
250
nz = 10000
(b) Block DCT
0 50 100 150 200 250
0
50
100
150
200
250
nz = 10000
(c) Wavelets
10 20 30 40 50 6020
25
30
35
40
45
50
55
60
65
70
PS
NR
Measurements( × 103)
DCT−l1−N
B_DCT−l1−N
DWT−l1−N
![Page 33: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/33.jpg)
Compressive Sensing
22
Different Sparsity DomainsWhat happens when we consider different sparsity domains?
0 50 100 150 200 250
0
50
100
150
200
250
nz = 10000
(a) DCT
0 50 100 150 200 250
0
50
100
150
200
250
nz = 10000
(b) Block DCT
0 50 100 150 200 250
0
50
100
150
200
250
nz = 10000
(c) Wavelets
10 20 30 40 50 6020
25
30
35
40
45
50
55
60
65
70
PS
NR
Measurements( × 103)
DCT−l1−N
B_DCT−l1−N
DWT−l1−N
![Page 34: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/34.jpg)
Compressive Sensing
23
But this is not for real...
All this happens because we FORCE sparsity!
Theorem
Reconstruction is exact if
M & S · µ2(Φ,Ψ) · log N
Important Parameters:• µ - Noiselets are highly incoherent with all considered domains• S - We are tempering with it! - Lets stop and see what happens...
![Page 35: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/35.jpg)
Compressive Sensing
24
Sparsity Errors
When Φ are Noiselet measurements the RIP holds and we can use:
Theorem
Let sS be the best S-sparse approximation of s.If the RIP holds, the solution s to
s = mins‖s‖l1 subject to ΘΩs = y
obeys
‖s − s‖l2 .1√S· ‖s − sS‖l1
![Page 36: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/36.jpg)
Compressive Sensing
25
Let’s verify this theorem empirically
![Page 37: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/37.jpg)
Compressive Sensing
26
Let’s verify this theorem empirically
10 20 30 40 50 6020
22
24
26
28
30
32
34
36
38
40
Measurements( × 103)
PS
NR
3.5k−Sparse
6k−Sparse
10k−Sparse
14k−Sparse
10 20 30 40 50 6020
25
30
35
40
45
50
PS
NR
Measurements( × 103)
10 20 30 40 50 6020
25
30
35
40
45
50
55
60
65
Measurements( × 103)
PS
NR
Test image LenaMeasurements Sparsity is forced Sparsity is not forced
M = 20k S = 3.5k PSNR = 28.8 PSNR = 26.6M = 25k S = 6k PSNR = 30.7 PSNR = 27.8M = 35k S = 10k PSNR = 33.0 PSNR = 30.2M = 40k S = 14k PSNR = 34.9 PSNR = 31.5
![Page 38: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/38.jpg)
Compressive Sensing
26
Let’s verify this theorem empirically
10 20 30 40 50 6020
22
24
26
28
30
32
34
36
38
40
Measurements( × 103)
PS
NR
3.5k−Sparse
6k−Sparse
10k−Sparse
14k−Sparse
10 20 30 40 50 6020
25
30
35
40
45
50
PS
NR
Measurements( × 103)
Test image LenaMeasurements Sparsity is forced Sparsity is not forced
M = 20k S = 3.5k PSNR = 28.8 PSNR = 26.6M = 25k S = 6k PSNR = 30.7 PSNR = 27.8M = 35k S = 10k PSNR = 33.0 PSNR = 30.2M = 40k S = 14k PSNR = 34.9 PSNR = 31.5
![Page 39: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/39.jpg)
Compressive Sensing
27
Different Sparsity Domains
10 20 30 40 50 60 [DCT−l
2−L] [B_DCT−l
2−L] [DCT−l
1−N] [B_DCT−l
1−N] [DWT−l
1−N] [SVD−l
1−N] [TV−l
1−N]
10 20 30 40 50 6010
20
30
40
50
60
70
80
Measurements( × 103)
PS
NR
(b) Phantom
10 20 30 40 50 6010
20
30
40
50
60
70
80
Measurements( × 103)
PS
NR
(c) Lena
10 20 30 40 50 6010
20
30
40
50
60
70
80
Measurements( × 103)
PS
NR
(d) Camera man
10 20 30 40 50 6010
20
30
40
50
60
70
80
Measurements( × 103)
PS
NR
(e) Text
![Page 40: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/40.jpg)
Compressive Sensing
27
Different Sparsity Domains
10 20 30 40 50 60 [DCT−l
2−L] [B_DCT−l
2−L] [DCT−l
1−N] [B_DCT−l
1−N] [DWT−l
1−N] [SVD−l
1−N] [TV−l
1−N]
10 20 30 40 50 6010
20
30
40
50
60
70
80
Measurements( × 103)
PS
NR
(b) Phantom
10 20 30 40 50 6010
20
30
40
50
60
70
80
Measurements( × 103)
PS
NR
(c) Lena
10 20 30 40 50 6010
20
30
40
50
60
70
80
Measurements( × 103)
PS
NR
(d) Camera man
10 20 30 40 50 6010
20
30
40
50
60
70
80
Measurements( × 103)
PS
NR
(e) Text
![Page 41: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/41.jpg)
Compressive Sensing
27
Different Sparsity Domains
10 20 30 40 50 60 [DCT−l
2−L] [B_DCT−l
2−L] [DCT−l
1−N] [B_DCT−l
1−N] [DWT−l
1−N] [SVD−l
1−N] [TV−l
1−N]
10 20 30 40 50 6010
20
30
40
50
60
70
80
Measurements( × 103)
PS
NR
(b) Phantom
10 20 30 40 50 6010
20
30
40
50
60
70
80
Measurements( × 103)
PS
NR
(c) Lena
10 20 30 40 50 6010
20
30
40
50
60
70
80
Measurements( × 103)
PS
NR
(d) Camera man
10 20 30 40 50 6010
20
30
40
50
60
70
80
Measurements( × 103)
PS
NR
(e) Text
![Page 42: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/42.jpg)
Compressive Sensing
27
Different Sparsity Domains
Total Variation (TV)
minx‖x‖TV subject to y = ΦΩx
• l1-norm of the (appropriately discretized) gradient.
![Page 43: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/43.jpg)
Compressive Sensing
27
Different Sparsity Domains
10 20 30 40 50 60 [DCT−l
2−L] [B_DCT−l
2−L] [DCT−l
1−N] [B_DCT−l
1−N] [DWT−l
1−N] [SVD−l
1−N] [TV−l
1−N]
10 20 30 40 50 6010
20
30
40
50
60
70
80
Measurements( × 103)
PS
NR
(b) Phantom
10 20 30 40 50 6010
20
30
40
50
60
70
80
Measurements( × 103)
PS
NR
(c) Lena
10 20 30 40 50 6010
20
30
40
50
60
70
80
Measurements( × 103)
PS
NR
(d) Camera man
10 20 30 40 50 6010
20
30
40
50
60
70
80
Measurements( × 103)
PS
NR
(e) Text
![Page 44: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/44.jpg)
Compressive Sensing
27
Different Sparsity Domains
Singular Value Decomposition (SVD)
• Calculated for each image!• Upper bounds - insights into performance limitations.
![Page 45: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/45.jpg)
Compressive Sensing
27
Different Sparsity Domains
10 20 30 40 50 60 [DCT−l
2−L] [B_DCT−l
2−L] [DCT−l
1−N] [B_DCT−l
1−N] [DWT−l
1−N] [SVD−l
1−N] [TV−l
1−N]
10 20 30 40 50 6010
20
30
40
50
60
70
80
Measurements( × 103)
PS
NR
(b) Phantom
10 20 30 40 50 6010
20
30
40
50
60
70
80
Measurements( × 103)
PS
NR
(c) Lena
10 20 30 40 50 6010
20
30
40
50
60
70
80
Measurements( × 103)
PS
NR
(d) Camera man
10 20 30 40 50 6010
20
30
40
50
60
70
80
Measurements( × 103)
PS
NR
(e) Text
![Page 46: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/46.jpg)
Compressive Sensing
28
Measurement Errors
Measurements: y = ΦΩx + n
• n is a random variable with normal distribution
Theorem
If the RIP holds and ‖n‖l2 ≤ ε, the solution s to
s = mins‖s‖l1 s.t. ‖ΘΩs − y‖l2 ≤ ε
obeys
‖s − s‖l2 .1√S· ‖s − sS‖l1 + ε
![Page 47: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/47.jpg)
Compressive Sensing
29
Gaussian Errors
10 20 30 40 50 6015
20
25
30
35
40
45
50
55
60
65
70
Measurements( × 103)
PS
NR
var = 0.1var = 0.5var = 1var = 2var = 3var = 4var = 5var = 10
![Page 48: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/48.jpg)
Compressive Sensing
30
Compression
Number of Measurements × Number of Bits
• To compress we have to quantize the acquired data!
![Page 49: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/49.jpg)
Compressive Sensing
31
Quantization
−300 −200 −100 0 100 200 3000
5
10
15
−300 −200 −100 0 100 200 3000
200
400
600
800
Each measurement may assume K values.
![Page 50: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/50.jpg)
Compressive Sensing
32
Quantization
• Larger quatization steps: less bits• Smaller quatization steps: less errors
![Page 51: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/51.jpg)
Compressive Sensing
33
Rate
Rate = number of bits per pixel
Original Image:• N pixels• 8 bits/pixel
Acquired Measurements:• M N coefficients• Hy bits/coefficients
Rate =MN
Hy bits/pixel
If Rate < 8 data is being compressed!
![Page 52: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/52.jpg)
Compressive Sensing
34
The Rate × Distortion Criteria
=⇒ which is the best compression scheme?
![Page 53: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/53.jpg)
Compressive Sensing
34
The Rate × Distortion Criteria
=⇒ which is the best compression scheme?
![Page 54: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/54.jpg)
Compressive Sensing
35
Rate × Distortion
step = 0.01step = 0.02step = 0.05step = 0.1step = 0.2step = 0.5
step = 100
step = 1step = 2step = 3step = 4step = 5step = 10step = 20step = 50
10 20 30 40 50 6010
20
30
40
50
60
70
80
90
100
Measurements x 103
PS
NR
![Page 55: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/55.jpg)
Compressive Sensing
35
Rate × Distortion
0 2 4 6 8 10 12 14 16 180
10
20
30
40
50
60
70
80
90
100
step = 0.01step = 0.02step = 0.05step = 0.1step = 0.2step = 0.5step = 1step = 2step = 3step = 4step = 5step = 10step = 20step = 50step = 100
Rate
PS
NR
![Page 56: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/56.jpg)
Compressive Sensing
35
Rate × Distortion
0 2 4 6 8 10 12 14 16 180
10
20
30
40
50
60
70
80
90
100
step = 0.01step = 0.02step = 0.05step = 0.1step = 0.2step = 0.5step = 1step = 2step = 3step = 4step = 5step = 10step = 20step = 50step = 100
Rate
PS
NR
![Page 57: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/57.jpg)
Compressive Sensing
35
Rate × Distortion
step = 0.01step = 0.02step = 0.05step = 0.1step = 0.2step = 0.5step = 1step = 2step = 3step = 4step = 5step = 10step = 20step = 50step = 100
0 2 4 6 8 10 12 14 16 180
10
80
90
100
![Page 58: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/58.jpg)
Compressive Sensing
35
Rate × Distortion
0 2 4 6 8 10 12 14 16 180
10
20
30
40
50
60
70
80
90
100
step = 0.01step = 0.02step = 0.05step = 0.1step = 0.2step = 0.5step = 1step = 2step = 3step = 4step = 5step = 10step = 20step = 50step = 100
Rate
PS
NR
![Page 59: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/59.jpg)
Compressive Sensing
36
Both Sparsity and Measurement Errors
A real example: real image and quantized measurements.
Recovery Strategies:• Ψ = DCT• Ψ = block DCT• Ψ = DWT• TV- minimization• Ψ = SVD - of each image!
![Page 60: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/60.jpg)
Compressive Sensing
37
Both Sparsity and Measurement Errors
0 1 2 3 4 5 6 7 8 Jpeg2000 DCT−l
1−N B_DCT−l
1−N DWT−l
1−N TV−N SVD−l
1−N
0 1 2 3 4 5 6 7 80
10
20
30
40
50
60
70
80
90
100
Rate
PS
NR
(b) Phantom
0 1 2 3 4 5 6 7 80
10
20
30
40
50
60
70
80
90
100
Rate
PS
NR
(c) Lena
0 1 2 3 4 5 6 7 80
10
20
30
40
50
60
70
80
90
100
Rate
PS
NR
(d) Camera man
0 1 2 3 4 5 6 7 80
10
20
30
40
50
60
70
80
90
100
Rate
PS
NR
(e) Text
![Page 61: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/61.jpg)
Compressive Sensing
38
Sparsity × Quantization Errors
‖x − x‖l2 .
εq +1√S‖xS − x‖l1︸ ︷︷ ︸
εs
,
The reconstruction error in CS is of the order of the maximum of thequantization (εq) and sparsity errors (εs).
![Page 62: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/62.jpg)
Compressive Sensing
39
(a) DCT (b) DWT
(c) TV (d) SVD
Figure: Rate × PSNR of image Lena
![Page 63: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/63.jpg)
Compressive Sensing
40
(a) DCT (b) DWT
(c) TV (d) SVD
Figure: Measurements × PSNR of image Lena
![Page 64: Compressive Sensing - Examples in Image Compressionw3.impa.br/~aschulz/CS/Lecture4.pdf · (a) DCT 0 50 100 150 200 250 0 50 100 150 200 250 nz = 10000 (b) Block DCT 0 50 100 150 200](https://reader034.vdocuments.site/reader034/viewer/2022050607/5faec46459090d4cd25b3107/html5/thumbnails/64.jpg)
Compressive Sensing
41
Lets wrap it up!
CS ApplicationsImage acquisition and compression• Interesting, but• Limitations ...
TomorowApplications in graphics and vision• One Pixel Camera• Dual Photography