ph.d. thesis presentation: a study of priors and algorithms for signal recovery by convex...
TRANSCRIPT
![Page 1: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/1.jpg)
A Study of Priors and Algorithms for Signal Recovery
by Convex Optimization Techniques
Shunsuke Ono
Yamada Lab.
Dept. Communications and Integrated Systems
Tokyo Institute of Technology
2014/06/12
![Page 2: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/2.jpg)
General Introduction
2
![Page 3: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/3.jpg)
3
Signal Recovery Problem
Signal recovery is a fundamental problem in signal processing
• signal reconstruction
• image restoration
• compressed sensing
• tensor completion...
signal recovery problems = inverse problems of the form:
observation
Goal: estimate from and
noise contamination linear degradation
How to resolve this ill-posed/ill-conditioned problem?
unknown signal
![Page 4: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/4.jpg)
4
Prior and Convex Optimization
Some a priori information on signal of interest, e.g.,• sparsity
• smoothness
• low-rankness
should be taken into consideration.
desired signal
convex function
convex set
1. D. P. Palomar and Y. C. Eldar, Eds., Convex Optimization in Signal Processing and Communications, Cambridge University Press, 2009.
2. J.-L. Starck et al., Sparse Image and Signal Processing: Wavelets, Curvelets,Morphological Diversity. Cambridge University Press, 2010.
3. H. H. Bauschke et al., Eds., Fixed-Point Algorithm for Inverse Problems in Science and Engineering, Springer-Verlag, 2011.
a-priori information convex function =: prior
• l1-norm [Donoho+ ‘03; Candes+ ‘06]
• total variation (TV) [Rudin+ ‘92; Chambolle ‘04]
• nuclear norm [Fazel ‘02; Recht et al. ‘11]
advantage: 1. local optimal = global optimal
2. flexible framework
A powerful approach: convex optimization [see, e.g., 1-3]
![Page 5: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/5.jpg)
5
Optimization Algorithms for Signal Recovery
Optimization algorithms for signal recovery must deal with
• useful priors = nonsmooth convex function
• problem scale = often more than 10^4
proximal splitting methods [e.g., Gabay+ ‘76; Lions+ ‘79; Combettes+ ‘05; Condat ‘13]
• first-order (without Hessian)
• nonsmooth functions
• multiple constraints
Why do we need more?
Useful priorsEfficient
algorithms
![Page 6: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/6.jpg)
Motivation & Goal
6
# prior: signal-specific properties are NOT fully exploited.
=> undesired results, e.g., texture degradation, color artifact, …
# algorithm: CANNOT deal with sophisticated constraints.
=> only the intersection of projectable convex sets.
Goal: design priors and algorithms to resolve them.
![Page 7: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/7.jpg)
7
Structure of The Dissertation
Chap. 1 General Introduction
Chap. 2 Preliminaries
Chap. 3 Image restoration with component-wise
use of priors
Chap. 4 Blockwise low-rank prior for cartoon-texture
image decomposition and restoration
Chap. 5 Priors for color artifact reduction
in image restoration
Chap. 6 A hierarchical convex optimization algorithm
with primal-dual splitting
Chap. 7 An efficient algorithm for signal recovery with
sophisticated data-fidelity constraints
Chap. 8 General conclusion
Main
chapters
priors
algorithms
![Page 8: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/8.jpg)
8
Structure of The Dissertation
Chap. 1 General Introduction
Chap. 2 Preliminaries
Chap. 3 Image restoration with component-wise
use of priors
Chap. 4 Blockwise low-rank prior for cartoon-texture
image decomposition and restoration
Chap. 5 Priors for color artifact reduction
in image restoration
Chap. 6 A hierarchical convex optimization algorithm
with primal-dual splitting
Chap. 7 An efficient algorithm for signal recovery with
sophisticated data-fidelity constraints
Chap. 8 General conclusion
priors
algorithms
Main
chapters
![Page 9: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/9.jpg)
9
Chap. 4 Blockwise low-rank prior for cartoon-texture
image decomposition and restoration
![Page 10: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/10.jpg)
Background
10
![Page 11: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/11.jpg)
Cartoon-Texture Decomposition Model
11
image cartoon texture
Assumption: image = sum of two components
optimization problem [Meyer ‘01; Vese+ ‘03; Aujol+ ’05; ~ Schaeffer+ ‘13 ]
advantage: 1. prior suitable to each component
2. extraction of texture
priors for each component data-fidelity to
![Page 12: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/12.jpg)
Cartoon-Texture Decomposition Model
12
image
optimization problem [Meyer ‘01; Vese+ ‘03; Aujol+ ‘05; ~ Schaeffer+ ‘13 ]
total variation (TV) [Rudin 92]: suitable for cartoon
cartoon texture
![Page 13: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/13.jpg)
Cartoon-Texture Decomposition Model
13
image
optimization problem [Meyer ‘01; Vese+ ‘03; Aujol+ ‘05; ~ Schaeffer+ ‘13 ]
Texture is rather difficult to model…
cartoon texture
![Page 14: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/14.jpg)
Existing Texture Priors
14
G norm frame2001 ~ 2005 ~ 2013
noise
fine pattern
local adaptivity
capability
prior
->
[Schaeffer+ ‘13][Meyer ‘01]
[Aujol+ ‘05]
[Ng+ ‘13]
[Daubechies+ ‘05]
[Elad+ ’05]
[Fadili+ ‘10]
![Page 15: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/15.jpg)
Contribution
15
Propose a prior for a better interpretation of texture.
2001 ~ 2005 ~ 2013
Noise
Fine pattern
Local adaptivity
Proposed
prior
G norm frame[Schaeffer+ ‘13][Meyer ‘01]
[Aujol+ ‘05]
[Ng+ ‘13]
[Daubechies+ ‘05]
[Elad+ ’05]
[Fadili+ ‘10]
![Page 16: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/16.jpg)
SVD, Rank and Nuclear Norm
16
• singular value decomposition (SVD)
nonzero singular values
• number of nonzero singular values
• nuclear norm
tightest convex relaxation of rank [Fazel 02]
* applications to robust PCA (sparse + low-rank)[Gandy & Yamada ‘10; Candes+ ‘11; Gandy, Recht, Yamada ‘11]
rank
nuclear norm
![Page 17: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/17.jpg)
Proposed Method
17
![Page 18: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/18.jpg)
How To Model Texture?
18
globally dissimilar but locally well-patterned
Any block is approximately low-rank after suitable shear.
![Page 19: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/19.jpg)
Any block is approximately low-rank after suitable shear.
Proposed Prior: Block Nuclear Norm (1/2)
19
Definition: pre-Block-nuclear-norm (pre-BNN)
nuclear normpositive weight
Important property of pre-BNN
Pre-BNN is tightest convex relaxation of
weighted blockwise rank
* Generalization of [Fazel ‘02]
![Page 20: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/20.jpg)
Proposed Prior: Block Nuclear Norm (2/2)
20
Definition: Block Nuclear Norm (BNN)
periodic expansion operator (overlap) shear operator
Any block is approximately low-rank after suitable shear.
BNN becomes small,
i.e., good texture prior.
![Page 21: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/21.jpg)
Cartoon-Texture Decomposition Using BNN
21image cartoon texture
• Patterns running in different directions are separately extracted.
• Proximal splitting methods can solve the problem after reformulation.
proposed cartoon-texture decomposition model
texture (K=3) sub-texture 1 sub-texture 2 sub-texture 3
various shear angles
![Page 22: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/22.jpg)
22
Experimental ResultsCASE 1: pure decomposition compared with a state-of-the-art decomposition [Schaeffer & Osher, 2013]
image
cartoon
texture
cartoon
texture
[Schaeffer & Osher 2013] “A low patch-
rank interpretation of texture,” SIAM J.
Imag. Sci. [Schaeffer & Osher 2013] proposed
![Page 23: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/23.jpg)
23
Experimental ResultsCASE 2: blur+20%missing pixelscompared also with [Schaeffer & Osher 2013]
PSNR: 23.20
SSIM: 0.6613
PSNR: 23.75
SSIM: 0.6978
observation [Schaeffer & Osher 2013] proposed
![Page 24: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/24.jpg)
24
Experimental ResultsCASE 2: blur+20%missing pixelsCompared with a state-of-the-art decomposition [Schaeffer & Osher, 2013]
PSNR: 23.20
SSIM: 0.6613
PSNR: 23.75
SSIM: 0.6978
observation [Schaeffer & Osher 2013] proposed
![Page 25: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/25.jpg)
25
Chap. 5 Priors for color artifact reduction
in image restoration
![Page 26: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/26.jpg)
Background
26
![Page 27: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/27.jpg)
Color Artifact in Image Restoration
27
restored by an existing prior
original
observation
color artifact
![Page 28: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/28.jpg)
Color-Line Property
28color-line
restored by
an existing priororiginal
corrupted
# color-line: RGB entries are linearly distributed in local regions.[Omer & Werman ‘04]
![Page 29: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/29.jpg)
Contribution
29color-line
original
corrupted
Propose a prior for promoting color-line property.
reconstructed
existing +
proposed prior
restored by
an existing prior
![Page 30: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/30.jpg)
Proposed Method
30
![Page 31: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/31.jpg)
Mathematical Modeling of Color-Line
31
color image
B G R
-th local region
(e.g., block)
Vectorize
matrix for
-th local region
Define matrices for every local region of a color image.
![Page 32: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/32.jpg)
color-line property low-rankness of
Proposed Prior: Local Color Nuclear Norm
32
number of local regions
key principle
rank( ) = 1exact cases
Local Color Nuclear Norm (LCNN)
![Page 33: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/33.jpg)
Proposed Prior: Local Color Nuclear Norm
33
key principle
color-line property small singular values of
practical casesrank( ) ≠ 1
but is small
Suppressing LCNN promotes the color-line property.
Local Color Nuclear Norm (LCNN)
![Page 34: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/34.jpg)
Application to Denoising
34
color-line
Proximal splitting methods are applicable after reformulation.
smoothness [VTV]
dynamic range
data-fidelity robust to
Impulsive noise
: color image contaminated by impulsive noise
optimization problem
[VTV] Bresson et al. “Fast dual minimization of the vectorial total variation norm and
applications to color image processing”, Inverse Probl. Img., 2008.
![Page 35: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/35.jpg)
Experimental Results
35
observation VTV VTV+LCNN original
22.95, 7.59 24.30, 5.80
25.12, 3.07 27.22, 2.58
(PSNR, D2000) (PSNR, D2000)
![Page 36: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/36.jpg)
36
Chap. 6 A hierarchical convex optimization algorithm
with primal-dual splitting
![Page 37: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/37.jpg)
Background
37
![Page 38: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/38.jpg)
NOT uniqueUnique
NOT strictly convexStrictly convex
.
38
contains infinitely many solutions non-strict convexity of .
Solutions of Convex Optimization Problems
Solution set of a convex optimization problem
Solutions could be considerably different in another criterion.
![Page 39: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/39.jpg)
39
Hierarchical Convex Optimization
ideal strategy: hierarchical convex optimization:
highly involved (≠the intersection of projectable convex sets)
proximal splitting methods cannot solve the problem.
selector: smooth convex function
via fixed point set characterization[e.g., Yamada ‘01; Ogura & Yamada‘03; Yamada, Yukawa, Yamagishi ‘11]
Definition: nonexpansive mapping
computable nonexpansive mapping on a certain Hilbert space
![Page 40: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/40.jpg)
40
Hierarchical Convex Optimization
fixed point set characterized problem
Hybrid Steepest Descent Method (HSDM) [e.g., Yamada ‘01; Ogura & Yamada ‘03]
nonexpansive mapping gradient of selector
Q. What kinds of are available?
![Page 41: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/41.jpg)
41
• Forward-Backward Splitting (FBS) method [Passty ’79; Combettes+ ‘05]
• Douglas-Rachford Splitting (DRS) method [Lions+ ‘79; Combettes+ ‘07]
Two characterizations underlying proximal splitting methods
are given in [Yamada, Yukawa, Yamagishi ‘11].
Q. Can we deal with a more flexible formulation?
Nonexpansive Mappings for
Definition: proximity operator [Moreau ‘62]
![Page 42: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/42.jpg)
42
• Forward-Backward Splitting (FBS) method [Passty ’79; Combettes+ ‘05]
• Douglas-Rachford Splitting (DRS) method [Lions+ ’79; Combettes+ ‘07]
• Primal-Dual Splitting (PDS) method [Condat ‘13; Vu ‘13]
Nonexpansive Mappings for
Two characterizations underlying proximal splitting methods
are given in [Yamada, Yukawa, Yamagishi ‘11].
![Page 43: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/43.jpg)
43
• Primal-Dual Splitting (PDS) method [Condat ‘13; Vu ‘13]
Contribution
• reveal convergence properties
• modify gradient computation
• extract operator-theoretic idea from [Condat 13]
• reformulate in a certain product space
incorporate
hierarchical convex optimization by HSDM
![Page 44: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/44.jpg)
Proposed Method
44
![Page 45: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/45.jpg)
45
Outline
Reformulate in the canonical product space with dual problem
Extract & incorporate fixed point set characterization from [Condat ‘13]
Install another inner product for nonexpansivity of by [Condat ‘13]
Apply HSDM with modified gradient computation w.r.t.
![Page 46: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/46.jpg)
46
Reformulation in The Canonical Product Space
solution set of the first stage problem (=primal problem)
solution set of the dual problem of the first stage problem
By letting
Note:
![Page 47: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/47.jpg)
47
Incorporation of PDS Characterization
Extract the PDS fixed point characterization from [Condat ‘13]
![Page 48: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/48.jpg)
48
Activation of Nonexpansivity
is nonexpansive NOT on the canonical product space
Definition: canonical inner product of
BUT on the following space with another inner product [Condat ‘13]
where
: strongly positive bounded linear operator
![Page 49: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/49.jpg)
49
Solver via HSDM
NOTE:
We can apply HSDM [e.g., Yamada ‘01; Ogura & Yamada ‘03]
![Page 50: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/50.jpg)
50
Convergence of HSDM with PDS
Assumptions:
Convergence 1:
Convergence 2:
Recall
Definition: distance function
![Page 51: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/51.jpg)
51
Application to Signal Recoveryunknown signal
Gaussian noisedegradation
observation model:
first stage problem:
priornumerical rangedata-fidelity
hierarchical convex optimization problem:
non-strictly convex
another prior
to specify
a better solution
Definition: indicator function
![Page 52: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/52.jpg)
52
Application to Signal Recoveryunknown signal
Gaussian noisedegradation
observation model:
first stage problem:
hierarchical convex optimization problem:
non-strictly convex
another prior
to specify
a better solution
Definition: indicator function
![Page 53: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/53.jpg)
53
Experimental Results
original
observed
no-
hierarchical
proposed
![Page 54: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/54.jpg)
54
General Conclusion
Chap. 3 Image restoration with component-wise
use of priors
Chap. 4 Blockwise low-rank prior for cartoon-texture
image decomposition and restoration
Chap. 5 Priors for color artifact reduction
in image restoration
Chap. 6 A hierarchical convex optimization algorithm
with primal-dual splitting
Chap. 7 An efficient algorithm for signal recovery with
sophisticated data-fidelity constraints
priors: to model signal-specific properties
algorithms: to deal with involved constraints
We have developed novel priors and algorithms for signal recovery.
![Page 55: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/55.jpg)
Related Publications
55
# Journal Papers[J1] S. Ono, T. Miyata, I. Yamada, and K. Yamaoka, "Image Recovery by
Decomposition with Component-Wise Regularization,"
IEICE Trans. Fundamentals, vol. E95-A, no. 12, pp. 2470-2478, 2012.
(Best Paper Award from IEICE)
[J2] S. Ono, T. Miyata, and I. Yamada, "Cartoon-Texture Image Decomposition
Using Blockwise Low-Rank Texture Characterization,"
IEEE Trans. Image Process., vol. 23, no. 3, pp. 1028-1042, 2014.
[J3] S. Ono and I. Yamada, "Hierarchical Convex Optimization with Primal-Dual
Splitting,“ submitted to IEEE Trans. Signal Process (accepted conditionally
in May. 2014).
[J4] S. Ono and I. Yamada, "Signal Recovery Using Complicated Data-Fidelity
Constraints,“ in preparation.
![Page 56: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/56.jpg)
Related Publications
56
# Articles in Proceedings of International Conferences (reviewed)[C1] S. Ono, T. Miyata, and K. Yamaoka, "Total Variation-Wavelet-Curvelet
Regularized Optimization for Image Restoration," IEEE ICIP 2011.
[C2] S. Ono, T. Miyata, I. Yamada, and K. Yamaoka, "Missing Region Recovery by
Promoting Blockwise Low-Rankness," IEEE ICASSP 2012.
[C3] S. Ono and I. Yamada, "A Hierarchical Convex Optimization Approach for High
Fidelity Solution Selection in Image Recovery,'' APSIPA ASC 2012, (Invited).
[C4] S. Ono and I. Yamada, "Poisson Image Restoration with Likelihood Constraint
via Hybrid Steepest Descent Method," IEEE ICASSP 2013.
[C5] S. Ono, M. Yamagishi, and I. Yamada, "A Sparse System Identification by Using
Adaptively-Weighted Total Variation via A Primal-Dual Splitting Approach,"
IEEE ICASSP 2013.
[C6] S. Ono and I. Yamada, "A Convex Regularizer for Reducing Color Artifact in
Color Image Recovery,“ IEEE Conf. CVPR 2013.
[C7] I. Yamada and S. Ono, "Signal Recovery by Minimizing The Moreau Envelope
over The Fixed Point Set of Nonexpansive Mappings," EUSIPCO 2013, (invited).
[C8] S. Ono and I. Yamada, “Second-Order Total Generalized Variation Constraint,”
IEEE ICASSP 2014.
[C9] S. Ono and I. Yamada, “Decorrelated Vectorial Total Variation,” IEEE Conf. CVPR
2014 (to appear).
![Page 57: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/57.jpg)
Other Publications
57
# Journal Papers[J5] S. Ono, T. Miyata, and Y. Sakai, "Improvement of Colorization Based Coding by
Using Redundancy of The Color Assignment Information and Correct Color
Component," IEICE Trans. Information and Systems, vol. J93-D, no. 9, pp.
1638-1641, 2010 (in Japanese).
[J6] H. Kuroda, S. Ono, M. Yamagishi, and I. Yamada, "Exploiting Group Sparsity in
Nonlinear Acoustic Echo Cancellation by Adaptive Proximal Forward-Backward
Splitting," IEICE Trans. Fundamentals, vol.E96-A, no.10, pp.1918-1927, 2013.
[J7] T. Baba, R. Matsuoka, S. Ono, K. Shirai, and M. Okuda, "Image Composition Using
A Pair of Flash/No-Flash Images by Convex Optimization,“ IEICE Transactions on
Information and System, 2014 (in Japanese, to appear)
![Page 58: Ph.D. Thesis Presentation: A Study of Priors and Algorithms for Signal Recovery by Convex Optimization Techniques](https://reader034.vdocuments.site/reader034/viewer/2022042723/5875da181a28ab8f438b729f/html5/thumbnails/58.jpg)
Other Publications
58
# Articles in Proceedings of International Conference (reviewed)[C10] S. Ono, T. Miyata, and Y. Sakai, "Colorization-Based Coding by Focusing on
Characteristics of Colorization Bases," PCS 2010.
[C11] M. Yamagishi, S. Ono, and I. Yamada, "Two Variants of Alternating Direction
Method of Multipliers without Inner Iterations and Their Application to Image
Super-Resolution,'' IEEE ICASSP 2012.
[C12] S. Ono and I. Yamada, "Optimized JPEG Image Decompression with Super-
Resolution Interpolation Using Multi-Order Total Variation," IEEE ICIP 2013
(top 10% of all accepted papers).
[C13] K. Toyokawa, S. Ono, M. Yamagishi, and I. Yamada, "Detecting Edges of
Reflections from a Single Image via Convex Optimization,“ IEEE ICASSP 2014.
[C14] T. Baba, R. Matsuoka, S. Ono, K. Shirai, and M. Okuda, "Flash/No-flash Image
Integration Using Convex Optimization,“ IEEE ICASSP 2014.
* Many other articles in proceedings of domestic conferences