total variation and geometric regularization for inverse problems regularization in statistics...
TRANSCRIPT
Total Variation and Geometric Regularization for Inverse Problems
Regularization in StatisticsSeptember 7-11, 2003
BIRS, Banff, Canada
Tony ChanDepartment of Mathematics, UCLA
Outline
• TV & Geometric Regularization (related concepts)• PDE and Functional/Analytic based• Geometric Regularization via Level Sets Techniques• Applications (this talk):
– Image restoration
– Image segmentation
– Elliptic Inverse problems
– Medical tomography: PET, EIT
Regularization: Analytical vs Statistical
• Analytical: – Controls “smoothness” of continuous functions– Function spaces (e.g. Sobolov, Besov, BV)– Variational models -> PDE algorithms
• Statistical:– Data driven priors– Stochastic/probabilistic frameworks– Variational models -> EM, Monte Carlo
Taking the Best from Each?
• Concepts are fundamentally related: – e.g. Brownian motion Diffusion Equation
• Statistical frameworks advantages: – General models
– Adapt to specific data
• Analytical frameworks advantages:– Direct control on smoothness/discontinuities, geometry
– Fast algorithms when applicable
Total Variation Regularization
dxuuTV ||)(
• Measures “variation” of u, w/o penalizing discontinuities.
• |.| similar to Huber function in robust statistics.
• 1D: If u is monotonic in [a,b], then TV(u) = |u(b) – u(a)|, regardless of whether u is discontinuous or not.
• nD: If u(D) = char fcn of D, then TV(u) = “surface area” of D.
• (Coarea formula)
• Thus TV controls both size of jumps and geometry of boundaries.
• Extensions to vector-valued functions
• Color TV: Blomgren-C 98; Ringach-Sapiro, Kimmel-Sochen
drdsfdxufnR
ru
)(||}{
The Image Restoration ProblemA given Observed image z
Related to True Image u
Through Blur K
And Noise n
Blur+NoiseInitial Blur
Inverse Problem: restore u, given K and statistics for n.
Keeping edges sharp and in the correct location is a key problem !
nuKz
Total Variation Restoration
2||||2
1)()(min zKuuTVuf
u
0n
uGradient flow:
)(||
)( *zKKuKu
uugut
anisotropic diffusion data fidelity
dxuuTV ||)(
* First proposed by Rudin-Osher-Fatemi ’92.
* Allows for edge capturing (discontinuities along curves).
* TVD schemes popular for shock capturing.
Regularization:
Variational Model:
Comparison of different methods for signal denoising & reconstruction
Image Inpainting (Masnou-Morel; Sapiro et al 99) Disocclusion
Graffiti Removal
Unified TV Restoration & Inpainting model
EDE
dxdyuudxdyuuJ ,||2
||][ 20
,0)(||
0
uuu
ue
.0; ,,DzEz
e
(C- J. Shen 2000)
TV Inpaintings: disocclusion
Examples of TV Inpaintings
Where is the Inpainting Region?
TV Zoom-in
Inpaint Region: high-res points that are not low-res pts
Edge Inpainting
edge tube T
No extra data are needed. Just inpaint!
Inpaint region: points away from Edge Tubes
Extensions
• Color (S.H. Kang thesis 02)• “Euler’s Elastica” Inpainting (C-Kang-Shen 01)
– Minimizing TV + Boundary Curvature
• “Mumford-Shah” Inpainting (Esedoglu-Shen 01)– Minimizing boundary + interior smoothness:
S S
SudSu 2
,||(min
Geometric Regularization
• Minimizing surface area of boundaries and/or volume of objects
• Well-studied in differential geometry: curvature-driven flows
• Crucial: representation of surface & volume• Need to allow merging and pinching-off of
surfaces• Powerful technique: level set methodology
(Osher/Sethian 86)
Level Set Representation (S. Osher - J. Sethian ‘87)
Inside C
Outside C
Outside C0
0
0
0C
nn
n
||
,||
Normal
divKCurvaturen
Example: mean curvature motion
* Allows automatic topology changes, cusps, merging and breaking.
• Originally developed for tracking fluid interfaces.
0),(|),( yxyxC
C= boundary of an open domain
Application: “active contour”
Initial Curve Evolutions Detected Objects
objects theof boundaries on the stop tohas curve the
in objectsdetect to curve a evolve
: imagean giving
0
0
uC
u
Basic idea in classical active contours
Curve evolution and deformation (internal forces):
Min Length(C)+Area(inside(C)) Boundary detection: stopping edge-function (external forces)
Example:
0)(lim , ,0
tgggt
puGug
||1
1|)(|
00
Snake model (Kass, Witkin, Terzopoulos 88)
1
0
1
0
2 |)))(((||)('|)(inf dssCIgdssCCFC
1
0
|)))(((||)('|2)(inf dssCIgsCCFC
Geodesic model (Caselles, Kimmel, Sapiro 95)
Limitations - detects only objects with sharp edges defined by gradients
- the curve can pass through the edge
- smoothing may miss edges in presence of noise
- not all can handle automatic change of topology
Examples
A fitting term “without edges”
)(
220
2
)(
10 ||||CoutsideCinside
dxdycudxdycu
where Cuaveragec
Cuaveragec
outside )(
inside )(
02
01
Fit > 0 Fit > 0 Fit > 0 Fit ~ 0
Minimize: (Fitting +Regularization)
Fitting not depending on gradient detects “contours without gradient”
)( )(
220
210
21,,
||||
))((||),,(inf21
Cinside Coutside
Ccc
dxdycudxdycu
CinsideAreaCCccF
An active contour model “without edges”
Fitting + Regularization terms (length, area)
C = boundary of an open and bounded domain
|C| = the length of the boundary-curve C
(C. + Vese 98)
Mumford-Shah Segmentation 89
S S
SudSuuu ))(||(min 22
,0
S=“edges”
MS reg: min boundary + interior smoothness
CV model = p.w. constant MS
Variational Formulations and Level Sets(Following Zhao, Chan, Merriman and Osher ’96)
The Heaviside function
The level set formulation of the active contour model
0),( :),( yxyxC
0 if ,0
0 if ,1)(
H
dxdyHCinsideArea
HC
)())((
|)(|||Length
dxdyHcyxudxdyHcyxu
dxdyHHccF
ccFcc
))(1(|),(|)(|),(|
)(|)(|),,(
),,(inf
220
210
21
21,, 21
))),((1()),((),( 21 yxHcyxHcyxu
The Euler-Lagrange equations
),,(inf 21, ,21
ccFcc
dxdyH
dxdyHu
cdxdyH
dxdyHu
c))(1(
))(1(
)( ,)(
)(
)(0
2
0
1
),,(for Equation yxt
),(),,0(
)()(||
)(
0
220
210
yxyx
cucudivt
2 1 andfor Equations cc
Using smooth approximations for the Heaviside and Delta functions
Advantages
Automatically detects interior contours!
Works very well for concave objects
Robust w.r.t. noise
Detects blurred contours
The initial curve can be placed anywhere!
Allows for automatical change of topolgy
Experimental ResultsC of Evolution ),( Averages 21 cc
A plane in a noisy environment
Europe nightlights
0
0
2
1
0
0
2
1
0
0
2
1
0
0
2
1
0
0
4-phase segmentation2 level set functions
2-phase segmentation1 level set function
}0{
:Curves
}0{}0{
:Curves
21
Multiphase level set representations and partitions allows for triple junctions, with no vacuum and no overlap of phases phases 2 ),...,( 1
nn
Example: two level set functions and four phases
|)(||)(|
))(1))((1(||)())(1(||
))(1)((||)()(||),(
Energy
))(1))((1()())(1(
))(1)(()()(
vectorConstant ),,,(
functionsset level The ),(
21
212
000212
010
212
100212
110),(
21002101
21102111
00011011
21
HH
dxdyHHcudxdyHHcu
dxdyHHcudxdyHHcucFInf
HHcHHc
HHcHHcu
ccccc
c
0,0 ,0,0 ,0,0 ,0,0
:segmentsor phases 4 ),( functionsset level 2
21212121
21
Phase 11 Phase 10 Phase 01 Phase 00
mean(11)=45 mean(10)=159 mean(01)=9 mean(00)=103
An MRI brain image
S S
SudSfuu ))(||(min 22
,
References for PDE & Level Sets in Imaging
* IEEE Tran. Image Proc. 3/98, Special Issue on PDE Imaging
* J. Weickert 98: Anisotropic Diffusion in Image Processing
* G. Sapiro 01: Geometric PDE’s in Image Processing
• Aubert-Kornprost 02: Mathematical Aspects of Imaging Processing
• Osher & Fedkiw 02: “Bible on Level Sets”
• Chan, Shen & Vese Jan 03, Notices of AMS
