poster caip 2013 - inpainting
DESCRIPTION
Patch-based (or “pattern-based”) inpainting, is a popular processing technique aiming at reconstructing missing regions in images, by iteratively duplicating blocks of known image data (patches) inside the area to fill in. This kind of method is particularly effective to process wide image areas, thanks to its ability to reconstruct textured data. Nevertheless, “pathological” geometric configurations often happen, leading to visible reconstruction artefacts on inpainted images, whatever the chosen pattern-based inpainting algorithm. In this paper we focus on these problematic cases and propose a generic spatial-blending technique that can be adapted to any type of patch-based inpainting methods in order to reduce theses problematic artefacts.TRANSCRIPT
Spatial Patch Blending Algorithm for Artefact Reduction inPatch-based Inpainting Methods.
Maxime Daisy, David Tschumperle and Olivier LezorayGREYC – CNRS UMR 6072, ENSICAEN, and University of Caen6 Bd Marchal Juin, F-14050 Caen cedex 4, France
Patch-based inpainting: entry-level algorithm
step step step
initialization
priority computation
C D
P
C
C'
Ii = 0
i = 1 i = 1 i = 11 2 3
=(a) (b)
(c)
δΩ
δΩ'(d)
DC
patch reconstruction confidence update
Problems and solution
Problems| Always pathological cases of reconstruction artefacts (cf. Fig. 1(b))| Appearance of seams between reconstruction patches
Solution
1. Artefact detection ⇒ locations of locally bad reconstructions
2. Spatial patch blending ⇒ reconstruction patches seams reduction
1. Artefact detection
a. Reconstruction artefact locations ?
i) Existence of sharp variations in I ⇒ high ‖∇I‖ii) Reconstruction patch locations U locally very different ⇒ high div(U)
b. Break field R(p): strength of artefacts, combination of hypothesis i) and ii)
∀p ∈ Ω, R(p) = ‖∇I(p)‖ · | div(U)(p) |α
where α is a normalization factor.
c. Blending amplitude map
∀p ∈ Γ, σ(p) = ρ×∑r∈E
wb(p,r)
maxq∈Γ
∑r∈E
wb(q,r)with wb(p, q) a Gaussian function
(a) Patch-based inpainting result. (b) Detected break points. (c) Detected artefacts areas.
2. Spatial Patch Blending
• Principle: remove seams between reconstruction patches
(d) Masked image. (e) One patch later. (f) Three patches
later.
(g) Inpainting result. (h) blending result.
• Equation: compose an image J with a set Ψp of reconstruction patches ψqcentered at each q, from a neighbourhood of p
J i(p) =
∑ψq∈Ψp
w(q,p) ψiq(p−q)
ε+∑
ψq∈Ψp
w(q,p)
with w(q, p) a gaussain weight based on a distance from q to p.
Comparison with a synthetic case
(a) Masked color image. (b) Criminisi inpainting result.
(c) Diffusion PDE inpainting result. (d) Criminisi + our spatial patch blending result.
Results and comparison with state-of-the-art methods
Our method is already embedded inside a G’MIC plugin for GIMP:http://gmic.sourceforge.net/gimp.shtml
Groupe de Recherche en Informatique, Image, Automatique et Instrumentation of Caen – France http://www.greyc.fr/