Background Estimation

Mehdi Ghayoumi, MD Iftakharul Islam, Muslem Al-SaidiDepartment of Computer Science

Kent State University,Kent, OH 44242.

Objective• Fill in the area of an image based on existing background• User selects an area, which is then filled based on surrounding

pixels• Smooth transitions

Introduction

• Object Removal

– Remove object(s) from image

– Fill the hole with information extracted from the surrounding area.

Filled region should look “realistic” to the human eyes

Example

Source Image Target Final Image

Greedy Approach• A Greedy Patch-based Image Inpainting Framework

Diffusion-based Approach

The idea is to track perfectly the local geometry of the damaged

image and allowing diffusion only in the isophotes curves

direction.

Exemplar Based Approach

Idea

1. Sample color values of the surrounding area

2. Generate textures with sampling result to fill the hole

Criminisi’s Algorithm• Assign each pixel with a priority value• Give linear structures higher priorities

Criminisi’s Algorithm

P(p) = C(p)D(p)

Confidence term

Data term

p

Iq pqC

pC

)(

)()(

pp nI

pD

)(

1. Compute the filling priority

Criminisi’s Algorithm

• (a) The confidence term assigns high filling priority to out-pointing appendices (in green) and low priority

to in-pointing ones (in red), thus trying to achieve a smooth and roughly circular target boundary. (b) The

data term gives high priority to pixels on the continuation of image structures (in green) and has the effect

of favoring in-pointing appendices in the direction of incoming structures.

Effects of data and confidence terms

Criminisi’s Algorithm

2. Search for the best matching patch

Criminisi’s Algorithm

In this step, the algorithm fills the region corresponding to Ψp∩Ω by

replicating the corresponding region in the best matching patch Ψ ^q to the

target patch Ψp. Besides, the boundary of the target region δΩ has to be

renewed.

3. Copy the best matching patch information and refresh the

boundary of target region

Criminisi’s Algorithm(cont.)• Structure Propagation by exemplar-based texture synthesis

Criminisi’s Algorithm(cont.)

Improved Criminisi’s Algorithm(cont.)

Expected Results

Input Output

Future Work

• Implementing Algorithms in JAVA• Make and install its Plugin in Imagej

Future Work

• More accurate propagation of curve structures• Solve the problems

References

• A. Criminisi, P. Perez, K. Toyama. Region filling and object removal by exemplar-based Inpainting, IEEE Transactions on Image Processing,2004.

• Christine Guillemot and Olivier Le Meur ,Image Inpainting, Signal Processing Magazin,IEEE,2014.

• Jing Wang and et all, Robust object removal with an exemplar-based image inpainting approach ,Neurocomputing, IEEE,2014.

Thanks!