Ch.8Efficient Coding of Visual Scenes byGrouping and Segmentation

Bayesian Brain

Tai Sing Lee and Alan L. Yuille

2008-12-22

Heo, Min-Oh

Contents

Introduction Computational Theories for Scene Segmentation

¨ Weak-membrane model A Computational Algorithm for the Weak-Membrane

Model Generalization of the Weak-Membrane Model

¨ Region competition model¨ Affinity-based model¨ Integration segmentation with shape properties

Biological Evidence¨ Go on to the next speaker…

Introduction

Conjecture ¨ Areas V1 and V2 compute a segmentation

for more compact and parsimonious encoding of images¨ The neural processes are representative of neural mech-

anisms that operate in other areas of the brain for per-forming other higher-level tasks.

Computational Theories for Scene Segmentation Choosing the representation W of the regions

which best fits the image data D based on Mini-mum Description Length (MDL) principle.

Taking logarithm

Encoding cost

Computational Theories for Scene Segmentation Weak-Membrane Model

Data term: Gaussian white noise

Smoothness term: variation on the estimated image intensity is smooth within each region

Penalty term:On the length of the boundaries

d(x,y) : intensity values of images (input image)u(x,y) : unobserved smoothed version of d(x,y)B : set of the boundaries between regionsE(u,B) : encoding cost

Mumford D. and Shah J, Optimal approximations by piecewise smooth functions and associated variational problems. Communications on Pure and Applied Mathematics, 42:577-685, 1989

Computational Theories for Scene Segmentation Reformulation

¨ to deal with boundaries easilyd(x,y) : intensity values of images (input image)u(x,y) : unobserved smoothed version of d(x,y)l(x,y) : line process variables which take on values in [0,1]E(u,l) : encoding cost

Ambrosio L, Tortorelli VM, On the approximations of free discontinuity problems. Preprints di Matermatica, 86, Pisa, Italy: Scuola Normale Superiore, 1990

A Computational Algorithm forthe Weak-Membrane Model

A Computational Algorithm forthe Weak-Membrane Model

Continuation methods¨ At large p, the energy function is con-

vex.¨ As p approaches zero, the energy func-

tion transform back to the original func-tion which can have many local minima.

¨ Strategy (successive gradual relaxation) Initialize p0 with large value, perform steep-

est descent . And decrease p to p1, do it again. Repeat the process. Empirically it yields good results

A Computational Algorithm forthe Weak-Membrane Model The steepest descent equations

¨ The system relaxes to an equilibrium as p decrease from 1 to 0

¨ As p decrease, the boundary responses contract spa-tially to the exact location

Ru: positive rate constant w.r.t uRl : positive rate constant w.r.t l

A Computational Algorithm forthe Weak-Membrane Model

A Computational Algorithm forthe Weak-Membrane Model Segmentation of an image by the weak-membrane model

Generalization of the Weak-Membrane Model Natural Images have …

¨ Texture¨ Shade¨ Color ¨ Shape¨ Material Properties¨ Lighting conditions

How can we segment images with these properties?¨ Region competition model¨ Affinity-based model¨ Integrate segmentation with the estimation of 3D shape properties

Generalization of the Weak-Membrane Model Region competition models

Rr: Region sets for each modelar : model type index variable θr : the parameters of the model

Tu Z, Zhu SC, Image segmentation by data-driven Markov chain Monte Carlo. IEEE Transactions on Pattern Analysis and Machine Intelli-gence, 24(5), 2002

Generalization of the Weak-Membrane Model Regions can be encoded as one of these types

¨ Gaussian model of the intensity in the region¨ Shading model

the image intensity follows a simple parameterized form

¨ Simple texture/clutter model

Generalization of the Weak-Membrane Model

Affinity-based Model¨ Affinity weights wij between different image pixels vi and vj

¨ Define a graph with image pixels and the weights. Assigning a label to each image pixel so that pixels with the same

labels define a region

n: the number of pixels k : the number of labels :

Yu SX, Shi J, Multiclass Spectral Clustering. Proc. of the 9th International Conference on Computer Vision, 313-319, 2003

Generalization of the Weak-Membrane Model Integration segmentation with shape properties

¨ Additional constraint on the surface normal Occlusion border

Ω : a subregion of the imaged(x, y) : the intensity of the image at location (x, y) : reflectance function based on standard Lambertian model is surface gradient at position (x, y) is the light source

Generalization of the Weak-Membrane Model Example: Surface interpolation process

¨ (a) input image¨ (b) initial estimate of

surface by needle map¨ (c) rendering with (b)¨ (d) final estimate of

surface orientations¨ (e) shaded rendering

Biological Evidence

Let me introduce the next speaker!