probabilistic models for images markov random fields

42
Probabilistic Models for Images Markov Random Fields plications in Image Segmentation and Texture Modeli Ying Nian Wu UCLA Department of Statistics IPAM July 22, 2013

Upload: kovit

Post on 05-Jan-2016

60 views

Category:

Documents


0 download

DESCRIPTION

Probabilistic Models for Images Markov Random Fields Applications in I mage S egmentation and Texture Modeling Ying Nian Wu UCLA Department of Statistics IPAM July 22, 2013. Outline Basic concepts, properties, examples Markov chain Monte Carlo sampling Modeling textures and objects - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Probabilistic Models for Images Markov Random Fields

Probabilistic Models for Images

Markov Random FieldsApplications in Image Segmentation and Texture Modeling

Ying Nian WuUCLA Department of Statistics

IPAM July 22, 2013

Page 2: Probabilistic Models for Images Markov Random Fields

Outline•Basic concepts, properties, examples•Markov chain Monte Carlo sampling•Modeling textures and objects•Application in image segmentation

Page 3: Probabilistic Models for Images Markov Random Fields

Markov Chains

Pr(future|present, past) = Pr(future|present)future past | presentMarkov property: conditional independence limited dependenceMakes modeling and learning possible

Page 4: Probabilistic Models for Images Markov Random Fields

Markov Chains (higher order)

Temporal: a natural orderingSpatial: 2D image, no natural ordering

Page 5: Probabilistic Models for Images Markov Random Fields

Markov Random Fields

all the other pixels

Nearest neighborhood, first order neighborhood

Markov Property

From Slides by S. Seitz - University of Washington

Page 6: Probabilistic Models for Images Markov Random Fields

Markov Random Fields

Second order neighborhood

Page 7: Probabilistic Models for Images Markov Random Fields

Markov Random Fields

Can be generalized to any undirected graphs (nodes, edges)Neighborhood system: each node is connected to its neighbors neighbors are reciprocalMarkov property: each node only depends on its neighbors

Note: the black lines on the left graph are illustrating the 2D grid for the image pixels they are not edges in the graph as the blue lines on the right

Page 8: Probabilistic Models for Images Markov Random Fields

Markov Random Fields

What is

Page 9: Probabilistic Models for Images Markov Random Fields

Cliques for this neighborhood

Hammersley-Clifford Theorem

normalizing constant, partition function

potential functions of cliques

From Slides by S. Seitz - University of Washington

Page 10: Probabilistic Models for Images Markov Random Fields

Cliques for this neighborhood

Hammersley-Clifford Theorem

a clique: a set of pixels, each member is the neighbor of any other member

From Slides by S. Seitz - University of Washington

Gibbs distribution

Page 11: Probabilistic Models for Images Markov Random Fields

Cliques for this neighborhood

Hammersley-Clifford Theorem

a clique: a set of pixels, each member is the neighbor of any other member

……etc, note: the black lines are for illustrating 2D grids, they are not edges in the graph

Gibbs distribution

Page 12: Probabilistic Models for Images Markov Random Fields

Cliques for this neighborhood

Ising model

From Slides by S. Seitz - University of Washington

Page 13: Probabilistic Models for Images Markov Random Fields

Ising model

Challenge: auto logistic regression

pair potential

Page 14: Probabilistic Models for Images Markov Random Fields

Gaussian MRF model

continuous

Challenge: auto regression

pair potential

Page 15: Probabilistic Models for Images Markov Random Fields

Sampling from MRF Models

Markov Chain Monte Carlo (MCMC)• Gibbs sampler (Geman & Geman 84)• Metropolis algorithm (Metropolis et al. 53)• Swedeson & Wang (87)• Hybrid (Hamiltonian) Monte Carlo

Page 16: Probabilistic Models for Images Markov Random Fields
Page 17: Probabilistic Models for Images Markov Random Fields

Gibbs Sampler

Simple one-dimension distribution

Repeat: • Randomly pick a pixel • Sample given the current values of

Page 18: Probabilistic Models for Images Markov Random Fields

Gibbs sampler for Ising model

Challenge: sample from Ising model

Page 19: Probabilistic Models for Images Markov Random Fields

Metropolis Algorithm

Repeat: • Proposal: Perturb I to J by sample from K(I, J) = K(J, I)• If change I to J otherwise change I to J with prob

energy function

Page 20: Probabilistic Models for Images Markov Random Fields

Metropolis for Ising model

Challenge: sample from Ising model

Ising model: proposal --- randomly pick a pixel and flip it

Page 21: Probabilistic Models for Images Markov Random Fields

Modeling Images by MRFIsing model

Exponential family model, log-linear model maximum entropy model

unknown parameters

features (may also need to be learned)

reference distribution

Hidden variables, layers, RBM

Page 22: Probabilistic Models for Images Markov Random Fields

Modeling Images by MRF

Given

How to estimate

• Maximum likelihood • Pseudo-likelihood (Besag 1973) • Contrastive divergence (Hinton)

Page 23: Probabilistic Models for Images Markov Random Fields

Maximum likelihood

Given

Challenge: prove it

Page 24: Probabilistic Models for Images Markov Random Fields

Stochastic Gradient

Given

Generate

Analysis by synthesis

Page 25: Probabilistic Models for Images Markov Random Fields

Texture Modeling

Page 26: Probabilistic Models for Images Markov Random Fields
Page 27: Probabilistic Models for Images Markov Random Fields
Page 28: Probabilistic Models for Images Markov Random Fields
Page 29: Probabilistic Models for Images Markov Random Fields
Page 30: Probabilistic Models for Images Markov Random Fields
Page 31: Probabilistic Models for Images Markov Random Fields
Page 32: Probabilistic Models for Images Markov Random Fields
Page 33: Probabilistic Models for Images Markov Random Fields
Page 34: Probabilistic Models for Images Markov Random Fields
Page 35: Probabilistic Models for Images Markov Random Fields
Page 36: Probabilistic Models for Images Markov Random Fields
Page 37: Probabilistic Models for Images Markov Random Fields
Page 38: Probabilistic Models for Images Markov Random Fields

Modeling image pixel labels as MRF (Ising)

( , )i ix y

( , )i jx x

1

real image

label image

Slides by R. Huang – Rutgers University

MRF for Image Segmentation

Bayesian posterior

Page 39: Probabilistic Models for Images Markov Random Fields

Model joint probability

label

image

label-labelcompatibility

Functionenforcing

Smoothness constraint

neighboringlabel nodes

local Observations

image-labelcompatibility

Functionenforcing

DataConstraint

( , )

1( , ) ( , ) ( , )i j i i

i j i

P x x x yZ

x y

* *

( , )( , ) arg max ( , | )P

xx x y

region labels

image pixels

model param

.

Slides by R. Huang – Rutgers University

Page 40: Probabilistic Models for Images Markov Random Fields

*

1

( , ) ( , )2

2

2

2 2

arg max ( | )

1arg max ( , ) ( | ) ( , ) / ( ) ( , )

1arg max ( , ) ( , ) ( , ) ( , ) ( , )

( , ) ( ; , )

( , ) exp( ( ) / )

[ , , ]

i i

i i

i i i j i i i ji i j i i j

i i i x x

i j i j

x x

P

P P P P PZ

x y x x P x y x xZ

x y G y

x x x x

x

x

x

x x y

x y x y x y y x y

x y

( , )i ix y

( , )i jx xSlides by R. Huang – Rutgers University

MRF for Image Segmentation

Page 41: Probabilistic Models for Images Markov Random Fields

Inference in MRFs

– Classical• Gibbs sampling, simulated annealing • Iterated conditional modes

– State of the Art• Graph cuts• Belief propagation• Linear Programming • Tree-reweighted message passing

Slides by R. Huang – Rutgers University

Page 42: Probabilistic Models for Images Markov Random Fields

Summary•MRF, Gibbs distribution•Gibbs sampler, Metropolis algorithm•Exponential family model