spin models for image processing alexey abramov abramov _at_ physik3.gwdg.de georg-august...

Spin models for image processing

Alexey Abramovabramov _at_ physik3.gwdg.de

Georg-August University, Bernstein Center for Computational Neuroscience,

III Physikalisches Institut, Göttingen, Germany

Lecture

Alexey Abramov (BCCN, Göttingen) 10-03-11 2/33

It’s a data structure representing a generally rectangular grid of pixels, or points

of color, viewable via a monitor, paper, or other display medium.

Pixel is the smallest piece of

information in an image.

(R,G,B) = (120,70,90)

width

height

In zoomed view every square represents a

pixel.

What is raster graphics image (bitmap)?

Alexey Abramov (BCCN, Göttingen)

Partitioning of an input image into disjoint parts based on some image characteristics.

Image segmentation

input image segments

- homogeneity criteria based methods

- clustering

- region-based growing

- graph cuts

- mean shift segmentation

10-03-11 3/33


The methods of superparamagnetic clustering are nonparametric methods

which solve the segmentation problem by finding the equilibrium states of the

energy function of a ferromagnetic Potts model (without data term).

Superparamagnitec clustering of data

Before update After update

Neighborhood

ε2Di

Superparamagnetic phase regions of aligned spins correspond to a natural

partition of the image data (Blatt et al., 1996).

10-03-11 4/33


A spin variable σk , which can take on q discrete values v1, v2, …, vq , called

spin states, is assigned to each pixel of the image. The energy of the system is

described by

Potts model

where the function Jij is a coupling constant, determining the interaction

strength, where gi and gj are the respective color vectors of the pixels:

where Δ is a normalization constant.

10-03-11 5/33

One iteration of Metropolis algorithm

Current configuration S = 1 S = 2 S = 3 S = 4

Assumption for this example: Jij = 1

1. Ecur = -1

2. 4 → 1 Enew = -5

3. 4 → 2 Enew = 0

4. 4 → 3 Enew = -2

min Emin = -5

5. Enew < Ecur therefore 4 → 1 (move is accepted)Proposed configuration


Final configuration

10-03-11 6/33

Metropolis algorithm (suggested by Metropolis, 1953)


Ising model is a statistical model. The probability of each configuration of spins is

the Boltzmann distribution with inverse temperature β:

To generate configurations using the probability distribution is easiest using

the Metropolis algorithm:

1. pick a spin at random and compute its contribution to the energy Ecur

2. flip the value of the spin and calculate the new contribution Enew

3. if Enew < Ecur then the new state is accepted – a good move

4. if Enew > Ecur then the new state is accepted using the probability – a worse move

5. Repeat.

Simulated Annealing

A certain number of iterations are carried out at each temperature and then

the temperature is decreased. This is repeated until the system freezes into a steady

state.10-03-11 7/33


One iteration of parallel Metropolis algorithm

10-03-11 8/33

Implementation of Metropolis on GPU


1. Generate image tiles taking overlapping into account;

2. Construct CUDA data according to memory structure of GPU;

3. Launch kernel function on the GPU;

4. Decompose CUDA data.

1 2 3 4 5 j j+2j+1 j+3 j+4

1

2

3

4

5

i

i+1

i+2

BLOCK (0.0) BLOCK (1.0)

BLOCK (0.0)

BLOCK (0.1)

10-03-11 9/33

Metropolis algorithm (the cooling schedule)


A certain number of iterations are carried out at each temperature and then

the temperature is decreased. This is repeated until the system freezes into a

steady state.

10-03-11 10/33


Parallel Metropolis algorithm (fast cooling)

Domain fragmentation – large uniform areas are being split into sub-segments

despite the high attractive forces within them.

10-03-11 11/33

Configurations after short annealing


“Cluttered scene” “Filling a cup” “Tsukuba”

10-03-11 12/33


Simulated annealing with short-cut

Resolving domain fragmentation

10-03-11 13/33

Resolving Domain Fragmentation


For the spin configuration with q = 6 after 10 Metropolis iterations:

10-03-11 14/33

Detected object boundaries



10-03-11 15/33

Labeling of connected components



10-03-11 16/33

Fast connected-component labeling


Labeling of connected components in a binary image is one of the most fundamental

operations in pattern recognition: labeling is required whenever a computer needs to

recognize objects (connected components) in a binary image.

By use of the labeling operation, a binary image is transformed into a symbolic image

in which all pixels belonging to a connected component are assigned a unique label.

- Multi-scan algorithms scan an image in the forward and backward raster directions alternately to propagate label equivalences until no label changes.

- Two-scan algorithms complete labeling in two scans: during the first scan, they assign provisional labels to object pixels, and record label equivalences. Label equivalences are resolved during or after the first scan. During the second scan, all equivalent labels are replaced by their representative label.

- Hybrid algorithms. Algorithms that are between muti-scan algorithms and two-scan algorithms.

10-03-11 17/33

Conventional two-raster-scan algorithms


For an N x N binary image, we use b(x,y) to denote the pixel value at (x,y) in the image

where 1 ≤ x,y ≤ N, and V0 for the value of object pixels and Vb for that of background

pixels. We assume that V0 and Vb are larger than the value of any provisional label,

and V0 < Vb.

Mask for eight-connected connectivity

10-03-11 18/33



For an N x N binary image, we use b(x,y) to denote the pixel value at (x,y) in the image

where 1 ≤ x,y ≤ N, and V0 for the value of object pixels and Vb for that of background

pixels. We assume that V0 and Vb are larger than the value of any provisional label,

and V0 < Vb.

Provisional labeling by the first scan

10-03-11 19/33



With the mask for eight-connected connectivity, for an object pixel b(x,y) there are 16

possible cases in the mask.

(2) - (9) and (13) – (16) b(x,y) can take on any existing provisional label in the mask

without the need for consideration of label equivalences.

10-03-11 20/33



The order of processing:

b(x-1,y-1) → b(x,y-1) → b(x+1,y-1) → b(x-1,y)

b(x,y)

After b(x,y-1) is processed, the provisional labels for b(x-1,y-1) and b(x,y-1) should

have been combined in the same equivalent label set and have the same

representative label.

Because all provisional labels in the mask have the same representative label, b(x,y)

can take on any of the labels without consideration of any label equivalence.

10-03-11 21/33

The Karnaugh map


The Karnaugh map (K-map) is a method to simplify boolean algebra expressions. In

a Karnaugh map only one boolean variable changes in between adjacent squares.

Once the table is generated and the output possibilities are transcribed, the data is

arranged into the largest possible groups containing 2n cells and the minterm is

generated through the axiom laws of boolean algebra.

A

B

D

C

10-03-11 23/33

The Karnaugh map for the operation resolve


C1

C2

C3

C4

The operation resolve takes much

time compared to other operations,

it should be avoided as much as

possible.

10-03-11 24/33

The Karnaugh map for the operation resolve


if (C3 ≠ VB)

b(x,y) = C3;

else if (C1 ≠ VB)

b(x,y) = C1;

if (C4 ≠ VB)

resolve(C4,C1);

else if (C2 ≠ VB)

b(x,y) = C2;

if (C4 ≠ VB) resolve(C2,C4);

else if (C4)

b(x,y) = C4;

else

b(x,y) = m, m = m + 1.

PseudocodeAfter the first scan, the provisional labels that

are assigned to a connected component in the

given image will be combined in an equivalent

label set and hold the same representative

label. Then, by the second scan, each

provisional label is replaced by its

representative label. Thus, after the second

scan, all pixels in a connected component will

be assigned a unique label.

10-03-11 25/33

Possible accelerations on a GPU


Representative labels can be assigned to all object pixels simultaneously. Therefore

the second scan can be implemented on the GPU in a very efficient manner.

10-03-11 26/33

Execution time on CPU


CCL – the contour-tracing connected-component labeling;

SAUF – the scan plus array-based union-find algorithm;

RTS – the run-based two-scan algorithm.

10-03-11 27/33

Execution time on CPU


CCL – the contour-tracing connected-component labeling;

SAUF – the scan plus array-based union-find algorithm;

RTS – the run-based two-scan algorithm.

10-03-11 28/33


Metropolis procedure with proposed short-cut

10-03-11 29/33


Comparison with graph-based method

Original image Proposed method Graph-based 7

7 Felzenszwalb, et al.: Efficient graph-

based image segmentation. (2004)

10-03-11 30/33


Timing performance

Processing time Total computation time of all steps

512 x 640 px 153.1 ms | 24.4 x 103 ms

500.0 ms

Image size GPU CPU

graph-based (CPU)

Proposed method

10-03-11 31/33


Bibliography

Abramov et al., “Real-time image segmentation on a GPU” (2010)

von Ferber et al., “Cluster update algorithm and recognition” (2000)

Potts, “Some generalized order-disorder transformations” (1952)

Metropolis et al., “Equation of state calculations by fast computing machines” (1953)

Blatt et al., “Superparamagnetic clustering of data” (1996)

Liefeng He et al., “Fast connected-component labeling” (2009)

Felzenszwalb et al., “Efficient graph-based image segmentation” (2004)

10-03-11 32/33

Thank you for your attention !

QUESTIONS ?

Göttingen, 10.03.2011

spin models for image processing alexey abramov abramov _at_ physik3.gwdg.de georg-august...

Documents