multi-dimensional image analysis - petra christian university

Center for Image Processing

Multi-dimensional Image Analysis

Lucas J. van Vlietwww.ph.tn.tudelft.nl/~lucas

Center for Image ProcessingCenter for Image Processing

CBP course: Multi-Dimensional Image Analysis 2

Image Analysis Paradigm

Texture filtering

analysis

segmentation

Imagerestoration

Imageenhancement

sensorImageformation scene

pre-processing

classification

Measurements:Point: edge location, isophote curvaturesGlobal: size and shape descriptorsLocal: texture attributes anisotropy, orientation, scale



Multi-dimensional analysis

Goal: Use sampling-error free measurements for analog properties in digitized images

� Point measurements for object identification� Exact boundary location� (Principal) curvatures

� Global measurement for object description� Integrated object intensity (gray-volume)� Size in 2D: area and perimeter� Size in 3D: volume, surface area, length� Shape: Bending energy, Euler numbers

� Local texture analysis� Anisotropy, orientation, scale

Use gray-scale rather than binary operations to obtain high accuracy and precision



Better geometric measurements

� State-of-the-art cameras offer:� Large images of 1000 x 1000 pixels or more� 12 – 16 bit photometric information

� A binary image is disturbed by aliasing� Thresholding corrupts data irreversible

Faithful representation Binary representation



Location of curved edges

� Zero-crossing of second derivatives applied to blurred curved edges are biased.� Laplace: Dxx + Dyy = Dcc + Dgg outwards� in grad direction: Dgg inwards� PLUS 2Dgg + Dcc on edge !

� Results on an ellipse: object with slowly varying curvature.

PLUS

SDGDLaplace

PLUS

x

y

g

cg

cg

c g = gradientc = contour



Isophote curvature in 2D

An isophote is a curve of constant gray-value (level curve)� Curvature is the change of contour direction per unit length� Curvature (κ=1/R) with R the radius of the osculating circle

cc

g

DD

κ

−

=object contour

g =(Dx,Dy)

c =(-Dy,Dx) Usage: corner detection,dominant points, bending energy



Isophote curvature in 3D

� 2D isophote surface patch in 3D image space� Principal curvatures κ1 and κ2 along c1 and c2: 1 2c c g⊥ ⊥

1 11

c c

g

DD

κ

−

=2 2

2c c

g

DD

κ

−

=

1 2 0κ κ> > 2 1 0κ κ< < 1 20, 0κ κ> =

1 20, 0κ κ> < 1 2 0κ κ= =

Usage for local shape: elliptic (convex, concave), cylinder,saddle, flat



Sampling-error free measures

� A sampling-error free measurement is a digital measurement performed on a sampled image that is exactly equal to its analog counterpart

The “sum” of all samples is measured without thresholding and does not introduce a sampling error

( ) ( ) ( ) ( )2

12

sum , 2 0,0

iff :x y x y

sampling Nyquist

b b i j B

f f

π= ∆ ∆ ∆ ∆ =

>

∑

Digital track

=

Analog track

reconstruction

operation in analog space

samplingoperation in digital space



Sum() as measure for …

� Sum() is a sampling-error free measure

� Recipe for object measurements in gray-scale images:1. Transform the input image with the object into an output image

whose sum() is directly proportional to the feature to be measured

2. The transformation must consist of sampling-error free operations

3. Proper sampling is required to avoid aliasing

4. Bias correction terms can be extracted from the mathematical framework! (not empirically)



2D area & 3D volume

� Clip image a to produce a flat object on a flat background� Use “soft-clipping” to avoid aliasing

output after:

thresholding

hard-clipping

erf-clipping( )

( )

erf

2

3

clip

sumD

D

b a

Ab b

V

=

= =∑

linear input slope



2D perimeter, 3D surface area

� Transform the “flat”, bandlimited object into a contour

Analytical dilation / erosion by Taylor series around a smooth edge (with height = 1) over a distance δ =½

( ) ( )

( ) ( )

12

212

212

:

:

:

g gg

g gg

g

dilation b r b r b b

erosion b r b r b b

contour bδ

δ δ δ

δ δ δ

=

+ ≈ + +

− ≈ − + −

≈

a

b

|bg |



Shape: 2D bending energy

� 2D bending energy proportional to the bending energy of a deformed circular rod (Young ’74).

� The simply-connected, closed contour with the minimum bending energy is the circle (2π/R, not scale invariant).

b |bg| κ κ2

Differential geometry Image analysis

( )2 2be be gE s ds E bκ κ= =∑∫�



Shape: 3D bending energy

� Elastic rods (SCC space curves)� κ1 ⇒ cross-section of rod� κ2 ⇒ trajectory of rod

� circle has minimal bending energy

� Torque forces are neglected

� Deflected thin plates (SCC surfaces)� principal curvatures κ1 and κ2

� Poisson’s ratio p [0 ,½]. (let p = 0)� sphere has minimal energy (8π). � Dimensionless and therefore scaling

invariant.

( )22rodE s dsκ= ∫� ( )2 2

1 28plateE p dSπ κ κ= + +∫∫�

22rod gE bκ=∑ ( )2 2

1 2plate gE bκ κ= +∑

Differential geometry

Image analysis



Length in 3D

Resume: An erf-clipped object of unit intensity, bvolume: V = Σ bsurface area: A = Σ |bg|length: L = ??

� image b contains a cylinder (length L,radius R)Σ b = πR2 L = volumeΣ bg = 2πRL = areaΣ bgg = 2πL = length

� Length of spaghetti

� In the plane perpendicular to the string: g,cΣ bgg + bcc = 0 all g • c = 0Σ bgg = –Σ bcc = 2π (see Euler)

13 2D ggL b

π

= ∑

b

bcc

bgg



Shape: Euler numbers

� Euler numbers characterize the topology2D: number of objects – number of holes3D: number of objects – number of handles

(donut & coffee cup have Euler number = 0)

Differential geometry Image analysis

Hopf:

Gauss-Bonnet:

( ) 12 2

11 2 3 1 24

2

4

D g

D g

s ds N b

dS N b

π

π

κ π κ

κ κ π κ κ

= =

= =

∑∫

∑∫∫

�

�

b

bg

κ



Curvilinear structures

10 km 10 cm

1 cm

10 cm

∆z = 10 µm

� Anisotropy� Orientation� Scale� Curvature



Domains vs Scale

� Texture attributes are domain properties



Orientation and Scale

� Rotational invariant chirp image

Orientation mapby Gradient Structure Tensor

Scale mapby Gaussian Scale Space



Gradient Structure Tensor

� How to combine vectors?

ϕ

λ 1

λ 2

= ⋅ =

2

2

x x yt

x y y

f f f

f f fG g g

xf

yf



GST: anisotropy, orientation

� Closed-form solutions for:

12 2

2tan x y

x y

f f

f fϕ

−

= −

1 2

1 2anisotropy: A λ λ

λ λ

−=

+

( ) ( )

( ) ( )

2 22 2 2 211 2

2 22 2 2 212 2

4

4

x y x y x y

x y x y x y

f f f f f f

f f f f f f

λ

λ

= + + − +

= + − − +

ϕ

λ1 λ2

A



PVC particle roughness

� Contour information (shape features) failed to rank batches according to “quality”

� Lobes show up as lines rather than points� Measure the local anisotropy (ellipses)



Roughness = Integrated anisotropy

� Three scales: gradient, window, particle

λ

λ= − =

∑∑

2

11 0.44R

λ

λ= − =

∑∑

2

11 0.61R



Roughness distributions

� (Cumulative) distributions for N=30 particles can be used as roughness measure

� Roughness measure correlates well with product quality

0.39 0.42 0.45 0.48 0.51 0.54 0.57 0.

6

0.63 0.66 0.69 0.72

smoothmiddle

rough

0

2

4

6

8

10

12

coun

t

roughness

particle roughness histograms

smoothmiddlerough

0.39 0.42 0.45 0.48 0.51 0.54 0.57 0.60 0.63 0.66 0.69 0.72

smoothmiddle

rough0%

25%

50%

75%

100%

roughness

cumulative particle roughness histograms



Orientation-driven analysis

� Dominant orientation of Gradient Structure Tensor� Strongest peak in orientation space

Dominant orientationof GST yields the gradient2-weightedorientation map.

Strongest orientationin ϕ-space allows veryabrupt changes inorientation map.



Orientation space

� Decompose the image into narrow orientation bands

� Apply a nonlinear operator in orientation space:� Labeling yields segmentation� Peak selection for detection

ΦΦΦΦ ΦΦΦΦ−−−−1111

ΦΦΦΦ

selectivity



Circle in image-space …

� A circle in 2D image space becomes a double-helix in orientation space.

� Note that orientation-axis is periodic with π



Overlapping circles

� Two overlapping circles in 2D image space compose a single object.

� Since the circles cross at different orientations, they become separated in orientation space.



Orientation selectivity

� Green = first peak, Red = second peak, Blue = remainder

N=8 N=16 N=32

orientation

N=8

N=16



Multi-Scale

Series of images filtered of decreasing scale: Scale-space� Sample the scales logarithmically using filters of size = basescale

yields n scales per octave

Input imagescale 0

scale 1

scale 2

scale 3

scale 4

scale 5

var (scale 1)

var (scale 2)

var (scale 3)

var (scale 4)

var (scale 5)

Local variance between scales n and n-1.

{ }11 13212 ,2 ,2 ,...,2 nbase ∈

Scale difference ≈ scale derivative

Scal

e sp

ace

Scal

e sp

ace



Chirp exampleSc

ale-

spac

e

Scale derivative Spatial variance

Scal

e-sp

ace



Gaussian scale space

� Chirp of varying contrast� Low-pass filters of increasing scale:

color code (fine to course)� Normalization: sum per pixel is constant



Scale analysis

� Scale information reveals geological structures in seismic data

Absolute Normalized



Morphological scale space

� Measure the hole-size distributionThe image acts like a sieve

� Subtract closings of increasing scaleThe hole-size is the scale that closes the gap

2 4 8 16 32 64 128 256

Scale256 x 256



Labeling of holes by size

2 4 8 16 32 64 128 256 2 4 8 16 32 64 128 256



Pore-size distribution

2 4 8 16 32 64 128 256

Milk + substrateMilk + substrate + enzime C

Milk

Milk (blue) + substate (red) + enzyme (green)

Average pore-size distributions imagesN=32



Literature

pdf files available at: www.ph.tn.tudelft.nl/~lucas

� L.J. van Vliet and P.W. Verbeek, Better geometric measurements based on photometric information, Proc. IEEE Instrumentation and Measurement Technology Conf. IMTC94 (Hamamatsu, Japan, May 10-12), 1994, 1357-1360.

� G.M.P. van Kempen et al., The application of a local dimensionality estimator to the analysis of 3D microscopic network structures, in: SCIA'99, Proc. 11th Scandinavian Conference on Image Analysis (Kangerlussuaq, Greenland, June 7-11), 1999, 447-455.

� M. van Ginkel et al., Improved Orientation Selectivity for Orientation Estimation, in: SCIA'97, Proc. 10th Scandinavian Conference on Image Analysis (Lappeenranta, Finland, June 9-11), 1997, 533-537.

multi-dimensional image analysis - petra christian university

Documents