digital image processing chapter 11: image description and representation 12 september 2007 digital...

Digital Image ProcessingChapter 11:

Image Description and Representation

12 September 2007

Digital Image ProcessingChapter 11:

Image Description and Representation

12 September 2007

Image Representation and Description? Image Representation and Description?

Objective:To represent and describe information embedded in

an image in other forms that are more suitable than the image itself.

Benefits:- Easier to understand- Require fewer memory, faster to be processed- More “ready to be used”

What kind of information we can use?- Boundary, shape- Region- Texture- Relation between regions

Shape Representation by Using Chain Codes Shape Representation by Using Chain Codes

Chain codes: represent an object boundary by a connected sequence of straight line segments of specified lengthand direction.

4-directionalchain code


Why we focus on a boundary?The boundary is a good representation of an object shapeand also requires a few memory.

(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Examples of Chain Codes Examples of Chain Codes

Object boundary

(resampling)

Boundaryvertices




The First Difference of a Chain Codes The First Difference of a Chain Codes Problem of a chain code: a chain code sequence depends on a starting point.Solution: treat a chain code as a circular sequence and redefine thestarting point so that the resulting sequence of numbers forms an integer of minimum magnitude.

The first difference of a chain code: counting the number of directionchange (in counterclockwise) between 2 adjacent elements of the code.

Example:

1

2

3

0

Example: - a chain code: 10103322 - The first difference = 3133030 - Treating a chain code as a circular sequence, we get the first difference = 33133030

Chain code : The first difference 0 1 1 0 2 2 0 3 3 2 3 1 2 0 2 2 1 3 The first difference is rotational

invariant.

Polygon Approximation Polygon Approximation

Object boundary Minimum perimeterpolygon

Represent an object boundary by a polygon

Minimum perimeter polygon consists of line segments that minimize distances between boundary pixels.


Polygon Approximation:Splitting Techniques Polygon Approximation:Splitting Techniques

1. Find the line joiningtwo extreme points

0. Object boundary 2. Find the farthest pointsfrom the line

3. Draw a polygon(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Distance-Versus-Angle Signatures Distance-Versus-Angle Signatures

Represent an 2-D object boundary in term of a 1-D function of radial distance with respect to .


Boundary Segments Boundary Segments

Concept: Partitioning an object boundary by using vertices of a convex hull.

Convex hull (gray color)

Object boundary

Partitioned boundary


Input : A set of points on a cornea boundaryOutput: A set of points on a boundary of a convex hull of a cornea

1. Sort the points by x-coordinate to get a sequence p1, p2, … ,pnFor the upper side of a convex hull2. Put the points p1 and p2 in a list Lupper with p1 as the first point3. For i = 3 to n4. Do append pi to Lupper

5. While Lupper contains more than 2 points and the last 3 points in Lupper do not make a right turn

6. Do delete the middle point of the last 3 points from Lupper

TurnRightOK!

TurnRightOK!

TurnLeftNOK!

Convex Hull Algorithm Convex Hull Algorithm

For the lower side of a convex hull7. Put the points pn and pn-1 in a list Llower with pn as the first point8. For i = n-2 down to 19. Do append pi to Llower

10. While Llower contains more than 2 points and the last 3 points in Llower do not make a right turn11. Do delete the middle point of the last 3 points from Llower

12. Remove the first and the last points from Llower

13. Append Llower to Lupper resulting in the list L14. Return L

TurnRightOK!

TurnRightOK!

TurnLeftNOK!

Convex Hull Algorithm (cont.) Convex Hull Algorithm (cont.)

Skeletons Skeletons

Obtained from thinning or skeletonizing processes

Medial axes (dash lines)


Thinning Algorithm Thinning Algorithm

Neighborhoodarrangementfor the thinningalgorithm

Concept: 1. Do not remove end points2. Do not break connectivity3. Do not cause excessive erosion

Apply only to contour pixels: pixels “1” having at least one of its 8neighbor pixels valued “0”

p2p9 p3

p8

p7

p1 p4

p6 p5

Let98321)( pppppN

T(p1) = the number of transition 0-1 in the ordered sequence p2, p3, …

, p8, p9, p2.

Notation:

=Let

00 1

1

1

p1 0

0 1

Example

N(p1) = 4T(p1) = 3

Thinning Algorithm (cont.) Thinning Algorithm (cont.)

Step 1. Mark pixels for deletion if the following conditions are true. a)

b) T(p1) =1c)d)

6)(2 1 pN

0642 ppp0864 ppp

p2p9 p3

p8

p7

p1 p4

p6 p5

Step 3. Mark pixels for deletion if the following conditions are true. a)

b) T(p1) =1c)d)

6)(2 1 pN

0842 ppp0862 ppp

Step 4. Delete marked pixels and repeat Step 1 until no change occurs.

(Apply to all border pixels)

Step 2. Delete marked pixels and go to Step 3.

(Apply to all border pixels)

Example: Skeletons Obtained from the Thinning Alg. Example: Skeletons Obtained from the Thinning Alg.


Skeleton

Boundary Descriptors Boundary Descriptors

1. Simple boundary descriptors: we can use

- Length of the boundary- The size of smallest circle or box that can totally enclosing the object

2. Shape number

3. Fourier descriptor

4. Statistical moments

Shape Number Shape Number

Shape number of the boundary definition: the first difference of smallest magnitude

The order n of the shape number: the number of digits in the sequence

1

2

3

0


Shape Number (cont.) Shape Number (cont.)

Shape numbers of order 4, 6 and 8


Example: Shape Number Example: Shape Number

Chain code: 0 0 0 0 3 0 0 3 2 2 3 2 2 2 1 2 1 1

First difference:3 0 0 0 3 1 0 3 3 0 1 3 0 0 3 1 3 0

Shape No.0 0 0 3 1 0 3 3 0 1 3 0 0 3 1 3 0 3

1. Original boundary

2. Find the smallest rectanglethat fits the shape

3. Create grid4. Find the nearest Grid.


Fourier Descriptor Fourier Descriptor Fourier descriptor: view a coordinate (x,y) as a complex number (x = real part and y = imaginary part) then apply the Fourier transform to a sequence of boundary points.

)()()( kjykxks


1

0

/2)(1

)(K

k

KukeksK

ua Fourier descriptor :

1

0

/2)(1

)(K

k

KukeuaK

ks

Let s(k) be a coordinate of a boundary point k :

Reconstruction formula

Boundarypoints

Example: Fourier Descriptor Example: Fourier Descriptor

Examples of reconstruction from Fourier descriptors

1

0

/2)(1

)(ˆP

k

KukeuaK

ks

P is the number of Fourier coefficients used to reconstruct the boundary


Fourier Descriptor Properties Fourier Descriptor Properties


Some properties of Fourier descriptors

Statistical Moments Statistical Moments


1. Convert a boundary segment into 1D graph2. View a 1D graph as a PDF function3. Compute the nth order moment of the graph

1

0

)()()(K

ii

nin rgmrr

Definition: the nth moment

where

1

0

)(K

iii rgrm

Boundarysegment 1D graph

Example of moment:The first moment = meanThe second moment = variance

Regional Descriptors Regional Descriptors

Purpose: to describe regions or “areas”

1. Some simple regional descriptors- area of the region- length of the boundary (perimeter) of the region- Compactness

)(

)(2 RP

RAC

where A(R) and P(R) = area and perimeter of region R

2. Topological Descriptors3. Texture4. Moments of 2D Functions

Example: a circle is the most compact shape with C = 1/4

Example: Regional Descriptors Example: Regional Descriptors


White pixels represent “light of the cities”

% of white pixelsRegion no. compared to the total white pixels

1 20.4%2 64.0%3 4.9%4 10.7%

Infrared image of America at night

Topological Descriptors Topological Descriptors


Use to describe holes and connected components of the region

Euler number (E):

HCE

C = the number of connected componentsH = the number of holes

Topological Descriptors (cont.) Topological Descriptors (cont.)


E = -1

E = 0 Euler Formula

EHCFQV

V = the number of verticesQ = the number of edgesF = the number of faces

E = -2

Example: Topological Descriptors Example: Topological Descriptors


Original image:Infrared imageOf Washington D.C. area

After intensityThresholding(1591 connectedcomponents with 39 holes)Euler no. = 1552

The largestconnected

area (8479 Pixels)

(Hudson river)

After thinning

Texture Descriptors Texture Descriptors


Purpose: to describe “texture” of the region.

Examples: optical microscope images:

Superconductor(smooth texture)

Cholesterol(coarse texture)

Microprocessor(regular texture)

A

B C


Statistical Approaches for Texture Descriptors Statistical Approaches for Texture Descriptors

We can use statistical moments computed from an image histogram:

1

0

)()()(K

ii

nin zpmzz

where

1

0

)(K

iii zpzm

z = intensityp(z) = PDF or histogram of z

Example: The 2nd moment = variance measure “smoothness” The 3rd moment measure “skewness” The 4th moment measure “uniformity” (flatness)

ABC

Divide into areasby angles

Fourier Approach for Texture Descriptor Fourier Approach for Texture Descriptor

Originalimage

Fouriercoefficient

image

FFT2D+FFTSHIFT

Sum all pixelsin each area

Divide into areasby radius

Sum all pixelsin each area

0

)()( rSrS

0

1

)()(R

rrSS

Concept: convert 2D spectrum into 1D graphs

Fourier Approach for Texture Descriptor Fourier Approach for Texture Descriptor


Originalimage

2D Spectrum(Fourier Tr.)

S(r) S()

Anotherimage

Another S()

Moments of Two-D Functions Moments of Two-D Functions

x y

qppq yxfyxm ),(

00

10

m

mx

00

01

m

my

x y

qppq yxfyyxx ),()()(

0000 m 01001

1011011111 mymmxm

102020 mxm 010202 mym

012

20112121 22 mxmymxm

102

02111212 22 mymxmym

102

203030 23 mxmxm

012

020303 23 mymym

The moment of order p + q

The central moments of order p + q

Invariant Moments of Two-D Functions Invariant Moments of Two-D Functions

00

pqpq where 1

2

qp

The normalized central moments of order p + q

Invariant moments: independent of rotation, translation, scaling, and reflection

02201 211

202202 4

20321

212303 33 2

03212

12304

2

03212

123003210321

20321

21230123012305

33

33

0321123011

20321

2123002206

4

Example: Invariant Moments of Two-D Functions Example: Invariant Moments of Two-D Functions


1. Original image 2. Half size 3. Mirrored

4. Rotated 2 degree 5. Rotated 45 degree


Invariant moments of images in the previous slide

Example: Invariant Moments of Two-D Functions Example: Invariant Moments of Two-D Functions

Invariant moments are independent of rotation, translation,scaling, and reflection

Principal Components for Description Principal Components for Description

Purpose: to reduce dimensionality of a vector image while maintaining information as much as possible.

Let

Mean:

Covariance matrix

K

k

Txx

Tkk

Txxx K

E1

1}))({( mmxxmxmxC

Tnxxx ]...[ 21x

K

kkx K

E1

1}{ xxm

Hotelling transformation Hotelling transformation

1

1

1

......0

............

0...0

0...0

yC

Let )( xmxAy

Then we get

0}{ ym Ey

Txy AACC

and

Then elements of are uncorrelated. The component of y with the largest is called the principal component.

)( xmxAy

Where A is created from eigenvectors of Cx as followsRow 1 contain the 1st eigenvector with the largest eigenvalue.Row 2 contain the 2nd eigenvector with the 2nd largest eigenvalue.

….

Eigenvector and Eigenvalue Eigenvector and Eigenvalue

Eigenvector and eigenvalue of Matrix C are defined as

Let C be a matrix of size NxN and e be a vector of size Nx1.If

eCe

Where is a constant

We call e as an eigenvector and as eigenvalue of C

Example: Principal ComponentsExample: Principal Components


6 spectral imagesfrom an airborneScanner.


Example: Principal Components (cont.)Example: Principal Components (cont.)

Component 1 32102 931.43 118.54 83.885 64.006 13.40

Example: Principal Components (cont.)Example: Principal Components (cont.)

Original image After Hotelling transform

Principal Components for Description Principal Components for Description


Relational Descriptors Relational Descriptors


Structural Approach for Texture Descriptor Structural Approach for Texture Descriptor


digital image processing chapter 11: image description and representation 12 september 2007 digital...

Documents

chain code sequence

chain codes chain codes

d object boundary

directional chain code

boundary pixels

image representation

image description

boundary segments concept