digital image processing - scut.edu.cn · 2019-04-18 · 4/18/2019 digital image processing 39 ....

Xiangyu Yu

School of Electronic and Information Engineering,

South China University of Technology, P. R. China

[email protected]

DIGITAL IMAGE PROCESSING

LECTURE 10 REPRESENTATION & DESCRIPTION

REPRESENTATION & DESCRIPTION

After segmentation, we get pixels along the boundary or pixels contained in a region.

representing a segmented in 2 ways

in terms of its external characteristics (its boundary) focus on shape characteristics

in terms of its internal characteristics (pixels comprising the region) focus on regional properties, e.g., color, texture

sometimes, we may need to use both ways

G3E-P.817 4/18/2019 DIGITAL IMAGE PROCESSING 3

OVERVIEW

Description describes the region based on the chosen representation

e.g. representation boundary

description length of the boundary, orientation of the straight line joining its extreme points, and the number of concavities in the boundary.

Descriptors should be insensitive to changes in size, translation, rotation, …


SENSITIVITY

feature selected as descriptors should be as insensitive as possible to variations in size

translation

rotation

following descriptors satisfy one or more of these properties.


Image regions (including segments) can be represented by either the border or the pixels of the region. These can be viewed as external or internal characteristics, respectively.

G3E-P.818

REPRESENTATION

4/18/2019 DIGITAL IMAGE PROCESSING 6

REPRESENTATION

The segmentation techniques yield raw data in the form of pixels along a boundary or pixels contained in a region

these data sometimes are used directly to obtain descriptors

It is standard practice to use schemes that compact the segmented data into representations that facilitate the computation of descriptors.

standard uses techniques to compute more useful data (descriptors) from the raw data in order to decrease the size of data.

G3E-P.818


REPRESENTATION SCHEMES

Segmentation Raw data (pixels)

Representations

Computation of descriptors


Boundary Following: We assume (1) that we are working with binary images in which object and background points are labeled 1 and 0, respectively

(2) the images are padded with a border of 0s to eliminate the possibility of an object merging with the image border.

G3E-P.818 4/18/2019 DIGITAL IMAGE PROCESSING 9 G3C-P.514

Given a binary region R or its boundary, an algorithm for following the border of R, or the given boundary, consists of the following steps:

1. Let the starting point, b0, be the uppermost, leftmost point in the image that is labeled 1. Denote by c0 the west neighbor of b0. Clearley, c0 always is a background point. Examine the 8-neighbors of b0, starting at c0 and proceeding in clockwise direction. Let b1 denote the first neighbor encountered whose value is 1, and let c1 be the (background) point immediately preceding b1 in the sequence. Store the location of b0 and b1 for use in Step 5.

G3E-P.818 4/18/2019 DIGITAL IMAGE PROCESSING 10 G3C-P.514-515

2. Let b=b1 and c=c1

3. Let the 8-neighbors of b, starting at c and proceeding in a clockwise direction, be denoted by n1,n2,...,n8. Find the first nk labeled 1.

4. Let b=nk and c=nk-1

5. Repeat Step 3 and 4 until b=b0 and the boundary point found is b1. The sequence of b points found when the algorithm stops constitutes the set of ordered boundary points.

G3E-P.818-819 4/18/2019 DIGITAL IMAGE PROCESSING 11 G3C-P.515

The boundary-following algorithm works equally well if a region, rather than its boundary, is given in Fig.11.2

That is, procedure extracts the outer boundary of a binary region. If the objective is to find the boundaries of holes in a region(these are called the inner boundaries of the region), a simple approach is to extract the holes, a simple approach is to extract and treat them as 1-valued region on a background of 0s.


15

REPRESENTATION SCHEMES

Common external representation methods are: Chain code

Polygonal approximation

Signature

Boundary segments

Skeleton (medial axis)

4/18/2019 DIGITAL IMAGE PROCESSING

CHAIN CODES

Represent a boundary by a connected sequence of straight-line segments of specified length and direction.

Directions are coded using the numbering scheme

Based on 4- or 8-connectivity

Direction of each segment coded by a numbering scheme

G3E-P.820

4-connectivity 8 connectivity 4/18/2019 DIGITAL IMAGE PROCESSING 16

CHAIN CODES

Problems: the resulting chain of codes tends to be quite long

any small disturbances along the boundary due to noise or imperfect segmentation cause changes in the code that may not be related to the shape of the boundary(Easily disturbed by noise, and sidetracked)

Solution: Resampling using larger grid spacing

Normalizations


DIGITAL IMAGE PROCESSING 4/18/2019 18

Figure Connectivity in chain codes.

N3E-P.347 Images

take

n fr

om

Nix

on

& A

gua

do, Featu

re E

xtr

act

ion

& Im

age P

roce

ssin

g f

or

Com

put

er

Visio

n(3e)

(2012)

REPRESENTATION CHAIN CODE

4-direction

8-direction

19 4/18/2019 DIGITAL IMAGE PROCESSING

G3E-P.820-821 4/18/2019 DIGITAL IMAGE PROCESSING 20

CHAIN CODES

21

RESAMPLING FOR CHAIN CODES

0033…01

0766…12

Spacing of the sampling

grid affect the accuracy of

the representation

(4-directional)

(8-directional)



Figure Chain codes by different connectivity.

N3E-P.348 Images

take

n fr

om

Nix

on

& A

gua

do, Featu

re E

xtr

act

ion

& Im

age P

roce

ssin

g f

or

Com

put

er

Visio

n(3e)

(2012)


Figure Start point invariance in chain codes.

N3E-P.348 Images

take

n fr

om

Nix

on

& A

gua

do, Featu

re E

xtr

act

ion

& Im

age P

roce

ssin

g f

or

Com

put

er

Visio

n(3e)

(2012)

The chain code of boundary depends on the starting point. However, the code can be normalized with respect to starting point by a straightforward procedure.

We simply treat the chain code as a circular sequence of direction numbers and redefine the starting point so that the resulting sequence of numbers forms an integer of minimum magnitude.

We can normalize also for rotation (in angles that are integer multiples of the directions in Fig. 11.3) by using the first difference of the chain code instead of the code itself.

G3E-P.820-821 4/18/2019 DIGITAL IMAGE PROCESSING 24 G3C-P.516

NORMALIZATION FOR CHAIN CODES

With respect to starting point:

Make the chain code a circular sequence

Redefine the starting point which gives an integer of minimum magnitude

E.g. 101003333222 normalized to 003333222101

For rotation:

Using first difference (FD) obtained by counting the number of direction changes in counterclockwise direction

E.g FD of a 4-direction chain code 10103322 is 3133030

For scaling:

Altering the size of the resampling grid.


Chain codes can be based on either 4-connectedness or 8-connectedness.

The first difference of the chain code: This difference is obtained by counting the number of direction changes (in a counterclockwise direction) For example, the first difference of the 4-direction chain code 10103322 is 3133030.

Assuming the first difference code represent a closed path, rotation normalization can be achieved by circularly shifting the number of the code so that the list of numbers forms the smallest possible integer.

Size normalization can be achieved by adjusting the size of the resampling grid.


CHAIN CODES

DIFFERENTIAL CHAIN CODE


Chain code representations for a feature outline starting at the blue pixel

and proceeding clockwise, showing the conventional direction codes and

below them the differential chain code R6E-P.613

Normalization for rotation – first difference

Counting (counterclockwise) the number of direction changes that separate two adjacent element of the code

Normalization for starting position – shape number

The first difference of smallest magnitude

Normalization for size

Multi-scaling resampling

CHAIN CODES NORMALIZATION

POLYGONAL APPROXIMATIONS

a digital boundary can be approximated with arbitrary accuracy by a polygon

To capture the essence of the boundary shape with the fewest possible polygonal segments.

not trivial and time consuming

Various methods: Minimum perimeter polygons

Merging techniques (least square error line fit)


Polygonal approximations: to represent a boundary by straight line segments, and a closed path becomes a polygon.

The number of straight line segments used determines the accuracy of the approximation.

Only the minimum required number of sides necessary to preserve the needed shape information should be used (Minimum perimeter polygons).

A larger number of sides will only add noise to the model.



32

MINIMUM PERIMETER POLYGONS

Two techniques: merging & splitting


Minimum perimeter polygons: (Merging and splitting)

Merging and splitting are often used together to ensure that vertices appear where they would naturally in the boundary.

A least squares criterion to a straight line is used to stop the processing.



REPRESENTATION POLYNOMIAL APPROXIMATIONS

Merging Techniques Splitting Techniques

34

1S

2S


35

MERGING TECHNIQUE

Repeat the following steps:

Merging unprocessed points along the boundary until the least-square error line fit exceed the threshold.

Store the parameters of the line

Reset the LS error to 0

Intersections of adjacent line segments form vertices of the polygon.

Problem: vertices of the polygon do not always correspond to inflections (e.g. corners) in the original boundary.


G3E-P.829


Merging technique problem: corners

Solution: Splitting: to subdivide a segment successively into two parts until a given criterion is satisfied.

Objective: seeking prominent inflection points.


37

SPLITTING TECHNIQUE

Subdivide a segment successively into two parts until a specified criterion is met.

For example


G3E-P.830

SPLITTING TECHNIQUES

1. find the major axis 2. find minor axes which perpendicular to major

axis and has distance greater than a threshold 3. repeat until we can’t split anymore


G3E-P.830

SIGNATURES

A 1-D functional representation of a boundary

Basic idea : reduce the boundary representation to a 1-D function, which might be easier to describe than a 2-D boundary

One simple approach : Plot the distance from the centroid to the boundary as a function of angle. It is invariant to translation, but not to rotation and scaling.

Rotation : select the farthest point from the centroid as the starting point

Scaling : normalize the function by variance

G3E-P.830-831

The idea behind a signature is to convert a two

dimensional boundary into a representative one

dimensional function.


G3E-P.831

map 2D function to 1D function 4/18/2019 DIGITAL IMAGE PROCESSING 40

SIGNATURES

REPRESENTATION SIGNATURE

Distance signature of circle shapes

Distance signature of rectangular shapes

41

θr

Ar(θ)

4

2

3

4

5

4

3

2

7

4

2

θ

A

r(θ)

4

2

3

4

5

4

3

2

7

4

2

θ

2A

A


SIGNATURES

Invariable to translation, but depends on rotation & scaling

normalization of the procedure is necessary

e.g. select the same starting point


For the signature just described Invariant to translation (everything w.r.t. the centroid)

Depending on rotation and scaling

How to remove the dependence on rotation? Selecting point farthest from the centroid

Farthest point on the eigen axis (more robust)

Using the chain code Starting at the point farthest from the reference point or using the major axis of the region can be used to decrease

dependence on rotation.

How to remove the dependence on scaling? Scale all functions to span the range, say, [0,1] (not very robust)

Normalize by the variance Scale invariance can be achieved by either scaling the signature function to fixed amplitude or by dividing the function values

by the standard deviation of the function.


SIGNATURES

Changes in size change the amplitude values

Scaling the functions so that they always span the same range of values, e.g. [0,1] might help.

Disadvantage: it only depends on minimum & maximum values.


SIGNATURES

Another way to generate signatures:

Plot the angle between a line tangent to the boundary and a reference line as a function of position along the boundary

e.g. horizontal segments would correspond to straight lines as the tangent angle would be constant there.

Slope-density function = a histogram of tangent-angle values.


BOUNDARY SEGMENTS

Boundary segments: decompose a boundary into segments.

Decomposition of a boundary into segments reduces the boundary’s complexity and simplifies the description process.

Use of the convex hull of the region enclosed by the boundary is a powerful tool for robust decomposition of the boundary.

Convex hull H of a set S is the smallest convex containing S

the set different H-S is called convex deficiency D of the set S

Scheme: independent of size and orientation.



BOUNDARY SEGMENTS

BOUNDARY SEGMENTS

Prior to partitioning, smooth the boundary:

e.g. by replacing each pixel by the average coordinates of m of its neighbors along the boundary

or use a polygonal approximation prior to finding the convex deficiency


THE SKELETON OF A REGION

To reduce a plane region to a graph

by e.g. obtaining the skeleton of the region via thinning.


Use skeleton to represent a region

Skeletonizing (thinning) a region

Medial axis transformation (MAT) : for each point (p) in a region R, find its closest neighbor in the boundary (B). If it has more than one closest neighbor in B, then this point belongs to the medial axis of the region (skeleton)

Computationally expensive

a fast MAT algorithm for binary image

G3E-P.834



Find medial axis transformation (MAT):

The MAT of region R with border B is found as:

For each point p in R, we find its closest neighbor in B.

If p has more than one, it belongs to the Medial Axis (skeleton) of R.


Skeletons: produce a one pixel wide graph that has the same basic shape of the region, like a stick figure of a human. It can be used to analyze the geometric structure of a region which has bumps and “arms”.

G3E-P.834

medial axis (skeleton) 4/18/2019 DIGITAL IMAGE PROCESSING 52

SKELETONS


To improve computational efficiency, in essence we perform thinning:

Edge points of a region are iteratively deleted if:

End points are not deleted

Connectedness is not broken

No excessive erosion is caused


G3E-P.835

Before a thinning algorithm:

A contour point is any pixel with value 1 and having at least one 8-neighbor valued 0.

Let

98321 ...)( pppppN

29832

1

,, ,...,,

sequence ordered in the

ns transitio1-0 ofnumber the:)(

ppppp

pT

G3E-P.835

REPRESENTATION SKELETONS

Step 1: Flag a contour point p1 for deletion if the following conditions are satisfied

6)(2 )a( 1 pN

1)( b)( 1 pT

0 )c( 642 ppp

0 )d( 864 ppp

G3E-P.835


Step 2: Flag a contour point p1 for deletion again. However, conditions (a) and (b) remain the same, but conditions (c) and (d) are changed to

0 )c'( 842 ppp

0 )'d( 862 ppp

G3E-P.835



Step1:

(a)

(b)

(c)

(d)

Step2:

(c’)

(d’)

57

12 ( ) 6N p

1( ) 1T p

4 6 8 0p p p

2 4 6 0p p p

2 4 8 0p p p

2 6 8 0p p p


G3E-P.835

A thinning algorithm: (1) applying step 1 to flag border points for deletion

(2) deleting the flagged points

(3) applying step 2 to flag the remaining border points for deletion

(4) deleting the flagged points

This procedure is applied iteratively until no further points are deleted.



One application of skeletonization is for character recognition.

A letter or character is determined by the center-line of its strokes, and is unrelated to the width of the stroke lines.

REPRESENTATION SKELETONS: EXAMPLE



Representative Shape Descriptors

R6E-P.600

61

DESCRIPTORS

Boundary information can be used directly, or converted into features that describe properties of a region, such as Length of the boundary

Orientation of straight line joining its extreme points

Number of concavities in the boundary


Boundary information can be used directly, or converted into features that describe properties of a region. There are several simple geometric measures that can be useful for describing a boundary.

The length of a boundary: the number of pixels along a boundary gives a rough approximation of its length.

diameters

eccentricity

Orientation of straight line joining its extreme points

Number of concavities in the boundary

Curvature: the rate of change of slope

To measure a curvature accurately at a point in a digital boundary is difficult

The difference between the slops of adjacent boundary segments is used as a descriptor of curvature at the point of intersection of segments


BOUNDARY DESCRIPTORS

63

BOUNDARY DESCRIPTORS (PROPERTIES)

Length: approximated by the # of pixels

Diameter:

Basic rectangle: defined by

Major axis: the line giving the diameter

Minor axis: the line perpendicular to major axis

Eccentricity: ratio of the 2 axes

ji

ji

ppD ,max,

G3E-P.837 4/18/2019 DIGITAL IMAGE PROCESSING

LENGTH OF A BOUNDARY

Length:

the number of pixels along a boundary

give a rough approximation of its length

Number of vertical and horizontal components + √2 times the number of diagonal components


DIAMETERS

Diameter:

D is a distance measure

pi and pj are points on the boundary B

Major axis (connecting the two extreme points)

)],([max)(Diam,

jiji

ppDB


ECCENTRICITY

ratio of the major to the minor axis

major axis = the line connecting the two extreme points that comprise the diameter

minor axis = the line perpendicular to the major axis


CURVATURE

Curvature: Rate of change of slope

i.e. using the difference between the slopes of adjacent boundary segments, which have been represented as straight lines, as a descriptor of curvature at the point of intersection of the segments.

Convex segment: change in slope at p is nonnegative

Concave segment: change in slope at p is negative

Ranges in the change of slope: Less than 10° line

More than 90° corner


BOUNDARY DESCRIPTORS

Other Boundary Descriptors:

Shape numbers

Fourier descriptors

Moments


Boundary Descriptors Shape Numbers

First difference

The shape number of a boundary is defined as the first difference of smallest magnitude.

The order n of a shape number is defined as the number of digits in its representation.

G3E-P.838

4-directional code


Boundary Descriptors Fourier Descriptors

This is a way of using the Fourier transform to analyze the shape of a boundary.

The x-y coordinates of the boundary are treated as the real and imaginary parts of a complex number.

Then the list of coordinates is Fourier transformed using the DFT.

The Fourier coefficients are called the Fourier descriptors.

The basic shape of the region is determined by the first several coefficients, which represent lower frequencies.

Higher frequency terms provide information on the fine detail of the boundary.


73

FOURIER DESCRIPTOR Given a set of boundary points (x0, y0), (x1, y1), (x2, y2), …,. (xK-1, yK-1).

Points on the boundary can be treated as a complex number s(k)=x(k)+jy(k)

Represent the sequence of coordinates as s(k)=[xk, yk], for k=0, 1, 2, …, K-1.

Using complex representation for each point:

s(k) = xk + jyk

Advantage: reducing 2D to 1D problem


BOUNDARY DESCRIPTORS FOURIER DESCRIPTORS (1)

Step1:

Step2: (DFT)

Step3: (reconstruction)

if a(u)=0 for u>P-1

Disadvantage: Just for closed boundaries

74

( ) ( ) ( )s k x k jy k

1 2 /

0

1( ) ( )

K j uk K

ka u s k e

K

1 2 /

0( ) ( )

P j uk K

us k a u e



What’s the reason that previous Fourier descriptors can’t be used for non-closed boundaries?

How can we use the method to descript non-closed boundaries?

(a)linear offset

(b)odd-symmetric

extension

75

•Original segment

s1(k)

(x0, y0)

(xK1, yK1) s2(k)

(b) Linear

offset

s3(k)

(c) Odd symmetric extension

Step 2

Step 3



The proposed method is used not only for non-closed boundaries but also for closed boundaries.

Why we used proposed method to descript closed boundaries rather than previous method?



Figure An ellipse and its Fourier description.

N3E-P.350 Images

take

n fr

om

Nix

on

& A

gua

do, Featu

re E

xtr

act

ion

& Im

age P

roce

ssin

g f

or

Com

put

er

Visio

n(3e)

(2012)

DIGITAL IMAGE PROCESSING 4/18/2019

78

Figure Example of angular Fourier descriptors.

N3E-P.368 Images

take

n fr

om

Nix

on

& A

gua

do, Featu

re E

xtr

act

ion

& Im

age P

roce

ssin

g f

or

Com

put

er

Visio

n(3e)

(2012)


Figure Example of a contour defined by elliptic

Fourier descriptors.

N3E-P.372 Images

take

n fr

om

Nix

on

& A

gua

do, Featu

re E

xtr

act

ion

& Im

age P

roce

ssin

g f

or

Com

put

er

Visio

n(3e)

(2012)


Figure Fourier approximation.

N3E-P.376 Images

take

n fr

om

Nix

on

& A

gua

do, Featu

re E

xtr

act

ion

& Im

age P

roce

ssin

g f

or

Com

put

er

Visio

n(3e)

(2012)


Figure Example of elliptic Fourier descriptors.

N3E-P.377 Images

take

n fr

om

Nix

on

& A

gua

do, Featu

re E

xtr

act

ion

& Im

age P

roce

ssin

g f

or

Com

put

er

Visio

n(3e)

(2012)


Figure Reconstructing an ellipse from a Fourier

description.

N3E-P.351 Images

take

n fr

om

Nix

on

& A

gua

do, Featu

re E

xtr

act

ion

& Im

age P

roce

ssin

g f

or

Com

put

er

Visio

n(3e)

(2012)


Unrolling a feature shape by plotting the x- and y-coordinates of boundary points as a function of position along the boundary (the red and blue graphs) and combining them as real and imaginary parts of complex numbers (the magenta line).

R6E-P.614

84

1D FOURIER (REVIEW) Fourier descriptor : the discrete Fourier transform (DFT) of s(k) is

The complex coefficients a(u), for u=0,1, 2, …, K-1, are called the Fourier descriptor of the boundary

The inverse Fourier restores s(k)

1

0

/21 K

k

KukjeksK

ua

1

0

/2K

u

Kukjeuaks


85

FOURIER APPROXIMATION Usually, only the first few coefficients are used to reprsent the shape (k=0:M, M<N)

Using only the first P coefficients (a(u)=0 for u>P)

Please note that k still ranges from 0 to K-1

The smaller P becomes, the more detail that is lost on the boundary

There are ways to make the descriptor insensitive to translation, rotation, and scale changes.

1

0

/2ˆP

u

Kukjeuaks


Boundary Descriptors Fourier Descriptors


BOUNDARY DESCRIPTORS POLYNOMIAL APPROXIMATION(1)

Lagrange Polynomial

Cubic Spline Interpolation

87

0 ,0 , ,

0

( ) ( ) ( ) ( ) ( ) ( ) ( )n

n n n n k n k

k

P x f x L x f x L x f x L x

0 1 1,

0 1 1

( ) ( )( ) ( )( )

( ) ( )( ) ( )

k k nn k

k k k k k k n

x x x x x x x xL x

x x x x x x x x

( 1)

0 1

( ( ))( ) ( ) ( )( ) ( )

( 1)!

n

n

f xf x P x x x x x x x

n

( ) ( )e f x P x

x

S(x)

0x 1x 2x 3x 4x 5x 6x 7nx

0S

1S

4S

5S6S

1 1 1 1

' '

1 1 1

" "

1 1 1

( ) ( ) ( )

( ) ( )

( ) ( )

j j j j j

j j j j

j j j j

S x f x S x

S x S x

S x S x



Proposed method(1)

Step1: rotate the boundary and let two end point locate at x-axis

Step2: use second order polynomial to approximate the boundary

88

( )f x

x

( ')f x

'x

( ')f x

'x

y

0 'x1 'nx

a

b

( , )2

ab

(0,0)( ,0)a

2

2

4ˆ ( ' )

2

b ay x b

a

212 2

20

4ˆ' ' ( ' )

2

n

j j

j

b ae y y y x b

a



Proposed method(2)

If the boundary is closed, how can we do?

Step1: use split approach divide the boundary to two parts.

Step2: use parabolic function to fit the boundary.

89

1 'y

2 'y

1y

2y

1y

2y


Boundary Descriptors Statistical Moments

Moments are statistical measures of data.

They come in integer orders.

Order 0 is just the number of points in the data.

Order 1 is the sum and is used to find the average.

Order 2 is related to the variance, and order 3 to the skew of the data.

Higher orders can also be used, but don’t have simple meanings.


Boundary Descriptors Statistical Moments

Let r be a random variable, and g(ri) be normalized (as the probability of value ri occurring), then the moments are

1

0

)()()(K

k

i

n

in rgmrr

1

0

)( whereK

i

ii rgrm

G3E-P.843

MOMENTS OF SHAPE

The evaluation of moments represents a systematic method of shape analysis.

The most commonly used region attributes are calculated from the three low-order moments.

Knowledge of the low-order moments allows the calculation of the central moments, normalized central moments, and moment invariants.


96

REGIONAL DESCRIPTORS

Simple descriptors:

Area

Perimeter

Compactness: (perimeter)2/area

Mean and median of the gray levels.

Topological descriptors:

# of holes

# of connected components

topological descriptors

texture


SIMPLE DESCRIPTORS

The area of a region The number of pixels contained within its boundary

The perimeter of a region The length of its boundary

The compactness of a region perimeter * perimeter / area

Dimensionless, and thus insensitive to scale changes

Insensitive to orientation

Q: What shape gives the minimal compactness

The mean and median of the gray levels

The minimum and maximum gray-level values

The number of pixels with values above and below the mean


SHAPE ANALYSIS

Shape measurements are physical dimensional measures that characterize the appearance of an object.

The goal is to use the fewest necessary measures to characterize an object adequately so that it may be unambiguously classified.

The shape may not be entirely reconstructable from the descriptors, but the descriptors for different shapes should be different enough that the shapes can be discriminated.


Region Properties

Once a binary image has been processed we could obtain properties about the regions in the processed image.

Some of those properties are

Area, centroid, perimeter, perimeter length, circularity of the region and second circularity measure



Table 11 1 Representative Shape Descriptors

R6E-P.600

R6C-P.393

Area and Centroid

Area A = sum of all the 1-pixels in the region.

Centroid [r’,c’] is the average location of the pixels in the region

r’ = 1/A*Sum of all the rows in the region

c’ = 1/A*Sum of all the columns in the region.


AREA

The area is the number of pixels in a shape.

The convex area of an object is the area of the convex hull that encloses the object.

original image net area filled area convex area

PERIMETER

The perimeter [length] is the number of pixels in the boundary of the object.

The convex perimeter of an object is the perimeter of the convex hull that encloses the object.

perimeter external perimeter convex perimeter

A simple definiton of the perimeter of a region without holes i set of its interior border pixels.

A pixel of a region is a border pixel if it has some neighboring pixel that is outside the region.

When 8-connectivity is used to determine whether a pixel inside the region is connected to a pixel outside the region, the resulting set of perimeter pixel is 4-connected.

When 4-connectivity is used to determine whether a pixel inside the region is connected to a pixel outside the region, the resulting set of perimeter pixel is 8-connected.


PERIMETER

THE PERIMETER:


The length of Perimeter

To compute length |P| of perimeter P, the pixels in P must be ordered in a sequence P=<(r0,c0),. . . , (rk-1,ck-1)>, each pair of successive pixels in the sequence being neighbor, including the first and last pixels.


MAJOR AND MINOR AXES

The major axis is the (x,y) endpoints of the longest line that can be drawn through the object. The major axis endpoints (x1,y1) and (x2,y2) are by computing the pixel distance between every combination of border pixels in the object boundary and finding the pair with the maximum length.

The minor axis is the (x,y) endpoints of the longest line that can be drawn through the object whilst remaining perpendicular with the major-axis. The minor axis endpoints (x1,y1) and (x2,y2) are found by computing the pixel distance between the two border pixel endpoints.


ASPECT RATIO The major-axis length of an object is the pixel distance between the major-axis endpoints.

The minor-axis length of an object is the pixel distance between the minor-axis endpoints

The aspect ratio measures the ratio of the objects height to its width:

width

heightratioaspect


COMPACTNESS (FORMFACTOR)

Compactness is defined as the ratio of the area of an object to the area of a circle with the same perimeter:

2perimeter

area4scompactnes

• A circle is used as it is the object with the most

compact shape: the measure takes a maximum

value of 1 for a circle


COMPACTNESS (CONT.)


CIRCULARITY OR ROUNDNESS

A measure of roundness or circularity (area-to-perimeter ratio) which excludes local irregularities can be obtained as the ratio of the area of an object to the area of a circle with the same convex perimeter:

• Roundness equals 1 for a circular object and less than 1

for an object that departs from circularity, except that it

is relatively insensitive to irregular boundaries.

2

convexperimeter

area4roundness


Circularity(1):

With the area A and perimeter P defined, a common measure of the circularity of the region is the length of the perimeter squared divided by the area.


ROUNDNESS (CONT.)


CONVEXITY

Convexity is the relative amount that an object differs from a convex object. A measure of convexity can be obtained by forming the ratio of the perimeter of an object’s convex hull to the perimeter of the object itself:

external

convex

perimeter

perimeterconvexity

• This will take the value of 1 for a convex object, and will

be less than 1 if the object is not convex, such as one

having an irregular boundary.


CONVEXITY (CONT.)


SOLIDITY

Solidity measures the density of an object. A measure of solidity can be obtained as the ratio of the area of an object to the area of a convex hull of the object:

convex

net

area

areasolidity

• A value of 1 signifies a solid object, and a value

less than 1 will signify an object having an

irregular boundary (or containing holes).


SOLIDITY (CONT.)


EXTENSION

Extension is a measure of how much the shape differs from the circle. It takes value of zero if the shape is circular and increases without limit as the shape become less compact

1112 lnlnlog cE


DISPERSION

Dispersion is the minimum extension that can be attained by compressing the shape uniformly. There is a unique axis, the long axis of the shape, along which the shape must be compressed in order to minimize its extension.

Dispersion is invariant to stretching, compressing or shearing the shape in any direction

2121212 ln2

1ln

2

1log cD


ELONGATION

Elongation is the measure how much the shape must be compressed along its long axis in order to minimize the extension

Elongation never take a value of less than zero or greater than extension

2

1

2

1

2

12 ln

2

1ln

2

1log

cL


CELL

SH

APE

No Extension Dispersion Elongation

1 0.1197 0.0542 0.0655

2 0.3998 0.0524 0.3474

3 0.7575 0.0550 0.7025

4 0.8725 0.1740 0.6985

5 0.0920 0.0262 0.0659

6 0.3784 0.0277 0.3506

7 0.7411 0.0313 0.7099

8 0.8591 0.1434 0.7157

9 0.0816 0.0138 0.0679

10 0.3617 0.0160 0.3457

11 0.7243 0.0199 0.7044

12 0.8350 0.1029 0.7321


FIBER LENGTH

This gives an estimate as to the true length of a threadlike object.

Note that this is an estimate only. The estimate is fairly accurate on threadlike objects with a formfactor that is less than 0.25 and gets worse as the formfactor increases.

4

area16perimeterperimeterlength thread

2


FIBER WIDTH

This gives an estimate as to the true width of a threadlike object.

Note that this is an estimate only. The estimate is fairly accurate on threadlike objects with a formfactor that is less than 0.25 and gets worse as the formfactor increases.

4

area16perimeter-perimeter widththread

2


AVERAGE FIBER LENGTH

The number of skeleton end-points estimates the number of fibers (half the number of ends)

Average length:

Picture size =

3.56 in x 3.56 in

Skeletonization

Total length = 29.05 in

Number of

end-points = 14

points end ofnumber 0.5

length totallengthfiber average

Length=4.15 in


EUCLIDEAN DISTANCE MAPPING

Euclidean Distance Map (EDM) converts a binary image to a grey scale image in which pixel value gives the straight-line distance from each originally black pixel within the features to the nearest background (white) pixel.

EDM image can be thresholded to produce erosion, which is both more isotropic and faster than iterative neighbor-based morphological erosion.



Principal axes:

Eigenvectors of the covariance matrix

Ratio of large to small eigenvalue: insensitive to scale and rotation



Other descriptors:

Mean and median of gray levels

Min. and max. gray-level values

# pixels with values above and below the mean


Regional Descriptors Example


TOPOLOGICAL DESCRIPTORS

Topology:

properties of figures that are unaffected by deformations

no tearing or joining though rubber sheet distortions.



Examples:

number of of holes H

number of of connected components C:

A subset of maximal size such that any two points can be joined by a connected curve lying entirely within the subset.

Euler number E: E = C-H

also a topological property


Regional Descriptors Topological Descriptors

Topological property 1:

the number of holes (H)


the number of connected

components (C)



Regions represented by straight line segments (polygonal network):

–V: # of vertices

–Q: # of edges

–F: # of faces Euler formula:

V-Q+F = C-H = E




Euler number: the number of connected components subtract the number of holes

E = C - H

E=0 E= -1


REGIONAL DESCRIPTORS TOPOLOGICAL E = V - Q + F = C – H

E: Euler number

V: the number of vertices

Q: the number of edges

F: the number of faces

C: the number of

connected component

H: the number of holes


Representation & Description



Topological

property 4:

the largest

connected

component.


TEXTURE DESCRIPTORS

Statistical approach

Describe the smoothness, coarseness, and graininess of the of region

Structural approach

Describe the arrangement

Spectral approach

Use Fourier spectrum to detect global periodicity

141


Regional Descriptors Texture

Texture is usually defined as the smoothness or roughness of a surface.

In computer vision, it is the visual appearance of the uniformity or lack of uniformity of brightness and color.

There are two types of texture: random and regular. Random texture cannot be exactly described by words or equations; it must be described statistically. The surface of a pile of dirt or rocks of many sizes would be random.

Regular texture can be described by words or equations or repeating pattern primitives. Clothes are frequently made with regularly repeating patterns.

Random texture is analyzed by statistical methods.

Regular texture is analyzed by structural or spectral (Fourier) methods. 4/18/2019 DIGITAL IMAGE PROCESSING 142

REGIONAL DESCRIPTORS TEXTURE

Statistical approaches

smooth, coarse, regular

nth moment:

2th moment:

is a measure of gray level contrast(relative smoothness)

3th moment:

is a measure of the skewness of the histogram

4th moment:

is a measure of its relative flatness

5th and higher moments:

are not so easily related to histogram shape

143

1

0( ) ( ) ( )

L n

n i iiu z z m p z

1

0( )

L

i iim z p z


Regional Descriptors Statistical Approaches

Let z be a random variable denoting gray levels and let p(zi), i=0,1,…,L-1, be the corresponding histogram, where L is the number of distinct gray levels.

The nth moment of z:

The measure R:

The uniformity:

The average entropy:

1

0

)( whereL

i

ii zpzm

)(log)( 2

1

0

i

L

i

i zpzpe

)(1

11

2 zR

1

0

2 )(L

i

izpU

1

0

)()()(L

k

i

n

in zpmzz

G3E-P.850-851

The second moment is the variance and can be used to describe the smoothness

The third moment (skewness): measure skewness of the histogram

The fourth moment (kurtosis): measure the flatness of the histogram

Special moments

0

1

1

0

146

Moments - examples


G3E-P.850

TEXTURE DESCRIPTORS


Regional Descriptors Statistical Approaches

Smooth Coarse Regular


Regional Descriptors Structural Approaches

Structural concepts:

Suppose that we have a rule of the form S→aS, which indicates that the symbol S may be rewritten as aS.

If a represents a circle and the meaning of “circle to the right” is assigned to a string of the form aaaa… .


Fourier spectrum is ideally suited for describing the directionality of periodic or almost periodic 2-D patterns in an image

Global texture patterns can be easily distinguishable as concentrations of high-energy bursts in the spectrum

Prominent peaks give the principal direction of the texture patterns

The location of the peaks give the fundamental spatial period of the patterns

Eliminating any periodic components via filtering leaves nonperiodic image elements which can then be described by statistical techneques

Spectral approach

G3E-P.859-860

Regional Descriptors Spectral Approaches

For non-random primitive spatial patterns, the 2-dimensional Fourier transform allows the patterns to be analyzed in terms of spatial frequency components and direction.

It may be more useful to express the spectrum in terms of polar coordinates, which directly give direction as well as frequency.

Let is the spectrum function, and r and are the variables in this coordinate system.

For each direction , may be considered a 1-D function .

For each frequency r, is a 1-D function.

A global description:

0

)()( rSrS

),( rS

0

1

)()(R

r

rSS

),( rS

)(rS

)(rS

G3E-P.860

152

Spectral approaches - examples

0

1

0

)()(

)()(

R

r

rSS

rSrS

G3E-P.860


Regional Descriptors Spectral Approaches

Regional Descriptors Moments of Two-Dimensional Functions

For a 2-D continuous function f(x,y), the moment of order (p+q) is defined as

The central moments are defined as

,...3,2,1,for ),(

qpdxdyyxfyxm qp

pq

dxdyyxfyyxx qp

pq ),()()(

00

01

00

10 and wherem

my

m

mx


If f(x,y) is a digital image, then

The central moments of order up to 3 are

x y

qp

pq yxfyyxx ),()()(

00

00

00 ),(),()()( myxfyxfyyxxx yx y

0)(),()()( 00

00

1010

01

10 mm

mmyxfyyxx

x y

0)(),()()( 00

00

0101

10

01 mm

mmyxfyyxx

x y

10110111

00

011011

11

11

),()()(

mymmxm

m

mmmyxfyyxx

x y


The central moments of order up to 3 are

1020

02

20 ),()()( mxmyxfyyxxx y

0102

20

02 ),()()( mymyxfyyxxx y

01201121

12

21 22),()()( mxmymxmyxfyyxxx y

10021112

21

12 22),()()( mymxmymyxfyyxxx y

10

2

2030

03

30 23),()()( mxmxmyxfyyxxx y

01

2

0203

30

03 23),()()( mymymyxfyyxxx y


The normalized central moments are defined as

00

pq

pq

,....3,2for 12

where

qpqp

G3E-P.862


A seven invariant moments can be derived from the second and third moments:

2

0321

2

123003210321

2

0321

2

1230123012305

2

0321

2

12304

2

0321

2

12303

2

11

2

02202

02201

)()(3))(3(

)(3)())(3(

)()(

)3()3(

4)(

G3E-P.863


This set of moments is invariant to translation, rotation, and scale change.

2

0321

2

123003213012

2

0321

2

1230123003217

0321123011

2

0321

2

123002206

)()(3))(3(

)(3)())(3(

))((4

)()()(

G3E-P.863

Table 11.3 Moment invariants for the images in Figs. 11.25(a)-(e).



DIGITAL IMAGE PROCESSING 4/18/2019

163

Figure Horizontal axis ellipse moments.

N3E-P.385 Images

take

n fr

om

Nix

on

& A

gua

do, Featu

re E

xtr

act

ion

& Im

age P

roce

ssin

g f

or

Com

put

er

Visio

n(3e)

(2012)


Figure Describing a shape by centralised moments.

N3E-P.387 Images

take

n fr

om

Nix

on

& A

gua

do, Featu

re E

xtr

act

ion

& Im

age P

roce

ssin

g f

or

Com

put

er

Visio

n(3e)

(2012)


Figure Describing a shape by invariant moments.

N3E-P.389 Images

take

n fr

om

Nix

on

& A

gua

do, Featu

re E

xtr

act

ion

& Im

age P

roce

ssin

g f

or

Com

put

er

Visio

n(3e)

(2012)

Connected Components

Components are objects that share at least one common neighbor (in 4- or 8- neighborhood).

Defn: A connected component labeling of binary image B is a labeled image LB in which the value of each pixel is the label of its connected component.


MORPHOLOGY

Morphology deals with form and structure

Mathematical morphology is a tool for extracting image components useful in:

representation and description of region shape

preprocessing (filtering, thinning, etc.)


BASIC MORPHOLOGICAL ALGORITHMS

Purpose:

to extract image components that are useful in the representation and description of shape.

Boundary Extraction:

) ()( BAAA


BASIC MORPHOLOGICAL ALGORITHMS

Extraction of Connected Components:

ABXX kk )( 1k=1,2,3,…

Where X0=p

when Xk=Xk-1 the algorithm has converged.


171

INTRODUCTION

Applicable to boundaries and regions.

First developed by Hotelling

Goal: Remove the correlation among the element of a random vector.

Aside: x is a random vector if each element xi of x is a random variable.

What is a random variable?


172

MEAN AND COVARIANCE is a population of random vectors of length n

Mean vector is

Covariance matrix is defined as

K

k

kkx xK

xEm1

1

T

xx

K

k

T

kk

T

xkxkx mmxxK

mxmxEC 1

1

Expected value

Kkkx

..1


173

EXAMPLE

Considering 4 vectors x1=(0,0,0)T, x2=(1,0,0)T, x3=(1,1,0)T and x4=(1,0,1)T

Mean vector and covariance matrix are

How to interpret entries of the covariance matrix?

1

1

3

4

1xm

311

131

113

16

1xC


174

WHAT TO DO WITH CX?

We want to transform the vectors such that elements of the new vectors are uncorrelated.

Making the off-diagonal elements of covariance matrix 0.

Cx is real and symmetric, so there exists a set of n orthonormal eigenvectors, ei with corresponding eigenvalues in descending order. i


175

HOTELLING TRANSFORM (PRINCIPAL-COMPONENT ANALYSIS)

Let A be a matrix in the form

Create a new set of vectors

T

n

T

T

e

e

e

A2

1Largest eigenvalue

Smallest eigenvalue

Kkky

..1

xmxAy


176

MY AND CY FOR Mean vector is 0 (zero vector)

Covariance matrix is

Exactly what we want: 0 off-diagonal elements

Kkky

..1

n

T

xy AACC

0

0

2

1

Cx and Cy share the same

eigenvalues and

eigenvectors

Still remember matrix

diagonalization?


177

RECONSTRUCTION OF X FROM Y

A is orthonormal, i.e. A-1 = AT

x can be recovered from corresponding y

Approximation can be made by using the first k eigenvectors of Cx to form Ak

x

T myAx

x

T

k myAx ˆ


178

APPLICATION: BOUNDARY DESCRIPTION

e1 and e2 are the

eigenvectors of the

object


179

(CON’D)

are the points on the boundary

x=(a,b)T, where a and b are the coordinate values w.r.t. x1- and x2-axes.

The result of Hotelling (principal component) transformation is a new coordinate system:

Origin at the centroid of the

boundary points

Axes are the direction of the

eigenvectors of Cx

Kkkx

..1


180

(CON’D) Aligning the data with the eigenvectors, i.e.

, we get

Aside: is the variance of component yi along eigenvector ei

xmxAy

i


181

DESCRIPTION INVARIANT TO …

Rotation:

Aligning with the principal axes removes rotation

Scaling:

Normalization using eigenvalues (variance)

Translation:

Accounted for by centering the object about its mean


RELATIONAL DESCRIPTORS

To organize boundaries & regions to exploit any structural relationships that may exist between them.

Example:



In the previous example:

Recursive relationship involving the primitive elements a and b.

Rewriting rules:

S aA

A bS, and

A b



When dealing with disjoint structures, tree descriptors are used:

A tree T is a finite set of one or more nodes for which:

There is a unique node $ designated the root

The remaining nodes are partitioned into m disjointed sets T1, …, Tm, each of which in turn is a tree called a subtree of T.



The tree frontier is the set of nodes at the bottom of the tree (leaves), taken in order from left to right.



Two types of information are important (in a tree):

Information about a node stored as a set of words describing the node

Information relating a node to its neighbors stored as a set of pointers to those neighbors


digital image processing - scut.edu.cn · 2019-04-18 · 4/18/2019 digital image processing 39 ....

Documents