shape representation shape representation - what for? 4 4 to reason about an entity we must first...
Post on 21-Dec-2015
222 views
TRANSCRIPT
![Page 1: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/1.jpg)
Shape Shape RepresentationRepresentation
![Page 2: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/2.jpg)
Shape Representation - what for?Shape Representation - what for?
To reason about an entity we must first represent the entity.
![Page 3: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/3.jpg)
• Real world
Image processing
• Shape representation
•Image understanding \ shape recognition
011001011001000100010001000000100111
00101001100111101
Shape Representation - what for?Shape Representation - what for?
![Page 4: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/4.jpg)
TopicsTopics
DefinitionsAttributesPopular Strategies
TopicsTopics
![Page 5: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/5.jpg)
DefinitionsDefinitionsObject Abstraction Representation
![Page 6: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/6.jpg)
ObjectObject
An object is something that
can be seen or touched,
material thing
(Oxford dictionary)
![Page 7: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/7.jpg)
AbstractionAbstraction
Idea of quality separate from
actual examples
(Oxford dictionary)
• ball
• sphere
![Page 8: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/8.jpg)
RepresentationRepresentation
A way of symbolizing an object
2222 zyxr
![Page 9: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/9.jpg)
Attributes Of A Good Attributes Of A Good RepresentationRepresentation
sufficient wide domain unique unambiguous generative stable convenient All will be explained in next All will be explained in next
slidesslides
![Page 10: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/10.jpg)
SufficientSufficient
...Is this representation sufficient enough? Depends on the application
![Page 11: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/11.jpg)
Wide DomainWide Domain
Able to represent many different classes of entities E.g. numbers for elements in a queue
123
![Page 12: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/12.jpg)
UniqueUnique every distinct member of its domain has a
single distinct representation.
Not unique : dog dog dog
unique : pitbull collie cocker-spaniel
![Page 13: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/13.jpg)
UnambiguousUnambiguous
An entity may have different representations but no two distinct entities may have a common representation.
3 three III 2 two II
![Page 14: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/14.jpg)
GenerativeGenerative Capable of directly generating
(recovering) the represented entity
Chain-code :436476872832
12
3
45
6
7
8
![Page 15: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/15.jpg)
StableStable small perturbations do not induce large changes
in the representation of the entity.
![Page 16: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/16.jpg)
ConvenientConvenient A representation may exhibit all the characteristics
that we have discussed so far and yet not be convenient for a task.– e.g. an assembly line robotassembly line robot who’s task is to filter out
rectangels from other shapes might use chaine code representation.
Yet for a police computerized camera that is supposed to compare faces in the crowd to a data base of suspects this is not enough.
![Page 17: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/17.jpg)
Popular Strategies for shape Popular Strategies for shape encodingencoding
Volume based :– Describe the object volumetrically
– Combination of primitive volumes commonly used, eg. cubes, tetrahedra, and discs
– Provide more access to global relationships
– Do not provide direct surface information about the object
– Can represent only closed (boundaryless) surfaces
e.g.
Symmetric Axis Transform
![Page 18: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/18.jpg)
Popular Strategies for shape Popular Strategies for shape encodingencoding
Surface based :– Surface of object represented by a single closed
parametric grid.
– Better suited for partial surfaces.
E.g.
Parametric Bicubic Patches
Gaussian- Image Representations
![Page 19: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/19.jpg)
Popular StrategiesPopular Strategies Others:
– Chain code.– Distance vs. Angle.– Fourier Transform Moment– Generalized Cylinders – Visual Potential
We will discuss these We will discuss these strategiesstrategies
![Page 20: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/20.jpg)
Chain CodeChain Code
![Page 21: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/21.jpg)
Chain CodeChain Code
![Page 22: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/22.jpg)
This example shows that the Chain Code is This example shows that the Chain Code is independent of Location, Starting Point and independent of Location, Starting Point and
orientationorientation
![Page 23: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/23.jpg)
AttributesAttributes
1. wide domain
2. unique
3. unambiguous
4. generative - 2D only
5. stable - depends on tolerance
![Page 24: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/24.jpg)
Distance vs. AngleDistance vs. Angle
![Page 25: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/25.jpg)
Distance vs. AngleDistance vs. Angle
1. Find point of balance.
2. Measure distance to edges.
3. Plot the graph of distance vs. angle.
Taking the dist. vs. angle one step further we get ...
![Page 26: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/26.jpg)
FourierFourier DescriptorDescriptor
![Page 27: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/27.jpg)
AttributesAttributes
1. wide domain
2. unique
3. unambiguous
4. generative
5. stable - depends upon tolerance
![Page 28: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/28.jpg)
MomentsMoments 1. find center of mass
2. find the moment:
![Page 29: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/29.jpg)
Feature extraction takes segmented image data and outputs a list of features which are combined to give the feature vector v.– Area of a binary object can be calculated by simply adding up the
pixel values in the neighbourhood.– Perimeter obtained by subtracting eroded object from original and
adding up the resulting pixel values.– Compactness is a basic roundness measurement. Circle has low
compactness.– Correlation is the convolution of an image with a reference shape.
Zero-and first-order moments give area and centroid.
Conclusion: Conclusion: How to calculate area How to calculate area and centroid position using moments.and centroid position using moments.
![Page 30: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/30.jpg)
What are Moments?What are Moments?1. Used to calculate:
1. area,
2. position,
3. other more complex shape features such as elongation.
2. Provides a general framework.
3. For an image A, where each pixel brightness is denoted by Ai,j, the moment of order k+l is given by:
i j
jilk
kl Ajim ,
![Page 31: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/31.jpg)
Area = Area = mm0000
i jji
i jji
i jji
i jji
lkkl
Am
AAjim
ji
lk
Ajim
,00
,,00
00
00
,
11
1,1
0,0
i j
jiAArea ,
![Page 32: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/32.jpg)
Object PositionObject PositionCalculate the object position
for a specific example.
Using the same process show how you
can generate a general expression
for object position.
Show that this general expression is equal
to a combination of zero- and first-order moments.
![Page 33: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/33.jpg)
j 1 2 3 4 5 6 7j 1 2 3 4 5 6 7i i 11 22 33 44 55 66 77
Determine pixel at center of pattern. Its position gives the object position.
Called the centroid,– denoted by (ic,jc).
ic is the average i value of all white pixels.
jc is the average j value of all white pixels.
Object PositionObject Position
![Page 34: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/34.jpg)
How do we calculate How do we calculate jjcc??
j 1 2 3 4 5 6 7j 1 2 3 4 5 6 7i i 11 22 33 44 55 66 77
j 1 2 3 4 5 6 7j 1 2 3 4 5 6 7
Generate 1-d plot of the number of Generate 1-d plot of the number of pixels with a particular value of j versus jpixels with a particular value of j versus j
![Page 35: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/35.jpg)
PositionPosition
j 1 2 3 4 5 6 7j 1 2 3 4 5 6 7
49.3
17/66172524124
55644322
c
c
c
j
j
Areaj
jc = sum of j values
over all white pixels divided by
the total number of white pixels Total number of white pixels =Area
![Page 36: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/36.jpg)
Position Position jjcc
i jji
i jji
i jji
j ijic
ii
ii
i iii
ii
ii
iic
AAjAAjj
AreaAAAAAAAj
,,,,
7,6,5,4,3,2,1, 7654321
j 1 2 3 4 5 6 7j 1 2 3 4 5 6 7
i
jiA ,
![Page 37: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/37.jpg)
Centroid PositionCentroid Position Centroid position, (ic,jc) can
be calculated using zero-, m00,
and first-order, m10 and m01, moments.
jc = m01 / m00
i jji
i jji
c
i jji
i jji
i jji
i jji
lkkl
A
Aj
j
AjAjim
lk
Am
Ajim
,
,
,,10
01
,00
,
1,0
![Page 38: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/38.jpg)
How do we calculate How do we calculate iicc??
j 1 2 3 4 5 6 7j 1 2 3 4 5 6 7 i i 11 22 33 44 55 66 77
i i 11 22 33 44 55 66 77
![Page 39: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/39.jpg)
PositionPosition
43.4
17/731714121516124
272635444322
c
c
c
i
i
Areai
ic = sum of i values over all pixels divided by the
total number of pixels Total number of pixels
=Areai 1 2 3 4 5 6 7i 1 2 3 4 5 6 7
![Page 40: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/40.jpg)
Position Position iicc
i jji
i jji
i jji
i jjic
j jj
jj
jjj
jj
jj
jjc
AAiAAii
AreaAAAAAAAi
,,,,
,7,6,5,4,3,2,1 7654321
i 1 2 3 4 5 6 7i 1 2 3 4 5 6 7
j
jiA ,
![Page 41: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/41.jpg)
Centroid PositionCentroid Position
Centroid position, (ic,jc) can be calculated using zero-, m00,
and first-order, m10 and m01, moments.
ic = m10 / m00
i jji
i jji
c
i jji
i jji
i jji
i jji
lkkl
A
Ai
i
AiAjim
lk
Am
Ajim
,
,
,,01
10
,00
,
0,1
![Page 42: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/42.jpg)
Centroid PositionCentroid Position (ic,jc)=(4,4)
j 1 2 3 4 5 6 7j 1 2 3 4 5 6 7i i 11 22 33 44 55 66 77
XX
![Page 43: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/43.jpg)
2-2-D Invariant D Invariant DescriptorsDescriptors
![Page 44: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/44.jpg)
Why Invariant Descriptors?Why Invariant Descriptors? Shape-Based Image Retrieval 2-D Invariant Descriptors
– Invariant to translation, rotation, scale change
Moment invariants (region-based measure)– Hu's moment invariants [Hu‘61]
• M.-K. Hu, Pattern recognition by moment invariants, Proc. IRE, vol. 49, p.1428, Sept. 1961
– Taubin's moment invariants [Taubin‘92]– Flusser's moment invariants [Flusser‘93]– Zernike moments [Taegu‘80]
• M. R. Taegue, Image analysis via the general theory of moments, J. Opt. Soc, Amer., vol. 70, pp. 920-930, Aug. 1980
Fourier descriptors [Zhan‘72] (boundary-based measure)
• C. T. Zhan, R. S. Roskies, Fourier descriptors for plane closed curves, IEEE Trans. Comput., vol. C-21, 1972, pp. 269-281
![Page 45: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/45.jpg)
What Kind of?What Kind of?
1,...,0))},(),({( Niiyix
1,...,0),()()( Niijyixiz
1
0
)2
exp()()(N
i N
kijizkZ
12
,...,2,1,...,2
},|)1(|
|)(|{
mmk
Z
kZ
Boundary pointsBoundary points
Considering in complex planeConsidering in complex plane
Fourier descriptorsFourier descriptors
NormalizationNormalization
Fourier descriptorsFourier descriptors
)8(),...,( 81 mFFF
![Page 46: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/46.jpg)
What Kind of?What Kind of?
Syx
qppq yxSm
),(
)(
)(
)(,
)(
)(,)()()(
00
01
00
10
),( Sm
Smy
Sm
SmxyyxxS
Syx
qppq
,..3,2,12
,)(00
qpqp
S pqpq
Moment invariantsMoment invariants
Region pointsRegion points
MomentMoment
Central momentCentral moment
Normalized central momentNormalized central moment
}1),(|),{( yxfyxS
Various waysVarious ways
![Page 47: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/47.jpg)
Moment InvariantsMoment Invariants
])()(3)[)(3(
])(3))[()(3(
))((4
])())[((
])())[()(3(
])(3))[()(3(
)()(
)3()3(
4)(
20321
2123003211230
20321
21230123003217
0321123011
20321
2123002206
20321
2123003210321
20321
21230123012305
20321
212304
20321
212303
211
202202
02201
H
H
H
H
H
H
H
M
M
M
M
M
M
M),...,( 71 HHH MMM
• HuHu[‘61] : 2nd-3rd order NCM -> 7 invariant moments[‘61] : 2nd-3rd order NCM -> 7 invariant moments
![Page 48: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/48.jpg)
• TaubinTaubin & Cooper[‘92] : the concept of covariant matrix & Cooper[‘92] : the concept of covariant matrix
-> affine invariant moments-> affine invariant moments
)(/, 00'
'02
'11
'11
'20
11 SmM pqpq
ILLMll
lL T
11
2221
11 ,0
qp
Syxpq yylxxlxxl
Sm)]()([)]([
)(
12221
),(11
00
'
2
1,
2
1,, 21
'042
'131
'222
'131
'22
'311
'222
'312
'402
'22'
031'12
'211
'121
'21
'301'
12
cc
ccc
cc
ccc
Mcc
ccM
),...,( 81 TTT MMM
)(41
),(),(),( '04
'40122212221212 TT MMMMMM
CM of order 2 -> 2x2 matrixCM of order 2 -> 2x2 matrix
Lower triangular matrixLower triangular matrix
New momentsNew moments
3rd-4th order moments -> two matrices3rd-4th order moments -> two matrices
Moment invariantsMoment invariants
Moment InvariantsMoment Invariants
![Page 49: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/49.jpg)
MomentMoment
![Page 50: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/50.jpg)
MomentsMoments
since we get invariant values the moments are not affected by transition or rotation.
By normalizing the values, we can make the
moment indifferent to scaling
![Page 51: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/51.jpg)
Attributes of momentsAttributes of moments
1. wide domain
2. unique - no
3. unambiguous - no
4. generative - no
5. stable - no
![Page 52: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/52.jpg)
)2(
)34(
)61296
8186
12966(
)]()()([
3446(
)(
322
23104
21240122231220440
9006
22212310440
6005
230
3022130
20211123002
211
221
202201230
20220
033031112210211200330021120
032121120
21202
220032102
220031211
220
303
320
11004
2211230021221033011
212032120
7003
212
221
32130
3123003122130
203
230
10002
2110220
4001
F
F
F
F
F
F
M
M
M
M
M
M
),...,( 61 FFF MMM
• FlusserFlusser & Suk[‘93]: affine moment invariants & Suk[‘93]: affine moment invariants
• Taegue[‘80]: Taegue[‘80]: ZernikeZernike moments -> moment invariants moments -> moment invariants• Invariant under rotationInvariant under rotation
• Invariant under general 2-D affine transformationInvariant under general 2-D affine transformation
• Normalization -> invariant under translation and scalingNormalization -> invariant under translation and scaling• translate origin to centroidtranslate origin to centroid• scale so that max distance from centroid == 1scale so that max distance from centroid == 1• ignore first two momentsignore first two moments
2/|)|(
0
2
1,),(
*
!2/|)|(!2/|)|(!
)!()1()(
),exp()(),(,),(1
22
mn
s
snsnm
nmnmyxSyxnmnm
smnsmns
snR
jmRVVn
A
evenis||and,,...,1,0 mnnmnn
|| nmA
),...,( 101 ZZZ MMM
|||,| 1100 AA
|||,||,||,||,||,||,||,||,||,| 55535144424033312220 AAAAAAAAAA
![Page 53: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/53.jpg)
)2(
)34(
)61296
8186
12966(
)]()()([
3446(
)(
322
23104
21240122231220440
9006
22212310440
6005
230
3022130
20211123002
211
221
202201230
20220
033031112210211200330021120
032121120
21202
220032102
220031211
220
303
320
11004
2211230021221033011
212032120
7003
212
221
32130
3123003122130
203
230
10002
2110220
4001
F
F
F
F
F
F
M
M
M
M
M
M
),...,( 61 FFF MMM
• FlusserFlusser & Suk[‘93]: affine moment invariants & Suk[‘93]: affine moment invariants
• Taegue[‘80]: Taegue[‘80]: ZernikeZernike moments -> moment invariants moments -> moment invariants• Invariant under rotationInvariant under rotation
• Invariant under general 2-D affine transformationInvariant under general 2-D affine transformation
• Normalization -> invariant under translation and scalingNormalization -> invariant under translation and scaling• translate origin to centroidtranslate origin to centroid• scale so that max distance from centroid == 1scale so that max distance from centroid == 1• ignore first two momentsignore first two moments
2/|)|(
0
2
1,),(
*
!2/|)|(!2/|)|(!
)!()1()(
),exp()(),(,),(1
22
mn
s
snsnm
nmnmyxSyxnmnm
smnsmns
snR
jmRVVn
A
evenis||and,,...,1,0 mnnmnn
|| nmA
),...,( 101 ZZZ MMM
|||,| 1100 AA
|||,||,||,||,||,||,||,||,||,| 55535144424033312220 AAAAAAAAAA
![Page 54: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/54.jpg)
What’s the Best?What’s the Best? 44Size : 337 x 145Size : 337 x 145Rotation : -400,400,8Rotation : -400,400,8Scaling : -2,2,0.04Scaling : -2,2,0.04
![Page 55: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/55.jpg)
Recent Solution.Recent Solution. Difficulty (Current Problem in Digital Image
Analysis)– Moment invariants are not sufficient for distinguishing– Sensitive to noise
Region-based measure + boundary-based measure– Gives the better average retrieval efficiency
Two-stage similarity scheme – Moment invariants (1st retrieval stage) -> n images– Fourier descriptors (2nd verification stage) -> m < n
images
![Page 56: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/56.jpg)
Comparison of computation time. (unit=sec)--> Indigo2 IMPACT (MIPS R1000), 337 x 145, 718 boundary points, 17761 region points
Method FD HM TM FM ZM FH FZ
Time 0.01 0.02 0.02 0.02 0.93 0.03 0.94
• Comparison to other similarity schemes (Top5 Comparison to other similarity schemes (Top5 / Top 10)/ Top 10)--> 10 rabbit images / 85 animal images--> 10 rabbit images / 85 animal images
Query\method FD HM TM FM ZM FH FZquery 1 5/8 4/7 4/7 2/3 5/9 5/8 5/9query 2 5/8 5/8 4/5 2/5 5/9 5/8 5/9query 3 5/8 2/6 1/1 1/3 4/8 5/8 5/8query 4 5/8 5/7 4/6 1/3 5/9 5/8 5/9
FD=Fourier descriptorsFD=Fourier descriptors
HM=Hu's moment invariantsHM=Hu's moment invariants
TM=Taubin's moment invariantsTM=Taubin's moment invariants
FM=Flusser's moment invariantsFM=Flusser's moment invariants
ZM=Zernike momentsZM=Zernike moments
FH=FD+HMFH=FD+HM
FZ=FD+ZMFZ=FD+ZM
Recent Solution.Recent Solution.
![Page 57: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/57.jpg)
QueryQuery
FH/FZ Top 5FH/FZ Top 5
FH/FZ Top 10FH/FZ Top 10 FZ Top 10FZ Top 10 MissMiss
FalseFalse
Database: 1195 animal imagesDatabase: 1195 animal imagesSadegh Abbasi, Farzin Mokhtarian, Josef Kittler, S. SclaroffSadegh Abbasi, Farzin Mokhtarian, Josef Kittler, S. Sclaroffhttp://www.ee.surrey.ac.uk/Research/VSSP/imagedb/demo.htmhttp://www.ee.surrey.ac.uk/Research/VSSP/imagedb/demo.htmll
![Page 58: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/58.jpg)
Parametric Parametric Bicubic PatchesBicubic Patches
Split the object to simpler bicubic patches that can be represented by relatively simple mathematical equations.
![Page 59: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/59.jpg)
Piecewise Parametric SurfacesPiecewise Parametric Surfaces
![Page 60: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/60.jpg)
Piecewise PatchesPiecewise Patches
![Page 61: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/61.jpg)
Bezier PatchesBezier Patches
![Page 62: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/62.jpg)
Bezier PatchesBezier Patches
![Page 63: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/63.jpg)
AttributesAttributes
1. generative
2. stable
3. popular in computer graphics
4. Possibility of several equally acceptable bicubic approximations to any given surface makes it inappropriate for surface matching
![Page 64: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/64.jpg)
Symmetric Axis Symmetric Axis TransformTransform
• A.k.a Blum Transform, Medial Axis Transform.• Formally : the object is the logical union of all its maximal discs.
![Page 65: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/65.jpg)
May be thought of as grass fire spreading from the border inwards.
Description : – Locus of centers a.k.a symmetric axis.Locus of centers a.k.a symmetric axis.– Radius at each pointRadius at each point
Symmetric Axis TransformSymmetric Axis Transform
![Page 66: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/66.jpg)
AttributesAttributes
1. wide domain
2. unique
3. unambiguous
4. generative
5. not stable - small changes affect dramatically
6. Primarily used in biological applications
![Page 67: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/67.jpg)
Generalized Generalized CylindersCylinders
Binford 1971 an extension of SAT. Shape represented by an ordinary cylinder sweeping the cross-section along
an arbitrary space curve (axis/spine)
![Page 68: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/68.jpg)
Generalized CylindersGeneralized Cylinders
Decomposition of 3D shape-description problems into lower-order problems
![Page 69: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/69.jpg)
Generalized CylindersGeneralized CylindersA generalized cylinder is thus defined by
a cross section, a cross section, an axisan axis a sweeping rule. a sweeping rule.
![Page 70: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/70.jpg)
Generalized CylindersGeneralized Cylinders
![Page 71: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/71.jpg)
Generalized CylindersGeneralized Cylinders
![Page 72: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/72.jpg)
AttributesAttributes
1. wide domain - depends on implementation (redundancy vs. wide domain )
2. unique
3. generative
4. stability is doubtful
5. Decomposition of complex structures is difficult.
6. Used mainly for object recognition
7. been used in working systems.
![Page 73: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/73.jpg)
Gaussian Gaussian ImageImage
Surface normal vector information for any object can be mapped onto a unit sphere, called the Gaussian sphere.
Mapping is called the Gaussian image of the object.
![Page 74: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/74.jpg)
Extended Gaussian ImageExtended Gaussian Image
The mapping is: Surface normals for each point of the object are placed so that their
tails lie at the center of the Gaussian spheretails lie at the center of the Gaussian sphere
heads lie on a point on the sphere appropriate to heads lie on a point on the sphere appropriate to the particular surface orientationthe particular surface orientation
![Page 75: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/75.jpg)
Extended Gaussian Image (EGI)Extended Gaussian Image (EGI)
We can extend this process so that a weight is assigned to each point on the Gaussian sphere
equal to the area of the surface having the given normal This mapping is called the extended Gaussian image (EGI). Weights are represented by vectors parallel to the surface
normals, with length equal to the weight
![Page 76: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/76.jpg)
EGIEGIExtended Gaussian Image (EGI)Extended Gaussian Image (EGI)
![Page 77: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/77.jpg)
AttributesAttributes
1. wide domain
2. not unique
3. not unambiguous
4. not generative
5. stable - with limitations
6. invariant to translation and scaling
7. position independence
![Page 78: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/78.jpg)
Visual Visual PotentialPotential
Aspect - topological structure of the singularities in a single view.
A graph in which each node represents an aspect of the object, and each edge the possibility of transiting from one aspect to another under motion of the observer [Turner74].
Represents in a concise way any visual experience an observer can obtain by looking at the object when traversing any orbit through space.
![Page 79: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/79.jpg)
Visual PotentialVisual Potential
![Page 80: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/80.jpg)
AttributesAttributes
1. wide domain
2. unique
3. unambiguous
4. generative
5. not stable
![Page 81: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/81.jpg)
SummarySummary
The above methods are deterministic; in practice, uncertainties in shape that result from the noise in measurement have to be considered [Ayache88, Ikeuchi88]
The above methods are based on geometric models, they are more appropriate for describing specific objects, particularly artificial ones with regular structures
![Page 82: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/82.jpg)
ConclusionConclusion
The Proposed Hybrid Method– FD + Hu's/Zernike MI– FD + Hu's MI
Future Works– Invariant to 3-D perspective
transform– Recognition of partially
recovered/occluded objects
![Page 83: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/83.jpg)
SummarySummary
Symbolic models are more appropriate for natural objects that are better defined in terms of generic characteristics (e.g.. small, red, rough) than precise shape; useful for matching
![Page 84: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/84.jpg)
ReferencesReferences
Books :– Nalwa “a guided tour of computer vision”
– Gonzales, “Image analysis and computer vision”
– Jain
Web:– http://web.mit.edu/manoli/ecimorph/www/code/MMorph.html
– http://www.cs.ubc.ca/~clee/cg/proj/index.html
– http://www.dai.ed.ac.uk/CVonline/LOCAL_COPIES/MARSHALL/node53.html#figuresame_EGI
– http://www.cs.uwa.edu.au/~cheng/index.html
![Page 85: Shape Representation Shape Representation - what for? 4 4 To reason about an entity we must first represent the entity](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d605503460f94a418c6/html5/thumbnails/85.jpg)
[email protected]@[email protected]@etri.re.krHoward Schultz, UMass Howard Schultz, UMass Yaron BerlinskyYaron Berlinsky
SourcesSources