integrating color and spatial information for cbir
DESCRIPTION
Integrating Color And Spatial Information for CBIR. NTUT CSIE D.W. Lin 2003.8.26. References. L. Cinque, G. Ciocca, S. Levialdi, A. Pellicano, and R. Schettini, “Color-based image retrieval using spatial-chromatic histograms,” Image and Vision Computing , 19 (2001) 979-986 - PowerPoint PPT PresentationTRANSCRIPT
Integrating Color And Integrating Color And Spatial Information for Spatial Information for
CBIRCBIR
NTUT CSIE D.W. Lin2003.8.26
References
L. Cinque, G. Ciocca, S. Levialdi, A. Pellicano, and R. Schettini, “Color-based image retrieval using spatial-chromatic histograms,” Image and Vision Computing, 19 (2001) 979-986
M.S. Kankanhalli, B.M. Mehtre, and H.Y. Huang, “Color and spatial feature for content-based image retrieval,” Pattern Recognition Letters, 20 (1999) 109-118
S. Berretti, A.D. Bimbo, and E. Vicario, “Spatial arrangement of color in retrieval by visual similarity,” Pattern Recognition 35(2002) 1661-1674
The representation of color
Color histogram– Global color histogram– Global color histogram + spatial info.– (fixed) Partition + local color histogram
Dominant color– Extracting the representative colors of image
via VQ or clustering (e.g. k-means algorithm)– Spatial info. can be attained
• Histogram refinement• Specific-color pixel distribution (single, pair, triple …)• Edge histogram …
Non-adaptive
Spatial-chromatic histograms [1]
SCH – global color histogram with info. about (single) pixel distribution
SCH attempts to answer:– How many meaningful colors? color space
quantization– Where the pixels having the same color?
location of region(with same color)– How are these pixels spatially arranged?
distribution of region
SCH – Color representation
Color representation– CIELAB Munsell ISCC-NBS
CIELAB– CIEXYZ CIELAB
CIELUV– a, b: opponent color
( green red, blue yellow )
SCH – Color representation
Munsell color system– Hue value chroma
SCH – Color representation
ISCC-NBS Centroid Color System– Partitioning the Munsell color system into 267
blocks, each blocks represented by an unique linguistic tag and the block centroid (Munsell coordinates)
Using back-propagation NN to transform – CIELAB Munsell ISCC-NBS
SCH – Feature vector
The definition of SCH for image ISI(k) = (hI(k), bI(k), σI(k))
– k: kth quantized color (1~c)– hI(k): pixel amount(ratio)– bI(k): baricenter (normalized mean coordinates)– σI(k): standard deviation of (spread)
Properties:– Insensitive to scale changes(via normalization)– Compact representation and rapidly computing
SCH – Similarity measure
Similarity function
– c: number of quantized color– d(·): Euclidean distance– : max. distance2
SCH – Effectiveness measure
– S: relevant items in DB– : retrieved set (short list) for a query– : relevant items in retrieved set
Precision if
Recall if
S
S
s
IE
EI
II
EI
RR
RR
RR
RR
IRq
IRq
ERq
Color and spatial clustering [2]
k-means algorithm– Iteration version– Two-pass version– VQ (LBG algorithm)
Proposed color clustering (two-pass)– Generating a new cluster while d(p, Ci) > T– Merging those clusters with small population
to the nearest cluster
Color and spatial clustering
Spatial clustering– Based on the clustered color layer– Using connected components labeling to
separate the spatial clusters– Discarding those clusters with small
population or lower density(embedded rectangular)
Feature vectors
For image I, color clusters can be givenCci = {Ri, Gi, Bi, λci, xci, yci} i: 1..m (number of color cluster)Ri, Gi, Bi: representative color of clusterλci: pixel ratio of cluster to totalxci, yci : centroid of cluster
fc={Cci|i=1, 2, …, m} Do the same to color-spatial clusters
Similarity measure
1: color distance between color cluster(RGB) 2: relative frequency of pixels of color cluster () 3: spatial distance between color cluster(x, y) 4: relative frequency of pixels of color-spatial cluster
() 5: spatial distance between color-spatial cluster(x, y)
5544332211),( IQD
Spatial arrangement of color[3]
The back-projection from dominant colors to the image results in an exceedingly complex model(e.g. [2])
Authors proposed a descriptor, called weighted walkthroughs, to capture the binary directional relationship of two complex sets of pixels
Weighted walkthroughs
– The model can be extended to represent the relationship of two sets A, B
– w11 evaluates the number of pixel pairs aA and bB such that b is upper right from a
+,+ w+1+1
aB
+, -
-, +
-, -
bbB
abjabiji yxyyCxxCB
Baw dd1,,
bbA B
aaabjabiji yxyxyyCxxCBA
BAw dddd1,,
Ci: characteristic function of negative/positive real number|B| : area of B i,j : ±1
Compositional computation
Reducing the region to a set of rectangular The weight between A and B1B2 can be derived
by linear combination of A/B1 and A/B2
Distance of WW
3 directional indexes:– wH(A, B) = w1, 1 (A, B) + w1, -1 (A, B)
– wV(A, B) = w-1, 1 (A, B) + w1, -1 (A, B)
– wD(A, B) = w-1, -1 (A, B) + w1, 1 (A, B)
Spatial distance
A B A
B
+,+ w+1+1
aB
+, -
-, +
-, -
A
B
DDVVHH wwwwwwwwDs ,
Arrangements comparing
Image model:< E, a, w >
E: set of spatial entities (color-clustered region)
a: E A { anya} (chromatic label)
w: E E W { anys} (spatial description)
Arrangements comparing
Distance between image model Q and D
: injective function(interpretation, association between query and model image)
– DA : chromatic distance (L*u*v*)– DS : spatial distance– Nq : number of entities in query
qN
k
k
hhkhks
Nq
kkkA qqqqDqqDDQ
1
1
11
,,,)1(,,
Future works
Finish the color-spatial study(geometric-enhanced histogram)
Study wavelet and JPEG 2000