IntentSearch: Capturing User

Intention for One-Click Internet

Image Search

Presented by

DILSHA V V

IT09106008

MESCE

Guided by

NISHA T M

Objective

Search engine which helps to interpret users search

intention by using ONE-CLICK query image

Contents

Introduction

Existing System

Proposed System

4 steps used

Search techniques

Visual feature design

Adaptive Weight Schema

Features for query categorization

Image Clustering

Advantages

Future enhancement

Conclusion

References

Introduction•Novel Internet image search

•Search engine which helps to interpret users‟ search

intention by using ONE-CLICK query image

•Uses 4 steps for image searching

•Text based information of query word and visual content of

query image to expand the image pool

Introduction(cont..)

User search intention only by query keywords is

difficult because text based image search suffers from..

•Ambiguity of query keywords

•User doesn't have enough knowledge

•Hard for users to describe the visual content of target

images

•Easier search by using both textual and visual content

of query

•Web-scale image search engines mostly rely on

surrounding text features.

•Users‟ search intention by only by query keywords

Existing System

Proposed system

•Image search on the basis of both textual and visual

content of images

•Image pool is re-ranked based on textual and visual

features

Fig. 1: Top-ranked images returned from „Bing‟ using

“apple” as query

4 steps used

Key contribution is to capture the users‟ search intention from

this one-click query image in four steps

•Adaptive similarity

•Keyword expansion

•Visual query expansion

•Image pool expansion

Adaptive similarity

User always has specific intention when submitting a

query image

Categorized into one of the predefined adaptive weight

categories, such as “portrait” and “scenery.”

Correspondence between the query image and its

proper similarity measurement reflects the user

intention.

Keyword expansion

Query keywords are expanded to capture users‟ search

intention inferred from the visual content of query

images

A word w is suggested as an expansion of the query

Image pool expansion

Retrieved by text-based search accommodates images

with a large variety of semantic meanings

More accurate query by keywords is needed to narrow

the intention and retrieve more relevant images.

The user to click on one of the suggested keywords

Both visual and textual information captured are

automatically added into the text query and enlarge the

image pool

Visual query expansion

One query image is not diverse enough to capture the

user‟s intention.

To learn visual and textual similarity metrics, which

are more robust and more specific to the query, for

image reranking.

Search techniques

The user first submits query keywords q.

A pool of image is retrieved by text-based search

User is asked to select the query image from image pool

The query image is classified as one of the predefined

adaptive weight categories

Images in the pool are re-ranked based on their visual

similarities to the query image

Similarities are computed using the weight schema

Visual feature design

Existing features : Gist ,

SIFT,

Daubechies Wavelet ,

Histogram of Gradient (HoG)

New features : Attention guided Color Signature,

Color spatialet (CSpa) ,

Multilayer Rotation Invarient ( EOH),

Facial Feauter

Adaptive Weight Schema

•Weight schema is used for similarity calculations

•Lets take image i and j…

Adaptive similarity between i & j

Sq(i , j) = ∑fm=1

αmq sm(i , j)

where sm(i , j) similarity between i and j on feature m

f is the visual feature

αmq is the express the importance of feature m for

measuring similarity

Existence of faces, the number of faces in the image

Percentage of the image frame taken up by the face region

Coordinate of face center relative to the centre of image

Directionality

Color Spatial Homogeneousness (variance of values in

different blocks of Color Spatialet)

Total energy of edge map obtained from Canny Operator

Edge Spatial Distribution

Features for query categorization

•Image is divided into clusters

•Each word wi has ti clusters

C(wi)= { ci,1 ,.............,ci,ti }

•Visual distance between the query image and a cluster c is

calculated as the mean of the distances between the query

image and the images in c.

•The cluster Ci,j with the minimal distance is chosen as

visual query expansion and its corresponding word wi .

q = wi + q‟

Image Clustering

• Duplicate images

• User friendly

• Easy search for a particular image(on the internet)

• Can find the image is real or not

Advantages

Disadvantages

Future enhancement

•Query log data, which provides valuable co-occurrence

information of keywords , for keyword expansion

•Can be improved by including duplicate detection in

the future work

Conclusion

•Internet image search approach which only requires

one-click user feedback

•Intention specific weight schema

•Without additional human feedback

•Possible for industrial scale image search by both text

and visual content

References

[1] Y. Zhang, Z. Jia, and T. Chen, “Image Retrieval with

Geometry-Preserving Visual Phrases,” Proc. IEEE Int‟l

Conf. Computer Vision and Pattern Recognition, 2011.

[2] J. Cui, F. Wen, and X. Tang, “IntentSearch:

Interactive On-Line Image Search Re-Ranking,” Proc.

16th ACM Int‟l Conf. multimedia,2008.

[3] “Bing Image Search,” http://www.bing.com/images,

2012.

[4] J. Deng, A.C. Berg, and L. Fei-Fei, “Hierarchical

Semantic Indexing for Large Scale Image Retrieval,”

Proc. IEEE Int‟l Conf.Computer Vision and Pattern

Recognition, 2011.

References(cont…)

[5] Y. Cao, C. Wang, Z. Li, L. Zhang, and L. Zhang, “Spatial-

Bag-of-Features,” Proc. IEEE Int‟l Conf. Computer Vision

and Pattern Recognition, 2010.

[6] J. Deng, A.C. Berg, and L. Fei-Fei, “Hierarchical

Semantic Indexing for Large Scale Image Retrieval,” Proc.

IEEE Int‟l Conf.Computer Vision and Pattern Recognition,

2011.

[7] Y. Huang, Q. Liu, S. Zhang, and D.N. Metaxas, “Image

Retrieval via Probabilistic Hypergraph Ranking,” Proc. IEEE

Int‟l Conf.Computer Vision and Pattern Recognition, 2011

THANK YOU