Download - Slides
![Page 1: Slides](https://reader034.vdocuments.site/reader034/viewer/2022042608/55cf9ae9550346d033a3fa65/html5/thumbnails/1.jpg)
IntentSearch: Capturing User
Intention for One-Click Internet
Image Search
Presented by
DILSHA V V
IT09106008
MESCE
Guided by
NISHA T M
![Page 2: Slides](https://reader034.vdocuments.site/reader034/viewer/2022042608/55cf9ae9550346d033a3fa65/html5/thumbnails/2.jpg)
Objective
Search engine which helps to interpret users search
intention by using ONE-CLICK query image
![Page 3: Slides](https://reader034.vdocuments.site/reader034/viewer/2022042608/55cf9ae9550346d033a3fa65/html5/thumbnails/3.jpg)
Contents
Introduction
Existing System
Proposed System
4 steps used
Search techniques
Visual feature design
Adaptive Weight Schema
Features for query categorization
Image Clustering
Advantages
Future enhancement
Conclusion
References
![Page 4: Slides](https://reader034.vdocuments.site/reader034/viewer/2022042608/55cf9ae9550346d033a3fa65/html5/thumbnails/4.jpg)
Introduction•Novel Internet image search
•Search engine which helps to interpret users‟ search
intention by using ONE-CLICK query image
•Uses 4 steps for image searching
•Text based information of query word and visual content of
query image to expand the image pool
![Page 5: Slides](https://reader034.vdocuments.site/reader034/viewer/2022042608/55cf9ae9550346d033a3fa65/html5/thumbnails/5.jpg)
Introduction(cont..)
User search intention only by query keywords is
difficult because text based image search suffers from..
•Ambiguity of query keywords
•User doesn't have enough knowledge
•Hard for users to describe the visual content of target
images
•Easier search by using both textual and visual content
of query
![Page 6: Slides](https://reader034.vdocuments.site/reader034/viewer/2022042608/55cf9ae9550346d033a3fa65/html5/thumbnails/6.jpg)
•Web-scale image search engines mostly rely on
surrounding text features.
•Users‟ search intention by only by query keywords
Existing System
![Page 7: Slides](https://reader034.vdocuments.site/reader034/viewer/2022042608/55cf9ae9550346d033a3fa65/html5/thumbnails/7.jpg)
Proposed system
•Image search on the basis of both textual and visual
content of images
•Image pool is re-ranked based on textual and visual
features
![Page 8: Slides](https://reader034.vdocuments.site/reader034/viewer/2022042608/55cf9ae9550346d033a3fa65/html5/thumbnails/8.jpg)
Fig. 1: Top-ranked images returned from „Bing‟ using
“apple” as query
![Page 9: Slides](https://reader034.vdocuments.site/reader034/viewer/2022042608/55cf9ae9550346d033a3fa65/html5/thumbnails/9.jpg)
4 steps used
Key contribution is to capture the users‟ search intention from
this one-click query image in four steps
•Adaptive similarity
•Keyword expansion
•Visual query expansion
•Image pool expansion
![Page 10: Slides](https://reader034.vdocuments.site/reader034/viewer/2022042608/55cf9ae9550346d033a3fa65/html5/thumbnails/10.jpg)
Adaptive similarity
User always has specific intention when submitting a
query image
Categorized into one of the predefined adaptive weight
categories, such as “portrait” and “scenery.”
Correspondence between the query image and its
proper similarity measurement reflects the user
intention.
![Page 11: Slides](https://reader034.vdocuments.site/reader034/viewer/2022042608/55cf9ae9550346d033a3fa65/html5/thumbnails/11.jpg)
Keyword expansion
Query keywords are expanded to capture users‟ search
intention inferred from the visual content of query
images
A word w is suggested as an expansion of the query
![Page 12: Slides](https://reader034.vdocuments.site/reader034/viewer/2022042608/55cf9ae9550346d033a3fa65/html5/thumbnails/12.jpg)
Image pool expansion
Retrieved by text-based search accommodates images
with a large variety of semantic meanings
More accurate query by keywords is needed to narrow
the intention and retrieve more relevant images.
The user to click on one of the suggested keywords
Both visual and textual information captured are
automatically added into the text query and enlarge the
image pool
![Page 13: Slides](https://reader034.vdocuments.site/reader034/viewer/2022042608/55cf9ae9550346d033a3fa65/html5/thumbnails/13.jpg)
Visual query expansion
One query image is not diverse enough to capture the
user‟s intention.
To learn visual and textual similarity metrics, which
are more robust and more specific to the query, for
image reranking.
![Page 14: Slides](https://reader034.vdocuments.site/reader034/viewer/2022042608/55cf9ae9550346d033a3fa65/html5/thumbnails/14.jpg)
Search techniques
The user first submits query keywords q.
A pool of image is retrieved by text-based search
User is asked to select the query image from image pool
The query image is classified as one of the predefined
adaptive weight categories
Images in the pool are re-ranked based on their visual
similarities to the query image
Similarities are computed using the weight schema
![Page 15: Slides](https://reader034.vdocuments.site/reader034/viewer/2022042608/55cf9ae9550346d033a3fa65/html5/thumbnails/15.jpg)
Visual feature design
Existing features : Gist ,
SIFT,
Daubechies Wavelet ,
Histogram of Gradient (HoG)
New features : Attention guided Color Signature,
Color spatialet (CSpa) ,
Multilayer Rotation Invarient ( EOH),
Facial Feauter
![Page 16: Slides](https://reader034.vdocuments.site/reader034/viewer/2022042608/55cf9ae9550346d033a3fa65/html5/thumbnails/16.jpg)
Adaptive Weight Schema
•Weight schema is used for similarity calculations
•Lets take image i and j…
Adaptive similarity between i & j
Sq(i , j) = ∑fm=1
αmq sm(i , j)
where sm(i , j) similarity between i and j on feature m
f is the visual feature
αmq is the express the importance of feature m for
measuring similarity
![Page 17: Slides](https://reader034.vdocuments.site/reader034/viewer/2022042608/55cf9ae9550346d033a3fa65/html5/thumbnails/17.jpg)
Existence of faces, the number of faces in the image
Percentage of the image frame taken up by the face region
Coordinate of face center relative to the centre of image
Directionality
Color Spatial Homogeneousness (variance of values in
different blocks of Color Spatialet)
Total energy of edge map obtained from Canny Operator
Edge Spatial Distribution
Features for query categorization
![Page 18: Slides](https://reader034.vdocuments.site/reader034/viewer/2022042608/55cf9ae9550346d033a3fa65/html5/thumbnails/18.jpg)
•Image is divided into clusters
•Each word wi has ti clusters
C(wi)= { ci,1 ,.............,ci,ti }
•Visual distance between the query image and a cluster c is
calculated as the mean of the distances between the query
image and the images in c.
•The cluster Ci,j with the minimal distance is chosen as
visual query expansion and its corresponding word wi .
q = wi + q‟
Image Clustering
![Page 19: Slides](https://reader034.vdocuments.site/reader034/viewer/2022042608/55cf9ae9550346d033a3fa65/html5/thumbnails/19.jpg)
![Page 20: Slides](https://reader034.vdocuments.site/reader034/viewer/2022042608/55cf9ae9550346d033a3fa65/html5/thumbnails/20.jpg)
![Page 21: Slides](https://reader034.vdocuments.site/reader034/viewer/2022042608/55cf9ae9550346d033a3fa65/html5/thumbnails/21.jpg)
![Page 22: Slides](https://reader034.vdocuments.site/reader034/viewer/2022042608/55cf9ae9550346d033a3fa65/html5/thumbnails/22.jpg)
![Page 23: Slides](https://reader034.vdocuments.site/reader034/viewer/2022042608/55cf9ae9550346d033a3fa65/html5/thumbnails/23.jpg)
• Duplicate images
• User friendly
• Easy search for a particular image(on the internet)
• Can find the image is real or not
Advantages
Disadvantages
![Page 24: Slides](https://reader034.vdocuments.site/reader034/viewer/2022042608/55cf9ae9550346d033a3fa65/html5/thumbnails/24.jpg)
Future enhancement
•Query log data, which provides valuable co-occurrence
information of keywords , for keyword expansion
•Can be improved by including duplicate detection in
the future work
![Page 25: Slides](https://reader034.vdocuments.site/reader034/viewer/2022042608/55cf9ae9550346d033a3fa65/html5/thumbnails/25.jpg)
Conclusion
•Internet image search approach which only requires
one-click user feedback
•Intention specific weight schema
•Without additional human feedback
•Possible for industrial scale image search by both text
and visual content
![Page 26: Slides](https://reader034.vdocuments.site/reader034/viewer/2022042608/55cf9ae9550346d033a3fa65/html5/thumbnails/26.jpg)
References
[1] Y. Zhang, Z. Jia, and T. Chen, “Image Retrieval with
Geometry-Preserving Visual Phrases,” Proc. IEEE Int‟l
Conf. Computer Vision and Pattern Recognition, 2011.
[2] J. Cui, F. Wen, and X. Tang, “IntentSearch:
Interactive On-Line Image Search Re-Ranking,” Proc.
16th ACM Int‟l Conf. multimedia,2008.
[3] “Bing Image Search,” http://www.bing.com/images,
2012.
[4] J. Deng, A.C. Berg, and L. Fei-Fei, “Hierarchical
Semantic Indexing for Large Scale Image Retrieval,”
Proc. IEEE Int‟l Conf.Computer Vision and Pattern
Recognition, 2011.
![Page 27: Slides](https://reader034.vdocuments.site/reader034/viewer/2022042608/55cf9ae9550346d033a3fa65/html5/thumbnails/27.jpg)
References(cont…)
[5] Y. Cao, C. Wang, Z. Li, L. Zhang, and L. Zhang, “Spatial-
Bag-of-Features,” Proc. IEEE Int‟l Conf. Computer Vision
and Pattern Recognition, 2010.
[6] J. Deng, A.C. Berg, and L. Fei-Fei, “Hierarchical
Semantic Indexing for Large Scale Image Retrieval,” Proc.
IEEE Int‟l Conf.Computer Vision and Pattern Recognition,
2011.
[7] Y. Huang, Q. Liu, S. Zhang, and D.N. Metaxas, “Image
Retrieval via Probabilistic Hypergraph Ranking,” Proc. IEEE
Int‟l Conf.Computer Vision and Pattern Recognition, 2011
![Page 28: Slides](https://reader034.vdocuments.site/reader034/viewer/2022042608/55cf9ae9550346d033a3fa65/html5/thumbnails/28.jpg)
THANK YOU