WhittleSearch: Image Search with Relative Attribute FeedbackCVPR 2012
Adriana Kovashka Devi Parikh Kristen Grauman
University of Texas at Austin Toyota Technological Institute Chicago (TTIC)
Approach
•Dataset
•At each iteration, the top K < N ranked images
Binary Relevance Feedback
Binary Relevance Feedback
•Current reference set
•Score function
Relative Attribute Feedback
Learning Relative Attributes
•Mechanical Turk (MTurk)
Updating the scoring function from feedback•Feedback form
▫“ What I want is more/less/similarly m than image Itr ”
Three cases
Score function of Relative Attribute Feedback
•Take the intersection of all F feedback
Hybrid Feedback Approach
x denotes the Cartesian product
Experimental Results
Datasets• Shoes
▫14,658 shoe images from like.com▫attributes—‘pointy-at-the-front’, ‘open’, ‘bright-
in-color’, ‘covered-with-ornaments’, ‘shiny’, ‘highat-the-heel’, ‘long-on-the-leg’, ‘formal’, ‘sporty’, and ‘feminine’
• PubFig▫the Public Figures dataset of human faces▫772 images from 8 people and 11 attributes
• OSR▫the Outdoor Scene Recognition dataset of
natural scenes▫2,688 images from 8 categories and 6 attributes
Datasets
•Training set ▫750 triplets of images (i , j , k) from each
dataset
•Score correlation▫Normalized Discounted Cumulative Gain at
top K (NDCG@K)▫K = 50
Iteration experiments on the three datasets
Amount of feedback
Ranking accuracy with first feedback
•Attribute meaning▫“amount of perspective” on a scene is less
intuitivethan “shininess” on shoes