User Adaptive Image Ranking for Search Engines

Maryam Mahdaviani Nando de FreitasLaboratory for Computational Intelligence

University of British Columbia

• Screen shot of apple/red apple/red apple fruit

• Screen shot of tiger

Image Retrieval systems mainly use linguistic

features (e.g. words) and not visual cues

Word Polysemy is a common problem in

IR system

legend

before after

Page 2 Page 3 Page 11

How do we do it? Instance Preference Learning by Gaussian Processes

• We want to learn a better ranking from m pair-wise relations: for

• We use the standard hierarchical Bayes probit model [Hebrich et al, NIPS 06; Wei Chu et al, ICML 05]

mk ,...,1kk uv

How do we do it? Instance Preference Learning by Gaussian Processes

• It then follows that :

• The posterior can be easily computed either using MCMC, Laplace’s method, mean field or Expectation Propagation.

before after

legend

Can also do Active Preference Learning

• The system prompts user with intelligent questions to increase the confidence in ranking

• The user can stop questioning once she is annoyed

• The system re-ranks the images based on the preferences

• We calculate for each unlabeled pair; pick the maximum and query the user accordingly [Wei Chu et al, NIPS 05]

before after

legend

?

Water is hardlegend

Conclusion and Future Directions

• We applied state-of-the-art preference learning algorithm for image ranking

• In future we should work on:

Improving the HCI

Improving the vision

Conducting using study

Expand the idea to other search Learning from many sources

Thank You!

Questions? Feedback?

Acknowledgment:

The code for this work has been built on Wei Chu’s supervised preference learning package, which is available online