time-sensitive web image ranking and retrieval via dynamic multi-task regression
DESCRIPTION
Time-Sensitive Web Image Ranking and Retrieval via Dynamic Multi-Task Regression. Gunhee Kim Eric P. Xing. School of Computer Science, Carnegie Mellon University. February 6, 2013. Image Ranking and Retrieval. Goal: Find the images for a given query. Text-based image retrieval. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Time-Sensitive Web Image Ranking and Retrieval via Dynamic Multi-Task Regression](https://reader036.vdocuments.site/reader036/viewer/2022081515/568161dd550346895dd1ecd6/html5/thumbnails/1.jpg)
1
Time-Sensitive Web Image Ranking and Retrieval via Dynamic Multi-Task
Regression
Gunhee Kim Eric P. Xing
School of Computer Science, Carnegie Mellon University
February 6, 2013
![Page 2: Time-Sensitive Web Image Ranking and Retrieval via Dynamic Multi-Task Regression](https://reader036.vdocuments.site/reader036/viewer/2022081515/568161dd550346895dd1ecd6/html5/thumbnails/2.jpg)
Image Ranking and Retrieval
Goal: Find the images for a given query
ex. Cardinal Text-based image retrieval
2
![Page 3: Time-Sensitive Web Image Ranking and Retrieval via Dynamic Multi-Task Regression](https://reader036.vdocuments.site/reader036/viewer/2022081515/568161dd550346895dd1ecd6/html5/thumbnails/3.jpg)
Image Ranking and Retrieval
ex. Cardinal
northern_cardinal_glamour.jpgFile name
http://www.allaboutbirds.org/guide/Northern_Cardinal/id
Url
3
Text-based image retrieval
• Ambiguity and noise due to mismatch.
• Scalable and successful so far
Goal: Find the images for a given query
![Page 4: Time-Sensitive Web Image Ranking and Retrieval via Dynamic Multi-Task Regression](https://reader036.vdocuments.site/reader036/viewer/2022081515/568161dd550346895dd1ecd6/html5/thumbnails/4.jpg)
Recent Image Ranking and Retrieval
Various efforts to improve text-based image search
User relevance feedback [Wang et al. CVPR 11]
Text-based search by apple
chosen by a user
Reranking on visual features
Pseudo-relevance feedback[Liu et al. CVPR 11]
Human labeled training data[Yang et al. MM10]
Image click data [Jain et al. WWW11]
4
![Page 5: Time-Sensitive Web Image Ranking and Retrieval via Dynamic Multi-Task Regression](https://reader036.vdocuments.site/reader036/viewer/2022081515/568161dd550346895dd1ecd6/html5/thumbnails/5.jpg)
5
Time-Sensitive Image Ranking and Retrieval
From experiments of 7.5 millions of Flickr images of 30 topicswe found three good reasons …
Discovery of temporal patterns of Web image collections
• [D08] Dakka et al. CIKM 2008• [M09] Metzler et al. SIGIR 2009• [K10] Kulkani et al, WSDM 2011
• [V11] Amodeo et al, CIKM2011• [R12] Radinsky et al, WWW 2012• …..
No previous work using temporal info on image retrieval
[Related work] Exploring temporal dynamics of Web queries• Popular search keywords and relevant documents change over time.• ex) Keyword search, Product search, News recommendation
![Page 6: Time-Sensitive Web Image Ranking and Retrieval via Dynamic Multi-Task Regression](https://reader036.vdocuments.site/reader036/viewer/2022081515/568161dd550346895dd1ecd6/html5/thumbnails/6.jpg)
6
Why Time-Sensitive Image Retrieval? (1/3)
1. Knowing when search takes place is useful to infer users' implicit intents.
Cardinal: (1) the red bird in America.
Fall to Winter (Sep. ~ Feb.)
Bing
(2) Arizona cardinals (football) (3) St. Louis cardinals (baseball)
Spring to Fall (Mar. ~ Oct.)
• Severely redundant. Almost identical all year long.
![Page 7: Time-Sensitive Web Image Ranking and Retrieval via Dynamic Multi-Task Regression](https://reader036.vdocuments.site/reader036/viewer/2022081515/568161dd550346895dd1ecd6/html5/thumbnails/7.jpg)
7
Why Time-Sensitive Image Retrieval? (1/3)
1. Knowing when search takes place is useful to infer users' implicit intents.
at May 4, 2009
at Feb. 7, 2009 Football
Bing
Ourresults
baseball
• Diversity can make search interesting.
![Page 8: Time-Sensitive Web Image Ranking and Retrieval via Dynamic Multi-Task Regression](https://reader036.vdocuments.site/reader036/viewer/2022081515/568161dd550346895dd1ecd6/html5/thumbnails/8.jpg)
8
Why Time-Sensitive Image Retrieval? (2/3)
2. Timing suitability can be used as a complementary attribute to relevance.
Bing
at May 4, 2009
at Feb. 7, 2009
Ourresults
• There are so many almost equally good images.Background: snow
Background: Green Baby birds or eggs
![Page 9: Time-Sensitive Web Image Ranking and Retrieval via Dynamic Multi-Task Regression](https://reader036.vdocuments.site/reader036/viewer/2022081515/568161dd550346895dd1ecd6/html5/thumbnails/9.jpg)
9
Why Time-Sensitive Image Retrieval? (3/3)
3. Temporal information is synergetic in personalizedimage retrieval.
Louisville Men's College Basketball
At Nov. 7, 2009 for user 30033302
Each user’ term usages are relatively stationary, and predictable once they are learned.
![Page 10: Time-Sensitive Web Image Ranking and Retrieval via Dynamic Multi-Task Regression](https://reader036.vdocuments.site/reader036/viewer/2022081515/568161dd550346895dd1ecd6/html5/thumbnails/10.jpg)
10
Algorithm
Regularized multi-task regression on multivariate point process
• Goal: Scalably learn temporal models for each topic keyword.
• Multi-task framework: allows multiple image descriptors.
• Several regularization schemes
• Personalization by locally-weighted learning
![Page 11: Time-Sensitive Web Image Ranking and Retrieval via Dynamic Multi-Task Regression](https://reader036.vdocuments.site/reader036/viewer/2022081515/568161dd550346895dd1ecd6/html5/thumbnails/11.jpg)
11
Thank you !Stop by our poster!
![Page 12: Time-Sensitive Web Image Ranking and Retrieval via Dynamic Multi-Task Regression](https://reader036.vdocuments.site/reader036/viewer/2022081515/568161dd550346895dd1ecd6/html5/thumbnails/12.jpg)
12
Multivariate Point Process Models
Given a stream of hornet pictures up to T
Clustering by descriptor 1 Clustering by descriptor 2
Time t1 t2 t3 t5 t6 t7 t9 t10
1st descriptor (v1) 2nd descriptor (v2)
![Page 13: Time-Sensitive Web Image Ranking and Retrieval via Dynamic Multi-Task Regression](https://reader036.vdocuments.site/reader036/viewer/2022081515/568161dd550346895dd1ecd6/html5/thumbnails/13.jpg)
13
Regularized GLM on Point Processes
Given a stream of hornet pictures up to T
Time t1 t2 t3 t5 t6 t7 t9 t10
Formulate a regression between occurrence rates and covariates.
Covariates: any likely factors to be associated with image occurrence (ex. Time, season, and other external events)
Compute sparse regularized MLE solutions For each visual cluster, we select only a small number of strong factors.
(v1,v2) (3, 2) (3, 2) (3, 2) (2, 1) (1, 3) (1, 4) (2, 3) (2, 1)
![Page 14: Time-Sensitive Web Image Ranking and Retrieval via Dynamic Multi-Task Regression](https://reader036.vdocuments.site/reader036/viewer/2022081515/568161dd550346895dd1ecd6/html5/thumbnails/14.jpg)
14
A Toy Example of Image Reranking
Peaked in summer
(Aquarium)
(Sea tour)
(Ice hockey) Peaked in January
Covariates: only year and months
Stationary