recod @ mediaeval 2014: diverse social images retrieval
TRANSCRIPT
Recod @ MediaEval 2014:Diverse Social Images Retrieval
Rodrigo T. Calumby, Vinícius P. Santana,Felipe S. Cordeiro, Otávio A. B. Penatti,
Lin T. Li, Giovani Chiachia, Ricardo da S. Torres
[email protected][rtcalumby, vpsantana, fscordeiro]@ecomp.uefs.br
[lintzyli, chiachia, rtorres]@ic.unicamp.br
UEFS
Acknowledgments: UEFS/PROBIC, Samsung Research Institute Brazil, CNPq, and FAPESP (2013/11359-0).
Face detectionGeographic
Visual
inputlist
outputlist
Credibility
Clustering
Selection
Filtering
Re-rankingDiversification
PROPOSED APPROACH
inputlist
outputlist
Filtering
Re-ranking
Diversification
Geo Filter
Face FilterNumFacesFilter> 1 face → non-relevant
(location: Christ the Redeemer, Rio de Janeiro)
1-NN Classifier
Features1) number of faces2) biggest face size 3) smallest face size 4) average face size 5) total face size
Validation (devset) leave-one-location-out
Runs (testset)target: full devset
Test Image:
kNN
10km radius limit from reference lat/long of the location
(location: Iguazu Falls, Brazil/Argentina)
FaceClassifierFilter
Filtering
Re-ranking
Diversification
Visual CredibilityCredScore:
visualScore X faceProportion X tagSpecificity
Original Re-ranked
(location: Casa Batlló, Barcelona)
Location representatives:
Original Re-ranked
RelScore (nth image):
Finalcore: CredScore x RelScore
1
√n+14
inputlist
outputlist
Filtering
Re-ranking
Diversification
Clustering
Selection- Descending cluster size- Most relevant item from each cluster
(location: Arc de Triomphe, Paris)
- kMedoids: 50 clusters- Initial medoids: rank offset positions
Output list
RESULTS – OFFICIAL MEASURES
Run Filtering Re-ranking Diversification P@20 CR@20 F1@20
1GeoFilter and NumFacesFilter
-kMedoids (BoVWsparse
max +
HOG)0.7130 0.4030 0.5077
2GeoFilter and NumFacesFilter
- kMedoids (Cosine) 0.6976 0.4139 0.5133
3GeoFilter and NumFacesFilter
Visual re-ranking (CM3x3 + HOG + BIC)
kMedoids (BoVWsparsemax
+
HOG + Cosine)0.7016 0.4177 0.5168
4GeoFilter and NumFacesFilter
Visual re-ranking (CM3x3 + HOG + BIC) and Credibility
re-rankingkMedoids (CN3x3) 0.7598 0.4288 0.5423
5GeoFilter and
FaceClassifierFilter
Visual re-ranking (CM3x3 + HOG + BIC) and Credibility
re-rankingkMedoids (CN3x3) 0.7407 0.4076 0.5206
CONCLUSIONS
- Geographic and face-based filtering improved precision.
- The simple NumFacesFilter outperformed the FaceClassifierFilter.
- Visual and credibility re-ranking improved precision.
- kMedoids outperfomed MMR on devset and was applied for the runs.
- Inter and inner cluster sorting were effective for improving relevance.
Recod @ MediaEval 2014:Diverse Social Images Retrieval
Thank you!
[email protected][rtcalumby, vpsantana, fscordeiro]@ecomp.uefs.br
[lintzyli, chiachia, rtorres]@ic.unicamp.br
UEFS
Acknowledgments: UEFS/PROBIC, Samsung Research Institute Brazil, CNPq, and FAPESP (2013/11359-0).
Rodrigo T. Calumby, Vinícius P. Santana, Felipe S. Cordeiro, Otávio A. B. Penatti, Lin T. Li, Giovani Chiachia, Ricardo da S. Torres