WHERE SHOULD SALIENCY MODELS LOOK NEXT ?
Zoya Bylinskii, Adrià Recasens, Ali Borji, Aude Oliva, Antonio Torralba, Frédo Durand “Where Should Saliency Models Look Next?”, ECCV2016
Hello!I am Junting PanI am here because I love to give presentations. You can find me at [email protected]
Saliency mapSaliency map is a probability distribution map, that describe where human observers look in images.
It can provide important clues to human image understanding :- Main focus- Action or event- Participants
Regions of interest to human
Breakthroughs because of ….
× Prediction score increment has benn stable for looong time since ..
× CNN comes !!× End to end manner : feat. extraction, feat.
integration & saliency prediction.
Breakthroughs because of ….
× Prediction score increment has benn stable for looong time since ..
× CNN comes !!× End to end manner : feat. extraction, feat.
integration & saliency prediction.
Evaluation scores h
ave begun
to saturate
A picture is worth a thousand words
A complex idea can be conveyed with just a single still image, namely making it possible to absorb large amounts of data quickly.
SALICON modelCNN applied at 2 different image scales : small & BIG
BEST MODELS AT mit benchmarkDeepFixFCN built on top of the VGG.
10 MOST REPRESENTATIVE images
0,97 of Spearman correlation relative to their ranking on all dataset images
Name all image regions under the fixation map
95 percentile threshold
651 regions over 300 images
20 users and 2 MTurk task
Face saliency is underestimated when faces are small, non-frontal, or not centered in an image
Sometimes the actions in a scene are more salient to human observers than the participants, but saliency models can overestimate the relative saliency of the faces
Not all people are equally important
× Assign importance score to each face (using fixation gt and predicted map.
× Relative ordering assign by saliency model does not match by the importance given by human fixations.
Understanding the text ...
× The description of a warning or a book are more informative to observers than the warning or book title..
× Only piece of English text..
Objects and action
× Objects of gaze and/or action are usually missed
× Detecting objects of action remains a problem area for saliency model..
Conclusion
Models continue to under-predict crucial image regions containing people, actions, and text.
These are the regions with greatest semantic importance in an image, and become essential for saliency applications