leveraging stereopsis for saliency analysis yuzhen niu † yujie geng † xueqing li ‡ feng liu...
TRANSCRIPT
![Page 1: Leveraging Stereopsis for Saliency Analysis Yuzhen Niu † Yujie Geng † Xueqing Li ‡ Feng Liu † † Department of Computer Science Portland State University](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649ca45503460f94964c25/html5/thumbnails/1.jpg)
Leveraging Stereopsis for Saliency Analysis
Yuzhen Niu† Yujie Geng† Xueqing Li‡ Feng Liu†
†Department of Computer SciencePortland State University Shandong University
Portland, OR, 97207 USA ‡School of Computer Science and Technology
Jinan, Shandong, 250101 China
![Page 2: Leveraging Stereopsis for Saliency Analysis Yuzhen Niu † Yujie Geng † Xueqing Li ‡ Feng Liu † † Department of Computer Science Portland State University](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649ca45503460f94964c25/html5/thumbnails/2.jpg)
Outline
• Introduction• Stereo Saliency• Experiments• Conclusion
![Page 3: Leveraging Stereopsis for Saliency Analysis Yuzhen Niu † Yujie Geng † Xueqing Li ‡ Feng Liu † † Department of Computer Science Portland State University](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649ca45503460f94964c25/html5/thumbnails/3.jpg)
Introduction
• The performance of saliency analysis methods depends on feature contrast.
• When an object does not exhibit distinct visual features, it becomes challenging for saliency detection.
• Stereopsis provides an additional depth cue and plays an important role in the human vision system.
![Page 4: Leveraging Stereopsis for Saliency Analysis Yuzhen Niu † Yujie Geng † Xueqing Li ‡ Feng Liu † † Department of Computer Science Portland State University](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649ca45503460f94964c25/html5/thumbnails/4.jpg)
Introduction
![Page 5: Leveraging Stereopsis for Saliency Analysis Yuzhen Niu † Yujie Geng † Xueqing Li ‡ Feng Liu † † Department of Computer Science Portland State University](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649ca45503460f94964c25/html5/thumbnails/5.jpg)
Introduction
• Two approach– Computes stereo saliency based on the global
disparity contrast in the input image.– Leverages domain knowledge in stereoscopic
photography.
![Page 6: Leveraging Stereopsis for Saliency Analysis Yuzhen Niu † Yujie Geng † Xueqing Li ‡ Feng Liu † † Department of Computer Science Portland State University](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649ca45503460f94964c25/html5/thumbnails/6.jpg)
Stereo Saliency
• Stereo saliency analysis works on disparity map.
• We apply the SIFT flow method to disparity estimation for its robustness.
![Page 7: Leveraging Stereopsis for Saliency Analysis Yuzhen Niu † Yujie Geng † Xueqing Li ‡ Feng Liu † † Department of Computer Science Portland State University](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649ca45503460f94964c25/html5/thumbnails/7.jpg)
Stereo Saliency
• Stereo Saliency from Disparity Contrast– Extend a recent color contrast-based saliency
detection method[3] for disparity contrast analysis.
[3] M. Cheng, G. Zhang, N. J. Mitra, X. Huang, and S. Hu. Global contrast based salient region detection. In IEEE CVPR, pages 409–416, 2011.
![Page 8: Leveraging Stereopsis for Saliency Analysis Yuzhen Niu † Yujie Geng † Xueqing Li ‡ Feng Liu † † Department of Computer Science Portland State University](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649ca45503460f94964c25/html5/thumbnails/8.jpg)
Stereo Saliency
• Stereo Saliency from Disparity Contrast
w(p, q): the spatial distance between p and q
dv(p, q): the disparity difference between pixel p and q
![Page 9: Leveraging Stereopsis for Saliency Analysis Yuzhen Niu † Yujie Geng † Xueqing Li ‡ Feng Liu † † Department of Computer Science Portland State University](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649ca45503460f94964c25/html5/thumbnails/9.jpg)
Stereo Saliency
![Page 10: Leveraging Stereopsis for Saliency Analysis Yuzhen Niu † Yujie Geng † Xueqing Li ‡ Feng Liu † † Department of Computer Science Portland State University](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649ca45503460f94964c25/html5/thumbnails/10.jpg)
Stereo Saliency
• Domain Knowledge Assisted Saliency Analysis– Unique features of stereoscopic photography give
useful cues for saliency analysis.
– Two rules to compute the stereo saliency.1. Objects with small disparity magnitudes (e.g. in the comfort
zone) tend to be salient.2. Objects popping out from the screen tend to be salient.
![Page 11: Leveraging Stereopsis for Saliency Analysis Yuzhen Niu † Yujie Geng † Xueqing Li ‡ Feng Liu † † Department of Computer Science Portland State University](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649ca45503460f94964c25/html5/thumbnails/11.jpg)
Stereo Saliency
![Page 12: Leveraging Stereopsis for Saliency Analysis Yuzhen Niu † Yujie Geng † Xueqing Li ‡ Feng Liu † † Department of Computer Science Portland State University](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649ca45503460f94964c25/html5/thumbnails/12.jpg)
Stereo Saliency
• Follow Rule 1– Assign big saliency values to regions with small
disparity magnitudes.
![Page 13: Leveraging Stereopsis for Saliency Analysis Yuzhen Niu † Yujie Geng † Xueqing Li ‡ Feng Liu † † Department of Computer Science Portland State University](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649ca45503460f94964c25/html5/thumbnails/13.jpg)
Stereo Saliency
• Follow Rule 2– Objects with negative disparities are perceived
popping out from the screen.
![Page 14: Leveraging Stereopsis for Saliency Analysis Yuzhen Niu † Yujie Geng † Xueqing Li ‡ Feng Liu † † Department of Computer Science Portland State University](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649ca45503460f94964c25/html5/thumbnails/14.jpg)
Stereo Saliency
• Combine Rule 1 and 2– When an image only has negative disparities.• Rule 2 > Rule 1
– When an image has both negative and positive disparities.• Rule 1 > Rule 2
![Page 15: Leveraging Stereopsis for Saliency Analysis Yuzhen Niu † Yujie Geng † Xueqing Li ‡ Feng Liu † † Department of Computer Science Portland State University](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649ca45503460f94964c25/html5/thumbnails/15.jpg)
Stereo Saliency
• Combine Rule 1 and 2
![Page 16: Leveraging Stereopsis for Saliency Analysis Yuzhen Niu † Yujie Geng † Xueqing Li ‡ Feng Liu † † Department of Computer Science Portland State University](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649ca45503460f94964c25/html5/thumbnails/16.jpg)
Stereo Saliency
• Modulate the original saliency map with local contrast-based saliency analysis– Disparities change little in each row in some of the
background areas.
![Page 17: Leveraging Stereopsis for Saliency Analysis Yuzhen Niu † Yujie Geng † Xueqing Li ‡ Feng Liu † † Department of Computer Science Portland State University](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649ca45503460f94964c25/html5/thumbnails/17.jpg)
Stereo Saliency
![Page 18: Leveraging Stereopsis for Saliency Analysis Yuzhen Niu † Yujie Geng † Xueqing Li ‡ Feng Liu † † Department of Computer Science Portland State University](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649ca45503460f94964c25/html5/thumbnails/18.jpg)
Stereo Saliency
• Stereo Saliency Map
![Page 19: Leveraging Stereopsis for Saliency Analysis Yuzhen Niu † Yujie Geng † Xueqing Li ‡ Feng Liu † † Department of Computer Science Portland State University](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649ca45503460f94964c25/html5/thumbnails/19.jpg)
Experiments
• Stereoscopic Image Database– Three users are asked to enclose the most salient
object in each image with a rectangle.– Ask a user to manually segment the salient
object(s) in each of the images.
![Page 20: Leveraging Stereopsis for Saliency Analysis Yuzhen Niu † Yujie Geng † Xueqing Li ‡ Feng Liu † † Department of Computer Science Portland State University](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649ca45503460f94964c25/html5/thumbnails/20.jpg)
Experiments
• Performance Evaluation
![Page 21: Leveraging Stereopsis for Saliency Analysis Yuzhen Niu † Yujie Geng † Xueqing Li ‡ Feng Liu † † Department of Computer Science Portland State University](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649ca45503460f94964c25/html5/thumbnails/21.jpg)
Experiments
![Page 22: Leveraging Stereopsis for Saliency Analysis Yuzhen Niu † Yujie Geng † Xueqing Li ‡ Feng Liu † † Department of Computer Science Portland State University](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649ca45503460f94964c25/html5/thumbnails/22.jpg)
Experiments
• Automatic Salient Object Segmentation
![Page 23: Leveraging Stereopsis for Saliency Analysis Yuzhen Niu † Yujie Geng † Xueqing Li ‡ Feng Liu † † Department of Computer Science Portland State University](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649ca45503460f94964c25/html5/thumbnails/23.jpg)
Experiments
• Limitations– The performance of our methods depend on the
quality of disparity maps.– Stereoscopic photography rules may conflict with
each other for some images.– Stereo saliency is useful only if a salient object
stays at a different depth than its surroundings.
![Page 24: Leveraging Stereopsis for Saliency Analysis Yuzhen Niu † Yujie Geng † Xueqing Li ‡ Feng Liu † † Department of Computer Science Portland State University](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649ca45503460f94964c25/html5/thumbnails/24.jpg)
Conclusion
• Developed two methods for stereo saliency detection.
• Stereo saliency is a useful complement to existing visual saliency analysis.