window-based approach for fast stereo correspondence
DESCRIPTION
Window-based Approach For Fast Stereo Correspondence. Raj Kumar Gupta, Siu-Yeung Cho IET Computer Vision, 2013. Outline. Introduction Related Work Proposed Method Experimental Results Conclusion. Introduction. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/1.jpg)
1
Window-based Approach For Fast Stereo Correspondence
Raj Kumar Gupta, Siu-Yeung ChoIET Computer Vision, 2013
![Page 2: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/2.jpg)
2
Outline•Introduction•Related Work•Proposed Method•Experimental Results•Conclusion
![Page 3: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/3.jpg)
3
Introduction•Using two correlation windows to improve the performance of the algorithm •3*3 and 9*9•Real-time suitability •more than 10 frame/s on CPU in case of 320 × 240-sized image pair with disparity value 16
![Page 4: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/4.jpg)
4
Related Work•Local methods are usually base on correlation.•Area-based (NCC, SAD, SSD)•Feature-based: rely on feature extraction and match local cues (BF, GF)
•Bigger window size, more information, more blurred.
![Page 5: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/5.jpg)
5
Outline•Introduction•Related Work•Proposed Method•Experimental Results•Conclusion
![Page 6: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/6.jpg)
6
Flow Chart
![Page 7: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/7.jpg)
7
Flow Chart
![Page 8: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/8.jpg)
8
Initial Matching•Matching cost computation: SAD
Left Right
d
![Page 9: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/9.jpg)
9
Problem in disparity selection
a. Determine disparity easily for unique minimum valueb. Ambiguous disparity in case of multiple minimac. Matching cost calculated at point (205, 230) of Tsukuba image
![Page 10: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/10.jpg)
10
Initial Matching: large correlation window
•Matching cost computation: SAD + penalty
•Penalty term
• Disparity computation
![Page 11: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/11.jpg)
11
Problem in disparity selection
![Page 12: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/12.jpg)
12
Initial Matching: small correlation window
•Only those disparity values that are carried by neighbouring pixels.•Matching cost computation without penalty
•N: the disparity values of the neighbouring pixels.•Avoid local minima and speed up
![Page 13: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/13.jpg)
13
Flow Chart
![Page 14: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/14.jpg)
14
Unreliable pixel detection• left–right cross-checking
![Page 15: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/15.jpg)
15
Disparity Interpolation•Search for pixels with reliable disparity value in its eight neighbouring pixels.•Compute similarity of unreliable pixel and its reliable neighbor.
![Page 16: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/16.jpg)
16
Flow Chart
![Page 17: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/17.jpg)
17
Disparity Refinement
![Page 18: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/18.jpg)
18
Outline•Introduction•Related Work•Proposed Method•Experimental Results•Conclusion
![Page 19: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/19.jpg)
19
Experimental Results•Computation time of the proposed algorithm for different window sizes on Tsukuba image.(image size 384 × 288 with 16 disparity labels)
![Page 20: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/20.jpg)
20
• Percentage error in non-occluded (nocc), whole image (all) and near depth discontinuities (disc) for different window sizes for all four images (Tsukuba, Venus, Teddy and Cones)
![Page 21: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/21.jpg)
21
Experimental Results
• a. Without using small correlation window Ws and the disparity refinement step• b. Without using the disparity refinement step• c. Without using small correlation window Ws• d. With all four steps on Tsukuba image
![Page 22: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/22.jpg)
22
Experimental Results
![Page 23: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/23.jpg)
23
![Page 24: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/24.jpg)
24
![Page 25: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/25.jpg)
25
Experimental Results•Comparison the performance of the proposed algorithm with other correlation-based algorithms.
![Page 26: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/26.jpg)
26
Experimental Results
![Page 27: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/27.jpg)
27
Reference• [24] Gupta, R., Cho, S.-Y.: ‘Real-time stereo matching using adaptive binary window’
(3D Data Processing, Visualization and Transmission, 2010)• [25] Zhang, K., Lu, J., Lafruit, G., Lauwereins, R., Gool, L.V.: ‘Real-time accurate
stereo with bitwise fast voting on Cuda’. Int. Conf. Computer Vision Workshops, 2009, pp. 540–547• [26] Humenberger, M., Zinner, C., Weber, M., Kubinger, W., Vincze, M.:‘A fast
stereo matching algorithm suitable for embedded real-time systems’, Comput. Vis. Image Underst., 2010, 114, (11),pp. 1180–1202• [27] Gong, M., Yang, Y.: ‘Near real-time reliable stereo matching using
programmable graphics hardware’. IEEE Conf. Computer Vision and Pattern Recognition, 2005, pp. 924–931• [28] Richardt, C., Orr, D., Davies, I., Criminisi, A., Dodgson, N.: ‘Real-time
spatiotemporal stereo matching using the dual-cross-bilateral grid’. European Conf. Computer Vision, 2010, vol. 6313, pp. 510–523• [29] Ambrosch, K., Kubinger, W.: ‘Accurate hardware-based stereo vision’,Comput.
Vis. Image Underst., 2010, 114, (11), pp. 1303–1316
![Page 28: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/28.jpg)
28
![Page 29: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/29.jpg)
29
![Page 30: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/30.jpg)
30
![Page 31: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/31.jpg)
31
![Page 32: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/32.jpg)
32
![Page 33: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/33.jpg)
33
![Page 34: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/34.jpg)
34
Conclusion•A new correlation-based stereo-matching approach.• Large window improves at non-textured image regions• Small window improves at depth discontinuities
•The CPU implementation computes at a speed of more than 10 frame/s.•Easily implemented on GPU. •The proposed method can be used in real-time applications to reconstruct the 3D structures with great accuracy at object boundaries.
![Page 35: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/35.jpg)
35
Codebook based Stereo Matching for Natural User
Interface Sung-il Kang and Hyunki Hong
2013 IEEE International Conference on Consumer Electronics (ICCE)
![Page 36: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/36.jpg)
36
Outline•Introduction•Proposed Method•Experimental Results•Conclusion
![Page 37: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/37.jpg)
37
Introduction•Interactive user interface has been one of the major topics in consumer electronics.•Gesture based user interface• Interactive smart TV, Nintendo Wii, Sony PlayStation3 Move, and Microsoft Kinect.
•Propose a stereo system implemented on GPGPU for real-time performance.•Employ codebook to solve occlusion.
![Page 38: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/38.jpg)
38
Flow chart
![Page 39: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/39.jpg)
39
Proposed Method•Pre-processing• Laplace od Gaussian (LoG) filter for alleviating the lighting effects.
•Cost initialization• AD+Census[6]
[6] X. Mei, X. Sun, M. Zhou, H. Wang, and X. Zhang, “On building an accurate stereo matchng system on graphics hardware,” Proc. of GPUCV, pp. 467-474, 2011. http://www.camdemy.com/media/4724
![Page 40: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/40.jpg)
40
Proposed Method•Cost aggregation[6]
• Cross-based aggregation• Color similarity and the length constraint
•Refinement[6,7]
• Left-right consistency check• Iterative region voting• Sub-pixel enhancement
d
[7] Q. Yang, C. Engels, R. Yang, H. Stewenius, and D. Nister, “Stereo matching with color-weighted correlation, hierarchical belief propagation and occlusion handling,” IEEE Transactions on PAMI, 2009.
![Page 41: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/41.jpg)
41
Proposed Method
Occlusion? Find codeword
UpdatecodewordCodeword?
Yes
No
Yes
No
Add a newcodeword
![Page 42: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/42.jpg)
42
Experimental Results• Device: Intel Quad 2.66GHz with Nvidia GTX460. • Stereo images are captured by a Bumblebee 3 from Point Grey Inc. • Time: 80~110ms/frame• Stereo matching is implemented on GPU.• The codebook generation and its evaluation is on CPU.
![Page 43: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/43.jpg)
43
Experimental Results
![Page 44: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/44.jpg)
44
Experimental Results
[8] K. J. Yoon and I. S. Kweon, “Adaptive support-weight approach for correspondence search,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, pp. 650-656, 2005.[9] C. Richardt, D Orr, I Davies, and A Criminisi, “Real-time spatiotemporal stereo matching using the dual-cross-bilateral grid,” Proc. of ECCV, 2010.
![Page 45: Window-based Approach For Fast Stereo Correspondence](https://reader036.vdocuments.site/reader036/viewer/2022062410/56815aad550346895dc84e47/html5/thumbnails/45.jpg)
45
Conclusion•Propose a stereo system implemented on GPGPU for real-time performance.•Good performance at static background Only.