improved census transforms for resource-optimized stereo vision
DESCRIPTION
Improved Census Transforms for Resource-Optimized Stereo Vision. Wade S. Fife, Member, IEEE, James K. Archibald, Senior Member, IEEE. IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 23, NO. 1, JANUARY 2013. Outline. Introduction Related Work Proposed Algorithm - PowerPoint PPT PresentationTRANSCRIPT
Improved Census Transforms for Resource-Optimized Stereo Vision
Wade S. Fife, Member, IEEE, James K. Archibald, Senior Member, IEEE
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 23, NO. 1, JANUARY 2013
2
Outline• Introduction• Related Work• Proposed Algorithm
• Sparse Census Transform• Generalized Census Transform• Hardware Implementation
• Experimental Results• Conclusion
3
Introduction
4
Introduction• The challenges:
• The enormous amount of computation required to identify the corresponding points in the images.
• It is critical to…
• maximize the accuracy and throughput of the stereo system • while minimizing the resource requirements
5
Objective• Propose the sparse census transforms :
• Reduce the resource requirements of census-based systems• Maintain correlation accuracy
• Propose the generalized census transforms :
• A new class of census-like transforms • Increase the robustness and flexibility
6
Related Work
7
Related Work• Census Transform :
• Color• Gradient
8
Related Work• After aggregation step:
Census on colors Census on gradients
9
Related Work• Sparse census[6] :
• Half of the bits
X
[6] C. Zinner, M. Humenberger, K. Ambrosch, and W. Kubinger, “An optimized software-based implementation of a census-based stereo matching algorithm,” in Proc. 4th ISVC, 2008, pp. 216–227.
The computation costs for the hamming distances are quite large.
10
Related Work• Mini-census[8] :
X
[8] N.-C. Chang, T.-H. Tsai, B.-H. Hsu, Y.-C. Chen, and T.-S. Chang,“Algorithm and architecture of disparity estimation with mini-census adaptive support weight,” IEEE Trans. Circuits Syst. Video Technol., vol. 20, no. 6, pp. 792–805, Jun. 2010.
11
Related Work• Mini-census[8] :
• Mini-census adaptive support weight
[8] N.-C. Chang, T.-H. Tsai, B.-H. Hsu, Y.-C. Chen, and T.-S. Chang,“Algorithm and architecture of disparity estimation with mini-census adaptive support weight,” IEEE Trans. Circuits Syst. Video Technol., vol. 20, no. 6, pp. 792–805, Jun. 2010.
12
Related Work• Mini-census[8] :
13
ProposedAlgorithm
14
Sparse Census Transform• Definition :
• N: the set of points within a T T window around p• : a new set of N•
P’
P
15
Transform Point Selection• Goal : minimize the size of the census transform vector
• Challenge: Must quantify how much each point in the transform window contributes to overall correlation accuracy
• Test correlation accuracy:
• Define a sparse census transform consisting of a single point (| | = 1)• Determine how consistently this point leads to correct correlation• 13 13 correlation window (aggregation)
16
Transform Point Selection• Go
Tsukuba Venus Average
Teddy Cones
Bright: Higher correlation accuracy
25 25 neighborhood
18
Transform Point Selection• Further from the center : value decreasing
• Very near the center : less effective
• It is best to choose points that are neither too far from nor too close to the center pixel.
• Optimal distance : 2 pixels• If the image is noisy should be slightly further
from the center
19
Transform Point Selection•
Tsukuba Venus Average
Teddy Cones
Bright: Higher correlation accuracy
37 37 neighborhood
Tsukuba Venus
Teddy Cones
With Gaussian noise( = 5.12)
20
Proposed Sparse Census Transform• Very good correlation accuracy can be achieved using very sparse transforms.
16-point 12-point 8-point
4-point 2-point 1-point
21
Experimental Results
22
Generalized Census Transform• Goal : greater freedom in choosing the census transform design
• Definition : redrawing the transform as a graph
3 3 census
3 3 correlation(aggregation)
23
Generalized Census Transform• As..
• (1)transform neighborhoods become more and more sparse• (2)fewer pixels are used in the correlation process
• selection of points to include in the transform becomes more critical
2-point 2-edge
Horizontal + Vertical
24
Generalized Census Transform
symmetric
25
Proposed Generalized Census Transform• Benefits :
• Often require a smaller census transform window (memory)• Increased robustness under varying conditions (noise)
16-edge 12-edge 8-edge
4-edge 2-edge 1-edge
26
Experimental Results
27
Experimental Results
28
Hardware Implementation• Pipelining : to increase throughput in an FPGA implementation
(Field Programmable Gate Array)
Range : 0~3
3 2 1 0
3 2 1 0
3 2 1 0
3 2 1 0
3 2 1 0
One input pixel per clock cycle &Output one disparity result per clock cycle
29
Hardware Implementation• Correlation window sum (Aggregation) :
30
ExperimentalResults
12-edge 4-edgeFull 7x7 censusGround TruthLeft Image
12-edge 4-edgeFull 7x7 censusGround TruthLeft Image
12-edge 4-edgeFull 7x7 censusLeft Image
34
Experimental Results
LUTs (look-up tables) : the amount of logic required to implement the methodFFs : the number of 1-bit registers (the amount of pipelining used)RAMs : the number of 18-kbit block memoriesFreq. : the maximum operating frequency reported by synthesis
𝟖𝟖% ↓ 𝟔𝟏% ↓
35
Conclusion
36
Conclusion• Proposed and analyzed in this paper:
• A range of sparse census transforms
• reduce hardware resource requirements• attempting to maximize correlation accuracy.• often better than or nearly as good as the full census
• Generalized census transforms
• increased robustness in the presence of image noise