combining shape from shading and stereo: a variational ...andrés bruhn [email protected]...
TRANSCRIPT
Combining Shape from Shading and Stereo: A Variational Approach for theJoint Estimation of Depth, Illumination and Albedo
Daniel [email protected]
Institute for Visualization and Interactive SystemsUniversity of Stuttgart, Germany
Yong Chul [email protected]
Institute for Visualization and Interactive SystemsUniversity of Stuttgart, Germany
Michael Breuß[email protected]
Institute for Applied Mathematics and ScientificComputing, BTU Cottbus-Senftenberg, Germany
Andrés [email protected]
Institute for Visualization and Interactive SystemsUniversity of Stuttgart, Germany
Shape from shading (SfS) and stereo are twofundamentally different strategies for image-based 3-D reconstruction. While approaches forSfS infer the depth solely from pixel intensities,methods for stereo are based on finding corre-spondences across images.
In this paper we propose a joint variationalmethod that combines the advantages of bothstrategies. By integrating recent stereo andSfS models into a single minimisation frame-work, we obtain an approach that exploits shad-ing information to improve upon the recon-struction quality of robust stereo methods. Tothis end, we fuse a Lambertian SfS approachwith a robust stereo model and supplementthe resulting energy functional with a detail-preserving anisotropic second-order smoothnessterm. Moreover, we extend the novel model insuch a way that it jointly estimates depth, albedoand illumination. This in turn makes the ap-proach applicable to objects with non-uniformalbedo as well as to scenes with unknown illu-mination.
Experiments for synthetic and real-worldimages show the advantages of our combinedapproach: While the stereo part overcomes thealbedo-depth ambiguity inherent to all SfS meth-ods, the SfS part improves the degree of detailsof the reconstruction compared to pure stereomethods. An example of the reconstructionquality of our combined approach using onlytwo views is given in Figure 1. As one can see,the reconstructed depth is quite detailed. More-over, the computed illumination direction as wellas the estimated albedo look reasonable.
[1] C. Strecha, W. von Hansen, L. Van Gool, P. Fua,and U. Thoennessen. On benchmarking cameracalibration and multi-view stereo for high resolu-tion imagery. In Proc. IEEE Conference on Com-puter Vision and Pattern Recognition, 2008.
Figure 1: Two-view results for the Fountain-P11[1]. Top to bottom: Reference image, computeddepth (shaded reconstruction), computed illumi-nation direction, computed albedo.