recap from previous lecture
DESCRIPTION
Recap from Previous Lecture. Tone Mapping Preserve local contrast or detail at the expense of large scale contrast. Changing the brightness within objects or surfaces unequally leads to halos. We are now transitioning to more geometric reasoning about light. Exam 1 Results. - PowerPoint PPT PresentationTRANSCRIPT
Slide 1
Recap from Previous LectureTone MappingPreserve local contrast or detail at the expense of large scale contrast.Changing the brightness within objects or surfaces unequally leads to halos.We are now transitioning to more geometric reasoning about light.Exam 1 ResultsAverage Grade: 80%Question 17 not graded.Project 2 Resultshttp://www.cs.brown.edu/courses/cs129/results/proj2/zyp/http://www.cs.brown.edu/courses/cs129/results/proj2/damoreno/Project 2 ResultsJonathon Mace
4Project 2 ResultsXiaofeng Tao
Project 2 ResultsChen Xu
Project 2 ResultsJonathan Koh
Project 2 ResultsMarvin Arroz
Stereo and Depth Estimation (Sz 11)cs129: Computational PhotographyJames Hays, Brown, Fall 2012Slides from Steve Seitz and Robert Collins
3d movies are trendy lately, but arguably they should be called stereo 3d movies because movies already have many 3d cues.9How is depth estimated?Time of Flight
Humans can learn to echolocate!
Stereo VisionNot that important for humans, especially at longer distances. Perhaps 10% of people are stereo blind.Many animals dont have much stereo overlap in their fields of view
Public Library, Stereoscopic Looking Room, Chicago, by Phillips, 1923
Teesta suspension bridge-Darjeeling, India
Woman getting eye exam during immigration procedure at Ellis Island, c. 1905 - 1920 , UCR Museum of Phography
Mark Twain at Pool Table", no date, UCR Museum of Photography
Stereoscene pointoptical centerimage planeStereoBasic Principle: TriangulationGives reconstruction as intersection of two raysRequires camera pose (calibration)point correspondenceStereo correspondenceDetermine Pixel CorrespondencePairs of points that correspond to same scene pointEpipolar ConstraintReduces correspondence problem to 1D search along conjugate epipolar linesepipolar planeepipolar lineepipolar lineFundamental matrixLet p be a point in left image, p in right image
Epipolar relationp maps to epipolar line l p maps to epipolar line l Epipolar mapping described by a 3x3 matrix F
It follows thatllpp
Fundamental matrixThis matrix F is calledthe Essential Matrixwhen image intrinsic parameters are knownthe Fundamental Matrixmore generally (uncalibrated case)
Can solve for F from point correspondencesEach (p, p) pair gives one linear equation in entries of F
8 points give enough to solve for F (8-point algorithm)see Marc Pollefeys notes for a nice tutorial
Stereo image rectification33Stereo image rectificationreproject image planes onto a commonplane parallel to the line between optical centerspixel motion is horizontal after this transformationtwo homographies (3x3 transform), one for each input image reprojectionC. Loop and Z. Zhang. Computing Rectifying Homographies for Stereo Vision. IEEE Conf. Computer Vision and Pattern Recognition, 1999.34Stereo matching algorithmsMatch Pixels in Conjugate Epipolar LinesAssume brightness constancyThis is a tough problemNumerous approachesA good survey and evaluation: http://www.middlebury.edu/stereo/ Your basic stereo algorithm
For each epipolar lineFor each pixel in the left imagecompare with every pixel on same epipolar line in right imagepick pixel with minimum match costImprovement: match windowsThis should look familar...
Window sizeSmaller window Larger window
W = 3W = 20Better results with adaptive windowT. Kanade and M. Okutomi, A Stereo Matching Algorithm with an Adaptive Window: Theory and Experiment,, Proc. International Conference on Robotics and Automation, 1991. D. Scharstein and R. Szeliski. Stereo matching with nonlinear diffusion. International Journal of Computer Vision, 28(2):155-174, July 1998
Effect of window sizesmaller window: more detail, more noisebigger window: less noise, more detailStereo results
Ground truthSceneData from University of TsukubaSimilar results on other images without ground truth
38Results with window search
Window-based matching(best window size)
Ground truthBetter methods exist...
Better MethodBoykov et al., Fast Approximate Energy Minimization via Graph Cuts, International Conference on Computer Vision, September 1999.Ground truth
For the latest and greatest: http://www.middlebury.edu/stereo/ Stereo as energy minimizationWhat defines a good stereo correspondence?Match qualityWant each pixel to find a good match in the other imageSmoothnessIf two pixels are adjacent, they should (usually) move about the same amount
Stereo as energy minimizationExpressing this mathematicallyMatch qualityWant each pixel to find a good match in the other image
SmoothnessIf two pixels are adjacent, they should (usually) move about the same amount
We want to minimizeThis is a special type of energy function known as an MRF (Markov Random Field)Effective and fast algorithms exist:Graph cuts, belief propagation.for more details (and code): http://vision.middlebury.edu/MRF/ Great tutorials available online (including video of talks)
Depth from disparityfxxbaselinezCCXf
Camera calibration errorsPoor image resolutionOcclusionsViolations of brightness constancy (specular reflections)Large motionsLow-contrast image regionsStereo reconstruction pipelineStepsCalibrate camerasRectify imagesCompute disparityEstimate depthWhat will cause errors?Active stereo with structured light
Project structured light patterns onto the objectsimplifies the correspondence problem
camera 2camera 1projector
camera 1projectorLi Zhangs one-shot stereoActive stereo with structured light
Laser scanningOptical triangulationProject a single stripe of laser lightScan it across the surface of the objectThis is a very precise version of structured light scanning
Digital Michelangelo Projecthttp://graphics.stanford.edu/projects/mich/
Laser scanned models
The Digital Michelangelo Project, Levoy et al.Laser scanned modelsThe Digital Michelangelo Project, Levoy et al.
Laser scanned models
The Digital Michelangelo Project, Levoy et al.Laser scanned models
The Digital Michelangelo Project, Levoy et al.Laser scanned models
The Digital Michelangelo Project, Levoy et al.