image stitching shangliang jiang kate harrison. what is image stitching?
TRANSCRIPT
Introduction
• Are you getting the whole picture?– Compact Camera FOV = 50 x 35°– Human FOV = 200 x 135°
Introduction
• Are you getting the whole picture?– Compact Camera FOV = 50 x 35°– Human FOV = 200 x 135°– Panoramic Mosaic = 360 x 180°
Recognizing Panoramas
• 2D Rotations (, )– Ordering matching images
• 1D Rotations ()– Ordering matching images
Recognizing Panoramas
• 1D Rotations ()– Ordering matching images
• 2D Rotations (, )– Ordering matching images
Recognizing Panoramas
• 1D Rotations ()– Ordering matching images
• 2D Rotations (, )– Ordering matching images
Overview
• Feature Matching– SIFT Features– Nearest Neighbor Matching
• Image Matching• Bundle Adjustment• Image Compositing• Conclusions
SIFT Features
• SIFT features are…– Geometrically invariant to similarity transforms,
• some robustness to affine change
– Photometrically invariant to affine changes in intensity
Overview
• Feature Matching– SIFT Features– Nearest Neighbor Matching
• Image Matching• Bundle Adjustment• Image Compositing• Conclusions
Nearest Neighbor Matching
• Find k nearest neighbors for each feature– k number of overlapping images (we use k = 4)
• Use k-d tree– k-d tree recursively bi-partitions data at mean in the
dimension of maximum variance– Approximate nearest neighbors found in O(nlogn)
Overview
• Feature Matching– SIFT Features– Nearest Neighbor Matching
• Image Matching• Bundle Adjustment• Image Compositing• Conclusions
Overview
• Feature Matching• Image Matching
– Random Sample Consensus (RANSAC) for Homography
– Probabilistic model for verification
• Bundle Adjustment• Image Compositing• Conclusions
Overview
• Feature Matching• Image Matching
– Random Sample Consensus (RANSAC) for Homography
– Probabilistic model for verification
• Bundle Adjustment• Image Compositing• Conclusions
Overview
• Feature Matching• Image Matching
– Random Sample Consensus (RANSAC) for Homography
– Probabilistic model for verification
• Bundle Adjustment• Image Compositing• Conclusions
Overview
• Feature Matching• Image Matching
– RANSAC for Homography– Probabilistic model for verification
• Bundle Adjustment• Image Compositing• Conclusions
Overview
• Feature Matching• Image Matching• Bundle Adjustment
– Error function
• Image Compositing• Conclusions
Overview
• Feature Matching• Image Matching• Bundle Adjustment
– Error function
• Image Compositing• Conclusions
Error function
• Sum of squared projection errors
– n = #images– I(i) = set of image matches to image i– F(i, j) = set of feature matches between images i,j
– rijk = residual of kth feature match between images i,j
• Robust error function
Overview
• Feature Matching• Image Matching• Bundle Adjustment
– Error function
• Image Compositing• Conclusions
Open questions
• Advanced camera modeling– radial distortion, camera motion, scene motion,
vignetting, exposure, high dynamic range, flash …
• Full 3D case – recognizing 3D objects/scenes in unordered datasets
Credits
• Automatic Panoramic Image Stitching Using Invariant Features, 2007– Matthew Brown and David G. Lowe (Uni. of British
Columbia)
• Recognising Panoramas, 2003– Matthew Brown and David G. Lowe (Uni. of British
Columbia)– 2003– Thanks for the slides!
• Image Alignment and Stitching: A Tutorial, 2006– Richard Szeliski (Microsoft)