cs131 panoramic image stitching - stanford computer vision lab
TRANSCRIPT
PA1 Session 18-Oct-13Jiayuan Ma
CS131Panoramic Image Stitching
Jiayuan Ma18-Oct-13
1
Thursday, October 17, 13
PA1 Session 18-Oct-13Jiayuan Ma
Agenda• Objective• Main flow• Skeleton code• Results
2
Thursday, October 17, 13
PA1 Session 18-Oct-13Jiayuan Ma
Objective
3
Multiple images into one panorama!
Thursday, October 17, 13
PA1 Session 18-Oct-13Jiayuan Ma
Main Flow
• Detect key points
4
Thursday, October 17, 13
PA1 Session 18-Oct-13Jiayuan Ma
Detect Key Points
5
Thursday, October 17, 13
PA1 Session 18-Oct-13Jiayuan Ma
Skeleton Code• Detect key points (Done for you!)– Under KeypointDetect
6
[feature, DoG pyr, Gaussian pyr] = detect_features(input image)
Thursday, October 17, 13
PA1 Session 18-Oct-13Jiayuan Ma
Main Flow
• Detect key points• Build the SIFT descriptors
7
(v1, v2, . . . , v128)(u1, u2, . . . , u128)
Thursday, October 17, 13
PA1 Session 18-Oct-13Jiayuan Ma
Build the SIFT Descriptors
8
This is just an illustration!
Thursday, October 17, 13
PA1 Session 18-Oct-13Jiayuan Ma
Skeleton Code• Build the SIFT descriptors– Read this paper http://www.cs.ubc.ca/~lowe/papers/
ijcv04.pdf first! • Input– Gaussian pyramid– key point location– key point scale index
• Output– A set of 128-dim vectors
9
Thursday, October 17, 13
PA1 Session 18-Oct-13Jiayuan Ma
Skeleton Code• Build the SIFT descriptors (30 lines of code)– Compute gradient magnitude and orientation– For each key point• Find a patch (tricky round-off)• Compute orientation of the patch• Build the histogram (edge case)
10
Thursday, October 17, 13
PA1 Session 18-Oct-13Jiayuan Ma
Main Flow
• Detect key points• Build the SIFT descriptors• Match SIFT descriptors
11
(v1, v2, . . . , v128)(u1, u2, . . . , u128)
Thursday, October 17, 13
PA1 Session 18-Oct-13Jiayuan Ma
Match SIFT Descriptors• Euclidean distance between descriptors
12
Thursday, October 17, 13
PA1 Session 18-Oct-13Jiayuan Ma
Skeleton Code• Match SIFT descriptors (6 lines of code)– Input: D1, D2, thresh (default 0.7)– Output: match [D1’s index, D2’s index]– Try to use one for loop– Useful command• repmat• sort
13
Thursday, October 17, 13
PA1 Session 18-Oct-13Jiayuan Ma
Main Flow
• Detect key points• Build the SIFT descriptors• Match SIFT descriptors• Fitting the transformation
14
(v1, v2, . . . , v128)(u1, u2, . . . , u128)
T =
2
4t11 t12 t13t21 t22 t230 0 1
3
5
Thursday, October 17, 13
PA1 Session 18-Oct-13Jiayuan Ma
Fitting the transformation• 2D transformations
15
Thursday, October 17, 13
PA1 Session 18-Oct-13Jiayuan Ma
Skeleton Code• Fit the transformation matrix
• Six variables– each point give two equations– at least three points
• Least squares
16
H =
2
4h11 h12 h13
h21 h22 h23
0 0 1
3
5
Thursday, October 17, 13
PA1 Session 18-Oct-13Jiayuan Ma
Main Flow
• Detect key points• Build the SIFT descriptors• Match SIFT descriptors• Fitting the transformation• RANSAC
17
(v1, v2, . . . , v128)(u1, u2, . . . , u128)
Thursday, October 17, 13
PA1 Session 18-Oct-13Jiayuan Ma
RANSAC• A further refinement of matches
18
Thursday, October 17, 13
PA1 Session 18-Oct-13Jiayuan Ma
Skeleton Code• RANSAC– ComputeError
19
�����
2
4x2
y21
3
5�H
2
4x1
y11
3
5�����2
Thursday, October 17, 13
PA1 Session 18-Oct-13Jiayuan Ma
Main Flow
• Detect key points• Build the SIFT descriptors• Match SIFT descriptors• Fitting the transformation• RANSAC
20
(v1, v2, . . . , v128)(u1, u2, . . . , u128)
Thursday, October 17, 13
PA1 Session 18-Oct-13Jiayuan Ma
Image Stitching• Almost done for you
• Recall from PS0– imtransform–maketform
21
Thursday, October 17, 13
PA1 Session 18-Oct-13Jiayuan Ma
Skeleton Code• Multiple Stitch (2 lines of code)– A simplified case of real-world scenario– Transformation is associative and invertible– Useful command• pinv
22
Thursday, October 17, 13
PA1 Session 18-Oct-13Jiayuan Ma
Skeleton Code• Tester.m– Scripts that help you to get started
• Evaluate.m– Scripts that tests your solution• Load fixed input from checkpoint• Run your implementation• Compare results with reference solution
23
Thursday, October 17, 13
PA1 Session 18-Oct-13Jiayuan Ma
Requirement• Due Date: 5pm Oct 28, 2013• Electronic submission only– [email protected]
• Code + Report– SIFT invariance and why it helps– DoG v.s. Dense SIFT– Why RANSAC– Your own stitches– Error discussion
24
Thursday, October 17, 13
PA1 Session 18-Oct-13Jiayuan Ma
Results
25
Thursday, October 17, 13
PA1 Session 18-Oct-13Jiayuan Ma
Results
26
Thursday, October 17, 13
PA1 Session 18-Oct-13Jiayuan Ma
Some Advice• About choosing your own images
• Questions?
27
Thursday, October 17, 13