computer vision: models, learning and inference chapter 15 models for transformations
Post on 31-Mar-2015
227 Views
Preview:
TRANSCRIPT
Computer vision: models, learning and inference
Chapter 15 Models for transformations
2
Structure
2Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Transformation models• Learning and inference in transformation models• Three problems– Exterior orientation– Calibration– Reconstruction
• Properties of the homography• Robust estimation of transformations• Applications
3
Transformation models
3Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Consider viewing a planar scene• There is now a 1 to 1 mapping between points
on the plane and points in the image• We will investigate models for this 1 to 1
mapping– Euclidean– Similarity– Affine transform– Homography
4
Motivation: augmented reality tracking
4Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
5
• Consider viewing a fronto-parallel plane at a known distance D.
• In homogeneous coordinates, the imaging equations are:
Euclidean Transformation
5Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
3D rotation matrix becomes 2D (in plane)
Plane at known distance D
Point is on plane (w=0)
6
Euclidean transformation
6Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Simplifying
• Rearranging the last equation
7
Euclidean transformation
7Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Simplifying
• Rearranging the last equation
8
Euclidean transformation
8Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Pre-multiplying by inverse of (modified) intrinsic matrix
9
Euclidean transformation
9Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
or for short:
Homogeneous:
Cartesian:
For short:
10
Similarity Transformation
10Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Consider viewing fronto-parallel plane at unknown distance D
• By same logic as before we have
• Premultiplying by inverse of intrinsic matrix
11
Similarity Transformation
11Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Simplifying:
Multiply each equation by :
12
Similarity Transformation
12Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Simplifying:
Incorporate the constants by defining:
13
Similarity Transformation
13Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Homogeneous:
Cartesian:
For short:
14
Affine Transformation
14Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Homogeneous:
Cartesian:
For short:
15
Affine Transform
15Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Affine transform describes mapping well when the depth variation within the
planar object is small and the camera is far away
When variation in depth is comparable to distance to object then the affine
transformation is not a good model. Here we need the homography
16
Projective transformation / collinearity / homography
16Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Start with basic projection equation:
Combining these two matrices we get:
17
Homography
17Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Homogeneous:
Cartesian:
For short:
18
Modeling for noise
18Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• In the real world, the measured image positions are uncertain.
• We model this with a normal distribution• e.g.
19
Structure
19Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Transformation models• Learning and inference in transformation models• Three problems– Exterior orientation– Calibration– Reconstruction
• Properties of the homography• Robust estimation of transformations• Applications
20
Learning and inference problems
20Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Learning – take points on plane and their projections into the image and learn transformation parameters
• Inference – take the projection of a point in the image and establish point on plane
21
Learning and inference problems
21Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
22
• Maximum likelihood approach
• Becomes a least squares problem
Learning transformation models
22Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
23
Learning Euclidean parameters
23Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Solve for transformation:
Remaining problem:
24
• This is an orthogonal Procrustes problem. To solve: • Compute SVD• And then set
Learning Euclidean parameters
24Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Has the general form:
25
Learning similarity parameters
25Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Solve for rotation matrix as before• Solve for translation and scaling factor
26
Learning affine parameters
26Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Affine transform is linear
• Solve using least-squares solution
27
Learning homography parameters
27Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Homography is not linear – cannot be solved in closed form. Convert to other homogeneous coordinates
• Both sides are 3x1 vectors; should be parallel, so cross product will be zero
28
• Write out these equations in full
• There are only 2 independent equations here – use a minimum of four points to build up a set of equations
Learning homography parameters
28Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
29
• These equations have the form , which we need to solve with the constraint
• This is a “minimum direction” problem– Compute SVD– Take last column of
• Then use non-linear optimization
Learning homography parameters
29Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
30
Inference problems
30Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Given point x* in image, find position w* on object
31
In the absence of noise, we have the relation , or in homogeneous coordinates
Inference
31Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
To solve for the points, we simply invert this relation
32
Structure
32Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Transformation models• Learning and inference in transformation models• Three problems– Exterior orientation– Calibration– Reconstruction
• Properties of the homography• Robust estimation of transformations• Applications
33
Problem 1: exterior orientation
33Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
34
Problem 1: exterior orientation
34Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Writing out the camera equations in full
• Estimate the homography from matched points• Factor out the intrinsic parameters
35
Problem 1: exterior orientation
35Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• To estimate the first two columns of rotation matrix, we compute this singular value decomposition
• Then we set
36
Problem 1: exterior orientation
36Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Find the last column using the cross product of first two columns
• Make sure the determinant is 1. If it is -1, then multiply last column by -1.
• Find translation scaling factor between old and new values
• Finally, set
37
Structure
37Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Transformation models• Learning and inference in transformation models• Three problems– Exterior orientation– Calibration– Reconstruction
• Properties of the homography• Robust estimation of transformations• Applications
38
Problem 2: calibration
38Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
39
Structure
39Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
One approach (not very efficient) is to alternately
• Optimize extrinsic parameters for fixed intrinsic
• Optimize intrinsic parameters for fixed extrinsic
Calibration
40Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Then use non-linear optimization.
41
Structure
41Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Transformation models• Learning and inference in transformation models• Three problems– Exterior orientation– Calibration– Reconstruction
• Properties of the homography• Robust estimation of transformations• Applications
42
Problem 3 - reconstruction
42Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Transformation between plane and image:
Point in frame of reference of plane:Point in frame of reference of camera
43
Structure
43Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Transformation models• Learning and inference in transformation models• Three problems– Exterior orientation– Calibration– Reconstruction
• Properties of the homography• Robust estimation of transformations• Applications
44
Transformations between images
44Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• So far we have considered transformations between the image and a plane in the world
• Now consider two cameras viewing the same plane
• There is a homography between camera 1 and the plane and a second homography between camera 2 and the plane
• It follows that the relation between the two images is also a homography
45
Properties of the homography
45Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Homography is a linear transformation of a ray
Equivalently, leave rays and linearly transform image plane – all images formed by all planes that cut the same ray bundle are related by homographies.
46
Camera under pure rotation
46Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Special case is camera under pure rotation.Homography can be showed to be
47
Structure
47Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Transformation models• Learning and inference in transformation models• Three problems– Exterior orientation– Calibration– Reconstruction
• Properties of the homography• Robust estimation of transformations• Applications
48
Robust estimation
48Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Least squares criterion is not robust to outliers
For example, the two outliers here cause the fitted line to be quite wrong.
One approach to fitting under these circumstances is to use RANSAC – “Random sampling by consensus”
49
RANSAC
49Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
50
RANSAC
50Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
51
Fitting a homography with RANSAC
51Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Original images Initial matches Inliers from RANSAC
52
Piecewise planarity
52Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Many scenes are not planar, but are nonetheless piecewise planarCan we match all of the planes to one another?
53
Approach 1 – Sequential RANSAC
53Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Problems: greedy algorithm and no need to be spatially coherent
54
Approach 2 – PEaRL (propose, estimate and re-learn)
54Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Associate label l which indicates which plane we are in• Relation between points xi in image1 and yi in image 2
• Prior on labels is a Markov random field that encourages nearby labels to be similar
• Model solved with variation of alpha expansion algorithm
55
Approach 2 – PEaRL
55Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
56
Structure
56Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Transformation models• Learning and inference in transformation models• Three problems– Exterior orientation– Calibration– Reconstruction
• Properties of the homography• Robust estimation of transformations• Applications
57
Augmented reality tracking
57Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
58
Fast matching of keypoints
58Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
59
Visual panoramas
59Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
60
Conclusions
• Mapping between plane in world and camera is one-to-one
• Takes various forms, but most general is the homography
• Revisited exterior orientation, calibration, reconstruction problems from planes
• Use robust methods to estimate in the presence of outliers
60Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
top related