uncalibrated camera
DESCRIPTION
Uncalibrated Camera. Lecture 5 Uncalibrated Geometry & Stratification. calibrated coordinates. Linear transformation. pixel coordinates. Uncalibrated Camera. Calibration with a rig Uncalibrated epipolar geometry Ambiguities in image formation Stratified reconstruction - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Uncalibrated Camera](https://reader030.vdocuments.site/reader030/viewer/2022032612/56813297550346895d992dcf/html5/thumbnails/1.jpg)
MASKS © 2004 Invitation to 3D vision
Uncalibrated Camera
Lecture 5Uncalibrated Geometry
& Stratification
![Page 2: Uncalibrated Camera](https://reader030.vdocuments.site/reader030/viewer/2022032612/56813297550346895d992dcf/html5/thumbnails/2.jpg)
MASKS © 2004 Invitation to 3D vision
Uncalibrated Camera
pixelcoordinates
calibratedcoordinates
Linear transformation
![Page 3: Uncalibrated Camera](https://reader030.vdocuments.site/reader030/viewer/2022032612/56813297550346895d992dcf/html5/thumbnails/3.jpg)
MASKS © 2004 Invitation to 3D vision
Overview
• Calibration with a rig
• Uncalibrated epipolar geometry
• Ambiguities in image formation
• Stratified reconstruction
• Autocalibration with partial scene knowledge
![Page 4: Uncalibrated Camera](https://reader030.vdocuments.site/reader030/viewer/2022032612/56813297550346895d992dcf/html5/thumbnails/4.jpg)
MASKS © 2004 Invitation to 3D vision
Uncalibrated Camera
xyc
• Pixel coordinates
• Projection matrix
Uncalibrated camera
• Image plane coordinates • Camera extrinsic parameters
• Perspective projection
Calibrated camera
![Page 5: Uncalibrated Camera](https://reader030.vdocuments.site/reader030/viewer/2022032612/56813297550346895d992dcf/html5/thumbnails/5.jpg)
MASKS © 2004 Invitation to 3D vision
Taxonomy on Uncalibrated Reconstruction
• is known, back to calibrated case
• is unknown Calibration with complete scene knowledge (a rig) –
estimate Uncalibrated reconstruction despite the lack of knowledge
of Autocalibration (recover from uncalibrated images)
• Use partial knowledge Parallel lines, vanishing points, planar motion, constant
intrinsic
• Ambiguities, stratification (multiple views)
![Page 6: Uncalibrated Camera](https://reader030.vdocuments.site/reader030/viewer/2022032612/56813297550346895d992dcf/html5/thumbnails/6.jpg)
MASKS © 2004 Invitation to 3D vision
Calibration with a Rig
Use the fact that both 3-D and 2-D coordinates of feature points on a pre-fabricated object (e.g., a cube) are known.
![Page 7: Uncalibrated Camera](https://reader030.vdocuments.site/reader030/viewer/2022032612/56813297550346895d992dcf/html5/thumbnails/7.jpg)
MASKS © 2004 Invitation to 3D vision
Calibration with a Rig
• Eliminate unknown scales
• Factor the into and using QR decomposition
• Solve for translation
• Recover projection matrix
• Given 3-D coordinates on known object
![Page 8: Uncalibrated Camera](https://reader030.vdocuments.site/reader030/viewer/2022032612/56813297550346895d992dcf/html5/thumbnails/8.jpg)
MASKS © 2004 Invitation to 3D vision
Uncalibrated Epipolar Geometry
• Epipolar constraint
• Fundamental matrix
• Equivalent forms of
![Page 9: Uncalibrated Camera](https://reader030.vdocuments.site/reader030/viewer/2022032612/56813297550346895d992dcf/html5/thumbnails/9.jpg)
MASKS © 2004 Invitation to 3D vision
Properties of the Fundamental Matrix
Imagecorrespondences
• Epipolar lines
• Epipoles
![Page 10: Uncalibrated Camera](https://reader030.vdocuments.site/reader030/viewer/2022032612/56813297550346895d992dcf/html5/thumbnails/10.jpg)
MASKS © 2004 Invitation to 3D vision
Properties of the Fundamental Matrix
A nonzero matrix is a fundamental matrix if has a singular value decomposition (SVD) with
for some .
There is little structure in the matrix except that
![Page 11: Uncalibrated Camera](https://reader030.vdocuments.site/reader030/viewer/2022032612/56813297550346895d992dcf/html5/thumbnails/11.jpg)
MASKS © 2004 Invitation to 3D vision
• Fundamental matrix can be estimated up to scale
• Find such F that the epipolar error is minimized
Estimating Fundamental Matrix
• Denote
• Rewrite
• Collect constraints from all points
Pixel coordinates
![Page 12: Uncalibrated Camera](https://reader030.vdocuments.site/reader030/viewer/2022032612/56813297550346895d992dcf/html5/thumbnails/12.jpg)
MASKS © 2004 Invitation to 3D vision
• Project onto the essential manifold:
• cannot be unambiguously decomposed into pose and calibration
Two view linear algorithm – 8-point algorithm
• Solve the LLSE problem:
• Compute SVD of F recovered from data
• Solution eigenvector associated with smallest eigenvalue of
![Page 13: Uncalibrated Camera](https://reader030.vdocuments.site/reader030/viewer/2022032612/56813297550346895d992dcf/html5/thumbnails/13.jpg)
MASKS © 2004 Invitation to 3D vision
Calibrated vs. Uncalibrated Space
![Page 14: Uncalibrated Camera](https://reader030.vdocuments.site/reader030/viewer/2022032612/56813297550346895d992dcf/html5/thumbnails/14.jpg)
MASKS © 2004 Invitation to 3D vision
Calibrated vs. Uncalibrated Space
Distances and angles are modified by S
![Page 15: Uncalibrated Camera](https://reader030.vdocuments.site/reader030/viewer/2022032612/56813297550346895d992dcf/html5/thumbnails/15.jpg)
MASKS © 2004 Invitation to 3D vision
Motion in the distorted space
• Uncalibrated coordinates are related by
• Conjugate of the Euclidean group
Calibrated space Uncalibrated space
![Page 16: Uncalibrated Camera](https://reader030.vdocuments.site/reader030/viewer/2022032612/56813297550346895d992dcf/html5/thumbnails/16.jpg)
MASKS © 2004 Invitation to 3D vision
What Does F Tell Us?
• can be inferred from point matches (eight-point algorithm)
• Cannot extract motion, structure and calibration from one fundamental matrix (two views)
• allows reconstruction up to a projective transformation (as we will see soon)
• encodes all the geometric information among two views when no additional information is available
![Page 17: Uncalibrated Camera](https://reader030.vdocuments.site/reader030/viewer/2022032612/56813297550346895d992dcf/html5/thumbnails/17.jpg)
MASKS © 2004 Invitation to 3D vision
Decomposing the Fundamental Matrix
• Decomposition of is not unique
• Unknown parameters - ambiguity
• Corresponding projection matrix
• Decomposition of the fundamental matrix into a skew symmetric matrix and a nonsingular matrix
![Page 18: Uncalibrated Camera](https://reader030.vdocuments.site/reader030/viewer/2022032612/56813297550346895d992dcf/html5/thumbnails/18.jpg)
MASKS © 2004 Invitation to 3D vision
Projective Reconstruction
• From points, extract , followed by computation of projection matrices and structure • Canonical decomposition
• Given projection matrices – recover structure
• Projective ambiguity – non-singular 4x4 matrix
Both and are consistent with the epipolar geometry – give the same fundamental matrix
![Page 19: Uncalibrated Camera](https://reader030.vdocuments.site/reader030/viewer/2022032612/56813297550346895d992dcf/html5/thumbnails/19.jpg)
MASKS © 2004 Invitation to 3D vision
Projective Reconstruction
• Given projection matrices recover projective structure
• This is a linear problem and can be solve using linear least-squares
• Projective reconstruction – projective camera matrices and projective structure
Euclidean Structure Projective Structure
![Page 20: Uncalibrated Camera](https://reader030.vdocuments.site/reader030/viewer/2022032612/56813297550346895d992dcf/html5/thumbnails/20.jpg)
MASKS © 2004 Invitation to 3D vision
Euclidean vs Projective reconstruction
• Euclidean reconstruction – true metric properties of objects lenghts (distances), angles, parallelism are preserved • Unchanged under rigid body transformations• => Euclidean Geometry – properties of rigid bodies under rigid body transformations, similarity transformation
• Projective reconstruction – lengths, angles, parallelism are NOT
preserved – we get distorted images of objects – their distorted 3D counterparts --> 3D projective reconstruction
• => Projective Geometry
![Page 21: Uncalibrated Camera](https://reader030.vdocuments.site/reader030/viewer/2022032612/56813297550346895d992dcf/html5/thumbnails/21.jpg)
MASKS © 2004 Invitation to 3D vision
Homogeneous Coordinates (RBM)
3-D coordinates are related by:
Homogeneous coordinates:
Homogeneous coordinates are related by:
![Page 22: Uncalibrated Camera](https://reader030.vdocuments.site/reader030/viewer/2022032612/56813297550346895d992dcf/html5/thumbnails/22.jpg)
MASKS © 2004 Invitation to 3D vision
Homogenous and Projective Coordinates
• Homogenous coordinates in 3D before – attach 1 as the last
coordinate – render the transformation as linear transformation
• Projective coordinates – all points are equivalent up to a scale
• Each point on the plane is represented by a ray in projective space
2D- projective plane 3D- projective space
• Ideal points – last coordinate is 0 – ray parallel to the image plane • points at infinity – never intersects the image plane
![Page 23: Uncalibrated Camera](https://reader030.vdocuments.site/reader030/viewer/2022032612/56813297550346895d992dcf/html5/thumbnails/23.jpg)
MASKS © 2004 Invitation to 3D vision
Vanishing points – points at infinity
Representation of a 3-D line – in homogeneous coordinates
When -> 1 - vanishing points – last coordinate -> 0
Similarly in the image plane
![Page 24: Uncalibrated Camera](https://reader030.vdocuments.site/reader030/viewer/2022032612/56813297550346895d992dcf/html5/thumbnails/24.jpg)
MASKS © 2004 Invitation to 3D vision
Ambiguities in the image formation
• Potential Ambiguities
• Ambiguity in K (K can be recovered uniquely – Cholesky or QR)
• Structure of the motion parameters
• Just an arbitrary choice of reference frame
![Page 25: Uncalibrated Camera](https://reader030.vdocuments.site/reader030/viewer/2022032612/56813297550346895d992dcf/html5/thumbnails/25.jpg)
MASKS © 2004 Invitation to 3D vision
Ambiguities in Image Formation
• For any invertible 4 x 4 matrix
• In the uncalibrated case we cannot distinguish between camera imaging word from camera imaging distorted world
• In general, is of the following form
• In order to preserve the choice of the first reference frame we can restrict some DOF of
Structure of the (uncalibrated) projection matrix
![Page 26: Uncalibrated Camera](https://reader030.vdocuments.site/reader030/viewer/2022032612/56813297550346895d992dcf/html5/thumbnails/26.jpg)
MASKS © 2004 Invitation to 3D vision
Structure of the Projective Ambiguity
• 1st frame as reference
• Choose the projective reference frame
then ambiguity is
• can be further decomposed as
![Page 27: Uncalibrated Camera](https://reader030.vdocuments.site/reader030/viewer/2022032612/56813297550346895d992dcf/html5/thumbnails/27.jpg)
MASKS © 2004 Invitation to 3D vision
Stratified (Euclidean) Reconstruction
• General ambiguity – while preserving choice of first reference
frame
• Decomposing the ambiguity into affine and projective one
• Note the different effect of the 4-th homogeneous coordinate
![Page 28: Uncalibrated Camera](https://reader030.vdocuments.site/reader030/viewer/2022032612/56813297550346895d992dcf/html5/thumbnails/28.jpg)
MASKS © 2004 Invitation to 3D vision
Affine upgrade
• Upgrade projective structure to an affine structure
• Exploit partial scene knowledge Vanishing points, no skew, known principal point
• Special motions Pure rotation, pure translation, planar motion,
rectilinear motion• Constant camera parameters (multi-view)
![Page 29: Uncalibrated Camera](https://reader030.vdocuments.site/reader030/viewer/2022032612/56813297550346895d992dcf/html5/thumbnails/29.jpg)
MASKS © 2004 Invitation to 3D vision
Affine upgrade using vanishing points
Maps the points
To points with affine coordinates
How to compute
Vanishing points – last homogeneous affine coordinate is 0
![Page 30: Uncalibrated Camera](https://reader030.vdocuments.site/reader030/viewer/2022032612/56813297550346895d992dcf/html5/thumbnails/30.jpg)
MASKS © 2004 Invitation to 3D vision
Affine Upgrade
Need at least three vanishing points
3 equations, 4 unknowns (-1 scale) Solve for
Set up and update the projectivestructure
![Page 31: Uncalibrated Camera](https://reader030.vdocuments.site/reader030/viewer/2022032612/56813297550346895d992dcf/html5/thumbnails/31.jpg)
MASKS © 2004 Invitation to 3D vision
Euclidean upgrade
• We need to estimate remaining affine ambiguity
• In the case of special motions (e.g. pure rotation) – no projective ambiguity – cannot do projective reconstruction
• Estimate directly (special case of rotating camera – follows)
• Multi-view case – estimate projective and affine ambiguity together
• Use additional constraints of the scene structure (next)• Autocalibration (Kruppa equations)
Alternatives:
![Page 32: Uncalibrated Camera](https://reader030.vdocuments.site/reader030/viewer/2022032612/56813297550346895d992dcf/html5/thumbnails/32.jpg)
MASKS © 2004 Invitation to 3D vision
Euclidean Upgrade using vanishing points
• Vanishing points – intersections of the parallel lines
• Vanishing points of three orthogonal directions
• Orthogonal directions – inner product is zero
• Provide directly constraints on matrix
• S – has 5 degrees of freedom, 3 vanishing points – 3 constraints (need additional assumption about K)
• Assume zero skew and aspect ratio = 1
![Page 33: Uncalibrated Camera](https://reader030.vdocuments.site/reader030/viewer/2022032612/56813297550346895d992dcf/html5/thumbnails/33.jpg)
MASKS © 2004 Invitation to 3D vision
Geometric Stratification
![Page 34: Uncalibrated Camera](https://reader030.vdocuments.site/reader030/viewer/2022032612/56813297550346895d992dcf/html5/thumbnails/34.jpg)
MASKS © 2004 Invitation to 3D vision
Special Motions – Pure Rotation
• Calibrated Two views related by rotation only
• Mapping to a reference view – rotation can be estimated
• Mapping to a cylindrical surface
![Page 35: Uncalibrated Camera](https://reader030.vdocuments.site/reader030/viewer/2022032612/56813297550346895d992dcf/html5/thumbnails/35.jpg)
MASKS © 2004 Invitation to 3D vision
Pure Rotation - Uncalibrated Case
• Calibrated Two views related by rotation only
• Conjugate rotation can be estimated
• Given C, how much does it tell us about K (or S) ?
• Given three rotations around linearly independent axes – S, K can be estimated using linear techniques• Applications – image mosaics
![Page 36: Uncalibrated Camera](https://reader030.vdocuments.site/reader030/viewer/2022032612/56813297550346895d992dcf/html5/thumbnails/36.jpg)
MASKS © 2004 Invitation to 3D vision
Overview of the methods
![Page 37: Uncalibrated Camera](https://reader030.vdocuments.site/reader030/viewer/2022032612/56813297550346895d992dcf/html5/thumbnails/37.jpg)
MASKS © 2004 Invitation to 3D vision
Summary
![Page 38: Uncalibrated Camera](https://reader030.vdocuments.site/reader030/viewer/2022032612/56813297550346895d992dcf/html5/thumbnails/38.jpg)
MASKS © 2004 Invitation to 3D vision
Direct Autocalibration Methods
satisfies the Kruppa’s equations
The fundamental matrix
If the fundamental matrix is known up to scale
Under special motions, Kruppa’s equations become linear.
Solution to Kruppa’s equations is sensitive to noises.
![Page 39: Uncalibrated Camera](https://reader030.vdocuments.site/reader030/viewer/2022032612/56813297550346895d992dcf/html5/thumbnails/39.jpg)
MASKS © 2004 Invitation to 3D vision
Direct Stratification from Multiple Views
we obtain the absolute quadric contraints
From the recovered projective projection matrix
Partial knowledge in (e.g. zero skew, square pixel) renders the above constraints linear and easier to solve.
The projection matrices can be recovered from the multiple-view rank method to be introduced later.
![Page 40: Uncalibrated Camera](https://reader030.vdocuments.site/reader030/viewer/2022032612/56813297550346895d992dcf/html5/thumbnails/40.jpg)
MASKS © 2004 Invitation to 3D vision
Summary of (Auto)calibration Methods