Precise Omnidirectional Camera Calibration

Dennis Strelow, Jeffrey Mishler, David Koes, and Sanjiv Singh

Overview (1)

Projection model for omnidirectional cameras that accounts for the full rotation and translation between camera and mirror

Projection model handles noncentral omnidirectional cameras

Calibration algorithm determines relative position from one omnidirectional image of known 3D targets

Overview (2)

One image sufficient for accurate calibration of transformation

Full calibration allows shape-from-motion and epipolar matching even if camera-mirror misalignment is severe

Full model improves shape-from-motion and epipolar geometry results even if the camera and mirror are closely aligned

Omnidirectional cameras

Omnidirectional projections (1)

The mirror point m determines the projection

Omnidirectional projections (2)

Finding the mirror point is…

one-dimensional (z only) if the mirror and camera are assumed aligned

closed form for aligned single viewpoint cameras

Omnidirectional projections (3)

If the camera and mirror are not aligned, then two constraints determine m

Equiangular Cameras (1)

Equiangular cameras (2)

Relative rotation, translation between axes distorts projections

Calibration (1)

Calibration (2)

Least squares error to be minimized:

Known: 2D projections xi, 3D points pi

Unknown: Camera position Rc, tc; mirror-to-camera transformation (implicit in ∏)

2 (Rc pi tc) x i

2

i1

n

Experiments (1)

Basic questions about calibration:

1. Does the calibration produce the correct mirror-to-camera transformation?

2. Is the model correct, e.g., is it possible to perform SFM with misaligned a mirror?

3. Is the full model worthwhile if the mirror is nearly aligned?

Experiments (2)

Three lab sequencesMirror and camera axes:

1. Closely aligned 2. Moderate misalignment 3. Severe misalignment

Experiments (3)

Experiments (4)

Performed shape-from-motion on each sequence using each of three calibrations:

A. Calibrate nothing B. Calibrate mirror-camera distance C. Calibrate rotation and translation

Calibration B is interesting because computing the projection in this case is a one-dimensional problem

Experiments (5): residuals

Difference between observed target image location and reprojected location (pixels)

Cal. A Cal. B Cal. C

Seq. 1 1.92 1.91 1.34

Seq. 2 4.01 3.85 1.40

Seq. 3 6.50 6.30 1.36

Experiments (6): values

Sequences differ mainly in tx

Standard deviations are small

tx (cm)

Seq. 1 0.0052 ± 0.0035

Seq. 2 0.21 ± 0.0053

Seq. 3 0.37 ± 0.0056

Experiments (7): apex reproj.

Difference between observed screw center and reprojected mirror apex

Difference

Seq. 1 5.0 pixels

Seq. 2 6.0 pixels

Seq. 3 4.7 pixels

Experiments (8): SFM

Shape-from-motion average reprojection errors (pixels) and depth errors (cm)

Cal. A Cal. B Cal. C

Seq. 1 0.40 / 3.3 0.41 / 3.4 0.37 / 2.0

Seq. 2 1.1 / 9.9 1.1 / 9.6 0.43 / 1.9

Seq. 3 1.9 / 15.9 1.92 / 15.2 0.38 / 1.9

Experiments (9): epipolar error

Average distance in pixels from epipolar line to correct match

Cal. A Cal. B Cal. C

Seq. 1 0.68 0.69 0.64

Seq. 2 1.3 1.4 0.71

Seq. 3 2.1 2.1 0.64