Camera from F

Computing K from 1 image

HZ8.8

IAC and K

HZ8.5

Cameramatrixfrom F

HZ9.5

IAC

HZ3.5-3.7, 8.5

Extractingcamera

parameters

HZ6.2

Cameramatrix

HZ6.1

Calibrationusing Q*

HZ19.3Hartley 92

F does not uniquely identify a camera pair

• a fundamental matrix F relates two images, and therefore two cameras• F already gives indirect information about the camera through the epipole (the null vector

of F)• we will now see that much fuller information about both cameras can be extracted from F,

in particular the two camera matrices• but a fundamental matrix does not uniquely identify two camera matrices• Theorem: Let H be a homography of 3-space. The fundamental matrix associated

with camera pair (P,P’) is the same as the fundamental matrix associated with camera pair (PH, P’H).

• Proof: if (x = PX, x’ = P’X) is a point correspondence in the 1st camera system arising from 3d point X, then (x = (PH)(H-1X), x’ = (P’H)(H-1X)) is a point correspondence in the 2nd camera system arising from 3d point H-1X

• thus, upon finding F, we only know the camera matrix up to a homography• analogous to affine rectification: extra work is required to remove this projective freedom

and get to metric structure of the camera (stay tuned!)• luckily this is the only degree of freedom: if two camera pairs have the same F, then they

must differ by a homography (Thm 9.10)• HZ254

F does not depend on world frame

• camera matrix depends on image frame and world frame, since x = PX is dependent on the image coordinates x and the world coordinates X

• but the above result is telling us that the fundamental matrix is independent of the world frame

• we are happy to relax to within a similarity of the truth, but we are relaxing to within a projectivity of the truth, which pleases us less– we will need calibration techniques – (these calibration techniques may be interpreted as ways of

protecting the absolute conic)

• HZ253

Camera matrix from F• let’s see what we can get from F• we assume that the first camera matrix is always [I 0]

– we can always normalize so that it is, using some homography• we are solving for the “relative offset” of the 2nd camera from the 1st

• Lemma: F is fundamental matrix of camera pair (P,P’) iff P’tFP is skew-symmetric.• Proof: P’tFP is skew-symmetric Xt (P’tFP) X = 0 for all X x’t F x = 0 where x’ =

P’X and x = PX, the images of X F is fundamental matrix of (P,P’)• Theorem: The camera matrices associated with F may be chosen to be [I 0] and [SF

e’] where– e’^t F = 0 (2nd image epipole)– S = any skew-symmetric matrix– proof: just check that [SF e’]^t F [I 0] is skew-symmetric

• Corollary: The camera matrices associated with F may be chosen to be [I 0] and [[e’]x F e’].

– choose S = [e’]x to guarantee a rank 3 camera matrix P’– see proof on 256– ironic: to guarantee rank 3 P’ (necessary, so good), we force rank 2 M’ (a bit bad)

• HZ255-6

Pseudoinverse P+

• the camera matrix P is rectangular (3x4) so it does not have an inverse

• but we want to use the notion of inverse to talk about the preimage of an image point

• a rectangular matrix P has a pseudoinverse P+

• P+ = (Pt P)-1 Pt – note that Pt P is square– note that P+ degenerates to P-1 when P is square and

nonsingular• x = A+b is the least-squares solution of Ax=b if A

is full rank (Trefethen 81)

F from P

• two camera matrices do uniquely specify a fundamental matrix• let P and P’ be two camera matrices, and e’ the epipole in the 2nd

image– note: e’ can be calculated as P’C (image of other camera center)

• the associated fundamental matrix of these 2 cameras: – F = [e’]x P’ P+

– note: P’P+ is the homography H in the earlier definition of F = [e’]x H– proof idea: epipolar line of x is built from the epipole e’ and the camera-2-

image of a typical point of the camera-1-preimage of x (or P’ P+x)– proof: Fx = L’ = e’ x P’ (P+ x) so F = [e’]x P’ P+

• note how the fundamental and camera matrices are independent of the actual contents of the image; only dependent on the camera setup

• HZ244