camera calibration - umiacsramani/cmsc828d/lecture9.pdfwhat is camera calibration? • primarily,...
Post on 07-Mar-2018
234 Views
Preview:
TRANSCRIPT
What is Camera Calibration?
• Primarily, finding the quantities internal to the camera that affect the imaging process§ Position of image center in the image
• It is typically not at (width/2, height/2) of image
§ Focal length§ Different scaling factors for row pixels
and column pixels§ Skew factor§ Lens distortion (pin-cushion effect)
Motivation• Good calibration is important when we need
to § Reconstruct a world model: Virtual L.A. project§ Interact with the world
• Robot, hand-eye coordination
Image plane
We see a square of known size
Evaluation of position of a square for 2 focal lengths (red and blue projection geometry)
Scaling of Rows and Columns in Image
• Camera pixels are not necessarily square• Camera output may be analog (NTSC)• Image may be obtained by digitizing card§ A/D converter samples NTSC signal
Camera NTSC signal DigitizingMonitor display
CCD/CMOS
Compound Lens Imaging
• Inexpensive single lens systems distort image at its periphery
• Compound lenses may be used to reduce chromatic effects and pin-cushion effects
C
CameraImage plane
Image plane for equivalent pinhole camera is not camera image plane
Principal planes
Center of Projection
Nodal Point
f
f
Variety of Techniques
• VERY large literature on the subject• Work of Roger Tsai influential• Linear algebra method described here§ Can be used as initialization for iterative non
linear methods.
• Some interesting methods use vanishing points
Calibration Procedure
• Calibration target: 2 planes at right angle with checkerboard patterns (Tsai grid)§ We know positions of pattern corners only with
respect to a coordinate system of the target§ We position camera in front of target and find
images of corners§ We obtain equations that describe imaging and
contain internal parameters of camera• As a side benefit, we find position and orientation of
camera with respect to target (camera pose)
Image Processing of Image of Target
• Canny edge detection• Straight line fitting to detected linked edges• Intersecting the lines to obtain the image
corners• Matching image corners and 3D target
checkerboard corners § By counting if whole target is visible in image
• We get pairs (image point)--(world point)),,(),( iiiii ZYXyx →
Central Projection
s
si
s
si
zy
fy
zx
fx
=
=
Scene point(xs , ys , zs )
Image point(xi , yi , f)x
z
Cf
ycenter of projection
Image plane
=
1000
1000000
s
s
s
zyx
ff
wvu
wvywux ii /,/ ==
If world and image points are represented by homogeneous vectors, central projection is a linear transformation:
Transformation From Lengths to Pixels
s
si
s
si
zy
fy
zx
fx
=
=
Image point(xi , yi , f)
xpix
ypix
s
ssyiypix
s
ssxixpix
zyzy
kfyyky
zxzx
kfxxkx
00
00
+=+=
+=+=
y
x
=
1000
1000
0
'''
0
0
s
s
s
y
x
zyx
yx
wvu
αα
'/'
'/'
wvy
wux
pix
pix
=
=then
yy
xx
kfkf
==
αα
with
Image center
x0
y0
Transformation uses:
• image center (x0, y0)
• scaling factors kx and ky
Internal Camera Parameters
=
1000
1000
'''
0
0
s
s
s
y
x
zyx
yxs
wvu
αα
yy
xx
kfkf
−==
αα
with
• αx and αy “focal lengths” in pixels
• x0 and y0 coordinates of image center in pixels
•Added parameter s is skew parameter
• K is called calibration matrix. Five degrees of freedom.
•K is a 3x3 upper triangular matrix
'/'
'/'
wvy
wux
pix
pix
=
=
[ ]33 0|IK=
=
01000010
0001
1000
00
0
1000 0
0
0
0
y
xs
y
xs
y
x
y
x
αα
αα
From Camera Coordinatesto World Coordinates
(Xs , Ys , Zs )
Image point(xi , yi , f)
x
z
C
y
Image plane
Z
X
Y(xs , ys , zs )
OM
From Camera Coordinatesto World Coordinates 2
(Xs , Ys , Zs ) in world cordinate system(xs , ys , zs ) in image cordinate system
O
MX
z
x
ki
I
KJ
jC
KJIkjikjiOMCOCM
SSSzyxSSS ZYXTTTzyx +++++=+++=
K.iJ.iI.i SSSxS ZYXTx +++=
+
=
S
S
S
z
y
x
S
S
S
ZYX
TTT
zyx
K.kJ.kI.kK.jJ.jI.jK.iJ.iI.i
Z
CO = T translation vector
Homogeneous Coordinates
+
=
S
S
S
z
y
x
S
S
S
ZYX
TTT
zyx
K.kJ.kI.kK.jJ.jI.jK.iJ.iI.i
=
111S
S
S
z
y
x
S
S
S
ZYX
TTT
zyx
000K.kJ.kI.kK.jJ.jI.jK.iJ.iI.i
=
11
1S
S
S
S
S
S
ZYX
zyx
T30
TR
From Camera Coordinatesto World Coordinates 3
(Xs , Ys , Zs )(xs , ys , zs )
O
MX
z
x
ki
I
KJ
jC
K)-J)-I)-kjiOC-OMCM
CSCSCSSSS ZZYYXXzyx ((( ++=++=
K.i)-J.i)-I.i)- CSCSCSS ZZYYXXx ((( ++=
)C-(XRxcam~
=
Z
)C-R(T~
=
system coordinate in world expressed OC vector is C~
Linear Transformation from World Coordinates to Pixels
=
11
~
1S
S
S
S
S
S
ZYX
zyx
T30
CR-R
[ ]
=
1'''
s
s
s
zyx
wvu
33 0|IK
[ ]
=
⇒
11
~
'''
S
S
S
ZYX
wvu
T3
33 0CR-R
0|IK
=
⇒
1'''
S
S
S
ZYX
wvu
P XPx =
• Combine camera projection and coordinate transformation matrices into a single matrix P
• P has 11 degrees of freedom:• 5 from triangular calibration matrix K, 3 from R and 3 from
• P is a fairly general 3 x 4 matrix •left 3x3 submatrix KR is non-singular
Properties of Matrix P
[ ] X0
CR-R0|IKx
T3
33
=
1
~
[ ] [ ] [ ]C-|IRCR-R0
CR-R0|I 3T
333
~~1
~==
[ ]XC-|IRKx 3~
=
[ ]C-|IRKP 3~
=
C~
• Further simplification of P:
Calibration
• 1. Estimate matrix P using scene points and their images
• 2. Estimate the interior parameters and the exterior parameters
§ Left 3x3 submatrix of P is product of upper-triangular matrix and orthogonal matrix
[ ]C-|IRKP 3~
=
Finding Camera Translation• Find homogeneous coordinates of C in the
scene • C is the null vector of matrix P§ P C = 0:
• Find null vector C of P using SVD§ C is the unit singular vector of P corresponding to the
smallest singular value (the last column of V, where P = U D VT is the SVD of P)
[ ]C-|IRKP 3~
=
=
0000
1100010001
c
c
c
c
c
c
ZYX
ZYX
Finding Camera Orientation andInternal Parameters
• Left 3x3 submatrix M of P is of form M=K R§ K is an upper triangular matrix§ R is an orthogonal matrix
• Any non-singular square matrix M can be decomposed into the product of an upper-triangular matrix K and an orthogonal matrix Rusing the RQ factorization§ Similar to QR factorization but order of 2 matrices is
reversed
RQ Factorization of M
• Compute
• Multiply M by Rx. The resulting term at (3,2) is zero because of the values selected for c and s
• Multiply the resulting matrix by Ry, after selecting c’ and s’ so that the resulting term at position (3,1) is set to zero
• Multiply the resulting matrix by Rz, after selecting c’’ and s’’ so that the resulting term at position (2,1) is set to zero
−=
−=
−=
1000''''0''''
,'0'
010'0'
,00
001cssc
cs
sc
cssc zyx RRR
2/1233
232
322/12
33232
33
)(,
)( mmm
smm
mc
+=
+−=
RKRRRKMKRRRM Tx
Ty
Tzzyx ==⇒=
Computing Matrix P
• Use corresponding image and scene points§ 3D points Xi in world coordinate system§ Images xi of Xi in image
• Write xi = P Xi for all i• Similar problem to finding projectivity
matrix H (i.e. homography) in homework
Improved Computation of P
• xi = P Xi involves homogeneous coordinates, thus xi and P Xi just have to be proportional:
• Let p1T, p2
T, p3T be the 3 row vectors of P
=
iT3
iT2
iT1
i
XpXpXp
XP
0=× ii XPx
−−−
=×
iT1ii
T2i
iT3ii
T1i
iT2ii
T3i
ii
XpXpXpXpXpXp
XPx''''''
vuuwwv
0''
''''
=
−−
−⇒
3
2
1
T4
Tii
Tii
Tii
T4
Tii
Tii
Tii
T4
ppp
0XXX0X
XX0
uvuw
vw vector112 a is
ppp
3
2
1
×
Improved Computation of P, cont’d
• Third row can be obtained from sum of u’itimes first row - v’i times second row
• So we get 2 independent equations in 11 unknowns (ignoring scale)
• With 6 point correspondences, we get enough equations to compute matrix P
0''
''''
=
−−
−
3
2
1
T4
Tii
Tii
Tii
T4
Tii
Tii
Tii
T4
ppp
0XXX0X
XX0
uvuw
vw
A p = 0
Solving A p = 0
• Linear system A p = 0• When possible, have at least 5 times as
many equations as unknowns (28 points) • Minimize || A p || with the constraint
|| p || = 1§ P is the unit singular vector of A corresponding
to the smallest singular value (the last column of V, where A = U D VT is the SVD of A)
• Called Direct Linear Transformation (DLT)
Improving P Solution with Nonlinear Minimization
• Find p using DLT• Use as initialization for nonlinear
minimization of § Use Levenberg-Marquardt iterative minimization
2),( ii
i PXx∑ d
Radial Distortion
• We have assumed that lines are imaged as lines
• Not quite true for real lenses§ Significant error for cheap optics and for short
focal lengths
Scene Image Corrected image
Radial Distortion Modeling• In pixel cordinates the correction is written
• Minimize using lines known to be straight(x’,y’) is radial projection of (x,y) on straight line
...1)(
)()(
)()()()(
221
20
20
2
00
00
+++=
−+−=
−=−−=−
rrrL
yyxxr
yyrLyyxxrLxx
c
c
κκ
with(x0, y0)
(x, y)(xc, yc)
∑ −+−=i
2221 )'()'(),( ciicii yyxxf κκ
(x, y)
(x’, y’)
Distorted lineAnd ideal line:
References
• Multiple View Geometry in Computer Vision, R. Hartley and A. Zisserman, Cambridge University Press, 2000, pp. 138-183
• Three-Dimensional Computer Vision: A Geometric Approach, O. Faugeras, MIT Press, 1996, pp. 33-68
• “A Versatile Camera Calibration Technique for 3D Machine Vision”, R. Y. Tsai, IEEE J. Robotics & Automation, RA-3, No. 4, August 1987, pp. 323-344
top related