lecture3 camera calibration-3 · 2015-01-27 · silvio savarese lecture 3 - 14#jan#15...
TRANSCRIPT
![Page 1: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/1.jpg)
Lecture 3 - Silvio Savarese 13-‐Jan-‐15
Professor Silvio Savarese Computational Vision and Geometry Lab
Lecture 3Camera Models 2 & Camera Calibration
![Page 2: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/2.jpg)
Lecture 3 - Silvio Savarese 14-‐Jan-‐15
• Recap of camera models• Camera calibration problem• Camera calibration with radial distortion • Example
Some slides in this lecture are courtesy to Profs. J. Ponce, F-F Li
Reading: [FP] Chapter 1 “Geometric Camera Calibration” [HZ] Chapter 7 “Computation of Camera Matrix P”
Lecture 3Camera Models 2 & Camera Calibration
![Page 3: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/3.jpg)
Pinhole perspective projectionProjective camera
f = focal length
f
![Page 4: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/4.jpg)
Pinhole perspective projection
x
y
xc
C’=[uo, vo]
Projective camera
f = focal lengthuo, vo = offset (note a different convention w.r.t. lecture 2)
f
yc
![Page 5: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/5.jpg)
Units: k,l [pixel/m]f [m]
[pixel],α βNon-square pixels
Projective camera
f = focal lengthuo, vo = offset
→ non-square pixels,α β
f
![Page 6: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/6.jpg)
f
x
y
xc
yc
C’ = [uo, vo]θ
Projective camera
f = focal lengthuo, vo = offset
→ non-square pixels,α βθ = skew angle
![Page 7: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/7.jpg)
Projective camera
!P =
α −α cotθ uo 0
0 β
sinθvo 0
0 0 1 0
&
'
(((((
)
*
+++++
xyz1
&
'
((((
)
*
++++
f = focal lengthuo, vo = offset
→ non-square pixels,α βθ = skew angle
K has 5 degrees of freedom!
f
![Page 8: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/8.jpg)
f
Projective camera
Ow
kw
jwR,T
f = focal lengthuo, vo = offset
→ non-square pixels,α βθ = skew angleR,T = rotation, translation
wPTR
P4410×
"#
$%&
'=
![Page 9: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/9.jpg)
f
f = focal lengthuo, vo = offset
→ non-square pixels,α β
Projective camera
wPMP =!
[ ] wPTRK=Internal parameters
External parameters
θ = skew angleR,T = rotation, translation
Ow
kw
jwR,T
![Page 10: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/10.jpg)
P'3×1 =M Pw = K3×3 R T"#
$% 3×4
Pw4×1
=
m1
m2
m3
!
"
####
$
%
&&&&
PW =m1PWm2PWm3PW
!
"
####
$
%
&&&&
→ P'E = (m1Pwm3Pw
, m2Pwm3Pw
)E
f
Projective camera
Ow
kw
jwR,T
!!!
"
#
$$$
%
&
=
3
2
1
mmm
M
![Page 11: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/11.jpg)
Theorem (Faugeras, 1993)
!!!
"
#
$$$
%
&
=
3
2
1
aaa
A[ ] [ ] ][ bATKRKTRKM ===
[Eq.13; lect. 2]
![Page 12: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/12.jpg)
Exercise!
Ow
iw
kw
jwR,T
M = K R T!"
#$
= (m1Pwm3Pw
, m2Pwm3Pw
)
P
P’
f
P'E = ( f xwzw, f yw
zw)
=
f 0 0 00 f 0 00 0 1 0
!
"
###
$
%
&&&
Pw =
xwywzw1
!
"
#####
$
%
&&&&&
à
= K I 0!"
#$
![Page 13: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/13.jpg)
Projective camera
p’q’r’ O
f’R
Q
P
![Page 14: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/14.jpg)
Weak perspective projection
R
Q
P
O
f’
When the relative scene depth is small compared to its distance from the camera
zo
p’q’r’ Q_
P_
R_
π
![Page 15: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/15.jpg)
Weak perspective projection
O
f’ zo
p’q’r’ Q_
P_
R_
x' = f 'zx
y' = f 'zy
!
"
###
$
###
→
x ' = f 'z0x
y' = f 'z0y
!
"
##
$
##
Magnification m
R
Q
Pπ
![Page 16: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/16.jpg)
!"
#$%
&=
10bA
M[ ]TRKM = !"
#$%
&=
1vbA
O
f’ zo
p’q’r’ Q_
P_
R_R
Q
P
π
à
Projective (perspective) Weak perspective
Weak perspective projection
![Page 17: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/17.jpg)
!!!
"
#
$$$
%
&
=
!!!
"
#
$$$
%
&
=
10002
1
3
2
1
mm
mmm!
!!
"
#
$$$
%
&
=
!!!
"
#
$$$
%
&
=
1PP
P W2
W1
W
3
2
1
mm
mmm
)P,P( 21 ww mm→E
wPMP =!!"
#$%
&=
10bA
M
magnification
!!!
"
#
$$$
%
&
=
3
2
1
mmm
wPMP =! !"
#$%
&=
1vbA
M!!!
"
#
$$$
%
&
=
!!!
"
#
$$$
%
&
=
W3
W2
W1
W
3
2
1
PPP
Pmmm
mmm
)PP,
PP(
3
2
3
1
w
w
w
w
mm
mm
→E
Perspective
Weak perspective
![Page 18: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/18.jpg)
Orthographic (affine) projectionDistance from center of projection to image plane is infinite
x' = f 'zx
y' = f 'zy
!
"
###
$
###
→x ' = xy' = y
!"#
$#
![Page 19: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/19.jpg)
Pros and Cons of These Models
• Weak perspective much simpler math. – Accurate when object is small and distant.– Most useful for recognition.
• Pinhole perspective much more accurate for scenes. – Used in structure from motion or SLAM.
![Page 20: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/20.jpg)
The Kangxi Emperor's Southern Inspection Tour (1691-1698) By Wang Hui
Weak perspective projection
![Page 21: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/21.jpg)
Weak perspective projection
The Kangxi Emperor's Southern Inspection Tour (1691-1698) By Wang Hui
![Page 22: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/22.jpg)
Lecture 3 - Silvio Savarese 14-‐Jan-‐15
• Recap of camera models• Camera calibration problem• Camera calibration with radial distortion • Example
Some slides in this lecture are courtesy to Profs. J. Ponce, F-F Li
Lecture 3Camera Calibration
Reading: [FP] Chapter 1 “Geometric Camera Calibration” [HZ] Chapter 7 “Computation of Camera Matrix P”
![Page 23: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/23.jpg)
Projective camera
!!!
"
#
$$$
%
& −
=
100
v0
ucot
K o
o
sinθβ
θαα
!!!
"
#
$$$
%
&
=T3
T2
T1
Rrrr
!!!
"
#
$$$
%
&
=
z
y
x
ttt
T
43×
wPMP =! [ ] wPTRK=Internal parameters
External parameters
![Page 24: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/24.jpg)
Goal of calibration
wPMP =! [ ] wPTRK=
Estimate intrinsic and extrinsic parameters from 1 or multiple images
Change notation:P = Pw
p = P’!!!
"
#
$$$
%
& −
=
100
v0
ucot
K o
o
sinθβ
θαα
!!!
"
#
$$$
%
&
=T3
T2
T1
Rrrr
!!!
"
#
$$$
%
&
=
z
y
x
ttt
T
43×
![Page 25: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/25.jpg)
Calibration Problem
•P1… Pn with known positions in [Ow,iw,jw,kw]
jC
Calibration rig
![Page 26: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/26.jpg)
Calibration Problem
•P1… Pn with known positions in [Ow,iw,jw,kw]•p1, … pn known positions in the image
jC
Calibration rig
image
pi
Goal: compute intrinsic and extrinsic parameters
![Page 27: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/27.jpg)
Calibration Problem
jC
Calibration rig
How many correspondences do we need?• M has 11 unknown • We need 11 equations • 6 correspondences would do it
image
pi
![Page 28: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/28.jpg)
Calibration Problem
imagejC
Calibration rig
In practice, using more than 6 correspondences enables more robust results
pi
![Page 29: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/29.jpg)
Calibration Problem
jC
ii PMP → !"
#$%
&=→
i
ii v
up
!!!
"
#
$$$
%
&
=
3
2
1
Mmmm
!!!!
"
#
$$$$
%
&
=
i3
i2
i3
i1
PPPP
mmmm
in pixels [Eq. 1]
![Page 30: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/30.jpg)
Calibration Problem
i3
i1i P
Pumm
=
i2i3i P)P(v mm =→
i1i3i P)P(u mm =→
i3
i2i P
Pvmm
=
!"
#$%
&
i
i
vu
!!!!
"
#
$$$$
%
&
=
i3
i2
i3
i1
PPPP
mmmm
0P)P(v 23 =−→ iii mm
→ ui (m3 Pi )−m1 Pi = 0
[Eq. 1]
[Eqs. 2]
![Page 31: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/31.jpg)
Calibration Problem
……
0)( 12131 =− PPv mm
0)( 11131 =− PPu mm
0)( 23 =− iii PPv mm
0)( 13 =− iii PPu mm
0)( 23 =− nnn PPv mm
0)( 13 =− nnn PPu mm
[Eqs. 3]
![Page 32: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/32.jpg)
Block Matrix Multiplication
!"
#$%
&=!
"
#$%
&=
2221
1211
2221
1211
BBBB
BAAAA
A
What is AB ?
!"
#$%
&
++
++=
2222122121221121
2212121121121111
BABABABABABABABA
AB
![Page 33: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/33.jpg)
!!!
"
#
$$$
%
&
=T3
T2
T1def
mmm
m
1 11
1 1 1def
nn
n n
PPP P
PP
P P
TT T
T TT
TT Tn
T TTn
uv
uu
! "−$ %
−$ %$ %=$ %$ %−$ %$ %−& '
00
P0
0
!
Calibration Problem
2n x 1212x1
1x44x1
…
Homogenous linear system
knownunknown
0)( 12131 =+− PPv mm
0)( 11131 =+− PPu mm
0)( 23 =+− nnn PPv mm
0)( 13 =+− nnn PPu mm
0=P m [Eq. 4]
![Page 34: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/34.jpg)
Homogeneous M x N Linear Systems
P m 0=
Rectangular system (M>N)• 0 is always a solution
Minimize |P m|2 under the constraint |m|2 =1
M=number of equations = 2nN=number of unknown = 11
• To find non-zero solution
N
M
![Page 35: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/35.jpg)
• How do we solve this homogenous linear system?
Calibration Problem
• Via SVD decomposition!
0=P m
![Page 36: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/36.jpg)
Calibration Problem
1212T
121212n2 VDU ×××
Last column of V gives m
M
SVD decomposition of P
Why? See pag 592 of HZ
!!!
"
#
$$$
%
&
=T3
T2
T1def
mmm
m
0=P m
![Page 37: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/37.jpg)
Extracting camera parameters
M = ρ
![Page 38: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/38.jpg)
Extracting camera parameters
A
!!!
"
#
$$$
%
&
=T3
T2
T1
Aaaa
[ ]TRK=
3
1a±
=ρ
!!!
"
#
$$$
%
&
=
3
2
1
bbb
b
Estimated values
)( 312 aa ⋅= ρou
)(v 322
o aa ⋅= ρ
( ) ( )3231
3231cosaaaaaaaa
×⋅×
×⋅×=θ
Intrinsic
b !!!
"
#
$$$
%
& −
=
100
v0
ucot
K o
o
sinθβ
θααρ
M =
Box 1
![Page 39: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/39.jpg)
Theorem (Faugeras, 1993)
![Page 40: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/40.jpg)
Extracting camera parameters
!!!
"
#
$$$
%
&
=T3
T2
T1
Aaaa
[ ]TRK=
!!!
"
#
$$$
%
&
=
3
2
1
bbb
b
Estimated values
Intrinsic
θρα sin312 aa ×=
θρβ sin322 aa ×=
ρ
A b
![Page 41: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/41.jpg)
Extracting camera parameters
A
!!!
"
#
$$$
%
&
=T3
T2
T1
Aaaa
b
[ ]TRK=
!!!
"
#
$$$
%
&
=
3
2
1
bbb
b
Estimated values
Extrinsic( )
32
321 aa
aar×
×=
3
33 a
ar ±=
132 rrr ×= b1KT −= ρ
ρ
![Page 42: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/42.jpg)
Degenerate cases
•Pi’s cannot lie on the same plane!• Points cannot lie on the intersection curve of two quadric surfaces
![Page 43: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/43.jpg)
Lecture 3 - Silvio Savarese 14-‐Jan-‐15
• Recap of projective cameras • Camera calibration problem• Camera calibration with radial distortion• Example
Some slides in this lecture are courtesy to Profs. J. Ponce, F-F Li
Lecture 3Camera Calibration
Reading: [FP] Chapter 1 “Geometric Camera Calibration” [HZ] Chapter 7 “Computation of Camera Matrix P”
![Page 44: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/44.jpg)
No distortion
Pin cushion
Barrel
Radial Distortion– Image magnification (in)decreases with distance from the optical axis– Caused by imperfect lenses– Deviations are most noticeable for rays that pass through the edge of
the lens
![Page 45: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/45.jpg)
Radial Distortion
ii
ii p
vu
PM
100
00
00
1
1
=!"
#$%
&→
!!!!!!
"
#
$$$$$$
%
&
λ
λd
v
vucvbuad 222 ++=
u
To model radial behavior
∑±==
3
1p
2ppdκ1λ
Polynomial function
Distortion coefficient
[Eq. 5] [Eq. 6]
Sλ
Image magnification decreases with distance from the optical center
![Page 46: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/46.jpg)
!"
#$%
&=
i
ii v
up
!!!
"
#
$$$
%
&
=
3
2
1
Qqqq
!!!!
"
#
$$$$
%
&
=
i3
i2
i3
i1
PPPP
qqqq
ii
ii p
vu
PM
100
00
00
1
1
=!"
#$%
&→
!!!!!!
"
#
$$$$$$
%
&
λ
λ
Q
!"#
=
=
PPvPPu
2i3i
i1i3i
qqqq
Is this a linear system of equations?
Radial Distortion
No! why? [Eqs.7]
![Page 47: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/47.jpg)
X = f (Q)
measurements parameters
f( ) is the nonlinear mapping
-Newton Method-Levenberg-Marquardt Algorithm
• Iterative, starts from initial solution • May be slow if initial solution far from real solution • Estimated solution may be function of the initial solution• Newton requires the computation of J, H• Levenberg-Marquardt doesn’t require the computation of H
General Calibration Problem
!!!!
"
#
$$$$
%
&
=
i3
i2
i3
i1
PPPP
qqqq
!"
#$%
&
i
i
vu
i=1…n
[Eq .8]
![Page 48: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/48.jpg)
A possible algorithm
1. Solve linear part of the system to find approximated solution 2. Use this solution as initial condition for the full system3. Solve full system using Newton or L.M.
General Calibration Problem
X = f (Q)
!!!!
"
#
$$$$
%
&
=
i3
i2
i3
i1
PPPP
qqqq
!"
#$%
&
i
i
vu
i=1…n
measurements parameters
[Eq .8]
f( ) is the nonlinear mapping
![Page 49: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/49.jpg)
Typical assumptions:
- zero-skew, square pixel- uo, vo = known center of the image
General Calibration Problem
X = f (Q)
measurement parameter
f( ) is nonlinear!!!!
"
#
$$$$
%
&
=
i3
i2
i3
i1
PPPP
qqqq
!"
#$%
&
i
i
vu
![Page 50: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/50.jpg)
Can we estimate m1 and m2 and ignore the radial distortion?
Radial Distortion
!"
#$%
&=
i
ii v
up
!!!!
"
#
$$$$
%
&
=
i3
i2
i3
i1
PPPP
1
mmmm
λ
d
v
uHint: slopevu
i
i =
!!!!
"
#
$$$$
%
&
=
i3
i2
i3
i1
PPPP
qqqq
![Page 51: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/51.jpg)
Estimating m1 and m2…
Radial Distortion
!"
#$%
&=
i
ii v
up
!!!!
"
#
$$$$
%
&
=
i3
i2
i3
i1
PPPP
1
mmmm
λ
0)()( 121111 =− PuPv mm0)()( 21 =− iiii PuPv mm
0)()( 21 =− nnnn PuPv mm…
L n= 0 !"
#$%
&=
2
1
mm
n
Tsai [87]
i
i
i
i
i
i
i
i
PP
PPPP
vu
2
1
3
2
3
1
)()()()(
mm
mmmm
==
Get m1 and m2 by SVD
[Eq .9]
[Eq .10][Eq .11]
![Page 52: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/52.jpg)
Once that m1 and m2 are estimated…
Radial Distortion
!"
#$%
&=
i
ii v
up
!!!!
"
#
$$$$
%
&
=
i3
i2
i3
i1
PPPP
1
mmmm
λ
3m is non linear function of 1m 2m λ
There are some degenerate configurations for which m1 and m2 cannot be computed
, ,
![Page 53: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/53.jpg)
Lecture 3 - Silvio Savarese 14-‐Jan-‐15
Some slides in this lecture are courtesy to Profs. J. Ponce, F-F Li
Lecture 3Camera Calibration • Recap of projective cameras • Camera calibration problem• Camera calibration with radial distortion• ExampleReading: [FP] Chapter 1 “Geometric Camera Calibration”
[HZ] Chapter 7 “Computation of Camera Matrix P”
![Page 54: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/54.jpg)
Calibration ProcedureCamera Calibration Toolbox for MatlabJ. Bouguet – [1998-2000] http://www.vision.caltech.edu/bouguetj/calib_doc/index.html#examples
![Page 55: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/55.jpg)
Calibration Procedure
![Page 56: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/56.jpg)
Calibration Procedure
![Page 57: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/57.jpg)
Calibration Procedure
![Page 58: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/58.jpg)
Calibration Procedure
![Page 59: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/59.jpg)
Calibration Procedure
![Page 60: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/60.jpg)
Calibration Procedure
![Page 61: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/61.jpg)
Calibration Procedure
![Page 62: lecture3 camera calibration-3 · 2015-01-27 · Silvio Savarese Lecture 3 - 14#Jan#15 •.Recap.of.cameramodels •.Cameracalibration.problem •.Cameracalibration.with.radial.distortion](https://reader033.vdocuments.site/reader033/viewer/2022050103/5f4223c718cfc32c1f038fff/html5/thumbnails/62.jpg)
Next lecture
• Single view reconstruction