robot vision ss 2009 matthias rüther 1 robot vision lesson 4: camera models and calibration...

Robot Vision SS 2009 Matthias Rüther 1

ROBOT VISION Lesson 4: Camera Models and Calibration

Matthias Rüther

Slides partial courtesy of Marc Pollefeys Department of Computer Science

University of North Carolina, Chapel Hill


Content

Camera Models– Pinhole Camera

– CCD Camera

– Finite Projective Camera

– Affine Camera

– Pushbroom Camera

Calibration– Inner Orientation

– Nonlinear Distortion

– Calibration using Planar Targets

– Calibration using a 3D Target

The Cyclops, 1914 by Odilon Redon


Basic Pinhole Camera Model


TT ZfYZfXZYX )/,/(),,(

101

0

0

1

Z

Y

X

f

f

Z

fY

fX

Z

Y

X



101

0

0

Z

Y

X

f

f

Z

fY

fX

101

01

01

1Z

Y

X

f

f

Z

fY

fX

PXx

0|I)1,,(diagP ff



Tyx

T pZfYpZfXZYX )/,/(),,(

principal pointT

yx pp ),(

0

0

1 01 1

x x

y y

X XfX Zp f p

Y YfY Zp f p

Z ZZ

Principal Point Offset


101

0

0

Z

Y

X

pf

pf

Z

ZpfY

ZpfX

y

x

x

x

camX0|IKx

1y

x

pf

pf

K calibration matrix

Principal Point Offset


C~

-X~

RX~

cam

X10

RCR

1

10

C~

RRXcam

Z

Y

X

camX0|IKx XC~

|IKRx

t|RKP C~

Rt PXx

Camera Rotation and Translation


1yy

xx

p

p

K

11y

x

y

x

pf

pf

m

m

K

CCD Camera


1yy

xx

p

ps

K

1yy

xx

p

p

K

C~

|IKRP

non-singular

11 dof (5+3+3)

decompose P in K,R,C?

4p|MP 41pMC

~ MRK, RQ

{finite cameras}={P3x4 | det M≠0}

If rank P=3, but rank M<3, then cam at infinity

Finite Projective Camera


PXx

MdDp|MPDx 4

Forward projection (3D -> 2D)

Back-projection (2D -> 3D)

xPX 1PPPP

TT IPP

(pseudo-inverse)

0PC

λCxPλX

1

p-μxM

1

pM-

0

xMμλX 4

-14

-1-1

xMd -1

CD

Action of Projective Cameras on Points

D…direction


1yx

xx

p

ps

K

1

arctan(1/s)

for CCD/CMOS, always s=0

Image from image, s≠0 possible(non coinciding principal axis)

HPresulting camera:

When is skew non-zero?


00

dP

Camera center at infinity

0Mdet

Affine and non-affine cameras

Definition: affine camera has P3T=(0,0,0,1)

Moving the Camera Center to Infinity


Affine Cameras


1000

0010

0001

P canonical representation

10

0KK 22

affine calibration matrix

principal point is not defined

Parallel Projection: Summary


Orthographic projection

Scaled orthographic projection

1000

0010

0001

P

10

tRH

1T1

2T2

r

P r

0 1

t

t

1T1

2T2

r

P r

0 1/

t

t

k

(5dof)

(6dof)

A Hierarchy of Affine Cameras


Weak perspective projection

ktt

y

x

/10rr

1α

αP 2

1T1

1T

(7dof)



1. Affine camera=camera with principal plane coinciding with ∞

2. Affine camera maps parallel lines to parallel lines

3. No center of projection, but direction of projection PAD=0(point on ∞)

Affine camera

ktts

y

x

A

/10rr

1α

αP 2

1T1

1T

(8dof)

1000P 2232221

1131211

tmmmtmmm

A

affine 44100000100001

affine 33P

A



T)(X X,Y,X,T T),,(PX wyx T)/,( wyx

Straight lines are not mapped to straight lines!(otherwise it would be a projective camera)

(11dof)

Pushbroom Cameras


ZYX

pppppp

yx

232221

131211(5dof)

Null-space PC=0 yields camera center

Also decomposition c~|IRKP 22222232

Line Cameras


Camera calibration


ii xX ? P

Problem Statement


ii PXx

ii PXx

0Ap

Basic Equations


0Ap

minimal solution

Over-determined solution

5½ correspondences needed (say 6)

P has 11 dof, 2 independent eq./points

n 6 points

Apminimize subject to constraint

1p

1p̂3 3p̂

P

Basic Equations


Geometric Error


Gold Standard algorithm

ObjectiveGiven n≥6 2D to 2D point correspondences {Xi↔xi’}, determine the Maximum Likelihood Estimation of P

Algorithm

(i) Linear solution:

(a) Normalization:

(b) DLT:

(ii) Minimization of geometric error: using the linear estimate as a starting point minimize the geometric error:

(iii) Denormalization:

ii UXX~ ii Txx~

UP~

TP -1

~ ~~


Calibrated camera, position and orientation unkown

Pose estimation

6 dof 3 points minimal (4 solutions in general)

Exterior Orientation


short and long focal length

Nonlinear Distortion

Radial Component



Radial Component


Correction of radial distortion

Computing the parameters of the distortion function(i) Minimize with additional unknowns(ii) Straighten lines(iii) …

Correction of radial Distortion



Tangential Component

Distortion function:


Distortion Workflow (KU wp1)

Project world points: x = P*X

Normalize projected points: xn = K-1 * x

Apply distortion to xn : xd = fdistort(xn)

Denormalize: xp = K * xd

robot vision ss 2009 matthias rüther 1 robot vision lesson 4: camera models and calibration...

Documents