robotica - home.deib.polimi.ithome.deib.polimi.it/restelli/mywebsite/pdf/lezione2.pdf · 13/10/2006...

Robotica Robotica

Anno accademico 2006/2007

Davide [email protected]

13/10/200613/10/2006 Corso di Robotica 06/07Corso di Robotica 06/07Davide Migliore - [email protected] Migliore - [email protected]

22

TodayToday

● The last episode ● A survey on Geometry Trasformations ● More useful information about Cameras● How to calibrate a camera

– Example from Bouguet Calibration Toolbox


33

The last episodeThe last episode


44

Modeling projectionModeling projection

Projection equations– Compute intersection with PP of ray from (x,y,z) to COP– Derived using similar triangles (on board)

– Ignoring the final image coordinate: (from 3D to 2D)

x= fXZ

y= f YZ

X ,Y , Z T f XZ, fYZT


55

Principal Point OffsetPrincipal Point Offset

X ,Y , Z T f XZ px , f

YZ p y

T

● The origin oh coordinates in image planes is not at the principal point p x , p yT

XYZ1 fXZpx

fYZp yZ =[ f 0 px 0

0 f p y 00 0 1 0 ] XYZ1


66

Camera Calibration Matrix (K)Camera Calibration Matrix (K)

K=[ f 0 px0 f p y0 0 1 ]

x=K [ I∣0] X cam● Rewriting the equation:

● The camera is assumed to be located at the origin of Euclidean coordinate system, with principal axis of camera pointing down the Z-axis: this system may be called camera coordinate frame.


77

Camera Rotation and TraslationCamera Rotation and Traslation

X cam=R X− C is an inhomogeneus point in world coordinate frame

represents the same point in camera coordiante frame

represents the coordinate of the camera centre in world coordinate frame

X cam=[R −R C0 1 ][ XY1 ]

x=KR [ I∣− C ] X

X

X cam

C


88

Camera Rotation and TraslationCamera Rotation and Traslation

● It is often convenient not to make the camera centre explicit:

X cam=R Xt t=−R C

P=K [R∣t ] Camera MatrixCamera Matrix


99

TrasformationsTrasformations


1010

Projective TrasformationsProjective Trasformations

A projectivity is an invertible mapping h from P2 to itself such that three points x1,x2,x3 lie on the same line if and only if h(x1),h(x2),h(x3) do.

Definition:

A mapping h:P2→P2 is a projectivity if and only if there exist a non-singular 3x3 matrix H such that for any point in P2 reprented by a vector x it is true that h(x)=Hx

Theorem:

Definition: Projective transformation

=

3

2

1

333231

232221

131211

3

2

1

'

'

'

x

x

x

hhh

hhh

hhh

x

x

x

xx' H=or

projectivity=collineation=projective transformation=homography


1111

Mapping between planesMapping between planes

central projection may be expressed by x’=Hx(application of theorem)


1212

RRemoving projective distortionemoving projective distortion

333231

131211

3

1

'

''

hyhxh

hyhxh

x

xx

++++==

333231

232221

3

2

'

''

hyhxh

hyhxh

x

xy

++++==

( ) 131211333231' hyhxhhyhxhx ++=++( ) 232221333231' hyhxhhyhxhy ++=++

select four points in a plane with know coordinates

(linear in hij)

(2 constraints/point, 8DOF ⇒ 4 points needed)

Remark: no calibration at all necessarybetter ways to compute


1313

A hierarchy of trasformationsA hierarchy of trasformations

• Projective linear group

• Affine group (last row (0,0,1))

• Euclidean group (upper left 2x2 orthogonal)

– Oriented Euclidean group (upper left 2x2 det 1)

• Alternative, characterize transformation in terms of elements or quantities that are preserved or invariant

e.g. Euclidean transformations (traslation and rotation) leave distances unchanged


1414

Class I: IsometriesClass I: Isometries

● Trasformation of the plane R2 that preserve Euclidean distance (iso = same, metric= measure )

● A planar Euclidean trasformation can be written:

−=

1100

cossin

sincos

1

'

'

y

x

t

t

y

x

y

x

θθεθθε

1=ε1−=ε

Orientation preserving (Euclidean trasformation):

Orientation reversing:

x0

xx'

==

1

tT

RHE IRR =T

Invariants: length, angle, area


1515

Class II: SimilarityClass II: Similarity

● An isometry composed with an isotropic scaling

● And in block form:

−=

1100

cossin

sincos

1

'

'

y

x

tss

tss

y

x

y

x

θθθθ

x0

xx'

==

1

tT

RH

sS IRR =T

Invariants: ratios of length, angle, ratios of areas, parallel lines


1616

Class III: Affine tran.Class III: Affine tran.

● A non-singular linear trasformation followed by a traslation

● And in block form:

=

11001

'

'

2221

1211

y

x

taa

taa

y

x

y

x

x0

xx'

==

1

tT

AH A

Invariants: parallel lines, ratios of parallel lengths, ratios of areas


1717

Class IV: Projective tras.Class IV: Projective tras.

● A general non-singular linear trasformation of homogeneous coordinate

xv

xx'

==

vP T

tAH ( ) T

21,v vv=

Invariants: cross-ratio of four points on a line (ratio of ratio)


1818

What is Cross-Ratio?What is Cross-Ratio?

● 1D case:

x'x 22×=H( ) T21, xx

=

22

11detx,x

ji

jiji xx

xxCrossx1,x2,x3,x4=∣x1,x3∣∣x2,x4∣∣x2,x3∣∣x1,x4∣


1919

The line at infinityThe line at infinity

● Affine trasformation:

● Projective trasformation:

=

000 2

1

2

1

x

xx

x

vAA

T

t

+

=

2211

2

1

2

1

0v

xvxvx

xx

x

vAA

T

t


2020

Overview transformationsOverview transformations

1002221

1211

y

x

taa

taa

1002221

1211

y

x

tsrsr

tsrsr

333231

232221

131211

hhh

hhh

hhh

1002221

1211

y

x

trr

trr

Projective8dof

Affine6dof

Similarity4dof

Euclidean3dof

Concurrency, collinearity, order of contact (intersection, tangency, inflection, etc.), cross ratio

Parallellism, ratio of areas, ratio of lengths on parallel lines (e.g midpoints), linear combinations of vectors (centroids). The line at infinity l∞

Ratios of lengths, angles.The circular points I,J

lengths, areas.


2121

Overview transformations in Overview transformations in 3D3D

10

tRT

s

Projective15dof

Affine12dof

Similarity7dof

Euclidean6dof

Intersection and tangency

Parallellism of planes,Volume ratios, centroids,The plane at infinity π∞

The absolute conic Ω∞

Volume

10

tRT

[A t0T 1]

[A tV T v ]


2222

More useful information about Cameras


2323

CCD CamerasCCD Cameras

● The pinhole camera model assumes that image coordinate Euclidean– Equal scales in both axial direction

● In CCD there is the possibility of non-square pixels

● Image coordinates measured in pixel introduces unequal scale factors in each direction


2424

CCD CamerasCCD Cameras

● If the number of pixels per unit distance coordinates are mx and my in the x and y direction:

● Where:

– e represent the focal length of camera in term of pixel dimensions

– is the principal point in terms of pizel dimensions with coordinates

K=[x 0 x00 y y00 0 1 ]

x= f mx y= f m y

x0= x0 , y0x0=mx px y0=m y p y


2525

Finite Projective cameraFinite Projective camera

● For added generality:

● Where s is referred to as the skew parameter– Usually is zero for most normal cameras

K=[x s x00 y y00 0 1 ]


2626

Finite Projective cameraFinite Projective camera

● A camera for which the calibration matrix K is of the previous form will be called a finite projective camera

● Notes:– The matrix KR is 3x3 non-singular (a signature)

– Usually you have P so decomposition is needed

P=KR [ I∣− C ]

P=M [ I∣M−1 p4]=KR [ I∣− C ]P=[ p11 p12 p13 p14p21 p22 p23 p24p31 p32 p33 p34

]


2727

Camera anatomyCamera anatomy

•Camera center

•Column points

•Principal plane

•Axis plane

•Principal point

•Principal ray


2828

CameraCamera centercenter

● Do you remember?

● Then

P=M [ I∣M−1 p4]=KR [ I∣− C ]

C= CT ,1T

C=−M−1 p41


2929

Column VectorsColumn Vectors

● Vanishing points of the world coordinate X,Y,Z

p1=PD x

Dx=1000


3030

Row VectorsRow Vectors

● The row of the projective camera are 4-vectors which may be interpreted geometrically as particular world plane

P=[ p11 p12 p13 p14p21 p22 p23 p24p31 p32 p33 p34

]=[P1T

P2T

P3T]


3131

Principal PlanePrincipal Plane

● The plane through the camera centre parallel to the image plane

It consists of the set of point X which are imaged on the line at infinity of the image

=

1p

p

p

0 3

2

1

Z

Y

X

y

x

T

T

T


3232

Axis PlaneAxis Plane

● Points that satisfy or

=

1p

p

p0

3

2

1

Z

Y

X

w

yT

T

T

P1T X=0 P2T X=0


3333

The principal pointThe principal point

● The principal axis is the line passing through the camera centre C with direction perpendicular to the principal plane P3 (the axis intersects the image plane at principal point)

principal point

( )0,,,p̂ 3332313 ppp=

∞

330 Mmp̂Px ==P=[M∣p4]


3434

Camera CalibrationCamera Calibration

● Common approach

Planar patternNon-planar pattern


3535

Camera Calibration from Camera Calibration from Planar PatternsPlanar Patterns

● ICCV Zhang’99: “Flexible Calibration by Viewing a Plane From Unknown Orientations”

m = [Camera Projection Matrix] M

A [R t]

observedestimate: A [R t] M

Minimize:


3636


● ICCV Zhang’99: “Flexible Calibration by Viewing a Plane From Unknown Orientations”

m = [Camera Projection Matrix] M

A [R t]

observedestimate: A [R t] M

Minimize:

• Two steps:• Find an initial solution for A [R t]• Minimize the objective function

using the initial solution


3737


• Finding an initial solution– First step

• Estimate the image homography matrix H for each image– Second step

• Solve for b in the linear system:• b yields the intrinsic parameter matrix A.

V b = 0


3838


● Intrinsic Parameters:

● Extrinsic Parameters

robotica - home.deib.polimi.ithome.deib.polimi.it/restelli/mywebsite/pdf/lezione2.pdf · 13/10/2006...

Documents