eecs 442 – computer vision single view...

EECS 442 Computer vision

Single view metrology

Review calibration Lines and planes at infinity Absolute conic Estimating geometry from a single image Extensions

Reading: [HZ] Chapters 2,3,8

Calibration Problem

jC

ii PMP

=

i

ii v

up

In pixels

[ ]TRKM =

=

100v0

ucot

10000

001

K o

o

sin1

World ref. system

Calibration Problem

jC

ii PMP

=

i

ii v

up

Need at least 6 correspondences11 unknown

[ ]TRKM =

Pinhole perspective projectionOnce the camera is calibrated...

[ ]TRKM =C

Ow

-Internal parameters K are known-R, T are known but these can only relate C to the calibration rig

Pp

Can I estimate P from the measurement p from a single image?

Pinhole perspective projectionRecovering structure from a single view

C

Ow

Pp

unknown known Known/Partially known/unknown

Recovering structure from a single viewSaxena, Sun, Ng, 06

Review calibration Lines and planes at infinity Absolute conic Estimating geometry from a single image Examples

Lines in a 2D plane

0cbyax =++

-c/b

-a/b

=

cba

l

If x = [ x1, x2]T l 0cba

1xx T

2

1

=

l

x

y

Lines in a 2D plane

Intersecting lines

llx = l

l

Proof

lll lll

0l)ll( =0l)ll( =

lxx

lx x is the intersecting point

x

y

x

Points and planes in 3D

=

1xxx

x3

2

1

=

dcba

0dczbyax =+++

x

y

z

0xx T =

How about lines in 3D? Lines have 4 degrees of freedom - hard to represent in 3D-space Can be defined as intersection of 2 planes

Points at infinity (ideal points)

0x,xxx

x 33

2

1

=

What happens if x3 = 0 ?

=

cba

l

=

cba

l

=0a

b)cc(llLets intersect two parallel lines:

Agree with the general idea of two lines intersecting at infinity

l

l

Lines infinity l

T

2

1

0xx

Set of ideal points lies on a line called the line at infinityHow does it look like?

0100

=

Indeed:

l

=

100

l

Projective transformation of a line (in 2D)

lHl T=

=

bvtA

H

?lH T =

is it a line at infinity?

?lH TA =

=

=

=

100

100

10

100

10 TTTT

AtAtA


lH T

horizon

- Recognize the horizon line- Measure if the 2 lines meet at the horizon

- if yes, these 2 lines are // in 3DRecognition helps reconstruction!Humans have learnt this

Are these two lines parallel or not?

Vanishing points

[ ]0IKM =dv K=

Points where parallel lines intersect in 3D(= ideal points in 2D)

d=direction of the line

d

C

v

Vanishing points - example

C2

v2

C1

v1

R,T

star

11

11

1 KK

vvd

=

21

21

2 KK

vvd

=

21 R dd = R

v1, v2: measurementsK = known and constant

11

11

1 KK

vvd

=

21

21

2 KK

vvd

=

R

Planes at infinity

=

1000

x

y

z

2 planes are parallel iff their intersections is a line that belongs to

Vanishing lines

C

n

Parallel planes intersect the plane at infinity in a common line the vanishing line (horizon)

horizTK ln =


lH T

horizon

horizTK ln =

Conics in 2D

- If a point x C

0feydxcybxyax 22 =+++++

In homogeneous coordinates:

0xCxT =

=

f2/e2/d2/ec2/b2/d2/ba

C

=

1xx

x 21

0xCxT =- If a line l is tangent to a conic in x xCl =

1T PCPC =- Projective transformation of conics:

Circular points

l

+=0i

1pi

=0i

1p j

Circular points(point at infinity)

Circular points are fixed under similarity transformation

=

=

0i1

0i1

100tcosssinstsinscoss

pH yx

is

i2=-1

Conic C

Circular points define a degenerate

conic called ; anyC

==+0x

0xx

3

22

21

Cx

=

000010001

C0xCxT =

l

+=0i

1pi

=0i

1p j

Circular points(point at infinity)

is fixed under similarity transformationC

In 3D: absolute conic Cis a

Any x satisfies:

=

01

11

0xxT =

==++

0x0xxx

4

23

22

21

is fixed under similarity transformation

Projective transformation of conics in 3D1T PCPC =

Projective transformation of

?1 = PP T

1T )KR()KR(

1T )KR(I)KR( =11TT KRRK =

=== 1T11T )KK(KRRK

[ ]TRKM =

=

01

11


1T )KK( =

It is not function of R, T

=

654

532

421

symmetric

02 = zero-skew 31 =square pixel

02 =

1.

2.

3. 4.


1T )KK( =

Why this is useful?

Angle between 2 scene lines

2T21

T1

2T1

vvvvvvcos

=

C

d1v1

dv K=v2

d2

If 90= 0vv 2T1 =

Angle between 2 scene lines

90=

0vv 2T1 =

v1v2

1T )KK( =Constraint on K

Single view calibration - example

v2

v3

0vv 2T1 =

0vv 3T1 =

0vv 3T2 =

=

654

532

421

31 =02 =

Compute :6 unknown5 constraints known up to scale

1T )KK( =K (Cholesky factorization)

v2

Single view reconstruction - example

lh

knownK horizTK ln = = Scene plane orientation in

the camera reference system

Select orientation discontinuities


lH T

horizon

- Recognize the horizon line- Measure if the 2 lines meet at the horizon

- if yes, these 2 lines are // in 3DRecognition helps reconstruction!Humans have learnt this


Single view reconstruction - example

Recover the structure within the camera reference systemNotice: the actual scale of the scene is NOT recovered

C

- Recognize the horizon line- Measure if the 2 lines meet

at the horizon- if yes, these 2 lines are // in 3D

Recognition helps reconstruction!Humans have learnt this


Criminisi & Zisserman, 99

http://www.robots.ox.ac.uk/~vgg/projects/SingleView/models/merton/merton.wrl

La Trinita' (1426) Firenze, Santa Maria Novella; by Masaccio (1401-1428)

http://www.robots.ox.ac.uk/~vgg/projects/SingleView/models/hut/hutme.wrl

Manually select: Vanishing points and lines; Planar surfaces; Occluding boundaries; Etc..

Single view reconstruction - drawbacks

Automatic Photo Pop-upHoiem et al, 05

Automatic Photo Pop-upHoiem et al, 05

http://www.cs.uiuc.edu/homes/dhoiem/projects/software.html

Software:

Single Image Depth ReconstructionSaxena, Sun, Ng, 05

A software: Make3DConvert your image into 3d model

Single Image Depth ReconstructionSaxena, Sun, Ng, 05

http://make3d.stanford.edu/

http://make3d.stanford.edu/images/view3D/185http://make3d.stanford.edu/images/view3D/931?noforward=truehttp://make3d.stanford.edu/images/view3D/108

http://make3d.stanford.edu/images/view3D/185http://make3d.stanford.edu/images/view3D/931?noforward=true

Next lecture:

Multi-view geometry (epipolar geometry)

EECS 442 Computer visionSingle view metrology Slide Number 2Slide Number 3Slide Number 4Slide Number 5Slide Number 6Slide Number 7Slide Number 8Slide Number 9Slide Number 10Slide Number 11Slide Number 12Slide Number 13Slide Number 14Slide Number 15Slide Number 16Slide Number 17Slide Number 18Slide Number 19Slide Number 20Slide Number 21Slide Number 22Slide Number 23Slide Number 24Slide Number 25Slide Number 26Slide Number 27Slide Number 28Slide Number 29Slide Number 30Slide Number 31Slide Number 32Slide Number 35Slide Number 36Slide Number 37Slide Number 38Slide Number 39Slide Number 41Slide Number 42Slide Number 43Slide Number 44Slide Number 45Slide Number 46Slide Number 47Slide Number 48

eecs 442 – computer vision single view...

Documents