new cs664 computer vision 5. image geometry - cornell university · 2008. 2. 7. · 11 2-d rotation...

CS664 Computer Vision

5. Image Geometry

Dan Huttenlocher

2

First Assignment

Wells paper handout for question 1Question 2 more open ended– Less accurate approximations

• Simple box filtering doesn’t work

– Anisotropic, spatially dependent

3

Image Warping

T

T

f

f g

g

Image filtering: change range of imageg(x) = T f(x)

Image warping: change domain of image

g(x) = f(T(x))

4

Feature Detection in Images

Filtering to provide area of support– Gaussian, bilateral, …

Measures of local image difference– Edges, corners

More sophisticated features are invariant to certain transformations or warps of the image– E.g., as occur when viewing direction changes

5

Parametric (Global) Warping

Examples of parametric warps:

translation rotation

affine projectivecylindrical

6

Parametric (Global) Warping

p’ = T(p) What does it mean that T is global?– Same function for any point p– Described by a few parameters, often matrix

T

p = (x,y) p’ = (x’,y’)

⎥⎦

⎤⎢⎣

⎡=⎥

⎦

⎤⎢⎣

⎡yx

yx

M''

p’ = M*p

7

Scaling

Scaling a coordinate means multiplying each of its components (axes) by a scalarUniform scaling means this scalar is the same for all components:

× 2

8

Non-uniform scaling: different scalars per component:

Scaling

X × 2,Y × 0.5

9

Scaling

Scaling operation:

Or, in matrix form:

byyaxx

==''

⎥⎦

⎤⎢⎣

⎡⎥⎦

⎤⎢⎣

⎡=⎥

⎦

⎤⎢⎣

⎡yx

ba

yx

00

''

scaling matrix S

What’s inverse of S?

10

2-D Rotation

θ

(x, y)

(x’, y’)

x’ = x cos(θ) - y sin(θ)y’ = x sin(θ) + y cos(θ)

About origin

11

2-D Rotation

This is easy to capture in matrix form:

Even though sin(θ) and cos(θ) are nonlinear functions of θ,– x’ is a linear combination of x and y– y’ is a linear combination of x and y

Inverse transformation, rotation by –θ– For rotation matrices, det(R) = 1 and

( ) ( )( ) ( ) ⎥

⎦

⎤⎢⎣

⎡⎥⎦

⎤⎢⎣

⎡ −=⎥

⎦

⎤⎢⎣

⎡yx

yx

θθθθ

cossinsincos

''

TRR =−1

R

12

2x2 Transformation Matrices

What types of transformations can be represented with a 2x2 matrix?

2D Identity? (A rotation)

yyxx

==''

⎥⎦⎤

⎢⎣⎡⎥⎦⎤

⎢⎣⎡=⎥⎦

⎤⎢⎣⎡

yx

yx

1001

''

2D Scale around (0,0)?

ysy

xsx

y

x

*'

*'

=

=⎥⎦

⎤⎢⎣

⎡⎥⎦

⎤⎢⎣

⎡=⎥

⎦

⎤⎢⎣

⎡yx

ss

yx

y

x

00

''

13



2D Rotate around (0,0)?

yxyyxx

*cos*sin'*sin*cos'

Θ+Θ=Θ−Θ= ⎥

⎦

⎤⎢⎣

⎡⎥⎦

⎤⎢⎣

⎡ΘΘΘ−Θ

=⎥⎦

⎤⎢⎣

⎡yx

yx

cossinsincos

''

2D Shear?

yxshyyshxx

y

x

+=+=

*'*' ⎥

⎦

⎤⎢⎣

⎡⎥⎦

⎤⎢⎣

⎡=⎥

⎦

⎤⎢⎣

⎡yx

shsh

yx

y

x

11

''

14



2D Mirror about Y axis?

yyxx

=−=

''

⎥⎦⎤

⎢⎣⎡⎥⎦⎤

⎢⎣⎡−=⎥⎦

⎤⎢⎣⎡

yx

yx

1001

''

2D Mirror over (0,0)?

yyxx

−=−=

''

⎥⎦⎤

⎢⎣⎡⎥⎦⎤

⎢⎣⎡

−−=⎥⎦

⎤⎢⎣⎡

yx

yx

1001

''

15

All 2D Linear Transformations

Linear transformations are combinations of …– Scale,– Rotation,– Shear, and– Mirror

Properties of linear transformations:– Origin maps to origin– Lines map to lines– Parallel lines remain parallel– Ratios are preserved– Closed under composition

⎥⎦

⎤⎢⎣

⎡⎥⎦

⎤⎢⎣

⎡=⎥

⎦

⎤⎢⎣

⎡yx

dcba

yx

''

⎥⎦⎤

⎢⎣⎡⎥⎦⎤

⎢⎣⎡⎥⎦⎤

⎢⎣⎡⎥⎦⎤

⎢⎣⎡=⎥⎦

⎤⎢⎣⎡

yx

lkji

hgfe

dcba

yx

''

16

Linear Transformations as Change of Basis

Any linear transformation is a basis!

j =(0,1)

i =(1,0)

p

p=4i+3j = (4,3) p’=4u+3vpx’=4ux+3vxpy’=4uy+3vy

v =(vx,vy)

u=(ux,uy)

p’

pp ⎥⎦

⎤⎢⎣

⎡=⎥

⎦

⎤⎢⎣

⎡⎥⎦

⎤⎢⎣

⎡=

yy

xx

yy

xx

vuvu

vuvu

34

'

17



2D Translation?

y

x

tyytxx

+=+=

''

Only linear 2D transformations can be represented with a 2x2 matrix

NO!

18

Homogeneous Coordinates

How can we represent translation as matrix?

Homogeneous coordinates– Represent coordinates in 2 dimensions with a 3-

vector

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡⎯⎯⎯⎯ →⎯⎥

⎦

⎤⎢⎣

⎡

1

coords shomogeneou yx

yx

y

x

tyytxx

+=+=

''

19


Add a 3rd coordinate to every 2D point– (x, y, w) represents a point at 2D location

(x/w, y/w)– (x, y, 0) represents a point at infinity– (0, 0, 0) is not allowed

1 2

1

2 (2,1,1) or (4,2,2) or (6,3,3)

x

y

20


⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

=100

1001

y

x

tt

ranslationT

y

x

tyytxx

+=+=

''

How can we represent translation as matrix?

Last column of homogeneous matrix

21

Translation

Example of translation in homogeneous coordinates

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡++

=⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

=⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

111001001

1''

y

x

y

x

tytx

yx

tt

yx

tx = 2ty = 1

22

Homogeneous 2D Transformations

Basic 2D transformations as 3x3 matrices

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡ΘΘΘ−Θ

=⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

11000cossin0sincos

1''

yx

yx

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

11001001

1''

yx

tt

yx

y

x

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

11000101

1''

yx

shsh

yx

y

x

Translate

Rotate Shear

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

11000000

1''

yx

ss

yx

y

x

Scale

23

Affine Transformations

Affine transformations are …– Linear transformations, and– Translations

Properties of affine transformations:– Origin does not necessarily map to origin– Lines map to lines– Parallel lines remain parallel– Ratios are preserved– Closed under composition– Models change of basis– Maps any triangle to any triangle (or parallelogram)

⎥⎥⎦

⎤

⎢⎢⎣

⎡

⎥⎥⎦

⎤

⎢⎢⎣

⎡=

⎥⎥⎦

⎤

⎢⎢⎣

⎡

wyx

fedcba

wyx

100''

24

Projective Transformations

Projective transformations …– Affine transformations, and– Projective warps

Properties of projective transformations:– Origin does not necessarily map to origin– Lines map to lines– Parallel lines do not necessarily remain parallel– Ratios are not preserved– Closed under composition– Models change of basis– Maps any quadrilateral to any quadrilateral

⎥⎥⎦

⎤

⎢⎢⎣

⎡

⎥⎥⎦

⎤

⎢⎢⎣

⎡=

⎥⎥⎦

⎤

⎢⎢⎣

⎡

wyx

ihgfedcba

wyx

'''

25

Matrix Composition

Transformations can be combined (composed) by matrix multiplication

⎥⎥⎦

⎤

⎢⎢⎣

⎡

⎟⎟⎟

⎠

⎞

⎜⎜⎜

⎝

⎛

⎥⎥⎦

⎤

⎢⎢⎣

⎡

⎥⎥⎦

⎤

⎢⎢⎣

⎡ΘΘΘ−Θ

⎥⎥⎦

⎤

⎢⎢⎣

⎡=

⎥⎥⎦

⎤

⎢⎢⎣

⎡

wyx

sysx

tytx

wyx

1000000

1000cossin0sincos

1001001

'''

p’ = T(tx,ty) R(Θ) S(sx,sy) p

26

2D image transformations

• Nested set of groups

27

Affine Invariants

Affine transformations preserve collinearity of points: – If 3 points lie on the same line, their images under affine

transformations also line on the same line and, – The middle point remains between the other two points

Concurrent lines remain concurrent (images of intersecting lines intersect), Ratio of length of line segments of a given line remains constant Ratio of areas of two triangles remains constant, Ellipses remain ellipses and the same is true for parabolas and hyperbolas

28

3D Interpretations

Orthographic projection plus scaling of Euclidean motion in 3D world yields 2D affine transformation– Often called weak perspective imaging model

Set of coplanar points in 3D– Undergo rigid body motion– Scaling – analogous to perspective size with

distance but same scaling for all the points– Projection into image plane (drop coordinate)

29

Orthographic Projection

Also called “parallel projection”: (x, y, z) → (x, y)What’s the projection matrix (homogeneous coords)?

Image World

30

Weak Perspective Imaging

Scaling in addition to parallel projection

Composed with 3D rigid body motion (6 dof)

Equivalent to affine transformation of plane (6 dof) up to reflection – state without proof

31

Projection – Pinhole Camera Model

Projection equations– Compute intersection with PP of ray from (x,y,z) to COP– Derived using similar triangles– Parallel projection where d infinite

32

Perspective Projection

Homogeneous coordinates

Divide by third coordinate

Scaling projection matrix, no effect

33

Perspective Projection

Composed with 3D rigid body motion (6 dof)Focal length, d, and distance z two additional parametersEquivalent to projective transformation of plane (homography)– 3x3 matrix in homogeneous coordinates, 8 dof– Again state without proof

2D affine and projective transformations correspond to images of plane in space under rigid motion, different imaging models

34

Image Warping

Given coordinate transform (x’,y’) =T(x,y) and source image f(x,y)How do we compute transformed image g(x’,y’) = f(T(x,y))?

x x’

T(x,y)

f(x,y) g(x’,y’)

y y’

35

f(x,y) g(x’,y’)

Forward Warping

Send each pixel f(x,y) to its corresponding location

(x’,y’) = T(x,y) in the second image

x x’

T(x,y)

Q: What about when pixel lands “between” two pixels?

y y’

36

f(x,y) g(x’,y’)

Forward Warping

Send each pixel f(x,y) to its corresponding location

(x’,y’) = T(x,y) in the second image

x x’

T(x,y)

Q: What about when pixel lands “between” two pixels?

y y’

A: Distribute color among neighboring pixels (x’,y’)– Known as “splatting”

Points at pixel centers

37

f(x,y) g(x’,y’)xy

Inverse Warping

Get each pixel g(x’,y’) from its corresponding location

(x,y) = T-1(x’,y’) in the first image

x x’

Q: What about when pixel comes from “between” two pixels?

y’T-1(x,y)

38

f(x,y) g(x’,y’)xy

Inverse Warping

Get each pixel g(x’,y’) from its corresponding location

(x,y) = T-1(x’,y’) in the first image

x x’

T-1(x,y)

Q: What about when pixel comes from “between” two pixels?

y’

A: Interpolate color value from neighbors– Nearest neighbor, bilinear, Gaussian, bicubic

39

Bilinear Interpolation (Reminder)

Sampling at f(x,y):

40

Forward vs. Inverse Warping

Q: Which is better?

A: Usually inverse – eliminates holes– However, requires an invertible warp function – not

always possible...

affine projective

41

Reference

R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, 2nd Ed., Cambridge Univ. Press, 2003.

new cs664 computer vision 5. image geometry - cornell university · 2008. 2. 7. · 11 2-d rotation...

Documents