06 warping
TRANSCRIPT
-
7/30/2019 06 Warping
1/34
Image Warping
15-463: Computational PhotographyAlexei Efros, CMU, Fall 2005
Some slides from Steve Seitz
http://www.jeffrey-martin.com
http://www.jeffrey-martin.com/http://www.jeffrey-martin.com/ -
7/30/2019 06 Warping
2/34
Image Warping
image filtering: change range of image
g(x) = T(f(x))
f
x
T
f
x
f
x
T
f
x
image warping: change domain of image
g(x) = f(T(x))
-
7/30/2019 06 Warping
3/34
Image Warping
T
T
f
f g
g
image filtering: change range of image
g(x) = h(T(x))
image warping: change domain of image
g(x) = f(T(x))
-
7/30/2019 06 Warping
4/34
Parametric (global) warping
Examples of parametric warps:
translation rotation aspect
affine perspective cylindrical
-
7/30/2019 06 Warping
5/34
Parametric (global) warping
Transformation T is a coordinate-changing machine:
p =T(p)
What does it mean that T is global? Is the same for any point p
can be described by just a few numbers (parameters)
Lets represent T as a matrix:p =M*p
T
p = (x,y) p = (x,y)
=
yx
yx M
''
-
7/30/2019 06 Warping
6/34
Scaling
Scaling a coordinate means multiplying each of its components bya scalar
Uniform scaling means this scalar is the same for all components:
2
-
7/30/2019 06 Warping
7/34
Non-uniform scaling: different scalars per component:
Scaling
X 2,
Y 0.5
-
7/30/2019 06 Warping
8/34
Scaling
Scaling operation:
Or, in matrix form:
byyaxx
==''
=
y
x
b
a
y
x
0
0
'
'
scaling matrix S
Whats inverse of S?
-
7/30/2019 06 Warping
9/34
2-D Rotation
(x, y)
(x, y)
x = x cos() - y sin()y = x sin() + y cos()
-
7/30/2019 06 Warping
10/34
2-D Rotation
x = r cos ()y = r sin ()x = r cos ( + )y = r sin ( + )
Trig Identityx = r cos() cos() r sin() sin()y = r sin() sin() + r cos() cos()
Substitute
x = x cos() - y sin()y = x sin() + y cos()
(x, y)
(x, y)
-
7/30/2019 06 Warping
11/34
2-D Rotation
This is easy to capture in matrix form:
Even though sin() and cos() are nonlinear functions of, x is a linear combination of x and y
y is a linear combination of x and y
What is the inverse transformation?
Rotation by For rotation matrices, det(R) = 1 so
( ) ( )
( ) ( )
=
y
x
y
x
cossin
sincos
'
'
TRR =1
R
-
7/30/2019 06 Warping
12/34
2x2 Matrices
What types of transformations can berepresented with a 2x2 matrix?
2D Identity?
yy xx =='' = yxyx 10 01''
2D Scale around (0,0)?
ysy
xsx
y
x
*'
*'
=
=
=
y
x
s
s
y
x
y
x
0
0
'
'
-
7/30/2019 06 Warping
13/34
2x2 Matrices
What types of transformations can berepresented with a 2x2 matrix?
2D Rotate around (0,0)?
yxy yxx *cos*sin' *sin*cos' += =
=
yx
yx
cossinsincos
''
2D Shear?
yxshy
yshxx
y
x
+=
+=
*'
*'
=
y
x
sh
sh
y
x
y
x
1
1
'
'
-
7/30/2019 06 Warping
14/34
2x2 Matrices
What types of transformations can berepresented with a 2x2 matrix?
2D Mirror about Y axis?
yy xx = ='' = yxyx 10 01''
2D Mirror over (0,0)?
yyxx
==
''
=
y
x
y
x
10
01
'
'
-
7/30/2019 06 Warping
15/34
2x2 Matrices
What types of transformations can berepresented with a 2x2 matrix?
2D Translation?
y
x
tyytxx
+= +=''
Only linear 2D transformationscan be represented with a 2x2 matrix
NO!
-
7/30/2019 06 Warping
16/34
All 2D Linear Transformations
Linear transformations are combinations of Scale,
Rotation,
Shear, and Mirror
Properties of linear transformations: Origin maps to origin
Lines map to lines
Parallel lines remain parallel Ratios are preserved
Closed under composition
=
y
x
dc
ba
y
x
'
'
=
yx
lk
ji
hg
fedcba
yx
''
-
7/30/2019 06 Warping
17/34
Linear Transformations as Change of Basis
Any linear transformation is a basis!!! Whats the inverse transform? How can we change from any basis to any basis?
What if the basis are orthogonal?
j =(0,1)
i =(1,0)
p
p=4i+3j = (4,3) p=4u+3vpx=4ux+3vxpy=4uy+3vy
v =(vx,vy)
u=(ux,uy)
p
pp
=
=
yy
xx
yy
xx
vu
vu
vu
vu
3
4'
-
7/30/2019 06 Warping
18/34
Homogeneous Coordinates
Q: How can we represent translation as a 3x3matrix?
y
x
tyy
txx
+=+=
'
'
-
7/30/2019 06 Warping
19/34
Homogeneous Coordinates
Homogeneous coordinates represent coordinates in 2dimensions with a 3-vector
1
coordsshomogeneou
y
x
y
x
-
7/30/2019 06 Warping
20/34
Homogeneous Coordinates
Q: How can we represent translation as a 3x3matrix?
A: Using the rightmost column:
= 10010
01
y
x
t
t
ranslationT
y
x
tyy
txx
+=
+=
'
'
T l i
-
7/30/2019 06 Warping
21/34
Translation
Example of translation
++
=
=
11100
10
01
1
'
'
y
x
y
x
ty
tx
y
x
t
t
y
x
tx = 2
ty = 1
Homogeneous Coordinates
H C di t
-
7/30/2019 06 Warping
22/34
Homogeneous Coordinates
Add a 3rd coordinate to every 2D point (x, y, w) represents a point at location (x/w, y/w)
(x, y, 0) represents a point at infinity
(0, 0, 0) is not allowed
Convenientcoordinate system torepresent many
usefultransformations
1 2
1
2(2,1,1) or (4,2,2) or (6,3,3)
x
y
B i 2D T f ti
-
7/30/2019 06 Warping
23/34
Basic 2D Transformations
Basic 2D transformations as 3x3 matrices
=
1100
0cossin
0sincos
1
'
'
y
x
y
x
=
1100
10
01
1
'
'
y
x
t
t
y
x
y
x
=
1100
01
01
1
'
'
y
x
sh
sh
y
x
y
x
Translate
Rotate Shear
=
1100
00
00
1
'
'
y
x
s
s
y
x
y
x
Scale
Affi T f ti
-
7/30/2019 06 Warping
24/34
Affine Transformations
Affine transformations are combinations of
Linear transformations, and
Translations
Properties of affine transformations:
Origin does not necessarily map to origin
Lines map to lines
Parallel lines remain parallel
Ratios are preserved
Closed under composition
Models change of basis
=
w
yx
fedcba
w
yx
100
''
P j ti T f ti
-
7/30/2019 06 Warping
25/34
Projective Transformations
Projective transformations
Affine transformations, and
Projective warpsProperties of projective transformations:
Origin does not necessarily map to origin
Lines map to lines
Parallel lines do not necessarily remain parallel
Ratios are not preserved
Closed under composition
Models change of basis
=
w
yx
ihg
fedcba
w
yx
'
''
Matrix Composition
-
7/30/2019 06 Warping
26/34
Matrix Composition
Transformations can be combined bymatrix multiplication
=
w
yx
sysx
tytx
w
yx
1000000
1000cossin0sincos
1001001
'''
p = T(tx,ty) R() S(sx,sy) p
2D image transformations
-
7/30/2019 06 Warping
27/34
2D image transformations
These transformations are a nested set of groupsClosed under composition and inverse is a member
Image warping
-
7/30/2019 06 Warping
28/34
Image warping
Given a coordinate transform (x,y) =h(x,y) and asource image f(x,y), how do we compute atransformed image g(x,y) =f(T(x,y))?
x x
T(x,y)
g(x,y)
f(x,y)
y y
Forward warping
-
7/30/2019 06 Warping
29/34
f(x,y) g(x,y)
Forward warping
Send each pixel f(x,y) to its corresponding location
(x,y) =T(x,y) in the second image
x x
T(x,y)
Q: what if pixel lands betweentwo pixels?
y y
Forward warping
-
7/30/2019 06 Warping
30/34
f(x,y) g(x,y)
Forward warping
Send each pixel f(x,y) to its corresponding location
(x,y) =T(x,y) in the second image
x x
T(x,y)
Q: what if pixel lands betweentwo pixels?
y y
A: distribute color among neighboring pixels (x,y) Known as splatting
Inverse warping
-
7/30/2019 06 Warping
31/34
f(x,y) g(x,y)x
y
Inverse warping
Get each pixel g(x,y) from its corresponding location
(x,y) =T-1(x,y) in the first image
x x
yT-1(x,y)
Q: what if pixel comes from betweentwo pixels?
Inverse warping
-
7/30/2019 06 Warping
32/34
f(x,y) g(x,y)x
y
Inverse warping
Get each pixel g(x,y) from its corresponding location
(x,y) =T-1(x,y) in the first image
x x
T-1(x,y)
Q: what if pixel comes from betweentwo pixels?
y
A: Interpolate color value from neighbors nearest neighbor, bilinear, Gaussian, bicubic
Bilinear interpolation
-
7/30/2019 06 Warping
33/34
Bilinear interpolation
Sampling at f(x,y):
Forward vs inverse warping
-
7/30/2019 06 Warping
34/34
Forward vs. inverse warping
Q: which is better?
A: usually inverseeliminates holes
however, it requires an invertible warp functionnot always possible...