CSC505

Texture Mapping

when shading just won’t do

CSC505

Shading

• Recall that surface shading (Gouraud/Phong) gave us a means for creating the illusion of a smooth 3D surface on a 2D display– Surface features are dynamically controlled based on light

source (value and location) and viewer location

• What about when we want a textured surface without increasing object complexity?– We could add more and more triangles resulting in more

and more surface normals but there is a limit

• This is where we resort to texture mapping

CSC505

Creating the Illusion of 3D Texture

• An analogy:• Consider the creation of a stage set for a play

– Prop hands build elaborate 3D buildings with all the painful details or…

– Prop hands cut appropriate shapes out of planar (flat) pieces of plywood

– The planar pieces are painted with appropriate designs

– Stage lighting and positioning gives the audience the feel of a 3D set

CSC505

Creating the Illusion of 3D Texture

• We want to achieve a similar effect in our rendering of graphic scenes

• We could create our surface out of lots and lots of small triangles each with its own surface normal

• Then, using shading and lighting models we could create the appearance of a textured scene

• Consider a stucco wall (and all the design and computation that would go into it)

CSC505

Texture Mapping

• Also known as image-based texture mapping

• Think of texture mapping as gift wrapping a present– You select the wrapping paper with the desired

pattern– You cover the package with the wrapping paper– In essence the wrapping paper appears as increased

surface detail on the package without adding any geometric complexity (it’s still flat and smooth)

CSC505

Texture Mapping

• Similar approach as used in polygon shading– Based on surface orientation (polygon vertices), lighting,

and other effects (for example, fog)– Scan through the interior points of a polygon (triangle)

interpolating a color value for each

• But, rather than interpolating a color we select one from a pre-stored image (the texture map)– Similar approach as used in ray tracing– Polygon vertices describe which image pixels to place at

which interior polygon points– The result is a complex looking surface on the polygon

with no more than a look-up table operation

CSC505

Texture Mapping

• Three basic steps:– Map points on a 3D surface into the 2D domain

• Similar to what we did with Gouraud shading

– Map the points in the 2D domain into a unit square• Clamping, modulus, multiplication, division operations

– Map the points within the unit square to image coordinates

• Scale the unit square up to the texture image size

CSC505

Texture Map (Image)

• 2-Dimensional array of color values

CSC505

Texture Map (Image)

height

width

PIXel ELements (pixels)

except that now we call them TEXture ELements (texels)

This is called texel-space

CSC505

Texture Coordinates

• Texel-space is good for representing/storing the texture image

• But for mapping it onto a polygon it is awkward due to the dependence on the width and height of the texture image

• Thus, we define something called UV-space where U and V are texture coordinates

CSC505

UV-Space

height

width

Origin(U,V) = (0.0, 0.0)

(U,V) = (1.0, 1.0)

(U,V) = (1.0, 0.0)

(U,V) = (0.0, 1.0)

• UV-Space is independent of texture image height and width

CSC505

UV-Space – Texture Image

• Things to be careful of if you’re creating your own texture mapper:– Origins may be in different locations

• Texture image – (0,0) is typically in the upper left with positive “y” going downward

• UV space – (0,0) is in the lower left with positive “v” going upward (graphics coordinates)

– Coordinate order may be swapped• Texture image – typically indexed [row][column]• UV space – indexed [column][row]

CSC505

Mapping

• So, all we have to do is convert our polygon coordinates to texture coordinates

• We’ll concentrate on triangles only since this is how graphic hardware works

• The steps are– Map barycentric coordinates (s, t) to UV-space coordinates

(u, v)

– Clamp or wrap (u, v) to the unit square

– Multiply (u, v) by (width, height) of the texture image

– Assign the texture image pixel to the triangle point

CSC505

Barycentric Coordinates(refresher)

• P0, P1, P2 is the triangle to be textured

• P is the point we are currently mapping (inside the triangle)

• w is a vector from P0 to P (e.g. P – P0)

• Define:

s = b / c

t = a / c• a and b are the areas of their

respective triangles and c is the area of the entire triangle

P2

P1

P0

Pa

b

u = P1 – P0

v = P2 – P0

w

CSC505

Vector Cross Product

• Areas of the various triangles can be computed using the vector cross product– Vector cross product between two vectors returns a vector

– Magnitude of the cross product can be computed using the trigonometric sin function

vwwvvwwvvwwv yxyxxzxzzyzywv ,,

sinwvwv

CSC505

Vector Cross Product

• The magnitude (length) of the cross product (vector) is equal to the area of the parallelogram with v and w as sides

• This turns out to be real handy since we are looking for the area of a triangle which happens to be half the area of the parallelogram

v

w

v X w

CSC505

a

2

uwa

Barycentric Coordinates

P2

P1

P0

Pa

b

u = P1 – P0

v = P2 – P0

w

P1

P0

P

u

w

wX u

• Similar for triangle b (v x w) and entire triangle (v x u)

CSC505

Mapping

• Now that we have the barycentric coordinates s and t we can map them to UV-space using the matrix equation:

• Where v1, v2, and v3 are the coordinates of the texture map that correspond to the vertices of the triangle we’re mapping onto (see next slide)

13

3

32

32

31

31 t

s

v

u

vv

uu

vv

uu

v

u

vv

vvvv

vvvv

CSC505

Resultant mapping

Mapping

Texture mapV1=(0, h-1)

V2=(0, 0)

V3=(w-1, h-1)

Triangle to be textured

height

width

CSC505

Mapping

• The mapping of texture map coordinates to triangle vertices is generally done by the graphic designers using tools created by programmers (i.e. it is not a real time operation)

• If you do your own texture mapper you’ll probably get the coordinates wrong at least once (it took me 3 or 4 times to get it right)

• The goal of texture mapping is to minimize distortions due to image warping

• There are schemes for anti-aliasing by neighborhood averaging texels

• There are schemes for wrapping in the event that you allow mappings to fall outside the unit square of UV-space

CSC505

Demo Program

Texture Map

Triangles

Result

By The Way

• Remember the blood spatter question I posed (to myself) last week?

• No one does it computationally

• They all use a form of texture mapping called decal application– A small texture map (picture) of a blood spatter

that is mapped onto the unfortunate character on the receiving end of the dastardly blow

CSC505

CSC505

Image Warping

CSC505

Image Warping

• In texture mapping we tried to not alter the image in “unnatural” ways– The goal is to make things look realistic

• In image warping the goal is to distort the image– We want to make things look unrealistic

CSC505

Image Warping

• In texture mapping we map portions of the texture image to a triangle– We want the process to be as efficient as possible

• In image warping we map the entire rectangular source image to a quadrilateral (convex)– We could break it up into triangles if we wanted to

but the original mapping is specified for a quadrilateral area

CSC505

Image Mapping

u

v Sourceimage

x

y Destinationimage

• Origins may be swapped (image processing style) but are typically consistent

• Contrary to what we saw in texture mapping

CSC505

Image Warping Mappings

• Various mapping techniques are possible– Affine– Projective– Bilinear– Beier-Neely– Piecewise cubic

• We’ll look at bilinear

CSC505

Image Warping

• Two basic methods:– Forward mapping: for each pixel in the source

image we determine the location it goes to in the destination image

– Inverse mapping: for each location in the destination image we determine which source image pixel goes into it

CSC505

Bilinear Mapping

• Here’s the setup…

DestinationImage

SourceImage

CSC505

Bilinear Mapping

• The equations that model a bilinear mapping between quadrilaterals are:

• To do our warp we need the values of the eight coefficients C1 through C8

• Fortunately we have eight equations from which to calculate them (we’re given 4 point correspondences)

cyxcycxcycyxcycxcx

8765

4321

'

'

CSC505

Bilinear Mapping

cyxcycxcycyxcycxcx

800706050

400302010

'

'

cyxcycxcycyxcycxcx

811716151

411312111

'

'

cyxcycxcycyxcycxcx

822726252

422322212

'

'

cyxcycxcycyxcycxcx

833736353

433332313

'

'

(x0,y0) (x1,y1)

(x2,y2)(x3,y3)

SourceImage

(x0’,y0’) (x1’,y1’)

(x2’,y2’)(x3’,y3’)

DestinationImage

CSC505

Bilinear Mapping

• All we need to do is to solve the equations using any standard numerical technique– Gaussian Elimination w/back substitution– Gauss-Jordan Elimination– LUD Decomposition

• Then we loop through the pixels using either forward or inverse mapping– Note that the roles of the (x, y) and (x’, y’) pairs

determine which direction we are mapping

CSC505

Bilinear Mapping

• By doing an inverse mapping you are guaranteed to produce an image without holes– Forward mapping may miss some destination pixel locations due to

round-off errors

• Typically one uses “nearest neighbor” mapping to select the pixel (just truncate the floating point location)

• Alternatively:– You can add filtering (neighborhood averages around pixels) to

eliminate aliasing– You can compute the mapping based on the corners of an individual

pixel and do a weighted mean based on how much of each source pixel the destination pixel covers

CSC505

Image Warping

• Finally, you can add alpha blending to create an image morph

Image Warp

CSC505

CSC505

PhotoMosaic

• Very simple process, conceptually

1. Load the image that you want to mosaic

2. Create a dictionary of tiling images– One entry for every color of the image of step 1

– Resized to your desired tile size

3. Create “super pixels” in the image of step 1– Average pixels together of the desired tile size

4. Replace “super pixels” with tile image of similar color

Inputs

CSC505

Tiling Image(will be adjusted to 256 different average intensities)

(will be resized to 16x16)

Input image to be mosaic’ed

Super pixels

CSC505

16x16 average

Output

CSC505

Output

CSC505