Displaced Subdivision Surfaces of Animated Meshes

Hyunjun Leea, Minsu Ahnb, Seungyong Leea,∗

aPohang University of Science and Technology (POSTECH)bSamsung Advanced Institute of Technology (SAIT)

Abstract

This paper proposes a novel technique for converting a given animated mesh into a series of displaced subdivisionsurfaces. Instead of independently converting each frame in the animated mesh, our technique produces displacedsubdivision surfaces that share the same topology of the control mesh and a single displacement map. We firstpropose a conversion framework that enables sharing the same control mesh topology and displacement map amongframes, and then present the details of the components in the framework. Each component is specifically designed tominimize the conversion errors that can be caused by enforcing a single displacement map. The resulting displacedsubdivision surfaces have a compact representation, while reproducing the details of the original animated mesh.The representation can also be used for efficient rendering by modern graphics hardware that supports acceleratedrendering of subdivision surfaces.

Keywords: displaced subdivision surfaces; animated meshes

1. Introduction

Modern video games, immersive applications, andcomputer-based movies make extensive use of complexand highly detailed animated models to achieve real-istic results. A variety of methods have been devel-oped to efficiently represent and render animated mod-els. Multiresolution subdivision surfaces provide an ef-fective approach for handling animated models [7], andhave been implemented as popular tools in the state-of-the-art modelers. They have also become a format ofchoice with the emergence of GPU-based tessellationmethods in modern graphics hardware. However, al-though the design of new subdivision-based animatedmodels is relatively simple, no effective method hasbeen developed to extract them from existing animatedmeshes, which were acquired using 3D scanning of dy-namic objects [1, 18].

An animated mesh consists of a sequence of meshes,where the meshes have the same topology (vertex con-nectivity) but changing vertex positions. A displacedsubdivision mesh consists of a control mesh and a dis-placement map, where the final shape is determined by

∗Corresponding authorEmail addresses: crowlove@postech.ac.kr (Hyunjun Lee),

minsu.ahn@samsung.com (Minsu Ahn), leesy@postech.ac.kr(Seungyong Lee)

adding the displacement map after recursive subdivisionof the control mesh [12]. In this paper, we consider theconversion of an animated mesh into a series of dis-placed subdivision surfaces. Surprisingly, this subjecthas received little attention from research communitydespite potential advantages of the conversion. In addi-tion to the rendering efficiency [24, 2], displaced subdi-vision surfaces can be used as a compact representationof animated models which reduces storage requirement.

To understand the problem, it is worth pointing outthat simply running the displaced subdivision surfacetechnique for a static mesh [12] on each mesh of theanimated mesh sequence is not sufficient. The con-trol meshes from different meshes would have differ-ent topologies and different displacement maps. Conse-quently, rendering and storage of the resulting displacedsubdivision meshes would be inefficient. This problemcan be resolved if we enforce the resulting displacedsubdivision surfaces to share the same topology of thecontrol mesh and a single displacement map. This rep-resentation can save the amount of storage and alloweffective rendering of animated meshes.

In this paper, we propose a conversion frameworkfrom an animated mesh into a series of displaced subdi-vision surfaces that share the same control mesh topol-ogy and displacement map. However, using a singledisplacement map may cause shape conversion errors

Preprint submitted to SMI 2011 March 24, 2011

in the resulting displaced subdivision surfaces. Differ-ent mesh frames may need different displacements anda single displacement map may not be enough to re-produce all the details. Our framework is based on theprevious work [12] for static meshes, but we extend it tohandle animated meshes while specifically minimizingthe approximation errors that can be caused by a singledisplacement map.

Our contributions can be summarized as follows.

• We propose a conversion framework for animatedmeshes to displaced subdivision surfaces. The out-put displaced subdivision surfaces share the sameconnectivity of control meshes and a single dis-placement map, which requires less storage and iseasy to render with modern graphics hardware.

• We analyze the sources of shape conversion errorscaused by using a single displacement map, anddesign each component of our framework to min-imize the approximation errors. Experimental re-sults demonstrate that our conversion frameworkreduces the approximation errors.

2. Related Work

Displaced subdivision surfaces. Lee et al. introduceddisplaced subdivision surfaces [12]. They proposed amethod to approximate a highly detailed surface usinga coarse mesh, called the control mesh, and a displace-ment map. A good approximation of the original sur-face can be reconstructed from the data in two steps:A smooth mesh is defined as the limit surface of thecontrol mesh by Loop subdivision [14], and vertices arethen translated in the direction of their normals by thedistance stored in the displacement map. Displaced sub-division surfaces are a powerful tool for modeling largeand highly detailed objects. Furthermore, methods haverecently been introduced to directly render displacedsubdivision surfaces on GPU [26, 15, 4].

Simplification of animated meshes. To simplify de-forming meshes, Mohr and Gliecher [16] proposed anextension of the QEM algorithm using the sum ofquadrics of all frames in the animation. Kircher andGarland [10] used a progressive approach to simplifydeforming meshes, and proposed a reclustering tech-nique to improve simplification results. DeCoro andRusinkiewicz [6] proposed a method for simplifyingarticulated meshes, and Landreneau and Schaefer [11]further improved the technique by optimizing vertex po-sitions. In this paper, we propose a simplification tech-

nique for an animation mesh which considers the ap-proximation error due to a single displacement map.

Global mesh parameterization. Lee et al. proposed aglobal mesh parameterization technique, which is calledMAPS [13]. The original mesh is first partitioned intoa set of triangular charts, and then the vertices of theoriginal mesh are parameterized onto the charts. Byregularly resampling the triangular charts, the origi-nal mesh can be remeshed and approximated. Severalmethods have been proposed to refine the parameteri-zation, improving inter-chart continuity and smoothness[9, 20, 19]. While previous methods use local and globaloptimization techniques to optimize the parameteriza-tion, we use simple filtering to post-process the param-eterization. We show our technique effectively smoothsthe parameterization while considering motion informa-tion of a given animated mesh.

3. Overview

From an input sequence of triangles meshes, we ex-tract a series of control meshes having the same ver-tex connectivity which share a single displacement map.Our goal is to minimize the shape conversion errorsof the resulting displaced subdivision surfaces fromthe original mesh sequence. To achieve the goal, wefirst analyze the source of approximation errors, andthen present our conversion framework with compo-nents specifically handling the errors.

3.1. Motion-based algorithm

For a static mesh, Lee et al. [12] minimized the shapeconversion error by first making the control mesh simi-lar to the original, and then using a displacement map toexpress remaining details. Our conversion framework isbased on Lee et al.’s algorithm. However, we cannot di-rectly apply this algorithm for displacement subdivisionsurfaces when an input is a sequence of meshes, not astatic mesh.

If we use only one displacement map shared amongall frames, shape conversion errors will occur betweenthe original meshes and the displaced subdivision sur-faces, because we cannot express all the details of allframes with only one displacement map (see Fig. 1). Wecan divide the errors into two types:

• Geometry-based errors: Differences between thesubdivision surfaces of the control meshes and theinput meshes. By minimizing these errors, we canreduce the displacement values needed after subdi-vision to recover the original shapes.

2

Figure 1: 1D illustration of motion-based errors. Red dots are verticesof original meshes and blue dots are vertices of subdivided meshes. Avertex v in the subdivided mesh is projected onto the original mesh.Displacements from three frames are different and they cannot be ex-pressed with a single displacement map.

• Motion-based errors: Differences of displacementvalues among all frames. To express displacementsof all frames as one value, which will be saved inthe single displacement map, we have to minimiz-ing these errors.

We should minimize both geometry-based and motion-based errors to accurately reproduce the original shapesin the input mesh sequence.

3.2. Overall process

We first define the notations. Let Mo, Mc, Ms, andMd be the original, control, subdivided, and resultingdisplaced mesh sequences, respectively. Let D be theshared single displacement map. A specific frame is re-ferred by t. For example, Mo(t) and Mc(t) denote theoriginal and control meshes at frame t, respectively.

For the displaced subdivision surface obtained froma static mesh, Lee et al.’s algorithm [12] can be brieflyexplained as follows. First, a simplification process isused to obtain an initial control mesh Mc. The secondstage optimizes the location of vertices in Mc in order tominimize the geometry-based error. Finally, a displace-ment map is computed to capture the remaining details.

Similar to Lee et al.’s algorithm, we use thegeometry-based errors during the simplification and op-timization processes. In addition, we propose motion-aware simplification and parameterization smoothing,which reflect motion information to reduce the requireddisplacements for shape recovery. The following stepssummarize the process of obtaining the displaced sub-division surfaces from a sequence of meshes.

1. We simplify the original animated mesh Mo intoa sequence of initial control meshes Mc usinggeometry- and motion-based error metrics, andbuild the initial mapping from Mo onto Mc (Sec.4).

2. We smooth the initial mapping of Mo onto Mc toreduce parameterization artifacts (Sec. 5.1).

3. We subdivide Mc using Loop subdivision to ob-tain the subdivided meshes Ms, and compute thedisplacement map D, which contains the averagedisplacements of all frames between correspond-ing positions in Mo and Ms (Sec. 5.2).

4. We further optimize the mapping of Mo onto Mc

to reduce the shape conversion errors between Mo

and the displaced subdivision surfaces Md that areobtained by adding D to Ms. We then recomputeD using the optimized mapping (Sec. 5.3).

5. We optimize the vertex positions and local framesof the control meshes Mc to reduce the shape con-version errors, and compute the final displacementmap D (Sec. 6).

In our conversion process, the displacement map D isinitially computed in Step 3 after parameter smoothingin Step 2. To reduce the shape conversion errors of Md

from Mo, D is then updated in Steps 4 and 5 by opti-mizing the mapping from Mo onto Mc, and the vertexpositions and local frames of Mc.

4. Simplification

To generate control meshes Mc, we simplify origi-nal meshes Mo by performing a sequence of full-edgecollapses (ecols). Using a priority queue of edges, wechoose and remove the candidate edge with the low-est error (see Fig. 2(b)). Then, the two vertices ofthe removed edge are mapped onto faces of the sim-plified mesh using barycentric coordinates (see Fig. 2(c), (d)). We use the geometry of the first frame meshin the mesh sequence to determine the barycentric coor-dinates. Simplification using ecol together with vertexmapping is repeated until the simplified mesh containsa given number of vertices.

The error metric for simplification should be chosento minimize both geometry- and motion-based errors.We propose two error metrics, one for each of the twotypes of errors. We then combine the two metrics todefine an error metric for ecol.

4.1. Metric for geometry-based error

For the metric for geometry-based errors, we use thesum of QEM for all frames [16], which is defined by

EG(u, v) =

N∑t=1

v(t)T Qtv(t), (1)

where Qt is the quadric error matrix for an edge (u, v)at frame t, and v(t) is the optimized vertex position afterecol of the edge. N is the number of frames.

3

Figure 2: Full edge collapse and mapping of original vertices ontothe simplified mesh. (a) The edge with the lowest cost is selected.Gray dots are vertices of original meshes previously mapped. (b) Theedge is collapsed and connectivity information updated. (c) The twovertices of the removed edge are mapped onto faces of simplified sur-faces. (d) Mapping information of all mapped vertices is adjusted.

4.2. Metric for motion-based error

After simplification, the displacement map D is com-puted as the average of the differences between the sub-divided meshes Ms and the original meshes Mo (Step5 in Sec. 3.2). However, if a vertex in Mo and its cor-responding position in Ms have different motions, thedisplacement values for the vertex will vary among dif-ferent frames. In this case, taking the average of thosevalues may cause large motion-based errors.

To minimize motion-based errors, vertices in Mo andtheir mapped positions in Ms should have similar mo-tions. It can be achieved if every ecol preserves the cur-rent motion around the collapsed edge. Two propertiesare desired to preserve the current motion. First, an ecolshould not introduce a new motion. When an edge (u, v)is collapsed, two vertices u and v are merged into a newvertex v. Then, the motions of v before and after ecolshould be similar. Second, an ecol should not be per-formed if the vertices around the edge have non-uniformmotions. If an ecol is performed in such a case, the posi-tions of new vertices v will be determined inconsistentlyamong frames and the motions of triangles neighboringv will change accordingly. As a result, the original ver-tices mapped onto those triangles (Fig. 2 (d)) will havechanged motions after ecol.

We define the motion-based error metric for simplifi-cation by

EM(u, v) =

N∑t=2

∑v∈R(v)

∥∥∥Att−1v(t − 1) − v(t)

∥∥∥2. (2)

R(v) is the set of 1-ring neighbor vertices of v, including

v itself, in the simplified mesh obtained by ecol. Att−1 is

an affine transformation matrix between frames t−1 andt, which is computed by

arg minAt

t−1

∑v∈{R(u)∪R(v)}

∥∥∥Att−1v(t − 1) − v(t)

∥∥∥2, (3)

where R(u) ∪ R(v) are 1-ring neighbor vertices of thecollapsed edge (u, v) before ecol is performed. By min-imizing Eq. (3), At

t−1 approximates the motions of thevertices around the edge to be collapsed.

In Eq. (2), we measure the amount of motion changesby computing the squared differences of the predictedand actual positions of vertex v and its 1-ring neighbor-hood. The predicted positions at frame t are determinedby applying At

t−1 to the positions at frame t − 1. If themotion of v is different from the one approximated byAt

t−1, EM will become bigger. Also, if the motions of thevertices around the collapsed edges are non-uniform,At

t−1 may not accurately approximate all the motions,and the predicted positions using At

t−1 will have largedifferences from the actual ones, increasing EM .

4.3. Simplification algorithm

We define the error metric E for each edge by

E(u, v) = EG(u, v) + λEM(u, v), (4)

where λ ∈ R+ is a weight of EM relative to EG. Toevaluate E(u, v), we need to determine the optimizedposition v for each frame. A simple method would beto use the quadric error matrix Qt, considering only thegeometry-based error at frame t. In this paper, we usea more elaborated method to determine v. For the firstframe, we determine v(1) using Q1. For the remain-ing frames t, we consider both geometry- and motion-based errors. We first compute At

t−1 and transformv(t − 1) using it. Then, v(t) is determined by minimiz-ing v(t)T Qtv(t) + λ

∥∥∥Att−1v(t − 1) − v(t)

∥∥∥2. In our exper-

iments, we could obtain slightly better results with thisapproach than the simple method.

In our simplification algorithm, we use affine trans-forms to minimize the motion-based errors. When ver-tices follow affine transformations among frames, thevariance of displacements computed after simplificationmay not be minimized, e.g., the displacements could belinearly increasing. In other words, rigid-body trans-forms could be more suitable to reduce the varianceof displacements. However, our conversion frameworkcontains a later stage of local frame optimization (Sec.6.2), which helps affine transforms to reduce the dis-placement variances.

4

λ = 0

λ = 1

Figure 3: Simplification result. (top) w/o motion weight; (bottom)w/ motion weight. Notice that with motion weight, more vertices arepreserved for joints of the model and triangle shapes are adjusted tominimize motion-based error.

In Eq. (3), an affine transform may not be defined ifall the vertices are contained in a planer region. In thiscase, we use a rigid-body transform instead. Detailsabout computing affine and rigid-body transforms canbe found in [17].

When we compute EM with Eq. (2), instead of the1-ring neighbor vertices of v, we can use the originalvertices that have been mapped on the 1-ring neigh-bor faces of v during simplification so far. This alter-native approach would be more accurate because wemeasure the motion difference between the original ani-mated meshes Mo and the simplified meshes. However,this approach takes much more computation, and in ourexperiments, the additional gain in reducing the vari-ances of displacements was not big.

Fig. 3 shows an example of our simplification result.With non-zero λ, we can consider both geometry- andmotion-based errors when we simplify the original ani-mated mesh.

5. Parameterization

During simplification, control meshes Mc are ob-tained from the original meshes Mo and the mappingfrom Mo onto Mc is constructed. Note that the con-trol meshes Mc in different frames share the same ver-tex connectivity, and the mapping from Mo onto Mc isalso shared among frames. Since the mapping directlyaffects the sampling of the displacement map (Sec. 5.2),good parameterization is important for the quality of re-sulting displaced subdivision surfaces Md. To achievebetter parameterization, we smooth and optimize theinitial mapping between Mo and Mc.

Figure 4: 1D illustration of parameterization smoothing. (top) beforeparameterization smoothing; (bottom) after parameterization smooth-ing. Blue lines represent the original mesh, and red lines are the con-trol mesh. v in the original mesh is parameterized (black arrow) ontothe control mesh. If we smooth parameterization without consideringmotion information, the mapping of v onto the control mesh can bechanged to another face, introducing more variations to the displace-ments.

We first perform parameterization smoothing to sup-press artifacts that may be contained in the initial map-ping. We then sample the displacement map D usingthe smoothed parameterization. Finally, we optimizethe parameterization using D so that the motion-basederror is reduced, and resample D using the optimizedparameterization.

5.1. Parameterization smoothingThe initial parameterization from Mo to Mc may have

artifacts. It may include flipping of mapped triangles,and some triangles in Mo may have been mapped ontotoo small or too large regions of Mc. We can sup-press those artifacts by smoothing the initial mapping.However, simple smoothing may incur another prob-lem. During simplification, with the motion-based er-ror metric (Eq. (2)), the vertices in Mo that have similarmotions have been mapped onto the same face in Mc.Simple smoothing may map these vertices with simi-lar motions onto different faces, increasing the motion-based error (Fig. 4).

To preserve the similarity of vertex motions, we usemotion-based parameterization smoothing. Let u be avertex in Mo, and p be the mapping position of u on aface in Mc. We update the mapping position p using aweighted average of mapping positions of 1-ring neigh-bor vertices of u:

p′ =1k

∑v∈R(v)

f (u, v)q, (5)

where v is a 1-ring neighbor vertex of u in Mo, q is themapping position of v on Mc, and k is the normaliza-

5

w/o param. smoothing w/ param. smoothing

Figure 5: Parameterization smoothing example. With parameteriza-tion smoothing, the vertex positions in subdivision surface Ms are uni-formly distributed while vertices near joints are kept dense in order tominimize the motion-based error.

tion factor. f (u, v) is the weighting function, which isdefined by

f (u, v) = exp

−∑N

t=2

∥∥∥Att−1(u) − At

t−1(v)∥∥∥2

2σ2r

, (6)

where Att−1(u) and At

t−1(v) are the affine transform ma-trices of u and v from frame t−1 to frame t, respectively(see Eq. (3)). σr is a given user parameter.

If the motions of u and v are similar to each other,the weight function f (u, v) has a high value and thesmoothed mapping positions p′ and q′ on Mc will beclose. Thus, we can smooth the mapping on the controlmesh Mc while retaining vertices with similar motionson the same face of Mc. Fig. 5 shows an example.

5.2. Displacement map samplingAfter parameterization smoothing, we apply k steps

of Loop subdivision to the control mesh Mc to ob-tain subdivided meshes Ms. The mapping between Mc

and Ms is naturally defined by subdivision operations.Then, since we have both mappings from Ms to Mc andfrom Mo to Mc, we can establish mapping between Ms

and Mo using Mc as the common domain, in a similarway to vertex resampling in [13].

Let vs be a vertex in Ms and qs be the projection ofvs onto Mo. The projection qs is specified by a face fo

in Mo and barycentric coordinates (w1,w2,w3). That is,qs(t) =

∑3k=1 wkvok(t) for frame t, where vok are three

vertices of fo. Then, the displacement for vs at frame tis computed by ds(t) = (qs(t) − vs(t)).

To represent a 3D displacement vector ds, we use alocal frame, instead of the 3D world coordinates. Thedisplacement vectors ds for vs at different frames maydiffer due to shape change, but the difference also in-cludes the local motion among the frames. To minimize

the variances of displacement vectors among frames,the local frame should change following the local mo-tion of the input mesh sequence. Let t, b, and n be thethree axes of a local frame, which are tangent, bitangent,and normal vectors, respectively.

We assign a local frame to each face fc of the con-trol meshes Mc as follows. We set face normal as n, thefirst edge of face fc as t and the cross product of n and tas b. We use this local frame to represent the displace-ment vectors of the vertices in Ms generated from facefc when Mc is subdivided to Ms.

Let L = [t b n] be a local frame, which is representedby a 3× 3 matrix. The transform of a displacement vec-tor ds into L can be computed by Lds. For each vertexvs in Ms, we transform the displacement vectors at allframes using the corresponding local frames, and takethe average of the transformed displacement vectors.We then store the single displacement vectors computedat all vertices of Ms into the initial displacement map D.

5.3. Parameterization optimization

The single displacement map D introduces shapeconversion errors between the displaced subdivisionsurfaces Md and the given mesh sequence Mo. We canreduce the errors by updating the projections qs for ver-tices vs in Ms and recomputing the displacement mapD. Note that the mapping of Mo onto Mc, which affectsthe projections qs, has been determined using the firstframe mesh in Mo, and may not be optimal to reducethe approximation errors for all frames.

The updated projection qs for vs can be computed by

arg minwk

N∑t=1

‖qs(t) − (vs(t) + L(t)ds)‖2+α

3∑k=1

(wk−wk)2,

(7)where wk is the current barycentric coordinate, L(t) isthe local frame for vs at t, ds is the single displace-ment for vs, and α is a regularization factor. With theupdated barycentric coordinates wk, we have qs(t) =∑3

k=1 wkvok(t) at frame t. For vertices vs in Ms, theupdated projections qs determine the positions on Mo

which are nearby the current projections qs and mini-mize the deviations from the displaced subdivided ver-tices. By recomputing the displacement map D usingthese positions, we can reduce the shape conversion er-rors of Md.

However, since the updated projections for vertices vs

in Ms are independently computed in Eq. (7), they mayintroduce artifacts, such as triangle flipping. Instead ofdirectly using the updated projections to recompute D,

6

we modify the mapping from Mo onto Mc with the up-dated projections and use the modified mapping to re-compute D. For a vertex vs in Ms, let ps be its mappedposition on the control mesh Mc, which is determinedby subdivision operation. Then, for the projection qs ofvs onto Mo, its mapped position on Mc, determined bythe mapping from Mo onto Mc, is the same as ps be-cause the projection from Ms to Mo has been obtainedthrough Mc. With the updated projections qs, we mod-ify the mapping from Mo onto Mc so that qs, instead ofqs, should be mapped onto ps.

A mapping from Mo onto Mc is represented bymapped positions of the original vertices in Mo onto Mc

(Fig. 2 (d)). The modified mapping can be obtained by

arg minpoi

Ns∑k=1

∥∥∥∥∥∥∥No∑i=1

wkipoi − psk

∥∥∥∥∥∥∥2

, (8)

where poi be the mapped position on Mc of a vertex voi

in Mo, and psk be the mapped position of a vertex vsk

in Ms. No and Ns are the numbers of vertices in Mo

and Ms, respectively. For vertex vsk, wki is the updatedbarycentric coordinate obtained by Eq. (7) if voi belongsto the triangle in Mo which contains the original projec-tion qsk of vsk. Otherwise, it is zero.

In Eq. (8), the positions poi mapped onto differentfaces of Mc do not lie on the same plane. To avoidglobal parameterization of Mc onto a planar domain, weperform local optimizations of Eq. (8) as follows:

1. For each face fc in Mc, the local region includingthe adjacent faces of the face fc is parameterizedonto 2D, as well as vertices in Ms and Mo that aremapped onto the local region.

2. Fix the mapped positions of vertices in Mo that areat the boundary of the local region.

3. Optimize the mapping of vertices from Mo to Mc

in the local region using the localized version ofEq. (8).

6. Optimization of Control Meshes

After we compute the displacement map D using pa-rameterization smoothing and optimization, the shapeconversion errors of the displaced subdivision surfacesMd from the original mesh sequence Mo can be furtherreduced by optimizing the control meshes Mc. We firstiteratively update the vertex positions of Mc and the dis-placement map D to reduce the errors. We then optimizethe local frames for faces in Mc, and compute the finaldisplacement map D.

6.1. Optimization of control mesh verticesWe can represent the vertex positions in Ms in terms

of those in Mc using the mapping between Mc and Ms.Let PC and PS be matrices where each row containsx, y, z-coordinates of vertex positions in Mc and Ms, re-spectively. For each frame t, PS (t) can be representedby a linear transformation of PC(t),

PS (t) = SkPC(t) (9)

where Sk is the product of the first k subdivision matri-ces.

Let PD be a matrix containing the final displaced po-sitions of vertices in Md as their rows. Then, PD(t) isrepresented by

PD(t) = PS (t) + L(t)D = SkPC(t) + L(t)D, (10)

where L(t) denotes the local frames and D is the dis-placement map. L(t)D represents the displacementstransformed by local frames.

To optimize the vertex positions of Mc and the dis-placement map D, we adapt the method proposed in[12] to handle a mesh sequence. Let PO be a matrix thatcontains the projections qs on Mo of vertices vs in Ms

computed by parameterization optimization (Sec. 5.3).We first optimize the positions PC(t) at each frame t byminimizing the squared errors with the current displace-ment map D:

arg minPC (t)

‖PO(t) − (SkPC(t) + L(t)D)‖2 . (11)

We then update the subdivided positions PS (t) using theoptimized PC(t) at each frame t, and recompute the dis-placement map D using the differences between PO andthe updated PS . We repeat these steps a few times toiteratively update the vertex positions of Mc and D. Fig.6 shows an optimization result.

6.2. Optimization of local framesSince the local frames for the displacement vectors

have effects on the final displaced subdivision meshesMd, we optimize them to further reduce the shape con-version errors. For each face fc in Mc at frame t, weoptimize the local frame L(t) by

arg minL(t)

∑i

‖qsi(t) − {vsi(t) + L(t)dsi}‖2 , (12)

where vsi is a vertex in Ms subdivided from face fc, qsi isthe projection of vsi onto Mo, and dsi is the displacementvector for vsi.

After we optimize the local frames, we compute thefinal displacement map using the updated local frames.That is, for each vertex vs in Ms, we apply the updatedlocal frames to the displacements ds(t) at all frames t,and take the average.

7

Figure 6: Optimization of a control mesh Mc. (top) before optimiza-tion; (bottom) after optimization. The head of the horse (second col-umn) does not have any motion, so it is unchanged during the opti-mization process. However, the joints (third column) have a lot ofmotion and vertex positions in this region are optimized to fit the mo-tion.

Figure 7: Average conversion error chart of Table 1.

7. Results and Discussion

We tested our conversion framework with several an-imated meshes. Table 1 shows RMSE results of ourframework. In the table, we sequentially added eachcomponent of our framework and measured the RMSEchange between the original mesh sequence and the fi-nal displaced subdivision meshes. Fig. 7 visualizes av-erage RMSE values of Table 1. With the componentsin our framework, error values dropped almost by half.Fig. 11 shows the visualizations of RMSEs of our re-sults. Our framework introduces small amount of errorseven though it uses a single displacement map.

Review of each componentIn our experiments, optimization of vertex positions

and local frames of the control meshes showed the high-est quality improvements. These steps directly optimizethe shape conversion errors by Eqs. (11) and (12), whileothers try to reduce the errors indirectly.

RMSE (×10−4) gainw/o motion w/ motion (%)

horse-gallop 4.51 3.50 22.35dance 5.88 4.21 28.42

punching 6.60 4.83 26.77snake 1.15 1.18 -2.03

rabbit-run 8.05 7.51 6.76

Table 2: Conversion errors w/ and w/o motion-aware simplification.The other parameters for conversion are set the same; we simplifiedeach mesh to 600 vertices, then subdivided twice.

In Fig. 7, the conversion error did not drop much withmotion aware simplification. If control mesh optimiza-tion is not performed, motion aware simplification maynot reduce the error because it can harm the geometricquality of control meshes by considering motions in themesh sequence. However, motion aware simplificationconsiders motion similarity of neighboring vertices, andas a result, and the original vertices mapped onto thesame face of the control mesh would have similar mo-tions. Then, the optimization with control meshes willbe very effective for reducing the conversion errors. Ta-ble 2 compares the conversion errors with and withoutmotion aware simplification.

Parameterization smoothing is designed to handle ar-tifacts in the mapping between the original meshes andthe simplified meshes, and it improves the shapes of re-sulting displaced subdivision meshes with regular spac-ing of vertices. For parameterization optimization, weused an approximation method to optimize the objec-tive, and the effect was not big.

In our framework, parameterization optimization andthe following displacement update is performed onlyonce (Sec. 5.3). To further optimize the parameteriza-tion and the displacement map, those step can be iter-ated. However, in our experiments, such iteration didnot improve the quality of the final displaced subdi-vision meshes. Similarly, local frame optimization ofcontrol meshes can be iterated, but we perform the op-timization only once (Sec. 6.2).

After optimizing local frames, we should save the lo-cal frame data (t,b,n) for each face of control meshes ateach frame. The data size of local frames could be rela-tively large if the control mesh size is not much smallerthan the original mesh, e.g., when the original mesh isalready simple. However, usually, the control mesh ismuch simpler than the original, and the data size of localframes does not have much effect on the data reductionratio of our conversion framework.

8

Model info. RMSE (×10−4) gain#. verts. #. faces frames none w/ motion w/ param. w/ opt. Mc w/ opt. L (%)

horse-gallop 8,430 16,856 48 6.80 7.71 7.59 4.59 3.50 48.5dance 7,061 14,118 50 7.30 7.57 6.55 4.95 4.21 42.3

punching 19,694 39,384 48 8.43 7.89 7.89 6.07 4.83 42.6snake 9,179 18,354 42 2.67 2.63 2.53 1.47 1.18 56.0

rabbit-run 12,410 24,816 36 16.52 18.42 18.09 10.17 7.51 54.6

Table 1: RMSE table of our algorithm. We added each component one by one (from left to right) and measured RMSEs. (w/ motion) w/ motionaware simplification; (w/ param.) w/ parameterization filtering; (w/ opt. Mc) w/ control mesh optimization; (w/ opt L) w/ local frame optimization.We simplified each mesh to 600 vertices then subdivided twice.

Model characteristics

In Table 1, rabbit-run showed the biggest conversionerror. The model was carefully designed by a humanartist, and is already well shaped with many sharp fea-tures. It is difficult to obtain a good conversion resultfor such a model.

The dance and punching models have similar RM-SEs. Although they have different numbers of verticesand different motions, both are human body models,where motions are concentrated on the joints. In ad-dition, both models were acquired by motion capturemethods. Consequently, the two models have similarcharacteristics, and our framework converted both mod-els with similar performances.

In Table 2, motion aware simplification showed quiteamount of error reduction when applied to dance andpunching models. Our simplification method has simi-lar properties to the previous simplification methods de-signed for articulated meshes, where human body is atypical example of articulated meshes.

For snake, motion aware simplification harms theconversion quality. This model has a very smooth mo-tion, and the conversion error itself is already small. Insuch a case, motion aware simplification may not im-prove the conversion quality.

Storage

We compared our framework with the conventionaltechnique that uses a displacement map for each frame.In our experiments, we used 100∼200 fewer vertices(200∼400 in the case of not using local frame optimiza-tion) for the control meshes of the conventional tech-nique to achieve similar RMSE values to our method.However, still, the conventional technique took muchmore storage to keep the displacement maps for allframes rather than a single displacement map of ourframework.

Fig. 8 shows storage comparison. The amount of dataand RMSE are measured for several numbers of control

mesh vertices. With similar RMSE values, our resultstake less than half the storage.

We also applied a dynamic mesh compression tech-nique to our conversion result and the original animatedmesh, and compared the rate-distortion curves. Wecompressed the original animated mesh and our controlmesh sequence using the Coddyac algorithm with com-pressed PCA bases [21, 22]. To compress our singledisplacement map, we used OpenEXR library [8].

Fig. 9 shows the rate-distortion curves for horsemodel with compressed data sizes and RMSEs of theoriginal animated mesh and our conversion result. TheRMSE value of our result quickly converges to our orig-inal conversion error and remains constant although thedata size increases. However, before the RMSE valueconverges, compressed data of our conversion resulttake less storage than that of the original mesh at thesame RMSEs.

In Fig. 9, local frame optimization for control mesheswas not performed, and we did not store local frames.If we used local frame optimization, we could furtherreduce RMSEs while taking more storage. Still RM-SEs of our results are quite small (less than 5 × 10−4 inaverage) even without optimization of local frames.

Application to GPU-based renderingThere have been several techniques utilizing subdi-

vision surfaces for efficient rendering [24, 2, 15]. Inthe techniques, details of models are stored in the GPUmemory once, and only a small amount data (changingvertex positions and local frames of control meshes) istransmitted from CPU to GPU in the rendering time.Consequently, the bandwidth bottleneck is greatly re-duced and the overall rendering performance is in-creased.

Our conversion framework can easily be adapted forthose rendering techniques. In this paper, we experi-ment with the technique proposed by Loop et al. [15].We first simplify Mo to extract Mc. Then we convertMc into quad meshes [23]. The vertex mapping from

9

Figure 8: Rate-distortion curve of several models. (−) with a singledisplacement map; (�) with a displacement map for each frame. Theamount of data is greatly reduced by using a single displacement map.

Figure 9: Rate-distortion curve of the compression result of horse.(blue) size of the compressed data of the original animated mesh. (red)size of the compressed data of our conversion results (control meshesand their displacement map).

Mo onto Mc is updated accordingly during the conver-sion process. Then we subdivide the converted con-trol meshes using Catmull-Clark subdivision [5]. Othercomponents of our framework, such as parameterizationsmoothing and control mesh optimization, can easily beapplied to quad meshes. As the result of the conversion,we obtain a sequence of simplified quad meshes Md thatshare one displacement map among frames.

Fig. 10 shows a rendering example. We used the tech-nique presented by Castano [4] for rendering. Recenttechniques [3, 25] can be integrated into the renderingpipeline to handle displacement maps, although we didnot use them in our experiments.

Figure 10: GPU rendering example. (top) subdivided once; (bottom)subdivided three times.

8. Conclusions

We have presented an effective framework to extractdisplaced subdivision surfaces from animated meshes.Our framework is designed to convert an animated meshwith a single displacement map. With our framework,the conversion results achieve small RMSE values withsmall data sizes. Our framework can be used for appli-cations, such as real-time rendering of animated mesheson GPU and animated mesh compression.

There are rooms for further improvements in ourframework. A better method for motion-aware sim-plification will help reduce the motion differences ofthe control meshes from the original animated mesh.A more robust method for parameterization of origi-nal vertices onto the control mesh will improve the ver-tex sampling of the final displaced subdivision meshes.Similarly, a more robust method for projecting the sub-divided vertices onto the original meshes will be usefulto handle complex mesh shapes. In this paper, we used3D vectors for the displacement map, but a displace-ment map with scalar values (displacements in the nor-mal directions of local shapes) would reduce the storagerequirement although achieving small shape conversionerrors will become more challenging.

Acknowledgements

We would like to thank Libor Vasa for his help mak-ing compression results. We wish to thank Robert W.Sumner and Jovan Popovic for providing horse-gallop.We also thank Sang Il Park and Jessica K. Hodgins forproviding punching. Many thanks to Igor Guskov forproviding dance and snake. rabbit-run is courtesy ofBig Buck Bunny (http://www.bigbuckbunny.org). This

10

Figure 11: Visualizations of relative errors. Blue color indicates bigger error values. (top) w/o our algorithm; (bottom) w/ our algorithm.

work was supported by the Brain Korea 21 Project in2010, the Industrial Strategic Technology DevelopmentProgram of MKE/MCST/KEIT (KI001820, Develop-ment of Computational Photography Technologies forImage and Video Contents) and the Basic Science Re-search Program of MEST/NRF (2010-0019523).

References

[1] Allen, B., Curless, B., Popovic, Z., 2002. Articulated body de-formation from range scan data. ACM Trans. Graph. 21, 612–619.

[2] Boubekeur, T., Schlick, C., 2005. Generic mesh re-finement on gpu. In: Proceedings of the ACM SIG-GRAPH/EUROGRAPHICS conference on Graphics hardware.pp. 99–104.

[3] Burley, B., Lacewell, D., 2008. Ptex: Per-face texture mappingfor production rendering. In: Eurographics Symposium on Ren-dering 2008. pp. 1155–1164.

[4] Castano, I., 2008. Next-generation rendering of subdivision sur-faces. SIGGRAPH exhibitor tech sessions.

[5] Catmull, E., Clark, J., 1978. Recursively generated b-spline sur-faces on arbitrary topological meshes. Computer-Aided Design10 (6), 350–355.

[6] DeCoro, C., Rusinkiewicz, S., 2005. Pose-independent simplifi-cation of articulated meshes. In: Proceedings of the 2005 sym-posium on Interactive 3D graphics and games. pp. 17–24.

[7] DeRose, T., Kass, M., Truong, T., 1998. Subdivision surfaces incharacter animation. In: SIGGRAPH ’98. pp. 85–94.

[8] Industrial Light & Magic, 2010. OpenEXR.URL http://www.openexr.com

[9] Khodakovsky, A., Litke, N., Schroder, P., 2003. Globallysmooth parameterizations with low distortion. ACM Trans.Graph. 22 (3), 350–357.

[10] Kircher, S., Garland, M., 2005. Progressive multiresolutionmeshes for deforming surfaces. In: SCA ’05: Proceedings of

the 2005 ACM SIGGRAPH/Eurographics symposium on Com-puter animation. pp. 191–200.

[11] Landreneau, E., Schaefer, S., 2009. Simplification of articulatedmeshes. Computer Graphics Forum 28, 347–353(7).

[12] Lee, A., Moreton, H., Hoppe, H., 2000. Displaced subdivi-sion surfaces. In: Proceedings of the 27th annual conference onComputer graphics and interactive techniques. SIGGRAPH ’00.pp. 85–94.

[13] Lee, A. W. F., Sweldens, W., Schroder, P., Cowsar, L., Dobkin,D., 1998. MAPS: multiresolution adaptive parameterization ofsurfaces. In: SIGGRAPH ’98: Proceedings of the 25th annualconference on Computer graphics and interactive techniques.pp. 95–104.

[14] Loop, C., 1987. Smooth subdivision surfaces based on triangles.Ph.D. thesis, University of Utah.

[15] Loop, C., Schaefer, S., March 2008. Approximating catmull-clark subdivision surfaces with bicubic patches. ACM Trans.Graph. 27, 8:1–8:11.

[16] Mohr, A., Gleicher, M., 2003. Deformation sensitive decima-tion. Tech. rep., University of Wisconsin.

[17] Muller, M., Heidelberger, B., Teschner, M., Gross, M., July2005. Meshless deformations based on shape matching. ACMTrans. Graph. 24, 471–478.

[18] Park, S. I., Hodgins, J. K., 2006. Capturing and animating skindeformation in human motion. ACM Trans. Graph. 25, 881–889.

[19] Pietroni, N., Tarini, M., Cignoni, P., 2010. Almost isometricmesh parameterization through abstract domains. IEEE Trans-actions on Visualization and Computer Graphics 16, 621–635.

[20] Ray, N., Li, W. C., Levy, B., Sheffer, A., Alliez, P., 2006. Peri-odic global parameterization. ACM Trans. Graph. 25 (4), 1460–1485.

[21] Vasa, L., Skala, V., 2007. Coddyac: Connectivity driven dy-namic mesh compression. In: 3DTV Conference, 2007. pp. 1–4.

[22] Vasa, L., Skala, V., 2009. Cobra: Compression of the basisfor pca represented animations. Computer Graphics Forum 28,1529–1540.

[23] Velho, L., 2000. Quadrilateral meshing using 4-8 clustering. In:CILANCE’00. pp. 61–64.

11

[24] Vlachos, A., Peters, J., Boyd, C., Mitchell, J. L., 2001. Curvedpn triangles. In: Proceedings of the 2001 symposium on Inter-active 3D graphics. pp. 159–166.

[25] Yuksel, C., Keyser, J., House, D. H., 2010. Mesh colors. ACMTrans. Graph. 29 (2), 15:1–15:11.

[26] Zhou, K., Huang, X., Xu, W., Guo, B., Shum, H.-Y., July 2007.Direct manipulation of subdivision surfaces on GPUs. ACMTrans. Graph. 26, 91.

12

horse-gallop

dance

rabbit-run

punching

Figure 12: Results with various models. All models are simplified to 600 vertices and subdivided twice.

13