Geometry VideosSymposium on Computer Animation 2003

Hector M. BriceñoCollaborators: Pedro V. Sander, Leonard McMillan, Steven

Gortler, and Hugues Hoppe

22

2 2

Motivation Many sources of 3D

Animation data: Motion Capture Visual Hulls Physical Simulations Sensor Data Skilled Animators

Wide variety of formats, data, and reconstruction schemes…

33

3 3

Problem: Sharing 3D Animations

Render a Video of the animation Use the similar software and/or

hardware Use static mesh compression for

each frame DEMO DEMO

44

4 4

Approach: By representing manifold 3D objects using

a global 2D parametrization (mapping) it is possible to use existing video techniques to represent 3D animations.

55

5 5

Assumptions of Geometry Videos

One or more manifold surfaces Consistent connectivity through the

duration of the animation No changes in topology Can undergo arbitrary deformations

as well as rigid-body transformations

66

6 6

Outline Related Work Geometry Images and Geometry

Videos Cuts Parametrization Compression

Exploiting Temporal Coherence Results Future Work and Conclusions

77

7 7

Related Work: Mesh Compression

Maintaining connectivity: Topological Surgery

[Taubin98] Progressive Meshes

[Hoppe96] Spectral Compression

[Karni00] Re-parametrizing:

Semi-regular: Progressive Compression [Khodakovsky00]

Fully regular: Geometry Images: fully regular [Gu02]

88

8 8

Related Work: Animated Meshes MPEG4, VRML Animated Meshes “Multi-Resolution Dynamic Meshes with Arbitrary

Deformations” [Shamir00] “Representing Animations by PCA” [Alexa00] “Compression of Time-dependent geometry”

[Lengyel99] “Dynapack” [Ibarria03]

99

9 9

Related Work: Video MPEG

Spatial, Temporal, SNR Scalability, Motion Compensation, High Compression, VBR…

Other… Layered Coding L-DCT [Amir96] Multi-resolution Video [Finkelstein96]

LOD both time and space. NAIVE [Briceno99]

Graceful degradation, error resilience

1010

10 10

Geometry Images Represents a manifold surface in 3D

space as an 2D array of 3D points. Works in 3 steps:

Cutting: maps 3D surfaces to manifold Parametrization

Maps 3D space -> 2D parameter space Rasterization and Compression

1111

11 11

Parametrization Maps 3D manifold surface onto 2D

square Different criteria or metrics: Conformal,

Area-preserving, Geometric-Stretch

1212

12 12

Rasterization/Compression Sample points of parametrization

obtain a 2D grid of triplets (x,y,z) Compress resulting “image”

DEMO

1313

13 13

Cutting: Geometry Image Iteratively Cut and Reparametrize

Final

1414

14 14

Animated Meshes: Approach

How do we cut, parametrize and compress considering a time-sequence of meshes?

1515

15 15

Cutting: Animations Animation frames should have the

same cut and parametrization

No Correspondence

c

Different Cuts and Parametrization

1616

16 16

Cuts, how to pick? Looking at single frame might miss

something?

Approach: find a global cut considering all frames.

1717

17 17

Global Cut Cut from frame

2 misses spike on frame 1 and spikes on frame 3

Cut

2

Glo

bal C

ut

Frame 1 Frame 2 Frame 3

1818

18 18

Global Cut: how it works Run the iterative algorithm on all frames at

the same time. Pick worst avg. face on all

parametrizations…

Fram

e 1

Fram

e 2

1919

19 19

Parametrization: Animation Cut and parametrization has

to be fixed for all frames in order to use one texture for whole animation

We currently apply the global cut to the first frame and compute parametrization on that frame.

2020

20 20

Compression Spatial Compression:

Wavelets: Can support multiple levels of detail…

Temporal Compression Predictive Coding similar to MPEG Use affine transformations for predictor

2121

21 21

Encoder Architecture

Basic Delta Encoder Uses affine transformations

ReferenceFrame

InputFrame

Cut &Parametrize

Rasterize/Encode Diff

Transform

Decode

2222

22 22

Transformations: Global Global Trans. form a good

approximation

Frame 2Frame 1 Transformed Frame 1

2323

23 23

Transformations: Global con’t Global cannot capture well

deformations within the object

Frame 1

Frame 2

Predictor of Frame 2 from Frame 1

2424

24 24

Transformations: Local Apply transformation on

chartsFrame 1

Frame 2

Predictor

2525

25 25

Transformations: Local w/ Spread & Blend

Spread. Include neighbors in the computation of the transformation

Blend between patches.Target

PredictorNo blendNo spread

Predictorw/blendw/spread

d c

2626

26 26

Results Comparing Geometry Images Comparison to PCA Predictive Coding: Transformations

Global Local

Timing/Performance Level of Detail

2727

27 27

Comparing Geometry Images: Snake

2828

28 28

Comparison to PCA

2929

29 29

Transformations: Global vs. Local

3030

30 30

Transformation Performance

DEMO

2bpv P

8bpv I Baseline

4bpv P

8bpv P8bpv B

s d

3131

31 31

Performance Timings Finding Cut (one frame): 2-7 mins Finding Cut (100 frames): 3-5 hrs Parametrization: 2-6 mins Encoding: 2-3 fps @ 256x256 Encoding: 6-16 fps @ 64x64 Decoding: 10 fps @ 256x256 Decoding: 30-60 fps @ 64x64

3232

32 32

Level of Detail

3333

33 33

Future Work Video Compression Transformations Chartification Parametrization Non-manifold objects

3434

34 34

Conclusions Geometry Video as way to encode

and represent 3D animations Can use many of the 2D Video

Techniques/Features Spatial/Temporal scalability Error resiliency

Many other features to be exploited, i.e. fast clipping and hardware implementation…

3535

35 35

Acknowledgements Collaborators: Pedro Sander,

Leonard McMillan, Steven Gortler, Hughes Hoppe, and Gu Xianfen.

Animations: Matthias Mueller and Daniel VlasicQuestions

?