introduction to data-driven animation

Introduction to Data-driven Animation

Jinxiang Chai Computer Science and Engineering

Texas A&M University

Data-driven animation

- Motion graphs

- Motion interpolations

- Statistical motion synthesis

Outline

Motion capture data are easy to capture

But we cannot capture all kinds of motion variations

- different subjects - different styles - different emotions

Key idea: reuse prerecorded motion data to achieve new goals!

Motivations for Data-driven Approaches

Data-driven Animation

Goal: convert motion data into a usable form.

Can we automate this?– Must preserve realism and provide control

Motion model User specifications

MotionMotion

processing

Motion data


- Motion graphs



Outline

Motion Graphs: Key Ideas

Given: lots of prerecorded motion clips

Concatenate them to create new motions!

Motion ConcatenationMotion capture region Virtual environment

Obstacles

Sketched path

Motion ConcatenationMotion capture region Virtual environment

Unstructured Input Data

A number of motion clips • Each clip contains many frames• Each frame represents a pose

Unstructured Input Data

Connecting transition • Between similar frames

Graph Construction

Building Motion Graphs

So how can we find transition points between motion clips?

Building Motion Graphs

- Every pair of frames has a distance.

- Transitions are local minima below a threshold.

Motion 2 Frames

Mot

ion

1 Fr

ames

Finding Similar Frames

• Need derivatives (velocity, acceleration, etc.)

• Compare motion in joint angle space or 3d point space?

• Must account for coordinate invariance

– Different camera ≠ different motion!

Distance Metric

For more detail, refer to [Kovar and Gleicher, Lee et al]

Finding Transition Points

Transition thresholds control quality vs. flexibility tradeoff.

Threshold = 0 cm Threshold = 8 cm Threshold = 16 cm

Structures of Motion Graphs

Motion data structure: a graph of frames/poses

Avoid dead-ends: finding strongly connected components

Contact states: avoid transition to dissimilar contact state

Interacting with Motion Graphs

So given a motion graph, how can we generate an animation sequence?

Interacting with Motion Graphs

So given a motion graph, how can we generate an animation sequence?

- Random graph walk: Any sequence of edges is a motion!

Using Motion Graphs

How can we control synthesized motions (e.g., moving from point A to point B, speeds, walking directions)?

- Graph search: Find graph walks that minimize a cost function.

Path Synthesis

Goal: extract motion that follows a path.

User’s path ( )

Motion’s path ( )

Minimize

2)()( i

ii sPsP

P

P

Motion Control

Goal: extract motion (M) that satisfied constraints (C) specified by the user

Minimize

),( CMG

Results

See videos [click here]!

http://faculty.cs.tamu.edu/jchai/projects/Avatar/

Discussion

Pros: + Fully automatic: work on unstructured data + High-quality animation: motion concatenations + Easy to control: graph search

Cons - Poor generalization: cannot produce new poses - Control accuracy: cannot generalize new poses - Not compact: needs to retain original mocap data - Scalability


- Motion graphs



Outline

Motion Interpolations: Key Ideas

Given: lots of prerecorded motion clips

Interpolating motions to achieve new goals!

Motion Interpolations: Key Ideas

In research: more than decades [e.g., Rose et al. 98]In games for a long time

- Interpolating motions needs build correspondences between motion examples- Thus, motion interpolations require structurally similar motion examples!

Canonical timelinet

Time warping functions

w(t)

t

Motion Decomposition

Canonical timelinet


w(t)

t

Reference motion


Canonical timelinet


w(t)

t

Contact Transitions


Canonical timelinet


w(t)

Motion 1

t


Canonical timelinet


w(t)

t


Canonical timelinet


w(t)

t


Using dynamic time warping!

Canonical timelinet


w(t)

t


Canonical timelinet


w(t)

Motion 2

t


Canonical timelinet


w(t)

t


Motion Representation

Registered motions Time warping functions

Contact Transitions

Motion Annotation

Preprocessed motions mi

Motion Annotation


For each motion mi, we annotate the motion with control parameters si

- such as walking speed, direction, step size, kicking directions and positions, etc.

Motion Annotation


Motion space: m

Control parameter space: s

Motion Interpolations and Control


Motion space: m


How can we generate an animation that achieves the goals specified c* by the user?

Motion Interpolations and Control


Motion space: m


How can we generate an animation that achieves the goals specified c* by the user? Scattered data interpolation

Scatter Data Interpolations


Motion space: m


How can we generate an animation that achieves the goals specified c* by the user? Scattered data interpolation

Scattered Data Interpolations

Motion space: m


Many techniques for scattered data interp.:

- Local interpolation/regression [Kovar and Gleicher 04]- Radial basis functions [Rose et al 98]- Gaussian processes [Mukai and Kuriyama

05]

Results

Geostatistical motion interpolation [Mukai and Kuriyama 05]

- check youtube video [click here]

http://www.youtube.com/watch?v=qtNsF-nTp4A

Discussions

Pros: + high-quality animation + good generalization: motion interpolation and

extrapolation + particularly suitable for high-level motion control

Cons - all examples must be structurally similar. - not compact: needs to retain original mocap data - not suitable for detailed kinematic motion control

such as such key frames - lack of planning schemes for task level control such

as moving from one point to another.


- Motion graphs



Outline

Bayesian Motion Synthesis

Goal: Find the most likely motion x from control inputs c specified by the user

)|Pr(maxarg cxx



)|Pr(maxarg cxx

)Pr()Pr()|Pr(maxarg cxxc

x



)|Pr(maxarg cxx

)Pr()Pr()|Pr(maxarg cxxc

x

)Pr()|Pr(maxarg xxcx

Likelihood: How well motion matches control input? Motion priors: How

natural motion is?

Human motiondatabase

Human motionanalysis

Motionoptimization

Human motion prior

User-defined constraints

Generate natural human motion from a small set of user-defined constraints

Statistical Motion Synthesis

Statistical Dynamic Model

Character pose

Low-dimensional pose space

yt = C xt + D



Statistical dynamic model

xt = A1xt-1 +…+ Amxt-m+ B ut

Control input

Temporal prediction

Complexity of Statistical Dynamic Model

Character pose


yt = C xt + D



Statistical dynamic model


Control input

Temporal prediction

dim(ut)

dim(xt)

Statistical Dynamic Model Learning



Character pose


yt = C xt + D


Control input

Temporal prediction

Dynamic model matrices:

A1,…,Am, B, C, D

Statistical Dynamic Model Learning



Dynamic model matrices:

A1,…,Am, B, C, D

Dynamic system order:

m, dimensionality of xt and ut

Character pose


yt = C xt + D


Control input

Temporal prediction

Reconstruction error

Full-body motion data

Xt = A1Xt-1+…+AmXt-m+B ut

Statistical linear dynamic model:

m = 3, dim(ut) = 4, error = 0.7 deg

Yt = C Xt + D

Generate natural motion from a small set of user-defined constraints

Overview: Offline Animation Control



Motionoptimization

Human motion prior


Constrained Motion Optimization



Motionoptimization

Human motion prior


Generate natural motion from a small set of user-defined constraints


Motionoptimization

Human motion prior


Smoothness term

yyE Tsmoothness

),,....,()(ln 1 tmtttt uxxxpup

2212 ttt CxCxCx

User-defined constraints:

Objective function:

Motion prior Motion smoothness

Sequential Quadratic Programming

t ttt

K

j

m

tiititjjtjux CxCxCxBuxAxuN 2

211

2

1, 2)),;(log(minarg


Mjcxf jj ,1)(

Results

Click here

http://faculty.cs.tamu.edu/jchai/projects/motion-optimization/

Discussions

Pros: + good generalization and accurate motion control due

to the use of statistical models + compact: only needs to keep model parameters + suitable for kinematic motion control

Cons - often need to specify contact constraints across the

entire animation. - does not support task-level control (e.g., move from

one point to another) - better models are needed to match the synthesis

quality of motion graphs

Summary

The use of prerecorded motion data allows us to create high-quality controllable animation for human characters

Ideal data-driven animation techniques - High-quality: realistic animation without noticeable

visual artifacts - control accuracy: often require strong generalizability

to achieve new tasks. - control flexibility: support motion control in both

kinematics and task/behavior level. - scalability: scale up well to huge and heterogeneous

datasets. - compact: demand small or moderate memory sizes.

introduction to data-driven animation

Documents