motion capture in movies motion capture · 8 cameras + 120 hz + special tape = raw point data 10...

Motion CaptureMotion Capture

22

Motion Capture in MoviesMotion Capture in Movies

33

Motion Capture in GamesMotion Capture in Games

44

55

Magnetic Capture SystemsMagnetic Capture Systems

❚❚ TetheredTethered

❚❚ Sensitive to metalSensitive to metal

❚❚ Low frequency (60Hz)Low frequency (60Hz)

66

Mechanical Capture SystemsMechanical Capture Systems

❚❚ Any environmentAny environment

❚❚ Measures joint anglesMeasures joint angles

❚❚ Restricts the motionRestricts the motion

77

Optical motion captureOptical motion capture

❚❚ Place markers on the actorPlace markers on the actor

❚❚ Cameras can Cameras can determine marker determine marker positionspositions

88

Optical Capture SystemsOptical Capture Systems

❚❚ 8 or more cameras8 or more cameras

❚❚ Restricted volumeRestricted volume

❚❚ High Frequency (240Hz)High Frequency (240Hz)

❚❚ OcclusionsOcclusions

99

How Does It Work?How Does It Work?

8 cameras + 120 Hz + Special tape = Raw Point Data

1010

Optical motion capture processOptical motion capture process

1.1. Find the skeleton dimensions Find the skeleton dimensions and exact marker positions on the bodyand exact marker positions on the body

2.2. Perform a motion trialPerform a motion trial

3.3. Compute marker positions from camera imagesCompute marker positions from camera images

4.4. Identify and uniquely label markersIdentify and uniquely label markers

5.5. Calculate joint angles from maker pathsCalculate joint angles from maker paths

1111







1212

Problem StatementProblem Statement

1313

Automatic CalibrationAutomatic Calibration

Design Goals:

Fully automaticAny skeletonAccurate

1414

InputInput

Generic Skeleton

Actor’s kinematics structure,and roughhandle positions

CalibrationData

Initial path data that exercises all of the subject’s DOFs

1515

Independent VariablesIndependent Variables

DOFsDOFs

Bone lengthsBone lengths

Handle offsetsHandle offsets

Global scaleGlobal scale

1616







1717




3.3. Compute marker paths from camera imagesCompute marker paths from camera images



1818

Marker IdentificationMarker Identification

(X0, Y0, Z0)

(X1, Y1, Z1)

(X2, Y2, Z2)

θ

At each frame, motion capture gives us a set of points

We would like something more intuitive

1919

Marker Identification ProblemsMarker Identification Problems

Making sense of raw data…2020







2121

IK Problem DefinitionIK Problem Definition

1.1. Create a handle on bodyCreate a handle on body❙❙ position or orientationposition or orientation

2.2. Pull on the handlePull on the handle

3.3. IK figures out how joint IK figures out how joint angles should changeangles should change

2222

Inverse KinematicsInverse Kinematics

Inputs:An articulated skeleton with handles. Desired positions for handles.

Outputs:Joint angles that move handles to desired positions.

2323

Inverse Kinematics (Inverse Kinematics (con’tcon’t))

We are solving IK on a complex model (~50 DOFs and 30 handles).

Motion capture data often contains missing markers.

Many different formulations for IK problem, would like to use one that is best for motion capture data.

2424

More FormallyMore Formally

q actor state vector(joint bundle)

Let:

C(q) constraint functions that pull handles

Then:

solve for q such that C(q) = 0

2525

What’s a Constraint?What’s a Constraint?

❚❚ Can be rich, Can be rich, complicatedcomplicated

❚❚ But most common is But most common is very simple:very simple:

❚❚ Position constraint just Position constraint just sets difference of two sets difference of two vectors to zero:vectors to zero:

θt, φt, σt

θc

θf, φf

desired position desired position dd

xh,yh,zh, θh, φh, σh

q=[ xh,yh,zh,θh, φh, σh, θt, φt, σt, θc, θf, φf ]

handle handle h(q)h(q)C(q) = h(q) - d = 0

2626

Constraint derivativesConstraint derivatives

t, t, t

c

f,

f

desired position desired position dd

xh,yh,zh, h, h, h

q=[ xh,yh,zh, h, h, h, t, t, t, c, f, f ]

handle handle h(q)h(q)

C(q) = h(q) - d = 0C(q) = h(q) - d = 0

( ) ( )C h∂ ∂=∂ ∂

q qq q

2727

Computing DerivativesComputing Derivatives

❙❙ Apply the chain ruleApply the chain rule

❙❙ Need to know how to compute Need to know how to compute derivatives for each transformationderivatives for each transformation

xh,yh,zh, h, h, h

t, t, t

c

f,

f

hs

y

x

z

vshs

( , , ) ( , , ) ( ) )), , (( ,wst t th h h h h h

cf

c cf

h x y z hθ φ σ θ φθθθσ θ φ∂ =

∂∂∂

T R TR T TRR

( , , ) ( , , ) ( )( ), , ) ( ,w st t th h ch h h h f fh x y z hθ φ σ θ φ σ φθ θ= T R TR RT TR

2828

Jacobian MatrixJacobian Matrix

❚❚Can compute Jacobian Can compute Jacobian for each constraint / for each constraint / handle handle

❚❚Value of Jacobian Value of Jacobian depends on current statedepends on current state

❚❚Jacobian Jacobian linearlylinearly relates relates joint angle velocity to joint angle velocity to constraint velocityconstraint velocity

… θe … x . 0 .

y . 1 .

z . 0 .

Cq

ehandle C

Ce

2929

Unconstrained OptimizationUnconstrained Optimization

❚❚MinimizeMinimize

❚❚Move in the direction of the objective function Move in the direction of the objective function gradient:gradient:

2( ) ( ) ( )i ii

G q G q w C q′ = +∑

2 ii i

i

o

CG Gw C

q q q

Gq q

qα

′ ∂∂ ∂= +∂ ∂ ∂

′∂= +∂

∑

3030

RealReal--time Motion Capturetime Motion Capture

GenericSkeleton

SampleMotion

Match markers tohandles.

Calibrate the skeleton.

Begin capture.

Initialize markermatch.

Compute joint angleswith IK.

Update marker match.

Cal

ibra

tio

n Real-tim

e

3131

Motion capture as UIMotion capture as UI

❚❚ Use acting for animation interfaceUse acting for animation interface

3232

Motion TransformationMotion Transformation

❙❙ Start with a mocap sequenceStart with a mocap sequence

❙❙ Edit it to fit the needs of the animationEdit it to fit the needs of the animation

❙❙ Try to be as close to the original motion as Try to be as close to the original motion as possiblepossible

motion capture in movies motion capture · 8 cameras + 120 hz + special tape = raw point data 10...

Documents