CS 527 – Computer Animation October 17, 2006

Estimating Cloth Simulation Parameters from Video

Kiran S. Bhat, Christopher D. Twigg, Jessica K. Hodgins, Pradeep K. Khosla, Zoran Popovic and Steven M. Seitz,School of Computer Science, Carnegie Mellon University

Department of Computer Science and Engineering, University of Washington

Presented by:Ratko Jagodic

CS 527 – Computer Animation October 17, 2006

Cloth

• Need realism for mixed CG/real scenes

• Realism now possible

CS 527 – Computer Animation October 17, 2006

Simulating Cloth

• Cloth characteristics determined by resistance to:– Bending, stretching, shearing, external forces,

aerodynamic effects, friction and collisions

• Great results are achievable with the right set of parameters

• BUT, difficult and time consuming to choose parameters

• Different approach: estimate parameters from real-life video of cloth movement

CS 527 – Computer Animation October 17, 2006

Related Work

• Various cloth models:– Using stiff springs (Baraff and Witkin)– Bending energy model (Choi and Ko)

• Modeling cloth collisions and friction:– Untangling cloth– Preemptively avoiding collisions

• Estimating parameters:– Mechanical tests measuring force needed for deforming

(Breen)– Searching for parameters that match a piece of real cloth

draped over a sphere (Jojic and Huang)– Neither considered dynamic parameters

CS 527 – Computer Animation October 17, 2006

Proposed Approach

• Estimate parameters from the video of moving cloth pieces

• Choose simulated cloth parameters based on a cloth model

• Apply an optimization function until the simulated cloth behaves like the real one from the video

CS 527 – Computer Animation October 17, 2006

Cloth Model

• Cloth model requirements:– Realism and practicality– Scalable to varying resolutions of cloth– Input parameters independent of

meshing

• Baraff and Witkin cloth model was chosen with minor adjustments

• Collisions based on Bridson’s model (again, slightly modified)

CS 527 – Computer Animation October 17, 2006

A Metric for Matching Simulation to Video

• Two sequences compared frame-by-frame• Average error is computed across the whole

sequence• Perceptually motivated metric

– Human perceptual system is sensitive to moving edges in video

– Moving edges = folds in cloth

• The metric returns the difference in folds between the real and simulated cloth

CS 527 – Computer Animation October 17, 2006

Fold Detection and Representation

• Project striped pattern on pieces of cloth to eliminate lighting and material reflectance

• Compute dominant orientations of edge pixels angle map

• Threshold the gradient of the angle map gradient mask

CS 527 – Computer Animation October 17, 2006

Fold and Silhouette Comparison

Angle map from video Angle map from simulation

Angle map difference Angle map diff x gradient mask

Silhouette Comparison – the difference between the shapes

CS 527 – Computer Animation October 17, 2006

Optimization

• Before optimization:– 4 values are chosen for each parameter across its entire

range– Then simulate the fabric for each of those 4 points and

compare with the real fabric.– Use the best of the 4 points to start the optimization with

• Optimization:

– Uses simulated annealing for n-variable optimization

silhk

foldkk EEE

CS 527 – Computer Animation October 17, 2006

Experiments

• Perform simpler experiments that would yield parameters for the cloth model = calibration

• Fabric:– Linen, fleece, satin and knit

• Static Test:– How the fabric naturally hangs under gravity

• Waving Test:– One corner is fixed and the other moved back and forth

• Swatches of cloth were used– give reasonable estimates without the need to optimize

on complex fabric geometries

CS 527 – Computer Animation October 17, 2006

Experimental Setup

• Setup:– Vicon motion capture system for locating two top corners

of fabric– Motion capture markers for calibrating the camera and

the projector– 2.8 GHz Xeon processor

• Trials:– 2 trials per fabric– 1 trial = 50 frames of video– each takes 50 hours to optimize

• Parameters:– Bend, stretch, shear, bend damping, stretch damping,

shear damping, linear drag, quadratic drag, drag degradation

CS 527 – Computer Animation October 17, 2006

Results – Static Test

• Good estimates for the parameters that have low variability (stretch and bend)

• Bad estimates for high variability parameters since they do not contribute much to the total error (dynamic parameters like damping)

CS 527 – Computer Animation October 17, 2006

Results – Waving Test

• Added air drag parameters• Results good but probably needed longer or more

complex sequences since a few parameters still had high variability

CS 527 – Computer Animation October 17, 2006

More Results

• Evaluated obtained parameters on longer sequences (150 frames) = good match

• Evaluated on a more complex piece of cloth (a skirt) = approximately correct

CS 527 – Computer Animation October 17, 2006

Conclusion

• The estimation produces parameters that yield appealing results.

• Realism in the end is still dependent on the cloth model so good ones are needed

• Future work:– Different metrics could be explored (e.g. time varying as

opposed to frame-by-frame)– More complex garments may need additional test that

mimic behaviors better suited to a particular application of fabric (such as form fitting pants)

– Different parameters are needed for more complex effects (e.g. stretching in one direction causing shrinking in the other)

CS 527 – Computer Animation October 17, 2006

Video

• Movie

CS 527 – Computer Animation October 17, 2006

Questions?

CS 527 – Computer Animation October 17, 2006

Simulated Annealing