evolving character controllers for collision preparation

09/08/06 Oregon State University Rose / Metoyer

Evolving Character Controllersfor Collision Preparation

Robert Rose09/08/06


Agenda• Motivation• Overview• Background• Character Model• Evolving Character Controllers for Collision Preparation• Results• Conclusion


Motivation• Video Games

– Games have many collisions with human characters– Recent focus has been on collision response:

• Ragdoll (pure physics)• Hybrid Response (Zordan05, Mandel04)

– Little attention has been given to collision preparation– A complete preparation and response system would increase a

game’s realism dramatically

DOA4 - Tecmo NCAA 2003 - EA ABC Television


Motivation• Exploration of Evolutionary Robotics

– Developing controllers for robots and physically simulated characters by hand is a difficult problem

– Evolving Virtual Creatures (Sims94)– Evolved human locomotion (Smith98, Wolff03)– A strong sense…

• Holy grail: complete dynamics-based control of human characters• Evolutionary techniques must be the key!

Sims94Smith98 Wolff03


Motivation• Video


Overview• Physically simulate character model• Control system

– Desired joint angles

• Test environment– Throw a projectile at the character

• Evaluation system– Measure “pain” perceived by character

• Optimization system– Genetic algorithms


Background• Motion Capture vs. Physical Simulation• Physically Simulated Humans• Controlling Physically Simulated Humans

– Special-purpose controllers• Bruderlin89, Raibert91, Laszlo96, Takashima90, Hodgins95

– Combining controllers• de Garis90, Faloutsos01

– Developing controllers by hand is not optimal• Time consuming• Re-targeting difficult (or not possible)


Background• Evolution of Human Character Controllers

– Genetic Algorithms• Roberts03, Wyeth03

– Genetic Algorithms + Neural Networks• de Garis90, Smith98, Nolfi00

– Genetic Programming• Gritz95/97, Wolff03

Gritz95 Roberts03


Background• Collision Response

– Motion tracking - Zordan02• Applies torques to character to track motion capture data• Weaken control after impact, then gradually strengthen

– Fall control - Mandel04• Transitions to fall controller after impact or tripping• Motion capture resumes after character is in “rest” state



– Hybrid Control• Dynamic Response for Motion Capture Data, Zordan05• Use control system to bring character to nearest motion data• Blend out of control, into motion capture after a short period of time



– Endorphin - Natural Motion• Offline solution; “multi-pass simulation technique” for building

“uncapturable” motion capture sequences• Contains 40-50 preprogrammed controllers that can be applied to your

character. Ex: Stagger backwards and fall over• All controllers end in a rest state (ragdoll)


Character Model• Physically Simulated Character Model

– 9 segments, 8 joints, 18 DOF– Uniform density segments– Simulated in Novodex


Character Model• 1 DOF Joint Control

– PD-servos - “spring” controller• Ball and Socket Joint Control

– Our solution is an extension of PD-servos to 3 dimensions

• Controller Stability– Precision loss is our worst enemy

• Causes high-frequency oscillations - smoothing outputs helps– Overpowered on the twist axis

• Damping the twist axis torque helps


Character Model• Pain Mesh

– Color codes areas of the character model according to “pain regions”


Evolving Character Controllers• Testing Controllers

– Evolutionary process generates a lot of controllers– Fitness measures performance of controller during a single test– Multiple tests are necessary to ascertain “total fitness”

• Evaluating Controllers– Fitness metrics: pain, energy, distance

• Generating Controllers– Chromosome format– Selection process– Mutation and crossover


Testing Controllers• 36 tests per controller

– 9 tests according to the threat grid– x 4 tests for each mode of joint failure

• Threat grid– We desire “general” solutions rather than solutions that train for

blocking a specific case– 9 tests with “jitter”


Testing Controllers• Joint Failure

– Humans typically block with both hands and duck the head– We desire solutions that use all faculties of the character model– Joint failure forces the character to defend itself using:

• Both arms• Only the right arm, then only the left arm• No arms at all


Evaluating Controllers• Evaluation of a test run requires a fitness function

– Fitness = pain + energy + distance

• Pain Metric– How effectively did the controller minimize the amount of pain

perceived by the character?

• Energy Metric– How much energy did the controller use to defend the character?

• Distance Metric– How far away did the controller block the projectile from the

character’s head?


Evaluating Controllers• Pain Metric

– Goal: Find controller that cause the least amount of harm to come to the character

– Premise: Getting hit in some parts of your body hurts more than others

– Method:• Take the point of impact pi and use it to look up the “pain region” the

character was hit in -- each region has a pain multiplier• Apply the pain multiplier to the force of the impact• Sum for every point of impact


Evaluating Controllers• Pain Metric

– Pain multipliers developed ad hoc


Evaluating Controllers• Energy Metric

– Goal: We want to find “realistic” solutions– Premise: Humans don’t “over-compensate” movement– Method: Penalize controller based on energy consumption

• Picking a large value for ct prevents solutions that over-compensate• But, picking too large value has undesirable consequences…

– Energy metric overtakes the pain metric– Converges on solutions that only minimize energy

• ct = 0.001


Evaluating Controllers• Distance Metric

– Goal: We want to find different styles of poses– Premise: Humans sometimes block far from their head– Method: Penalize controllers that block close to the head

• Take the distance from the first point of impact (p0) to the head (h)• Round this to the nearest integer• Apply distance multiplier cd

• cd = 0.015


Generating Controllers• To generate controllers, we use Genetic Algorithms

– Encode desired joint angles as a chromosome– Tournament selection process– Perform mutation and crossover to generate new chromosomes– Slope of solutions over a window determines end


Results


Genetic Algorithm Parameters• Before we could begin testing our pain measurement

theories we needed to stabilize the parameters to the GA• Tested four mutation schemes (below)• Tested two tournament sizes (4, 8)• Tested eight population sizes (100-800 increments of 100)


Genetic Algorithm Parameters• Mutation Schemes A-D

Fitness vs. Time


Genetic Algorithm Parameters• Tournament Sizes 4,8 - Population Sizes 100-800

Fitness vs. Time


Genetic Algorithm Parameters• Conclusion:

– Mutation Scheme A• Clear winner, others didn’t converge as reliably

– Tournament size didn’t seem to matter• Others have observed the same behavior (Gritz97)• 4 was chosen arbitrarily

– Population size not so clear• Population sizes of 400 and over always converged on a solution• Population sizes under 400 converged, but more sporadically• Ultimately, convergence time was the decider: 400 seemed optimal

• Footnote:– Early in this work I used culling as a selection process…– Simulations took overnight to converge on a solution!


Results• Evolution in progress


Results• Results at various angles


Results• Hand emphasis: hands-only collision


Results• Video


Conclusion• Collision Preparatory Poses

– We successfully generated preparatory poses automatically– Tweaking the fitness function and collision space gave us different

styles of results– Our system occasionally gave us some odd solutions

• Blocking with the back of the hands - Ow!• The mysterious “inverted elbow” pose - I wish I could do that!

• Opportunities– Energy metric enhancements

• Consider the direction torque is being applied in - no back of hands– Joint limit penalties

• Penalize the system for placing the character in positions that would fracture a bone on impact


Conclusion• Evolving Character Controllers

– Automated development of human character controllers is a hard problem!

• Developing realistic controllers requires a large search space• Evaluation of human movement requires codifying what is “human”• Our problem domain works well because of its simplicity

– Ultimately, it’s about what the artist wants• Artist codification of a problem, combined with pain measurement

could be a powerful tool

evolving character controllers for collision preparation

Documents