Markerless Motion Tracking

University of Southern DenmarkSeptember 15 , 2011

Alex CzarowiczOrganic Motion Inc.

2

Introduction Industries and Apps BioStage Discussion Motion Capture Techniques

Marker-based or wired Optical / Infrared Video-based Electromagnetic Electromechanical

The Human is the Marker

1. Video data streaming • Multiple cameras capture the person from different

angles

2. 3D Reconstruction• Shape from silhouette algorithm creates three

dimensional representation of the person

3. Character Fitting• Human skeleton is then fit into 3D Model

4

Digital YouReal You

Bones

Texture

Mesh

Data

1. Collect Video

The Human is the Marker

2. 3D Reconstruction

3. Mapping

5

Software

• Markerless 3D Motion Tracking

• Tracks Humans and Objects

• Real-Time

• 3D data output (21 bones)

• Hybrid System

• Open, modular architecture

Markerless Tracking

6

Hardware

• Video cameras (FW B)

• 640 x 480 resolution (VGA)

• 30 - 200 fps

• Scalable scanning space

• Indoor, controlled environment

• Volume Calibration

• Record and save also videos

Markerless Tracking

7

Introduction Industries and Apps BioStage Discussion

1. Entertainment OpenStage V2.0

2. Life Sciences BioStage V2.0

3. Training and Simulation

4. Public Installations

Markets / Applications

8

BioStage Model

• Tracking of 21 Segments

• Static user calibrationFully automatic or semi-automatic via calibration posesLandmarks directly measured or derived from separate measurements

BioStage Model

9

Live Demo – Vicomtech Spain

-

OM Admin Software

Stick Figure and Voxel Figure

Live Demo

10

Data format and output

• 3D bone matrices and bone lengths

• SDK (C++)

• Plugins • Innovative Sports Training - MotionMonitor

(Biomechanics)• Autodesk MotionBuilder (Animation)

• 3D mesh/texture internally available

Data Output

11

Integration – Animation

CryEngine

Unreal 3D

Engine

Virtools

OpenStage

Via SDK

MotionBuilder

Via Plug-In

XYZ 3D App

3DMaxMaya

12

Integration – Life Science

Kinetics Electro-myography

Body-- Dimensions

-Structure- Proportion

Position,Angle, Velocity,

etc.

Forces,Moments, Pressure

Distribution

Muscle-action-

potiential

Data integration & Correlation

Kinesiological Analysis

“Body” “Motion” “Forces” “Neuromusc. Control”

Anthro-pometry

Kinematics

BioStage

13

TMM Motion Analysis Software

BioStageKinematics

Force Plates Forces and Moments

EMG Muscle Activity

Video

Via Plugin

Integration – Life Science

14

Live Demo – Vicomtech Spain

-

MotionBuilder and MotionMonitor

-

Show Avatar in MB and Skeleton in MM

Live Demo

15

Motion Tracking – In the Academics

1. Digital Media and related• Game Development and Animation• Training and Simulation• Computer Science• Theatre and Dance

2. Life Science and related• Kinesiology• Exercise Science• Physical Therapy and Rehabilitation• Health, Physical Education, and Recreation

16

1. Motion Analysis …..• Diagnosis & Evaluation Tool• Monitoring Tool • Decision Making Tool

2. …… in• Academics and Research• Clinical Applications• Sport

Motion Analysis – Life Sciences

17

1. Interactive Games

2. Life Science, Training and Simulationhttp://www.youtube.com/user/OrganicMotion#p/c/3EF8710D28235BE0/7/txw5z69ZLfg

3. Digital World UF – Rehab on Soldiers with PTSD

4. World Builder Game

5. OpenStage 2.0

6. Muscle View7. Ski Game http://

www.youtube.com/watch?v=FZ9u5H6Ssow

Samples

18

Steps during a typical Motion Capture session

• Calibration of capture area and/or subject

• Subject set-up

• Tracking of subject’s movement

• Clean-up/Post-processing of data

• Report generation

Workflow Motion Capture

19

Steps and time (approx.)with markerbased systemsCalibration Subject Setup Motion

Tracking Post-Processing Report

5-30min 15-45min 5min 15-120min 5min

1min 5min 5min

Calibration Motion Tracking

Report

Total: 11min

Total: 45min – 195min

Steps and time with OM

Workflow How is OM different from others?Points of Difference

20

• FMT - evidenced-based functionally-oriented

music therapy – Minea Bengsten

• Motion meets Music

http://www.youtube.com/watch?v=sYRir9dHrps

• IDMAA – University of Florida, James Oliverio

Projects - Music

21

• Therapist plays acoustic piano and clients are offered drums and cymbals to be played with specially designed drumsticks.

• Recorders and other wind instruments are also used for breathing development.

• By placing the instruments in certain predetermined positions the client is stimulated to act in collaboration with therapist and at the same time making use of his motor skills, all in all encouraging the progress of development.

• By changing the position of the instruments, varying posture between sitting and standing, the therapist also can stimulate several sensory an motor systems in a structured way

Projects - FMT

22

• Tactile perception - quality, cold and warmth, different materials in the drumsticks, form, distance, direction, angle and position in the grip of the hands, pressure against the supporting part of the body, etc.

• Perception of vibrations - quality, strength, direction, tempo, frequency in sound waves sent to the brain from the hands, supporting parts of the body, vibrations from wind instruments and body movement, etc.

• Proprioception - direction, distance, strength, tempo, quality, etc.

• Vestibular perception - position, stability, direction, tempo of movement, flow of movement, strength of movement and muscles, etc.

• Viscelar perception - strength, use of energy, tempo, etc.

Projects - FMT

23

The variation in placement of the instruments also gives the chance for a great processing of motor activity;

• Development of gross motor ability - is promoted while working with breathing, balance, stability of the body, cross movements, parallel movement in both sides of the body and coordination between hand and feet. It is also promoted when stimulating rotation of the upper part of the body while the trunk is stable in sitting or standing position.

• Development of fine motor skills - is promoted while working with the grip and position of the hands, movement of the wrists, angle of the hands and eye-to-hand coordination.

Projects - FMT

24

Accuracy and Validation

Validation and Accuracy Tests - Overview• Internal – data consistency and reproducibility of results

over repeated trials

• Independent Studies by German Sport University Cologne – Institute for Biomechanics and Orthopedics (Prof. Brueggemann)

• OM with Goniometer

• OM with FAB (inertial system)

• OM with 2D video analysis

• OM with Vicon

25

Benefits

• Reducing time and cost doing Motion Capture• Eliminating subject setup• Instant access • Real-time visual feedback and data• Minimized operating/maintenance cost

• Multi-discipline and interdepartmental use

• Non-intrusive – movements less contrived, natural

• Remote access via distributed network

• True data consistency across setups and operators

Benefits

26

Introduction Organic Motion BioStage Q/A

Thank you!

Q & A

Alex Czarowiczalex@organicmotion.com

Organic Motion, Inc.15 West 37th Street, 7th Floor

New York, NY 10018212-776-6100 x 211

Questions