3d based hand gesture recognition and tracking

8/7/2019 3d based hand gesture recognition and tracking

http://slidepdf.com/reader/full/3d-based-hand-gesture-recognition-and-tracking 1/31

WELCOME TO THE TECHNICAL

SEMINAR ON

3D MODEL3D MODEL--BASED HANDBASED HAND

GESTURE RECOGNITION ANDGESTURE RECOGNITION AND

TRACKINGTRACKING

3/15/2011 1:51:27 PM

BY

VENKATESH.G(1MV06EC411)



Topic

3/15/2011 1:51:27 PM2

y 1. Introduction

y 2. Human Hand Modeling

y 3. Feature Selection and Extraction

y 4. Model-Based Hand Posture Recognition

y 5. Hand Motion Tracking

y 6. Conclusion

y

Refs.



3/15/2011 1:51:28 PM3

y Hand gestures:y Purpose of human gestures: conversational, controlling,

manipulative, and communicative.

yObjective is to track the hand in a general background and to beable to recognize dynamic gestures in real time.

y Three objectives-simulation, real world video streaming & multiplecamera data fusion.

y Suggest a road map for future development to reach the final goal.



2. Human Hand Modeling

3/15/2011 1:51:28 PM5

y Representations of a hand and 3-D model

y Human hand motion has 26 DOF

y Global configuration: six DOF, representing the pose of the hand (position

and orientation).y Local configuration: 20 angular DOF of fingers

y DIP and PIP joint each has one degree of freedom for rotation

y MCP joint has two degrees of freedom

y Finger motion constraints are applied to define the ranges each

finger may move within.




3/15/2011 1:51:28 PM6




3/15/2011 1:51:28 PM7

y Kinematic model is augmented with shape information to

generate appearances of a hand seen in 2D images

y A 3-D model has been built in graphic programming

environment as shown.y Palm is represented by a flat, chamfered rectangular

y Each segment of fingers was approximated by a sphere-ended

cylinder with a unique dimention.

y Each joint is modeled using a rotation matrix, with a pre-defined

range.



3/15/2011 1:51:28 PM8


3-D hand model:

pose1: open palm, pose2: fist



3/15/2011 1:51:28 PM9


3-D hand model:

pose3: pointing, pose4: victory




3/15/2011 1:51:28 PM10

y All 21 local DOFs are modeled with static and dynamic

constraints.

y Different fingers are color-coded just for easy identification.

Actual models will use skin color.y 2-D projections of any posture at any angle can be easily obtained

by manipulating the model in 3-D space and performing a

perspective projection.

y For global configuration, only one DOF is implemented: rotationalong virtical axis.



3/15/2011 1:51:28 PM12

3. Feature Selection and Extraction

Hand feature: silhouette images

pose1: open palm, pose2: fist



3/15/2011 1:51:28 PM13


Hand feature: silhouette images

pose3: pointing, pose4: victory




3/15/2011 1:51:28 PM14

y Skin color segmentation

y Canny edge detector (Implemented)

y Hand shape normalization (dimension)

y 3D features:

y Stereo cameras obtain 3D images

y Depth info helping for cluttered backgrounds

y Acquired surface is matched to the model surface



4. Model-Based Hand Posture

Recognition

3/15/2011 1:51:28 PM16

y Edge model of the target image is

superimposed onto the distance

image.

y Avg/Max of distance values that edge

model hits gives Chamfer Distance.



3/15/2011 1:51:28 PM17

4. Model-Based Hand Posture Recognition

An example of an image (for the V pose)



Block Diagram of the Prototype

3/15/2011 1:51:28 PM21



5. Hand Motion Tracking

3/15/2011 1:51:28 PM22

y Bayesian tracking

y Multi-resolution partitioning of the state space.

y Particle filtering

yApproximate arbitrary distributions with a set of random samples.

y Deal with clutter and ambiguous situations more effectively, by

multiple hypotheses.

y Tree-based filtering and searching

y

Cluster prototype: a group of similar shape templates.



Simulation

3/15/2011 1:51:28 PM24

Fingertips: Finger posture and Joint angles

y Finger posture: use detected finger markers to determine a

reachable range by the finger along the camera view

directiony The reachable linear segment is then sampled at constant lengths

to calculate a finger posture hypothesis .

y Thumb: binary search of a lookup table of all feasible end

positions

y Other fingers: solved by error model analysis technique



Comments:

3/15/2011 1:51:28 PM29

y A prototype of 3-D model-based hand tracking in a general

environment with unconstrained background.

y Recognize dynamic gestures in real-time.

y Dataglove is used to validate the proposed framework.y Colored markers are used to assist palm and finguretip

recognition.



3D Model-Based Hand Gesture

Recognition and Tracking

3/15/2011 1:51:28 PM3

1

3d based hand gesture recognition and tracking

Documents