kinematic model of the hand using computer visionenglish).pdf · kinematic hand model solver...
TRANSCRIPT
![Page 1: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/1.jpg)
Kinematic Model of the Hand
using Computer Vision
Edgar Simo Serra
Barcelona, Spain
Barcelona - April, 2011
![Page 2: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/2.jpg)
Summary
Obtain hand movements through computervision
Reconstruct hand kinematics from data
![Page 3: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/3.jpg)
Index
1 Overview
2 Kinematic Hand Model
3 Hand Detection
4 Kinematic Synthesis
5 Non-linear solver
6 Results
![Page 4: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/4.jpg)
Overview
![Page 5: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/5.jpg)
Motivation
EMG Signals ⇐⇒ Hand Movement
Mapping of Electromyographic (EMG) signals
Understanding hand movements
Prosthesis design
Exoskeleton design
![Page 6: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/6.jpg)
Objectives
1 Design of a theoretical kinematic model of thehand
2 Detection of real hand characteristics withcomputer vision
3 Solver to adjust the theoretical model to theexperimental data
![Page 7: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/7.jpg)
Kinematic Hand Model
![Page 8: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/8.jpg)
State of the Art
Robotic Grippers
Prosthetic Hands
Match function not form
Simplified kinematic models
![Page 9: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/9.jpg)
State of the Art
Robotic Grippers
Prosthetic Hands
Match function not form
Simplified kinematic models
![Page 10: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/10.jpg)
State of the Art
Robotic Grippers
Prosthetic Hands
Match function not form
Simplified kinematic models
![Page 11: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/11.jpg)
Hand Anatomy
5 Fingers
14 Joints
27 Bones
Complex articulatedsystem
![Page 12: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/12.jpg)
Hand Topology
3 common joints (wrist)
Two fingers with 4 joints (index, middle)
Three fingers with 5 joints (third, fourth,thumb)
Thumb has different joint structure
![Page 13: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/13.jpg)
Hand Topology
(a) Full hand. (b) Index finger. (c) Middle finger.
(d) Third finger. (e) Fourth finger. (f) Thumb.
![Page 14: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/14.jpg)
Hand Model
26 degrees of freedomAll revolute jointsStrong similarity with bone structure
![Page 15: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/15.jpg)
Detection of Hand Characteristics
![Page 16: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/16.jpg)
State of the Art
Contact approachLow portabilityHigh costNeed for calibrationLow resolution
![Page 17: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/17.jpg)
State of the Art
Contact approachLow portabilityHigh costNeed for calibrationLow resolution
Non-contact ApproachFocus on realtimeLow precisionApplications in augmentedreality
![Page 18: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/18.jpg)
Approach
Tracking of nailsNon-deformableFixed to hand structureEasily identifiableNot easy to identify rotation
Markers for robust designAllows obtaining of pose (position and rotation)
![Page 19: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/19.jpg)
Markers
Simple for robustness
Convey poseinformation
![Page 20: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/20.jpg)
Markers
Simple for robustness
Convey poseinformation
![Page 21: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/21.jpg)
Detecting Markers
1 Detect candidate points with Corner and edgedetector (Harris 1998)
2 Match candidate points with heuristics3 Use DAISY descriptors to do tracking (Tola
2010)
![Page 22: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/22.jpg)
Detecting Markers
1 Detect candidate points with Corner and edgedetector (Harris 1998)
2 Match candidate points with heuristics3 Use DAISY descriptors to do tracking (Tola
2010)
![Page 23: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/23.jpg)
PnP: Pose Estimation Problem
u = [A][R |t]p
Very important problem in computer vision
Identify camera position given n 2D-3Dcorrespondences u ↔ p
![Page 24: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/24.jpg)
Solving the Pose Estimation
Specific case 4 coplanar points: P4P
Single unique solution (Yuan 1989)Usage of combination of algorithms
Orthogonal Iteration (Hager et al. 2000)Robust Pose Estimation (Schweighofer et al. 2006)
![Page 25: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/25.jpg)
Error Analysis
0 50 100 150 200
0
50
100
Pose (0 degrees)
0 50 100 150
0
50
100
Pose (10 degrees)
0 50 100 150
0
50
100
Pose (20 degrees)
0 50 100 150
0
50
100
Pose (30 degrees)
−50 0 50 100 150
0
50
100
Pose (40 degrees)
−50 0 50 100 150
0
50
100
Pose (50 degrees)
−50 0 50 100 150
0
50
100
Pose (60 degrees)
−50 0 50 100 150
0
50
100
Pose (70 degrees)
−50 0 50 100 150
0
50
100
Pose (80 degrees)
![Page 26: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/26.jpg)
Error Analysis
0 5 10 150
50
100
Noise Standard Deviation (px)
Err
or
(pe
rce
nt)
T. Error (0 degrees)
Interval (99%)
Mean
0 5 10 150
50
100
Noise Standard Deviation (px)
Err
or
(pe
rce
nt)
T. Error (10 degrees)
0 5 10 150
50
100
Noise Standard Deviation (px)
Err
or
(pe
rce
nt)
T. Error (20 degrees)
0 5 10 150
50
100
Noise Standard Deviation (px)
Err
or
(pe
rce
nt)
T. Error (30 degrees)
0 5 10 150
50
100
Noise Standard Deviation (px)
Err
or
(pe
rce
nt)
T. Error (40 degrees)
0 5 10 150
50
100
Noise Standard Deviation (px)
Err
or
(pe
rce
nt)
T. Error (50 degrees)
0 5 10 150
50
100
Noise Standard Deviation (px)
Err
or
(pe
rce
nt)
T. Error (60 degrees)
0 5 10 150
50
100
Noise Standard Deviation (px)
Err
or
(pe
rce
nt)
T. Error (70 degrees)
0 5 10 150
50
100
Noise Standard Deviation (px)E
rro
r (p
erc
en
t)
T. Error (80 degrees)
![Page 27: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/27.jpg)
Error Analysis
0 5 10 150
20
40
60
Noise Standard Deviation (px)
Ro
tatio
n E
rro
r (p
erc
en
t)
R. Error (0 degrees)
Interval (99%)
Mean
0 5 10 150
20
40
60
Noise Standard Deviation (px)
Ro
tatio
n E
rro
r (p
erc
en
t)
R. Error (10 degrees)
0 5 10 150
20
40
60
Noise Standard Deviation (px)
Ro
tatio
n E
rro
r (p
erc
en
t)
R. Error (20 degrees)
0 5 10 150
20
40
60
Noise Standard Deviation (px)
Ro
tatio
n E
rro
r (p
erc
en
t)
R. Error (30 degrees)
0 5 10 150
20
40
60
Noise Standard Deviation (px)
Ro
tatio
n E
rro
r (p
erc
en
t)
R. Error (40 degrees)
0 5 10 150
20
40
60
Noise Standard Deviation (px)
Ro
tatio
n E
rro
r (p
erc
en
t)
R. Error (50 degrees)
0 5 10 150
20
40
60
Noise Standard Deviation (px)
Ro
tatio
n E
rro
r (p
erc
en
t)
R. Error (60 degrees)
0 5 10 150
20
40
60
Noise Standard Deviation (px)
Ro
tatio
n E
rro
r (p
erc
en
t)
R. Error (70 degrees)
0 5 10 150
20
40
60
Noise Standard Deviation (px)R
ota
tio
n E
rro
r (p
erc
en
t)
R. Error (80 degrees)
![Page 28: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/28.jpg)
Detection of Hand
Reliable and robust
Low error in pose estimation with little noise
Experimental dataset for adjusting theoreticalhand model
![Page 29: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/29.jpg)
Kinematic Synthesis
![Page 30: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/30.jpg)
Kinematic Synthesis
Generalized inverse kinematics problem
Changing structural parameters to solveproblem
Motion-to-form problem
Very complex and generally unsolved problem
![Page 31: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/31.jpg)
State of the Art
Simplifying the problemGroebner basesElimination theoryLinear product decomposition
![Page 32: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/32.jpg)
State of the Art
Simplifying the problemGroebner basesElimination theoryLinear product decomposition
Revolute joint structures solved
2R: 2 solutions (A. Perez andJ.M. McCarthy 2003)
![Page 33: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/33.jpg)
State of the Art
Simplifying the problemGroebner basesElimination theoryLinear product decomposition
Revolute joint structures solved
2R: 2 solutions (A. Perez andJ.M. McCarthy 2003)3R: 13 solutions in an interval(Lee et al. 2004)
![Page 34: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/34.jpg)
State of the Art
Simplifying the problemGroebner basesElimination theoryLinear product decomposition
Revolute joint structures solved
2R: 2 solutions (A. Perez andJ.M. McCarthy 2003)3R: 13 solutions in an interval(Lee et al. 2004)4R: Unsolved
![Page 35: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/35.jpg)
State of the Art
Simplifying the problemGroebner basesElimination theoryLinear product decomposition
Revolute joint structures solved
2R: 2 solutions (A. Perez andJ.M. McCarthy 2003)3R: 13 solutions in an interval(Lee et al. 2004)4R: Unsolved...26R: Unsolved (our problem)
![Page 36: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/36.jpg)
Kinematic Chain
F
[G]
S1
S2
S3
a12
d12
a23
[H]
Θ12
α12
α23
M
Articulated system formed by jointsDifferent topologies
SerialParallelLoopTree (hand model)
F
[G]
Common Joints
Branch 2
Branch 1
S1
S2
S3 S4
S41
2
![Page 37: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/37.jpg)
Plucker Coordinates
Line representation: S = s︸︷︷︸
orientation
+ǫ s0︸︷︷︸
moment
Only 2 representations per line (s and −s)
![Page 38: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/38.jpg)
Dual Quaternions
Homogeneous Matrix [H]︸ ︷︷ ︸
16 components
7−→ Dual Quaternion Q︸ ︷︷ ︸
8 components
Element of Clifford algebraUnit dual quaternions
Represents 3D displacements, points and linesTwo constraints: qq∗ = 1 and q · q0 = 0
![Page 39: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/39.jpg)
Exponential Map
Based on Lie Algebra
Allows joints to be written as1-parameter screws
Joint parameters: θ = θ + ǫdPrismatic: θ = 0Revolute: d = 0
d
Θ
S
S(θ) = eθ
2S = cos
θ
2+ sin
θ
2S
![Page 40: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/40.jpg)
Forward Kinematics
Determines end effector position based onjoints
Direct calculation for serial kinematictopologies
T (θ) = S1(θ1)S2(θ2) · · · Sn(θn)G
= eθ12S1e
θ22S2 · · · e
θn
2SnG
= (cosθ1
2+ sin
θ1
2S1) · · · (cos
θn
2+ sin
θn
2Sn)G
![Page 41: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/41.jpg)
Forward Kinematics
∆θ = θ − θref + ǫ(d − dref )
Q(∆θ) = S1(∆θ1)S2(∆θ2) · · · Sn(∆θn)
= e∆θ1
2S1e
∆θ22
S2 · · · e∆θn
2Sn
= (cos∆θ1
2+ sin
∆θ1
2S1)
· · · (cos∆θn
2+ sin
∆θn
2Sn)
![Page 42: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/42.jpg)
Design Equations
Input: task defined by m − 1 relativetransformations P1j = TjT
−11 , j = 2, . . . , m
Variables: joint axes and joint anglesEquations:
S1j(∆θ1j)S2j(∆θ2j)S3j(∆θ3j)
Sk4j(∆θk
4j) · · · Sk7j(∆θk
7j) − Pk1j = 0, k ∈ 1, 2,
S1j(∆θ1j)S2j(∆θ2j)S3j(∆θ3j)
Sk4j(∆θk
4j) · · · Sk8j(∆θk
8j) − Pk1j = 0, k ∈ 3, 4, 5,
j = 2, ..., m
![Page 43: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/43.jpg)
Non-linear Solver
![Page 44: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/44.jpg)
System Dimension
Relations between variables and equationsdecrease dimension
Equations use 3 algebraic sets:
Set Comp Ind Notesθ 1 1 Periodic with period 2π
S = s + ǫs0 6 4 s · s0 = 0 , ‖s‖ = 1
Q = q + ǫq0 8 6 q · q0 = 0 , qq∗ = 1
![Page 45: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/45.jpg)
System Dimension
System unknowns:
n0x = r( 4
︸︷︷︸
Structural
+(m − 1)︸ ︷︷ ︸
Joint
)
nx = r(6 + (m − 1))
System functions:
n0f = b6(m − 1)
nf = b8(m − 1)
![Page 46: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/46.jpg)
System Dimension
System must be solvable: n0f ≥ n0
x
m =4r
6b − r+ 1 > 0
6b − r > 0
Multiple solution possibilities
![Page 47: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/47.jpg)
System Reduction
System can be reduced to subsystems that canbe solved
Allows reducing the dimension of the system
b r n0x = n0
f nx nf m Notes5 26 780 832 1092 27 Full hand model2 11 528 550 726 45 (4R, 4R) fingers1 5 120 130 170 21 5R, no common
![Page 48: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/48.jpg)
Solution Bound
High dimensionality of the system
Important non-linearity of the equationsBezout’s theorem: D = d1d2 · · · dn
Serial chain: d = (3r)6 (A. Perez 2003)
di = 216(m−1) , i ∈ 1, 2,
di = 246(m−1) , i ∈ 3, 4, 5,
D =5∏
i=1
di ≈ 101058
![Page 49: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/49.jpg)
Numerical Solver
Global optimizer
Allow overdetermined systems
![Page 50: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/50.jpg)
Numerical Solver
Global optimizer
Allow overdetermined systemsLevenberg-Marquadt Algorithm
Local optimizerLeast squaresVery localVery difficult to parallelize
![Page 51: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/51.jpg)
Numerical Solver
Global optimizer
Allow overdetermined systemsLevenberg-Marquadt Algorithm
Local optimizerLeast squaresVery localVery difficult to parallelize
Genetic Algorithm“Global” optimizer(meta-heuristics)Easy to implementVery hard to tuneExtremely parallel
![Page 52: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/52.jpg)
Implementation
Hybrid solverReduces search space by minimizing withLevenberg-Marquadt algorithmGlobal optimizer by using genetic algorithm
![Page 53: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/53.jpg)
Implementation
Hybrid solverReduces search space by minimizing withLevenberg-Marquadt algorithmGlobal optimizer by using genetic algorithm
![Page 54: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/54.jpg)
Implementation
Hybrid solverReduces search space by minimizing withLevenberg-Marquadt algorithmGlobal optimizer by using genetic algorithm
![Page 55: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/55.jpg)
Results
![Page 56: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/56.jpg)
Solver
Parallel algorithm
Bottleneck in QR decomposition fromLevenberg-Marquadt algorithm
Average run:
Generations Runtime (hours)38 60
![Page 57: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/57.jpg)
Solver
Mutation
Solution
![Page 58: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/58.jpg)
Solutions
Non-anthropomorphic structure
Moves finger positions exactly the same as ahandNeeds support for constraints to distinguish
More complicated solver
![Page 59: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/59.jpg)
Solutions
Non-anthropomorphic structure
Moves finger positions exactly the same as ahandNeeds support for constraints to distinguish
More complicated solver
![Page 60: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/60.jpg)
Solutions
(a) Full hand. (b) Index finger. (c) Middle finger.
(d) Third finger. (e) Fourth finger. (f) Thumb.
![Page 61: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/61.jpg)
Results
Softwarelibdq: Library for manipulating dual quaternionsKinematic hand model solverSoftware to track fingernail markers
PublicationE. Simo-Serra, F. Moreno-Noguer and A.Perez-Gracia, “Design of non-anthropomorphicrobotic hands for anthropomorphic tasks”,ASME-IDETC, accepted.
![Page 62: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/62.jpg)
Conclusions
Major problems in engineeringPnP or pose estimation problemMotion-to-form or dimensional kinematic synthesisproblem
Results reflect maturity in both problems
![Page 63: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/63.jpg)
Conclusions
Major problems in engineeringPnP or pose estimation problemMotion-to-form or dimensional kinematic synthesisproblem
Results reflect maturity in both problemsHand detection
Low computation costReliable with slow movements
![Page 64: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/64.jpg)
Conclusions
Major problems in engineeringPnP or pose estimation problemMotion-to-form or dimensional kinematic synthesisproblem
Results reflect maturity in both problemsHand detection
Low computation costReliable with slow movements
Kinematic synthesisVery high complexityLarge number of solutionsNon-anthropomorphic solutions found
![Page 65: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/65.jpg)
Conclusions
Objectives1 Defined theoretical hand model2 Detected real hand characteristics with computer
vision3 Adjusted theoretical model to experimental data
![Page 66: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/66.jpg)
Conclusions
Objectives1 Defined theoretical hand model2 Detected real hand characteristics with computer
vision3 Adjusted theoretical model to experimental data
Anthropomorphic solution not foundFirst project to attempt dimensional kinematicsynthesis of tree-like topologiesKinematic synthesis is generally unsolved (3R isnot algebraically solved)Upper bound of 101058 solutions
![Page 67: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/67.jpg)
Future Work
Flexible generic tree-like topology dimensionalkinematic synthesis solver
Exchangeable backendsScripting supportSupport for constraints
![Page 68: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/68.jpg)
Future Work
Flexible generic tree-like topology dimensionalkinematic synthesis solver
Exchangeable backendsScripting supportSupport for constraints
Study of design equationsHow many solutions are there?What causes loss of dimensionality?How are solutions (and local minima) distributed?Can the a reduced model of the search space beused for initial estimation?Are matrix representation better at convergence?
![Page 69: Kinematic Model of the Hand using Computer VisionEnglish).pdf · Kinematic hand model solver Software to track fingernail markers Publication E. Simo-Serra, F. Moreno-Noguer and](https://reader034.vdocuments.site/reader034/viewer/2022051921/600e4c83a7423b4dec182e07/html5/thumbnails/69.jpg)
Thanks
(a) Full hand. (b) Index finger. (c) Middle finger.
Questions?
(d) Third finger. (e) Fourth finger. (f) Thumb.