evidential modeling for pose estimation fabio cuzzolin, ruggero frezza computer science department...
TRANSCRIPT
![Page 1: Evidential modeling for pose estimation Fabio Cuzzolin, Ruggero Frezza Computer Science Department UCLA](https://reader038.vdocuments.site/reader038/viewer/2022110116/5515eb14550346dd6f8b5167/html5/thumbnails/1.jpg)
Evidential modeling for Evidential modeling for pose estimationpose estimation
Fabio Cuzzolin, Ruggero Frezza
Computer Science Department
UCLA
![Page 2: Evidential modeling for pose estimation Fabio Cuzzolin, Ruggero Frezza Computer Science Department UCLA](https://reader038.vdocuments.site/reader038/viewer/2022110116/5515eb14550346dd6f8b5167/html5/thumbnails/2.jpg)
Myself
Master’s thesis on gesturegesture recognitionrecognition
at the University of Padova Ph.D. thesis on the theory of theory of
evidenceevidence Post-doc in Milan with the Image and
Sound Processing group Post-doc at UCLA in the Vision Lab
![Page 3: Evidential modeling for pose estimation Fabio Cuzzolin, Ruggero Frezza Computer Science Department UCLA](https://reader038.vdocuments.site/reader038/viewer/2022110116/5515eb14550346dd6f8b5167/html5/thumbnails/3.jpg)
Px
Py
P
F (s)x
F (s)y
y
x
y
x
My past work…
geometric approachgeometric approach to the theory of belief functions space of belief functions geometry of Dempster’s rule
![Page 4: Evidential modeling for pose estimation Fabio Cuzzolin, Ruggero Frezza Computer Science Department UCLA](https://reader038.vdocuments.site/reader038/viewer/2022110116/5515eb14550346dd6f8b5167/html5/thumbnails/4.jpg)
.. again ..
algebra of compatible frames linear independence on lattices action recognition and object
tracking metrics on the space of dynamical
models
![Page 5: Evidential modeling for pose estimation Fabio Cuzzolin, Ruggero Frezza Computer Science Department UCLA](https://reader038.vdocuments.site/reader038/viewer/2022110116/5515eb14550346dd6f8b5167/html5/thumbnails/5.jpg)
43
21
… and today’s talk
the pose estimation problemthe pose estimation problem
model-free pose estimationmodel-free pose estimation
evidential modelevidential model
experimental resultsexperimental results
![Page 6: Evidential modeling for pose estimation Fabio Cuzzolin, Ruggero Frezza Computer Science Department UCLA](https://reader038.vdocuments.site/reader038/viewer/2022110116/5515eb14550346dd6f8b5167/html5/thumbnails/6.jpg)
Pose estimation estimating the “posepose” (internal configuration)
of a moving body from the available images
salient image measurements: featuresfeatures
Qtq k ˆt=0
t=T
![Page 7: Evidential modeling for pose estimation Fabio Cuzzolin, Ruggero Frezza Computer Science Department UCLA](https://reader038.vdocuments.site/reader038/viewer/2022110116/5515eb14550346dd6f8b5167/html5/thumbnails/7.jpg)
Model-based estimation if you have an a-priori modela-priori model of the object .. .. you can exploit it to help (or drive) the
estimation
example: kinematic model
![Page 8: Evidential modeling for pose estimation Fabio Cuzzolin, Ruggero Frezza Computer Science Department UCLA](https://reader038.vdocuments.site/reader038/viewer/2022110116/5515eb14550346dd6f8b5167/html5/thumbnails/8.jpg)
Model-free estimation
if you do not have any information about the body..
the only way to do inference is to learn a maplearn a map between features and
poses directly from the data
this can be done in a training stagetraining stage
![Page 9: Evidential modeling for pose estimation Fabio Cuzzolin, Ruggero Frezza Computer Science Department UCLA](https://reader038.vdocuments.site/reader038/viewer/2022110116/5515eb14550346dd6f8b5167/html5/thumbnails/9.jpg)
Collecting training data motion capture system
3D locations of markers = pose
![Page 10: Evidential modeling for pose estimation Fabio Cuzzolin, Ruggero Frezza Computer Science Department UCLA](https://reader038.vdocuments.site/reader038/viewer/2022110116/5515eb14550346dd6f8b5167/html5/thumbnails/10.jpg)
Training data when the object performs some “significant”
movements in front of the camera … … a finite collection of configuration values
are provided by the motion capture system
… while a sequence of features is computed from the image(s)
q q
y y
Q~
1
1
T
T
![Page 11: Evidential modeling for pose estimation Fabio Cuzzolin, Ruggero Frezza Computer Science Department UCLA](https://reader038.vdocuments.site/reader038/viewer/2022110116/5515eb14550346dd6f8b5167/html5/thumbnails/11.jpg)
Learning feature-pose maps
Hidden Markov modelsHidden Markov models provide a way to build feature-pose maps from the training data
a Gaussian density for each state is set up on the feature space -> approximate feature spaceapproximate feature space
mapmap between each region and the set of training poses qk with feature value yk inside it
![Page 12: Evidential modeling for pose estimation Fabio Cuzzolin, Ruggero Frezza Computer Science Department UCLA](https://reader038.vdocuments.site/reader038/viewer/2022110116/5515eb14550346dd6f8b5167/html5/thumbnails/12.jpg)
Evidential model
approximate feature spaces ..
.. and approximate parameter space ..
.. form a family of compatible family of compatible frames: the evidential modelframes: the evidential model
![Page 13: Evidential modeling for pose estimation Fabio Cuzzolin, Ruggero Frezza Computer Science Department UCLA](https://reader038.vdocuments.site/reader038/viewer/2022110116/5515eb14550346dd6f8b5167/html5/thumbnails/13.jpg)
Estimation
these belief functions are projected onto the approximate parameter space ..
.. and combined through Dempster’s rule
a point-wise estimate of the pose is obtained by probabilistic approximation
new features are represented as belief functions ..
![Page 14: Evidential modeling for pose estimation Fabio Cuzzolin, Ruggero Frezza Computer Science Department UCLA](https://reader038.vdocuments.site/reader038/viewer/2022110116/5515eb14550346dd6f8b5167/html5/thumbnails/14.jpg)
Human body tracking
two experiments, two views
four markers on the right arm
six markers on both legs
![Page 15: Evidential modeling for pose estimation Fabio Cuzzolin, Ruggero Frezza Computer Science Department UCLA](https://reader038.vdocuments.site/reader038/viewer/2022110116/5515eb14550346dd6f8b5167/html5/thumbnails/15.jpg)
Feature extraction
three steps: original image, color segmentation, bounding box
185
94
161
38
185
94
161
38
![Page 16: Evidential modeling for pose estimation Fabio Cuzzolin, Ruggero Frezza Computer Science Department UCLA](https://reader038.vdocuments.site/reader038/viewer/2022110116/5515eb14550346dd6f8b5167/html5/thumbnails/16.jpg)
Performances comparison of three models: left view
only, right view only, both views
pose estimation yielded by the overall model
estimate associated with the “right” model
“left” model
ground truth
![Page 17: Evidential modeling for pose estimation Fabio Cuzzolin, Ruggero Frezza Computer Science Department UCLA](https://reader038.vdocuments.site/reader038/viewer/2022110116/5515eb14550346dd6f8b5167/html5/thumbnails/17.jpg)
Estimation errors Euclidean distance between real and
predicted marker position
marker 4
3cm
marker 2
8cm
![Page 18: Evidential modeling for pose estimation Fabio Cuzzolin, Ruggero Frezza Computer Science Department UCLA](https://reader038.vdocuments.site/reader038/viewer/2022110116/5515eb14550346dd6f8b5167/html5/thumbnails/18.jpg)
Visual estimate
Tk
kIkpI..1
)()(ˆˆ compares the actual image
with the weighted sum of the training images
![Page 19: Evidential modeling for pose estimation Fabio Cuzzolin, Ruggero Frezza Computer Science Department UCLA](https://reader038.vdocuments.site/reader038/viewer/2022110116/5515eb14550346dd6f8b5167/html5/thumbnails/19.jpg)
Conclusions pose estimation of unknown objects is a
difficult task a bottom-up model has to be built from the
data in a training session the DS framework allows to formalize the
idea of feature-pose maps in a natural way through the notion of compatible frames
Dempster’s combination provides a method to integrate features to increase robustness