a discriminative key pose sequence model for recognizing human interactions arash vahdat, bo gao,...
TRANSCRIPT
![Page 1: A Discriminative Key Pose Sequence Model for Recognizing Human Interactions Arash Vahdat, Bo Gao, Mani Ranjbar, and Greg Mori ICCV2011](https://reader034.vdocuments.site/reader034/viewer/2022051412/551c4383550346a5458b4668/html5/thumbnails/1.jpg)
A Discriminative Key Pose Sequence Model for Recognizing Human
Interactions
Arash Vahdat, Bo Gao, Mani Ranjbar, and Greg Mori
ICCV2011
![Page 2: A Discriminative Key Pose Sequence Model for Recognizing Human Interactions Arash Vahdat, Bo Gao, Mani Ranjbar, and Greg Mori ICCV2011](https://reader034.vdocuments.site/reader034/viewer/2022051412/551c4383550346a5458b4668/html5/thumbnails/2.jpg)
Goal
![Page 3: A Discriminative Key Pose Sequence Model for Recognizing Human Interactions Arash Vahdat, Bo Gao, Mani Ranjbar, and Greg Mori ICCV2011](https://reader034.vdocuments.site/reader034/viewer/2022051412/551c4383550346a5458b4668/html5/thumbnails/3.jpg)
Outline
• Introduction• Related methods• Modeling Human Interactions• Single Subject Key Pose Sequence Model• Interaction Key Pose Sequence Model• Learning the parameters• Experiments
![Page 4: A Discriminative Key Pose Sequence Model for Recognizing Human Interactions Arash Vahdat, Bo Gao, Mani Ranjbar, and Greg Mori ICCV2011](https://reader034.vdocuments.site/reader034/viewer/2022051412/551c4383550346a5458b4668/html5/thumbnails/4.jpg)
Introduction
• This paper focuses on recognizing interactions between individuals.
• The sequences of key poses between peoples will be combined into activity level.
• The activities to be recognized include hugging ,shaking hands,pointing, punching, kicking,pushing each others.
![Page 5: A Discriminative Key Pose Sequence Model for Recognizing Human Interactions Arash Vahdat, Bo Gao, Mani Ranjbar, and Greg Mori ICCV2011](https://reader034.vdocuments.site/reader034/viewer/2022051412/551c4383550346a5458b4668/html5/thumbnails/5.jpg)
Introduction
• Not every movement or pose by the target is relevant to the activity to be recognized.
• We use an examplar-based model by giving the pose a score to match the key poses.
• When people do some activities, they will do the key poses in a chronological order.
![Page 6: A Discriminative Key Pose Sequence Model for Recognizing Human Interactions Arash Vahdat, Bo Gao, Mani Ranjbar, and Greg Mori ICCV2011](https://reader034.vdocuments.site/reader034/viewer/2022051412/551c4383550346a5458b4668/html5/thumbnails/6.jpg)
Related methods
• Ryoo and Aggarwal [13]develop a matching kernel that considers spatial and temporal relations between space-time interest points.
• Yao et al. [23] use a Hough transform voting scheme from an interest point representation.
[13] M. Ryoo and J. Aggarwal. Spatio-temporal relationship match: Video structure comparison for recognition of complex human activities. In ICCV, 2009.[23] A. Yao, J. Gall, and L. Van Gool. A hough transform-based voting framework for action recognition. In CVPR, 2010. 2, 6
![Page 7: A Discriminative Key Pose Sequence Model for Recognizing Human Interactions Arash Vahdat, Bo Gao, Mani Ranjbar, and Greg Mori ICCV2011](https://reader034.vdocuments.site/reader034/viewer/2022051412/551c4383550346a5458b4668/html5/thumbnails/7.jpg)
Recognition result in [13]
![Page 8: A Discriminative Key Pose Sequence Model for Recognizing Human Interactions Arash Vahdat, Bo Gao, Mani Ranjbar, and Greg Mori ICCV2011](https://reader034.vdocuments.site/reader034/viewer/2022051412/551c4383550346a5458b4668/html5/thumbnails/8.jpg)
Modeling Human Interactions
• There are four things to know:• 1. Who is involved in the interaction?– Subject or object
• 2. When do the key poses occur? – The interval of the key poses
• 3. How are the key poses executed?– With hand or leg , powerful or weak
• 4. Where are the people when the key poses occur?
![Page 9: A Discriminative Key Pose Sequence Model for Recognizing Human Interactions Arash Vahdat, Bo Gao, Mani Ranjbar, and Greg Mori ICCV2011](https://reader034.vdocuments.site/reader034/viewer/2022051412/551c4383550346a5458b4668/html5/thumbnails/9.jpg)
Modeling Human Interactions
• we will assume F maximizes a model G that includes the latent variables H:
• The variables H are the answer of the four questions above.
![Page 10: A Discriminative Key Pose Sequence Model for Recognizing Human Interactions Arash Vahdat, Bo Gao, Mani Ranjbar, and Greg Mori ICCV2011](https://reader034.vdocuments.site/reader034/viewer/2022051412/551c4383550346a5458b4668/html5/thumbnails/10.jpg)
Single Subject Key Pose Sequence Model
![Page 11: A Discriminative Key Pose Sequence Model for Recognizing Human Interactions Arash Vahdat, Bo Gao, Mani Ranjbar, and Greg Mori ICCV2011](https://reader034.vdocuments.site/reader034/viewer/2022051412/551c4383550346a5458b4668/html5/thumbnails/11.jpg)
Single Subject Key Pose Sequence Model
• We represent each key pose by h:
• Denote K key poses of a sequense by H
![Page 12: A Discriminative Key Pose Sequence Model for Recognizing Human Interactions Arash Vahdat, Bo Gao, Mani Ranjbar, and Greg Mori ICCV2011](https://reader034.vdocuments.site/reader034/viewer/2022051412/551c4383550346a5458b4668/html5/thumbnails/12.jpg)
Single Subject Key Pose Sequence Model
• Exemplar Matching Link:
• Compute the pose connection strength to the exemplar poses.
![Page 13: A Discriminative Key Pose Sequence Model for Recognizing Human Interactions Arash Vahdat, Bo Gao, Mani Ranjbar, and Greg Mori ICCV2011](https://reader034.vdocuments.site/reader034/viewer/2022051412/551c4383550346a5458b4668/html5/thumbnails/13.jpg)
Single Subject Key Pose Sequence Model
• Activity-Key Pose Link:
– Compute the sequence similarity in the activity to give a score.
• Direct Root Model:
![Page 14: A Discriminative Key Pose Sequence Model for Recognizing Human Interactions Arash Vahdat, Bo Gao, Mani Ranjbar, and Greg Mori ICCV2011](https://reader034.vdocuments.site/reader034/viewer/2022051412/551c4383550346a5458b4668/html5/thumbnails/14.jpg)
Interaction Key Pose Sequence Model
• Update the model from individual to two people interaction.
• We should recognize who is subject or object.• The scoring function will be:
![Page 15: A Discriminative Key Pose Sequence Model for Recognizing Human Interactions Arash Vahdat, Bo Gao, Mani Ranjbar, and Greg Mori ICCV2011](https://reader034.vdocuments.site/reader034/viewer/2022051412/551c4383550346a5458b4668/html5/thumbnails/15.jpg)
Learning the parameters
• Define the scoring function E(x,y):
• Use multiclass linear SVM classifier to find the best parameters.
![Page 16: A Discriminative Key Pose Sequence Model for Recognizing Human Interactions Arash Vahdat, Bo Gao, Mani Ranjbar, and Greg Mori ICCV2011](https://reader034.vdocuments.site/reader034/viewer/2022051412/551c4383550346a5458b4668/html5/thumbnails/16.jpg)
Experiments
• The UT-Interaction dataset contains videos of 6 classes of human-human interactions.
• Set 1 is with a stationary background,and Set 2 is with slight background movement and camera jitter.
![Page 17: A Discriminative Key Pose Sequence Model for Recognizing Human Interactions Arash Vahdat, Bo Gao, Mani Ranjbar, and Greg Mori ICCV2011](https://reader034.vdocuments.site/reader034/viewer/2022051412/551c4383550346a5458b4668/html5/thumbnails/17.jpg)
Experiments
![Page 18: A Discriminative Key Pose Sequence Model for Recognizing Human Interactions Arash Vahdat, Bo Gao, Mani Ranjbar, and Greg Mori ICCV2011](https://reader034.vdocuments.site/reader034/viewer/2022051412/551c4383550346a5458b4668/html5/thumbnails/18.jpg)
Experiments
![Page 19: A Discriminative Key Pose Sequence Model for Recognizing Human Interactions Arash Vahdat, Bo Gao, Mani Ranjbar, and Greg Mori ICCV2011](https://reader034.vdocuments.site/reader034/viewer/2022051412/551c4383550346a5458b4668/html5/thumbnails/19.jpg)
Experiments
![Page 20: A Discriminative Key Pose Sequence Model for Recognizing Human Interactions Arash Vahdat, Bo Gao, Mani Ranjbar, and Greg Mori ICCV2011](https://reader034.vdocuments.site/reader034/viewer/2022051412/551c4383550346a5458b4668/html5/thumbnails/20.jpg)
Experiments
![Page 21: A Discriminative Key Pose Sequence Model for Recognizing Human Interactions Arash Vahdat, Bo Gao, Mani Ranjbar, and Greg Mori ICCV2011](https://reader034.vdocuments.site/reader034/viewer/2022051412/551c4383550346a5458b4668/html5/thumbnails/21.jpg)
Conclusion
• This paper focuses on the key poses method to recognize the interaction.
• The precision is over 90% and outperform other methods in UT-interaction dataset.