online motion puppetry for non-human...
TRANSCRIPT
![Page 1: Online Motion Puppetry for Non-human Charactersmrl.snu.ac.kr/workshop/Osteology2011/puppet_minji.pdf · 2016-08-24 · prob1 prob2 prob3 prob4 prob5 prob6 prob7 prob8 prob9 prob10](https://reader033.vdocuments.site/reader033/viewer/2022060301/5f083a537e708231d420f7f2/html5/thumbnails/1.jpg)
Online Motion Puppetry for Non-human Characters비인간형 캐릭터에 대한 실시간 조종
서울대학교 대학원컴퓨터 공학부
윤민지
![Page 2: Online Motion Puppetry for Non-human Charactersmrl.snu.ac.kr/workshop/Osteology2011/puppet_minji.pdf · 2016-08-24 · prob1 prob2 prob3 prob4 prob5 prob6 prob7 prob8 prob9 prob10](https://reader033.vdocuments.site/reader033/viewer/2022060301/5f083a537e708231d420f7f2/html5/thumbnails/2.jpg)
Introduction to Online Motion Puppetry Problems
http://www.youtube.com/watch?v=A_iPnC5sgRQ
![Page 3: Online Motion Puppetry for Non-human Charactersmrl.snu.ac.kr/workshop/Osteology2011/puppet_minji.pdf · 2016-08-24 · prob1 prob2 prob3 prob4 prob5 prob6 prob7 prob8 prob9 prob10](https://reader033.vdocuments.site/reader033/viewer/2022060301/5f083a537e708231d420f7f2/html5/thumbnails/3.jpg)
Introduction to Online Motion Puppetry Problems
![Page 4: Online Motion Puppetry for Non-human Charactersmrl.snu.ac.kr/workshop/Osteology2011/puppet_minji.pdf · 2016-08-24 · prob1 prob2 prob3 prob4 prob5 prob6 prob7 prob8 prob9 prob10](https://reader033.vdocuments.site/reader033/viewer/2022060301/5f083a537e708231d420f7f2/html5/thumbnails/4.jpg)
4/ 12
Related Work
Seol et al. SCA 2013
![Page 5: Online Motion Puppetry for Non-human Charactersmrl.snu.ac.kr/workshop/Osteology2011/puppet_minji.pdf · 2016-08-24 · prob1 prob2 prob3 prob4 prob5 prob6 prob7 prob8 prob9 prob10](https://reader033.vdocuments.site/reader033/viewer/2022060301/5f083a537e708231d420f7f2/html5/thumbnails/5.jpg)
Number of DOFs
Absolute position 3 (x, y, z)
Root-coordinate position 3 (x, y, z)
Distance from root joint 1
Displacement from parent joint 3 (x, y, z)
Ground height 1
Joint angle at parent marker 1
Displacement from end effectors 15 (3×5 end effectors)
Distance from end effectors 5 (for 5 end effectors)
Velocity 3 (x, y, z)
Acceleration 3 (x, y, z)
Square of above features 38
Exponential of above features 38
Dot product of two vector features 45
Multiplication of two scalar features 28
(x,y,z)
Related Work
![Page 6: Online Motion Puppetry for Non-human Charactersmrl.snu.ac.kr/workshop/Osteology2011/puppet_minji.pdf · 2016-08-24 · prob1 prob2 prob3 prob4 prob5 prob6 prob7 prob8 prob9 prob10](https://reader033.vdocuments.site/reader033/viewer/2022060301/5f083a537e708231d420f7f2/html5/thumbnails/6.jpg)
6 / 12
Challenges on …
• Limitation of pose to pose mapping
• Characters' Original Motion Style
(ex) Biped VS Quadruped manner
![Page 7: Online Motion Puppetry for Non-human Charactersmrl.snu.ac.kr/workshop/Osteology2011/puppet_minji.pdf · 2016-08-24 · prob1 prob2 prob3 prob4 prob5 prob6 prob7 prob8 prob9 prob10](https://reader033.vdocuments.site/reader033/viewer/2022060301/5f083a537e708231d420f7f2/html5/thumbnails/7.jpg)
7 / 12
Our Approach
Motion Motion Mapping Pose Mapping
: Predefined motions : Undefined motions
![Page 8: Online Motion Puppetry for Non-human Charactersmrl.snu.ac.kr/workshop/Osteology2011/puppet_minji.pdf · 2016-08-24 · prob1 prob2 prob3 prob4 prob5 prob6 prob7 prob8 prob9 prob10](https://reader033.vdocuments.site/reader033/viewer/2022060301/5f083a537e708231d420f7f2/html5/thumbnails/8.jpg)
Motion Mapping
What kind of predefined motions?
From the current motion, what is the most natural
motion?
![Page 9: Online Motion Puppetry for Non-human Charactersmrl.snu.ac.kr/workshop/Osteology2011/puppet_minji.pdf · 2016-08-24 · prob1 prob2 prob3 prob4 prob5 prob6 prob7 prob8 prob9 prob10](https://reader033.vdocuments.site/reader033/viewer/2022060301/5f083a537e708231d420f7f2/html5/thumbnails/9.jpg)
9 / 12
일정차트Motion Mapping
(a) (c)(b)
• Motion Classifier: classify input motion among predefined motions
• Background data for classifying undefined motion
What kind of predefined motions?
![Page 10: Online Motion Puppetry for Non-human Charactersmrl.snu.ac.kr/workshop/Osteology2011/puppet_minji.pdf · 2016-08-24 · prob1 prob2 prob3 prob4 prob5 prob6 prob7 prob8 prob9 prob10](https://reader033.vdocuments.site/reader033/viewer/2022060301/5f083a537e708231d420f7f2/html5/thumbnails/10.jpg)
10 / 12
일정차트Motion Mapping
From the current motion, what is the most natural motion?
![Page 11: Online Motion Puppetry for Non-human Charactersmrl.snu.ac.kr/workshop/Osteology2011/puppet_minji.pdf · 2016-08-24 · prob1 prob2 prob3 prob4 prob5 prob6 prob7 prob8 prob9 prob10](https://reader033.vdocuments.site/reader033/viewer/2022060301/5f083a537e708231d420f7f2/html5/thumbnails/11.jpg)
11 / 12
일정차트Motion Mapping
From the current motion, what is the most natural motion?Motion graph
![Page 12: Online Motion Puppetry for Non-human Charactersmrl.snu.ac.kr/workshop/Osteology2011/puppet_minji.pdf · 2016-08-24 · prob1 prob2 prob3 prob4 prob5 prob6 prob7 prob8 prob9 prob10](https://reader033.vdocuments.site/reader033/viewer/2022060301/5f083a537e708231d420f7f2/html5/thumbnails/12.jpg)
12 / 12
일정차트Motion Mapping
From the current motion, what is the most natural motion?Motion graph + probability
prob1
prob2
prob3
prob4
prob5
prob6
prob7
prob8
prob9
prob10
prob11
prob12
![Page 13: Online Motion Puppetry for Non-human Charactersmrl.snu.ac.kr/workshop/Osteology2011/puppet_minji.pdf · 2016-08-24 · prob1 prob2 prob3 prob4 prob5 prob6 prob7 prob8 prob9 prob10](https://reader033.vdocuments.site/reader033/viewer/2022060301/5f083a537e708231d420f7f2/html5/thumbnails/13.jpg)
13 / 12
일정차트Motion Mapping
From the current motion, what is the most natural motion?
prob1
prob2
prob3
prob4
prob5
prob6
prob7
prob8
prob9
prob10
Current Node
prob12
![Page 14: Online Motion Puppetry for Non-human Charactersmrl.snu.ac.kr/workshop/Osteology2011/puppet_minji.pdf · 2016-08-24 · prob1 prob2 prob3 prob4 prob5 prob6 prob7 prob8 prob9 prob10](https://reader033.vdocuments.site/reader033/viewer/2022060301/5f083a537e708231d420f7f2/html5/thumbnails/14.jpg)
14 / 12
일정차트Motion Mapping
From the current motion, what is the most natural motion?
prob1
prob2
Current Node
prob4
prob5
prob6
prob7
prob8
prob9
prob10
prob12
prob11
![Page 15: Online Motion Puppetry for Non-human Charactersmrl.snu.ac.kr/workshop/Osteology2011/puppet_minji.pdf · 2016-08-24 · prob1 prob2 prob3 prob4 prob5 prob6 prob7 prob8 prob9 prob10](https://reader033.vdocuments.site/reader033/viewer/2022060301/5f083a537e708231d420f7f2/html5/thumbnails/15.jpg)
Motion Mapping
What kind of predefined motions?
Walk!
From the current motion, what is the most natural
motion?Node3 from motion graph
![Page 16: Online Motion Puppetry for Non-human Charactersmrl.snu.ac.kr/workshop/Osteology2011/puppet_minji.pdf · 2016-08-24 · prob1 prob2 prob3 prob4 prob5 prob6 prob7 prob8 prob9 prob10](https://reader033.vdocuments.site/reader033/viewer/2022060301/5f083a537e708231d420f7f2/html5/thumbnails/16.jpg)
Motion Mapping
What kind of predefined motions?
Walk!
From the current motion, what is the most natural
motion?Node3 from motion graph
![Page 17: Online Motion Puppetry for Non-human Charactersmrl.snu.ac.kr/workshop/Osteology2011/puppet_minji.pdf · 2016-08-24 · prob1 prob2 prob3 prob4 prob5 prob6 prob7 prob8 prob9 prob10](https://reader033.vdocuments.site/reader033/viewer/2022060301/5f083a537e708231d420f7f2/html5/thumbnails/17.jpg)
17 / 12
Pose Mapping
Number of DOFs
Absolute position 3 (x, y, z)
Root-coordinate position 3 (x, y, z)
Distance from root joint 1
Displacement from parent joint 3 (x, y, z)
Ground height 1
Joint angle at parent marker 1
Displacement from end effectors 15 (3×5 end effectors)
Distance from end effectors 5 (for 5 end effectors)
Velocity 3 (x, y, z)
Acceleration 3 (x, y, z)
Square of above features 38
Exponential of above features 38
Dot product of two vector features 45
Multiplication of two scalar features 28
(x,y,z)
![Page 18: Online Motion Puppetry for Non-human Charactersmrl.snu.ac.kr/workshop/Osteology2011/puppet_minji.pdf · 2016-08-24 · prob1 prob2 prob3 prob4 prob5 prob6 prob7 prob8 prob9 prob10](https://reader033.vdocuments.site/reader033/viewer/2022060301/5f083a537e708231d420f7f2/html5/thumbnails/18.jpg)
18/ 12
Online Process ..
Input motion
Motion Mapping
Motion classifier & Motion graph
Pose MappingPose mapping
Predefined Motion
Undefined Motion
Final Motion
![Page 19: Online Motion Puppetry for Non-human Charactersmrl.snu.ac.kr/workshop/Osteology2011/puppet_minji.pdf · 2016-08-24 · prob1 prob2 prob3 prob4 prob5 prob6 prob7 prob8 prob9 prob10](https://reader033.vdocuments.site/reader033/viewer/2022060301/5f083a537e708231d420f7f2/html5/thumbnails/19.jpg)
19/ 12
Results
For predefined motions, original motion styles are well maintained.
![Page 20: Online Motion Puppetry for Non-human Charactersmrl.snu.ac.kr/workshop/Osteology2011/puppet_minji.pdf · 2016-08-24 · prob1 prob2 prob3 prob4 prob5 prob6 prob7 prob8 prob9 prob10](https://reader033.vdocuments.site/reader033/viewer/2022060301/5f083a537e708231d420f7f2/html5/thumbnails/20.jpg)
20/ 12
Results
The motion classifier succeeds in distinguishing undefined motions and the pose mappings work well based on input motion.
![Page 21: Online Motion Puppetry for Non-human Charactersmrl.snu.ac.kr/workshop/Osteology2011/puppet_minji.pdf · 2016-08-24 · prob1 prob2 prob3 prob4 prob5 prob6 prob7 prob8 prob9 prob10](https://reader033.vdocuments.site/reader033/viewer/2022060301/5f083a537e708231d420f7f2/html5/thumbnails/21.jpg)
21/ 12
Results
predefined Motion + undefined Motion → Final Motion
![Page 22: Online Motion Puppetry for Non-human Charactersmrl.snu.ac.kr/workshop/Osteology2011/puppet_minji.pdf · 2016-08-24 · prob1 prob2 prob3 prob4 prob5 prob6 prob7 prob8 prob9 prob10](https://reader033.vdocuments.site/reader033/viewer/2022060301/5f083a537e708231d420f7f2/html5/thumbnails/22.jpg)
Question & Answers
![Page 23: Online Motion Puppetry for Non-human Charactersmrl.snu.ac.kr/workshop/Osteology2011/puppet_minji.pdf · 2016-08-24 · prob1 prob2 prob3 prob4 prob5 prob6 prob7 prob8 prob9 prob10](https://reader033.vdocuments.site/reader033/viewer/2022060301/5f083a537e708231d420f7f2/html5/thumbnails/23.jpg)
23/ 12
Discussion & Future Works
• Expansion of motion feature vectors• Support Vector Regression learning for pose mappings