![Page 1: A Framework For Human and Robot-Partner Interactions · Human-Robot A Framework For Human and Robot-Partner Interactions DINH QUANG HUY Interdisciplinary Graduate School Supervisor](https://reader034.vdocuments.site/reader034/viewer/2022050409/5f863a0c74199421b63b4372/html5/thumbnails/1.jpg)
Human-Robot
A Framework For Human and Robot-Partner Interactions
DINH QUANG HUY
Interdisciplinary Graduate School
Supervisor : Prof. Seet Gim Lee, Gerald (MAE)
Co-supervisor : Prof. Nadia Magnenat-Thalmann (IMI)
![Page 2: A Framework For Human and Robot-Partner Interactions · Human-Robot A Framework For Human and Robot-Partner Interactions DINH QUANG HUY Interdisciplinary Graduate School Supervisor](https://reader034.vdocuments.site/reader034/viewer/2022050409/5f863a0c74199421b63b4372/html5/thumbnails/2.jpg)
Human-Robot
Scope of Presentation
• Introduction
• Framework
• Implementation
• Demonstration
• Discussion
• Conclusion
7/1/2016 2
![Page 3: A Framework For Human and Robot-Partner Interactions · Human-Robot A Framework For Human and Robot-Partner Interactions DINH QUANG HUY Interdisciplinary Graduate School Supervisor](https://reader034.vdocuments.site/reader034/viewer/2022050409/5f863a0c74199421b63b4372/html5/thumbnails/3.jpg)
Human-Robot INTRODUCTION
7/1/2016 3
High volume and low-mix manufacturing
Figure 1. The manufacturing of Automobiles
• Tasks are expected to be repeated many hundreds of thousands of times
• Robot actions are explicitly defined and programmed
• Cost and time spent to program and commission each robot would be negligible in this scenario
![Page 4: A Framework For Human and Robot-Partner Interactions · Human-Robot A Framework For Human and Robot-Partner Interactions DINH QUANG HUY Interdisciplinary Graduate School Supervisor](https://reader034.vdocuments.site/reader034/viewer/2022050409/5f863a0c74199421b63b4372/html5/thumbnails/4.jpg)
Human-Robot INTRODUCTION
7/1/2016 4
Low volume and high-mix manufacturing
Figure 2. A high-mix manufacturing task
• Robot programming and the related setup tasks are relatively complex.
• Robot programming is costly and sometimes is impossible.
![Page 5: A Framework For Human and Robot-Partner Interactions · Human-Robot A Framework For Human and Robot-Partner Interactions DINH QUANG HUY Interdisciplinary Graduate School Supervisor](https://reader034.vdocuments.site/reader034/viewer/2022050409/5f863a0c74199421b63b4372/html5/thumbnails/5.jpg)
Human-Robot INTRODUCTION
7/1/2016 5
According to ASSEMBLY's annual Capital Equipment Spending Survey, there's little
doubt that fewer manufacturers are performing high-volume assembly today than there
were 8 years ago
There is a need for a new paradigm
MASTER-SLAVE
INTERACTION
PARTNERSHIP
INTERACTION
![Page 6: A Framework For Human and Robot-Partner Interactions · Human-Robot A Framework For Human and Robot-Partner Interactions DINH QUANG HUY Interdisciplinary Graduate School Supervisor](https://reader034.vdocuments.site/reader034/viewer/2022050409/5f863a0c74199421b63b4372/html5/thumbnails/6.jpg)
Human-Robot INTRODUCTION
Current Forms of Human-Robot Interaction:
1. Anthropomorphism and Direct Interaction
2. Indirect Interaction
7/1/2016 6
Robot
Interaction
• Cursor Tracking
• User detection
• Advanced Recognitions
• User Feedbacks
• …
Localization
• Landmark detection
• Collision avoidance
Mediating Object
GUIs on Projection
Surface
User
Interaction
• Floor-based
• Hand-held device
• Gesture
• Cursor
• …
![Page 7: A Framework For Human and Robot-Partner Interactions · Human-Robot A Framework For Human and Robot-Partner Interactions DINH QUANG HUY Interdisciplinary Graduate School Supervisor](https://reader034.vdocuments.site/reader034/viewer/2022050409/5f863a0c74199421b63b4372/html5/thumbnails/7.jpg)
Human-Robot INTRODUCTION
Two main limitations:
1. Normal Projector is blurry for outdoor environment.
2. Security Problem.
7/1/2016 7
Figure 3. The typical HRI platform with floor projection [1]
![Page 8: A Framework For Human and Robot-Partner Interactions · Human-Robot A Framework For Human and Robot-Partner Interactions DINH QUANG HUY Interdisciplinary Graduate School Supervisor](https://reader034.vdocuments.site/reader034/viewer/2022050409/5f863a0c74199421b63b4372/html5/thumbnails/8.jpg)
Human-Robot PROPOSED PARADIGM
7/1/2016 8
• Robot’s role is elevated to “Approaching But Less Than” that of the human
• Human being is responsible for decisions with the assistance of an intelligent robot
• A user would desire to define less, and require the robot to make appropriate suggestions
• Lasers are a powerful source of light with thin shafts of bright light and colours. Laser beams can provide a dazzling display for industrial applications.
• Augmented Reality can be used as another way of exchanging information between human and robot for security purposes.
![Page 9: A Framework For Human and Robot-Partner Interactions · Human-Robot A Framework For Human and Robot-Partner Interactions DINH QUANG HUY Interdisciplinary Graduate School Supervisor](https://reader034.vdocuments.site/reader034/viewer/2022050409/5f863a0c74199421b63b4372/html5/thumbnails/9.jpg)
Human-Robot FRAMEWORK
7/1/2016 9
Figure 3. A Framework for Dynamic Human-Robot Interactions
![Page 10: A Framework For Human and Robot-Partner Interactions · Human-Robot A Framework For Human and Robot-Partner Interactions DINH QUANG HUY Interdisciplinary Graduate School Supervisor](https://reader034.vdocuments.site/reader034/viewer/2022050409/5f863a0c74199421b63b4372/html5/thumbnails/10.jpg)
Human-Robot IMPLEMENTATION
7/1/2016 10
Laser Projector
USB Camera
Laser Rangefinder
Robot’s Computer
(Ubuntu)
Human Interface Device
Wearable AR Device
Figure 6. Implementation of the Human-Robot Partnership Framework
Framework Implementation
![Page 11: A Framework For Human and Robot-Partner Interactions · Human-Robot A Framework For Human and Robot-Partner Interactions DINH QUANG HUY Interdisciplinary Graduate School Supervisor](https://reader034.vdocuments.site/reader034/viewer/2022050409/5f863a0c74199421b63b4372/html5/thumbnails/11.jpg)
Human-Robot IMPLEMENTATION
7/1/2016 11
Figure 6. Implementation of Multi-modal Handcontroller
Multi-Modal Handheld Device
Display for Laser Rangefinder
Spring-loaded Linear Potentiometers for finger
displacement inputs
IMU for hand motion inputs
Near-infrared Laser Rangefinder
Laser pointer
![Page 12: A Framework For Human and Robot-Partner Interactions · Human-Robot A Framework For Human and Robot-Partner Interactions DINH QUANG HUY Interdisciplinary Graduate School Supervisor](https://reader034.vdocuments.site/reader034/viewer/2022050409/5f863a0c74199421b63b4372/html5/thumbnails/12.jpg)
Human-Robot IMPLEMENTATION
7/1/2016 12
Figure 7. Epson AR glass for Augmented Reality Application
Augmented Reality Glass
![Page 13: A Framework For Human and Robot-Partner Interactions · Human-Robot A Framework For Human and Robot-Partner Interactions DINH QUANG HUY Interdisciplinary Graduate School Supervisor](https://reader034.vdocuments.site/reader034/viewer/2022050409/5f863a0c74199421b63b4372/html5/thumbnails/13.jpg)
Human-Robot IMPLEMENTATION
7/1/2016 13
Figure 8. MAVEN Robot with Lase Projection System
Mobile Robot with Laser Projection
![Page 14: A Framework For Human and Robot-Partner Interactions · Human-Robot A Framework For Human and Robot-Partner Interactions DINH QUANG HUY Interdisciplinary Graduate School Supervisor](https://reader034.vdocuments.site/reader034/viewer/2022050409/5f863a0c74199421b63b4372/html5/thumbnails/14.jpg)
Human-Robot FRAMEWORK
7/1/2016 14
Human and Robot-Partner Interaction
Figure 5. Flowchart for Human-Robot Dialogue
![Page 15: A Framework For Human and Robot-Partner Interactions · Human-Robot A Framework For Human and Robot-Partner Interactions DINH QUANG HUY Interdisciplinary Graduate School Supervisor](https://reader034.vdocuments.site/reader034/viewer/2022050409/5f863a0c74199421b63b4372/html5/thumbnails/15.jpg)
Human-Robot DEMONSTRATION
7/1/2016 15
1. Mobile robot use laser projector for providing visual feedback
Figure 6. Robot is projecting its upcoming actions: stop, move foward, rotate to the left or to the right
![Page 16: A Framework For Human and Robot-Partner Interactions · Human-Robot A Framework For Human and Robot-Partner Interactions DINH QUANG HUY Interdisciplinary Graduate School Supervisor](https://reader034.vdocuments.site/reader034/viewer/2022050409/5f863a0c74199421b63b4372/html5/thumbnails/16.jpg)
Human-Robot DEMONSTRATION
7/1/2016 16
1. Mobile robot use laser projector for providing visual feedback
Figure 5. Flowchart for Human-Robot Dialogue
VIDEO
![Page 17: A Framework For Human and Robot-Partner Interactions · Human-Robot A Framework For Human and Robot-Partner Interactions DINH QUANG HUY Interdisciplinary Graduate School Supervisor](https://reader034.vdocuments.site/reader034/viewer/2022050409/5f863a0c74199421b63b4372/html5/thumbnails/17.jpg)
Human-Robot DEMONSTRATION
7/1/2016 17
2. Robot control with Augmented Reality System
Figure 7. Robot status displaying and robot selection using AR system with handcontroller
![Page 18: A Framework For Human and Robot-Partner Interactions · Human-Robot A Framework For Human and Robot-Partner Interactions DINH QUANG HUY Interdisciplinary Graduate School Supervisor](https://reader034.vdocuments.site/reader034/viewer/2022050409/5f863a0c74199421b63b4372/html5/thumbnails/18.jpg)
Human-Robot DEMONSTRATION
7/1/2016 18
2. Robot control with Augmented Reality System and
VIDEO
![Page 19: A Framework For Human and Robot-Partner Interactions · Human-Robot A Framework For Human and Robot-Partner Interactions DINH QUANG HUY Interdisciplinary Graduate School Supervisor](https://reader034.vdocuments.site/reader034/viewer/2022050409/5f863a0c74199421b63b4372/html5/thumbnails/19.jpg)
Human-Robot DISCUSSION
7/1/2016 19
The framework promises two significant benefits pertaining to user experience and human workload with the identification of solutions to robot deployment:
• The task of “programming” is simplified by the process of selecting options, presented by the robot.
• The natural selection through pointing and reaffirming through pushbutton selection is natural and does not require “extensive” operator training.
The combination of see-through LCD display and high-speed laser projection help us not only to overcome the limitation of projection in a bright environment but also serve the purpose of cooperating and sharing tasks with other operator in the same working scenario.
![Page 20: A Framework For Human and Robot-Partner Interactions · Human-Robot A Framework For Human and Robot-Partner Interactions DINH QUANG HUY Interdisciplinary Graduate School Supervisor](https://reader034.vdocuments.site/reader034/viewer/2022050409/5f863a0c74199421b63b4372/html5/thumbnails/20.jpg)
Human-Robot CONCLUSION
7/1/2016 20
A new paradigm was proposed, which promises to reduce the task of deploying a robot under a “low-volume, high-mix” scenario. This is viewed as a more acceptable alternative to that of programming human identified routes at a fixed computer terminal, using a mouse and keyboard
A demonstration of an intuitive dialogue between human and robot using gestures and laser selection is presented via augmented reality. This allows the deployment away from the traditional constraints and promises a faster “training” cycle with its human centric approach
![Page 21: A Framework For Human and Robot-Partner Interactions · Human-Robot A Framework For Human and Robot-Partner Interactions DINH QUANG HUY Interdisciplinary Graduate School Supervisor](https://reader034.vdocuments.site/reader034/viewer/2022050409/5f863a0c74199421b63b4372/html5/thumbnails/21.jpg)
Human-Robot
THANK YOU FOR LISTENING
7/1/2016 21
![Page 22: A Framework For Human and Robot-Partner Interactions · Human-Robot A Framework For Human and Robot-Partner Interactions DINH QUANG HUY Interdisciplinary Graduate School Supervisor](https://reader034.vdocuments.site/reader034/viewer/2022050409/5f863a0c74199421b63b4372/html5/thumbnails/22.jpg)
Human-Robot Reference
7/1/2016 22