Intuitive and Adaptive Robotics Arm Manipulationusing Leap Motion Controller (LMC)D. Bassily, C. Georgoulas, J. Güttler, T. Linner, T. BockISR/Robotik 2014; 41st International Symposium on Robotics, Munich, Germany

Quan LeHCI Lab

04th, Feb 2015

▧ Introduction▧ Proposed concepts▧ Implementation▧ Research projects▧ Conclusion

Contents

Fig1. The 6-DOF Jaco robotic arm

2

- Populations around the world are rapidly ageing.- Ageing society faces numerous challenges in performing

simple task in Activities of Daily Living (ADLs).

Introduction3

- Caring of elderly and handicapped people becomes more and more important.

- Many technical aids have been developed, but provided limited functionality.

Introduction

Fig3. Mobile arm support (MAS)Fig2. JAECO/Rancho Multilink MAS

Mobile arm supports: evidencebased benefits and criteria for use, Journal of Spinal

Cord Medicine, 31 (2008) 388-393

4

BIG CONCEPT

5

Proposed concepts

Supporting elderly and disabled people in home environment- Reduce the involved operation complexity of some

common human-robot interactions.- The Leap Motion Controller (LMC) is used to operate the

Jaco Robot in intuitive way.- Keyboard or joystick is replaced by LMC.- Mapping user’s real arm and the Jaco arm.- Estimate and filter out the unwanted oscillation.

6

Proposed concepts

Tele-Operation- A method to operate a robot, while still being at a distance

from it.- Dangerous and risky application areas (space exploration,

surveillance, surgery, nuclear plants and underwater operation).

7

Proposed concepts

Tremor Detection- The sub-millimeter accuracy of the

controller could be seen as an opportunity for detecting symptoms related to hand tremor.

- Detect and process the user palm, sent out-put data to robotic arm, filter out unwanted oscillation, enable smooth operations.

- LMC can be efficiently used to validate and quantify the tremor and bradykinesia.

8

Coordinate system

transformation

Mapping Algorithm

Connect to Arduino micro-

controller

Steps of development9

Coordinate system transformation- Relate two systems together by performing some

rotations.

Steps of development

Fig4. Coordinate system transformation

10

Mapping algorithm

Steps of development

Fig5. The mapping algorithm

11

Calibration process- Performs a specific gesture.- Fix hand in 10 ~15 sec.- Threshold values ensure that the hand tremor is not reflect to the robotic arm.

Steps of development

Fig6. The Calibration process

12

Connect to Arduino micro-controller- Enhance the functionality of the system.

Steps of development

Fig7. Grasp and Release operation

13

Overall flow diagram

Steps of development

Fig8. The information flow diagram

14

ALL flat Research

USA2 Projects

Research projects15

- Offers possibilities for elderly people to work at their home environment and thus contribute to the society.

USA2 projects

Fig9. The USA² project demonstrator

Fig10. The schematic diagram of the system

16

- Contribute the quality of living of the ageing society and impaired people, and assist them in maintaining an independent lifestyle in the home environment.

- Implement system in AAL* fat, also to mobility such as wheelchairs, rollators,…

AAL flat research

*Ambient Assisted Living

17

AAL flat researchFig11. The bedroom terminal

Fig13. The kitchen terminal

Fig12. The Lisa Smart Wall

18

- The intuitive manipulation of robotics arm uses a new gesture and LMC tracking system with sub-millimeter accuracy.

- Introduce a simple and straighforward robotics arm minipulation scheme.

- Enable incorporation of robotics system into the home environment.

- LMC technology would be benefit to enhance the independence and autonomy of individuals or montitors and prevents some diseases, such as Parkinson.

Conclusion19

Thanks!

Any questions?