1

High Performance Computing symposium High Performance Computing symposium

HPC2000, April 16-20, 2000. Washington, D.C. HPC2000, April 16-20, 2000. Washington, D.C.

Samir OtmaneSamir Otmane E-mail : otmane@cemif.univ-evry.fr E-mail : otmane@cemif.univ-evry.fr

CEMIF, Complex System LaboratoryCEMIF, Complex System Laboratory Http : http://lsc.cemif.univ-evry.fr:8080/~otmane Http : http://lsc.cemif.univ-evry.fr:8080/~otmane 40 Rue du Pelvoux 91020 Evry, France40 Rue du Pelvoux 91020 Evry, France Tél Tél : 01/69/47/75/04 Fax : 01/69/47/75/99 : 01/69/47/75/04 Fax : 01/69/47/75/99

A R I T IA R I T IAAugmented ugmented RReality eality IInterface nterface for for TTelerobotic applications elerobotic applications

via via IInternetnternet

2

ContentsContents

IntroductionIntroduction Human Computer Interaction in Tele-robotic Human Computer Interaction in Tele-robotic

environment environment

• Interaction with local and remote task environmentInteraction with local and remote task environment Interactions with ARITIInteractions with ARITI System description :System description :

• Robot slave - Task board - Virtual fixture boardRobot slave - Task board - Virtual fixture board

• Hardware and communication boardHardware and communication board Experiments and ResultsExperiments and Results Conclusion and perspectivesConclusion and perspectives

3

IntroductionIntroduction- Tele-work -- Tele-work -

Master site Master site Communication supportCommunication support Slave site Slave site• The slave site is distant from the master site.The slave site is distant from the master site.• Information feedback is corrupted by a bandwidth Information feedback is corrupted by a bandwidth

limitation of communication support .limitation of communication support .• Time delay is not constant when using any Time delay is not constant when using any

communication network.communication network.• No portable and user-friendly Tele-work systems.No portable and user-friendly Tele-work systems.• Human performances are decreased during direct control Human performances are decreased during direct control

of remote Tele-manipulation task.of remote Tele-manipulation task.

Sending Sending ordersorders

Information Information feedbackfeedback

4

VisualVisual

• Interaction with a local environment and completion of task Interaction with a local environment and completion of task

through the use of a through the use of a Robotic InterfaceRobotic Interface . .

Interaction between Human Interaction between Human and and locallocal Task Environment Task Environment

VisualVisualAuditory Auditory

Tactile Tactile

Robotic Robotic SystemSystem

System System I / OI / O

HardwareHardware

Robotic Robotic InterfaceInterface

Task EnvironmentTask Environment

PerceptualPerceptual

CognitionCognition

MotorMotor

Human Human ProcessorProcessor

5

Interaction between Human Interaction between Human and and remoteremote Task Environment Task Environment

During interaction control of a remote robotic terminal tool, the user mustDuring interaction control of a remote robotic terminal tool, the user must

• Perform a physical action to initiate motion from the robot,Perform a physical action to initiate motion from the robot,

• Wait for the system to respond,Wait for the system to respond,

• Perceive the physical effect onto the robot and task environment,Perceive the physical effect onto the robot and task environment,

• Decide what to do next,Decide what to do next,

• Repeat the cycle until the task is completed.Repeat the cycle until the task is completed. Motor activity is initiated through Motor activity is initiated through interactioninteraction with a software interface via with a software interface via

Keyboard and mouse, joystick, master arm, etc...Keyboard and mouse, joystick, master arm, etc...

HHAARRDD

SSOOFFTT

Human Human OperatorOperator

Robotic InterfaceRobotic Interface Remote Robotic EnvironmentRemote Robotic Environment

NN

EE

TT

WW

6

Interactions withInteractions withA R I T IA R I T I

Three kinds of visual assistance are given to human Three kinds of visual assistance are given to human operator for operator for friendly human computer interactionfriendly human computer interaction using the using the ARITIARITI interface. These visual helps are devoted to : interface. These visual helps are devoted to :

Environment perceptionEnvironment perception Robot controlRobot control Robot supervisionRobot supervision

PerceptionPerception

ControlControl

SupervisionSupervision

Human Human OperatorOperator

HHAARRDD

NN

EE

TT

WW

Remote Remote EnvironmentEnvironment

Robotic InterfaceRobotic Interface

7

Assistance for Assistance for EnvironmentEnvironment PerceptionPerception

Several Virtual view pointsSeveral Virtual view points

++

Video image feedbackVideo image feedback

Perception modulePerception module

Human Human OperatorOperator

In Control moduleIn Control module

8

Assistance for Assistance for Robot ControlRobot Control

Virtual robotVirtual robot

++

Virtual FixturesVirtual Fixtures

Control moduleControl module

Human Human OperatorOperator

In Supervision moduleIn Supervision module

9

Assistance for Assistance for RobotRobot SupervisionSupervision

Textual information of the Textual information of the current task current task

++Overlaid Model / ImageOverlaid Model / Image

Supervision moduleSupervision module

Human Human OperatorOperator

10

System descriptionSystem descriptionRobot slaveRobot slave

The CEMIF experimental site is a mechanism with four DOF The CEMIF experimental site is a mechanism with four DOF (degrees of freedom)(degrees of freedom)• 2 Translations parallel to the ground2 Translations parallel to the ground• 2 Rotations (Site and Azimuth)2 Rotations (Site and Azimuth)

Peg Peg mounted mounted

on a turreton a turret

11

System descriptionSystem descriptionTask BoardTask Board

The robot is assumed to assemble (place) and disassemble The robot is assumed to assemble (place) and disassemble (pick) objects hanging on a metal stand(pick) objects hanging on a metal stand

ObjectsObjects

Metal Metal standstand

Head of the Head of the robot pegrobot peg

TargetTarget

12

System descriptionSystem description Virtual Fixtures BoardVirtual Fixtures Board

Human operator can create and use virtual fixtures to Human operator can create and use virtual fixtures to control the robot very easier.control the robot very easier.

Some examples of simple Virtual Fixtures (VF) :Some examples of simple Virtual Fixtures (VF) :

DiscDisc SphereSphere PlanPlan

Super-ellipsoid Super-ellipsoid ConeCone

CylinderCylinder PipePipeCube or Cube or SquareSquare

13

System descriptionSystem description Virtual Fixtures StructureVirtual Fixtures Structure

Name Identify the fixture

Type Simple or complex and active or passive. If complex, itcontains the different links to the combined fixtures

Referential Contains position and orientation of the fixture on X, Yand Z axis

Attachment Static or dynamic and contains the coordinates of thevirtual point or object where fixture will be attached

Effect zone Contains the equation of the volumetric form, surface orother any known geometric shape may be associated tothe VF

Pre-condition Contains the activation condition of fixture

Function Contains a set of actions to be performed inside thevirtual guide either by the robot, by the HO or by bothactors

Post-condition Contains the inactivation condition of fixture

14

System descriptionSystem description Hardware Hardware

ARITI system has been implemented on a PC Pentium 233 Mhz ARITI system has been implemented on a PC Pentium 233 Mhz with a 128 Mo RAM.with a 128 Mo RAM.

The PC is equipped with a Matrox Meteor video acquisition card The PC is equipped with a Matrox Meteor video acquisition card connected to a black and white camera.connected to a black and white camera.

The orders are sent via the RS232 serial link.The orders are sent via the RS232 serial link.

RS232 serial linkRS232 serial link

OrdersOrders Video acquisitionVideo acquisition

15

System descriptionSystem description SoftwareSoftware

ARITI system is implemented under LINUX operating system.ARITI system is implemented under LINUX operating system. ARITI interface is written based on JAVA object programming ARITI interface is written based on JAVA object programming

language language Video server is written using the C standard language.Video server is written using the C standard language. Control server is written using the C and ASM (Microprocessor Control server is written using the C and ASM (Microprocessor

Assembly Language )Assembly Language )

Video ClientVideo Client

Control ClientControl Client

-ARITI- -ARITI- INTERFACEINTERFACE

Applet JAVAApplet JAVA

L I N U X - O S -L I N U X - O S -

Video serverVideo server

Control serverControl server

C and ASMC and ASM

SS

OO

KK

EE

TT

16

To use the To use the ARITI ARITI system system

http://lsc.cemif.univ-evry.fr:8080/Projets/ARITIhttp://lsc.cemif.univ-evry.fr:8080/Projets/ARITI

A R I T I A R I T I SystemSystem

CameraCameraRobotRobot

WWW CLIENTS + Internet BrowserWWW CLIENTS + Internet Browser

17

The The ARITIARITI Display Display

http://lsc.cemif.univ-evry.fr:8080/Projets/ARITIhttp://lsc.cemif.univ-evry.fr:8080/Projets/ARITI

18

Experiments Experiments

Pick and place taskPick and place taskTele-operation modeTele-operation mode Control the real robot via the virtual robotControl the real robot via the virtual robot10 human operators (HO)10 human operators (HO)3 kinds of test3 kinds of test

Without Virtual FixturesWithout Virtual Fixtures With With passivepassive Virtual FixturesVirtual Fixtures With With active (attractive)active (attractive) Virtual Fixtures Virtual Fixtures

Each HO makes 10 tests for each kindEach HO makes 10 tests for each kind

19

ResultsResultsReach a cylinder N° 1Reach a cylinder N° 1

Reach a 3D target point on the peripheral of the cylinder N°1Reach a 3D target point on the peripheral of the cylinder N°1 Without virtual fixtures there is Without virtual fixtures there is 1,491,49 collision for each test collision for each test

20

ResultsResultsReach a cylinder N° 1Reach a cylinder N° 1

Imprecision errors on X, Y, Z axisImprecision errors on X, Y, Z axis

Err < 0,25 mm Err < 0,25 mm with active VFswith active VFs

Average timeAverage time

7,7 sec7,7 sec with active VFs with active VFs

21

ResultsResults Pick and place a cylinder N° 1Pick and place a cylinder N° 1

- Blue - with passive VFs.- Blue - with passive VFs. - Red - with active (attractive potential fields) VFs.- Red - with active (attractive potential fields) VFs.

Unhook a cylinder N° 1 Unhook a cylinder N° 1 Passive VFs - Passive VFs - 12,78 sec12,78 sec Active VFs - Active VFs - 9,5 sec9,5 sec

Place a cylinder into the stand, Place a cylinder into the stand, Passive VFs - Passive VFs - 37,96 sec37,96 sec

Active VFs - Active VFs - 7,86 sec7,86 sec

22

23

24

25

ConclusionConclusion

Virtual realityVirtual reality and and Augmented RealityAugmented Reality technologies are technologies are used to :used to : Overcome the instability of time delay,Overcome the instability of time delay, Complete or compensate the information feedback (video Complete or compensate the information feedback (video

feedback for instance)feedback for instance) Increase the human operator performancesIncrease the human operator performances Thanks to Virtual Fixtures :Thanks to Virtual Fixtures :

best accuracy < 0,25 mmbest accuracy < 0,25 mm best completion timebest completion time best safetybest safety

JAVA programming Language is used to implement the JAVA programming Language is used to implement the Man Machine Interface of Man Machine Interface of ARITIARITI system to : system to : Give a Give a portable system and portable system and

User-friendly Tele-work systemUser-friendly Tele-work system

26

PerspectivesPerspectives

Use the Use the ARITI ARITI system for mobile robot application, such assystem for mobile robot application, such as• Navigation, obstacles avoidance, to assist disable person.Navigation, obstacles avoidance, to assist disable person.

27

PerspectivesPerspectives

Add Audio and Tactile feedback modulesAdd Audio and Tactile feedback modules Extend ARITI system to do a cooperative Tele-WorkExtend ARITI system to do a cooperative Tele-Work

Extended Extended A R I T I A R I T I SystemSystem

User 1User 1

User 2User 2

User nUser n

..

..

..

Robot 1Robot 1

Robot 2Robot 2

Robot mRobot m

..

..

..

N N E E TT