Software Framework For Robot Application Developmenta constraint-based task programming approach

Dominick Vanthienen, TinneDe Laet, Joris De Schutter,

Herman Bruyninckx

Presented by Wilm Decre

Dept. of Mechanical Engineering,KU Leuven

06/05/2013

Introduction Modular Framework Experiments Conclusions

Robot application development

How much work to generate a laser tracing application for twoplatforms? Double as much work as for one?

2 / 26

Introduction Modular Framework Experiments Conclusions

Reuse?

Two things are required for reusability:1. a systematic approach for the specification of tasks2. a modular software framework

3 / 26

Introduction Modular Framework Experiments Conclusions

a systematic approach for task specification

� iTaSC 1

� instantaneous Task Specification and estimation using Constraints(iTaSC)

� constraint-based programming: robot nor task frame centric� auxiliary coordinates to

� express task constraints (feature coordinates), and� model geometric uncertainty (uncertainty coordinates)

� robot application = optimization problem consisting of a set ofconstraints, and one or multiple objective functions.

1J. De Schutter et al. Constraint-based task specification and estimation forsensor-based robot systems in the presence of geometric uncertainty, IJRR 2007.4 / 26

Introduction Modular Framework Experiments Conclusions

Concept

How to create a new iTaSCapplication

� empty Scene

� add Robots and Objects� define Object Frames� add SubTasks

� Virtual Kinematic Chains� Constraint/Controller

� add a Solver

5 / 26

Introduction Modular Framework Experiments Conclusions

Concept

How to create a new iTaSCapplication

� empty Scene� add Robots and Objects

� define Object Frames� add SubTasks

� Virtual Kinematic Chains� Constraint/Controller

� add a Solver

5 / 26

Introduction Modular Framework Experiments Conclusions

Concept

How to create a new iTaSCapplication

� empty Scene� add Robots and Objects� define Object Frames

� add SubTasks

� Virtual Kinematic Chains� Constraint/Controller

� add a Solver

o2b

o1a

o1b

5 / 26

Introduction Modular Framework Experiments Conclusions

Concept

How to create a new iTaSCapplication

� empty Scene� add Robots and Objects� define Object Frames� add SubTasks

� Virtual Kinematic Chains

� Constraint/Controller� add a Solver

i

h

f1bo2b

f2b

g

o1a

o1b

5 / 26

Introduction Modular Framework Experiments Conclusions

Concept

How to create a new iTaSCapplication

� empty Scene� add Robots and Objects� define Object Frames� add SubTasks

� Virtual Kinematic Chains� Constraint/Controller

� add a Solver

i

h

f1bo2b

f2b

g

o1a

o1b

5 / 26

Introduction Modular Framework Experiments Conclusions

Concept

How to create a new iTaSCapplication

� empty Scene� add Robots and Objects� define Object Frames� add SubTasks

� Virtual Kinematic Chains� Constraint/Controller

� add a Solver

i

h

f1bo2b

f2b

g

o1a

o1b

5 / 26

Introduction Modular Framework Experiments Conclusions

Concept

How to create a new iTaSCapplication

� empty Scene� add Robots and Objects� define Object Frames� add SubTasks

� Virtual Kinematic Chains� Constraint/Controller

� add a Solver

i

h

f1bo2b

f2b

gw

o1a

o1b

5 / 26

Introduction Modular Framework Experiments Conclusions

Modular software framework

� follow workflow of iTaSC methodology� division along two ’orthogonal directions’

1. separation of concerns2. hierarchy of three levels

6 / 26

Introduction Modular Framework Experiments Conclusions

Modular software framework

� follow workflow of iTaSC methodology� division along two ’orthogonal directions’

1. separation of concerns following the principle of the 5C’s 2

� Communication� Computation� Coordination� Configuration� Composition

2. hierarchy of three levels

2E. Prassler et al. The use of reuse for designing and manufacturing robots,Robot Standards project, white paper 20096 / 26

Introduction Modular Framework Experiments Conclusions

Modular software framework

� follow workflow of iTaSC methodology� division along two ’orthogonal directions’

1. separation of concerns following the principle of the 5C’s 2

2. hierarchy of three levels2.1 application level = iTaSC level and parts outside the iTaSC formalism2.2 iTaSC level = task level and iTaSC specification elements2.3 task level = functionalities that together form a single task

2E. Prassler et al. The use of reuse for designing and manufacturing robots,Robot Standards project, white paper 20096 / 26

Introduction Modular Framework Experiments Conclusions

Computation

application level components:� Drivers: hardware interfaces� Setpoint generators: desired values for the controllers of iTaSC� iTaSC: iTaSC level

iTaSC level components:� Robots and objects: kinematic and dynamic models of the robots

and objects ⇒ robot coordinates q� Tasks: task level� Solver: optimization problem solver resulting in the desired values

for the low-level robot controllers� Scene: core of the iTaSC composite component

7 / 26

Introduction Modular Framework Experiments Conclusions

Computation

task level components:� Virtual Kinematic Chain (VKC): model of the space between pairs

of object frames, defined on robots and objects ⇒ the state of theVKC, the auxiliary (feature) coordinates χf

� Constraint-Controller (CC): output equation y = f (χf , q) and thecontrol law

8 / 26

Introduction Modular Framework Experiments Conclusions

Computation

Scene Description

Constraint - Controller (CC)

Virtual Kinematic Chain (VKC)

Task

Driver

Hardware

iTaSC

Application

SceneRobot

Object

Solver

Setpoint Generator

reference implementation: Orocos components

9 / 26

Introduction Modular Framework Experiments Conclusions

Coordination

� fixed structure, hierarchical finite-state machine (FSM) for eachlevel element

� FSM’s of all levels form hierarchical, distributed statechart� pure event processors, independent of software component

framework� modelled with rFSM: a light-weight Lua-based statechart model 3

� reference implementation: rFSM models loaded in a SupervisorOrocos component

3M. Klotzbuecher and H. BruyninckxCoordinating robotic tasks and systems with rFSM Statecharts, JSER 201210 / 26

Introduction Modular Framework Experiments Conclusions

Coordination

Configure

Stop Start

Run

Application

iTaSC

Task 1 Task 2

11 / 26

Introduction Modular Framework Experiments Conclusions

Configuration, communication, composition

Configuration� configuration of each of the other concerns� loaded at configuration time� reference implementation: XML format filesCommunication� data and event communication� port structure� reference implementation: Orocos-ROS functionallity (port-topic)Composition� scripts details composition per level� reference implementation: Lua scripting language

12 / 26

Introduction Modular Framework Experiments Conclusions

Laser tracing example

How much work to switch between the following examples?

13 / 26

Introduction Modular Framework Experiments Conclusions

Laser tracing with a KUKA LWR

14 / 26

Introduction Modular Framework Experiments Conclusions

Laser tracing with a PR2

15 / 26

Introduction Modular Framework Experiments Conclusions

Laser tracing with a KUKA youBot

16 / 26

Introduction Modular Framework Experiments Conclusions

Laser tracing with a KUKA LWR: computation

Scene Description

CC_sixDof_pff

VKC_sixDof

laser tracing

lwr_driver

hardware

iTaSC

Application

scene

LWR

table

WDLS_prior_vel_solver

lissajous_generator

CC_cartesian

VKC_cartesian

CC_joint_limit_avoidance

move to reference frame

joint limit avoidance

transX, transY, rotX, rotY, rotZ, transZ

reference_frame

17 / 26

Introduction Modular Framework Experiments Conclusions

Laser tracing with a PR2: computation

Scene Description

CC_sixDof_pff

VKC_sixDof

laser tracing

pr2_driver

hardware

iTaSC

Application

scene

pr2

wall

WDLS_prior_vel_solver

lissajous_generator

CC_cartesian

VKC_cartesian

CC_joint_limit_avoidance

move to reference frame

joint limit avoidance

transX, transY, rotX, rotY, rotZ, transZ

reference_frame

18 / 26

Introduction Modular Framework Experiments Conclusions

Laser tracing with a KUKA youBot: computation

Scene Description

CC_sixDof_pff

VKC_sixDof

laser tracing

youbot_driver

hardware

iTaSC

Application

scene

youbot

ceiling

WDLS_prior_vel_solver

lissajous_generator

CC_cartesian

VKC_cartesian

CC_joint_limit_avoidance

move to reference frame

joint limit avoidance

transX, transY, rotX, rotY, rotZ, transZ

reference_frame

19 / 26

Introduction Modular Framework Experiments Conclusions

Laser tracing: coordinationcomposite_task_fsm.luaMoveToInitial MoveToHomeIdleTraceFigurez_off z_onz_constrainte_toggle_z_constrainte_toggle_robot_weightsrot_on rot_offrot_constraint circle/infinityjoint_weightspenalize_base penalize_arme_start_tracinge_stop_tracinge_cartesian_generator_moveFinished e_trace_circle / e_trace_infinitye_cartesian_generator_moveFinishede_move_to_homee_toggle_rot_constraint20 / 26

Introduction Modular Framework Experiments Conclusions

Laser tracing: coordinationReuse of the same coordination for all three cases

composite_task_fsm.lua

entry

raise_event("e_cartesian_tracing_PoseInputMode")

MoveToHome

raise_event("e_reference_frame_gen_start")

raise_event("e_reference_frame_gen_moveToHome")

exitraise_event("e_reference_frame_gen_stop")raise_event("e_cartesian_tracing_DoNothing")

entry

raise_event("e_cartesian_tracing_PoseInputMode")

MoveToInitial

raise_event("e_reference_frame_gen_start")

raise_event("e_reference_frame_gen_moveToInit")

exitraise_event("e_reference_frame_gen_stop")raise_event("e_cartesian_tracing_DoNothing")

entry

raise_event("e_robot_base_avoidance_Avoid")

raise_event("e_joint_limit_avoidance_Avoid")

entry

Idle

exit

TraceFigure

entry

raise_event("e_laser_tracing_Activate")

raise_event("e_lissajous_generator_start")

exitraise_event("e_lissajous_generator_stop")raise_event("e_laser_tracing_DoNothing")

entry

penalize_arm

raise_event("e_pen_arm")

entry

penalize_base

raise_event("e_pen_base")

joint_weights

entry

z_on

raise_event("e_laser_tracing_zOn")

entry

z_off

raise_event("e_laser_tracing_zOff")

z_constraint

entry

rot_on

raise_event("e_laser_tracing_rotOn")

entry

rot_off

raise_event("e_laser_tracing_rotOff")

rot_constraintcircle/infinity

21 / 26

Introduction Modular Framework Experiments Conclusions

Configuration

reconfiguration needed between the three cases only for� tasks

� possibly retuning of control gains� adapt joint limits for each robot

� robots and objects� adapt configuration to used robot

� setpoint generators� make sure reference frame is within robot workspace

22 / 26

Introduction Modular Framework Experiments Conclusions

Composition: Lua scriptsFor example: the iTaSC level compositionyoubotLoc:fromtab( {M={X_x=1,Y_x=0,Z_x=0,X_y=0,Y_y=1,Z_y=0,X_z=0,Y_z=0,Z_z=1},p={X=0.0,Y=0.0,Z=0.0}} )FixedObjectBaseLoc:fromtab( {M={X_x=1,Y_x=0,Z_x=0,X_y=0,Y_y=1,Z_y=0,X_z=0,Y_z=0,Z_z=1},p={X=0.0,Y=0.0,Z=2.0}} )

addRobot("youbot", youbotLoc)addRobot("FixedObject", FixedObjectBaseLoc)

addObjectFrame("youbot_gripper","gripper_palm_link", "youbot")addObjectFrame("youbot_base","plate_link", "youbot")addObjectFrame("ee_FixedObject","ee", "FixedObject")

addVirtualKinematicChain("VKC_cartesian_motion", "youbot_base", "youbot_gripper")addVirtualKinematicChain("VKC_laser_tracing", "ee_FixedObject", "youbot_gripper")

addConstraintController("CC_cartesian_motion", "", "VKC_cartesian_motion", 1)addConstraintController("CC_laser_tracing", "", "VKC_laser_tracing", 1)addConstraintController("CC_joint_limit_avoidance", "youbot", "", 1)

addSolver("Solver")23 / 26

Introduction Modular Framework Experiments Conclusions

Communication

� non-buffered vs. buffered connections� non-event-driven vs. event-driven ports� Orocos ports and ROS topics

24 / 26

Introduction Modular Framework Experiments Conclusions

ConclusionsiTaSC software framework� modular framework for constraint-based programming� modularity and reusability due to

1. separation of concerns2. division of each of the concerns into small entities within a hierarchy

� reference implementation using Orocos components, rFSMstatecharts, Lua/Orocos scripts, and XML configuration

Future work� formal modeling: Domain Specific Language (DSL) for

constraint-based programming� iTaSC DSL = application, soft-and hardware independent

programming language� further development of a software component framework

independent iTaSC functionality library25 / 26

Introduction Modular Framework Experiments Conclusions

Strategic research directions in robotic softwaredevelopmentMany problems are theoretically solved but almost none on thesoftware level, especially those working at the different levels ofabstraction that are needed in typical advanced robotics tasks.Why?� lack of common agreement on concepts and the resulting lack of

standards� poor separation of concerns: most software is

robot/platform/OS... specificSolutions?� The modelling approach as advocated by BRICS, Proteus,

Smartsoft, ROCK...: agree on models and standardize them, thenmake software implementations

26 / 26