the lamprey robot

Future and Emerging TechnologiesFuture and Emerging

Technologies

Biorobotics Science and Engineering: from Bio-Inspiration to Bio-

Paolo Dario

Scuola Superiore Sant’Anna, Pisa, Italy

Inspiration to Bio-Application

Outline of the talk

� Introduction to Biorobotics and Biomechatronics

� Biorobotics Science

� Building robots to investigate humans and animaland animal

� Biorobotics Engineering

� Robotics for surgery and endoscopy

� Robotics in rehabilitation and assistance

� Conclusions

Biorobotics and biomechatronic design

Engineering analysis Engineering analysis and modelingand modeling

Biological systemBiological system

Validation

Development of a physical model

Bio-mimetic Bio-mimetic robot

Bio-inspired Bio-inspired robot

ApplicationsApplications

Development of a biomedical robot

Biorobotics Science and Engineering

Biorobotics Science: using robotics to discover…

Biorobotics Engineering: using Biorobotics Engineering: using robotics to invent…

Outline of the talk

� Introduction to Biorobotics and biomechatronics






� Conclusions

I

O

I

O

I

O

I

O

HYPOTHESIS AND MODEL

Biorobotics Science

PHENOMENON TO BE

Gripforce

Loadforce

Movement

Gripforce

Loadforce

Movement

TO BE EXPLAINED

Biorobotics vs. simulation and animal models

Human modelHuman model

InteractionModel of Model of

interactioninteraction

World modelWorld modelWorldWorld

I

O

I

O

I

O

I

O

HYPOTHESIS AND MODEL

Biorobotics Science

PHENOMENON TO BE

IMPLEMENTATION IN A ROBOT

Gripforce

Loadforce

Movement

Gripforce

Loadforce

Movement

Validation of Validation of the modelthe model

EXPERIMENTComparison between robot and biological system performance

TO BE EXPLAINED

A robotic platform for validating a model of development of sensory-motor grasp control

Objectives:

� To increase knowledge of brain connectivity (architecture) and brain activity (functioning) concerning sensory motor coordination for object manipulation in children

To integrate an � To integrate an anthropomorphic robotic platform for graspingand manipulation to validate a neurophysiological model of the five learning phases of visuo-tactile-motor coordination in infants

PALOMA EU IST-FET Project IST-2001-33073

P. Dario, M.C. Carrozza, E. Guglielmelli, C. Laschi, A. Menciassi, S. Micera, F. Vecchi, “Robotics as a “Future and Emerging Technology: biomimetics, cybernetics and neuro-robotics in European projects”, IEEE RAM, Vol.12, No.2, June 2005, pp.29-43.

HYPOTHESIS AND

Biorobotics Science

IMPLEMENTATION

PHENOMENON TO BE

EXPLAINED(combination of HYPOTHESIS AND

MODEL (comparison between numerical results and

interpolated experimental data of living oligochaeta)

Validation of Validation of the modelthe model

EXPERIMENTComparison between robot and biological system performance

IMPLEMENTATION IN A ROBOT

A. Menciassi and P. Dario, Philos. Transact. Roy. Soc. A Math. Phys. Eng. , 2003

D. Accoto, P. Castrataro, P. Dario, J. Theor. Biology, 2004

(combination of friction and segment number for effective

locomotion)

The Scuola Superiore Sant’Anna “Zoo”

Oligochaeta Role of friction in locomotion Endoscopy of GI tract

Legged insects Modeling compliant substrates Endoscopy of GI tract

PolychaetaNew computational models of locomotion kinematics

Rescue, field robotics

Biological model Scientific problem Engineering application

locomotion kinematics

Swimming cells Swimming at low Re numbers Neuroendoscopy

Cricket Scale effects on locomotion Mobile sensor networks

LampreyNeuroscientific models of goal-driven locomotion

River exploration, new robots (soft bodied)

Octopus Motor performance of hydrostatic muscular limbs

Infinite DOF robots, rescue, marine

Plant roots Soil penetration mechanisms Environmental robotics

Mouse Animal-robot interaction Entertainment, …

Lamprey & Salamander-like robots

Scientific Problem addressed according to the EMBODIED INTELLIGENCE paradigm:

to investigate neuroscientific open issues on locomotion by means of physical on locomotion by means of physical platforms on which to implement

theoretical models of neural mechanisms

Phylogenetic tree

From aquatic to terrestrial locomotion

Salamander

INT

RO

DU

CT

ION

Lamprey

530

360Ichtyostega

INT

RO

DU

CT

ION

INTRODUCTION

Axial CPG

Forelimb CPG

Hindlimb CPG

Working hypothesis

Hindlimb CPG

Salamander CPG = Lamprey-like axial CPGaxial CPG

extended with 2 limb CPGslimb CPGs

Evolution of spinal locomotor CPG for locomotion

Lamprey Salamander Cat Human

FORWARD SWIMMING(« lamprey like »)

Salamander in anguilliform (lamprey-like) swimming

Ijspeert et coll., 2007

• Traveling waves of lateral

displacement passing downthe body.

FORWARD STEPPING(« crocodile like »)

Salamander in stepping locomotion.

Ijspeert et coll., 2007

• Standing waves of lateral

displacement with fixed nodesat pectoral and pelvic girdles.

LAMPETRA Expected Results

Advances in neuroscienceBetter models of goal directed locomotion, and in particular of:

• mechanisms addressing striatum/basal ganglia in the selection betweendifferent patterns of behaviors based on visual input, other senses andprevious experience;

• motivational control as in the case of hunger, aggression, sexual partner

Advances in ICT technology

• motivational control as in the case of hunger, aggression, sexual partnerselection, day/night cycle.

• Control: rethinking traditional control by exploiting interacting layers ofdifferent behaviours instead of adopting a more traditional approach ofmodelling and planning, allowing to control complex systems (thousandsof receptors, hundreds of actuators, multimodal sensory inputs).

• Hardware: New technologies for actuators, sensors and materialsenabling soft-bodied robotics.

forward backward

lag

Neural activity

Accurate

simulation but affected by

intrinsic simplifying

… need for an

embodied study for more accurate

investigation

Virtual models of the Lamprey

simplifying

hypotheses…

O. Ekeberg and S. Grillner.

Salamander-like robots

To verify models of the transition from aquatic to terrestrial locomotion in vertebrate evolution

An amphibious salamander robot

demonstrates how a primitive neural circuit for swimming can be extended by

phylogenetically more recent limb

oscillatory centers to explain the ability

of salamanders to switch between swimming and walking

Ijspeert, Cabelguen et al. “From swimming to walking with a salamander robot driven by

a spinal cord model” Science, 9 March 2007.

The Lamprey robot(skeletal system)

(Full, Cutkosky, et al., 2002)

The Lamprey robot

(first experiments)

Not stabilized head

Not progressive wave

The Model of Collaboration between Neuroscience and Robotics in Lampetra

NEURO-

SCIENCE

MODELS

NEW SCIENCE

ARTEFACTS

(existing early

prototypes)

(Newly designed)

HYBRID BIONIC

SYSTEMS

NEW TECHNOLOGY

Joint SSSA-KI investigation

Schematic: obstacle avoidance by steering and recovery of the original direction Involved forebrain structures

PALLIUMPALLIUMSTRIATUMSTRIATUM

PALLIDUMPALLIDUM

THALAMUSTHALAMUS TECTUM

Lamprey - Task no. 1: Obstacle Avoidance

Activation sequence: retina; thalamus; pallium and striatum; pallidum;

tectum-eye mvt.; tectum-body orienting mvt.; MLR-DLR.

RETINARETINA MLRMLR

DLRDLR

Locomotion

Eye mvt.

Body orienting

mvt.

Experimental artefact

Early (neuro-robotics) co-design

25 segments, wireless,

on board processing, 1 hour autonomy

Stretch receptors + vision + vestibular

sensors

Joint SSSA-KI investigation

Schematic: pursuit of a movable target

Lamprey - Task no. 2: Target Pursuing

Involved forebrain structures

PALLIUMPALLIUMSTRIATUMSTRIATUM

PALLIDUMPALLIDUM

THALAMUSTHALAMUS TECTUM

Activation sequence: retina; thalamus; pallium and striatum; pallidum;

tectum-eye mvt.; tectum-body orienting mvt.; MLR-DLR.

RETINARETINA MLRMLR

DLRDLR

Locomotion

Eye mvt.

Body orienting

mvt.

Experimental artefact

Early (neuro-robotics) co-design

25 segments, wireless,

on board processing, 1 hour autonomy

Stretch receptors + vision + vestibular

sensors

The Scuola Superiore Sant’Anna “Zoo”

Oligochaeta Role of friction in locomotion Endoscopy of GI tract

Legged insects Modeling compliant substrates Endoscopy of GI tract

PolychaetaNew computational models of locomotion kinematics

Rescue, field robotics

Biological model Scientific problem Engineering application

locomotion kinematics

Swimming cells Swimming at low Re numbers Neuroendoscopy

Cricket Scale effects on locomotion Mobile sensor networks

LampreyNeuroscientific models of goal-driven locomotion

River exploration, new robots (soft bodied)

Octopus Motor performance of hydrostatic muscular limbs

Infinite DOF robots, rescue, marine

Plant roots Soil penetration mechanisms Environmental robotics

Mouse Animal-robot interaction Entertainment, …

Novel Design Principles and Technologies for a New Generation of High Dexterity Soft-bodied Robots Inspired by the Morphology andBehaviour of the Octopus

OCTOPUS (2008-2012)

The new OCTOPUS Project has the objective of designing and developingan 8-arm robot inspired to the muscular structure, neurophysiology andmotor capabilities of the octopus (Octopus vulgaris).

BIOMIMETIC ROBOTICS

The ANGELS Project

ANGuilliform robot with ELectric Sense

Outline of the talk







� Conclusions

The Evolution of SurgeryTRADITIONAL SURGERY

MINIMALLY INVASIVE SURGERY

ENDOLUMINAL SURGERY

Micro-endoscope for spinal cord

FETAL Da Vinci CAS system

Endoscopic capsuleReconfigurable surgical system

FETAL SURGERY

Force-feedback scissor for fetal surgery

CELL SURGERY

Artificial virus for cell therapy

Early Detection of Colon Cancer Saves Life. Colonoscopy is the Gold Standard. But…

• Pain and disconfort for the patient

• Complex and demanding procedure for the doctor

• The active part of the colonoscope is the head, that incorporates the visualization system (optical fibers or camera, optics, illumination)camera, optics, illumination)

• The head must be inserted along the colon by maneuvering and pushing, from outside the body, a relatively stiff shaft

• These actions stretch the colon and originate pain

The case of endoscopic tools for gastrointestinal analysis

Problem: pain,

difficult

maneuverability…

…imitating the worm?

Painless Colonoscopy

Semi-autonomous

inchworm-like

locomotion

SOFT TAIL

CLAMPER

FLEXIBLE

BODY

BENDING

SECTION

CMOS

CAMERA

AND

LIGHT

SOURCE

The E-WORM

Painless

Colonoscopy

System

From “wired”painless colonoscopy to “wireless” GI endoscopy

The case of endoscopic tools for gastrointestinal analysis

Problem: pain,

difficult

maneuverability…

Solution:

inchworm

locomotion,

Problem: slow,

not adequate

for different

gut

diameters…

Solution: legged

locomotion, insect-

like capsular

endoscopy

…like a worm in the gut…

locomotion,

self-

adaptability

endoscopy

Wireless endoscopic capsules with activelocomotion system for the entire GI tract

Single capsule approach: swimming locomotion

Ingestion of liquid in context with the examination allows to obtain organ distension,

thus making possible a low power 3D locomotion in the stomach

Fine control of steering and speedFine control of position

Single capsule approach: swimming locomotion

Ex vivo test in stomach of pig filled with water

Fine control of steering and speedFine control of position as regards the water level

Single capsule approach: legged capsule for tubular organs

Obtaining an active locomotion in

tubular organs of the GI tract, that

cannot be inflated or filled with water,

means having propulsion mechanisms

able to open and distend the tissue

around the capsule.

1. Diameter: 11.1 mm;

2. Length: 28 mm (+camera);

3. 12 legs;

4. 2 DC brushless motors (NAMIKI);

5. Force at the leg’s tip of about 1N;

6. No frontal latex balloon required;

7. On board electronics drivers;

M. Quirini et al., ICRA 2007

7. On board electronics drivers;

8. Power consumption: 0.66 W. Stefanini et al. Int. J. Of Rob. Res.,

2006.

Patent filed

Single capsule approach: legged capsule for tubular organs

Free motion

Control panel

M. Quirini, S. Scapellato, A. Menciassi, P. Dario, F. Rieber, C.-N. Ho, S. Schostek, M.O. Schurr, “Feasibility proof of a legged locomotion capsule for the GI tract”, GASTROINTESTINAL ENDOSCOPY Vol. 67, No. 7, 2008

Simulator test

Colon test (5 cm/min)

AAssembling ssembling RReconfigurable econfigurable EEndoluminal ndoluminal

SSurgical system (ARESurgical system (ARES--FP6FP6--NEST)NEST)

Target

area

Operation

THE VISION:

Performing

complex

internal surgery by tiny

robots that

assemble within the body,

perform tasks, and are then removed

ARAKNES - Array of Robots Augmenting

the KiNematics of Endoluminal Surgery

The ultimate goal: to integrate the advantages of traditional open surgery, laparoscopic surgery (MIS), and robotics surgery into a deeply innovative

system for bi-manual, ambulatory, tethered, visible scarless surgery, based on an array of smart microrobotic instrumentation

Main intended interventions:

Endoluminal and transluminal surgery (bariatric surgery, local excision, others)Single-port laparoscopy

ARAKNES - Array of Robots Augmenting

the KiNematics of Endoluminal SurgeryOPERATIVE TOOLASSISTIVE TOOL

Control board Li-Po battery Module

Prototyping of the Basic ModulesPrototyping of the Basic Modules

Control board Li-Po battery Module

Motor (4 x 17.4 mm) 1.03g, 10.6 mNm

camera

Biopsy

forceps

Tissue

Storage

12 Modules

-Camera X1

-Forceps X1

-Storage X1

-Central X1

-Structural X8

Example of a multi-module robot

integrating a grasping tool

The ARAKNES Project has received funding from the European Community's Seventh Framework Programme (FP7/2007-2013) under grant agreement num. 224565.

Storage -Structural X8

Once the CNT (naturally magnetic thanks to residuals) are attached or internalized, cells can be concentrated in a desired compartment for subsequent localized therapy.

Magnet

Metastatic

TumorCNT

Cell

CNT can be functionalized to bind target

Cell manipulation with magnetic carbon nanotubes

Metastatic cells bound with CNTs

CNT can be functionalized to bind target cells (such as metastatic cells) or to be internalized by the cells; in this sense cells become magnetotactic and can be drag and collected by a permanent magnet.

Human Neuroblastoma cells (SH-SY5Y)displacement after 3 days in culture withMWNTs-modified medium. Control samplenot showed (with Nikon TE2000U invertedoptical microscope, magnifications 20x).

V. Pensabene, O. Vittorio, S. Raffa, A. Menciassi, P. Dario, “Neuroblastoma cells displacement by magnetic carbon nanotubes”, IEEE Trans. On NanoBioScience, 2008.

Outline of the talk







� Conclusions

Stimulation of

the sensory nerves to

provides

sensory

A “cybernetic”

prosthesis

controlled by

the brain

Extraction of

brain commands

from the motor nerves

sensory

feedback

neuralProsthetic hands and neural

The case of the cybernetic hand prosthesis

Neuroscience

Scientific and

technological results

FP5 & FP6

Prosthetic hands and neural

interfaces

CyberHand:

a robotic hand and neural

peripheral interfaces

implanted in a first human

patient (2008)

CYBERHAND

(FP5, 2002-2005)

Cybernetic

prosthetic hand

Life-like

perception

systems

NEUROBOTICS IP

(FP6, 2004-2008)

Hybrid Bionic

Systems

Beyond

Robotics

Mechanisms Sensors

External World Bio

mech

atro

nic

Mechatronic System

Tele

co

mm

un

icatio

n

“User/Patient”

“Mechatronic” (integrated)

Our philosophy: Human-Centred Design

for devices, systems and services

Power Supply

Actuators Embedded Control

Human-Machine Interface

Human User

Bio

mech

atro

nic

Syste

m In

teg

ratio

n

Tele

co

mm

un

icatio

n

(integrated) platform

“User/Therapist”

CYBERHAND Project: Development of a CYBERnetic HAND prosthesis

Electrodes for Recording and Stimulation in the PNS

Sieve ElectrodeIntegrated Electronics for Active Sieve Electrode

Sieve silicon electrode

Sieve Head with Counter Electrodes

Shaft Electrode

Platinum Electrodes on the Shaft Tripolar Cuff Electrodes

LIFE Electrodes

Information Society Technologies

SSSA prosthetic hands

MECHATRONIC HANDS FOR

CYBERHAND robotic hand• A robust stand-alone robotic hand provided with artificial sensors

• It will be implantanted in Rome• 5 fingers, 6 motors, 16 dof, 11 sensors (encoders and cable tension), extrinsic electronics and motors.

Neurobotics - The fusion of Neuroscience and Robotics, FP6-IST-001917 (www.neurobotics.info). A project funded by the Future and Emerging Technologies arm of the IST programme

Future and Emerging Technologies

HANDS FOR PROSTHETICS @ SSSA

SMARTHAND robotic hand• Research prosthetic hand• 5 fingers, 4 motors, 16 dof, 40 sensors, and electronic units integrated in the hand

Information Society Technologies

Clinical experimental set-up

Development of a robust prosthetic hand provided with an advanced sensory system

– The development of new PNS neural interfaces

– The short-term implant in humans of the tfLIFEs for the control of the hand prosthesis

Collaboration

between SSSA,

UCBM, IBMT, UAB

Neurobotics - The fusion of Neuroscience and Robotics, FP6-IST-001917 (www.neurobotics.info). A project funded by the Future and Emerging Technologies arm of the IST programme

Future and Emerging Technologies

CYBERHAND robotic hand

Commercial stimulation and recording system

Efferent processing, control, afferent stimulation (on a PC platform)

LIFE electrodeswith a transcutaneoustranscutaneousconnection

Cybernetic TransradialHand Design

CyberHand and NEUROBOTICS : a robotic hand and neural peripheral interfaces for human prosthetics

Scheme of the shortScheme of the short--term implant term implant of the CYBERHAND systemof the CYBERHAND system

tfLIFE electrodeswith a

transcutaneoustranscutaneousconnection

CYBERHAND prosthesis

Recording and Stimulating Circuitry

(outside the outside the body of the body of the subjectsubject)

Efferent processing, control, afferent

stimulation (on a PC platform)

How can How can robotics robotics technology technology contribute?contribute?

Which Which advantages?advantages?

The Rehabilitation ProcessThe Rehabilitation ProcessFunctionalAssessment

Functional Recovery

FunctionalSubstitution

Functional Surgery

Motor Therapy

advantages?advantages?

Which are the Which are the current current challenges?challenges?Assistive

devicesProfessional Training

Assessment of Residual Abilities

Reintegration into social life and working activity

Two classes of

rehabilitation machines for

robot-mediated therapy

� Systems for physical therapy therapy

� Systems for neuro-rehabilitation

Neurophysiological basis for

neurorehabilitation after stroke

� Brain motor areas which are “not used” but can generate upper limb movements if properly stimulated(Kwan, 1978)

� The same motor function can be activated by multiple (different, non contiguous) brain motor areas (Humprey, 1986, Sato e Tanji, 1989, Huntley & Jones, 1991) Tanji, 1989, Huntley & Jones, 1991)

� Multiple representations of the cortico-spinal output have been shown in the motor cortex. These representations are related to different motor functions and can present several spatial and temporal overlaps (Sanes, Donoghue et al., 1995) Rijntjes et al., J Neurosci, 1999

This situation proves the flexibility of the motor output organization and is a key issue to promote functional recovery after strokeMotor learning and plasticity can be exploited for neurorehabilitation therapy, i. e. to promote functional recovery

MIT-MANUS system: clinical trials at

ASL12

Prof. A. Battaglia, Dr. F. PosteraroReparto di Medicina Riabilitativa - Centro di Alta Specialità per la Riabilitazione dei Traumi Cranici e delle Gravi Cerebrolesioni Acquisite

Clinical Validation of the MEMOS systemClinical Validation of the MEMOS system

Starting position

Clinical trials (2003 – 2004) at Fondazione Maugeri, Veruno (Italy) –

Prof. Fabrizio Pisano, Division of NeurologyP1(X1,Y1)

P2(X2,Y2)

Final position

Colombo et al, 2004Micera et al., 2004

Outline of the talk






� Robotics in rehabiliation and assistance

� Conclusions

Conclusions

� The biological domain is particularly suitable (although not the only one) for exploring the potential of systematic collaboration between robotics and science

� Robotics can provide tools useful for scientific investigation and discovery, but it can be as well a very attractive research area for discovering basic principles underlying the functioning of living beingsprinciples underlying the functioning of living beings

� Medical applications of robots are increasingly important (for research and education) and successful (for industry and in clinical practice)

An industrial product deriving (2005) from FET research (2001-4): 100.000+ robots sold!

PALOMA EU IST-FET Project IST-2001-33073

Distributed by De Agostini

SpA in 8 countries

Conclusions

� The biological domain is particularly suitable (although not the only one) for exploring the potential of systematic collaboration between robotics and science

� Robotics can provide tools useful for scientific investigation and discovery, but it can be as well a very attractive research area for discovering basic principles underlying the functioning of living beingsprinciples underlying the functioning of living beings

� Medical applications of robots are increasingly important (for research and education) and successful (for industry and in clinical practice)

� Robotics and the “grand challenges” it poses are an extraordinary means to attract, motivate, educate and train many talented and enthusiastic young students

Thank you for your attention

the lamprey robot

Documents