effects of kinesthetic and cutaneous stimulation during the learning...

14
1 Effects of kinesthetic and cutaneous stimulation during the learning of a viscous force field Giulio Rosati, Fabio Oscari, Claudio Pacchierotti, and Domenico Prattichizzo Abstract—Haptic stimulation can help humans learn perceptual motor skills, but the precise way in which it influences the learning process has not yet been clarified. This study investigates the role of the kinesthetic and cutaneous components of haptic feedback during the learning of a viscous curl field, taking also into account the influence of visual feedback. We present the results of an experiment in which 17 subjects were asked to make reaching movements while grasping a joystick and wearing a pair of cutaneous devices. Each device was able to provide cutaneous contact forces through a moving platform. The subjects received visual feedback about joystick’s position. During the experiment, the system delivered a perturbation through (1) full haptic stimulation, (2) kinesthetic stimulation alone, (3) cutaneous stimulation alone, (4) altered visual feedback or (5) altered visual feedback plus cutaneous stimulation. Conditions 1, 2 and 3 were also tested with the cancellation of the visual feedback of position error. Results indicate that kinesthetic stimuli played a primary role during motor adaptation to the viscous field, which is a fundamental premise to motor learning and rehabilitation. On the other hand, cutaneous stimulation alone appeared not to bring significant direct or adaptation effects, although it helped in reducing direct effects when used in addition to kinesthetic stimulation. The experimental conditions with visual cancellation of position error showed slower adaptation rates, indicating that visual feedback actively contributes to the formation of internal models. However, modest learning effects were detected when the visual information was used to render the viscous field. Index Terms—Cutaneous stimulation, kinesthetic stimulation, haptic force feedback, adaptation, dynamic perturbation, rehabilitation, visual perturbation 1 I NTRODUCTION I N the last two decades, there has been a rapid increase in the number of research groups and companies developing robotic interfaces for the re- habilitation of persons with movement disabilities [1]. A variety of assistive control strategies have been designed, including robots that move limbs rigidly along fixed paths, robots that take action only if the patient’s performance fails to stay within some spatial or temporal boundary, and soft robots that form a model of the patient’s weakness [2], [3]. Mechanical devices for rehabilitation are, in fact, designed to interact with humans, guiding the upper limb through repetitive exercises based on a stereotyped pattern, and providing force feedback for sensorimotor type rehabilitative training [4]. One of the most important issues facing robotic movement therapy is the lack of knowledge on how motor learning during neuro-rehabilitation works [5]. Many researchers have studied motor adaptation to altered dynamic environments. A typical setting in this field consists of applying a viscous curl field during the execution of point-to-point reaching movements in the Giulio Rosati and Fabio Oscari are with the Dept. of Management and Engineering, University of Padua, via Venezia 1, 35131 Padua, Italy. E-mail: [email protected] Domenico Prattichizzo and Claudio Pacchierotti are with the Dept. of Information Engineering and Mathematics, University of Siena, Via Roma 56, 53100 Siena, Italy and with the Dept. of Advanced Robotics, Istituto Italiano di Tecnologia, via Morego 30, 16163 Genova, Italy. horizontal plane [6], [7]. Following the initial deviation from a straight trajectory (direct effect), subjects tended to adapt to the altered dynamic environment restoring the initial motion path (adaptation). The presence of aftereffects, as a result of perturbation removal, proved that the nervous system creates an internal model of the environment, able to predict the expected perturbing forces [8]. Feygin et al. investigated the use of haptics for skill training [9]. Subjects learned a 3D motion under three training conditions: haptic, visual, and visuo-haptic. They were then required to manually reproduce the movement under two recall conditions, i.e. with and without vision. Results indicated that haptic guidance is effective in training: while visual training was better for teaching the tra- jectory shape, temporal aspects of the task were more effectively learned from haptic guidance. Regarding the interaction between visual and haptic feedback during the adaptation of human reaching movements, Scheidt et al. showed how eliminating visual feedback of hand-path deviations from a straight-line reach prevents compensation of initial direction errors in- duced by perturbations [10]. Results showed that when visual feedback of movement was eliminated entirely, proprioception was enough to guide adaptive recovery of straight and smooth hand trajectories directed towards the final target. However, eliminating visual feedback of just the orthogonal hand-path errors did not lead to reduced direction errors. Finally, it is worth mentioning the work of Morris et al., who explored

Upload: others

Post on 15-Mar-2020

15 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Effects of kinesthetic and cutaneous stimulation during the learning …sirslab.dii.unisi.it/papers/2014/Rosati.ToHSI.2014.Haptics.Fin.pdf · Effects of kinesthetic and cutaneous

1

Effects of kinesthetic and cutaneousstimulation during the learning

of a viscous force fieldGiulio Rosati, Fabio Oscari, Claudio Pacchierotti, and Domenico Prattichizzo

Abstract—Haptic stimulation can help humans learn perceptual motor skills, but the precise way in which it influences thelearning process has not yet been clarified. This study investigates the role of the kinesthetic and cutaneous components of hapticfeedback during the learning of a viscous curl field, taking also into account the influence of visual feedback. We present theresults of an experiment in which 17 subjects were asked to make reaching movements while grasping a joystick and wearing apair of cutaneous devices. Each device was able to provide cutaneous contact forces through a moving platform. The subjectsreceived visual feedback about joystick’s position. During the experiment, the system delivered a perturbation through (1) fullhaptic stimulation, (2) kinesthetic stimulation alone, (3) cutaneous stimulation alone, (4) altered visual feedback or (5) alteredvisual feedback plus cutaneous stimulation. Conditions 1, 2 and 3 were also tested with the cancellation of the visual feedback ofposition error. Results indicate that kinesthetic stimuli played a primary role during motor adaptation to the viscous field, whichis a fundamental premise to motor learning and rehabilitation. On the other hand, cutaneous stimulation alone appeared not tobring significant direct or adaptation effects, although it helped in reducing direct effects when used in addition to kinestheticstimulation. The experimental conditions with visual cancellation of position error showed slower adaptation rates, indicating thatvisual feedback actively contributes to the formation of internal models. However, modest learning effects were detected when thevisual information was used to render the viscous field.

Index Terms—Cutaneous stimulation, kinesthetic stimulation, haptic force feedback, adaptation, dynamic perturbation,rehabilitation, visual perturbation

1 INTRODUCTION

IN the last two decades, there has been a rapidincrease in the number of research groups and

companies developing robotic interfaces for the re-habilitation of persons with movement disabilities [1].A variety of assistive control strategies have beendesigned, including robots that move limbs rigidlyalong fixed paths, robots that take action only if thepatient’s performance fails to stay within some spatialor temporal boundary, and soft robots that form amodel of the patient’s weakness [2], [3]. Mechanicaldevices for rehabilitation are, in fact, designed tointeract with humans, guiding the upper limb throughrepetitive exercises based on a stereotyped pattern,and providing force feedback for sensorimotor typerehabilitative training [4].

One of the most important issues facing roboticmovement therapy is the lack of knowledge on howmotor learning during neuro-rehabilitation works [5].Many researchers have studied motor adaptation toaltered dynamic environments. A typical setting in thisfield consists of applying a viscous curl field during theexecution of point-to-point reaching movements in the

• Giulio Rosati and Fabio Oscari are with the Dept. of Management andEngineering, University of Padua, via Venezia 1, 35131 Padua, Italy.E-mail: [email protected]

• Domenico Prattichizzo and Claudio Pacchierotti are with the Dept. ofInformation Engineering and Mathematics, University of Siena, ViaRoma 56, 53100 Siena, Italy and with the Dept. of Advanced Robotics,Istituto Italiano di Tecnologia, via Morego 30, 16163 Genova, Italy.

horizontal plane [6], [7]. Following the initial deviationfrom a straight trajectory (direct effect), subjects tendedto adapt to the altered dynamic environment restoringthe initial motion path (adaptation). The presenceof aftereffects, as a result of perturbation removal,proved that the nervous system creates an internalmodel of the environment, able to predict the expectedperturbing forces [8]. Feygin et al. investigated theuse of haptics for skill training [9]. Subjects learned a3D motion under three training conditions: haptic,visual, and visuo-haptic. They were then requiredto manually reproduce the movement under tworecall conditions, i.e. with and without vision. Resultsindicated that haptic guidance is effective in training:while visual training was better for teaching the tra-jectory shape, temporal aspects of the task were moreeffectively learned from haptic guidance. Regardingthe interaction between visual and haptic feedbackduring the adaptation of human reaching movements,Scheidt et al. showed how eliminating visual feedbackof hand-path deviations from a straight-line reachprevents compensation of initial direction errors in-duced by perturbations [10]. Results showed that whenvisual feedback of movement was eliminated entirely,proprioception was enough to guide adaptive recoveryof straight and smooth hand trajectories directedtowards the final target. However, eliminating visualfeedback of just the orthogonal hand-path errors didnot lead to reduced direction errors. Finally, it is worthmentioning the work of Morris et al., who explored

Page 2: Effects of kinesthetic and cutaneous stimulation during the learning …sirslab.dii.unisi.it/papers/2014/Rosati.ToHSI.2014.Haptics.Fin.pdf · Effects of kinesthetic and cutaneous

2

the use of haptic feedback to teach an abstract motorskill which requires recalling a sequence of forces [11].Participants were guided along a trajectory and askedto learn a sequence of one-dimensional forces via threeparadigms: haptic, visual, and visuohaptic. Resultsshowed that recall following visuohaptic training wassignificantly more accurate than recall following visualor haptic training alone, although haptic training wasinferior to visual training.

Although many works analyze the role of hapticsin the learning of a perceptual motor skill, it is stillnot clear how the central nervous system combinesconcurrent stimulation, such as proprioceptive, visualor haptic feedback. Moreover, the way the kinestheticand cutaneous components of haptic feedback caninfluence such a process has not yet been clarified.We believe that this topic is fundamental for a betterunderstanding of the role of haptic feedback in motorrehabilitation, and will help optimize the design ofnovel technological aids in this field.

1.1 Cutaneous and kinesthetic feedback

Most of the grounded haptic and rehabilitazion devicesprovide a combination of kinesthetic and cutaneousstimuli to the user, if we assume that the interaction ismediated by a stylus, a ball, or any other tool mountedon the device [12], [13]. Cutaneous stimuli are sensedby pressure receptors in the skin, and they are usefulfor recognizing the local properties of objects, such asshape, edges, embossings and recessed features [14],[15]. On the other hand, through muscle spindles andthe Golgi tendon organ, kinesthesia allows the userto sense the movement of neighbouring parts of thebody and the forces being exerted [16], [17], [18].

The cutaneous and kinesthetic stimuli applied bya grounded haptic device, however, cannot be decou-pled: the force provided is felt by the user both at thefingertips (cutaneous component), and at the muscleand joint level (kinesthetic component) [12], [13].An interesting approach consists of using cutaneousdevices to activate only the cutaneous componentof the haptic interaction, which has been found tobe a simple but effective solution for reducing themechanical complexity of haptic devices, while guar-anteeing adequate performance [12], [19], [20], [21],[22]. On the other hand, using a cutaneous devicetogether with a grounded haptic interface allowsone to independently control how much kinestheticand cutaneous stimulation is provided to the user[13]. The authors of [23], for example, exploited thisidea to design a stability controller which enhancedtransparency of passive teleoperation systems withforce reflection. A similar approach was used in[24] to enhance the transparency of a 7-DoF roboticteleoperation system.

1.2 Contribution

The aim of this paper is to investigate the role ofthe two components of haptic interaction during the

learning of a viscous curl field, taking into account atthe same time the influence of visual feedback in sucha process.

Toward this challenging objective, we present theresults of an experiment in which healthy subjectswere required to make reaching movements whilegrasping a joystick handle, receiving visual feedbackabout joystick position and wearing, at the same time,a pair of cutaneous devices. The cutaneous deviceswere chosen with the specific goal of simulating, asclose as possible, the cutaneous sensations provided bythe contact with a haptic handle. Various experimentalconditions were tested, using different combinationsof kinesthetic, cutaneous and visual stimuli, with theaim of untangling what actually causes the adaptation:kinesthetic sensations, cutaneous sensations, and/or vi-sual sensations. We hypothesize that all three separatecomponents of the stimulation would cause adaptation,even though the two components of haptic feedbackmay have different roles. This hypothesis is based onthe results of a previous study [12], in which cutaneousstimulation was effectively used in substitution offull haptic feedback during the execution of a needleinsertion task.

Section 2 describes the design of the experimentand hardware setup, Sec. 3 presents the results ofstatistical analysis, and Sec. 4 addresses the discussionand outlines prospective work.

2 METHODS

2.1 Subjects

A total of 17 healthy subjects participated in theexperiment. They were aged between 20 and 29 years(mean age 23.6 ± 3.0 years, 13 males, 4 females), 16right-handed and one left-handed. All participantsreported normal vision with no color blindness, andno hearing or cutaneous-sensibility problems. Writteninformed consent for participation in the experimentsand for the publication of this report was obtainedfrom all the subjects. The experiment received theethical approval of the Scientific Commission of theUniversity of Padua.

2.2 Setup

The experimental setup is shown in Fig. 1. The subjectsat on a chair, in front of an LCD screen and a 2-DoF haptic joystick [25]. A white panel preventedthe subject from seeing the hand and the joystick. Acustom plastic support was mounted at the top of thehandle, housing two cutaneous devices, one for thethumb and one for the index finger (Fig. 2b). The otherfingers were closed into a fist. In this way, the subjectgrasped the handle only with the fingers wearing thecutaneous devices. The direction of grasping (x) wasalways transversal with respect to the direction ofmotion (y), which was parallel to the sagittal plane

Page 3: Effects of kinesthetic and cutaneous stimulation during the learning …sirslab.dii.unisi.it/papers/2014/Rosati.ToHSI.2014.Haptics.Fin.pdf · Effects of kinesthetic and cutaneous

3

x

y

5/60

Fig. 1: Experimental setup and feedback system. A2-DoF haptic feedback joystick, equipped with twocutaneous devices, was placed on a table in front of aLCD screen, while a white panel prevented the subjectfrom seeing the hand and the joystick.

of the subject. To prevent changes in the perceiveddirection of force and allow for a comfortable grasp,the subjects were instructed to keep the wrist slightlyextended (Fig. 2b).

Each cutaneous device consisted of a static part,connected to the joystick, and a mobile platform(Fig. 2a). Three springs kept the platform in a referenceconfiguration when the device was not actuated. Threeservo-motors controlled the length of three wiresconnecting the static part to the mobile platform,allowing the latter to apply the requested force atthe user’s fingertip [24]. An estimation of the exertedforce was derived through a simple elastic model of thefingertip, with a linear relationship between platformdisplacement and resultant wrench. In other terms,we assumed that the platform configuration ξ wasproportional to the wrench wp = [fT

p mT

p ]T ∈ ℜ6

applied to the mobile platform

ξ = K−1wp, (1)

where K ∈ ℜ6×6 is the stiffness matrix of the fingertip.An isotropic elastic behavior was considered, so thatthe stiffness matrix became

K =

ktI3 0

0 krI3

with kt = 0.5 N/m and kr = 0.5 Nm/rad representingthe translational and rotational stiffness constants,respectively [26]. This approach has been validated bythe experiments carried out in [19], [19], [24].

In our experiments the cutaneous interfaces wereused as 1-DoF devices (all motors pulled the cablestogether), so that only the forces in the sagittal plane

(a) 3-DoF cutaneous device employed in this work.

(b) Experimental setup.

Fig. 2: Users had to grasp the 2-DoF haptic joystickwhile wearing two cutaneous devices, one on thethumb and one on the index finger.

of the finger were generated, roughly normal to thelongitudinal axis of the distal phalanx.

2.3 Experimental protocol

Subjects were asked to perform forward (+y direction)and backward (−y direction) 13 cm-long reachingmovements (reaches) between two fixed targets, asstraight as possible. The exercise was divided intogroups of 60 reaches (task), corresponding to 30 for-ward and 30 backward movements. The position of thehand with respect to the joystick was checked beforethe beginning of each task.

A fixed, red circular cursor on the LCD screenindicated the position of the current target. The target’sdiameter was 0.5 cm in the joystick space, correspond-ing to 1 cm on the screen. A green circular cursor of thesame size was printed on the screen, whose positionalong the y axis corresponded to the position of thejoystick in the back and forth direction. The x positionof the green cursor either denoted the lateral positionof the joystick (actual visual feedback: VFA), or wasset to zero (straight visual feedback: VFS). The lattermode corresponded to a cancellation of the left-rightposition error.

Subjects were asked to complete each reach (eitherforward or backward) in the time between two beeps of

Page 4: Effects of kinesthetic and cutaneous stimulation during the learning …sirslab.dii.unisi.it/papers/2014/Rosati.ToHSI.2014.Haptics.Fin.pdf · Effects of kinesthetic and cutaneous

4

a metronome playing at 33 bpm (one beep every 1.8 s).The metronome was used to standardize the durationof the reaches among subjects. Subjects were alloweda 30 second warm-up to practice the rhythm dictatedby the metronome. The devices, the metronome andthe graphic rendering were controlled by a real-timesoftware (Matlab/Simulink R2012b) running at 200 Hz.

During each task, a dynamic perturbation wasintroduced from the 11th to the 50th reach, afterwhich it was removed. The perturbation consistedof a viscous curl field, rendered to the subject in fivedifferent ways:

- Cutaneous + Kinesthetic Stimulation (CS+KS)The perturbation consisted of a viscous force fx,generated by the joystick. The lateral force wascomputed as a function of the velocity along themotion axis (y):

f =

fx

fy

=

b1,1 b1,2

b2,1 b2,2

·

vx

vy

(2)

where all the elements of the viscosity matrixwere set to zero except for b1,2 = 20 Ns/m.The endpoint force f is given in Newtons, thevelocity v in meters per second. It is worth notingthat both the cutaneous and kinesthetic stimuliwere provided by the haptic joystick, while thecutaneous devices were turned off.

- Cutaneous Stimulation (CS)The perturbation, as of Eq. 2, was provided bythe cutaneous devices, while the haptic joystickwas used to track the position of the hand only(it provided no force feedback). According tothe sign of fx, the cutaneous force was appliedeither to the index finger (fx > 0) or to thethumb (fx < 0). This stimulation resembled theforce field presented in the previous mode, butwithout the kinesthetic part of the perturbation.This approach of subtracting kinesthesia from thecomplete haptic interaction by means of cutaneousdevices was introduced in [12], and it is calledsensory subtraction.

- Kinesthetic Stimulation (KS)The joystick provided the same force feedback asin the first modality (CS+KS). Concurrently, thecutaneous devices were used to produce a 5 Nforce on the fingertip pushed by the joystick. Inthis way, the cutaneous stimulation originated bythe joystick was completely masked. In fact, abovea threshold of ∼2 N [27], [28], [29], the cutaneousreceptors do not provide any perceivable sensationof increasing force. This condition is the closestto a pure kinesthetic stimulation.

- Visual Distortion (VD)In this mode, both the joystick and the cutaneousdevices were switched off, and the perturbation

was generated by providing altered visual infor-mation on the lateral position of the hand (visuo-motor transformation). The lateral (x) position ofthe green cursor on the screen was set equal tothe position error ex, computed as the differencebetween lateral joystick position (xs) and a viscousdistortion (xr, reference position):

ex = xs − xr = xs − b · vy (3)

where xs and ex are given in meters, vy in metersper second, whilst b was set to −0.15 s. Thisdistortion emulated the effects of a lateral viscousforce field, without providing any kinesthetic orcutaneous stimuli.

- Visual + Cutaneous Stimulation (VD+CS)The visual feedback provided to the subjectswas the same as in the previous mode (VD).Concurrently, a cutaneous stimulation was gen-erated as computed in the cutaneous mode (CS).This stimulation augmented the quality of theemulated force field, by combining the visuomotortransformation with the stimulation of fingertips.

The five perturbations were combined with thevisual modes (VFA and VFS) into eight experimentalconditions (see Table 1): A, B and C included visual in-formation about the actual position of the hand (VFA),combined with the active stimulation (CS+KS, CS andKS, respectively); F, G and H were the counterpartsof A, B and C, with elimination of visual feedbackon lateral position error (VFS); D and E included thevisuomotor transformation (VD), while the subjectsreceived actual visual feedback (VFA) at baseline andafter removal of the perturbation; E included cutaneousfeedback also (CS).

The protocol consisted of eight tasks, each onecorresponding to one of the experimental conditions.The conditions were alternated using randomizedsequences. No information about the experimentalconditions was provided to the subjects, neither ontheir nature nor on the particular order with whichthey were going to be presented.

2.4 Data analysis

We grouped the reaches of each task into 8 main phases,according to the motor adaptation literature [6], [7],[30]:

1) Baseline: the initial movements performed with-out perturbation (reaches 1–10).

2) Direct effect: the first reach with exposure to theperturbation (reach 11).

3) Adaptation (Early): the following 12 reaches, dur-ing which the subjects started to adapt to theperturbation (reaches 12–24).

4) Adaptation (Medium): the central phase of adapta-tion (reaches 25–37).

Page 5: Effects of kinesthetic and cutaneous stimulation during the learning …sirslab.dii.unisi.it/papers/2014/Rosati.ToHSI.2014.Haptics.Fin.pdf · Effects of kinesthetic and cutaneous

5

PerturbationVisual feedback (VF)

Actual (A) Straight (S)

Haptic stimulation (CS+KS) A F

Cutaneous stimulation (CS) B G

Kineshetic stimulation (KS) C H

Visual distortion (VD) D -

Video-cutaneous stim. (VD+CS) E -

TABLE 1: Experimental conditions. Each subject per-formed eight different tasks in a randomized sequence.Visual feedback was provided during all the exper-iment, while perturbations were provided from the11th to the 50th reach. Note that in these reaches, forboth the VD and VD+CS case, the visual feedbackmodality was switched from VFA to VD in order togenerate the viscous visual distortion.

5) Adaptation (Late): the last 12 reaches before re-moving the perturbation (reaches 38–50).

6) After effect: the first reach after removing theperturbation (reach 51).

7) Re-adaptation (Early): the phase during whichthe subjects started to re-adapt to movementswithout perturbation (reaches 52–55).

8) Re-adaptation (Late): the last phase of re-adaptation (reaches 56–60).

For each participant, both position errors (alongthe x direction) and movement speeds (along the ydirection) have been analyzed in each reach.

The left–right average weighted position error [30], [31]was calculated as follows:

1

Mk

Mk∑

h=1

(

n2h∑

i=n1h

− sign (vyi) · (xsi − xri)

n2 − n1 + 1

)

(4)

where k denotes the phase and h the reach number;Mk is the number of reaches in phase k; xsi, xri and vyiare, respectively, the current x position, the x referenceposition and the y velocity of the hand (i−th samples);n1h and n2h define the portion of the reach beinganalyzed. The reference position was xri = 0 for all theexperimental conditions except for conditions D andE, in which xri was the viscous distortion computedas in Eq. 3.

Let Nh be the number of samples in reach h. Thefollowing portions of the reach have been considered:n1h = 1, n2h = Nh (entire reach); n1h = 1, n2h = Nh/2(first half of reach, in terms of travel); n1h = Nh/2 + 1,n2h = Nh (second half of reach). The first and thesecond half of the reach were analyzed separately withthe aim of catching differences in terms of feedforwardcontrol during adaptation. In fact, according to motoradaptation studies [32], [33], the initial error in areach can be interpreted as a result of predictiveor feedforward control while the remainder of thetrajectory is likely determined by feedback control.

Regarding movement speeds, the peak of vy andits temporal location within the reach (peak time) werecomputed for each reach and averaged within thephase.

Normality tests (Shapiro-Wilk normality test andD’Agostino-Pearson omnibus normality test) indicateda Gaussian distribution of all position and velocitymeasures. We run a two-way within-subjects ANOVAfor each metric, with the phases and the experimentalconditions as within factors. In the presence of signifi-cant effects, pair wise post-hoc comparisons (Tukey’stest) were performed.

For each experimental condition in which motoradaptation was found, we calculated the left-rightweighted position error of the first half of each reachwhen perturbation was applied (reaches 11 − 50).For each subject, such data were least-square fittedwith one-phase exponential curves, after smoothingwith a 2-trial central moving average, to model thelearning process as a function of trial number. Thedecay constants of the curves were compared bymeans of a one-way within subjects ANOVA, usingthe experimental condition as repeated measure, andpost hoc comparisons were performed (Sidak’s test).

Two participants exhibited large variable errors andwhen questioned after the experiment it was apparentthat they had misunderstood the task; thus their datawere excluded.

3 RESULTS

The mean trajectory and the average intra-subjectstandard deviation are shown in Fig. 3, 4, 5 for eachexperimental condition and phase.

The measures of peak movement speeds indicatedinteraction between the two factors (i.e., experimentalcondition and phase), with a prominent effect of phase(interaction: F (49, 686) = 1.675, p = 0.0033; condition:F (7, 98) = 0.5396, p = 0.8026; phase: F (7, 98) = 8.207,p < 0.0001). Comparable results were obtained fromthe analysis of peak time (interaction: F (49, 686) =1.606, p = 0.0065; condition: F (7, 98) = 0.7406, p =0.6381; phase: F (7, 98) = 5.748, p < 0.0001).

A limited number of significant comparisons re-sulted from Tukey’s test, mostly for conditions A andF (where full haptic feedback was used) and when thedirect or after effects were compared to baseline.

These results indicate that the use of metronomeroughly standardized velocity profiles among phasesand conditions. Only the external perturbation, es-pecially when provided in the form of full hapticinteraction, tended to induce variations in the profiles,although limited to the cycles immediately followingthe introduction or removal of the perturbation.

The average peak movement speed was0.149±0.033 m/s, while peak time was 0.555±0.071 son average (nearly one third of average reach time).The position of the peak, measured along the directionof reach, was at about 45% of the travel.

Page 6: Effects of kinesthetic and cutaneous stimulation during the learning …sirslab.dii.unisi.it/papers/2014/Rosati.ToHSI.2014.Haptics.Fin.pdf · Effects of kinesthetic and cutaneous

6

−30 −15 0 15 300

32.5

65

97.5

130Null e�ect

x [mm]

y [

mm

]

−30 −15 0 15 300

32.5

65

97.5

130Direct E�ect

x [mm]

y [

mm

]

−30 −15 0 15 300

32.5

65

97.5

130Adaptation (Early)

x [mm]

y [

mm

]

−30 −15 0 15 300

32.5

65

97.5

130Adaptation (Medium)

x [mm]

y [

mm

]

−30 −15 0 15 300

32.5

65

97.5

130Adaptation (Late)

x [mm]

y [

mm

]

−30 −15 0 15 300

32.5

65

97.5

130After E�ect

x [mm]

y [

mm

]

−30 −15 0 15 300

32.5

65

97.5

130Re−adaptation (Early)

x [mm]

y [

mm

]

−30 −15 0 15 300

32.5

65

97.5

130Re−adaptation (Late)

x [mm]

y [

mm

]

(a) Experimental condition A (VFA+CS+KS).

−30 −15 0 15 300

32.5

65

97.5

130Null e�ect

x [mm]

y [m

m]

−30 −15 0 15 300

32.5

65

97.5

130Direct E�ect

x [mm]

y [m

m]

−30 −15 0 15 300

32.5

65

97.5

130Adaptation (Early)

x [mm]

y [m

m]

−30 −15 0 15 300

32.5

65

97.5

130Adaptation (Medium)

x [mm]

y [m

m]

−30 −15 0 15 300

32.5

65

97.5

130Adaptation (Late)

x [mm]

y [m

m]

−30 −15 0 15 300

32.5

65

97.5

130After E�ect

x [mm]

y [m

m]

−30 −15 0 15 300

32.5

65

97.5

130Re−adaptation (Early)

x [mm]

y [m

m]

−30 −15 0 15 300

32.5

65

97.5

130Re−adaptation (Late)

x [mm]

y [m

m]

(b) Experimental condition B (VFA+CS).

−30 −15 0 15 300

32.5

65

97.5

130Null e�ect

x [mm]

y [m

m]

−30 −15 0 15 300

32.5

65

97.5

130Direct E�ect

x [mm]

y [m

m]

−30 −15 0 15 300

32.5

65

97.5

130Adaptation (Early)

x [mm]

y [m

m]

−30 −15 0 15 300

32.5

65

97.5

130Adaptation (Medium)

x [mm]

y [m

m]

−30 −15 0 15 300

32.5

65

97.5

130Adaptation (Late)

x [mm]

y [m

m]

−30 −15 0 15 300

32.5

65

97.5

130After E�ect

x [mm]

y [m

m]

−30 −15 0 15 300

32.5

65

97.5

130Re−adaptation (Early)

x [mm]

y [m

m]

−30 −15 0 15 300

32.5

65

97.5

130Re−adaptation (Late)

x [mm]

y [m

m]

(c) Experimental condition C (VFA+KS).

Fig. 3: Trajectories of experimental conditions A (VFA+CS+KS), B (VFA+CS), C (VFA+KS) in the different phasesof the experiment. Average trajectory (solid lines), reference path (dashed lines), and average intra-subjectstandard deviation (patches) of the group are shown. Forward and backward movements are shown in orangeand green respectively. The reference path is a vertical line fixed at x = 0. The average intra-subject standarddeviation is the mean of the standard deviation for the every single experimental condition in each phase,obtained by calculating the variance for each subject in each phase from the sample data.

Page 7: Effects of kinesthetic and cutaneous stimulation during the learning …sirslab.dii.unisi.it/papers/2014/Rosati.ToHSI.2014.Haptics.Fin.pdf · Effects of kinesthetic and cutaneous

7

−30 −15 0 15 300

32.5

65

97.5

130Null e�ect

x [mm]

y [

mm

]

−30 −15 0 15 300

32.5

65

97.5

130Direct E�ect

x [mm]

y [

mm

]

−30 −15 0 15 300

32.5

65

97.5

130Adaptation (Early)

x [mm]

y [

mm

]

−30 −15 0 15 300

32.5

65

97.5

130Adaptation (Medium)

x [mm]

y [

mm

]

−30 −15 0 15 300

32.5

65

97.5

130Adaptation (Late)

x [mm]

y [

mm

]

−30 −15 0 15 300

32.5

65

97.5

130After E�ect

x [mm]

y [

mm

]

−30 −15 0 15 300

32.5

65

97.5

130Re−adaptation (Early)

x [mm]

y [

mm

]

−30 −15 0 15 300

32.5

65

97.5

130Re−adaptation (Late)

x [mm]

y [

mm

]

(a) Experimental condition D (VD).

−30 −15 0 15 300

32.5

65

97.5

130Null e�ect

x [mm]

y [m

m]

−30 −15 0 15 300

32.5

65

97.5

130Direct E�ect

x [mm]

y [m

m]

−30 −15 0 15 300

32.5

65

97.5

130Adaptation (Early)

x [mm]

y [m

m]

−30 −15 0 15 300

32.5

65

97.5

130Adaptation (Medium)

x [mm]

y [m

m]

−30 −15 0 15 300

32.5

65

97.5

130Adaptation (Late)

x [mm]

y [m

m]

−30 −15 0 15 300

32.5

65

97.5

130After E�ect

x [mm]

y [m

m]

−30 −15 0 15 300

32.5

65

97.5

130Re−adaptation (Early)

x [mm]

y [m

m]

−30 −15 0 15 300

32.5

65

97.5

130Re−adaptation (Late)

x [mm]

y [m

m]

(b) Experimental condition E (VD+CS).

Fig. 4: Trajectories of experimental conditions D (VD) and E (VD+CS) in the different phases of the experiment.Average trajectory (solid lines), average reference path (dashed lines), and average intra-subject standarddeviation (patches) of the group are shown. Forward and backward movements are show in orange and greenrespectively. The average reference path is a velocity-proportional function as computed in Equation 3 by usingthe y velocity averaged over concerned trials of all subjects. The average intra-subject standard deviation isthe mean of the standard deviation for the every single experimental condition in each phase, obtained bycalculating the variance for each subject in each phase from the sample data.

Figure 6 shows the position metric and its inter-subject standard deviation in all phases, for the entirereach (Fig. 6a) and for its first (Fig. 6b) and secondhalf (Fig. 6c). Statistical analysis indicated a significantinteraction between experimental condition and phase(interaction: F (49, 686) = 31.09, p < 0.0001; condition:F (7, 98) = 30.40, p < 0.0001; phase: F (7, 98) = 216.80,p < 0.0001).

3.1 Direct and after effects

Pairwise post-hoc analyses indicated that all the con-ditions, except from B and G, exhibited significantdirect effects when the force field was first applied(p < 0.0001, comparison with baseline). Indeed, condi-

tions with CS alone, in presence of either VFA or VFS,showed no significant direct effects (B: p = 0.9998; G:p = 0.8175), regardless of whether the LCD screen pro-vided information on the lateral position (comparisonB–G: p = 0.7931).

Regarding the magnitude of the direct effect, nodifference was found between A and D (p = 0.6530),indicating that the visual perturbation, as providedin D, was effectively tuned to simulate the dynamicperturbation applied in A. Direct effects were compara-ble between conditions with the same visual feedback(A–C: p = 0.3546; D–E: p = 0.7931; F–H: p = 0.0661)when the position metric was calculated on the entirereach. However, C showed a significantly greater direct

Page 8: Effects of kinesthetic and cutaneous stimulation during the learning …sirslab.dii.unisi.it/papers/2014/Rosati.ToHSI.2014.Haptics.Fin.pdf · Effects of kinesthetic and cutaneous

8

−30 −15 0 15 300

32.5

65

97.5

130Null e�ect

x [mm]

y [

mm

]

−30 −15 0 15 300

32.5

65

97.5

130Direct E�ect

x [mm]

y [

mm

]

−30 −15 0 15 300

32.5

65

97.5

130Adaptation (Early)

x [mm]

y [

mm

]

−30 −15 0 15 300

32.5

65

97.5

130Adaptation (Medium)

x [mm]

y [

mm

]

−30 −15 0 15 300

32.5

65

97.5

130Adaptation (Late)

x [mm]

y [

mm

]

−30 −15 0 15 300

32.5

65

97.5

130After E�ect

x [mm]

y [

mm

]

−30 −15 0 15 300

32.5

65

97.5

130Re−adaptation (Early)

x [mm]

y [

mm

]

−30 −15 0 15 300

32.5

65

97.5

130Re−adaptation (Late)

x [mm]

y [

mm

]

(a) Experimental condition F (VFS+CS+KS).

−30 −15 0 15 300

32.5

65

97.5

130Null e�ect

x [mm]

y [m

m]

−30 −15 0 15 300

32.5

65

97.5

130Direct E�ect

x [mm]

y [m

m]

−30 −15 0 15 300

32.5

65

97.5

130Adaptation (Early)

x [mm]

y [m

m]

−30 −15 0 15 300

32.5

65

97.5

130Adaptation (Medium)

x [mm]

y [m

m]

−30 −15 0 15 300

32.5

65

97.5

130Adaptation (Late)

x [mm]

y [m

m]

−30 −15 0 15 300

32.5

65

97.5

130After E�ect

x [mm]

y [m

m]

−30 −15 0 15 300

32.5

65

97.5

130Re−adaptation (Early)

x [mm]

y [m

m]

−30 −15 0 15 300

32.5

65

97.5

130Re−adaptation (Late)

x [mm]

y [m

m]

(b) Experimental condition G (VFS+CS).

−30 −15 0 15 300

32.5

65

97.5

130Null e�ect

x [mm]

y [m

m]

−30 −15 0 15 300

32.5

65

97.5

130Direct E�ect

x [mm]

y [m

m]

−30 −15 0 15 300

32.5

65

97.5

130Adaptation (Early)

x [mm]

y [m

m]

−30 −15 0 15 300

32.5

65

97.5

130Adaptation (Medium)

x [mm]

y [m

m]

−30 −15 0 15 300

32.5

65

97.5

130Adaptation (Late)

x [mm]

y [m

m]

−30 −15 0 15 300

32.5

65

97.5

130After E�ect

x [mm]

y [m

m]

−30 −15 0 15 300

32.5

65

97.5

130Re−adaptation (Early)

x [mm]

y [m

m]

−30 −15 0 15 300

32.5

65

97.5

130Re−adaptation (Late)

x [mm]

y [m

m]

(c) Experimental condition H (VFS+KS).

Fig. 5: Trajectories of experimental conditions F (VFS+CS+KS), G (VFS+CS), H (VFS+KS) in the different phasesof the experiment. Average trajectory (solid lines), reference path (dashed lines), and average intra-subjectstandard deviation (patches) of the group are shown. The reference path is a vertical line fixed at x = 0.Forward and backward movements are show in orange and green respectively. The average intra-subjectstandard deviation is the mean of the standard deviation for the every single experimental condition in eachphase, obtained by calculating the variance for each subject in each phase from the sample data.

Page 9: Effects of kinesthetic and cutaneous stimulation during the learning …sirslab.dii.unisi.it/papers/2014/Rosati.ToHSI.2014.Haptics.Fin.pdf · Effects of kinesthetic and cutaneous

9

-25

-20

-15

-10

-5

0

5

10

Weig

hte

d P

osit

ion

Err

or

in X

dim

en

sio

n [

mm

] A B C D E F G H

(a) Entire reach.

-25

-20

-15

-10

-5

0

5

10

Weig

hte

d P

osit

ion

Err

or

in X

dim

en

sio

n [

mm

]

(b) First half of each reach.

-25

-20

-15

-10

-5

0

5

10

Weig

hte

d P

osit

ion

Err

or

in X

dim

en

sio

n [

mm

]

Null effect Direct Effect Adaptation (Early,Medium,Late) After Effect Re-Adaptation (Early, Late)

(c) Second half of each reach.

Fig. 6: Average weighted position error in the x direction of all the experimental conditions (A: VFA+CS+KS; B:VFA+CS; C: VFA+KS; D: VD; E: VD+CS; F: VFS+CS+KS; G: VFS+CS; H: VFS+KS), for different phases of theexperiment. The error bars represent the inter-subject standard deviation, i.e. the standard deviation computedin each phase by using the weighted position error of each subject as data set.

Page 10: Effects of kinesthetic and cutaneous stimulation during the learning …sirslab.dii.unisi.it/papers/2014/Rosati.ToHSI.2014.Haptics.Fin.pdf · Effects of kinesthetic and cutaneous

10

effect with respect to A in the first half of the reach(p = 0.0208), while H had a greater direct effect withrespect to F in the second half of the reach (p = 0.0004).

Thus, the first result is that the participantsexhibited comparable direct effects when what wasapplied was a sudden perturbation in the presenceof either a kinesthetic or an altered visual feedback,i.e., cutaneous feedback alone was not sufficient togenerate direct effects. On the other hand, whenused in addition to kinesthetic stimulation, cutaneousfeedback played a role in reducing the direct effect, atleast in selected portions of the reach. However, thislast point needs deeper investigation.

The direct effect was also compared between similarperturbation modalities with different visual feedback(VFA and VFS respectively), without finding significanteffects (A–F: p = 0.2893; C–H: p = 0.0514). This resultwas confirmed when direct effects were calculatedin the first half of the reach (A–F: p = 0.9993; C–H:p = 0.2481). However, F and H showed significantlygreater direct effects with respect to A and C in thesecond half of the reach (A–F: p = 0.0283; C–H: p <0.0001). This result is in accordance with Fig. 5a andFig. 5c, where the direct effect traces indicate little tono late movement compensation, due to the lack ofvisual information on the lateral position of the hand.

All the experimental conditions presenting directeffects showed also significant after effects when theperturbation was unexpectedly removed (A, C, F, H:p < 0.0001; D: p = 0.0014; E: p = 0.0125; comparisonwith baseline). Such effects were comparable in mag-nitude between conditions without visual distortion(A–C, F–H, A–F, C–H: p > 0.9999), while conditionsD and E presented significantly smaller after effectswith respect to the others (p < 0.05).

3.2 Adaptation

Following the direct effect, in the experimental con-ditions with VFA and VFS (A, C, F, H), the subjectsadapted to the alteration provided, reducing their posi-tion error. Indeed, the error in the last adaptation phasediffered significantly if compared with the direct effect(p < 0.0001). Instead, D and E didn’t present significantadaptation (D: p > 0.9999; E: p = 0.9227), and thefinal errors were similar between the two conditions(p > 0.9999). These results suggest that kinestheticstimulation plays a primary role in adaptation.

One-way ANOVA on the decay constants indicatedsignificant effect of experimental condition (F (3, 42) =7.790, p = 0.0003). The decay constants were smallerfor the conditions with VFA if compared with theones with VFS (A–F: p = 0.0320; C–H: p = 0.0478),suggesting that the visual information on lateral errorhelped the subjects to gain higher adaptation rates.On the other hand, adaptation rates were comparablein conditions sharing the same visual condition (A–C:p = 0.1570; F–H: p = 0.9971), indicating that cutaneousfeedback had little or null effect on adaptation.

By analyzing the first half of the reach, wherefeedforward control is likely to play a major role[32], [33], conditions D and E showed significantdifferences between direct effect and late adaptation(p < 0.0001). This result lies in accordance with [34],where subjects exposed to a visual viscous perturbationshowed adaptation in the first portion of the reach. Onthe other hand, the second half of the reach revealed noreduction of the position error in the reaches followingdirect effect.

4 DISCUSSION AND CONCLUSION

4.1 Cutaneous and kinesthetic stimulation

In our experimental setting, cutaneous cues simulatingthe presence of a viscous force field did not bringany significant direct or after effects when used alone,regardless of the visual feedback provided to thesubject (with or without cancellation of position error).Only small contributions, in terms of reduced directeffects, were found in selected portions of the reachwhen cutaneous stimulation was delivered togetherwith kinesthetic stimulation. On the other hand, all ex-perimental conditions including kinesthetic stimulation(A, C, F, H) brought significant adaptation, aftereffectsand re-adaptation, regardless of the cutaneous or visualfeedback concurrently being provided. This resultsuggests that kinesthetic stimulation plays a primaryrole in robot-induced motor adaptation, while littlecontribution is provided by the cutaneous part ofhaptic interaction.

The lack of importance of cutaneous feedback inour experiments, partly conflicts with our initial hy-pothesis, and with the numerous works highlightingthe prominent role of cutaneous forces in recognizingshapes [35], in curvature discrimination tasks [19],[36], and, more generally, in improving the illusion ofpresence in virtual and remote environments [12], [37],[38]. One explanation may be that the deformationof the finger pads, if not supported by a consistentmeasure of force by kinetic receptors, is not sufficientto produce the perception of viscosity [39]. It may alsobe that using wearable cutaneous devices, leaving theparticipant free to interact with the joystick in a morenatural way [12], [13], would have helped in betteroutlining the role of cutaneous stimulation.

For example, in [12], Prattichizzo et al. analyzedthe role of cutaneous feedback in teleoperation. Theysubstituted haptic force feedback, provided by a com-mon single-contact haptic interface, with cutaneousstimuli only, and registered the performance of 16subjects during a 1-DoF telemanipulation task. Theirresults showed improved performance with respectto traditional sensory substitution techniques, but stillnot as good as employing a kinesthetic stimulation.A similar result was also presented in [24], wherethe role of cutaneous force in a more challengingtelemanipulation task was analyzed. The participantswere asked to perform a peg-in-hole experiment in a

Page 11: Effects of kinesthetic and cutaneous stimulation during the learning …sirslab.dii.unisi.it/papers/2014/Rosati.ToHSI.2014.Haptics.Fin.pdf · Effects of kinesthetic and cutaneous

11

virtual scenario employing (1) cutaneous feedback only,(2) cutaneous and kinesthetic feedback, (3) kinestheticfeedback only, and (4) no force feedback at all. Similarlyto [12], cutaneous force showed worse performancesthan for experimental conditions where kinesthesiawas provided, but, unlike the results presented in thiswork, it showed better performances than for the caseproviding no force feedback at all.

The discrepancies between the results presented inthis work and the ones in [12], [24] can be related to thefact that they dealt with very different environments aswell as tasks. The cutaneous and kinesthetic feedbacksprovided in the previous works were related to theproperties of the virtual environment (i.e. the presenceof an object) and were always coherent with thevisual information being provided. Force feedbackwas always directed opposite to the motion of theuser’s hand and proportional to its displacement, asthe haptic interaction was designed according to thegod-object model [40]. Instead, in the work presentedhere, the force perturbation was always perpendicularto the motion of the user’s hand and proportional toits velocity, measured along the direction of motion.Moreover, in [12], [24], visual feedback provided verylimited information about the task being performed.This may have led users to concentrate more on theforce cues than they did in the experiments presentedhere, where visual cues had a prominent role.

4.2 Viscous visual and video-cutaneous perturba-tions

The importance of kinesthetic stimulation is also clearif we compare tasks with visual distortion (D, E) to theones in which kinesthetic stimuli were provided (A,C, F, H). All of them showed significant direct effects,but the latter had greater aftereffects and adaptation.

It must be underlined that, in conditions D and E,the subjects were forced to learn a velocity-dependenttrajectory model that consisted of moving along a non-straight curve (see reference paths in Fig. 4). On theopposite, in all other tasks the target trajectory wasstraight. To this regard, tasks D and E appeared differ-ent if compared to the other experimental conditions.Nonetheless, conditions in which viscous visual (D)and video-cutaneous perturbations (E) were providedto the subjects showed some learning effects, indicatedby significant aftereffect and reduced hand-path errorfollowing direct effect (limited to the analysis of firsthalf of each reach). This result is consistent with Bocket al. [34]. In that study, subjects were instructed topoint on a digitizing tablet when a viscous visualdistortion was introduced, by displacing a feedbackcursor proportionally to hand velocity. Also, a viscousforce field was provided to another group of subjects.By analyzing the average position error in the firstpart of motion, they found that participants were ableto adapt to both the visual distortion and to the forcefield. The adaptation rate of the group receiving visual

distortion, however, looked slower with respect to thatobtained with the force field.

Another similar study was performed by Wolpertet al. [41], in which the visual feedback of handposition was altered so as to increase the perceivedcurvature of the movement during self-paced point-to-point arm movements. Increasing the perceivedcurvature of normally straight sagittal movements ledto to significant corrective adaptation in the curvatureof the actual hand path: the movement became curved,thereby reducing the visually perceived curvature.However, in this case the distortion was not relatedto velocity.

We can summarize by saying that the contributionof kinesthesia to motor adaptation can only partiallybe replaced by visual information reproducing thealteration of motion which would result from theapplication of a lateral force field. This suggests thatthe information provided through the visual sensorychannel, if not corroborated by kinesthesia, is notsufficient to produce the same alterations in thesubject’s motor control. This conclusion is supportedby Hwang et al. [42], who tested the hypothesis thatproprioceptive states in which the limb is perturbeddominate the representation of limb state, by perform-ing a task where position of the hand during a reachwas correlated with patterns of force perturbation.

The additional cutaneous stimulation provided inour experiment during task E, which reproducedthe same effect on the fingertips which would resultfrom the application of a lateral force field, did notbring major improvements. In fact, tasks D and Ewere comparable in terms of direct and after effects,regardless of the presence of cutaneous stimuli, eventhough small increases of direct effect were observedin task E, in comparison with task D.

4.3 Contribution of visual feedback

By comparing tasks A vs F and C vs H, we can noticethat when visual feedback correctly resembles handmovements (A, C), motor adaptation becomes fasterwith respect to when visual cues are not informative onposition error (F, H). This suggests that visual feedback,if properly delivered, can influence adaptation to adynamic environment.

The role of visual feedback may not be limitedto affecting adaptation rates only, as evidenced byour experiment. For example, Melendez-Calderon etal. [43] found that subjects provided with reducedproprioceptive information were able to graduallyadapt to a viscous force field, using visual informationdiscrepant from proprioception. This happened whenproviding visual information about the position error,in addition to a constraining channel created by arobotic interface, which closed down lateral move-ments. In this setting, small proprioceptive errors wereused during the exposure phase but always in thepresence of kinesthetic information. Another important

Page 12: Effects of kinesthetic and cutaneous stimulation during the learning …sirslab.dii.unisi.it/papers/2014/Rosati.ToHSI.2014.Haptics.Fin.pdf · Effects of kinesthetic and cutaneous

12

finding is presented by Sarlegna et al. in [44]. Theyexamined the motor behavior of a deafferented patient,deprived of proprioception below the nose, to assessadaptation to new dynamic conditions in the absenceof limb proprioception. Although her impairment wasobvious in baseline reaching performance, the propri-oceptively deafferented patient clearly adapted to thenew force conditions. This finding shows that motoradaptation to a modified force field is possible withoutproprioception, and that vision can compensate forthe permanent loss of proprioception and update thecentral representation of limb dynamics.

In our experiment, during tasks F and H, a straightvisual feedback was provided to the users that isknown in the literature as a visual channel [10]. Thiscondition can be considered as a “false visual feedback”case, wherein the green cursor represented a projectionof the hand’s trajectory onto the straight line passingthrough the targets. Scheidt et al. [10] performed aseries of experiments exploring the integration ofvisual and proprioceptive estimates of hand-patherror during adaptation of reaching movements toa novel dynamic environment. Subjects grasped andmoved the handle of an instrumented robot, whichpushed the hand away from its intended target. Theyemployed three visual feedback conditions: accuratevisual feedback (concurrent visual and proprioceptivefeedback), no visual feedback (proprioception feedbackonly), and visual channel feedback. The latter conditionsignificantly impaired correction of initial directionerrors during reaching. In fact, these errors increasedwith repeated exposure to the field. This result ap-parently contrasts with that found in our experiment.However, one must notice that the motion task inScheidt’s experiment differed from the one presentedhere, since it included the reach of eight target locationsequally spaced around the periphery of a circle inthe horizontal plane, and the viscous force field wasa function of both directions (x and y). Secondly, nosubject in that study reported being aware of the visualchannel manipulation when asked to describe his/herexperience after that session. On the contrary, in ourexperiment most subjects declared they had realizedthat the visual information had been altered. Implicit,in this case, is the possibility that the subjects usedunaltered proprioceptive feedback of movement errorsto drive adaptive improvements in motor performance,disregarding the visual information.

4.4 Concluding remarks

A limitation of the present study is that the reach-ing task was essentially one-dimensional, whereasfuture studies should examine how kinesthetic andcutaneous feedback can drive adaptation during mul-tidimensional arm movements. Moreover, we did notsystematically explore how the rate of introduction ofthe perturbation may affect the motor system’s abilityto use cutaneous, kinesthetic and visual information.

In designing our experiment, we were inspired bythe protocol of the classic motor adaptation studyby Shadmehr and Mussa-Ivaldi [6], in which theauthors abruptly introduced a dynamic perturbation.However, studying the effect of a gradual introductionof the perturbations, especially in the case of a visualdistortion, may be very interesting. Recent studieshave suggested that the rate of introduction of aperturbation may affect the duration of the aftereffects, the amount of retention, and the pattern ofgeneralization [45]. Moreover, this may also call intoplay different neural substrates to drive the adaptation[46]. It is therefore possible that the rate of introductionof the perturbation, and the type of feedback, mayinteract to affect these factors as well.

It is also worth underlining that the results pre-sented in this paper may have been influenced by theparticular cutaneous devices employed. To this regard,future studies should investigate the usage of morewearable cutaneous interfaces, making the operatorable of interacting with the joystick handle in a morenatural way.

Finally, in this study we totally excluded auditoryfeedback from the protocol, although this stimulationmodality, when properly delivered, has proven to in-fluence both motor performance and motor adaptationduring the execution of reaching tasks [30], [31].

ACKNOWLEDGMENTS

The research leading to these results has receivedfunding from the European Union Seventh FrameworkProgramme FP7/2007-2013 under grant agreement n◦

270460 of the project “ACTIVE - Active ConstraintsTechnologies for Ill-defined or Volatile Environments”and under grant agreement n◦ 601165 of the project“WEARHAP - WEARable HAPtics for humans androbots”.

REFERENCES

[1] G. Rosati, “The place of robotics in post-stroke rehabilitation,”Expert Review of Medical Devices, vol. 7, no. 6, pp. 753–758, 2010.

[2] L. Marchal-Crespo and D. J. Reinkensmeyer, “Review of controlstrategies for robotic movement training after neurologic injury,”Journal of Neuroengineering and Rehabilitation, vol. 6, no. 1, pp.20–35, 2009.

[3] S. Hesse, H. Schmidt, and C. Werner, “Machines to supportmotor rehabilitation after stroke: 10 years of experience inberlin.” Journal of Rehabilitation Research and Development, vol. 43,no. 5, pp. 671–678, 2005.

[4] S. Masiero, E. Carraro, C. Ferraro, P. Gallina, A. Rossi, andG. Rosati, “Upper limb rehabilitation robotics after stroke: aperspective from the university of padua, italy,” Journal ofRehabilitation Medicine, vol. 41, no. 12, pp. 981–985, 2009.

[5] D. J. Reinkensmeyer, J. A. Galvez, L. Marchal, E. T. Wolbrecht,and J. E. Bobrow, “Some key problems for robot-assistedmovement therapy research: a perspective from the universityof california at irvine,” in Rehabilitation Robotics, 2007. ICORR2007. IEEE 10th International Conference on, 2007, pp. 1009–1015.

[6] R. Shadmehr and F. A. Mussa-Ivaldi, “Adaptive representationof dynamics during learning of a motor task.” Journal ofNeuroscience, vol. 14, no. 5 Pt 2, pp. 3208–3224, 1994.

[7] R. A. Scheidt, D. J. Reinkensmeyer, M. A. Conditt, W. Z. Rymer,and F. A. Mussa-Ivaldi, “Persistence of motor adaptationduring constrained, multi-joint, arm movements.” Journal ofNeurophysiology, vol. 84, no. 2, pp. 853–862, 2000.

Page 13: Effects of kinesthetic and cutaneous stimulation during the learning …sirslab.dii.unisi.it/papers/2014/Rosati.ToHSI.2014.Haptics.Fin.pdf · Effects of kinesthetic and cutaneous

13

[8] D. J. Reinkensmeyer, J. L. Emken, and S. C. Cramer, “Post-strokeupper extremity rehabilitation: a review of robotic systems andclinical results,” Annual Review of Biomedical Engineering, vol. 6,pp. 497–525, 2004.

[9] D. Feygin, M. Keehner, and R. Tendick, “Haptic guidance:Experimental evaluation of a haptic training method for a per-ceptual motor skill,” in Haptic Interfaces for Virtual Environmentand Teleoperator Systems, 2002. HAPTICS 2002. Proceedings. 10thSymposium on, 2002, pp. 40–47.

[10] R. A. Scheidt, M. A. Conditt, E. L. Secco, and F. A. Mussa-Ivaldi, “Interaction of visual and proprioceptive feedbackduring adaptation of human reaching movements,” Journalof Neurophysiology, vol. 93, no. 6, pp. 3200–3213, 2005.

[11] D. Morris, H. Tan, F. Barbagli, T. Chang, and K. Salisbury,“Haptic feedback enhances force skill learning,” in EuroHapticsConference, 2007 and Symposium on Haptic Interfaces for VirtualEnvironment and Teleoperator Systems. World Haptics 2007. SecondJoint, 2007, pp. 21–26.

[12] D. Prattichizzo, C. Pacchierotti, and G. Rosati, “Cutaneous forcefeedback as a sensory subtraction technique in haptics,” IEEETransactions on Haptics, vol. 5, no. 4, pp. 289–300, 2012.

[13] C. Pacchierotti, A. Tirmizi, and D. Prattichizzo, “Improvingtransparency in teleoperation by means of cutaneous tactileforce feedback,” ACM Transactions on Applied Perception. In Press,2014.

[14] I. Birznieks, P. Jenmalm, A. Goodwin, and R. Johansson,“Encoding of direction of fingertip forces by human tactileafferents,” The Journal of Neuroscience, vol. 21, no. 20, pp. 8222–8237, 2001.

[15] K. O. Johnson, “The roles and functions of cutaneousmechanoreceptors,” Current Opinion in Neurobiology, vol. 11,no. 4, pp. 455–461, 2001.

[16] V. Hayward, O. Astley, M. Cruz-Hernandez, D. Grant, andG. Robles-De-La-Torre, “Haptic interfaces and devices,” SensorReview, vol. 24, no. 1, pp. 16–29, 2004.

[17] B. B. Edin and N. Johansson, “Skin strain patterns providekinaesthetic information to the human central nervous system.”The Journal of Physiology, vol. 487, no. Pt 1, pp. 243–251, 1995.

[18] J. Gordon and C. Ghez, “Muscle receptors and spinal reflexes:the stretch reflex,” in Principles of Neural Science, E. R. Kandel,J. H. Schwartz, and T. M. Jessell, Eds. Mcgraw-Hill, 2000, pp.564–580.

[19] D. Prattichizzo, F. Chinello, C. Pacchierotti, and M. Malvezzi,“Towards wearability in fingertip haptics: a 3-DoF wearabledevice for cutaneous force feedback,” IEEE Transactions onHaptics, vol. 6, no. 4, pp. 506–516, 2013.

[20] A. A. Stanley and K. J. Kuchenbecker, “Evaluation of tactilefeedback methods for wrist rotation guidance,” IEEE Transac-tions on Haptics, vol. 5, no. 3, pp. 240–251, 2012.

[21] B. C. Lee, S. Chen, and K. H. Sienko, “A wearable device forreal-time motion error detection and vibrotactile instructionalcuing,” IEEE Transactions on Neural Systems and RehabilitationEngineering, vol. 19, no. 4, pp. 374–381, 2011.

[22] J. Lieberman and C. Breazeal, “TIKL: Development of awearable vibrotactile feedback suit for improved human motorlearning,” IEEE Transactions on Robotics, vol. 23, no. 5, pp. 919–926, 2007.

[23] C. Pacchierotti, A. Tirmizi, G. Bianchini, and D. Prattichizzo,“Improving transparency in passive teleoperation by combiningcutaneous and kinesthetic force feedback,” in Proc. IEEE/RSJInt. Symp. Intelligent Robots and Systems, 2014.

[24] C. Pacchierotti, F. Chinello, M. Malvezzi, L. Meli, and D. Prat-tichizzo, “Two finger grasping simulation with cutaneousand kinesthetic force feedback,” in Haptics: Perception, Devices,Mobility, and Communication. Eurohaptics, ser. Lecture Notes inComputer Science, 2012, pp. 373–382.

[25] S. Cenci, G. Rosati, D. Zanotto, F. Oscari, and A. Rossi, “Firsttest results of a haptic tele-operation system to enhance stabilityof telescopic handlers,” in Proceedings of the ASME 2010 10thBiennial Conference on Engineering Systems Design and AnalysisESDA2010, Istanbul, Turkey, July 12-14 2010.

[26] K. Park, B. Kim, and S. Hirai, “Development of a soft-fingertipand its modeling based on force distribution,” in Proc. of IEEEInternational Conference on Robotics and Automation, vol. 3, 2003,pp. 3169–3174.

[27] E. Serina, C. Mote et al., “Force response of the fingertip pulpto repeated compression–effects of loading rate, loading angle

and anthropometry,” Journal of Biomechanics, vol. 30, no. 10, pp.1035–1040, 1997.

[28] A. B. Vallbo, R. Johansson et al., “Properties of cutaneousmechanoreceptors in the human hand related to touch sensa-tion,” Human Neurobiology, vol. 3, no. 1, pp. 3–14, 1984.

[29] R. S. Johansson, “Tactile sensibility in the human hand: re-ceptive field characteristics of mechanoreceptive units in theglabrous skin area.” The Journal of Physiology, vol. 281, p. 101,1978.

[30] F. Oscari, R. Secoli, F. Avanzini, G. Rosati, and D. J. Reinkens-meyer, “Substituting auditory for visual feedback to adapt toaltered dynamic and kinematic environments during reaching,”Experimental Brain Research, vol. 221, no. 1, pp. 33–41, 2012.

[31] G. Rosati, F. Oscari, S. Spagnol, F. Avanzini, and S. Masiero,“Effect of task-related continuous auditory feedback duringlearning of tracking motion exercises,” Journal of NeuroEngi-neering and Rehabilitation, vol. 9, no. 79, 2012.

[32] T. Hunter, P. Sacco, M. Nitsche, and D. Turner, “Modulationof internal model formation during force field-induced motorlearning by anodal transcranial direct current stimulation ofprimary motor cortex,” The Journal of Physiology, vol. 587, no. 12,pp. 2949–61, 2009.

[33] R. Osu, E. Burdet, D. Franklin, T. Milner, and M. Kawato,“Different mechanisms involved in adaptation to stable andunstable dynamics,” Journal of Neurophysiology, vol. 90, no. 5,pp. 3255–69, 2003.

[34] O. Bock and M. Thomas, “Proprioception plays a different rolefor sensorimotor adaptation to different distortions,” HumanMovement Science, vol. 30, pp. 415–423, 2011.

[35] G. Jansson and L. Monaci, “Identification of real objectsunder conditions similar to those in haptic displays: providingspatially distributed information at the contact areas is moreimportant than increasing the number of areas,” Virtual Reality,vol. 9, no. 4, pp. 243–249, 2006.

[36] A. Frisoli, M. Solazzi, F. Salsedo, and M. Bergamasco, “A fin-gertip haptic display for improving curvature discrimination,”Presence: Teleoperators and Virtual Environments, vol. 17, no. 6,pp. 550–561, 2008.

[37] D. Prattichizzo, F. Chinello, C. Pacchierotti, and K. Minamizawa,“Remotouch: A system for remote touch experience,” in Proc.of IEEE International Workshop on Robot and Human InteractiveCommunication, 2010, pp. 676–679.

[38] A. M. Okamura, “Methods for haptic feedback in teleoperatedrobot-assisted surgery,” Industrial Robot: An International Journal,vol. 31, no. 6, pp. 499–508, 2004.

[39] S. Okamoto, M. Konyo, and S. Tadokoro, “Vibrotactile stimuliapplied to finger pads as biases for perceived inertial andviscous loads,” EEE Transaction on Haptics, vol. 4, no. 4, pp.307–315, Oct. 2011.

[40] C. Zilles and J. Salisbury, “A constraint-based god-objectmethod for haptic display,” in Proc. of IEEE/RSJ InternationalConference on Intelligent Robots and Systems, vol. 3, 1995, pp.146–151.

[41] D. M. Wolpert, Z. Ghahramani, and M. I. Jordan, “Are armtrajectories planned in kinematic or dynamic coordinates? anadaptation study,” Experimental Brain Research, vol. 103, pp.460–470, 1995.

[42] E. Hwang, M. Smith, and R. Shadmehr, “Dissociable effects ofthe implicit and explicit memory systems on learning controlof reaching,” Experimental Brain Research, vol. 173, no. 3, pp.425–437, 2006.

[43] A. Melendez-Calderon, L. Masia, R. Gassert, G. Sandini, andE. Burdet, “Force field adaptation can be learned using visionin the absence of proprioceptive error,” IEEE Transactions onNeural Systems and Rehabilitation Engineering, vol. 19, no. 3, pp.298–306, 2011.

[44] F. R. Sarlegna, N. Malfait, L. Bringoux, C. Bourdin, and J.-L.Vercher, “Force-field adaptation without proprioception: Canvision be used to model limb dynamics?” Neuropsychologia,vol. 48, pp. 60–67, Sep. 2009.

[45] S. E. Criscimagna-Hemminger, A. J. Bastian, and R. Shadmehr,“Size of error affects cerebellar contributions to motor learning,”Journal of Neurophysiology, vol. 103, no. 4, pp. 2275–2284, 2010.

[46] J.-J. O. de Xivry, S. E. Criscimagna-Hemminger, and R. Shad-mehr, “Contributions of the motor cortex to adaptive controlof reaching depend on the perturbation schedule,” CerebralCortex, vol. 21, no. 7, pp. 1475–1484, 2011.

Page 14: Effects of kinesthetic and cutaneous stimulation during the learning …sirslab.dii.unisi.it/papers/2014/Rosati.ToHSI.2014.Haptics.Fin.pdf · Effects of kinesthetic and cutaneous

14

Giulio Rosati received the MS degree withhonors in mechanical engineering from theUniversity of Padua, Italy, in 1999, and thePhD degree in Machine Mechanics from theUniversity of Brescia, Italy, 2003. Associateprofessor (since 2006) and assistant profes-sor (2004-2006) of Machine Mechanics at theUniversity of Padua, Italy. Visiting professorat the University of California at Irvine in2007. His research interests include haptics,robotics, industrial automation and medical

and rehabilitation robotics. He is a member of the Faculty Board ofthe Doctoral School in mechatronics engineering of the University ofPadua, Italy (since 2007).

Fabio Oscari received the Master degreemagna cum laude in 2008 in MechanicalEngineering and the PhD degree in 2012from the University of Padua, Italy. He partic-ipated to research projects regarding human-machine interaction systems for industrial(machine stability, flexible automatic systems)and medical applications (multimodal feed-back, post-stroke rehabilitation robots, teler-obotic systems). He involved in a researchactivity regarding the investigation on the role

of the auditory feedback in motor control and motor learning, atthe Dept. of Mechanical and Aerospace Engineering, University ofCalifornia Irvine (UCI) in 2010. He is currently a post-doc fellow atDepartment of Management and Engineering (Padua).

Claudio Pacchierotti (S’12) received theM.S. degree cum laude in Computer Engi-neering in 2011 from the University of Siena,Italy. He was an exchange student at theKarlstad University, Sweden in 2010. He iscurrently a Ph.D. student at the Dept. ofInformation Engineering and Mathematics ofthe University of Siena and at the Dept. ofAdvanced Robotics of the Italian Institute ofTechnology. His research interests includerobotics and haptics, focusing on cutaneous

feedback techniques, wearable devices, and haptics in roboticsurgery.

Domenico Prattichizzo (S’93 - M’95) re-ceived the M.S. degree in Electronics Engi-neering and the Ph.D. degree in Robotics andAutomation from the University of Pisa in 1991and 1995, respectively. Since 2002 AssociateProfessor of Robotics at the University ofSiena. Since 2009 Scientific Consultant atIstituto Italiano di Tecnologia, Genova Italy. In1994, Visiting Scientist at the MIT AI Lab. Co-author of the Grasping chapter of Handbookof Robotics Springer, 2008, awarded with two

PROSE Awards presented by the American Association of Publishers.Since 2007 Associate Editor in Chief of the IEEE Trans. on Haptics.From 2003 to 2007, Associate Editor of the IEEE Trans. on Roboticsand IEEE Trans. on Control Systems Technologies. Vice-chair forSpecial Issues of the IEEE Technical Committee on Haptics (2006-2010). Chair of the Italian Chapter of the IEEE RAS (2006-2010),awarded with the IEEE 2009 Chapter of the Year Award. Co-editor oftwo books by STAR, Springer Tracks in Advanced Robotics, Springer(2003, 2005). Research interests are in haptics, grasping, visualservoing, mobile robotics and geometric control. Author of more than200 papers in these fields.