hybrid inertial-robotic motion tracking for posture biofeedback … · 2018-09-14 · hybrid...

6
Hybrid Inertial-Robotic Motion Tracking for Posture Biofeedback in Upper Limb Rehabilitation* Arne Passon 1 , Thomas Schauer 1 , and Thomas Seel 1 Abstract— Rehabilitation robotics and neuromuscular stim- ulation have become widespread technologies for rehabilitation training of stroke and spinal cord injured patients. In this context, real-time tracking of the performed motion facilitates real-time control of the motion support and biofeedback about undesired compensatory motions. We consider a cable-driven robotic system for upper limb rehabilitation and extend it by two wearable inertial sensors. By sensor fusion of the robotic and inertial measurements, we obtain accurate estimates of the forearm and upper arm orientation and position, which cannot be obtained by either of both measurement systems alone. A real-time biofeedback is introduced to prevent undesired compensatory motions of the trunk and shoulder. The proposed methods are evaluated with respect to an optical reference system in a series of experimental trials with and without compensatory motions. Using only the robotic sensors yields average measurement errors of up to 24 cm for the shoulder position and 19 for the elbow angle. In contrast, the proposed hybrid sensor fusion achieves accuracies better than 6 cm and 4 , respectively. I. INTRODUCTION Stroke or spinal cord injury can lead to paresis of the upper limb [1], [2] and, as a result, often severely handicap the patients for the rest of their lives. Rehabilitation training is important to improve the physical condition of the patient in general. Moreover, in the therapy of stroke patients, it can help to regain lost motor functions. In modern rehabilitation settings, the patient is often supported by rehabilitation robotics or Functional Electrical Stimulation (FES) [3], [4]. Such systems may support the performed motion actively, they may facilitate objective motion assessment, and they may provide biofeedback and gamification context in a vir- tual reality. For all these purposes, accurate and unrestrictive motion tracking is a fundamental requirement. For the upper limb, many different systems with various sensors are available or have been proposed. Optical tracking systems are often considered as the gold standard in the field of human motion measurements, but their setup is mostly too complex and their cost too high for rehabilita- tion environments [5]. Inertial Measurement Units (IMUs) facilitate real-time motion analysis with comparably good accuracy as shown by [6]–[8]. However, due to the presence of ferromagnetic material and non-homogeneous magnetic fields in indoor environments, the accuracy of IMU-based motion tracking in the horizontal plane is limited [9], [10]. *The work was partially conducted within the research project BeMobil, which is supported by the German Federal Ministry of Education and Research (BMBF) (FKZ16SV7069K). 1 Arne Passon, Thomas Schauer, and Thomas Seel are with Control Systems Group at TU Berlin, Einsteinufer 17 EN-11, 10587 Berlin {passon,schauer,seel}@control.tu-berlin.de The current contribution considers the Diego system (Ty- romotion GmbH, Austria), which is an active cable-driven rehabilitation robot, relieving the arm weight and thus fa- cilitating longer therapy sessions as well as motions that would otherwise be unachievable for patients with weak upper limb motor function. As shown in Figure 1, both ropes are attached to the forearm, one at the wrist, the other close to the elbow. The Diego provides real-time measurements of the position of both cuffs. However, the supination/pronation of the forearm, which is of high interest during FES support of hand opening and closing, cannot be determined. The Diego system is capable of using the cuff positions to estimate the orientation of the upper arm under the premise that the shoulder remains at a nominal shoulder position defined initially. In practice, however, patients sometimes compensate weakness of the upper arm or forearm by shoul- der motions [11]–[13]. Instead of extending the elbow, for example, the shoulder is moved forward to reach an object [14]. Such compensatory motions are highly undesirable from a rehabiliation point of view [15]. To avoid these IMUs Ropes Tracked Box Fig. 1. Experimental Setup including the tracked box and the rehabilitation robot Diego (Tyromotion GmbH, Austria) with both ropes attached to the forearm. The measurement system is extended by the two IMUs at the upper arm and forearm. 2018 7th IEEE International Conference on Biomedical Robotics and Biomechatronics (Biorob) Enschede, The Netherlands, August 26-29, 2018 978-1-5386-8182-4/18/$31.00 ©2018 IEEE 1163

Upload: others

Post on 15-May-2020

8 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Hybrid Inertial-Robotic Motion Tracking for Posture Biofeedback … · 2018-09-14 · Hybrid Inertial-Robotic Motion Tracking for Posture Biofeedback in Upper Limb Rehabilitation*

Hybrid Inertial-Robotic Motion Tracking for Posture Biofeedback inUpper Limb Rehabilitation*

Arne Passon1, Thomas Schauer1, and Thomas Seel1

Abstract— Rehabilitation robotics and neuromuscular stim-ulation have become widespread technologies for rehabilitationtraining of stroke and spinal cord injured patients. In thiscontext, real-time tracking of the performed motion facilitatesreal-time control of the motion support and biofeedback aboutundesired compensatory motions. We consider a cable-drivenrobotic system for upper limb rehabilitation and extend it bytwo wearable inertial sensors. By sensor fusion of the roboticand inertial measurements, we obtain accurate estimates of theforearm and upper arm orientation and position, which cannotbe obtained by either of both measurement systems alone.A real-time biofeedback is introduced to prevent undesiredcompensatory motions of the trunk and shoulder. The proposedmethods are evaluated with respect to an optical referencesystem in a series of experimental trials with and withoutcompensatory motions. Using only the robotic sensors yieldsaverage measurement errors of up to 24 cm for the shoulderposition and 19◦ for the elbow angle. In contrast, the proposedhybrid sensor fusion achieves accuracies better than 6 cm and4◦, respectively.

I. INTRODUCTION

Stroke or spinal cord injury can lead to paresis of theupper limb [1], [2] and, as a result, often severely handicapthe patients for the rest of their lives. Rehabilitation trainingis important to improve the physical condition of the patientin general. Moreover, in the therapy of stroke patients, it canhelp to regain lost motor functions. In modern rehabilitationsettings, the patient is often supported by rehabilitationrobotics or Functional Electrical Stimulation (FES) [3], [4].Such systems may support the performed motion actively,they may facilitate objective motion assessment, and theymay provide biofeedback and gamification context in a vir-tual reality. For all these purposes, accurate and unrestrictivemotion tracking is a fundamental requirement.

For the upper limb, many different systems with varioussensors are available or have been proposed. Optical trackingsystems are often considered as the gold standard in thefield of human motion measurements, but their setup ismostly too complex and their cost too high for rehabilita-tion environments [5]. Inertial Measurement Units (IMUs)facilitate real-time motion analysis with comparably goodaccuracy as shown by [6]–[8]. However, due to the presenceof ferromagnetic material and non-homogeneous magneticfields in indoor environments, the accuracy of IMU-basedmotion tracking in the horizontal plane is limited [9], [10].

*The work was partially conducted within the research project BeMobil,which is supported by the German Federal Ministry of Education andResearch (BMBF) (FKZ16SV7069K).

1Arne Passon, Thomas Schauer, and Thomas Seel are with ControlSystems Group at TU Berlin, Einsteinufer 17 EN-11, 10587 Berlin{passon,schauer,seel}@control.tu-berlin.de

The current contribution considers the Diego system (Ty-romotion GmbH, Austria), which is an active cable-drivenrehabilitation robot, relieving the arm weight and thus fa-cilitating longer therapy sessions as well as motions thatwould otherwise be unachievable for patients with weakupper limb motor function. As shown in Figure 1, both ropesare attached to the forearm, one at the wrist, the other closeto the elbow. The Diego provides real-time measurements ofthe position of both cuffs. However, the supination/pronationof the forearm, which is of high interest during FES supportof hand opening and closing, cannot be determined.

The Diego system is capable of using the cuff positions toestimate the orientation of the upper arm under the premisethat the shoulder remains at a nominal shoulder positiondefined initially. In practice, however, patients sometimescompensate weakness of the upper arm or forearm by shoul-der motions [11]–[13]. Instead of extending the elbow, forexample, the shoulder is moved forward to reach an object[14]. Such compensatory motions are highly undesirablefrom a rehabiliation point of view [15]. To avoid these

IMUs

Ropes

TrackedBox

Fig. 1. Experimental Setup including the tracked box and the rehabilitationrobot Diego (Tyromotion GmbH, Austria) with both ropes attached to theforearm. The measurement system is extended by the two IMUs at the upperarm and forearm.

2018 7th IEEE International Conference on BiomedicalRobotics and Biomechatronics (Biorob)Enschede, The Netherlands, August 26-29, 2018

978-1-5386-8182-4/18/$31.00 ©2018 IEEE 1163

Page 2: Hybrid Inertial-Robotic Motion Tracking for Posture Biofeedback … · 2018-09-14 · Hybrid Inertial-Robotic Motion Tracking for Posture Biofeedback in Upper Limb Rehabilitation*

motions and to raise the patient’s awareness, biofeedbackstrategies have been proposed [16], including e.g. a musicphrase and using its volume to indicate the amount ofcompensation as described in [17].

Due to the described shortcomings of the rope-basedmeasurements, we propose an extended setup with IMUsadded to the upper arm and forearm. We develop a sensorfusion method that compensates the vulnerability of IMUs tomagnetic disturbances and yields full position and orientationinformation of the upper limb. This is achieved by deter-mining a pseudo-magnetometer signal from the measuredcuff positions. Finally, we propose a biofeedback strategyfor prevention of compensatory shoulder movements.

The remainder of this paper is organized as follows. InSection II, we present the setup and propose the methodsfor sensor fusion, shoulder position estimation and detectionof compensatory motions. The proposed methods are thenevaluated in Section III. A series of experimental trials isanalyzed in which a healthy subject tracks a rectangularpath with and without compensatory shoulder movements,hereinafter also termed compensatory movements (comp.mov.) and proper movements (proper mov.) respectively.The contribution ends with a conclusion in Section IV.

II. METHODS

A. Hardware Setup and Measurement Task

1) Rehabilitation Robotics Diego: The robotic systemDiego is depicted in Figures 1 and 2. It supports the weight ofthe arm by means of two actuated ropes, which are connectedto cuffs at the distal and proximal ends of the forearm.Spherical angles and the length of the rope are measuredby the Diego. Thus, real-time estimates of the position ofboth cuffs are available. The measurement is drift-free, butat a low sampling frequency (25 Hz) and noisy due to themeasurement principle (small changes of the rope’s sphericalangles result in large changes of the cuff position). Resultssolely by the robotic system are hereinafter also termedrobotic measurement (rob.).

2) Inertial Measurement Units (IMUs): Two wearableIMUs (100 Hz, MTx

TM, Xsens, Netherlands) are attached to

the upper arm and the forearm, as depicted in the Figures 1and 2. It is assumed that the orientation between IMU andforearm/upper arm is known either by careful attachment orby using methods that automatically determine this infor-mation from arbitrary movements of the arm [6], [18]. TheIMUs yield three-dimensional measurements of acceleration,angular rate and the magnetic field vector in their ownintrinsic coordinate system. Prior to each session, they arecalibrated to reduce the gyroscope biases to values below0.2 degrees per second.

3) Measurement task: Using the described measurements,we want to determine the orientations of upper arm andforearm as well as the positions of the shoulder, elbow, andwrist in real-time.

R

Rx

Ry

Rz

A

U

F I

driftingslowly

W

P

E

S

A . . . auxiliary CS

U . . . upper arm IMU CS

F . . . forearm IMU CS

I . . . inertial ref. CS

R . . . roboticscoord. system (CS) W . . . wrist

P . . . proximal cuff

E . . . elbow

S . . . shoulder

robotic system(rope actuator)

Fig. 2. Kinematic model of a patient performing a pick-and-place task,the inertial sensor frames, and the joint positions in robotics workspace.

B. Coordinate Systems and Notation

The robotic workspace coordinate system {Rx,Ry,Rz}is defined with the y-axis Ry pointing straight forward andthe z-axis Rz straight up, as illustrated in Figure 2. Theintrinsic measurement coordinate systems {Fx,Fy,Fz} ofthe forearm IMU and {Ux, Uy, Uz} of the upper arm IMUare characterized by the x-axes Fx and Ux being parallel tothe longitudinal (proximal-distal) axis of the forearm andupper arm, respectively. The reference coordinate system{Ix, Iy, Iz} of the inertial sensor fusion is determined bythe vertical z-axis Iz.

Upper and lower left indices are used to describe towhich coordinate system a vector is attached and in whichcoordinate systems it is described. For example, FRx is the x-axis of the forearm IMU (F) in coordinates of the workspaceframe (R). Trivially, FFx = R

Rx = [1, 0, 0]ᵀ and so on.Likewise, FRM is the rotation matrix that takes any vectorfrom F-coordinates toR-coordinates. The operator [·]normalizedenotes normalization of a vector to unit length.

C. Measurement of the Forearm Orientation

1) Sensor fusion approach: The positions RpW and RpP

of the cuffs at the wrist and at the proximal end of theforearm, respectively, are measured by the robot. The ori-entation of the forearm cannot be entirely determined fromthese positions since rotation around the longitudinal axis

1164

Page 3: Hybrid Inertial-Robotic Motion Tracking for Posture Biofeedback … · 2018-09-14 · Hybrid Inertial-Robotic Motion Tracking for Posture Biofeedback in Upper Limb Rehabilitation*

(supination/pronation) of the forearm leaves the positionsunchanged. By attaching an IMU to the forearm, we aimat overcoming this drawback as well as the low samplingrate and noise of the rope-based measurements. Determiningthe rotation matrix I

FM from the measured accelerations,angular rates and magnetic field vectors is a standard sen-sor fusion problem in inertial motion tracking. It can besolved, for example by a recently developed quaternion-based algorithm [9]. However, the magnetic field in indoorenvironments is not homogeneous and, moreover, disturbedby magnets in the cuffs of the Diego. Thus, we refrainfrom using magnetometer readings in the IMU sensor fusion.Instead of this, the robotic measurements are used to generatea pseudo-magnetometer signal that provides the missingheading information. This signal must be the F-coordinatesof a vector that always points into the same horizontal direc-tion of the fixed workframe coordinate system. Hereinafter,results by the sensor fusion approach are also termed hybridmeasurement (hyb.).

2) Mathematical solution: In order to calculate the de-sired pseudo-magnetometer signal, we aim at transformingRx into the measurement frame F of the IMU. Note that thedirect transformation from R to F is not known. Instead,we define an auxiliary coordinate system with vertical z-axis Az and an x-axis Ax that is aligned with the horizontalprojection of the longitudinal forearm axis, cf. Figure 2. Theaxes of A in coordinates of R and F are found by thefollowing constructions:

ARx =

[(RpW − RpP )− RRzT (RpW − RpP )

RRz]

normalize ,ARz = RRz , (1)ARy = ARz× ARx

and

AFx = [FFx− AFzT FFxAFz]normalize ,AFz = I

FMIIz , (2)

AFy = AFz× AFx .

We then obtain the pseudo-magnetometer signal

RFx = [AFx,

AFy,

AFz][

ARx,

ARy,

ARz]−1︸ ︷︷ ︸

RFM

RRx (3)

and provide it to the sensor fusion algorithm [9],which uses gyroscope-based predictions and accelerome-ter/magnetometer-based corrections to obtain the forearmorientation in real-time.

D. Measurement of the Wrist Position

A precise measurement of the wrist position is crucial inmany application scenarios, e.g. reaching, touching, grasp-ing, and similar movements that are recorded or supportedby the rehabilitation system. An estimate of the wrist positionis already given by the rope-based cuff position measurementRpW provided by the Diego system. Recall that this wristposition estimate is provided at a low sampling rate and isnoisy, as it is calculated from the rope-based spherical angles

measurements. Note as well that the forearm IMU measuresthe acceleration FaF of the forearm and that

FRMFaF − 9.8RRz (4)

yields the second time derivative of the wrist position. There-fore, we use a linear Kalman filter to fuse this accelerationmeasurement with the obtained rope-based position. Thisapproach yields a high sampling frequency and low-noiseestimate RpW,fused of the wrist position.

E. Measurement of Upper Arm OrientationAs in Section II-C, the IMUs are again used to directly

measure the segment orientations, here the ones of the upperarm. Recall that the magnetic field in indoor environmentsis not homogeneous and that we refrain from using mag-netometer readings. Combining only the accelerometer andgyroscope readings would yield an accurate inclination (rolland pitch) and heading of the upper arm. However, due toresidual gyroscope biases, the heading would drift at aboutten degrees per minute. Therefore, we employ the pseudo-magnetometer strategy from Section II-C.1 with a headingthat is determined from a rope-based estimation of the elbowposition and the nominal shoulder position.

The time constants of the sensor fusion algorithm [9] areset to assure that the gyroscope is trusted on timescales belowone minute and that only the residual drift is removed by thepseudo-magnetometer information. Therefore, the accuracyof the resulting upper arm heading remains high unless thetrue shoulder position deviates strongly from the nominalposition for more than half a minute.

1) Detection of Compensatory Motion and Biofeedback:If the user performs compensatory motions, then the reha-bilitation task is carried out in an uneffective and potentiallyharmful way. To prevent this situation, we propose to esti-mate the current shoulder position

RpS = RpW,fused − URM UUxlu −

FRM

FFxlf , (5)

where lu and lf are the lengths of the upper arm andforearm, and to perform a real-time monitoring whether RpS

leaves an acceptable region (e.g. ±10 cm, see Section III-B) around the nominal shoulder position for e.g. more thanthree seconds. Whenever such a compensation movement isdetected, the patient is asked to return to the original shoulderposition and to restart the exercise.

This biofeedback does not only assure that the patientperforms the motions properly. It also assures that we obtaina persistently accurate estimate of the upper arm orientationand shoulder position.

2) Elbow Flexion/Extension Angle: Using the upper armand forearm orientations, the flexion/extension (F/E) angleof the elbow joint is calculated by methods presented in [6],[18]. Note that the Diego system itself can also determinethe elbow F/E angle from the cuff positions, but it uses theassumption that the shoulder position is fixed. Therefore,that elbow angle measurement will be wrong whenever acompensation movement is present. In Section III-B, wewill compare this robotics-based elbow angle with the hybridIMU-and-robotics estimates.

1165

Page 4: Hybrid Inertial-Robotic Motion Tracking for Posture Biofeedback … · 2018-09-14 · Hybrid Inertial-Robotic Motion Tracking for Posture Biofeedback in Upper Limb Rehabilitation*

Fig. 3. Experimental Procedure: On the left side parts of a proper movement (proper mov.) are shown, whereas on the right side parts of a trial withcompensatory movements (comp. mov.) can be seen. The shoulder motion is highlighted in green and the arm orientations in blue.

III. EXPERIMENTAL VALIDATION

A. Experimental Setting and Procedure

A healthy subject is seated in front of the Diego asshown in Figure 1. A box-shaped object (aluminum suitcase)is placed on a table in front of the subject, so that thesubject is able to roughly track the rectangular path alongthe upper edge of the box without moving the trunk. Theropes and IMUs are attached as depicted in Figures 1 and 2.A weight relief of 6 N is chosen for each rope to guaranteetaut ropes, which reduces position measurement errors. Thesubject takes neutral pose while the experimenter defines thecurrent shoulder position as the nominal shoulder position.

Three trials are performed during which the wrist is movedapproximately along the rectangular path in counterclock-wise direction starting and ending at the right front (furtheraway) corner. Each trial is performed with the subject tryingto leave the shoulder near the nominal position and movingonly the upper arm and forearm as shown in the left pictureof Figure 3. Subsequently, the series of trials is repeatedwith the subject using almost no shoulder and elbow flexionbut instead almost only compensatory trunk and shouldermotions to follow the desired trajectory as can be seen onthe right picture of Figure 3.

B. Experimental Results

The methods proposed in Section II are applied to datarecorded during trials as described above. The results arepresented in Figures 4-6 as well as in Tables I-II. Figure 4shows the obtained estimate of the wrist position in thehorizontal plane during one trial. As desired (see Section II-D), the estimate is less noisy than the wrist position of therobotics and at the same time has a close tracking of theoriginal robotics data. The deviations from the tracked pathare a result of the imprecise tracking behavior of the subject.

The shoulder position is estimated using the upper arm andforearm orientations and the wrist position (see Section II-E).It is compared to the position determined by a camera thatwas positioned above the subject whose shoulder was markedby an adhesive label. The recorded video was evaluated using

the free software application Kinovea. For one exemplarytrial, Figure 5 shows the measured positions in the horizontalplane and over time. The fused estimate resembles the opticalreference (opt. ref.).

The average deviation of the estimated shoulder positionfrom the opt. ref. during an exemplary trial remained below6 cm for movements with compensation (comp. mov.) andbelow 3 cm for proper movements (proper mov.). Thecovered maximal Range of Motion (ROM) of the shoulderin the horizontal plane is defined as the longest diagonalbetween the edges of the movements of both series. Table Ishows that the percentage average deviation in dependencyof the ROM during a total of three performed trials remainedunder 10 % for both series of trials.

The measured elbow angles are shown in Figure 6 for one

−20 −10 0 10 20 30

0

10

20

30

40

X [cm]

Y[c

m]

Rob.Kalman-filteredTracked path

Fig. 4. Exemplary wrist position during one trial in the horizontal planeincluding the noisy and at a low sampling frequency estimate from therobotics (Rob.) as well as the less noisy and at a high sampling frequencyKalman-filtered result (fusion of the robotic position estimate with theacceleration measurement of the forearm IMU). The shown tracked pathis the manually measured real shape of the box.

1166

Page 5: Hybrid Inertial-Robotic Motion Tracking for Posture Biofeedback … · 2018-09-14 · Hybrid Inertial-Robotic Motion Tracking for Posture Biofeedback in Upper Limb Rehabilitation*

−30 −20 −10 0 10 20

−10

0

10

20

30

X [cm]

Y[c

m]

Opt. ref. Hyb.Proper mov.Comp. mov.

-20

0

20

X,Y

[cm

]

PROPER MOV. COMP. MOV.

0 50

20

40

||·|| 2

[cm

]

Deviation (Hyb.)Deviation (Rob.)Average (Hyb.)Average (Rob.)

Time [s]325 330

Fig. 5. Exemplary shoulder motion during one trial carried out with compensatory trunk movements (comp. mov.) and another without (proper mov.).Both are shown for the result from the optical reference (opt. ref.) and the one from the hybrid estimation (hyb.). On the left side, it is depicted in thehorizontal plane and on the right side the components are shown over time (X: solid, Y: dashed). Additionally the deviation of the estimate (hyb.) fromthe optical reference is depicted over time together with its average as well as the resulting deviation and its average of a supposed constant shoulderposition by the robotics (rob.). A potentially tolerated compensatory shoulder motion range of ±10 cm is shown as a gray circle in the horizontal planerespectively a band in the time chart.

TABLE IDEVIATIONS (IN CM) OF THE SHOULDER MOTION ESTIMATION FROM

THE OPT. REF. FOR A SERIES WITH COMPENSATORY TRUNK MOVEMENTS

AND ONE WITHOUT (EACH OF A TOTAL OF THREE TRIALS), WHERE THE

PERCENTAGE AVERAGES ARE IN DEPENDENCY OF THE COVERED

MAXIMAL ROM OF AROUND 60 CM. THE ROBOTIC MEASUREMENT

(ROB.) SUPPOSES A CONSTANT SHOULDER POSITION.

Type Method Average Maximumabs [cm] % abs [cm]

Proper mov. Rob. 7.82 13 16.59Hyb. 2.52 4 6.78

Comp. mov. Rob. 23.93 39 36.25Hyb. 5.40 9 10.34

exemplary trial. The hyb. estimate resembles the approxi-mated optical reference even during comp. mov.. Table IIshows that the percentage average deviation from the ROM(here the maximal covered range of elbow flexion) during atotal of three performed trials remained under 4 % for bothseries of trials, whereas the rob. measurement result fallsof to a percentage average deviation of up to 18 % duringcomp. mov..

C. Discussion

The results showed that the linear Kalman filter yieldsa low-noise wrist position at a high sampling frequency. Byusing this wrist position and the estimated orientations of thearm, it is possible to obtain an average tracking deviationof the shoulder position of less than six centimeters. Thedata of the trial during which the subject was trying tokeep the shoulder position still reveals that even with goodposture control, shoulder motions of up to ten centimetersmay occur when moving the wrist by half a meter. Therefore,

it seems reasonable to tolerate such motions and to aim fordetection of shoulder motions that deviate more than ten orfifteen centimeter from the nominal position, as indicated inFigure 5.

With errors below three (proper mov.) and six (comp.mov.) centimeters, the proposed method is clearly accurateenough to determine in real time whether the shoulder leavessuch an acceptable region around the nominal shoulder

0

50

100

Elb

owan

gle

[◦]

PROPER MOV. COMP. MOV.

Opt. ref. Rob. Hyb.

0 50

10

20

30

40

Dev

iatio

n[◦

] Hyb.Rob.Average (Hyb.)Average (Rob.)

Time [s]325 330

Fig. 6. Exemplary elbow angle during one trial carried out with compen-satory trunk movements (comp. mov. (red)) and another without (propermov. (blue)). Both are shown for the result from the approximated opticalreference (opt. ref.), the one from the hybrid estimation (hyb.) and therobotics (rob.).

1167

Page 6: Hybrid Inertial-Robotic Motion Tracking for Posture Biofeedback … · 2018-09-14 · Hybrid Inertial-Robotic Motion Tracking for Posture Biofeedback in Upper Limb Rehabilitation*

position. Therefore, the proposed setup and methods aresuitable for real-time biofeedback of compensatory shouldermotions.

TABLE IIDEVIATIONS (IN DEGREE) OF THE ELBOW ANGLE ESTIMATION FROM

THE APPROXIMATED OPT. REF. FOR A SERIES WITH COMPENSATORY

TRUNK MOVEMENTS AND ONE WITHOUT (EACH OF A TOTAL OF THREE

TRIALS), WHERE THE PERCENTAGE AVERAGES ARE IN DEPENDENCY OF

THE COVERED MAXIMAL ROM OF AROUND 109◦ .

Type Method Average Maximumabs [◦] % abs [◦]

Proper mov. Rob. 3.99 4 15.1Hyb. 3.78 3 9.86

Comp. mov. Rob. 19.36 18 40.28Hyb. 3.75 3 11.40

The proposed sensor fusion provides as well accurateelbow angles, as the errors remain below four degree evenin the case the of comp. movements. Using the biofeedbackfor comp. mov. to achieve only temporary strong deviationsof the nominal shoulder position, it is possible to assure thatwe persistently obtain accurate elbow angles. Consequently,using hyb. elbow angles combined with a biofeedback forcomp. mov., an active support of elbow flexion/extensionby FES is realizable.

IV. CONCLUSIONS

A method that allows the determination of the upper limbjoint angles by a sensor fusion of a robotic system andwearable IMUs was developed. It combines the advantagesof both systems and compensates their shortcomings. On theone hand, accurate heading information is obtained withoutusing magnetometers, i.e. without relying on a homogeneousmagnetic field. On the other hand, the low sampling fre-quency and the noise of the rope-based measurements of theDiego system are improved by sensor fusion with the inertialmeasurements, and additional degrees of freedom, such asforearm supination/pronation are tracked. This facilitates thecombination of the Diego system with active hand motionsupport systems, such as the neuroprosthesis devoloped in[8].

The developed algorithms yield an adequate estimation ofthe actual shoulder position. Thus, it is possible to performrehabilitation sessions with an automatic posture biofeedbackthat helps to avoid compensatory trunk and shoulder motions.In practice, the feedback might for example be implementedas a music phrase indicating the amount of compensationby its volume as proposed in [17]. Vibrotactile or visualfeedback represent alternative solutions, of which the latterone seems especially practical if the therapy already includesa virtual reality.

REFERENCES

[1] D. Popovic and T. Sinkjaer, Control of Movement for the PhysicallyDisabled, 1st ed. Springer, 2000.

[2] C. Gowland, H. deBruin, J. V. Basmajian, N. Plews, and I. Burcea,“Agonist and Antagonist Activity During Voluntary Upper-LimbMovement in Patients with Stroke,” Physical Therapy, vol. 72, no. 9,pp. 624–633, 1992.

[3] J. McCabe, M. Monkiewicz, J. Holcomb, S. Pundik, and J. J. Daly,“Comparison of robotics, functional electrical stimulation, and motorlearning methods for treatment of persistent upper extremity dysfunc-tion after stroke: Arandomizedcontrolled trial,” Archives of PhysicalMedicine and Rehabilitation, vol. 96, no. 6, pp. 981 – 990, 2015.

[4] L. Oujamaa, I. Relave, J. Froger, D. Mottet, and J.-Y. Pelissier,“Rehabilitation of arm function after stroke. literature review,” Annalsof Physical and Rehabilitation Medicine, vol. 52, no. 3, pp. 269 – 293,2009.

[5] H. Zhou and H. Hu, “Human motion tracking for rehabilitationasurvey,” Biomedical Signal Processing and Control, vol. 3, no. 1, pp.1 – 18, 2008.

[6] D. Laidig, P. Muller, and T. Seel, “Automatic anatomical calibrationfor imu-based elbow angle measurement in disturbed magnetic fields,”Current Directions in Biomedical Engineering, vol. 3, no. 2, p. 167170,2017.

[7] A. Filippeschi, N. Schmitz, M. Miezal, G. Bleser, E. Ruffaldi, andD. Stricker, “Survey of motion tracking methods based on inertialsensors: A focus on upper limb human motion,” Sensors, vol. 17,no. 6, 2017.

[8] M. Valtin, C. Salchow, T. Seel, D. Laidig, and T. Schauer, “Modularfinger and hand motion capturing system based on inertial and mag-netic sensors,” Current Directions in Biomedical Engineering, vol. 3,no. 1, p. 1923, 2017.

[9] T. Seel and S. Ruppin, “Eliminating the Effect of Magnetic Distur-bances on the Inclination Estimates of Inertial Measurement Units,”in IFAC World Congress 2017 (submitted), 2017.

[10] D. Laidig, T. Schauer, and T. Seel, “Exploiting kinematic constraintsto compensate magnetic disturbances when calculating joint anglesof approximate hinge joints from orientation estimates of inertialsensors,” in Proc. of 15th IEEE Conference on Rehabilitation Robotics(ICORR), London, UK, 2017, p. 971976.

[11] M. F. Levin, S. M. Michaelsen, C. M. Cirstea, and A. Roby-Brami,“Use of the trunk for reaching targets placed within and beyond thereach in adult hemiparesis,” Experimental Brain Research, vol. 143,no. 2, pp. 171–180, Mar 2002.

[12] M. F. Levin, D. G. Liebermann, Y. Parmet, and S. Berman, “Compen-satory versus noncompensatory shoulder movements used for reachingin stroke,” Neurorehabilitation and Neural Repair, vol. 30, no. 7, pp.635–646, 2016, pMID: 26510934.

[13] W. Liu, S. M. Waller, T. Kepple, and J. Whitall, “Compensatory armreaching strategies after stroke: Induced position analysis,” J RehabilRes Dev, vol. 50, no. 1, pp. 71–84, 2013, 23516085[pmid].

[14] J. V. Robertson and A. Roby-Brami, “The trunk as a part of thekinematic chain for reaching movements in healthy subjects andhemiparetic patients,” Brain Research, vol. 1382, pp. 137 – 146, 2011.

[15] S. M. Michaelsen, R. Dannenbaum, and M. F. Levin, “Task-specifictraining with trunk restraint on arm recovery in stroke,” Stroke, vol. 37,no. 1, pp. 186–192, 2006.

[16] B. A. Valdes and H. F. M. V. der Loos, “Biofeedback vs. game scoresfor reducing trunk compensation after stroke: a randomized crossovertrial,” Topics in Stroke Rehabilitation, vol. 0, no. 0, pp. 1–18, 2017,pMID: 29078743.

[17] H. Huang, S. L. Wolf, and J. He, “Recent developments in biofeed-back for neuromotor rehabilitation,” Journal of NeuroEngineering andRehabilitation, vol. 3, no. 1, p. 11, Jun 2006.

[18] P. Muller, M. A. Begin, T. Schauer, and T. Seel, “Alignment-free, self-calibrating elbow angles measurement using inertial sensors,” IEEEJournal of Biomedical and Health Informatics, vol. PP, no. 99, p. 11,2016.

1168