gaming and social interactions in the rehabilitation of...

Chapter #

Gaming and Social Interactions in the Rehabilitation of Brain Injuries: A Pilot Study with the Nintendo Wii ConsoleR.C.V. Loureiro, D. Valentine, B. Lamperd, C. Collin and W.S. Harwin

AbstractPhysical rehabilitation of brain injuries and strokes is a time consuming and

costly process. Over the past decade several studies have emerged looking at the use of highly sophisticated technologies, such as robotics and virtual reality to tap into the needs of clinicians and patients. While such technologies can be a valuable tool to facilitate intensive movement practice in a motivating and engaging environment, success of therapy also depends on self-administered therapy beyond hospital stay. With the emergence of low-cost gaming consoles such as the Nintendo Wii, new opportunities arise for home-therapy paradigms centred on social interactions and values, which could reduce the sense of isolation and other depression related complications. In this paper we examine the potential, user acceptance and usability of an unmodified Nintendo Wii gaming console as a low-cost treatment alternative to complement current rehabilitation programmes.

IntroductionAlthough an increased effort is placed on the recovery process of patients

following a brain injury such as a stroke, economic pressures, and lack of available human resources means that patients generally do not reach their full recovery potential when discharged from hospital following initial rehabilitation (Broeks et al., 1999). Although there is already evidence suggesting that the damaged motor system is able to reorganise in the presence of motor practice, optimal training

2 R.C.V. Loureiro, D. Valentine, B. Lamperd, C. Collin and W.S. Harwin

methodologies promoting such reorganisation remain unclear due to discrepancies of current rehabilitation therapy, quantification of dosage and rehabilitation type (Shadmehr and Mussa-Ivaldi, 1994). The recovery of upper limb function is particularly affected as the initial challenge is to stabilise the trunk and relearn minimum independence levels through gait re-learning. Robotic machines have been identified as a possible way to automate labour-intensive training paradigms, to improve patient access to therapy and to provide new tools for therapists. Several authors have already proposed the use of robots for the delivery of this type of physiotherapy (Krebs et al., 1999; Lum et al., 1999; Johnson et al., 1999; Reinkensmeyer et al., 2000; Loureiro et al., 2003; Hesse et al., 2003; Nef and Riener, 2005). One of the challenges still present today, is to how best use robotic technology to augment the physiotherapist’s skills (Harwin et al., 2006). On the contrary to public perception, robotic technology aims to be an advanced tool available to the physiotherapist and not a replacement. A robotic system is very unlikely to be able to amass all the skills of a physiotherapist, but it will be very good at conducting comparatively simple repetitive and manually intensive therapies. In this context, the physiotherapist would be doing all the clinical decisions and when suitable, considered and executed on the robot.

Robotic therapy is appealing because it can deliver complex therapies that would be too difficult for therapists to do, for instance provision of precise repeatable force and haptic feedback coupled with interesting and motivating visual feedback and/or the ability to augment movement errors to help correct a movement pattern (Patton et al.,2006). According to Hesse (Hesse et al., 2006) rehabilitation robots providing repetitive induced strategies are better positioned for therapy delivery to severely impaired patients in need of external movement assistance and support to overcome muscle weakness problems. Conversely, task-oriented therapy approaches are more suitable for mildly affected patients.

New strategies are being examined based on error augmentation (Patton et al., 2006), gravity assistance (Sukal et al., 2005), bilateral training and cueing (Johnson et al., 2006), exoskeletal training options (Nef and Riener, 2005), cooperative home-rehabilitation paradigms (Loureiro et al, 2006) and functional whole-arm (arm and hand) rehabilitation strategies (Loureiro et al., 2007; 2009) but there still is a need to provide better training strategies anchored in motor learning and neuroscience theories and affordable opportunities for intensive therapy.

Recent advances in computer power and graphics have prompted for a variety of innovative highly-engaging gaming technologies coupled with realistic virtual environments to reach the consumer market. Such technologies have the potential of providing an interesting and effective way of delivering rehabilitation exercises to people recovering from a brain injury or related cognitive deficit. Virtual environments have the potential of delivering safe and customisable training tailored to the patient’s disabilities with performance being monitored and positivity encouraged (Rizzo and Kim, 2005). The relatively low-cost of such gaming technologies, opens doors not only to its introduction in hospital acute rehabilitation programmes but also to home rehabilitation paradigms. In this context videogames could be used in the future where a group of people with similar disabilities could be engaged in group therapy. The games could be customised to allow the patient to succeed thus boosting morale and increase

Gaming and Social Interactions in the Rehabilitation of Brain Injuries: A Pilot Study with the Nintendo Wii Console 3

participation. Clinicians could log on remotely and adjust the therapy regimes, check progress and monitor safety. The next sections of this paper report on a pilot study evaluating the potential and acceptance of the Nintendo Wii console as additional tool to current rehabilitation practice.

Rehabilitation with the Nintendo Wii ConsoleAt the end of 2006 Nintendo launched their home videogame console named

Nintendo Wii and from the outset it has proved so popular that it has outsold the market competitors Microsoft and Sony (Lee, 2008). Its low-cost and revolutionary interactive game-play interface, the Wiimote (figure 1.1), has attracted attention from the traditional hardcore gamers to older people, encouraging group and family play. The Nintendo Wii has been so successful in attracting new class of gamers that soon nursing homes in the USA started using the console to keep elderly residents active with group activities (BBC News, 2008), followed by its use as part of treatment of burns victims in the south-East of England (BBC News Health, 2008).

B Button (trigger)

D-Pad

A Button

(a) (b)

Figure 1.1. Nintendo Wiimote interface. (a) Wiimote controller details; (b) User holding Wiimote. The IR camera is located on the darker area at the tip of the device.

Nintendo innovative Wiimote interface (figure 1.1) has been reversed engineered by computer enthusiasts, and used to illustrate the remote interaction capabilities of the Wiimote without the console in a variety of different applications (Lee, 2008). The Nintendo Wiimote (figure 1.1-a) is a handheld device packed with sensors such as buttons, a cursor, a 3-axis accelerometer and an IR camera. Two actuators - a speaker and a vibration motor - provide audio and tactile feedback. Data communication is made wirelessly via Bluetooth connectivity and users interact with the game by holding the Wiimote controller on their hand (figure 1.1-b) and pointing to the television set (figure 1.2) where often a sensor bar – containing infrared LEDs - is placed. The IR camera incorporated


inside the Wiimote is used to transform the x-y coordinates from the dot pairs produced by the infrared LEDs. The console software then uses this information together with the accelerometer data to determine the x-y coordinates and the rotation (yaw, pitch and roll) of the device. Distance to the television set is estimated by calculating the separation of two opposite infrared LEDs on the sensor bar relative to the camera view field (Lee, 2008).

Wii Therapy Usability StudyA small usability study was conducted at the Royal Berkshire NHS Foundation

Trust Hospital in Reading with nine acute subjects undergoing rehabilitation. Subjects played a variety of games for the Nintendo Wii console and interacted with the games using the Nintendo Wiimote. The Wii Sports and Wii Play game packs were selected for this study as they contain a variety of fun games involving varying degrees physical ability/mobility and cognitive function (figure 1.2) and are suitable to subjects with different interests and ability. Games such as bowling, tennis, shooting and air hockey promote unilateral play, whereas games such as golf, boxing and baseball are intrinsically more suitable for bilateral movements.

Figure 1.2. Subject playing bowling using the Nintendo Wii console

Games were first demonstrated by one of the occupational therapists (OT) delivering the rehabilitation intervention and played by the subjects either with support from the OT or unaided. Each of the involved OTs completed a simple questionnaire assessing the subjects’ clinical status prior to start the trial and on trial completion reported gaming limitations observed with each subject during the Wii sessions. At the end of the trial each subject was presented with a simple usability questionnaire rating their opinion of the Wii rehabilitation sessions.


Table 1.1 shows a summary of the subjects who have participated in the study. Subjects’ age ranged from 18 to 66 years (mean ± SD: 43.8 ± 17.1) and completed between 3 to 12 sessions (mean ± SD: 8.4 ± 3.2) of Wii therapy. Each session consisted of up to one hour of combined preparation/demonstration and game play. The recruited subjects all had a neurological diagnosis which ranged from cerebral vascular accidents, traumatic brain injuries to spinal cord disease and to other more rare conditions such as Beri Beri with Korsakoff’s syndrome (table 1.1). Impairments included weakness, reduced range of motion, reduced fine motor skills, pain, oedema, sensory loss, and cognitive deficits including reduced attention and concentration, slowed information processing and reduced memory.

Table 1.1. Subjects’ summary

Subject No.

AgeSessions

CompletedDiagnosis

Impairment affecting subject participation on the Wii sessions

1 66 10Spine Injury + Critical Illness Neuropathy

Unable to weight bear through right lower limb

2 55 12Cerebral Vascular Accident

Reduced active ROM and balance


Ataxia and reduced dexterity

4 31 10Berri Berri + Korsakoff's Syndrome

Reduced power, sensation and dexterity in hands, severe pain with diffi culties with memory, attention, cognition and reduced capacity to follow instructions

5 49 6Chronic Inflammatory Demyelinating Polyneuropathy

Reduced balance, sensation, coordination and fine motor skills

6 18 10 Traumatic Brain InjuryReduced coordination, balance and power in both upper limbs (L>R)


Reduced balance and coordination

8 28 10 Spinal Tumour Reduced balance and left shoulder pain

9 60 11 Cervical RadiculopathyReduced active ROM, oedema in both hands. Hand pain and reduced sensation

Results and discussionThe results obtained from the user opinion questionnaire shows that 44% of the

subjects responded that they had already played with the Nintendo Wii before starting the study (figure 1.3). All of the subjects reported that they enjoyed playing on the Nintendo Wii, and 89% of the subjects thought that the Nintendo Wii should be a regular part of their treatment sessions whilst in hospital, and would like one at home to carry on with treatment after discharge (figure 1.3). With regards to their ability to interact and hold the Wiimote while playing the games, 33% of the subjects (subject 3, 4 and 9) required help either from the OT


when movement was too jerky to stabilise the arm or used both hands to perform the movement. Interestingly, subject 4 used the left hand to operate the D-Pad (in the bowling game) and the right hand to perform the swinging movement. Subject 9 on the other hand, required occasional help to press A+B buttons together.

Although all games were initially demonstrated by the OT, subjects chose the game they wanted to play. Figure 1.4 shows that 78% of the participants preferred playing bowling with only 11% preferring air hockey. One of the reasons for the trend observed in figure 1.4 could be related to the fast pace required to play the game (e.g. air hockey) and to the ability to play a game in a relaxed way with more than one opponent (e.g. bowling).

44

100

89

89

33

0 50 100

% Response

Resp

onse

Need help to hold wiimote

Want Wii at home

Wii part of treatment

Enjoyed playing

Played before

Figure 1.3. Participants’ response to the user opinion questionnaire

78

67

44

44

22

44

11

0 20 40 60 80 100

% Played

Gam

e

Air Hockey

Shooting

Baseball

Boxing

Golf

Tennis

Bowling

Figure 1.4. Percentage of games played by participants


The self-evaluation responses (table 1.2) obtained with each subject after completing the Wii therapy sessions shows responses in line with the rehabilitation goals the OTs hoped to achieve during the Wii therapy sessions (table 1.4, second column). The results show a positive impact on improving subjects’ trunk and upper limb strength, balance, coordination, and on increased session participation, attention and concentration.

It is clear from this study that while the main responses are positive, certain aspects of the Wii therapy require attention and improvement, such as the hardware interface. The subjects’ gaming limitations observed by the OTs (table 1.4, fifth column) is in line with the participants’ comments and suggestions on how to improve the Wii interface for easy use (table 1.3). The typical response was that some of the games were too fast or the Wiimote too sensitive to movement change. Likewise interaction with the Wiimote buttons caused difficulties to some participants (subject 4, 5, 6, 9) who could not press the buttons due to their physical size or because of the reduced sensation on the participants’ fingers.

While OTs were positive about group activities – with participants engaging in competitions while playing bowling and boxing – contributing to improved function and cognitive deficits, games were at times too fast and high tone patients could not use the Wiimote. Although the gaming scores are useful for patient feedback, objective data on posture, range of motion, grip and dexterity would be useful to aid therapists’ patient assessment.

Table 1.2. Summary of participants’ response to question “How do you think using the Nintendo Wii has helped you?”

How do you think using the Nintendo Wii has helped you?Subject Comment

1 Strenghted my trunk; Involvement in game tasks takes mind off pain2 Improved my concentration and upper limbs

3 No response4 Helped getting me moving around and using my hands5 It has greatly improved my upper arm strength and fine motor skills.

It has also helped me improve my sitting balance and is now being used to work my standing balance

6It help me channel my anger, improve my balance and my hand control

7 Helped me with balance and concentration. Improved my dexterity, ability to getting involved in exercise, and helpd to get fit

8 Great cardiovascular exercise (boxing) and enjoyable leisure activity9 Helped me improve my grip and coordination


Table 1.3. Summary of participants’ comments and suggestions to question “How can we improve the Wii interface to make it easier to use?”

How can we improve the Wii interface to make it easier to use?No. Participant comment*

1 Reduce sensitivity or cursor movements2 Ability to restrict arm to concentrate on specific movements such as

flexion/extension only3 Have a wiimote with larger buttons 4 Easier game interaction: A+B buttons difficult to press together

with weak hand and reduced coordination* Several comments were repeated by different participants. To avoid repetition the main point is stated once

Table 1.4. Summary of Occupational Therapists’ assessment on participants’ involvement in the study


Left Right

1Increase UL strength, trunk control, standing balance and exercise tolerance

Full but weak

Full but weak

2

Improve balance, progress to palying standing, increase attention up to RW, improve muscle strength and hand-eye

Full but weak

Full but weaker than

Left

3

Increase participation in session; use of both UL, improve coordination or movement patterns, sequencing and timing

Full ataxic movement

Full ataxic movement

4Increase participation in session; improve coordination of limbs

Full but bilateral

weakness

Full but bilateral

weakness

5Improve upper limb ROM and strength, improve sitting balance and core stabillity

Reduced shoulder flexion

Reduced shoulder flexion

6Bilateral hand movements and use of left UL, Increase balance and coordination, anger management

Full but reduced

coordination and power

Full but reduced

coordination and power

7ability to stand during therapy session and completing game

FullNo

functional movement

8Working on increasing dynamic sitting balance

Reduced Left

shoulder due to pain

Full

9Improved coordination and grip of upper limbs

Limited due to oedema

Limited due to oedema

No response

None

Griping hand control due to oedema. Pressing A+B together. Feeling where button B was due to decreased sensation

None. Sat for majority of sessions

Confidence and tolerance

Sensitivity of cursor + infrared bar; control or direction when moving the controller

Complexity of controls, sensitivity of controls to ratation

Sensitivity of cursor on screen. Issue with which buttons on wiimote could be pressed

Subject Rehabilitation Goals hoping achieve during Wii sessions

Subject UL AROM level Subject gaming limitations observed

Reaching speed too quick, did not alow subject time to connect with ball. Coordination of swinging and releasing button

ConclusionFrom our experience in conducting this pilot study, we have found that the

participants’ comments provide an essential insight into usability issues of the Nintendo Wii console and the Wiimote as a low-cost alternative to physical and cognitive exercise in the hospital and potentially at the home. We believe that the Nintendo Wii and gaming in general have much to offer to brain injury and stroke rehabilitation. In particular, well-designed games can be highly motivating and engaging – if only addictive – while promoting limb movement and social interactions with friends, family and other people recovering from similar impairment.

It is evident from the results reported in this paper that this type of technology should be considered as an integral part of the rehabilitation process. However, the


potential and promise of such highly engaging gaming technologies would need to have the interface controls and game interaction customised to map the patients’ physical limitations to game play.

AcknowledgementsThe authors would like to thank all the subjects who have participated on the

study and to the occupational therapists at the Neurorehabilitation Unit of the Royal Berkshire NHS Hospital in Reading who have conducted the Wii therapy sessions.

ReferencesBBC News (2008). “Wii boost for care homes residents”, 30 January 2008,

http://news.bbc.co.uk/1/hi/wales/south_west/7218103.stmBBC News Health (2008). “Games therapy for burns victims”, 25 February 2008,

http://news.bbc.co.uk/1/low/health/7262552.stmBroeks J.G., Lankhorst G.J., Rumping K., and Prevo A.J. (1999). The long-term outcome of

arm function after stroke: results of a follow up study, Disabil & Rehabil, 21(8): 357-364.Harwin W.S., Patton J.L., and Edgerton V.R. (2006). Challenges and opportunities for

robot-mediated neurorehabilitation. Proceedings of the IEEE, 94(9): 1717-1726.Hesse S., Schmidt H. and Werner C. (2006). Machines to support motor rehabilitation after

stroke: 10 years of experience in Berlin. Journal of Rehabilitation Research and Development, 43(5): 671-678.

Hesse S., Schulte-Tigges G., Konrad M., Bardeleben A., and Werner C. (2003). Robot-assisted arm trainer for the passive and active practice of bilateral forearm and wrist movements in hemiparetic subjects. Arch Phys Med Rehabil, 84(6): 915–20.

Johnson M.J., Wisneski H.J., Anderson J., Nathan D., and Smith R. (2006). Development of ADLER: The activities of the daily living exercise robot. IEEE-EMBS Biomedical Robotics (BioRob 2006), February 2006, Pisa, Italy, pp. 881-886.

Johnson, M.J., Van der Loos, H.F.M, Burgar, C.G., and Leifer, L.J. (1999), Driver’s SEAT: simulation environment for arm therapy, ICORR’99: International Conference on Rehabilitation Robotics. Stanford, CA, USA, pp227-234.

Krebs H.I., Hogan N., Volpe B.T., Aisen M.L., Edelstein L., and Diels C. (1999). Robot-aided neuro-rehabilitation in stroke: three-year follow-up, ICORR’99: International Conference on Rehabilitation Robotics, Stanford, CA, USA, pp. 34-41.

Lee J.C. (2008). Hacking the Nintendo Wii Remote. IEEE Pervasive Computing, Vol. 7, 3, pp. 39-45.

Loureiro R., Amirabdollahian F., Topping M., Driessen B., and Harwin W. (2003). Upper Limb Mediated Stroke Therapy – GENTLE/s Approach. In Special Issue on Rehabilitation Robotics, Journal of Autonomous Robots, Springer, 15 (1), pp. 35-51 (ISSN 0929-5593).

Loureiro, R.C.V., and Harwin, W. S. (2007). Reach & Grasp Therapy: Design and Control of a 9-DOF Robotic Neuro-rehabilitation System. In Proceedings of IEEE 10th International Conference on Rehabilitation Robotics (ICORR 2007), Noordwijk, the Netherlands, June 12-14, pp. 757-763.


Loureiro, R.C.V., Johnson, M., and Harwin, W.S. (2006). Collaborative tele-rehabilitation: a strategy for increasing engagement. In Proceedings of the first IEEE International Conference on Biomedical Robotics and Biomechatronics. February 20-22, Pisa, Italy.

Loureiro, R.C.V., Lamperd B., Collin C. and Harwin, W.S. (2009). Reach & Grasp Therapy: Effects of the Gentle/G System Assessing Sub-acute Stroke Whole-arm Rehabilitation. In Proceedings of IEEE 11th International Conference on Rehabilitation Robotics (ICORR 2009), Kyoto, Japan, June 23-26, pp. 755-760.

Lum P.S., Van der Loos M., Shor P., and Burgar C.G. (1999). A robotic system for upper limb exercises to promote recovery of motor function following stroke. ICORR’99: International Conference on Rehabilitation Robotics, Stanford, CA, USA, pp. 235-239.

Nef T., and Riener R. (2005). ARMin - Design of a novel arm rehabilitation robot. 9th IEEE Conference on Rehabilitation Robotics, Chicago, USA, pp. 57-60.

Patton J.L., Stoykov M.E., Kovic M., and Mussa-Ivaldi F.A. (2006). Evaluation of robotic training forces that either enhance or reduce error in chronic hemiparetic stroke survivors," Exp Brain Res, 168: 368-383.

Reinkensmeyer D.J., Kahn L.E., Averbuch M., McKenna-Cole A., Schmit B.D., and Rymer W.Z. (2000). Understanding and treating arm movement impairment after chronic brain injury: progress with the ARM guide. Journal of Rehabilitation Research and Development, 37: 653-62.

Rizzo A., and Kim G.J. (2005). “A SWOT analysis of the field of virtual reality rehabilitation and therapy”, Presence: Teleoperators and Virtual Environments. Vol. 14, pp. 119-146.

Shadmehr R. and Mussa-Ivaldi F.A. (1994). “Adaptive representation of dynamics during learning of a motor task”, Journal of Neuroscience, Vol. 14, pp. 3208-3224.

Sukal T.M., Ellis M.D., and Dewald J.P.A. (2005). Dynamic characterization of upper limb discoordination following hemiparetic stroke. 9th IEEE International Conference on Rehabilitation Robotics, Chicago, USA, 519-521.

Title of paper: Gaming and Social Interactions in the Rehabilitation of Brain Injuries: A Pilot Study with the Nintendo Wii Console

Names of authors:R.C.V. Loureiro1, D. Valentine2, B. Lamperd2, C. Collin2 and W.S. Harwin1

Contact Author: R.C.V. Loureiro Email: [email protected]

Address:

1The University of Reading, School of Systems Engineering, Whiteknights Campus, Reading RG6 6AY (phone: +441183788219; fax: +441183788220)


2Royal Berkshire NHS Foundation Trust, London Road, Reading, RG1 5AN, UK

Submission type: formal paper

File name : Loureiro_et_al_CWUAAT2010_REVIEWED.doc

Version of Word used: 97-2003

gaming and social interactions in the rehabilitation of...

Documents