[IEEE 2013 IEEE Virtual Reality (VR) - Lake Buena Vista, FL (2013.3.18-2013.3.20)] 2013 IEEE Virtual Reality (VR) - VR solutions for improving physical therapy

Download [IEEE 2013 IEEE Virtual Reality (VR) - Lake Buena Vista, FL (2013.3.18-2013.3.20)] 2013 IEEE Virtual Reality (VR) - VR solutions for improving physical therapy

Post on 16-Dec-2016




2 download

Embed Size (px)


  • VR Solutions for Improving Physical TherapyCarlo CamporesiSchool of EngineeringUniversity of California,


    Marcelo KallmannSchool of EngineeringUniversity of California,


    Jay J. HanDepartment of Physical

    Medicine and RehabilitationUniversity of California,


    ABSTRACTWe present new solutions based on Virtual Reality technologies forimproving the delivery of physical therapy and rehabilitation. Threemain aspects are addressed: 1) the ability to allow therapists to cre-ate new exercises and therapy programs intuitively by direct demon-stration, 2) automatic therapy delivery and monitoring with the useof an autonomous virtual tutor that can monitor and quantitativelyassess the motions performed by the patient, and 3) networked col-laborative remote therapy sessions via connected applications dis-playing the motions of both the therapist and the patient.

    We also provide 3D assessment tools for monitoring changes inthe range of motion, and for allowing the visualization of a numberof properties during or after the execution of exercises. The pre-sented system has been implemented for a low-cost hardware solu-tion based on Kinect and for a high-end immersive virtual realityfacility.

    Index Terms: I.3.2 [Computing Methodologies]: ComputerGraphicsDistributed/network graphics.

    1 INTRODUCTIONRehabilitation and physical therapy are optimal when assessment,monitoring, adherence to the therapy program and patient engage-ment can be achieved. With recent technical advances developed inVirtual Reality (VR), innovative approaches to improve traditionalphysical therapy and rehabilitation practice can be explored.

    In traditional physical therapy and rehabilitation practice differ-ent processes are involved: physical examination, evaluation, as-sessment, therapy intervention, monitoring, and modification of thetherapy program according to patient recovery. Usually, after a pre-liminary step of diagnostic and quantitative measurements, a patientis guided by a trained therapist to perform specific therapeutic ex-ercises correctly. The tasks performed are designed according tothe recovery plan and imply repetitions where the therapist needsto evaluate the exercise both qualitatively and quantitatively.

    This process is usually intensive, time consuming, dependent onthe expertise of the therapist, and implies the collaboration of thepatient who is usually asked to perform the therapy multiple timesat home with no supervision [1, 5].

    Our solution allows therapists to model exercises by demonstra-tion and thus to customize exercises for specific patient needs. Li-braries of exercises can be developed for effective reuse in newtherapy programs. Therapy programs can be performed by a vir-tual character demonstrating exercises step by step, including mon-itoring and logging patient execution. Monitoring and progresstracking improves patient understanding, motivation and compli-ance, and also provides data gathering. Finally, the system alsoallows simultaneous networked sessions between remote patients

    e-mail: ccamporesi@ucmerced.edue-mail: mkallmann@ucmerced.edue-mail: jay.han@ucdmc.ucdavis.edu

    Figure 1: Our virtual collaborative system can run in two configura-tions: an inexpensive Kinect-based setup (left) and a high-end Im-mersive setup (right).

    and therapists sharing motion performances in real-time. The sys-tem also provides 3D assessment tools for monitoring the range ofmotion, and for allowing the visualization of a number of therapyparameters during or after execution of exercises.


    The proposed system was implemented in two main configurations.Immersive VR configuration: the main purpose of this configu-

    ration is to allow the therapist to immersively model customized ex-ercises by demonstration and to experience high-end visualizationof the performance of a patient. The patients motion can be gen-erated in real-time or it can be loaded from previously logged ses-sions. The application provides stereo visualization for enhancedcomprehension of the observed motions and data. The users upperbody motions are tracked using a precise motion tracking system(based on Vicon cameras). For simpler setup, the system is config-ured to only track markers attached to the hands, torso and head.The motion is calibrated and mapped to the avatar following thesame approach as described in the work of Camporesi et al. [3].When connected to a remote site, two avatars are displayed for rep-resenting the connected patient and therapist. Previously recordedsessions can also be played on any of the avatars. The avatars canbe visualized side-by-side or superimposed with transparency.

    Our experimental immersive setup consists of a Powerwall sys-tem composed of six rendering computers, a main rendering nodeand an external computer driving the devices and the motion cap-ture system. This choice provides a large immersive display thatenhances user engagement allowing a better spatial understandingand analysis of motions. The interaction with the application is alsofully immersive; thanks to virtual pointers and a 3D GUI interfacecontrolled by a Wiimote (the GUI provides menus, buttons, genericwidgets and panels).


    IEEE Virtual Reality 201316 - 20 March, Orlando, FL, USA978-1-4673-4796-9/13/$31.00 2013 IEEE

  • Kinect configuration: the second configuration is designed toassist patients when they perform their exercises. The patient istracked through a non-cumbersome tracking device (for example,Microsoft Kinect or similar) and a virtual character (or virtual ther-apist) helps the patient perform the prescribed daily therapy tasksby providing real-time monitoring, feedback and logging. This con-figuration is also suitable for use at homes or clinics.

    This configuration also provides two avatars when a networkedconnection is established. Even though the accuracy of Kinect islimited (and the accuracy drops when body occlusions occurs) itstill provides a good tradeoff between cost and portability [2, 4].Automatic motion detection mechanisms are provided to improvethe usability of the system. For example, automatic display of jointangles only when significant variation is detected, end of exerciseautomatically detected after a period of inactivity, etc.

    2.1 Virtual Exercise Modeling, Delivery and MonitoringBoth applications provide tools for modeling exercises and for de-livery and monitoring of the exercises. The option of creating cus-tomized exercises by demonstration enables the therapist to go be-yond recovery plans limited to a set of predefined exercises. Sev-eral interactive tools are available for assisting the therapist withcreating new exercises by demonstration. The therapist can recorddemonstrations and then customize them in different ways. Aftera validation process the motions can be saved and categorized ina database of exercises. The database is then used for fast con-struction of therapy programs using a desktop-mode interface ofthe application during consultation with patients.

    Both low-end and high-end configurations of the system includethe exercise modeling by demonstration capability. Even thoughthe two configurations offer similar capabilities, the accuracy of theemployed sensor will lead to different limitations. The immersiveconfiguration based on Vicon cameras is clearly more accurate forthe modeling task. On the other hand, the versatility of the Kinectsolution grants to the therapist more flexibility during a one to onesession, and gives a markerless, portable, and inexpensive solution.Together both systems complement each other well during the dif-ferent phases of long term therapy programs.

    The system can be employed as a tool to deliver exercises topatients at home, and can be also used during clinical appointmentsto measure and investigate the performance of a patient. In all casessessions can be logged and later re-loaded for analysis and progressassessment.

    When delivering a patients daily program, the virtual therapistcan start the session by demonstrating the exercises to the patient.In a subsequent step, the user is asked to follow the exercises whilethe application is recording the sensed motion. If the motion is de-tected to be significantly different than the demonstrated exercise,the appropriate visual feedback is provided to the user for motivat-ing an improved performance and for better understanding of theexercise. The level of expected compliance and repetitions untilcompliance can be personalized and defined by the therapist specif-ically for each patient.

    Four types of feedback were developed in order to provide vi-sual and quantitative information about the user motions (in real-time or off-line): visualization of end-effector trajectories; dynamicrange of motion measurements; distances from the target exerciseand joint angle measurements (virtual goniometer).

    2.2 Remote Therapist-Patient consultationThe system also allow patients and therapists to interact remotely inany configuration, saving travel costs, potentially increasing accessto health care, and allowing more frequent monitoring. The motionof each user participating to the virtual collaboration is mapped di-rectly to each respective avatar, and the avatars can be superimposedwith transparency or appear side-by-side in the applications.

    The communication between two peers in a collaborative ses-sion is based on a client-server UDP communication schema withadded packet ordering, guaranteed communication reliability andoptional data compression. The server application, after acceptingand validating an incoming connection, starts sending informationof the avatar of the current user (sender) and waits the update of theclients avatar (receiver). For instance, if the therapist applicationis started as a server, the therapists