usability of multi-modal home health monitoring devices used by older adults rita hubert pace...
TRANSCRIPT
Usability of Multi-modal Home Health Monitoring Devices
Used By Older Adults
Rita HubertPace University
April 12, 2008
2
Essence of the Idea
Technology
Older AdultsHealthcare
Usability of Home Health Monitoring DevicesUsed by Older Adults
3
Aging Population Statistics
The number of persons age 65 and over is expected to more than double to 71.5 million by 2030.
[Administration on Aging, United States Department of Health and Human Services, “Profile of Older Americans: 2004]
4
Care Giver Shortage
Fewer workers will be available to care for the aging population.
[Department of Health and Human Services, “The Future Supply of Long-Term Care Workers in Relation to the Aging Baby Boom Generation”, May 14, 2003.]
5
Increased Need for Long Term Health Care Workers
It is projected that between 2000 and 2010 there will be a need for 70% more health care workers in the home health care industry.
Projected Increase in Long Term Health Care Workers 2000 - 2010
0200,000400,000600,000800,000
1,000,0001,200,0001,400,000
Long TermCare
Facilites
ResidetialCare
HomeHealth Care
2000
2010
[Department of Health and Human Services, “The Future Supply of Long-Term Care Workers in Relation to the Aging Baby Boom Generation”, May 14, 2003.]
6
United States Healthcare Spending Projections
Medicare spending was $252.2 billion in 2002 and is projected at more than $500 billion by 2012.
[California HealthCare Foundation, “Snapshot Health Care Costs 101”, www.chcf.org, 2005]
7
United States Health Information Technology Spending
National Health Information Technology Spending per Person
$192
$31.85 $21.20 $11.43 $4.93 $0.43$0
$50
$100
$150
$200
$250
Organization of Economic Co-operation and Development (OECD) Statistics for Health Information Technology
Spending as of 2005 including: Electronic Medical Records Telehealth Electronic Ordering Systems Decision Support Tools Networks and Infrastructure
[Health Care Spending and Use of Information Technology in OECD Countries, Health Affairs, Vol 25, No 3, pp 819-831, 2006]
8
Older Adults with Chronic Diseases
Chronic Disease examples: Heart disease Hypertension Asthma Chronic Obstructive Pulmonary Disease (COPD)
Chronic Diseases account for 95% of the health care spending for older adults.
80% of older Americans have 1 chronic disease 50% of older Americans have 2 chronic diseases The average person 75 years of age has 3 chronic diseases. Chronic Diseases are the leading cause of death in America.
Diabetes
Stroke
Arthritis
[Centers for Disease Control, Health and Human Services,”The State of Aging and Health in America 2004”, www.cdc.gov/aging/pdf/state_of_Aging_and_Health_in_America_2004.pdf]
9
Relevance and Significance of the Research
Home health monitoring devices will provide older adults with the ability to live independently at home longer while under the care of the health provider.
Home health monitoring devices will reduce the need for caregivers and reduce costs. The Kaiser Permanente Tele-Home Health Research Project found mean cost savings
of $726 in the home health monitoring group.
New England Healthcare Institute and Massachusetts Technology Collaborative reports that studies show an annual cost reduction of $7,830 per patient per year for congestive heart failure and $747 per patient per year for diabetes with the use of home health monitoring devices.
The Pennsylvania Home Care Association study found that with home health monitoring devices, each nurse could care for 6 more patients, 17 rather than 11.
10
Definition: Home Health Monitoring
Home Health Monitoring or TeleMonitoring the use of technology devices in the home the collection of health parameters by the patient the transfer of the data to a remote monitoring
station healthcare provider review and actions
Some devices, not studied in this research, use videoconferencing and stethoscopes.
11
Home Health Monitor Measures
Weight Device 1 and Device 2 use a scale
Blood Pressure/Pulse Device 1 and Device 2 use a Blood Pressure Cuff (Device 2 uses a
separate machine with a separate ‘start’ button) Temperature
Device 1 uses a temperature sensor probe. Device 2 uses manual data entry of temperature measure performed with
home thermometer Blood Oxygen Saturation
Device 1 and Device 2 use a finger sensor clip 6 Health Questions
Device 1 uses both visual and auditory interaction for the questions. Device 2 uses only visual interaction for the questions.
12
Schema and Positioning of Telemedicine Segments [adapted from Dan and Luprano, 2003]
Medical Complexity
Technology Maturity Level
High
Research
Low
Tele-meeting
Telemonitoring
E-learning
TeleconsultingTelediagnostic
Teleoperation
DevelopmentCommercial
13
Definition: Usability
Usability is the study of the interaction between a computer-based device and the user of the device.
ISO 9241 Usability Standard - Usability is the effectiveness, efficiency and satisfaction.
Jakob Nielsen considers usability characteristics as being easy to use, easy to learn and easy to remember.
14
Home Health Monitor Device Multi-modal Usability Factors
Multi-modal Usability Factors
Audio• Male or Female voice • Volume adjustable• Languages available
Visual• Text size• Color• Contrast• Button size• Button spacing
15
Usability Methods According to Jakob Nielsen
Method Name
Users Needed
Main Advantage Main Disadvantage
Observation 3 or more Ecological validity; reveals users’ real tasks. Suggests functions and features.
Appointments hard to set up. No experimenter control.
Questionnaire At least 30 Finds subjective user preferences. Easy to repeat.
Pilot work needed (to prevent misunderstandings).
Interview 5 Flexible, in-depth attitude and experience probing.
Time consuming. Hard to analyze and compare.
Focus Group 6-9 per group Spontaneous reactions and group dynamics.
Hard to analyze. Low validity
[Nielsen, Jakob, Usability Engineering, Morgan Kaufman, Academic Press, 1993]
16
Healthcare Monitoring Usability Studies
Telephone-Linked Care for Diet Adherence in Dyslipidemia (2004) Method: telephone interviews and laboratory observation Sample size: 8
Home Asthma Telemonitoring System (2004) Method: telephone interviews and home-based field observation Sample size: 5
Informatics for Diabetes Education and Telemedicine (2003) Method: cognitive walkthrough and home-based field study observation Sample size for the field study observation: 25
Diabetes Glucometer (2001) Method: Survey and laboratory observation Sample size: 26 for the survey and 6 for the observation
17
Lessons Learned From Older Adult Research
Use individual interviews, rather than paper questionnaires or focus groups.
• Eliminate the use of ‘don’t know’ .
Obtain Qualitative data via open ended questions about feelings, problems and experiences with technology.
• Conduct in-home interviews to learned the most and see how the technology fits into home.
Older Adults are concerned about user friendliness of devices.
Use a video and audio tape recorder to record the tester using the device because it is easier to do transcripts from audio tape recordings
Let seniors try technology and ask questions in a supported environment
In-home studies provide a realistic setting for testing home health monitoring devices
18
Technology and Older Adult Research - Representative Sample Size
Experienced Computer Users
Inexperienced Computer Users
Technology Accepting Users
Will provide informed and constructive responses. Should result in useful ideas for improving service based on user’s previous experience. 10 Participants
Will provide ideas of how inexperienced consumers will react when they first acquire a product.5 Participants
Technology Resistant Users
Will highlight concerns from a technical perspective and which features are of value, even to technology resistant users.2 Participants
Will identify concerns of the wider consumer population. Useful if aim is to launch a simple service of interest to mass market.4 Participants
[Syme, Audrey and Roos Eisma, “How Representative is Your Older Adult Sample?”, HCI and the Older Population, Leeds, UK, September 7, 2004.]
19
Phase 1 Phase 1
Phase 3 Phase 3
Phase 2 Phase 2
Semi-structured Telephone Interviews 21 experienced Participants
Semi-structured Telephone Interviews 21 experienced Participants
Audio and Video Recording Observations for Device 1 Usage 7 Experienced Participants 10 Inexperienced Volunteers
Audio and Video Recording Observations for Device 1 Usage 7 Experienced Participants 10 Inexperienced Volunteers
Audio and Video Recording Observations for Device 1 and 2 Usage10 Volunteers
Audio and Video Recording Observations for Device 1 and 2 Usage10 Volunteers
Research Study Phases
20
Study Audience
Older Adults between 50 and 88 years of age.
Both males and females. Older adults living in rural New Hampshire. The Test Group
21 current and former Visiting Nurse Association patients who used a home health monitoring device for more than 7 days.
The Control Group 10 volunteers with no previous experience using
home health monitoring devices.
21
Qualitative Research Methods
Usability Method Semi-structured Telephone Interview
Qualitative MeasuresSatisfactionComments
Sample size 21
22
Quantitative Research Methods
Usability Method Field Study Observation in the Older Adult Home
using audio and video recording Quantitative Measures
Time to complete each taskTime to complete each sub-taskNumber of Errors
Sample size19Study group = 7 Control group = 12
23
Institutional Review Board (IRB) Process
Definition: ‘An institutional review board (IRB) is a Committee that has been
formally designated to approve, monitor, and review biomedical and behavioral research involving humans with the alleged aim to protect the rights and welfare of the research subjects. An IRB performs critical oversight functions for research conducted on human subjects that are scientific, ethical, and regulatory.’ [Wikipedia]
Pace University IRB Approval is required before beginning any Research with Human Subjects. Obtain Certificate on Human Subjects Research History, Ethics and
Requirements Complete the IRB Form Obtain IRB Approval Update IRB and obtain Approval for any Changes in Research and
Subjects Close IRB for your Research
24
Phase 1 - Participant Statistics
Phase 1 Home Health Monitoring Study – Semi-structured Telephone Interview
Study Statistics Ages: Number: Gender: 50-59 6 4F 2M 60-69 2 2F 70-79 8 2F 6M 80-89 5 3F 2M
11 Females 10 Males
Total 21 Participants
21 participants or 100% wear glasses 2 participants or 9.5 % wear hearing aid
25
Phase 1 - Satisfaction with Home Health Monitor Device 1 Used by Patients
Overall Satisfaction with the home health monitoring device by Patients
Response Percentage
Very satisfied 16 76% Satisfied 4 19% Dissatisfied 0 Very dissatisfied 1 5%
26
Phase 2 - Satisfaction with Home Health Monitor Device 1 Used by Volunteers
Overall Satisfaction with the home health monitoring Device 1 by Volunteers
Number Percentage
Very satisfied 8 80% Satisfied 2 20% Dissatisfied 0 Very dissatisfied 0
27
Phase 3 - Satisfaction with Home Health Monitor Device 2 Used by Volunteers
Overall Satisfaction with the home health monitoring Device 2
Number Percentage
Very satisfied 5 50% Satisfied 4 40% Dissatisfied 1 10% Very dissatisfied 0
28
Phase 1 – Device 1Visual and Button Comments
Button on top would be better. (T4) Can not read text but do by color. (T6) Buttons are too close together. Son’s finger pushes two buttons at a time. (T7) Buttons are flat and too close together. Unable to feel different buttons. Need
texture on some buttons to differentiate. Need better contrast, such as black machine and white buttons. (T10)
When you push the button you need to feel the push, so you know if you pushed enough. Display not large enough to see clearly. (T17)
Buttons difficult to use with long finger nails. Raised button would be better to use rather than flat button. (T18)
The buttons are difficult to push on front of machine, must put hand on top and push button. (T20)
29
Phase 2 – Device 1 Volunteer Visual and Button Comments
Move buttons on top of machine (V9) Pushing the ‘Start BP’ button with blood pressure cuff on one arm
and oxygen sensor on the finger of the other hand is difficult. Move buttons to top of machine. (V1)
Angle of display is difficult. (V5) Do not like looking down on screen. Did not like the colors, they
were hard to read. Suggest black buttons on white background. (V4)
The angle of the screen on front of the machine is difficult. LED is easier to read. (V6)
30
Phase 3 – Device 2Visual and Button Comments
Start BP button easy (V10) Circular buttons are easy. Separate button for BP is confusing. (V11) Clear visual screen. Directions are simple to follow. (V12) Selection buttons are easy. (V5) Buttons are easy. (V3) Easy visible screen and buttons. (V7) Button and the meaning of the scroll on the side of screen is difficult. Circular
scrolling with arrow buttons should be provided. (V9) Not separate start BP reading button. (V8) Cancel and OK buttons use is difficult. (V3) Reading screen is easy. (V6)
31
Hypothesis
1. Previous computer experience will decrease task time, regardless of age.
2. Previous computer experience will decrease error rate, regardless of age.
3. Persons with previous experience using the device will have faster task times than new users.
4. User device satisfaction ratings will be high for devices with low task times.
5. User device satisfaction ratings will be high for devices with low button press error rates
6. Discomfort with computers and technology results in longer performance task times for older adults.
7. Participants less than 65 years of age require less time to complete 6 health related questions than the participants over age 65
32
Phase 2 – Patients versus Volunteer Using Device 1 Summary Results
Device 1 Results
Number/Gender
Average Time for 6 Questions (Average)
Button Presses for 6 questions
(Average)
Button press errors
(Average)
Device Satisfaction
(Average on scale of 1-4)
Button easy to use Satisfaction
(Average on scale of 1-4)
Patients 6 Female 26.4 sec 6.3 presses 0.3 presses
1.7 rating 1.7 rating
Volunteers 6 Female
4 Male
34.6 sec 6 presses 0 presses 1.2 rating 1.4 rating
33
Phase 3 – Volunteers Using Device 1 and Device 2 Summary Results
Device 1 and Device 2
Results
Number/Gender
Average Time for 6 Questions (Average)
Button Presses for 6 questions
(Average)
Button press errors
(Average)
Device Satisfaction
(Average on scale of 1-4)
Button easy to use Satisfaction
(Average on scale of 1-4))
Device 1 Volunteers
6 Female
4 Male
30.1 sec 6 presses 0 presses 1.2 scale 1.4 scale
Device 2 Volunteers
6 Female
4 Male
71.1 sec 12.2 presses 0.7 presses 1.6 scale 1.6 scale
34
Hypothesis 1Previous computer experience will decrease task time responding to
questions, regardless of age.
Using Device 1 the average time for responding to 6 health related questions for test and control group participants
The computer experienced group completed the task an average of 24% faster than the computer inexperienced group.
The chi-squared test shows no significant difference between the computer experience and no computer experience group.
p=0.293763221
Average Task Time responding to 6 questions in seconds
Experienced Group (Patients)
N=6
26.4 seconds
Inexperienced Group
(Volunteers)
N=10
34.6 seconds
35
Device 1 More Experienced versus Less Experienced Test
Group Task Time Comparison
Device Experience Versus Task Time
2132 27 33 33
49
0
20
40
60
80
100
120
140
T20 T10 T12 T2 T11 T15
Participant ID
Days Using Device
Task Time inseconds
Summary: The most experienced persons were 21 and 33 seconds. The most inexperienced person was the longest with 49 seconds.
A larger sample size is needed to evaluate the statistical significance.
36
Hypothesis 2Previous computer experience will result in a
decreased error rate, regardless of age.
Summary: One experienced computer user had 11 errors and the remaining 5 computer users had a total of 8 errors.
A larger sample size is recommended to more accurately consider the comparison for this hypothesis.
Device 1 Button Press Errors
Device 2 Button Press Errors
Computer Experienced
0 Errors
N=7
19 Errors
N=6
Computer Inexperienced
3 Errors
N=9
19 Errors
N=6
37
Hypothesis 3Computer Experience versus Computer
Inexperience
Persons with experience using the device will have faster task times than inexperienced users.
The similarity of the task times shows that the usability of Device 1 is good because responding to 6 health related questions for Device 1 is easy to learn and use.
Average Task Time
Computer Experience
N=6
32.5 sec
Computer
In-experienced
N=10
30.1 sec
38
Hypothesis 4User Overall satisfaction ratings will be high for
devices with low task times
The task time in seconds to respond to 6 health related questions by 12 volunteers. Ten volunteers used Device 1 and ten volunteers used Device 2 Chi squared show significant differences at p
value of less that 0.01 Task time in seconds for the same 8
volunteers using Device 1 and Chi squared show significant differences at p
value of less that 0.01 Overall Device Satisfaction of the
participants with Device 1 and Device 2 There is no statistical significance and the
results are therefore similar. Although the Device 2 participants required
significantly longer times to complete the responses to 6 health related questions, their satisfaction ratings show similar ratings.
Average Task Time for 6 Questions
Average Task Time for measures
Device Satisfaction
Device 1 30.1 29.6 1.2
Device 2 71.1 72.3 1.6
39
Hypothesis 5User Button Ease of use satisfaction is high
for devices with low button error rates.
Comparing Device 1 and Device 2 total button press error rate for volunteers using vital signs and 6 health related questions.
The person with 11 button press errors indicated the buttons were easy to use. The person with 7 button press errors indicated the buttons were very easy to use. The person with 4 button press errors indicated the buttons were difficult to use.
The CHI-squared comparison of the button press errors is highly significant p = less than 0.01
The comparison of the responses of the button ease of use question shows no significance.
Button Press Errors
User
Satisfaction
Device 1 0-1 9=Very Easy
4=Easy
Device 2 2-11 2=Very Easy
4=Easy
1=Difficult
40
Hypothesis 6 Computer Comfortable versus Computer Uncomfortable
Participant Task Time Comparison
Computer Comfort Comparison
Task time range
Average Task time for 6 health related questions
Computer Comfortable
N=9
21-49 seconds
31 seconds
Computer Uncomfortable N=7
28-36 seconds
31 seconds
Summary: A larger sample size is needed to evaluate the statistical significance.
41
Hypothesis 7 Younger versus Older Groups Task Time Comparison
Age Group
Task time range
Average Task time for 6 health related questions
50-65
N=6
21-32 seconds
27.5 seconds
65-80
N=10
23-49 seconds
33.1 seconds
Summary: The range of the results and average task time are lower for the less than 65 age group in comparison to the greater than 65 age group.
However, a larger sample size is needed to evaluate the statistical significance.
42
Device 1 - Recommendations
Good use of bright primary colors Good use of multi-modal visual and auditory interaction Move buttons to the top of the device Use more spaces between the buttons Use raised buttons with different textures Use raised buttons with louder auditory feedback when pressing Use thicker high contrast letters and numbers on the monitor
screen
43
Device 2 - Recommendations
Good screen location and angle for viewing Good primary Button location, size and spacing Button color should use bright primary colors Contrast should be improved on the monitor, especially the
number pad (gray background with gray buttons) Use raised buttons with different textures Use raised buttons with louder auditory feedback of pressing Use thicker high contrast letters and numbers on the monitor
screen Add auditory reading for the health questions, in addition to
visual question on screen Use more auditory directions, if no patient response Repeat messages several times, if no patient response
44
Summary Button Recommendations
Button Location/Spacing – On top of machine or on an angle screen rather than on the front of the
machine Space between buttons to avoid double button press or incorrect button press
Button visual- Button color – bright colors Button contrast - high contrast with background color Button text color – color bright Button text contrast – high contrast with background color
Button Auditory – Button auditory feedback for button pressing
Button Tactile – Button texture – different for each button for visually limited persons to
differentiate buttons by texture Button tactile feedback upon pressing Raised button to differentiate from device
Button Operations – Minimize the number of buttons Minimize the number of button presses
45
Future Work
Larger Sample Size
Include more Males in the Observational group.
Inclusion of more Home Health Monitoring Devices
Conduct Satisfaction Rating and Device Interview after all Devices are
Tested
Interview Home Health Monitoring Nurses
Interview Physicians who recommend patients for Home Health
Monitoring
Setup Camera on Tripod in Home and ask Patients to do their own
recording.
46
Experience-Based Recommendations
Keep Focused on your Research Ask If you need IRB Approval, Apply ASAP Apply and Present at a Conference Doctorial
Consortium Set and Meet Short-term Goals Push Yourself It is definitely worth the effort
47
References
[1] Dan, Jean-Pierre and Jean Luprano, “Homecare: A Telemedicine Application” Medical Device Technology, December 2003, www.medicaldevicesonline.com, pp.25-27.
[2] Farzanfar, Ramesh, Joseph Fingelstein, Robert Friedman, “Testing the Usability of Two Automated Home-based Patient-Management Systems”, Journal of Medical Systems, Vol 28, No 2, April 2004, pp. 143-153.
[3] Goodman, Joy, Stephen Brewster, and Philip Gray, “Older People, Mobile Devices and Navigation”, HCI and the Older Population, Leeds, UK, September 7, 2004, www.acs.gla.ac.uk/utopia/workshop/.
[4] Health Care Spending and Use of Information Technology in OECD Countries, Health Affairs, Vol 25, No 3, pp 819-831, 2006
[5] Home Care Automation Report, “Telemedicine Leaders Recognize Home Telehealth”, vol 10, no 8, pp. 5-6, August 2005
[6] Johnston, B, L Wheeler, J Deuser, and K Sousa, “Outcomes of the Kaiser Permanente Tele-home Health Research Project”, Archives of Family Medicine, vol 9, no 1, 2000, pp 40-45.
[7] Kaufman, David, Vimla Patel, Charlyn Hilliman, Philip Morin, Jenia Pevzner, Ruth Weinstock, Robin Goland, Steven Shea, and Justin Starren, “Usability in the real world: assessing medical information technologies in patients’ homes”, Journal of Biomedical Informatics, vol 36, pp 45-60, 2003.
[8] Klecun-Dabrowska, Ela and Tony Cornford, “Evaluation and Telehealth-An Interpretative Study”, Proceedings of the 34th International Conference on System Sciences, pp 1-10, 2001.New England
[9] Healthcare Institute, “Advanced Technologies to Lower Health Care Costs and Improve Quality”, Massachusetts Technology Collaborative, http://www.nehi.net/, Fall 2003.
[10] Oorni , Kai, “What do we Know about Usability Evalation? A Critical View”, www.student.oulu.fi/~koorni/digilib2003.pdf, 2003[11] Pennsylvania Homecare Association and Pennsylvania State University, “The Financial Viability of Telehealth and Telehealth’s Impact on Home Health
Nurses” Telehealth Project Evaluation – Year 3, September 1, 2004 – August 31, 2005.
[12] Rogers, Wendy, Amy Mykityshyn, Regan Campbell and Arthur Fisk, “Analysis of a ‘Simple’ Medical Device”, Egonomics in Design, (Winter 2001) 6-14.