user interface design for medical devices - the relationship between usability and safety
TRANSCRIPT
The relationship between usability and safety29 April 2016Rich NewmanUsability Engineering ManagerBlackHägen [email protected]
User Interface Design for Medical Devices
2
Impact of Poor Usability
• Web Site, Software Application, Consumer Product– Poor conversion rate – loss of sales– Mistakes on orders, reports, etc.– User frustration
3
Impact of Poor Usability• Medical Device
– Incorrect medication or dose– Incorrect therapy setting– Incorrect interpretation of data
resulting in misdiagnosis
4
Usability Directly Relates to Safety• If a device is difficult to use, the user is more
likely to use it incorrectly• Can result in harm or death
– to patient, operator, or bystanders
5
Example 1: Therac 25• Radiation Therapy Device
– Used for both low and high dose therapy• Incorrect key sequence caused machine to switch to high dose
mode without engaging safety hardware*– User quickly corrected an incorrect key-press– Slow response to key press caused device to respond
incorrectly– Patients received 125x the normal dose of radiation
• 6 patients received radiation overdose– 5 deaths between 1985-1987– Machine recalled after 5th incident
*The Therac disaster resulted from a combination of software and usability problems. It lead to sweeping changes in safety critical software development. Major changes related to usability were not addressed until more recently.
6
Example 2: Infusion Pumps• Between 2005 and 2009, FDA reported:
– 56,000 infusion pump incidents– 710 deaths– 87 recalls
• Infusion devices account for up to 35% of all medication errors that result in significant harm
• A large percentage of these adverse events are due to programming errors that can be attributed to poor usability– Entering weight in pounds instead of kg – results in 2.2x overdose– Incorrectly placed decimal point – results in 10x under or over infusion– Select incorrect dose mode – mg/kg/min instead of mcg/kg/min – results in 1000x
overdose
Source: FDA White Paper: Infusion Pump Improvement Initiative, April 2010
7
How big is this problem?• Recalls
– 44% of medical device recalls due to design problems – More than 1/3 involved user interface
• Medical Device Incident Reports– FDA receives about 100,000 medical device incident
reports every year. – More than 1/3 can be attributed to usability problems– Small fraction of all the incidents that actually occur get
reportedSource: James Carstensen 1990 FDA presentation: “An internal FDA study some years ago showed that 44% of medical device recalls were the result of design problems and more than 1/3 involved the device-user interface (device, it’s labeling or instructions for use).”
8
What is Considered a Medical device• Physical Devices / Systems
– Robotic Surgery System– Defibrillator– Infusion pump– Pulse oximeter– Auto injector – Epi-Pen – Oral thermometer (analog or digital)
• Software– To monitor or configure devices
• Central monitoring station for a patient monitor– Image storage and viewing systems (PACS, etc.)
• Mobile apps– Used as an accessory to a regulated medical device– Used for diagnosis of disease or other conditions– Used for the cure, mitigation, treatment, or prevention of disease
9
Definition: User InterfaceAnything the user interacts with
• Display and touch screen (GUI)• Buttons and knobs• Mounting hardware• Cover, cap, or case• Disposables (needles, tubing, etc.)• Labeling• Training materials• Instructions for use (IFU)
Not just the GUI
10
Definition: USE ERROR
• Incorrect use called a USE ERROR– NOT a User Error– The User is not at Fault
• Device / UI Design is at fault– The design enabled or tolerated the
error
11
Causes of USE ERRORS• Perception
– Failure to see or hear information• Cognition (Processing)
– Forget information or step– Forget or apply rule incorrectly– Misinterpret information
• Action– Inability to reach a control– Use incorrect control– Use a control incorrectly– Failure to activate control
Source: FDA Draft Guidance: Applying Human Factors and Usability Engineering to Optimize Medical Device Design, 2011
12
Use Error = Device Defect
USE ERRORS treated the same as any other device fault
• A use error is considered a device defect– Same as hardware or software fault
• Must be reported to FDA or local regulatory body
13
What is Being Done?• Most countries now require proof that the UI is safe before
approving a medical device for sale– A new international standard went into effect in 2010
• IEC 62366 - Application of Usability Engineering to Medical Devices
• Revised in 2015– Outlines process for developing safe medical device user
interfaces
Devices will not be approved unless UI is shown to be safe
14
But UI Design is Subjective…
How can you prove the UI is safe?
15
Risk Based Process
• Adapted from proven processes used for hardware and software– Originated in aviation industry
• Integrate UCD methods to identify and mitigate risks
• Iterative Process– Loop back as new risks
identifiedSource: ANSI/AAMI HE75:2009
HF Risk Management Workflow
16
Define Intended Use, Users, Use Environment
Source: FDA Draft Guidance: Applying Human Factors and Usability Engineering to Optimize Medical Device Design, 2011
17
Factors that affect Safety: Users• Abilities
– Physical size, strength, and stamina,– Physical dexterity, flexibility, and coordination– Sensory abilities (i.e., vision, hearing, tactile
sensitivity)– Cognitive abilities, including memory– Familiarity with similar devices
• Factors that affect abilities– Age– Medical conditions– Literacy and language skills– General health status– Mental and emotional state– Medical expertise
18
Factors that affect Safety: Use Environment• Lighting
– Visibility of displays and indicators• Noise
– Ability to hear audible cues and alarms• Space
– Access to device, interference with cords and tubing• Activity
– Distraction
19
Factors that affect Safety: User Interface• Input
– Controls – type, size, spacing, actuation force, etc.– Size and Configuration of device
• Processing– Menu and control design– How system responds to user action – How feedback is provided to user
• Output– Displays and Indicators
• Brightness, Off angle visibility, Glare resistance• Font selection, size, color; screen layout
– Auditory • Alarms, Beeps, Ambient Noise (motors, fans, etc.)
• Other– Device labeling, packaging, – Training materials, Instruction for Use (IFU), and other reference materials.
20
Identify Use Related Hazards• Identify Known Problems
– Internal customer complaint files– Government and trade group databases
• FDA, EU, and others maintain complaint, adverse event, recall, and other databases
• Analytical Approaches– Interviews and focus groups– Contextual inquiry– Function and task analysis– Heuristic analysis / Expert review
• Formative Testing– Unanticipated Use Errors frequently found during formative testing
21
Estimate and Prioritize Risk• Use established risk analysis methods
– Failure Mode and Effect Analysis (FMEA)– Fault Tree Analysis (FTA)– Details outside of the scope of this talk
• Use results to prioritize and address potential hazards– Based mainly on level of harm– Frequency of occurrence also considered but less of a factor
• Iterate– Regularly review and update during development process
22
Implement Risk Controls
Source: AAMI/ IEC 62366 -1 Public Review Draft (AAMI/IEC DS-1 6236601-D)
23
Example – Displaying Units of Measure• Hazardous situation
– User enters weight in pounds instead of kg• Harm
– Weight based dose calculation is 2.2x too high resulting in overdose
• Fix– Display units with similar prominence as numeric field– Locate units adjacent to numeric entry
Weight (kg) Weight 100 45.4 kg
X Source: NPSA Design for Patient Safety – A guide to the design of Electronic Infusion Devices, Edition 1, 2010
24
Example – Displaying Numeric Values• Hazardous situation
– Misreading decimal numbers due to incorrect display of leading or trailing zeros– Misreading of large numbers
• Harm– Misdiagnosis to incorrect interpretation of data– Failure to detect data entry error
• Fix – Do not display trailing zeros (after the decimal point)– Use a leading zero (before decimal point) for numbers less than 1– Include thousands separators for large numbers
Source: NPSA Design for Patient Safety – Guidelines for safe on-screen display of medication information, Edition 1, 2010
Incorrect Correct
Dose 5.0 mg Dose 5 mg
Dose 5.30 mg Dose 5.3 mg
Volume .6 mL Volume 0.6 mL
Dose 10000 u Dose 10,000 u
25
Example – Numeric Keypad Design• Hazardous situation
– User presses wrong key resulting in incorrect numeric entry• Harm
– Incorrect therapy, overdose, underdose• Fix
– Always use the same keypad layout ; “Phone” recommended vs. “Calculator”
– Do not place ‘0’ and ‘.’ adjacent to each other – Keypad layout is only part of the fix
XSource: NPSA Design for Patient Safety – A guide to the design of Electronic Infusion Devices, Edition 1, 2010
7 8 94 5 61 2 3
0 .
1 2 34 5 67 8 9C 0 .
1 2 34 5 67 8 90 C .
X
26
Risk Control Example- The Total Solution• Each of the previous examples is only part of the solution
– Combine risk control methods for maximum benefit
1 2 34 5 67 8 90 C .
Weight
4,540 kg
Error – Weight Exceeds Limit
27
Formative Evaluations• Purpose is to inform the design
– Explore strengths and weaknesses of UI – Identify unanticipated USE ERRORS
• Test with representative users– Users with similar age, physical and cognitive abilities– Users with appropriate training and experience
• Simulate use environment– The more realistic the environment, the better the results
• Various methods of simulating treatment – mannequins, “dummy tummy”, …– Include other devices typically found in use environment
• Test labeling, training materials, IFU, etc.– Anything used to mitigate risk
• Iterate– Fix problems and test again
28
Summative (Validation) Testing
• Use production equivalent devices
• Test in realistic use environment– No think out loud; moderator probing, etc.
• 15-25 participants per user group
• Test essential and safety critical tasks– Leverage results of formatives to narrow scope
• Include training materials, IFU if normally used– Approximate training actual users will receive– Allow time between training and testing (“training decay”)
29
Summative Data Collection• Objective data
– Task success or failure, USE ERROR, close call– Reference to instructions for use– Need for assistance, evidence of difficulty or confusion– Unsolicited comments
• Subjective data (after use)– Discuss perceived reasons for any essential and critical task errors, failures
and difficulties – Solicit feedback on design of device, packaging, labeling and training
Source: Human Factors Engineering of Combination Products and the FDA, Molly Story, July 2012http://www.aami.org/hfconnect/FDA/HF_Engineering_Combination_Products_071712.pdf
30
Analysis of Summative Data• Analyze all USE ERRORS and failures
– Determine root cause and potential clinical consequences– Determine need to modify device, labeling, or training– Identify true residual risks
• USE ERRORS/failures are not of equal importance– Some errors might be frequent but inconsequential – Some errors might be rare but reveal a hazardous design deficiency that was not previously
recognized
• Perform Residual Risk Analysis– Can have USE ERRORS and still pass summative– USE ERRORS must pass residual risk analysis
• Is benefit greater than risk?
Source: Human Factors Engineering of Combination Products and the FDA, Molly Story, July 2012http://www.aami.org/hfconnect/FDA/HF_Engineering_Combination_Products_071712.pdf
31
Documentation• FDA Human Factors Report
– Summary of usability engineering inputs, activities, and outputs
1. Intended device users, uses, environments2. Description and images of device user interface3. Summary of known use problems4. User task descriptions, risk priorities, success criteria5. Summary of preliminary evaluations6. Validation study protocol, results and analysis7. Conclusion
Source: FDA Draft Guidance: Applying Human Factors and Usability Engineering to Optimize Medical Device Design, 2011
32
Further readingWEB• AAMI Human Factors Resources Page:
http://www.aami.org/hfconnect/– Links to major standards and other valuable Medical HF
Resources
• FDA Human Factors Home Page:http://www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/HumanFactors/default.htm
• FDA Guidance –Applying Human Factors and Usability Engineering to Medical Devices; February 3, 2016http://www.fda.gov/downloads/MedicalDevices/.../UCM259760.pdf
STANDARDS
• ANSI/AAMI/IEC 62366-1:2015 (Available for Purchase)Medical devices - Part 1: Application of usability engineering to medical devices http://my.aami.org/store/detail.aspx?id=6236601-PDF
• ANSI/ANSI HE75:2009(R)2013 (Available for Purchase) Human factors engineering—Design of medical devices
http://www.aami.org/publications/standards/he75.html – Specific design recommendations and related
information
BOOKS• Handbook of Human Factors in Medical Device Design
Edited by Matthew Weinger, Michael Wiklund, and Daryle Gardner-Bonneau
• Designing Usability into Medical ProductsBy Michael Wiklund and Stephen Wilcox
• Usability Testing of Medical Devicesby Michael Wiklund, Jonathan Kendler, and Allison Strochlic
34
Other Sources of Hazard Data • Consider entire product lifecycle
– Transport– Storage– Installation– Operation– Maintenance and Repair– Disposal
• Reasonably Foreseeable Misuse– Use of non-approved disposables– Use by unintended user groups – Use in unintended environment– Off label uses