usability & human factors decision support systems: a human factors approach this material...

Usability & Human Factors

Decision Support Systems: a Human Factors Approach

This material (Comp15_Unit7) was developed by Columbia University, funded by the Department of Health and Human Services, Office of the National Coordinator for Health Information Technology under Award Number 1U24OC000003.


Learning Objectives

2

• Discuss factors in understanding Human Decision Making

• Discuss Clinical Decision Support Systems (CDSS)

• Discuss computer provider order entry systems (CPOES) and clinical decision support systems (CDSS)

Health IT Workforce Curriculum Version 3.0/Spring 2012

Usability & Human Factors Decision Support Systems: A Human Factors Approach

Patient Safety

3Health IT Workforce Curriculum Version 3.0/Spring 2012


Human Factors Approach



Understanding Decisions



Medical Decision Making Research



Heuristics and Biases



Hindsight Bias



Hindsight Bias: So What?



Confirmation Bias



The Cost of Confirmation Bias



Classic DM Problem (Eddy, 1982)

• Estimate the probability that a woman has breast cancer given that she has a positive mammogram on the basis of the following information:

– The probability that a patient has breast cancer is 1%. (This provides the prior probability)

– If the patient has breast cancer, the probability that the radiologist will correctly diagnose it is 80% (This provides the sensitivity or hit rate)

– If the patient has a benign lesion (no breast cancer), the probability that the radiologist will misdiagnose it is 9.6% (This provides the false positive rate).

• What is the probability that a patient with a positive mammogram actually has breast cancer?

• Probability of breast cancer is only 7.8%, while Eddy reports that 95 out of 100

• 95 out of 100 doctors estimated this probability to be between greater than 75%



Framing Effect



Survival vs. Mortality

• McNeil et al (1982) presented a hypothetical lung cancer decision scenario to physicians and patients– The treatment options were radiation therapy, which had an immediate

higher survival (lower mortality) rate, but a lower 5 year survival rate. – Frame 1: treatments were described in terms of survival rates – Frame 2: treatments were described in terms of mortality rates

• Results: – Survival frame, clear preference for surgery, – Mortality frame, the two choices were preferred almost equally.

• One possible explanation is that the positive framing leads to more risk averse choices, while the negative framing increases risk-seeking decision making



DM in Naturalistic Settings



Decision Support Systems



Clinical Decision Support Systems



Star Trek Tricorder: The Ultimate Clinical Decision Support Tool

http://www.flickr.com/photos/bojo/4078685614/lightbox/#/



http://www.flickr.com/photos/bojo/4078685614/lightbox/

Forms of CDSS Advice



The Case for Clinical Decision Support



Degrees of CDSS Computerization



Degrees of Computerization Continued (6-10)



Computerized Provider Order Entry Systems (CPOE)



Promise of Order-Entry Systems



Some Advantages of CPOE Systems



A Drug-Drug Interaction Scenario

• “When ordering a new medication, a prescriber may not be aware that two drugs interact, or may not be keeping in mind the other medications that the patient is taking.

• As an example, consider the case of a hospitalized patient who is being treated with venlafaxine (Effexor) for chronic depression and develops an infection with a drug resistant bacterium requiring treatment with linezolid, a new antimicrobial agent.

• The interaction between linezolid and venlafaxine (serotonin syndrome -- altered mental status, including agitation, confusion and coma, neuromuscular hyperactivity, and autonomic dysfunction) is very severe but may not be known to the practitioner.

• While writing the order for linezolid, an alert screen can warn the practitioner that these two drugs should not be used together.

• The alert screen may offer the prescriber the opportunity to cancel the order, to discontinue

• the existing medication that interacts with the newly ordered medication, or to order a test that could detect the interaction or monitor therapy. The alert screen may prompt the physician to have a conversation with the patient regarding potential side effects of the medications. Any of these consequences of the decision support software could be beneficial.”

Kuperman , G., et al, (2007).



Challenges with Order Entry



CPOE Paradox



Cognitive Evaluation of Interaction with a CDSS



Weight-Based Heparin Ordering



Methods



CPOE Screen

Horsky, J., Kaufman, D. R., & Patel, V. L. (2005).



Decision Support Alert Weight-based IV Heparin Protocols

Horsky, J., Kaufman, D. R., & Patel, V. L. (2005).



Results - Presentation Salience



Results - User Behavior



Summary



Role of CPOE Systems in Facilitating Medical Errors



Information Errors: Fragmentation and Systems Integration Failure 1



Information Errors: Fragmentation and Systems Integration Failure 2



Human-Machine Interface Flaws



Automation Bias



Anti-Automation Bias

• Errors of dismissal, where computer advice is ignored

• Clinicians routinely disable or ignore the alarms or alerts on clinical monitoring devices – Legitimate reasons such as high false alarm

rates [or repetition of the same alarms – Less valid reasons such as not wanting to be

interrupted



Barriers to Prescriber Decision-Making and Clinical Workflow



15 Barriers to Prescriber Decision-Making

BARRIER DESCRIPTION

1. Poor Screen Display

Alert display does not support alert resolution and/or prescriber workflow

2. Inadequate Alert Specification

Alert does not provide information on why it was triggered and/or the potential problem

3. Actual or Perceived Lack of Evidence

Alert is not evidence-based, does not provide a reference to evidence that does exist

4. Unclear Level of Risk

Alert does not provide clear information on relative risk of harm for a given patient

5. Redundancy Repeated alerts within the same encounter or over multiple encounters for a given patient

1.1 Table: (Kaufman, 2010).



Barriers Continued 6. Low Alert Signal to Noise Ratio

Numerousness of alerts leads to information overload, prescriber desensitization, and potential for missing key alerts

7. Inadequate Allergy Logic

Alert system does not distinguish between true allergies and bothersome, but non-serious, side effects

8. Duplicate Workload

Alert duplicates other required work processes

9. Paper Prescriptions & Limited CPOE

Some medications are not or cannot be entered electronically, and therefore, are not reviewed by the alert system

10. Unclear System Capabilities

Alert system does not adequately reveal its capabilities/limitations to the prescriber; full functionality of the alert system is ambiguous

1.2 Table: (Kaufman, 2010).



Barriers Elaborated



Human Factors and Information Management



Situation Awareness



Mental Workload



CPOE/CDSS Design Recommendations



Designing for Better Workflow




Summary

• Computer-based decision support systems offer great promise for reduction of errors in medicine and facilitation of quality patient care

• Results on system efficacy have thus far been equivocal– CDSS can lead to new types of errors

• Adherence to usability/human factors principles can lead to superior design and enhanced performance




ReferencesReferences:

Kuperman, G. J., Bobb, A., Payne, T. H., Avery, A. J., Gandhi, T. K., Burns, G., et al. (2007). Medication-related clinical decision support in computerized provider order entry systems: a review. J Am Med Inform Assoc, 14(1), 29-40.

Coiera E, Westbrook J, Wyatt J. The safety and quality of decision support systems. Methods Inf Med. 2006;45 Suppl 1(suppl 1):20–5.me06010020

Eddy, D. M. (1982). Probabilistic reasoning in clinical medicine: Problems and opportunities. In D. Kahneman, P. Slovic & A. Tversky (Eds.), Judgment under uncertainty: Heuristics and biases (pp. 249-267). Cambridge, England: Cambridge University Press.

Horsky, J., Kaufman, D. R., & Patel, V. L. (2005). When you come to a fork in the road, take it: strategy selection in order entry. AMIA Annu Symp Proc, 350-354.

Institute of Medicine Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press, 2001.

J.A. Osheroff, J.M. Teich, B. Middleton, E.B. Steen, A. Wright and D.E. Detmer, A roadmap for national action on clinical decision support, J Am Med Inform Assoc 14 (2) (2007), pp. 141–145.

Karsh B-T. Clinical practice improvement and redesign: how change in workflow can be supported by clinical decision support. AHRQ Publication No. 09-0054EF; Rockville (MD): Agency for Healthcare Research and Quality; June 2009.

Khajouei R, Jaspers MW: CPOE system design aspects and their qualitative effect on usability. Stud Health Technol Inform 2008 , 136:309-14




ReferencesReferences:

Koppel, R., Metlay, J. P., Cohen, A., Abaluck, B., Localio, A. R., Kimmel, S. E., et al. (2005). Role of computerized physician order entry systems in facilitating medication errors. JAMA, 293(10), 1197-1203.

Kuperman, G. J., Bobb, A., Payne, T. H., Avery, A. J., Gandhi, T. K., Burns, G., et al. (2007). Medication-related clinical decision support in computerized provider order entry systems: a review. J Am Med Inform Assoc, 14(1), 29-40.

McNeil, B. J., Pauker, S. G., Sox, H. C., Jr., & Tversky, A. (1982). On the elicitation of preferences for alternative therapies. N Engl J Med, 306(21), 1259-1262.

Reckmann, M. H., Westbrook, J. I., Koh, Y., Lo, C., & Day, R. O. (2009). Does computerized provider order entry reduce prescribing errors for hospital inpatients? A systematic review. J Am Med Inform Assoc, 16(5), 613-623.

Russ, A. L., Zillich, A. J., McManus, M. S., Doebbeling, B. N., & Saleem, J. J. (2009). A human factors investigation of medication alerts: barriers to prescriber decision-making and clinical workflow. AMIA Annu Symp Proc, 2009, 548-552.

Teich JM, Osheroff JA, Pifer EA, Sittig DF, Jenders RA, Panel CDSER. Clinical decision support in electronic prescribing: Recommendations and an action plan. J Am Med Inform Assoc 2005;12(4):365-76.

Images:

Slide 18: Retreived on October 4th, 2010 form http://www.flickr.com/photos/bojo/4078685614/lightbox/#/

Slide 32: Horsky, J., Kaufman, D. R., & Patel, V. L. (2005). When you come to a fork in the road, take it: strategy selection in order entry. AMIA Annu Symp Proc, 350-354

Slide 33: Horsky, J., Kaufman, D. R., & Patel, V. L. (2005). When you come to a fork in the road, take it: strategy selection in order entry. AMIA Annu Symp Proc, 350-354



http://www.flickr.com/photos/bojo/4078685614/lightbox/

usability & human factors decision support systems: a human factors approach this material...

Documents

human factors approachsurvival

human factors approachclassic

human services

workforce curriculum

department of health

probability of breast

health information technology

positive mammogram