leading a horse to water: using automated reminders to increase use of online decision support
DESCRIPTION
Leading a Horse to Water: Using Automated Reminders to Increase Use of Online Decision Support. James J. Cimino and Dmitriy Borovtsov NIH Clinical Center and Department of Biomedical Informatics Columbia University College of Physicians and Surgeons. Knowledge Resources are Underused. - PowerPoint PPT PresentationTRANSCRIPT
Leading a Horse to Water:Using Automated Reminders to Increase
Use of Online Decision Support
James J. Cimino and Dmitriy Borovtsov
NIH Clinical Center
andDepartment of Biomedical Informatics
Columbia University College of Physicians and Surgeons
Knowledge Resources are Underused
Clinicians (physicians) with questions in clinical practice:
– sought answers only 45% of the time
– found answers only 34% of the time
– used online resources 18% of the time
* Ely JW, Osheroff JA, Maviglia SM, Rosenbaum ME. Patient-care questions that physicians are unable to answer. J Am Med Inform Assoc. 2007 Jul-Aug;14(4):407-14.
Health Resources
• Available from main menu
• Static list of resources
• No assistance with retrieval
Infobuttons
• Links inserted next to clinical information
• Resources selected based on likely information need
• Links are customized to assist with retrieval
Log File of HR vs IB 2006
2006 Health ResourcesInfobuttons
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
7000
6000
5000
4000
3000
2000
1000
0
Monthly Usage
Log File of HR vs IB 2006, 2007
2006 2007Health ResourcesInfobuttons
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
7000
6000
5000
4000
3000
2000
1000
0
Monthly Usage
Educational Interventions
• Orientation for new housestaff
• Orientation for new medical students
• Infobutton exercises for new medical students
Educational Interventions
2006Medical StudentsHousestaff
Jun Jul Aug Sep Oct Nov Dec
Monthly Usage
450
400
350
300
250
200
150
100
50
0
Educational Interventions
2006 2007Medical StudentsHousestaff
Jun Jul Aug Sep Oct Nov Dec
450
400
350
300
250
200
150
100
50
0
Monthly Usage
Usage of IB vs HR in Inpatient Meds
Health ResourcesInfobuttons
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
1400
1200
1000
800
600
400
200
0
Monthly Usage
Usage of IB in Outpatient Meds
Health ResourcesInfobuttons
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
800
700
600
500
400
300
200
100
0
Monthly Usage
Hypothesis
Users of the Health Resources page in a particular context will be especially receptive to a suggestion to use Infobuttons in the same context.
SendReminder
No
StopYesReminder
Sent in past2 months?
No
StopYesIB Used
in past2 months?
InterventionHR used in OM
HR = Health Resources PageOM = Outpatient Medications
E-MailDate: Fri, 2 Mar 2007 06:00:11 -0500 (EST)From: James Cimino <[email protected]>To: [email protected]: Getting Drug Information while Using WebCIS
Dear WebCIS User:
I am writing to let you know about a handy feature in WebCIS called the "infobutton". You will see it in places like the Outpatient Medication list - it is a little purple circle with a white letter "i". If you click on it, it will give you a list of topics that you can select to get more information about the drug next to the infobutton.
I hope you'll find it useful.
--Jim Cimino, Infobutton Project Manager
Reminders Sent
• 522 messages• 371 users
– 80 attending physicians– 189 housestaff– 29 nurses– 24 students– 49 other/unknown
• 173 users received two messages• 1 user received three messages• Average 89.9 days between messages
Usage of Infobuttons by Reminder Recipients
• 371 recipients of 552 reminders• 111 (20.1%) eventually used IBs in OM• 52 (9.4%) “early responders”
– 26 housestaff– 18 attending physicians– 8 other/unknown
• 201 used either IB or HR within one month• 25.9% “early responders with opportunity”• 9 early responders used IBs 8 to 97 times in
subsequent months
Intervention versus Control
552 Intervention525 Control
Days to First Use of Infobuttons after First Use of Health Resources Page
P<0.05 by Day 4
Effect of delay
0
100
200
300
400
1-2Days
3-5Days
>5Days
RemindersSent
EarlyResponders
Adverse Effects
• 522 reminders
• 12 replies– All were positive– Most were thanking us for infobuttons
Discussion
• Traditional training method appears to have limited effect
• Simple, spam-like reminder appears to have an effect on a substantial fraction (26%) of users
• Sustained effect in some (17%) of the early responders
Limitations
• Impersonal message
• Timing of reminder less-than ideal
• However: Logfile+e-mail approach is application independent
• Quasi-experimental design
• Convenience sample of controls
• However: Strong temporal association between intervention and outcome
Conclusions
• Automated, context-specific reminders are technically feasible, requiring few resources
• Reminders to use online information resources appear to be effective
• Timing of message is critical to success
Acknowledgments
This work was supported in part by:National Library of Medicine Grant R01LM07593
NIH Clinical Center intramural research funds
National Library of Medicine intramural research funds
The authors also thank:Dr. Jianhua Li for technical support
Dr. Rick Gallagher for log files
Dr. Krystl Haerian for statistical support