if you build it, will they come? · 2019-06-05 · if you build it, will they come? moving...
TRANSCRIPT
![Page 1: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/1.jpg)
@lauradesveaux @NoahIvers
If you build it, will they come?
Moving recipients from apathy to action
Laura Desveaux PhD, PT
lauradesveaux
Noah Ivers MD, PhD
NoahIvers
#AF2019
![Page 2: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/2.jpg)
Agenda#AF2019
• Introductions
• Review of theory
• Understanding failure and success
• Engaging end users
• Group activity → Practical strategies
• The role of context
• Group activity → Supportive coaching
![Page 3: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/3.jpg)
#AF2019@lauradesveaux @NoahIvers
RECOGNIZE OUR ASSUMPTIONS
3
![Page 4: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/4.jpg)
#AF2019@lauradesveaux @NoahIvers
BEFORE WE BUILD IT, WE NEED TO KNOW HOW IT WORKS
4
A&F Best Practices
![Page 5: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/5.jpg)
#AF2019@lauradesveaux @NoahIvers 5
Brown et al, Imp Sci 2019; 14:40.
![Page 6: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/6.jpg)
#AF2019@lauradesveaux @NoahIvers
Learning from failure
2 x 2 FACTORIAL TRIAL
Top 25%
- Frame
Mean
- Frame
Top 25%
+ Frame
Mean
+ Frame
Does the design of the report improve performance?
Learning from Failure
![Page 7: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/7.jpg)
#AF2019@lauradesveaux @NoahIvers 7
![Page 8: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/8.jpg)
#AF2019@lauradesveaux @NoahIvers 8
![Page 9: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/9.jpg)
#AF2019@lauradesveaux @NoahIvers
RESULTS OF LTC FACTORIAL
TRIAL?
9
![Page 10: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/10.jpg)
#AF2019@lauradesveaux @NoahIvers
…BUT ENGAGEMENT WAS SUBOPTIMAL
10
Learning from Failure
![Page 11: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/11.jpg)
#AF2019@lauradesveaux @NoahIvers
ENGAGEMENT
11
60%REGISTERED
38%
REACH23%
14%
33%
5%
Primary Care LTC
![Page 12: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/12.jpg)
#AF2019@lauradesveaux @NoahIvers
ENGAGEMENT DOESN’T JUST HAPPEN
12
A&F Best Practices
CoherenceCognitive
Participation(Collective)
ActionReflexive Monitoring
![Page 13: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/13.jpg)
#AF2019@lauradesveaux @NoahIvers
Strategies to Increase Engagement
Understand workflow
• Where is the drop-off?
• Where can you intervene?Consider the
evidence
• What has worked?
• How was it operationalized?Seek user feedback
• What do they say?
• How do they react?
![Page 14: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/14.jpg)
#AF2019@lauradesveaux @NoahIvers
Strategies to Increase Engagement
Select techniques and draft content
Recruit users and non-users
Co-create new content
Pretest content
Finalize operational details
Priority is to gauge user response vs. feedback
Bravo et al, JMIR Hum Factors 2018; 5:3.
1
2
3
4
5
![Page 15: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/15.jpg)
#AF2019@lauradesveaux @NoahIvers
Strategies to Increase Engagement
Remember:
User preference ≠ Scientific evidence
Consider context
![Page 16: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/16.jpg)
#AF2019@lauradesveaux @NoahIvers
Strategies to Increase Engagement
Vaisson et al, Curr Oncol 2019 [In press]
![Page 17: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/17.jpg)
#AF2019@lauradesveaux @NoahIvers
Strategies to Increase Engagement
Minor changes can make a small difference in engagement
Anticipated Regret
(n=2642)
Material Incentive
(n=2777)
Problem Solving
(n=2812)
Primary outcome: Any SAR logins during trial
SAR accessed (%) 21.8% 20.3% 19.5%
Adjusted RR [ 95% CI ] 1.07 [0.97-1.18] 0.95 [0.87-1.05] 0.87 [0.79-0.96]
Secondary outcome: Total number of SAR unique log-ins
Adjusted RR [ 95% CI ] 1.09 [0.95-1.25] 0.89 [0.77-1.02] 0.81 [0.71-0.93]
Secondary outcome: Adherence to cancer screening
Breast ca screening aRR [95% CI] 1.000 [0.997-1.002] 1.000 [0.997-1.003] 0.998 [0.995-1.001]
Cervical ca screening aRR [95% CI] 1.002 [0.998-1.003] 1.000 [0.998-1.002] 1.003 [1.001-1.006]
Colon ca screening aRR [95% CI] 1.000 [0.998-1.002] 1.000 [0.998-1.002] 1.000 [0.998-1.002]
![Page 18: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/18.jpg)
#AF2019@lauradesveaux @NoahIvers
Interactive Activity
What makes you engage?
![Page 19: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/19.jpg)
#AF2019@lauradesveaux @NoahIvers
Interactive Activity
Group Activity:
Designing Feedback
![Page 20: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/20.jpg)
#AF2019@lauradesveaux @NoahIvers
Interactive Activity1. Pair up with someone you don’t know
2. Explain your job & responsibilities (1 min each)
3. Guess feedback parameters for your partner (5 min)
→ Topics, Delivery, Source
4. Write down your ideal feedback parameters (5 min)
5. Discuss alignment and rationale
6. Would you engage with the feedback? Why?
7. Identify themes (if able)
![Page 21: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/21.jpg)
#AF2019@lauradesveaux @NoahIvers
What we’ve learned
How do you best deliver feedback?
It depends.
![Page 22: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/22.jpg)
#AF2019@lauradesveaux @NoahIvers
Learning from Failure
What is meaningful engagement?
![Page 23: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/23.jpg)
#AF2019@lauradesveaux @NoahIvers
• Goal must be clearly stated
• Utility must align with recipient goals
• Perceptions of feedback influence engagement
• Approach to practice influences perceptions of
feedback
MEANINGFUL CONTENT >> A STRONG VISUAL
Learning from Failure
![Page 24: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/24.jpg)
#AF2019@lauradesveaux @NoahIvers
Learning from failure
ESTABLISH MEANINGFUL CONTENT FIRST
…THEN CONSIDER USABILITY AND RELATIVE ADVANTAGE
![Page 25: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/25.jpg)
#AF2019@lauradesveaux @NoahIvers 25
Brown et al, Imp Sci 2019; 14:40.
![Page 26: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/26.jpg)
#AF2019@lauradesveaux @NoahIvers
Learning from failure
Threats to meaningful engagement:
The data sucks
My patients are different
I don’t know what to do with the data
1
2
3
How do primary care physicians engage with A&F?
![Page 27: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/27.jpg)
#AF2019@lauradesveaux @NoahIvers
Tenants of Feedback Intervention TheoryHysong et al, BMJ Qual Saf 2017; 26:4.
![Page 28: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/28.jpg)
@lauradesveaux @NoahIvers
Bridging the gap:
Facilitated Feedback
![Page 29: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/29.jpg)
#AF2019@lauradesveaux @NoahIvers
Facilitated Feedback
R2C2 → an evidence-based model on delivering
effective feedback to physicians
29
Rapport Building
Explore Reactions
Explore Content
Coach for Change
Sargeant et al, Acad Med 2015; 90(12).
![Page 30: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/30.jpg)
#AF2019@lauradesveaux @NoahIvers
Interactive Activity
Group Activity:
Feedback Coaching
![Page 31: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/31.jpg)
#AF2019@lauradesveaux @NoahIvers
Interactive Activity
1. Get back in your pairs
2. Designate one person as the coach and the other as
the recipient (who is now a family doctor)
3. Discuss the following feedback
![Page 32: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/32.jpg)
#AF2019@lauradesveaux @NoahIvers 40
High Dose Opioids Antibiotic prescribing rate
You Average
20
40
50
30
10
%
High Dose Opioids Antibiotics
![Page 33: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/33.jpg)
#AF2019@lauradesveaux @NoahIvers
Group Facilitation
41
Cooke et al, Imp Sci 2018; 13:136.
Precursors
Relationship building
Question choice
Usability
![Page 34: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/34.jpg)
#AF2019@lauradesveaux @NoahIvers
Group Facilitation
42
Cooke et al, Imp Sci 2018; 13:136.
• Navigate through reactions to data
• Understand their data
• Create a plan for change
Note: A “coaching-oriented approach” with prompts is essential
![Page 35: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/35.jpg)
#AF2019@lauradesveaux @NoahIvers
Positive Deviance Approach
• Identify organizations or individuals that are performing
well
• Highlight key strategies/practices that enable high
performance
• Share strategies with feedback recipients
43
Lawton et al, BMJQS 2014; 23.
![Page 36: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/36.jpg)
@lauradesveaux @NoahIvers
![Page 37: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/37.jpg)
#AF2019@lauradesveaux @NoahIvers
‘Beneficent Persuasion*’: How will data → QI?
Motivation (goal-commitment, social comparison, regret)
i.e., address outcome expectancy with relative risk reduction rather than number needed to treat
Cochrane 2011 Mar 16;(3):CD006776
Capability (self-efficacy, graded-entry, coaching)
i.e., connect trigger to desired action and plan for barriers
Psychol Health 2005, 20:143-160
Opportunity (addressing barriers, co-interventions)
i.e., make the desired action easier or no effort at all
NEJM. 2007 Sep 27;357(13):1340-4
*Ann Fam Med. 2010 May; 8(3): 260–26445
![Page 38: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/38.jpg)
#AF2019@lauradesveaux @NoahIvers 46
![Page 39: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/39.jpg)
#AF2019@lauradesveaux @NoahIvers
Design must
be fit for
purpose
![Page 40: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/40.jpg)
#AF2019@lauradesveaux @NoahIvers
Be clear
about the
purpose
PURPOSE FUELS APPETITE
![Page 41: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/41.jpg)
#AF2019@lauradesveaux @NoahIvers
Help people
understand
their score and
make plans to
improve
![Page 42: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/42.jpg)
#AF2019@lauradesveaux @NoahIvers
Be realistic
about your
goals
![Page 43: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/43.jpg)
#AF2019@lauradesveaux @NoahIvers
Help people
improve
together by
changing
processes, not
trying harder
![Page 44: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/44.jpg)
#AF2019@lauradesveaux @NoahIvers 52
![Page 45: If you build it, will they come? · 2019-06-05 · If you build it, will they come? Moving recipients from apathy to action Laura Desveaux PhD, PT laura.desveaux@wchospital.ca lauradesveaux](https://reader033.vdocuments.site/reader033/viewer/2022043015/5f37fdbce42518759114c12e/html5/thumbnails/45.jpg)
@lauradesveaux @NoahIvers