user-centered and theory-based design of a professional ......user-centered and theory-based design...
TRANSCRIPT
-
Lawani et al. BMC Med Inform Decis Mak (2021) 21:59 https://doi.org/10.1186/s12911-021-01396-y
RESEARCH ARTICLE
User-centered and theory-based design of a professional training program on shared decision-making with older adults living with neurocognitive disorders: a mixed-methods studyMoulikatou Adouni Lawani1, Yves Turgeon2, Luc Côté3, France Légaré4, Holly O. Witteman1, Michèle Morin5, Edeltraut Kroger6, Philippe Voyer7, Charo Rodriguez8 and Anik Giguere9*
Abstract Background: We know little about the best approaches to design training for healthcare professionals. We thus stud-ied how user-centered and theory-based design contribute to the development of a distance learning program for professionals, to increase their shared decision-making (SDM) with older adults living with neurocognitive disorders and their caregivers.
Methods: In this mixed-methods study, healthcare professionals who worked in family medicine clinics and home-care services evaluated a training program in a user-centered approach with several iterative phases of quantitative and qualitative evaluation, each followed by modifications. The program comprised an e-learning activity and five evidence summaries. A subsample assessed the e-learning activity during semi-structured think-aloud sessions. A sec-ond subsample assessed the evidence summaries they received by email. All participants completed a theory-based questionnaire to assess their intention to adopt SDM. Descriptive statistical analyses and qualitative thematic analyses were integrated at each round to prioritize training improvements with regard to the determinants most likely to influence participants’ intention.
Results: Of 106 participants, 98 completed their evaluations of either the e-learning activity or evidence summary (93%). The professions most represented were physicians (60%) and nurses (15%). Professionals valued the e-learning component to gain knowledge on the theory and practice of SDM, and the evidence summaries to apply the knowl-edge gained through the e-learning activity to diverse clinical contexts. The iterative design process allowed address-ing most weaknesses reported. Participants’ intentions to adopt SDM and to use the summaries were high at baseline and remained positive as the rounds progressed. Attitude and social influence significantly influenced participants’ intention to use the evidence summaries (P < 0.0001). Despite strong intention and the tailoring of tools to users,
© The Author(s) 2021. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creat iveco mmons .org/licen ses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creat iveco mmons .org/publi cdoma in/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
Open Access
*Correspondence: [email protected] VITAM Research Centre on Sustainable Health, Pavillon Landry-Poulin, Door A-1-2, 2nd floor, Room 2416, 2525 Chemin de la Canardière, Québec, QC G1J 0A4, CanadaFull list of author information is available at the end of the article
http://orcid.org/0000-0001-9928-7395http://creativecommons.org/licenses/by/4.0/http://creativecommons.org/publicdomain/zero/1.0/http://creativecommons.org/publicdomain/zero/1.0/http://crossmark.crossref.org/dialog/?doi=10.1186/s12911-021-01396-y&domain=pdf
-
Page 2 of 25Lawani et al. BMC Med Inform Decis Mak (2021) 21:59
BackgroundNeurocognitive disorders require making several difficult decisions to ensure older adults remain independent as long as possible, while maintaining their well-being and safety [1]. These decisions may cover daily life manage-ment (e.g. being home alone, day care, transportation, home book- keeping), arranging healthcare and support (e.g. diagnosis, medications, home care, domestic help, and respite care), community life (e.g. visiting family, moving house), and representing the person with demen-tia (e.g. advanced decisions about the end-of-life) [1]. Decisions made by older adults with neurocognitive dis-orders and their caregivers are generally not only based on clinical information, but also on social considerations (e.g. financial insecurity, availability of community-based organizations, social networks), requiring professionals to expand their knowledge and scope of practice beyond the boundaries of their professions [1, 2]. Since several acceptable alternatives exist for most of these decisions, the priorities of the older adults and those of their family/friend caregivers should guide decision making, together with the scientific evidence on the benefits and harms of the available options [3]. Shared decision-making (SDM) is an ideal approach for supporting older adults and their significant others in making these decisions collabo-ratively with the interprofessional healthcare team, as SDM is typically used in the context of uncertainty when the person’s preferences are central to the decision [4]. SDM is an approach where clinicians and patients share the best available evidence when faced with the task of making decisions, and where patients are supported to consider options, to achieve informed preferences [5]. Primary healthcare professionals (HCPs) should be trained in SDM and have access to patient decision aids tailored to the needs of older adults living with neurocog-nitive disorders and their caregivers, as these profession-als play a central role in the care and services provided to community-based older adults living with neurocognitive disorders [6–8].
Although SDM can improve the quality of life of patients with neurocognitive disorders and their car-egivers [9], decision-making in this context can be
challenging. The disabling and neurodegenerative nature of neurocognitive disorders may challenge decision-mak-ing by limiting communication with the person as the disease progresses [10–12]. Consequently, neurocogni-tive disorders are a major risk factor for exclusion from decision-making [11]. Some studies have described the issues and requirements involved with implementing SDM with this population [1, 2, 13–17], but as yet, there have been no studies on the essential characteristics of a training program in SDM for HCPs in caring for older adults with neurocognitive disorders.
Initiatives aimed at increasing the use of SDM by HCPs may comprise training programs, leaflets, financial incentives or email reminders; however evidence remains scarce on their effectiveness to change professional behaviour and improve patient/caregiver participation in decision-making [18, 19]. This project focuses on two strategies to facilitate the implementation of professional training programs in SDM. Firstly, these programs should consider the logistical challenges of attending educa-tional meetings for HCPs, especially those who work in remote areas [20]. In this project, we thus propose a distance training program that is accessible to all profes-sionals, even those living away from the larger centres where continuing professional development activities generally take place. Secondly, we propose using user-centered design to tailor a professional training pro-gram to the actual needs and barriers faced by HCPs, as this is a promising approach to ensure that training on SDM leads to actual behaviour change [21, 22]. Indeed, evidence from systematic reviews shows that continu-ing professional development programs built on well-conducted needs assessments are effective in changing clinicians’ behaviours [23]. Training needs assessments have traditionally been achieved through practice audits, questionnaires, environmental scans, or interviews, but the value of user-centered design to this end remains unexplored. User-centered design, which includes design thinking, consists of involving target users in several iter-ative rounds of evaluations and modifications, to tailor the design of a product to a given task and to the user’s experience [24–27]. In the field of healthcare, preliminary
certain factors external to the training program can still influence the effective use of these tools and the adoption of SDM in practice.
Conclusions: A theory-based and user-centered design approach for continuing professional development inter-ventions on SDM with older adults living with neurocognitive disorders and their caregivers appeared useful to identify the most important determinants of learners’ intentions to use SDM in their practice, and validate our initial interpretations of learners’ assessments during the subsequent evaluation round.
Keywords: Dementia, Aging, Continuing professional development, Curricular development, User experience, Intervention design, Behaviour change technique, Implementation
-
Page 3 of 25Lawani et al. BMC Med Inform Decis Mak (2021) 21:59
evidence suggests that user-centered design may enhance the implementation in practice of evidence-based infor-mation [26, 28, 29] and patient decision aids [30–33].
In this research project, we studied how user-centered design and theory contribute to the development of a dis-tance learning program to support HCPs in implement-ing SDM with older adults living with neurocognitive disorders and their family/friend caregivers. More specif-ically, the investigation was guided by the following over-arching research question: What are the features of the training program and design strategies that may increase HCPs’ intention to adopt SDM in this clinical context?
Theoretical frameworkSDM requires the adoption of a diverse set of behav-iours by HCPs [34, 35]. This project was thus based on the integrated framework proposed by Godin and al, according to which a behaviour may be predicted by a person’s intention (motivation) to adopt it (Fig. 1) [36]. A person’s intention may, in turn, be predicted by sev-eral determinants, including belief about consequences (the perceived advantage or disadvantage of adopting a behaviour), social influence (the perceived social pressure to adopt a behaviour), and beliefs about capabilities (per-ceived ease or difficulty of adopting a behaviour) [36]. In addition, intention can also be determined by habits/past behaviours and other social and emotional factors,
namely moral norms (the feeling of being obliged to adopt a behaviour) and role/identity (beliefs that a per-son of similar age, sex, or social position should adopt a behaviour) [36]. We then added to this general model the Technology Acceptance Model (TAM-2), which identifies usefulness and ease of use as two specific determinants which could predict users’ intention to use new informa-tion technology/information systems [37]. Despite the fact that these two domains are similar to, respectively, the Beliefs about Consequences and Beliefs about Capa-bilities domains described in the integrated framework, we added them to draw more attention to acceptance (usability and acceptability) of the two studied learning components [38], and thus complement the integrated framework, which is focused on motivation. Use of both the TAM-2 and integrated framework constructs allowed evaluating two dimensions of intention, namely motiva-tion (integrated framework) and acceptance (TAM-2).
MethodsStudy design and approachThis was a convergent, parallel, mixed-method study in which three HCP subsamples participated in the tailoring of a professional training program. A first subsample helped tailor the first component of the program (the e-learning activity), whereas the second and third helped tailor the second component (a series of five evidence summaries).
Beliefs about Capabilities
Moral Norm
Social Influence
Beliefs about Consequence
BEHAVIOUR INTENTION
Role and Identity
Individual Characteristics
Ease of Use
Usefulness
Fig. 1 Theoretical model framing the current research
-
Page 4 of 25Lawani et al. BMC Med Inform Decis Mak (2021) 21:59
We used user-centered design approaches for tailoring each component, by improving user experience of the pro-totypes during several iterative evaluations that each com-prised both quantitative and qualitative data collection [39].
Professional training program on shared decision makingDetails on the training program are reported elsewhere [40]. Briefly, the training program included two modali-ties: (1) a self-directed e-learning activity on SDM, and (2) five evidence summaries, decision boxes (DBs), to support decision-making at the point of care with older adults liv-ing with neurocognitive disorders and their caregivers. The self-directed e-learning activity covered the use of patient decision aids to implement SDM. It lasted about one hour, could be completed in several sittings, and was not specific to any clinical area. It was interactive and used narrated slides, videos, and interactive exercises. It also offered an asynchronous forum to discuss any question with an expe-rienced moderator.
The second modality of the training program consisted of a series of five evidence summaries describing the options available to older adults living with neurocognitive disor-ders who face five important and frequent decisions that we identified in a previous study [41]. These decisions were: (#1) choosing a support option to decrease caregiver bur-den; (#2) choosing a non-pharmacological treatment to manage agitation, aggression, or psychotic symptoms; (#3) deciding whether or not to stop driving following diagno-sis; and (#4) choosing an option to improve quality of life; and (#5) deciding whether or not to prepare a power of attorney (called a “protection mandate” in Quebec, Can-ada) covering health, property, and financial matters;. The evidence summaries followed the decision box (DB) tem-plate, which aims to provide stakeholders with evidence in a format that supports them in SDM [42, 43]. Biefly, these summaries met several of the standards set by the International Patient Decision Aids Collaboration [44]: (1) they described the health condition for which a decision is required; (2) they explicitly stated the decision to be taken into consideration; (3) they described all the options availa-ble for this decision; and (4) they described the positive and negative characteristics of each option. Their content was developed from rapid reviews and then revised by clinical experts, as described earlier [45]. The studied summaries
are available at www.decis ionbo x.ulava l.ca/en/ (Series on Older Adults – Problems with Thinking or Memory).
Population and recruitment strategyWe recruited convenience samples of HCPs from any profession (e.g., family physicians, nurses, and social workers) who practiced in family medicine clinics or homecare services in the province of Quebec, Canada. Of the primary care settings invited to participate in the project (46 clinics and 8 homecare services), 20 agreed to participate (16 clinics, 4 homecare services). Figure 2a, b describe the sample distribution for each training component. We carried three or four evalua-tions/tailoring rounds, with at least five HCPs during each round. These numbers respect human factors vali-dation testing [46].
Design process of the e‑learning activityE‑learning activity evaluationsAt study entry, all study participants completed a ques-tionnaire on their sociodemographic and professional characteristics. The study participants assigned to eval-uate the e-learning activity also completed a question-naire before and after exposure to the activity, to assess: (1) their preferred role in decision-making using the Control Preferences Scale [47, 48]; (2) their perception of the usefulness and ease of use of the program based on TAM-2 [49]; (3) their level of intention to adopt SDM, and the factors influencing that intention using the CPD-REACTION instrument [50]. To allow calcu-lating means for each of the studied factors, we added three items to the original CDP-REACTION instru-ment, giving it three items per domain.
These participants also assessed the e-learning activ-ity during a semi-structured think-aloud session that was screen-captured and audio-recorded using Flash-back (Blueberry Software). One of two trained mod-erators (DC, YT) conducted these sessions. After each section of the training, the moderator asked partici-pants about their perceptions on the content and learn-ing strategies used, and recorded any usability issues. At the end of the session, the moderator also asked participants to comment on the main benefits and inconveniences of SDM, which allowed us to describe the factors influencing participants’ beliefs about the
Fig. 2 User-centered design process to tailor the training program to user needs. DB #1 = Choosing a support option to decrease caregiver burden; DB #2 = Choosing a non-pharmacological treatment to manage agitation, aggression, or psychotic symptoms; DB #3 = Deciding whether or not to stop driving following diagnosis; DB #4 = Choosing an option to improve quality of life; DB #5 = Deciding whether or not to prepare a power of attorney; HCP = HCP; v = version
(See figure on next page.)
http://www.decisionbox.ulaval.ca/en/
-
Page 5 of 25Lawani et al. BMC Med Inform Decis Mak (2021) 21:59
-
Page 6 of 25Lawani et al. BMC Med Inform Decis Mak (2021) 21:59
consequences of using SDM, one of the components of the theoretical framework guiding this work (Fig. 1).
E‑learning activity tailoringInspired by Susan Michie’s mapping approach, we modified the e-learning activity by adding/enhanc-ing behaviour-change techniques targeting the factors likely to limit the adoption of SDM at each round [51, 52]. To this end, we initially imported the transcripts of the think-aloud sessions as source documents using NVivo coding software (QSR International, version 12). Two researchers (DC, AMCG) conducted a deductive/inductive thematic qualitative analysis of the factors likely to limit adoption of SDM [53]. More specifically, we initially used the theoretical domains framing the questionnaire as themes, and then added new themes as needed. We identified the weaknesses and strengths of the e-learning activity within each theme. We inter-preted qualitative and quantitative data together and, following this analysis, the coders (DC, AMCG), mod-erators (DC, YT), principal investigators (AMCG, FL), and a human factors expert co-investigator (HOW) discussed the functionality of the tutorial and modifica-tions to improve functionality and modify the program accordingly.
Design process of the decision boxes (DBs)Decision boxes evaluationsThe study participants assigned to evaluate the DBs received emails with a link to access a web-based ques-tionnaire to evaluate each DB, at a rate of one per week for five weeks. The DBs were also available on a website. The questionnaire served to (1) explain the purpose of the DB; (2) allow participants to access the DB under evaluation by clicking on a link; and (3) ask a series of questions about what they thought of the DB.
Congruent with the theoretical approach adopted, we used two questionnaires to assess the psychological con-struct ‘intention’ and its potential determinants. We used the CPD-REACTION questionnaire, which is based on the integrated framework described above and was cre-ated as a routine assessment of the impact of continuing professional education on practice [50]. We also assessed the usefulness and ease of use of the DBs, based on the TAM-2 [49]. For each of the CPD-REACTION and TAM-2 items described above, participants rated their perceptions using Likert items ranging from 1–7 (with 1 being the lowest). If ratings fell below four for any item, we then asked the HCPs to explain the reason for their rating in an open-text field, to allow us to understand the barriers they perceived to adopting SDM. At the end of the questionnaire, an open-text field also invited
participants to include any additional feedback on the DB.
We sent two weekly reminders to participants who had not completed their evaluations.
Decision boxes tailoringAfter each round, we used descriptive statistics to sum-marize participant ratings. We also imported the quali-tative comments made in the open-text fields of the questionnaire into a specialized software program (NVivo), and two researchers (MAL, AMCG) analyzed them using a thematic deductive/inductive qualitative analysis approach [53]. We initially used the theoreti-cal domains framing the questionnaire as themes, then added subthemes as needed [53]. We identified the weaknesses and strengths of the DBs within each of the theoretical domains, then broke them down further into emerging themes, to describe the weakness or strength. We resolved any disagreement by consensus between the two researchers.
An interdisciplinary expert panel subsequently met to review the qualitative and quantitative findings, and identified strategies to improve the DB, so as to limit the identified weaknesses. The panel consisted of a graphic designer (JB), a human factors engineer (HOW), an epi-demiologist (MAL), and four knowledge-translation researchers (AMCG, HOW, MAL, DC). The experts started by prioritizing each of the problems uncovered. Then, we determined the most appropriate solutions by considering the magnitude, frequency, and sever-ity of these problems, and modified the DBs so that HCPs could explain the pros and cons of health options, as understood from the DBs, to patients and their caregivers.
We used the same evaluation/tailoring process again in two more rounds, with new participants each time.
Final quantitative analysesWe used descriptive statistical analyses to summarize participants’ characteristics, their interest in each DB topic, their level of intention, and the potential predicting determinants of their intention. We used SAS (Version 9.4, copyright 2002–2012, SAS Institute Inc.) to conduct these descriptive statistics, and a significance level of 0.05.
Integration of quantitative and qualitative findingsAfter the entire series of evaluation/tailoring rounds, we integrated the quantitative and qualitative findings to generate conclusions on the factors influencing inten-tion, and on the changes in users’ intention as the rounds progressed.
-
Page 7 of 25Lawani et al. BMC Med Inform Decis Mak (2021) 21:59
ResultsStudy populationOne hundred and six HCPs from 20 clinics and home-care services located in 13 cities, agreed to participate in this study. A first cohort of 16 HCPs was assigned to help design the e-learning activity, a second cohort of 36 dif-ferent HCPs helped design the DBs in a three-round eval-uation process, and a third cohort of 54 helped design the DBs by providing comments in a fourth evaluation round. These cohorts are represented in Fig. 3.
In the three cohorts, 78% of participants were women, 60% were physicians, and 57% had fewer than 20 years of practice (Table 1).
Tailoring of the training programThe data that was used to tailor the training program is presented in this section and summarized in the Fig. 4. In the following two subsections, we successively report participants’ perceptions of the e-learning activity and decision boxes, and the strategies used to solve the problems discovered during the evaluation. In the third subsection, we present how participants’ perceptions of SDM supported the tailoring of the training program.
Tailoring of the e‑learning activity based on user experiencesParticipants expressed their appreciation of the content and design of the e-learning activity on numerous occa-sions, and in every section (Table 2). They especially liked the narrated slides, the quiz that informed them of actual patient numeracy levels, and the film depict-ing a simulated patient encounter during which SDM is implemented.
User-centered design initially allowed discover-ing several weaknesses of the e-learning activity. The approach then made it possible to remedy these weak-nesses and then evaluate the impact of the changes made during the next evaluation round. For example, participants mentioned several elements lacking clarity, either in terms of the training component or the con-tent (Table 3). We therefore systematically corrected the most specific issues (e.g. an acronym is undefined, a source is not cited) and they were not mentioned again by the participants in the following rounds. However, for the more general elements lacking clarity (e.g. the availability and content of the DBs, the main principles of SDM), we added an introductory video after Round #2. In Round #3, participants all mentioned appreciat-ing this introductory video.
106 participants were invited to participate
36 Were assigned to help design the DBs (rounds 1-3)
47 Were assigned to help design the DBs (round #4)
16 Were assigned to design he e-learning activity
99 Agreed to participate and completed the study entry questionnaire
26 Evaluated DB#1
18 Evaluated DB#3
23 Evaluated DB#2
22 Evaluated DB#4
19 Evaluated DB#5
21 Completed the evaluation in round #1
2 Answered only half of the questions
17 Completed the evaluation in round #2
18 Completed the evaluation in round #3
1 Answered only half the questions
6 Were lost to follow-up1 Was lost to follow-up16 Completed the study
Fig. 3 Description of participants’ samples in each of the three sub-studies. DB = decision box
-
Page 8 of 25Lawani et al. BMC Med Inform Decis Mak (2021) 21:59
Several other types of issues were corrected system-atically, such as usability issues, or features perceived as irrelevant or less valuable such as the videos with avatars, the introduction, or the training module explaining the evaluation of evidence quality. We either removed these elements, or made them optional for people interested in learning more.
We received several comments on the inapplicability of some of the examples across professions. Therefore, in the final version we diversified the professions displayed in the examples offered.
The activity required participants to extract the ben-efits and harms of patient decision aids from a scien-tific abstract of the Cochrane systematic review on their impact. Participants found the exercise difficult. We chose to try to improve the format of the exercise first, to make it easier. We therefore simplified the text of the abstract as much as possible, added pictograms and smi-ley faces to help identify the benefits and harms, and translated it into French, to further understanding of the information. We received no further negative comments from participants on this exercise thereafter.
Tailoring of the decision boxes based on user experiencesParticipants reported general interest in the topics cov-ered in the decision boxes (DBs), with an overall mean interest of 80% (± SD 11%). The DB perceived as least interesting concerned the power of attorney (67% ± SD 29%), and the one perceived as most interesting covered
Table 1 Characteristics of study participants
a City size < 1,000 = rural; 1,000–29,999 = small; 30,000–99,99 = medium; > 100,000 = large (Statistics Canada, 2011)
Participant characteristics Frequency (Total n = 99)
n %
Age (years) Under 30 16 16.1
30–39 27 27.3
40–49 27 27.3
50–59 23 23.2
60–69 5 5.1
Missing 1 1.0
Gender Female 77 77.8
Male 22 22.2
Profession Physician 59 59.6
Nurse 18 18.1
Social worker 13 13.1
Occupational therapist 6 6.1
Pharmacist 1 1.0
Physiotherapist 1 1. 0
Nutritionist 1 1. 0
City sizes of practice area a Small city 35 35.4
Medium city 7 7.1
Large city 57 57.6
Years of practice < 10 35 35.4
10–19 21 21.2
20–29 27 27.3
30–39 10 10.1
40–49 3 3.0
Unsure 3 3.0
Heathcare professional
Fun, easy, diversified
Valued content
Practical, hands-on
Visual, briefUsability,
clear navigation
Applicable across
professions
Decision box
RelevantImportantNovelInterestingComprehensiveClearTrustedRealisticFocused
Good designPractical, applied
Valued contentSpecific
SynthesizedSimple, clear
Conducive to dialogueSupport next steps
Fig. 4 HCPs’ experiences of the training program
-
Page 9 of 25Lawani et al. BMC Med Inform Decis Mak (2021) 21:59
Tabl
e 2
Stre
ngth
s of
e-le
arni
ng c
ompo
nent
of t
he p
rogr
am
Them
es a
nd s
ubth
emes
Illus
trat
ive
cita
tion
Valu
ed c
onte
ntIn
trod
uctio
n m
odul
e (o
verv
iew
of t
he a
ctiv
ity):
clea
r, re
leva
nt, c
ompr
ehen
sive
, and
trus
ted
“It w
as re
ally
com
preh
ensi
ve, a
nd th
e vi
deo
was
ver
y he
lpfu
l.” (N
urse
#16
, Rou
nd #
3)
Shar
ed D
ecis
ion-
mak
ing
mod
ule:
inte
rest
ing,
rele
vant
, nov
el, i
mpo
rtan
t, cl
ear,
uses
use
ful e
xam
ples
, and
use
of a
n ap
prop
riate
te
rmin
olog
y fo
r the
“Wat
chfu
l Wai
ting”
opt
ion
“Wha
t’s in
tere
stin
g is
the
fact
that
it d
oesn
’t ne
cess
arily
app
ly to
ev
ery
deci
sion
. You
kno
w, I
wou
ld h
ave
thou
ght t
hey
wou
ld s
ay:
‘Sha
red
deci
sion
-mak
ing
mus
t be
the
focu
s of
eve
ry d
ecis
ion
mad
e w
ith y
our p
atie
nts.’
[…].
I thi
nk it
’s go
od it
’s no
t bei
ng
pres
ente
d as
a d
ogm
a w
here
you
don
’t ha
ve th
e rig
ht to
mak
e a
deci
sion
.” (Ph
ysic
ian
#7, R
ound
#2)
Risk
Com
mun
icat
ion
mod
ule:
rele
vant
, app
ropr
iate
, allo
ws
unde
rsta
ndin
g pa
tient
per
cept
ions
of n
umbe
rs, a
nd in
tere
stin
g in
form
a-tio
n on
the
leve
l of e
vide
nce
“You
hav
e to
real
ize
that
peo
ple
don’
t all
have
the
sam
e le
vel o
f ed
ucat
ion,
and
the
way
I un
ders
tand
som
ethi
ng m
ay n
ot b
e th
e sa
me
way
my
patie
nt s
ees
it. It
mak
es y
ou re
aliz
e th
ings
; it
mak
es y
ou q
uest
ion
your
self
and
your
com
pete
ncie
s.” (P
hysi
cian
#3
, Rou
nd #
1)
Patie
nt P
refe
renc
es m
odul
e: fo
unda
tiona
l to
shar
ed d
ecis
ion-
mak
ing,
inte
rest
ing,
use
ful p
rom
pt o
f the
five
key
beh
avio
urs
in th
e SD
M p
roce
ss, a
nd v
alue
d co
nten
t on
the
clim
ate
cond
uciv
e to
dia
logu
e“I
foun
d th
is p
art r
eally
impo
rtan
t bec
ause
, in
my
opin
ion,
it’s
the
crux
of t
he a
ppro
ach.
Wha
t I m
ean
is th
at if
we
don’
t app
roac
h th
e m
atte
r pro
perly
with
the
patie
nt, t
hen
we’
re n
ot re
spec
ting
thei
r val
ues
and
pref
eren
ces,
and
we’
re m
issi
ng s
omet
hing
. So
this
is s
omet
hing
that
’s re
ally
mea
ning
ful t
o m
e.” (S
ocia
l wor
ker
#9, R
ound
2)
Impl
emen
tatio
n m
odul
e: in
tere
stin
g, v
alue
d ex
erci
se o
n th
e ba
rrie
rs/m
yths
in im
plem
entin
g SD
M, a
nd o
vera
ll su
ppor
tive
of im
ple-
men
tatio
n of
sha
red
deci
sion
-mak
ing
“I lik
e it
beca
use
it br
ings
us
back
a b
it to
the
pre-
test
with
any
pr
econ
ceiv
ed n
otio
ns w
e m
ay h
ave
had,
and
it d
ebun
ks th
em.
It pr
ovid
es s
olut
ions
. Hon
estly
, I th
ink
it’s
a re
ally
goo
d se
ctio
n.”
(Phy
sici
an #
13, R
ound
#3)
Inst
ruct
iona
l des
ign
supp
ortiv
e of
lear
ning
Nar
rate
d sl
ides
: app
ropr
iate
lear
ning
mod
ality
, cha
nges
of n
arra
tors
bet
wee
n m
odul
es a
re a
ppre
ciat
ed, i
mag
es a
nd v
isua
l des
ign
appr
ecia
ted
“I th
ink
it’s
good
bec
ause
it’s
inte
ract
ive,
ther
e’s
som
eone
talk
ing
to
you,
exp
lain
ing
it to
you
, with
slid
es. I
t’s e
asy
to u
nder
stan
d. T
hey
don’
t go
into
long
-win
ded
expl
anat
ions
.” (N
urse
#1,
Rou
nd #
1)
Film
dis
play
ing
impl
emen
tatio
n of
sha
red
deci
sion
-mak
ing:
goo
d sy
nthe
sis,
hand
s-on
, a re
alis
tic e
xam
ple
of S
DM
app
licat
ion
and
of d
ecis
ion
box
use,
the
dem
onst
ratio
n of
bes
t pra
ctic
es is
val
ued
(as
oppo
sed
to th
e de
mon
stra
tion
of b
ad p
ract
ices
), th
e in
ter-
activ
e ex
erci
se a
fter
vie
win
g of
the
film
is a
ppre
ciat
ed to
allo
w c
ritic
al a
ppra
isal
of t
he b
ehav
iour
dis
play
ed
“As
a m
atte
r of f
act,
it’s
prob
ably
the
mos
t int
eres
ting
part
bec
ause
in
the
begi
nnin
g it’
s al
l the
ory,
and
then
, it’s
mor
e ab
out p
ract
ice,
so
in m
y vi
ew, i
t’s e
xtre
mel
y re
leva
nt.” (
Phys
icia
n #5
, Rou
nd #
1)
Qui
z on
num
erac
y: a
llow
s be
com
ing
awar
e of
the
chal
leng
es o
f ris
k co
mm
unic
atio
n, re
quire
s re
flect
ion,
fun,
and
app
ropr
iate
lear
n-in
g m
odal
ity w
ith q
uest
ions
-ans
wer
s“S
ure,
it p
uts
us to
the
test
! [la
ughs
]. Bu
t it m
akes
you
real
ize
that
w
e al
l hav
e di
ffere
nt s
kills
ets,
incl
udin
g co
mpr
ehen
sion
… “
(Phy
sici
an #
3, R
ound
#1)
Exer
cise
on
the
barr
iers
/myt
hs re
late
d to
sha
red
deci
sion
-mak
ing:
app
ropr
iate
lear
ning
mod
ality
as
it is
inte
ract
ive,
real
istic
in th
at it
de
scrib
es ty
pica
l myt
hs p
ropa
gate
d by
HC
Ps“It
’s re
ally
inte
ract
ive.
I lik
e th
at. N
ot to
o m
uch
read
ing.
Poi
nt fo
rm. I
ju
st re
tain
info
rmat
ion
bett
er th
at w
ay.” (
Phys
icia
n #1
3, R
ound
#3)
Intr
oduc
tion
vide
o: c
lear
, brie
f, an
d vi
sual
“It p
uts
thin
gs in
to c
onte
xt n
icel
y. I
real
ly li
ke th
e pa
rt w
here
she
sh
ows
wha
t the
web
site
look
s lik
e an
d th
e di
ffere
nt fe
atur
es o
f it.
” (Ph
ysic
ian
#13,
Rou
nd #
3)
-
Page 10 of 25Lawani et al. BMC Med Inform Decis Mak (2021) 21:59
Tabl
e 2
(con
tinu
ed)
Them
es a
nd s
ubth
emes
Illus
trat
ive
cita
tion
Pers
onal
sto
ries
usin
g vi
deos
with
ava
tars
: int
egra
te m
uch
info
rmat
ion,
and
nic
e fo
rmat
“I’ll
rem
embe
r the
litt
le a
vata
r guy
. It’s
a g
ood
mem
ory
aid.
Lik
e po
p-up
rem
inde
rs: ‘O
h ye
ah, I
nee
d to
take
out
suc
h an
d su
ch a
th
ing’
or ‘
That
rem
inds
me
of m
y pa
tient
’… b
ecau
se w
e ha
ve n
o sh
orta
ge o
f doc
umen
ts, a
bout
ant
ibio
tics,
abou
t thi
s or
that
… I
foun
d it
inte
rest
ing,
the
use
of a
vis
ual a
spec
t lik
e th
at…
it’s
a bi
t ‘li
ghte
r’” (P
hysi
cian
#5,
Rou
nd #
1)
Ove
rall
prog
ram
: allo
ws
brin
ging
eve
ryon
e to
the
sam
e le
vel,
appr
ecia
ted
inte
grat
ion
with
the
dire
ctor
y of
dec
isio
n bo
xes,
valu
ed
the
dive
rsity
of l
earn
ing
mod
aliti
es, t
opic
s w
ell-i
llust
rate
d, a
ppro
pria
te le
ngth
, cle
ar n
avig
atio
n th
roug
hout
pro
gram
“The
inte
ract
ive
feat
ures
you
put
in w
ere
not t
oo lo
ng. T
hey
wer
e at
just
the
right
tim
e, a
nd it
mix
ed u
p th
e rh
ythm
a b
it. I
thou
ght
it w
as re
ally
goo
d.” (
Phys
icia
n #1
3, R
ound
#3)
Crit
ical
app
rais
al e
xerc
ise:
app
reci
ated
that
som
e pa
rt o
f the
text
is h
ighl
ight
ed, w
ell-s
ynth
esiz
ed, c
lear
, nov
el in
form
atio
n, a
ppre
ci-
ated
inte
ract
ivity
, cha
lleng
ing
“Of c
ours
e, I’m
real
ly o
n th
e gr
ound
and
ver
y ha
nds-
on. S
o, w
hen
I ha
ve to
dea
l with
rese
arch
find
ings
and
sci
entifi
c la
ngua
ge, y
ou
lose
me
a bi
t. Bu
t wha
t I fo
und
good
was
that
you
put
the
mai
n m
essa
ge in
bol
d. Y
ou d
on’t
have
to re
ad th
e w
hole
sta
tem
ent.
I ju
st re
ad th
e pa
rt in
bol
d an
d I g
ot th
e gi
st o
f it.”
(Soc
ial w
orke
r #1
5, R
ound
#3)
To L
earn
Mor
e se
ctio
n: a
ppre
ciat
ed“It
’s re
ally
goo
d be
caus
e th
ere
are
alw
ays
peop
le w
ho w
ant t
o le
arn
mor
e or
inve
stig
ate
furt
her.
If th
ere’
s so
met
hing
they
did
n’t
unde
rsta
nd, t
hey
can
go d
irect
ly to
the
sour
ces,
whi
ch I
thou
ght
was
goo
d.” (
Soci
al w
orke
r #15
, Rou
nd #
3)
-
Page 11 of 25Lawani et al. BMC Med Inform Decis Mak (2021) 21:59
Tabl
e 3
Wea
knes
ses
of e
-lear
ning
com
pone
nt o
f the
pro
gram
Them
es a
nd S
ubth
emes
Illus
trat
ive
cita
tion
Lack
of c
lari
tyTr
aini
ng m
odal
ity: m
ore
inst
ruct
ions
nee
ded
on w
hat i
s a
deci
sion
box
and
how
to a
cces
s th
em, e
xerc
ise
is u
ncle
ar“A
t firs
t I w
onde
red
wha
t the
dec
isio
n bo
x w
as. B
ut in
real
ity
they
’re to
ols.
It co
uld
be a
goo
d id
ea to
men
tion
wha
t it i
s, be
caus
e, in
the
begi
nnin
g, I
didn
’t kn
ow w
hat i
t was
. I th
ough
t it
was
som
ethi
ng o
ptio
nal,
but r
eally
it’s
not.”
(Phy
sici
an #
5, R
ound
#1
)
Unc
lear
con
tent
: im
port
ant i
nfor
mat
ion
shou
ld b
e hi
ghlig
hted
or b
olde
d, m
ore
prec
isio
ns re
quire
d, m
issi
ng in
form
atio
n[N
ote
from
the
auth
or: T
he p
artic
ipan
t is
disc
ussi
ng th
e cr
itica
l ap
prai
sal e
xerc
ise]
“I d
on’t
know
if p
eopl
e w
ill u
nder
stan
d it.
The
y ar
en’t
nece
ssar
ily s
impl
e te
rms.
It’s
not t
he k
ind
of th
ing
you
see
ever
y da
y. E
spec
ially
not
for c
linic
ians
who
don
’t do
muc
h re
sear
ch. F
or th
ose
who
’re in
jour
nal c
lubs
, it’s
usu
ally
abo
ut
inte
rven
tions
and
rela
tive
risks
… th
at s
ort o
f thi
ng.” (
Phys
icia
n #1
4, R
ound
#3)
Usa
bilit
yBr
owsi
ng is
sues
bet
wee
n pa
ges
or s
ectio
ns: n
ot in
tuiti
ve e
noug
h, n
eed
to c
larif
y ho
w to
mov
e to
the
next
sec
tion
afte
r com
plet
ing
an e
xerc
ise,
acc
ess
to o
ptio
nal c
onte
nt is
unc
lear
“I di
dn’t
know
whe
re to
clic
k. I’m
not
ver
y te
ch-s
avvy
. I w
asn’
t sur
e w
hat y
ou m
eant
by
‘Tar
get C
lient
ele.’
But
now
I un
ders
tand
that
it’
s pa
rt o
f the
intr
oduc
tion.
May
be y
ou c
ould
say
‘Intr
oduc
tion’
in
stea
d of
‘Tar
get C
lient
ele?
’” (Ph
ysic
ian
#11,
Rou
nd #
3)
Issu
es w
ith c
licka
ble
elem
ents
: ref
eren
ces
with
in n
arra
ted
slid
e sh
ows
shou
ld b
e cl
icka
ble,
ele
men
t dis
posi
tione
d, s
ome
slid
es
shou
ld b
e re
orde
red,
con
fusi
ons
betw
een
butt
ons
to m
ove
to th
e ne
xt p
age
“Clic
k he
re to
beg
in re
adin
g […
]. Th
e ar
row
is in
the
wro
ng p
lace
.” (P
hysi
cian
#7,
Rou
nd #
2)
Elem
ents
too
smal
l: im
ages
, nar
rate
d sl
ide
show
“It w
ould
be
good
to m
ake
the
butt
on m
ore
visi
ble.
I’m im
agin
ing
som
eone
who
’s no
t ver
y co
mfo
rtab
le w
ith c
ompu
ters
… y
ou
know
, som
etim
es, d
epen
ding
on
your
scr
een,
it c
an b
e ha
rder
to
see.”
(Phy
sici
an #
5, R
ound
#1)
Irre
leva
nt o
r of l
ess
valu
eCo
nten
t unr
ealis
tic: s
tatin
exa
mpl
e irr
elev
ant f
or p
eopl
e at
low
risk
of c
ardi
ovas
cula
r dis
ease
s, th
e fil
m d
ispl
ays
a si
mul
atio
n, n
ot
real
ity“In
real
ity, I
’m n
ot s
ure
peop
le w
ould
be
incl
ined
to ta
lk a
bout
tak-
ing
stat
ins
with
a p
atie
nt w
ho’s
low
risk
. I u
nder
stan
d; I’
ve s
een
the
stud
ies
show
ing
ther
e m
ay b
e so
me
bene
fits,
but a
t the
sa
me
time,
with
figu
res
like
that
, it’s
not
nec
essa
rily
a ca
se w
here
I w
ould
use
sha
red
deci
sion
-mak
ing.
” (Ph
ysic
ian
#14,
Rou
nd #
3)
Irrel
evan
t for
spe
cific
aud
ienc
es: c
linic
al in
form
atio
n no
t app
licab
le to
all
prof
essi
ons,
info
rmat
ion
mor
e re
leva
nt fo
r old
er H
CPs
, in
form
atio
n le
s us
eful
to e
xper
ienc
ed/le
ss e
xper
ienc
ed p
hysi
cian
s“T
his
part
is a
bit
mor
e ab
out t
he m
edic
al s
ide,
and
that
one
cou
ld
be m
ore
for s
ocia
l wor
kers
or o
ther
s. Su
re, t
here
are
thin
gs th
at
are
a bi
t les
s re
leva
nt fo
r me
that
I ca
n st
ill e
xplo
re fu
rthe
r, bu
t yo
u kn
ow, w
hen
I see
cer
tain
thin
gs, I
’m m
ore
likel
y to
just
refe
r th
em…
“ (S
ocia
l wor
ker #
9, R
ound
#2)
Inap
prop
riat
e le
arni
ng s
trat
egie
sVi
gnet
te w
ith a
vata
rs d
ispl
ayin
g pa
tient
cou
nsel
ing
stra
tegi
es: n
ot re
alis
tic e
noug
h, ro
bot-
like,
chi
ldis
h, in
form
atio
n is
too
theo
reti-
cal,
optio
nal c
onte
nt“W
hen
you’
re a
clin
icia
n an
d yo
u w
atch
that
, you
see
inte
rvie
w
tech
niqu
es w
ith th
e pa
tient
. The
pro
blem
with
the
vide
o is
that
th
e av
atar
s do
n’t h
ave
any
into
natio
n. W
hen
we’
re ta
ught
how
to
com
mun
icat
e w
ith o
ur p
atie
nts,
we’
re ta
ught
wha
t int
onat
ions
to
use
. But
in th
is c
ase,
ther
e is
no
into
natio
n. T
hey
alw
ays
spea
k in
exa
ctly
the
sam
e to
ne o
f voi
ce.” (
Phys
icia
n #5
, Rou
nd #
1)
-
Page 12 of 25Lawani et al. BMC Med Inform Decis Mak (2021) 21:59
Tabl
e 3
(con
tinu
ed)
Them
es a
nd S
ubth
emes
Illus
trat
ive
cita
tion
Crit
ical
app
rais
al e
xerc
ise
(Coc
hran
e re
view
on
the
impa
cts
of p
atie
nt d
ecis
ion
aids
): in
form
atio
n la
yout
sho
uld
be im
prov
ed, b
or-
ing,
read
ing
is le
ss a
ppre
ciat
ed a
s le
arni
ng s
trat
egy
“I ha
te re
adin
g st
udie
s, es
peci
ally
stu
dies
in E
nglis
h! I
get c
onfu
sed
with
the
wor
ds […
]. Ev
en in
Fre
nch,
I do
n’t l
ike
it, b
ut in
Eng
lish
it’s
even
wor
se! I
’m ju
st n
ot in
tere
sted
. Esp
ecia
lly w
ith th
e w
ay
it’s
pres
ente
d he
re. T
here
are
too
man
y fig
ures
and
, in
any
case
, it
real
ly d
oesn
’t in
tere
st m
e.” (P
hysi
cian
#2,
Rou
nd #
1)
Film
: unr
ealis
tic. c
omm
unic
atio
n of
sta
tistic
stil
l con
fusi
ng“T
he fa
ct re
mai
ns th
at it
’s a
sim
ulat
ed in
terv
iew
. You
kno
w, i
t’s n
ot
real
ity. I
t loo
ks n
ice,
but
in re
al li
fe, i
t’s n
ot o
ften
like
that
.” (Ph
ysi-
cian
#6,
Rou
nd #
2)
Too
long
: the
who
le p
rogr
am is
too
long
, som
e se
ctio
ns a
re to
o lo
ng, t
he fi
lm is
too
long
, som
e ex
erci
ses
are
too
long
“I th
ink
it’s
a bi
t lon
g in
pla
ces.
The
cont
ent o
n de
cisi
on a
id to
ols
was
per
haps
a b
it lo
ng. T
here
wer
e se
vera
l slid
es p
rese
ntin
g th
e to
ols,
and
I figu
re p
eopl
e w
ill b
e ab
le to
und
erst
and
it ea
sily
en
ough
. Aft
er a
ll, it
’s ai
med
at H
CPs
.” (Ph
ysic
ian
#7, R
ound
#2)
Dry
: rel
evan
t but
dry
(crit
ical
app
rais
al e
xerc
ise,
and
con
tent
on
evid
ence
app
rais
al)
“Whe
n yo
u ge
t to
this
par
t, it
shift
s fro
m v
isua
l mod
e, w
hich
is
easi
er to
follo
w, t
o a
mor
e pu
rely
info
rmat
iona
l mod
e, w
ith
wor
ds in
Eng
lish
and
all t
hat,
and
lots
of t
ext.
[…]T
he in
form
a-tio
n is
cle
ar e
noug
h, b
ut it
’s pe
rhap
s a
little
har
der t
o fin
d it.
It’s
not a
s si
mpl
e as
with
the
vide
os.” (
Phys
icia
n #5
, Rou
nd #
1)
Tech
nica
l iss
ues:
vid
eo d
oes
not w
ork,
diffi
culti
es m
ovin
g to
the
next
slid
es, f
eedb
ack
does
not
sho
w, l
oosi
ng th
e In
tern
et c
onne
c-tio
n“T
he v
ideo
feed
back
doe
sn’t
show
up,
onl
y te
xt.” (
Soci
al w
orke
r #1
5, R
ound
#3)
Bori
ng: fi
rst p
art i
s bo
ring,
read
ing
is b
orin
g, n
arra
ted
slid
e sh
ows
are
borin
g, la
ck o
f ani
mat
ion
“I’m
a v
ery
visu
al p
erso
n. W
hen
ther
e ar
e lo
ng s
tret
ches
of n
arra
-tio
n on
ly, I
just
sw
itch
off.” (
Phys
icia
n #1
3, R
ound
#3)
Term
inol
ogy
issu
es: a
cron
yms
shou
ld b
e de
fined
, som
e te
rms
that
are
too
stro
ng, u
ncle
ar te
rmin
olog
y, w
hat d
o “o
ptio
n” a
nd
“num
erac
y” m
ean?
, avo
id th
e te
rm “c
hoos
ing
the
optio
n of
doi
ng n
othi
ng”, s
ome
term
s ar
e di
fficu
lt to
und
erst
and
“A li
st o
f ava
ilabl
e he
alth
opt
ions
… I
can’
t pic
ture
exa
ctly
wha
t tha
t m
eans
.” (Ph
ysic
ian
#3, R
ound
#1)
Risk
com
mun
icat
ion
seem
s ch
alle
ngin
g“I
don’
t alw
ays
reca
ll al
l the
pro
babi
lity
perc
enta
ges.
In a
sin
gle
day.
I m
ight
see
fift
een
patie
nts,
all w
ith d
iffer
ent p
robl
ems.
I can
’t al
way
s re
mem
ber o
ff th
e to
p of
my
head
wha
t the
pro
babi
litie
s ar
e.” (P
hysi
cian
#2,
Rou
nd #
1)
Opt
iona
l con
tent
Ove
rall
desc
riptio
n of
the
prog
ram
, pro
gram
des
ign
team
, “Y
ou k
now
, we’
re s
o pu
shed
for t
ime
that
we
real
ly w
ant t
o ge
t st
raig
ht to
the
poin
t whi
ch, i
n th
is c
ase,
is th
e tr
aini
ng. I
get
it
that
it’s
real
ly w
ell d
one
and
all t
hat,
but I
wou
ld s
till t
ry to
con
-de
nse
it a
bit m
ore.”
(Phy
sici
an #
5, R
ound
#1)
GRA
DE
qual
ity o
f evi
denc
e as
sess
men
t: ou
tsid
e th
e pr
ogra
m’s
scop
e, in
form
atio
n th
at is
alre
ady
know
n, c
over
ed in
oth
er p
rogr
ams
“I fo
und
it le
ss re
leva
nt fo
r sha
red
deci
sion
-mak
ing,
whi
ch m
ade
it le
ss in
tere
stin
g.. [
…].
If I w
ante
d to
wat
ch a
web
inar
on
shar
ed
deci
sion
-mak
ing,
I w
ould
n’t w
ant t
o w
atch
that
.” (Ph
ysic
ian
#8,
Roun
d #2
)
Inco
mpl
ete,
mis
sing
info
rmat
ion:
mis
sing
sou
rces
, spe
cific
det
ails
mis
sing
, mis
sing
feed
back
on
som
e ex
erci
ses,
abbr
evia
tions
or
acro
nym
s ar
e no
t defi
ned
“I do
n’t k
now
if it
’s fe
asib
le, b
ut it
wou
ld h
ave
been
fun
to h
ave
a D
B in
han
d to
follo
w a
long
at t
he s
ame
time
as th
e tr
aini
ng s
es-
sion
.” (Ph
ysic
ian
#8, R
ound
#2)
Redu
ndan
t con
tent
: cou
ld b
e sh
orte
ned,
slig
htly
too
repe
titiv
e“A
t one
poi
nt th
ey g
ive
a lo
t of e
xam
ples
. May
be th
ere
are
som
e th
ings
that
cou
ld b
e re
mov
ed.” (
Phys
icia
n #7
, Rou
nd #
2)
-
Page 13 of 25Lawani et al. BMC Med Inform Decis Mak (2021) 21:59
Tabl
e 3
(con
tinu
ed)
Them
es a
nd S
ubth
emes
Illus
trat
ive
cita
tion
Info
rmat
iona
l or t
ypog
raph
ic e
rror
“’Are
you
sure
abo
ut w
hich
cho
ice
is be
st fo
r you
?’ Kn
owin
g so
me
of
my
elde
rly p
atie
nts,
I thi
nk th
is q
uest
ion
coul
d de
stab
ilize
them
or
mak
e th
em fe
el le
ss c
onfid
ent i
n th
eir d
ecis
ion.
I w
ould
n’t
have
wor
ded
it th
at w
ay. I
wou
ld h
ave
valid
ated
that
that
is
the
deci
sion
they
wan
t to
mak
e, b
ut I
find
it a
bit s
tron
g to
use
w
ords
like
‘cer
tain
’ or ‘
best
.’ Per
sona
lly, I
wou
ld h
ave
take
n a
gen-
tler a
ppro
ach
in th
at s
ense
.” (Ph
ysic
ian
#11,
Rou
nd #
3)
Patie
nt P
refe
renc
es s
ectio
n to
o sh
ort:
mor
e co
mm
unic
atio
n tip
s sh
ould
be
offer
ed“N
ot e
very
one
in th
e m
edic
al fi
eld
has
good
pat
ient
com
mun
ica-
tion
skill
s, so
I th
ink
you
coul
d ex
pand
this
sec
tion
a bi
t.” (S
ocia
l w
orke
r #15
, Rou
nd #
3)
-
Page 14 of 25Lawani et al. BMC Med Inform Decis Mak (2021) 21:59
non-pharmacological treatment to manage agitation, aggression, or psychotic symptoms (88% ± SD 11%). Participants mentioned that they liked the visual design of the DBs, because it facilitates their use in practice, especially the tabular format presenting risks and ben-efits/harms (Table 4). Several participants also expressed their satisfaction with the informational content of the DBs, especially the information about the options. Some mentioned that the DBs helped make them aware of the options, or found that the options were relevant. One person appreciated the fact that the DB was available to support older adults in realizing, on their own, that their driving might be dangerous. They thought this might help maintain their relationship with patients.
The limitations reported by study participants on the decision boxes allowed enhancing them to improve user experiences. Some of the modifications made were quite extensive. For example, several participants reported lower beliefs about their capabilities to use the DB to explain the pros and cons of health options to patients, because of the lack of accessibility to the services described in the DB (Table 5). We judged this a critical flaw, as if it remained unresolved, HCPs would not use the DB and adopt SDM. Therefore, to resolve this aspect, we added a section with a list of contact information for professional and community services available to imple-ment the options offered in the DB, such as massage ther-apy, music therapy, and physical activity. Another critical flaw was that several participants perceived that they might not have enough time to use the DB. We there-fore added some content describing the situations where SDM should be prioritized. We also added information about the possibility to delay decision-making to a subse-quent consultation, and thus limit the time needed to go through the complete SDM process.
In some cases, we chose not to attempt to resolve the issues raised. For example, some participants felt the statistics were too hard for patients and their family caregivers to understand. Since probabilities in numeri-cal formats are required to understand risks, we chose not to change the risks presentation in the DBs. Instead, we modified the e-learning activity by adding a module describing best practices to communicate risks to people with lower numeracy skills.
Similarly to the e-learning activity, many of the par-ticipants’ comments were specific and easily addressed. For example, participants mentioned that the informa-tion was too dense, the terminology challenging, or some usability issues. We therefore adjusted the content of the DBs to reduce their length and complexity, and thereby limit the time needed to understand them, without com-promising their meaning. We kept the use of jargon to a minimum, and added a glossary where we were unable to
find simplified terminology. We also synthesized the sci-entific information on the pros and cons of the options in summary tables where possible. When there were more than two options for a given clinical situation, we added a table setting out the potential options for the decision at hand on the first page, including the estimates of the probabilities of impacts for each option, as well as the corresponding page detailing the impacts.
A few physicians reported that the topic covered by DB #5 (Power of Attorney) was irrelevant to their pro-fessional role. In an earlier Delphi study [41], we identi-fied a need in primary care practices for decision support regarding this topic. We consequently attempted to improve DB #5 by making some of the legal aspects of the power of attorney easier to understand so that HCPs, especially physicians, can take ownership of the content and become more comfortable discussing it with their patients.
Tailoring content based on participants perceptions of SDMWe explored participants’ beliefs about consequences, or usefulness of SDM process after completing the e-learn-ing activity. Participants’ descriptions of the benefits and inconveniences of SDM (Table 6), were very useful to appraise participants knowledge after training, and tailor the content to improve knowledge. Several of these com-ments pointed to known barriers to adopting SDM. We added specific content to the existing modules to address each of these concerns. Overall, these comments led us to describe several strategies for adopting SDM in diverse clinical situations, for example, where time is limited, when there is an emergency and a decision cannot be delayed, when the patients’ preferences go against those of the professional, when risk is low, or when there are several decisions to be made. We also clarified the role of HCPs in situations where the patient’s choice appears contrary to public health recommendations.
Some of the participants’ comments after reading a decision box also point to a lack of knowledge about SDM, for example that DBs are of little use when stating their opinion or making recommendations (Table 6). To improve understanding of the SDM approach, we added an introductory paragraph to all DBs, entitled “What’s this document for?” which described the gen-eral SDM approach. We also added patient stories to most of the DBs, usually demonstrating an encounter between a patient and a clinician, to demonstrate the value of seeking patient priorities. The stories were created from testimonies gathered online and were validated by the expert panel. These strategies proved effective, as we received no more comments suggest-ing that SDM might not be well understood after these changes.
-
Page 15 of 25Lawani et al. BMC Med Inform Decis Mak (2021) 21:59
Changes in the level of intention as the rounds progressedE‑learning activityVisual examination of the quantitative descriptive results suggests that there was no change in participants’ intention to adopt SDM after their participation in the e-learning activity. Intention was relatively high and
stable across the three rounds, with a mean level of 6.8, on a scale ranging from 1 (low intention) to 7 (high inten-tion) (Additional file 1). Mean levels of potential factors influencing intention were also relatively high, ranging from 5.6 to 7.0, on a scale ranging from 1 to 7.
Table 4 HCPs’ perceptions of the strengths of the Decision Boxes
*DB #1 = Choosing a support option to decrease caregiver burden; DB #2 = Choosing a non-pharmacological treatment to manage agitation, aggression, or psychotic symptoms; DB #3 = Deciding whether or not to stop driving following diagnosis; DB #4 = Choosing an option to improve quality of life; DB #5 = Deciding whether or not to prepare a power of attorney.
Themes and Subthemes Illustrative citation*
Visual designGood visual design “Very nice tool with an excellent visual pres-
entation.” (Physician #22, Round #1, DB #3) ; “Well-designed tool. Easy to use.” (Physician #31, Round #2, DB #3)
Tabular format: tables are very clear and visual “Very clear, visually appealing tables. The Confi-dence in these results pictograms could be a bit more visible (black dots rather than a cross? Bigger circles?). The presentation page (page 1) is a bit dry to read, but essential for explaining the goal.” Physician #12, Round #1, DB #1)
Balance between benefits and harms is useful“Nice layout of benefits vs harms.” (Physician #74, Round #4, DB #2)
Informational contentValue of the information about the options “The box presents some very interesting options.”
(Physician #32, Round #3, DB #1)
Raises awareness about certain options “The role of case manager no longer exists in many CLSCs. Highly relevant and appropriate for our family caregivers who are unsure or unaware of which resources to turn to. I will definitely use it.” (Social worker #12b, Round #4, DB #1) ; “Great idea for improving our client service.” (Social worker #48, Round #4, DB #4)
Information allows HCPs to keep up-to-date and to empower patients “Nice tool that allows us to be more professional and access up-to-date knowledge. Also enables us to show that we respect the client’s values. Helps empower them.” (Social worker #48, Round #4, DB #1)
A tool to help older adults realize themselves their own risks “Very interesting toolbox for guiding and helping patients realize on their own that their driving may not be safe, instead of having the impres-sion, as a doctor, that you are taking away their license and their autonomy. It helps maintain the quality of the therapeutic relationship.” (Physician #36b, Round #4, DB #3)
Useful to remind me of something I already know “The information in the box will be helpful for refreshing my memory on the various power of attorney options.” (Physician #31, Round #1, DB #5)
Implementation of SDMUseful to adopt a shared decision-making approach in their practice “I have never (or rarely) discussed stopping driving
with a patient based on the risks and benefits to the patient. Rather, I tried to test the patient’s skills through tests without necessarily dwelling on his understanding of the risks of driving. Participation in this study will make me more likely to approach the risk-benefit aspect with the patient in the future.” (Physician #12, Round #3, DB #3)
-
Page 16 of 25Lawani et al. BMC Med Inform Decis Mak (2021) 21:59
Tabl
e 5
HCP
s’ pe
rcep
tion
s of
the
wea
knes
ses
of th
e D
ecis
ion
Boxe
s by
theo
reti
cal d
omai
n.
Theo
retic
al D
omai
n, W
eakn
ess
Illus
trat
ive
cita
tion*
Inte
ntio
nI h
ave
no in
tent
ion
of c
hang
ing
my
day-
to-d
ay a
ppro
ach.
“The
info
rmat
ion
in th
e de
cisi
on b
ox w
ill b
e us
eful
to m
e to
re
fresh
my
mem
ory
on th
e di
ffere
nt p
rote
ctio
n re
gim
es b
ut I
do
not b
elie
ve th
at I
will
cha
nge
my
way
of d
oing
thin
gs fr
om d
ay
to d
ay.” (
Phys
icia
n #3
1, R
ound
#1,
DB
#5)
Belie
f abo
ut c
onse
quen
ces
of u
sing
the
DB;
Use
fuln
ess
of th
e D
BTh
e D
B is
not
wel
l ada
pted
to e
very
pat
ient
, nor
is it
ada
pted
to e
very
situ
atio
n“T
he fa
cts
wer
e ge
nera
lly k
now
n to
me,
eve
n th
ough
I di
d no
t ha
ve p
reci
se s
tatis
tics.
In a
dditi
on, e
ach
case
is d
iffer
ent,
and
I do
not b
elie
ve th
at a
pply
ing
this
box
will
allo
w m
e to
bet
ter t
arge
t th
e ne
cess
ary
inte
rven
tions
.” (Ph
ysic
ian
#19,
Rou
nd #
2, D
B #3
)
DBs
do
not a
dd a
ny v
alue
to c
linic
al p
ract
ice
“We
unde
rsta
nd th
at q
ualit
y of
life
is th
e m
ain
obje
ctiv
e, b
ut th
e to
olbo
x, a
lthou
gh in
tere
stin
g, is
hea
vy a
nd b
rings
not
hing
mor
e to
the
clin
ical
leve
l.” (P
hysi
cian
#31
, V3,
DB
#5)
DBs
are
not
hel
pful
to a
nnou
nce
my
own
opin
ion/
reco
mm
enda
tion
"Wha
t is
mos
t diffi
cult
is to
ann
ounc
e ou
r opi
nion
s w
hen
they
ar
e ve
ry d
iver
gent
from
thos
e of
the
patie
nt o
r the
fam
ily, a
nd
I do
not b
elie
ve th
at th
e de
cisi
on b
ox h
elps
us."
(Phy
sici
an #
19,
Roun
d #2
, DB
#3)
Stat
istic
s ar
e us
eles
s"T
he D
B, w
hile
inte
rest
ing,
is c
umbe
rsom
e an
d br
ings
not
hing
m
ore
to c
linic
al p
ract
ice."
(Doc
tor #
31, R
ound
#3,
DB
#4)
The
sam
e in
form
atio
n ca
n be
offe
red
info
rmal
ly w
ithou
t pre
sent
ing
the
DB
“The
re a
re m
any
tool
s fo
r diff
eren
t asp
ects
of m
edic
ine,
and
the
use
of c
oncr
ete
tool
s is
diffi
cult,
but
the
idea
con
veye
d by
thes
e to
ols
can
be tr
ansm
itted
ver
bally
to p
atie
nts
info
rmal
ly th
roug
h ot
her i
nfor
mat
ion.
” (Ph
ysic
ian
#9, R
ound
#1,
DB
#5)
Info
rmat
ion
is h
arm
ful t
o in
terp
rofe
ssio
nal c
olla
bora
tion
"I w
as s
urpr
ised
by
the
stat
istic
s th
at 1
% to
3%
of t
hera
pist
s co
m-
mit
sexu
al a
cts
that
cou
ld h
inde
r the
pro
cess
of c
are.
Con
side
r-in
g th
e lo
w p
ropo
rtio
n of
men
, the
per
cent
age
of th
erap
ists
w
ith d
evia
nt b
ehav
iour
s se
ems
to m
e to
be
high
. I v
ery
muc
h do
ubt t
his
stat
istic
. As
I am
a m
an, I
find
that
this
sta
tistic
mig
ht
affec
t ref
erra
l and
the
confi
denc
e of
the
doct
or to
war
ds th
e th
erap
ists
." (S
ocia
l wor
ker #
69, R
ound
#4,
DB
#1)
Alre
ady
awar
e of
this
info
rmat
ion
“I w
as a
war
e of
the
fact
s, bu
t not
of t
he e
xact
sta
tistic
s.“ (P
hysi
cian
#1
9, R
ound
#2,
DB
#3)
Belie
f abo
ut c
apab
ilitie
s: I
mig
ht n
ot b
e ab
le to
use
the
DB
to p
rese
nt th
e pr
os a
nd c
ons
of h
ealth
opt
ions
to p
atie
nts
beca
use…
… it
is c
halle
ngin
g to
acc
ess
the
serv
ices
offe
ring
the
heal
th o
ptio
ns d
escr
ibed
in th
e D
B “A
cces
sibi
lity
rem
ains
a p
robl
em in
man
y re
gion
s of
Que
bec.
The
se
data
hav
e be
en c
olle
cted
to fa
cilit
ate
disc
ussi
on, b
ut th
e bu
lk
of th
e w
ork
that
will
ulti
mat
ely
incr
ease
the
patie
nt’s
qual
ity o
f lif
e is
not
the
DB,
but
the
long
-ter
m a
cces
sibi
lity
to th
is ty
pe o
f se
rvic
e.” (P
hysi
cian
#32
, Rou
nd #
2, D
B #4
).
…it
is c
halle
ngin
g to
pre
sent
sta
tistic
s"W
hen
patie
nts
and
thei
r fam
ily c
areg
iver
s ar
e in
our
offi
ce, t
alki
ng
abou
t sta
tistic
s is
unt
hink
able
. It i
s al
read
y ve
ry d
ifficu
lt to
onl
y ta
lk a
bout
the
real
ity o
f eve
ryda
y lif
e." (N
urse
#14
, Rou
nd #
2, D
B #1
)
-
Page 17 of 25Lawani et al. BMC Med Inform Decis Mak (2021) 21:59
*DB
#1=
Cho
osin
g a
supp
ort o
ptio
n to
dec
reas
e ca
regi
ver b
urde
n; D
B #2
= C
hoos
ing
a no
n-ph
arm
acol
ogic
al tr
eatm
ent t
o m
anag
e ag
itatio
n, a
ggre
ssio
n, o
r psy
chot
ic s
ympt
oms;
DB
#3=
Dec
idin
g w
heth
er o
r not
to s
top
driv
ing
follo
win
g di
agno
sis;
DB
#4=
Cho
osin
g an
opt
ion
to im
prov
e qu
ality
of l
ife; D
B #5
= D
ecid
ing
whe
ther
or n
ot to
pre
pare
a p
ower
of a
ttor
ney.
Tabl
e 5
(con
tinu
ed)
Theo
retic
al D
omai
n, W
eakn
ess
Illus
trat
ive
cita
tion*
…I d
on’t
have
tim
e‘’O
ffice
day
s ar
e be
com
ing
heav
ier a
nd h
eavi
er a
nd it
is n
ow p
os-
sibl
e to
use
the
serv
ices
of o
ur p
aram
edic
al s
taff
to c
onve
y m
ore
com
plet
e in
form
atio
n so
we
can
focu
s m
ore
on c
linic
al ta
sks.”
(P
hysi
cian
#31
, Rou
nd #
1, D
B #5
).
...I d
on’t
know
how
to a
cces
s th
e D
Bs.
“Acc
ess
to th
e D
B sh
ould
be
faci
litat
ed.” (
Nur
se #
15, R
ound
#2,
DB
#4)
Ease
of U
seTh
e D
B us
es u
nfam
iliar
jarg
on o
r ter
min
olog
y"T
he te
rms
used
are
lega
l ter
ms
and
it’s
a to
ugh
jarg
on fo
r me."
(P
hysi
cian
#20
, Rou
nd #
2, D
B #5
)
The
info
rmat
ion
is to
o de
nse
and
shou
ld b
e sy
nthe
size
d"T
he p
rese
ntat
ion
is to
o de
nse
and
too
com
plex
." (P
hysi
cian
#2,
Ro
und
#1, D
B #1
)
The
info
rmat
ion
is d
ifficu
lt to
und
erst
and
“I fo
und
the
late
st d
ata
on u
sing
vs
stop
ping
med
icat
ion
conf
us-
ing.
In th
e H
arm
s se
ctio
n, it
is h
ard
to in
terp
ret w
hat y
ou’re
tr
ying
to s
how
.” (Ph
ysic
ian
#32,
Rou
nd #
1, D
B #2
)
Usa
bilit
y of
the
DBs
cou
ld b
e im
prov
ed"T
he P
icto
gram
s fo
r the
Con
fiden
ce in
the
resu
lts s
ectio
n co
uld
be a
litt
le m
ore
visi
ble.
Cou
ld b
lack
dot
s be
use
d ra
ther
than
cro
sses
? Co
uld
the
circ
les
be e
nlar
ged?
" (Ph
ysic
ian
#12,
Rou
nd #
1, D
B #1
)
Soci
al in
fluen
ce: M
y co
lleag
ues
mig
ht n
ot u
se th
e D
B to
pre
sent
the
pros
and