clinical decision support systems for opioid prescribing ... · clinical decision support systems...
TRANSCRIPT
Clinical Decision Support Systems for Opioid Prescribing
for Chronic Non-cancer Pain in Primary Care Settings
by
Sheryl Maria Spithoff
A thesis submitted in conformity with the requirements
for the degree of Masters of Science
Graduate Department of the Institute of Medical Science
University of Toronto
© Copyright by Sheryl Spithoff 2019
ii
Clinical Decision Support Systems for Opioid Prescribing for
Chronic Non-cancer Pain in Primary Care Settings
Sheryl Spithoff
Masters of Science
Institute of Medical Science
University of Toronto
2019
Abstract
This thesis sought to provide an understanding of the potential benefits and possible limitations of clinical
decision support systems (CDSS) for opioid prescribing for chronic non-cancer pain (CNCP) in primary
care settings. Findings from the scoping review and the exploratory qualitative study indicated few potential
benefits and significant limitations. There were few studies and, although some reported that the CDSS led
to more appropriate prescribing, they used lower quality designs. None of the studies examined patient
outcomes or assessed for unintended consequences. Many had conflicts of interest. Developers did not
appear to be using evidence when designing a CDSS. Investigators did not follow guidance for the
evaluation of complex interventions. The research also demonstrated barriers to implementation for a
specific CDSS we evaluated, including increased work, an interrupted workflow and poor buy-in.
Therefore, more research is needed before widespread implementation of CDSSs for opioid prescribing for
CNCP in primary care settings.
iii
Acknowledgments
I would like to express my gratitude to my supervisor, Frank Sullivan, for his support and guidance
during this project. I would like to thank my thesis committee members: Mary Ann O’Brien for agreeing
to join my committee half-way through—without her expertise I could not have done this project; Tara
Gomes for reading drafts of my thesis and giving really insightful feedback just a few months after going
on maternity leave; and Meldon Kahan for many long conversations that helped me make sense of the
opioid prescribing landscape. I would also like to thank Stephanie Mathieson who—from the other side of
the world—agreed to participate in the scoping review and Leslie Carlin, a kindred spirit, whose genuine
excitement about qualitative research inspired me. A thank you also to Susan Hum for reading a draft of
the thesis and providing feedback. And most of all, a thank you to those who make my life happy, my
husband Pete and my daughter Anna.
Contributions
Mary Ann O’Brien, Stephanie Mathieson, Abhimanyu Sud, Qi Guan, Frank Sullivan, and Susan Hum
participated in the scoping review. I designed the study and did the screening, data extraction and
analysis. Mary Ann, Frank and Stephanie provided feedback on the design of the study. Abhimanyu,
Mary Ann, Qi, Stephanie and Susan assisted with screening, data extraction and contacting authors.
Kaitlin Fuller, University of Toronto librarian, answered questions about scoping reviews. The scoping
study did not receive specific financial support. Frank Sullivan, Mary Ann O’Brien, Leslie Carlin, Ivanka
Pribramska, and Saddaf Syed participated in the exploratory qualitative study. I designed the study and
conducted the analysis. Leslie conducted the interviews and participated in the analysis. Ivanka and
Saddaf assisted in coordination of the study. Frank and Mary Ann provided feedback on the design and
the analysis. The Institute for Safe Medication Practices (ISMP) Canada provided funding for the
exploratory qualitative study. University of Toronto Practice-based Research Network (UTOPIAN)
provided support to coordinate the exploratory qualitative study. Sheryl Spithoff was supported by a
Graduate Research Award from the Department of Family and Community Medicine, University of
Toronto.
iv
Table of Contents
Acknowledgments .......................................................................................................................... iii Table of Contents ........................................................................................................................... iv List of Figures and Tables ............................................................................................................. vii List of Appendices ....................................................................................................................... viii List of Abbreviations ..................................................................................................................... ix
Introduction ................................................................................................................................ 1 1.1 Research Problem ............................................................................................................... 1 1.2 Rationale ............................................................................................................................. 1 1.3 Aim and Specific Research Questions ................................................................................ 1 1.4 Overview ............................................................................................................................. 2
Chapter 2 Literature Review ............................................................................................... 2 Chapter 3 Scoping Review Study ....................................................................................... 2 Chapter 4 Exploratory Qualitative study ............................................................................ 2 Chapter 5 General Discussion ............................................................................................. 3
Literature Review ....................................................................................................................... 4 2.1 Chronic Non-Cancer Pain (CNCP) ..................................................................................... 4
2.1.1 Introduction ............................................................................................................. 4 2.1.2 Definition and Epidemiology .................................................................................. 4 2.1.3 Pathophysiology ...................................................................................................... 5 2.1.4 Burden of CNCP ..................................................................................................... 6 2.1.5 Pharmacological Treatments ................................................................................... 6 2.1.6 Non-pharmacological Treatments ........................................................................... 7 2.1.7 CNCP in Practice Settings ...................................................................................... 8 2.1.8 Summary ................................................................................................................. 9
2.2 Opioid Prescribing for CNCP ............................................................................................. 9 2.2.1 Introduction ............................................................................................................. 9 2.2.2 Physiology ............................................................................................................... 9 2.2.3 Opioids and CNCP .................................................................................................. 9 2.2.4 Opioid Prescribing ................................................................................................ 10 2.2.5 International Comparisons of Opioid Prescribing ................................................ 11 2.2.6 Roots of the Increase in Prescribing ..................................................................... 11 2.2.7 Harms from Prescribed Opioids ............................................................................ 12 2.2.8 The Opioid Crisis .................................................................................................. 13 2.2.9 Guidelines for Opioid Prescribing for CNCP ....................................................... 13 2.2.10 Adherence to Guidelines for Opioid Prescribing for CNCP ................................. 14 2.2.11 Summary ............................................................................................................... 14
2.3 Knowledge Translation ..................................................................................................... 14 2.3.1 Introduction ........................................................................................................... 14 2.3.2 History of KT ........................................................................................................ 15 2.3.3 Definition and Overview of KT ............................................................................ 15 2.3.4 CIHR KTA Cycle ................................................................................................. 16 2.3.5 KT Interventions ................................................................................................... 18 2.3.6 Guidelines and Indicators for Safer Opioid Prescribing for CNCP ...................... 18 2.3.7 Barriers to More Appropriate Opioid Prescribing for CNCP ............................... 19 2.3.8 KT Interventions for Opioid Prescribing for CNCP ............................................. 19
v
2.3.9 Summary ............................................................................................................... 20 2.4 Scoping Reviews ............................................................................................................... 20
2.4.1 Introduction ........................................................................................................... 20 2.4.2 History of Reviews ............................................................................................... 20 2.4.3 Systematic Reviews and Meta-analyses ............................................................... 21 2.4.4 Scoping Reviews ................................................................................................... 22 2.4.5 Summary ............................................................................................................... 23
2.5 Clinical Decision Support Systems (CDSSs) ................................................................... 23 2.5.1 Introduction ........................................................................................................... 23 2.5.2 Definition, Taxonomy and History ....................................................................... 23 2.5.3 Evidence of Effectiveness ..................................................................................... 24 2.5.4 Implementation Issues .......................................................................................... 25 2.5.5 CDSSs for Opioid Prescribing for CNCP ............................................................. 26 2.5.6 Prescription Drug Monitoring Programs (PDMPs) .............................................. 26 2.5.7 CDSSs in Primary Care Settings ........................................................................... 27 2.5.8 Summary ............................................................................................................... 27
2.6 Evaluation of Complex Interventions ............................................................................... 28 2.6.1 Introduction ........................................................................................................... 28 2.6.2 Description of Complex Interventions .................................................................. 28 2.6.3 Evaluation of Complex Interventions ................................................................... 29 2.6.4 Current Guidance for the Evaluation of Complex Interventions .......................... 29 2.6.5 Process Evaluations .............................................................................................. 30 2.6.6 Process Evaluation Terminology .......................................................................... 31 2.6.7 Conducting a Process Evaluation .......................................................................... 32 2.6.8 Role of Theory in Process Evaluations ................................................................. 34 2.6.9 Use of Theory to Guide the Implementation Process of CDSSs .......................... 35 2.6.10 Summary ............................................................................................................... 36
2.7 The Normalization Process Theory (NPT) ....................................................................... 36 2.7.1 Introduction ........................................................................................................... 36 2.7.2 Theory ................................................................................................................... 36 2.7.3 Development of the NPT ...................................................................................... 37 2.7.4 Goals of the NPT .................................................................................................. 37 2.7.5 Description of the NPT ......................................................................................... 38 2.7.6 NPT in Practice ..................................................................................................... 39 2.7.7 Use of the NPT in CDSS Implementation Studies ............................................... 39 2.7.8 Summary ............................................................................................................... 40
Clinical Decision Support Systems for Opioid Prescribing for Chronic Non-cancer Pain in
Primary Care Settings: a Scoping Review ............................................................................... 41 3.1 Focused Introduction ........................................................................................................ 41 3.2 Research Questions ........................................................................................................... 42 3.3 Methods ............................................................................................................................. 42
3.3.1 Overview ............................................................................................................... 42 3.3.2 Eligibility Criteria ................................................................................................. 43 3.3.3 Search Strategy ..................................................................................................... 44 3.3.4 Study Selection Process ........................................................................................ 44 3.3.5 Data Extraction and Outputs ................................................................................. 45 3.3.6 Data Synthesis ....................................................................................................... 45
3.4 Results ............................................................................................................................... 46
vi
3.4.1 Search Results ....................................................................................................... 46 3.4.2 Description of Study Settings, Population, Intervention and Description of the
CDSS ..................................................................................................................... 48 3.4.3 Description of Study Aims, Methodologies, Methods, Findings and Adherence
to Guidance ........................................................................................................... 48 3.4.4 Description of Funding Sources and Conflicts of Interest .................................... 49 3.4.5 Study Tables .......................................................................................................... 50
3.5 Discussion ......................................................................................................................... 55 3.6 Conclusion ........................................................................................................................ 55
A Description of the Normalization Process of a Clinical Decisions Support System for
Safer Opioid Prescribing for Chronic Non-cancer Pain into Primary Care Settings: an
Exploratory Qualitative Study .................................................................................................. 56 4.1 Focused Introduction ........................................................................................................ 56 4.2 Aim and Objective ............................................................................................................ 57 4.3 Methods ............................................................................................................................. 57
4.3.1 Overview ............................................................................................................... 57 4.3.2 Population of Interest and Sampling Methods ...................................................... 58 4.3.3 Data Collection and Preparation ........................................................................... 59 4.3.4 Description of the PCI .......................................................................................... 60 4.3.5 Implementation of the PCI in Practice .................................................................. 61 4.3.6 Ethics, Privacy and Conflicts of Interest ............................................................... 61 4.3.7 Analysis ................................................................................................................. 62 4.3.8 Rigour ................................................................................................................... 64
4.4 Results ............................................................................................................................... 64 4.5 Discussion ......................................................................................................................... 69 4.6 Conclusions ....................................................................................................................... 70
General Discussion ................................................................................................................... 71 5.1 Discussion ......................................................................................................................... 71 5.2 Implications ....................................................................................................................... 78 5.3 Strengths ........................................................................................................................... 79 5.4 Limitations ........................................................................................................................ 80 5.5 Conclusion ........................................................................................................................ 80 5.6 Future Directions .............................................................................................................. 81
References ..................................................................................................................................... 84 Appendix 3.1 PRISMA-ScR Checklist ....................................................................................... 116 Appendix 3.2 Protocol Scoping Review ..................................................................................... 117 Appendix 3.3 Medline Search Strategy ...................................................................................... 123 Appendix 3.4 Grey Literature Search ......................................................................................... 127 Appendix 4.1 Physician Interview Guide ................................................................................... 135 Appendix 4.3 Mapping Categories to NPT Constructs .............................................................. 140 Copyright Acknowledgements .................................................................................................... 142
vii
List of Figures and Tables
Figures
Figure 2.1: Knowledge to Action Cycle Page 27
Figure 3.1: PRISMA Flow Diagram Page 47
Tables
Table 2.1: Common types of CDSS Page 24
Table 2.2: What Makes an Intervention Complex Page 28
Table 2.3: Key Recommendations for Process Evaluations Page 32
Table 3.1: Study Overview: Setting, Population, Intervention and Descriptions of the CDSS Page 50
Table 3.2: Study Overview: Aim, Design and Summary of Relevant Findings Page 51
Table 3.3: Study Summary Characteristics Page 53
Table 3.4: Inclusion of Evidence-based Components Page 54
Table 3.5: Adherence to Guidance for Development and Evaluation of Complex Interventions Page 54
Table 4.1: The NPT Coding Framework Page 63
viii
List of Appendices
Appendix 3.1 PRISMA-ScR Checklist
Appendix 3.2 Protocol: Scoping Review
Appendix 3.3 Medline Search Strategy
Appendix 3.4 Grey Literature Search
Appendix 3.5 Data Extraction Form
Appendix 4.1 Physician Interview Guide
Appendix 4.2 Patient Check-in (PCI)
Appendix 4.3 Mapping Categories to NPT Constructs
ix
List of Abbreviations
APP Advanced Practice Provider
BPI Brief Pain Index
CADTH Canadian Agency for Drug and Technology in Health
CDC Centre for Disease Control
CDSS Clinical Decisions Support System
CIHR Canadian Institutes of Health Research
CHAT Case-finding and Help Assessment Tool
COMM Current Opioid Misuse Measure
CNCP Chronic Non-cancer Pain
DST Decision Support Tool
E-health Electronic Health
EQUATOR Enhancing the Quality and Transparency of Health Research
EMR Electronic Medical Record
HCP Health Care Provider
HQO Health Quality Ontario
ISMP Institute for Safe Medication Practices
KTA Knowledge to Action Cycle
MED Morphine Equivalent Dose
x
N/A Not Applicable
NP Nurse Practitioner
NPT Normalization Process Theory
OUD Opioid use disorder
PCI Patient Check-in
PHQ9 Patient Health Questionnaire
PCP Primary Care Provider
PDMP Prescription Drug Monitoring Program
PMP Prescription Monitoring Program
PRISMA Preferred Reporting Items for Systematic Reviews
PRISMA-CI Preferred Reporting Items for Systematic Reviews- Complex Interventions
PRISMA-ScR Preferred Reporting Items for Systematic Reviews- Scoping Reviews
PSS Practice Solutions
REB Research Ethics Board
RCT Randomized Controlled Trial
SD Standard Deviation
UTOPIAN University of Toronto Practice-based Research Network
1
Introduction
1.1 Research Problem
Prescriptions for opioids tripled in Canada (1) and the United States (U.S.) (2) over about 15 years.
Many of these prescriptions were issued by primary care providers (3–5). Prescribed opioids place people
at risk for harms including overdoses and deaths (6–9). Prescription opioids also spawned the current
opioid crisis (10,11). To address the individual and population level harms, the American Centre for
Disease Control (CDC) and the National Pain Centre in Canada released guidelines for prescribing
opioids for chronic non-cancer pain (CNCP) (1,12). Adherence, however, appears to be poor (13–17).
Barriers to change include inadequate training, poor access to resources, lack of supports and poor patient
buy-in (18–26).
1.2 Rationale
Clinical decisions support systems (CDSSs) may assist providers in prescribing opioids more
appropriately. CDSSs have a modest impact on process outcomes, such as safer prescribing (27–35). The
impact on patient outcomes is less clear with mixed outcomes and low quality of evidence (27–30).
Certain design components appear to increase likelihood of positive outcomes (36–39). CDSSs, however,
often fail to become integrated into healthcare processes (40–46). Additionally, they can be difficult to
develop and evaluate because they are complex interventions (47).
1.3 Aim and Specific Research Questions
The aim of my thesis was to gain an understanding of the potential benefits and possible limitations of the
use of CDSSs for opioid prescribing for CNCP in primary care settings.
My specific objectives were:
1) To report on the range and extent of current research on CDSSs for opioid prescribing for CNCP
in primary care clinical settings as well as the extent to which researchers are following best
evidence for CDSS components and current guidance for complex interventions.
2
2) To describe, in an exploratory study, the normalization process of a specific CDSS — the Patient
Check-in (PCI) — for more appropriate opioid prescribing for CNCP into a primary care setting.
For the first objective, we conducted a scoping review and for the second objective, we conducted an
exploratory qualitative study.
1.4 Overview
Chapter 2 Literature Review
In this chapter I present an overview of CNCP, including pharmacological and non-pharmacological
treatments. I also review the use of opioids for CNCP and the opioid crisis. I discuss barriers to provider
adherence to guidelines for opioid prescribing for CNCP. I discuss CDSSs as a knowledge translation
(KT) intervention that may assist primary care providers (PCPs) in prescribing opioids more
appropriately. I explain why CDSSs are complex interventions and review the current guidance for
development and evaluation. I discuss how process evaluations theory can be used to assist in the
evaluation of complex interventions. And finally, I provide an overview of a specific theory, the
Normalization Process Theory (NPT), that can assist in describing the normalization of a complex
interventions.
Chapter 3 Scoping Review Study
In this chapter I present our scoping review study. I start with a focused introduction and then present the
rationale and purpose of the review as well as the specific research questions. I provide the methods we
followed including eligibility criteria, search strategy, study selection process, data extraction and data
synthesis. I present results in tabular and narrative format. I include a brief discussion with an
interpretation of the findings and a conclusion.
Chapter 4 Exploratory Qualitative study
In this chapter I present our exploratory qualitative study. I start with a focused introduction and then
present the rationale and purpose of the review as well as the specific research questions. I provide the
methods including the population of interest, approach to sampling, data collection and preparation. I
provide a description of the specific CDSS and its implementation. I provide the analytic approach to the
interview data and how we used theory to guide the analysis. I also present issues related to privacy and
ethics as well as rigour. In the results section I give a summary of the findings and quotes to provide
support. In the focused discussion I provide an interpretation of the results and a conclusion.
3
Chapter 5 General Discussion
In this chapter I review my thesis research problem and rationale as well as my research aim and specific
objections. I then present an overview of the findings from both studies and review how the findings
answer the aim and specific objections. I review in detail how each finding relates to the aim and
objective and compares to other research in the same area. I present implications of the findings and
review the strengths and weaknesses of the research. I summarize the chapter in the conclusion. Finally, I
present possible future research directions based on the research findings.
4
Literature Review
2.1 Chronic Non-Cancer Pain (CNCP)
2.1.1 Introduction
CNCP is a common, debilitating and costly condition that causes a significant burden of disease in
Canada and throughout the world (48–50). In this section I will present an overview of chronic non-
cancer pain (CNCP) including prevalence, burden of disease and evidence for different treatment
modalities. I will also review the barriers to effective management of CNCP in clinical settings.
2.1.2 Definition and Epidemiology
Chronic pain is typically defined as pain that is present for more than 3 months and present beyond the
expected time of tissue healing (51,52). The definition of CNCP includes the caveat that the pain is not
associated with a cancer (53,54). International estimates of the prevalence of chronic pain range widely
from less than 10% to almost 50% (55–58) with most studies reporting prevalence between 10 and 30%
(48). Estimates of neuropathic pain (a subtype of chronic pain) also range widely; a recent systematic
review put the range of prevalence of neuropathic pain between 3 and 18% with a weighted average at 7%
(48). Some patients may have both non-neuropathic (nociceptive) and neuropathic pain (59). Very few
studies report on the prevalence of CNCP (60). Cancer-related chronic pain, however, appears to
contribute about 1% to the prevalence of chronic pain: an internet survey of a representative panel of the
American population, found that the prevalence of chronic pain (defined as pain present for at least 6
months) was 30.7% and cancer-related pain “contributed 1%” to this prevalence (55). One of the few
studies on the prevalence of CNCP, a Danish study, reported rates of 19% (61), similar to the prevalence
of chronic pain in many studies (48).
A recent systematic review and meta-analysis sought to identify factors that led to the differences in
prevalence estimates (62). The authors found that definitions were highly inconsistent between studies
(and found no improvement in consistency over time). They also found that studies that used interviews
found a lower prevalence than surveys and that this effect was more marked for men than women. The
difference in prevalence estimates between surveys and interviews is consistent with results from earlier
reviews on chronic pain (48,56).
5
Studies have consistently found that chronic pain increases with age (63–65). As a result, the prevalence
of chronic pain is expected to increase over time as the world’s population ages (66). Most studies also
found that women have higher rates of chronic pain than men (60,65,67–69). Chronic pain also appears
to be more common in those with mental health problems, substance use disorder, lower education levels,
lower socioeconomic status, and in those who are unemployed (63,64,67,68). It is unclear if prevalence
varies by a country’s Human Development Index (HDI), a measure that reflects life expectancy,
education and standard of living (70).
Even though chronic pain, by definition, lasts for at least three months, it appears to persist much longer
for many individuals. In a Canadian study, over 50% of respondents had chronic pain for more than 10
years and 25% had it for more than 20 years (69). One European and a Canadian study reported that
common locations of pain are back, neck and spine, knee, leg and shoulder (57,69). Chronic widespread
pain - diffuse pain, associated with malaise and mood changes - is also very common. A recent meta-
analysis that included studies from around the world put the prevalence at 11.8% (95% confidence
intervals: 10.3-13.3) (71). There are two major Canadian national surveys on the prevalence of chronic
pain published in the past decade. Reitsma and colleagues used data from seven National Population
Health Surveys and the Canadian Community Surveys from 1994 to 2008. They found the prevalence
ranged from 15.1% to 18.9%. They used the question “Are you usually free from pain or discomfort” and
included all those who responded “no” as having chronic pain (72). Schopflocher and colleagues
conducted a telephone survey between 2007 and 2008 and used a screening questionnaire that defined
chronic pain as pain present for at least six months. They found the prevalence of chronic pain to be
18.9% for those over age 18 (69).
2.1.3 Pathophysiology
Acute pain occurs as a result of tissue damage or inflammation. Cold, heat, and mechanical insults to
somatic or visceral structures lead to the release of inflammatory mediators and activation of nociceptors.
The pain signal is transmitted to the spinal cord, which modulates the signal and transmits it to the brain
where interpretation occurs (73,74). Pain is also modulated by descending pathways (75). Damage to the
central or peripheral nervous system causes neuropathic pain (51). The development of chronic pain is
not well understood (76). It depends on factors besides the degree and type of tissue damage or
inflammation, and the primary etiology of the pain may be unknown (51). Instead of adaption after tissue
healing, the pain system becomes more sensitive and the up-regulation can spread to surrounding
6
structures (74). Research suggests that sensitization happens at many levels from the peripheral to the
central nervous system (75,77).
2.1.4 Burden of CNCP
Individuals report that CNCP has devastating impact on their quality of life, with more severe pain having
a more severe impact (48). CNCP often affects all aspects of life, from activities of daily living to work
to personal relationships (48,60,78,79). One European study found that 60% of those with chronic pain
had difficulty leaving home to work, 19% had lost their job because of chronic pain, and 21% had been
diagnosed with depression (57). Not surprisingly, CNCP leads to lower levels of activity (80). Chronic
pain results in high health care utilization (57); those with CNCP make an average of six to ten visits per
year to see a health care provider (48). The total costs to society are enormous. An American study
estimated the total costs of chronic pain to American society was $560 to $635 billion dollars in 2010
(50). A 2011 Canadian study put the national total cost at $43 billion per year (49). A recent study in
Ontario, Canada estimates the incremental cost to manage chronic pain to be $1742 per individual/year
(81).
2.1.5 Pharmacological Treatments
Historically, CNCP treatment focused on pharmacological treatments within a biomedical model (53,66).
Opioids are best studied, but appear to cause more harm than benefit for most people. Most of the older
experimental studies that showed some improvement in pain were less than three months in duration and
did not include an active comparator (12,54). A recent well-designed experimental study showed opioids
had no benefit for osteoarthritis pain compared to other pharmacological treatments at 12 months (82).
Observational studies indicate that opioids lead to worse outcomes with respect to pain and function, and
cause significant harm including addiction, overdose and premature death (12,54). There is little
evidence for the effectiveness of other pharmacological treatments. A recent systematic review on
anticonvulsants also concluded that there was no benefit from anticonvulsants in CNCP that was not
neuropathic in nature, and also stated that gabapentinoid types of anticonvulsants have “a higher risk for
adverse events” (83). Cannabis has little evidence in CNCP (84) and a recent Canadian guideline
recommends that it should be avoided (85). There is little evidence to support use of anti-depressants,
such as selective serotonin reuptake inhibitors (SSRIs) or tricyclics in non-neuropathic CNCP (86). With
respect to specific conditions, systematic reviews on low back pain found no benefit from acetaminophen,
some benefit from nonsteroidal anti-inflammatory drugs (NSAIDs) (but smaller than previously reported)
7
and “modest” improvements from duloxetine (87). NSAID use may be limited by side effects and risk of
adverse effects (88). A recent systematic review of gabapentinoids in low back pain concluded evidence
was limited and side effects significant without “demonstrated benefit” (89). Acetaminophen leads to
minimal improvements in osteoarthritis pain (90,91). There is evidence to support some pharmacological
treatments for neuropathic pain. A 2015 high quality systematic review and meta-analysis, however,
estimates that publication bias has led to about a 10% overestimate of benefit (92). According to this
analysis, the most effective drugs for neuropathic pain are tricyclic antidepressants, serotonin-
noradrenaline reuptake inhibitors (duloxetine and venlafaxine), and gabapentinoids. Incorporating
tolerability and safety data, the authors made a strong recommendation for these medications and weak
recommendations for lidocaine and capsaicin patches and tramadol. They recommended strong opioids as
third line and stated data was lacking for cannabinoids. The Canadian Pain Guidelines make similar first-
line recommendations (93). Several other recent studies support these conclusions. A Cochrane review
concludes that there is very weak evidence from small studies with high risk of potential bias to support
use of tramadol in neuropathic pain (94). Another recent review concludes cannabis has limited evidence
for neuropathic pain and can be associated with adverse mental health problems (95). Many neuropathic
pain conditions have been studied individually (e.g. diabetic neuropathy, post-herpetic neuralgia, lumbar
radiculopathy) with recommendations for specific conditions (96). For example, caudal epidural
injections may improve outcomes for radiculopathy but the evidence is of very low quality (97,98). As
with non-neuropathic CNCP, observational studies indicate opioids do not lead improve function and
cause addiction and overdoses for those with neuropathic pain (99).
2.1.6 Non-pharmacological Treatments
Given the failure of pharmacological therapies to provide significant relief, researchers have turned to
alternatives. Psychological therapies have been well researched: there are currently over 100 randomized
controlled trials (RCTs) examining the impact of psychological therapies on CNCP included in four
Cochrane systematic reviews; however, many studies are small and of poor quality (100). In an overview
of these reviews, the authors concluded that cognitive behavioural therapy (CBT) shows a small to
moderate benefit for CNCP, but that the evidence is likely to change with more studies (100). A more
recent systematic review examined the impact of psychological therapies on other outcomes for those
with CNCP (101). They found that psychological therapies had moderate reductions on health care
utilization and time lost from work as compared to waiting list, care as usual and active controls.
Therapies included CBT, mindfulness and Acceptance and Commitment therapy (ACT). Other recent
systematic reviews have found that mindfulness, Tai chi, yoga and ACT appear promising (87,102–104).
8
Internet-based psychological therapies also show promise: a review found some improvements in pain
and disability in those with all types of chronic pain, but only improvements in depression and anxiety for
those with non-headache pain conditions (105). There is insufficient evidence to determine the impact of
psychological therapies on neuropathic pain (106). A recent Cochrane “review of reviews” evaluated
exercise as a treatment modality for CNCP. It found that the evidence is of low quality but there is
indication of some benefit. Additionally the risk of harm from exercise is low and exercise provides many
other benefits (53). A systematic review on low back pain also supports the role of exercise (107). Poor
adherence to an exercise regimen, however, may limit its effectiveness (108). Acupuncture appears to
lead to some improvement in chronic pain. In RCTs where it is compared to a sham procedure, it reduced
pain in CNCP (109). Multi-modal (or multi-disciplinary) approaches are gaining traction. They appear to
improve low back pain outcomes, decreased pain and disability (110).
2.1.7 CNCP in Practice Settings
Providers in Canada and around the world report difficulties managing CNCP in practice (111–113).
Patients also appear to be dissatisfied with treatment for CNCP in practice: one large European study
found that 1/3 of those with chronic pain were currently not in treatment, and of those in treatment, 40%
reported inadequate alleviation of their pain (57). In a Canadian survey, 40% of people with CNCP
reported that their first interaction with a health care provider about chronic pain was a negative
experience and only 20% reported feeling hopeful after the interaction (114). Canadian and American
studies report that a lack of providers’ skills, knowledge and confidence managing CNCP is a major
factor (115–117). A underlying reason may be a lack of appropriate training: Canadian, American and
international studies indicate inadequate training in the management of chronic pain during medical
school and residency (116–121). Another factor in Canada, the USA, and internationally appears to be
poor access to appropriate treatment modalities as well as inadequate local practice supports (122,123).
Patients report long waits to see a specialist (114,124); Canadian patients reported an average of 18
months to see a pain specialist. In the USA, VA patients report cost and transportation as barriers to
treatment (125). Communication barriers and differing views between patients and provider may also
present a problem. Authors of a non-systematic review report that patients often have different goals than
their providers and have difficulties getting their concerns heard by the provider (112). Another factor
may be stigma: a survey of residents in the USA found 30% used derogatory terms to describe people
with CNCP (126). In a study in North England, patients reported a lack of empathy as a major barrier to
care (124).
9
2.1.8 Summary
CNCP is a common condition with significant impacts on function and quality of life (48–50). Providers
in Canada, the U.S. and internationally, often do not have appropriate training, supports or resources to
manage CNCP (116–121) . In the next section, I will review how in the late 1990s, Purdue Pharma in
North America saw this as ripe conditions for marketing their new opioid drug— Oxycontin (127).
2.2 Opioid Prescribing for CNCP
2.2.1 Introduction
Prescriptions for opioids for CNCP have dramatically increased in Canada and the U.S in the past twenty
years (1,12). Many of the prescriptions are issued by PCPs (3–5). In this section I will demonstrate how
opioids became widely used for CNCP despite the lack of evidence to support their use. I will also review
the subsequent harms from the increase in opioid prescriptions and current recommendations for opioid
prescribing for CNCP.
2.2.2 Physiology
Opioids are natural, synthetic or semi-synthetic substances that bind to opioid receptors. Humans have
three major opioid receptors: mu, kappa and delta(128). The effect of an opioid is determined by which
receptor it binds to, how strongly it binds and whether it has an agonist or antagonist effect. The mu
receptor is responsible for the pain-relieving effects and euphoria from opioids. It is found in the nervous
system and gastrointestinal tract
2.2.3 Opioids and CNCP
Opioids effectively reduce acute pain (129); however, there is limited evidence to support their use in
CNCP. Most experimental studies are too short to evaluate long-term outcomes (1,12). As stated by
Chou and colleagues in a systematic review on opioids for chronic pain in 2015:
“No study of opioid therapy versus placebo, no opioid therapy, or non-opioid therapy evaluated
long-term (>1 year) outcomes related to pain, function, or quality of life.” (9).
A more recent RCT did examine longer term outcomes: it found that opioids had no benefit for
osteoarthritis pain compared to other pharmacological treatments at 12 months (82). Observational
10
studies indicate that opioids for CNCP do not lead to reduced pain or better functional outcomes (130–
132), perhaps in part due to neuro-adaption (133). Tapering appears to improve both pain and function
(134–137). Additionally, the dramatic increase in opioid prescribing at the population level has not
reduced the prevalence of chronic pain (138).
2.2.4 Opioid Prescribing
Prescriptions for opioids have dramatically increased. In Canada, the dispensing of prescription opioids
increased from 10,209 defined daily doses (2001 and 2003) to 30,540 (2012 and 2014) (1). In the USA. in
2015, the morphine milligram equivalents per capita were three times higher than in 1999 (2). PCPs
provided many of these prescriptions. In 2009, in the U.S. family practitioners, general practitioners and
osteopaths provided about 30% of all opioid prescriptions (4). When combined with internal medicine,
these primary care specialties provided almost half of all opioid prescriptions between 2007 and 2012 (3).
The rate of prescribing in primary care specialties increased over 5% in the five years. In Ontario,
Canada, family doctors provided almost half of new opioid prescriptions (5). Emergency medicine
physicians also increased opioid prescribing for pain in 2000 to 2010 in the U.S. (139,140). An
American study found that although emergency medicine physicians provide a much smaller absolute
number of opioid prescriptions - 1.4% of the total - they had a much higher prescribing rate (opioid
prescriptions/total prescriptions) than family practice (20.7% versus 5.6 %) (3). Opioid prescribing in
Canada and the United States has leveled off in the past several years (3,141). However, in the U.S.
prescribing is still three times higher per capita than in the 1990s (2). In Canada the population-adjusted
opioid prescribing decreased about 9% (defined daily doses) from 2012 to 2016 -- with most of the
decline between 2015 and 2016 (142). Additionally, the total number of people per 1000 dispensed
opioids decreased from 2012 to 2016. However, the proportion of people prescribed strong opioids
chronically remained steady (8% of all those prescribed). There is substantial variability in opioid
prescribing in Canada and the U.S. that is not accounted for by levels of pain (143–145). In Ontario, the
top quintile of opioid prescribers issued opioid prescriptions 55 times more often than the lowest quintile
(146). There is also variation in type of opioid prescribed between regions. In a recent study, Ontario
physicians had the highest annual rate of high-dose oxycodone and fentanyl prescribing compared to
other Canadian provinces (147).
11
2.2.5 International Comparisons of Opioid Prescribing
Internationally, many high income countries besides Canada and the U.S. have had an uptick in opioid
prescribing and associated harms (10,148–153). However, the U.S. and Canada far outpace all other
countries in opioids consumed (154). They are the highest consumers of opioids per capita in the world
(155). The major factor contributing to this high rate of use appears to be a marketing campaign by
Purdue (the pharmaceutical company that produces Oxycontin, a long-acting opioid), a campaign focused
on the U.S. and Canada (10,11,127,156). Additionally, Canada and the U.S. have looser restrictions on
narcotic prescribing; widespread use and acceptance of psychotropic drugs; and high expectations for
effective medical treatments (154).
2.2.6 Roots of the Increase in Prescribing
Up until the 1990s, most physicians only prescribed opioids for acute severe pain or for cancer-related
pain and palliative care (10), apparently because of concerns about addiction and overdose (157). Some
physicians may have remembered the lessons from the American prescription opioid crisis in the early
20th century:
“It is daily becoming better known that opium, its derivatives and co-caine[sic] are being used in
alarming amounts all over this country. Various factors, such as the careless prescribing of these
drugs by physicians, the spread of habit from person to person, the cupidity of druggists and
patent medication manufacturers, and vice and dissipation are responsible for the existing
conditions (158).”
In the 1980s, a small pharmaceutical company, Purdue, developed a “contin” system for morphine that
turned a short-acting narcotic into one that lasted up to 12 hours (159). It was approved for cancer
patients in end-of-life care, a small market. In 1996 Purdue decided to try to expand the market for long-
acting opioids to the much larger chronic pain market (159). Purdue selected oxycodone, a synthetic
opioid, and created Oxycontin. To encourage physicians to prescribe the new drug, Purdue launched a
marketing campaign of unprecedented proportions (127). One key component of the campaign was the
use of physicians to market to other physicians. In the U.S., Purdue hired and trained over 2500
physicians to be part of its speaker bureau to present at conferences and at drug company-funded dinners
(160). The conferences and drugs dinners were often accredited by medical organizations, lending them
legitimacy. This approach appeared to work:
12
“…internal Purdue records indicate that doctors who attended these seminars in 1996 wrote
OxyContin prescriptions more than twice as often as those who didn’t (159).”
In total Purdue funded over 20,000 pain-related programs in the U.S. (160). No such data are available in
Canada. The marketing campaign contained many messages that were not supported by evidence. In
addition to claiming that Oxycontin was effective for chronic pain, Purdue also claimed that it was less
addictive than short-acting narcotics (127). Eventually it became obvious that many with an opioid
addiction preferred Oxycontin, as crushing the tablet disabled the slow release system and delivered a
high dose of narcotic all at once. A justice department report revealed that Purdue may have been aware
of the drug’s significant addictive potential soon after the drug’s release in 1996, but did not disseminate
this information (161). The Oxycontin campaign was one of the most successful in the history of the
pharmaceutical industry. Sales soared from $44 million in 1996 to over $1.5 billion in 2002 (160).
Purdue was eventually found guilty of misleading physicians and patients and had to pay $634 million in
fines (127).
2.2.7 Harms from Prescribed Opioids
Prescribed opioids place people at risk for addictions, cardiovascular events, motor vehicle collisions,
fractures, fatal and non-fatal overdoses, and death (6–9,162). A 2015 systematic review on opioids for
chronic pain included ten fair quality studies that investigated the risk of opioid misuse and opioid use
disorder (OUD) among people prescribed opioids (9). (OUD is defined as a problematic pattern of opioid
use leading to clinically significant impairment or distress (163).) In primary care settings (three studies),
rates of opioid misuse ranged from 0.6% to 8% and dependence (the Diagnostic and Statistical Manual of
Mental Disorders IV term for a more severe OUD) ranged from 3% to 26%. In pain clinics (seven
studies) prevalence of dependence ranged from 2% to 14%. The review also reported that OUD appears
to be related to opioid dose: one of the studies compared low-dose and high-dose therapy and found that
the rate of OUD was 0.7% with low-dose and was 6.1% with high-dose therapy (164). Factors associated
with misuse in the systematic review were past substance use disorders, younger age, depression and use
of psychotropic medications (9). Risk of overdose and overdose death from prescribed opioids is dose-
related (6–8). An American cohort study found that risk of overdose increased 3.7 fold for those on doses
between 50 to 100 MED and 8.9 fold for those on doses over 100 MED compared to those on doses less
than 20 MED (6). An American study of the Veterans Health Administration reported that the adjusted
hazard ratio of overdose death for those on doses of over 100 morphine equivalent dose (MED)
compared to less than 20 MED was 7.18 for those with chronic pain (165). A case control study in
13
Ontario, Canada of patients with non-malignant pain eligible for publicly-funded prescription drug
coverage, found that risk of death tripled at 200 MED (8). Another Ontario study that followed patients
on opioids for CNCP for 13 years found that 1/550 died from an opioid overdose (166). This increased to
1/32 for those who escalated doses to over 200 MED.
2.2.8 The Opioid Crisis
The increase in opioid prescribing led to an increase in population level harms and spawned our current
opioid crisis (10,11). In the U.S. in 2016, more than 42,000 died from an opioid overdose (167), more
than triple the number in 1999 (168). In Canada, in 2017 there were 3996 opioid-related deaths, up from
3005 in 2016 (169). The rates of OUD in the general population increased from 0.3% in 2003 to 0.9% in
2013 in U.S. (170). In 2015, over 2.5 million Americans had OUD (171). Canadian statistics for OUD
are not available. The opioid crisis has evolved overtime. The flood of prescription opioids, and
subsequent increase in opioid misuse and addiction created a market that drug cartels eagerly filled with
illicit opioids (172). Difficulty in accessing prescription opioids may have also helped the illicit market
(173). Use of illicit opioids is riskier than use of prescription opioids because of the lack of quality
control (174,175). Opioids and heroin are frequently contaminated with fentanyl and fentanyl analogues
some of which have a potency of 10,000 times that of morphine (174,176). As a result, there has been a
shift in cause of deaths from prescription opioids to fentanyl and fentanyl analogues (168,174,177,178),
mostly from illicit sources (174,176). In Canada, there was a relative increase of 34% in the number of
opioid-related deaths in 2017 compared 2016, and the proportion of deaths involving fentanyl increased
from 55% to 72% (169). The CDC in the U.S. reports that half the increase in deaths from 2013 on was
the result of fentanyl and heroin combined (179).
2.2.9 Guidelines for Opioid Prescribing for CNCP
To address the harms, and the lack of long-term benefit from opioids for CNCP, recent guidelines
recommend a major shift in approach. The 2016 CDC Guideline for Prescribing Opioids for Chronic Pain
and the 2017 Canadian Guideline for Opioids for Chronic Non-Cancer Pain recommend against the use of
opioids for CNCP in most circumstances (1,12). Both recommend restricting the opioid dose to no more
than 50 MED for most patients (although this is a weak recommendation in the Canadian guideline) and
rarely prescribing above 90 MED. These guidelines also recommend more careful prescribing for patients
already prescribed opioids. They recommend a slow, patient-directed taper of opioids to a lower dose or
to complete discontinuation, particularly if opioids have not been effective, or if patients are experiencing
side effects. The guidelines also recommend that providers encourage patients to try other treatment
14
modalities for CNCP, including other pharmacological and non-pharmacological treatments, such as
exercise and cognitive behavioural therapy. They recommend providers give overdose prevention
education, prescribe naloxone, and avoid co-prescribing benzodiazepines. The guidelines recommend
providers seek to identify aberrant behaviours that may indicate opioid misuse or an addiction, and
connect these patients to treatment.
2.2.10 Adherence to Guidelines for Opioid Prescribing for CNCP
The literature shows that most providers prescribing practices do not align with the guidelines (13–17). A
recent systematic review reported that although prescribers only use opioids when other approaches have
failed, and avoid doses above 200 MED, they often failed to discontinue opioids when they are ineffective
for pain (180). Other studies reported that physician and residents were only partially compliant with
ordering urine drug tests and using opioid contracts (14,17,181). Making these behavior changes is
unlikely, without substantial supports, education and training (182,183). Education or training alone does
not appear to be effective at improving the appropriateness of opioid prescribing. McCracken and
colleagues assessed whether training could improve prescribing and found that although it increased
knowledge, it did not lead to changes in practice behavior (15). Kahan and colleagues also found that an
educational intervention had no impact on opioid prescribing (184).
2.2.11 Summary
Patients continue to experience substantial opioid related harms in Canada and the U.S. from prescribed
opioids. To prescribe opioids more appropriately, providers need to make a number of complex changes
to how they address chronic pain and prescribe opioids. In the next section, I will review why this type of
complex behavior change is difficult to achieve.
2.3 Knowledge Translation
2.3.1 Introduction
In the last section I reviewed current guideline recommendations for more appropriate opioid prescribing
for CNCP. To follow these guidelines requires complex behavioural changes. The field of knowledge
translation (KT) can provide assistance. In this chapter, I provide an overview of KT, including a review
of KT strategies that have been employed to change opioid prescribing behavior. (185).
15
2.3.2 History of KT
The field of KT emerged in the late 1990s over concern about the large gap between knowledge and
practice in all areas of medicine (186). A seminal study found that patients only receive 55% of
appropriate treatments available (187). Some areas had a larger gap than others, but in all areas of
medicine, treatments were overused, underused and misapplied. Researchers reported that the reasons
included the overwhelming amount of medical literature; lack of resources; financial disincentives; lack
of appropriate provider knowledge and skills; and poor patient buy-in (188,189). By the early 2000s, it
was clear that passive continuing medical education did not lead to significant changes to clinical practice
(183). As a result, researchers proposed evidence-based KT to close the knowledge to practice gap
(182,183).
2.3.3 Definition and Overview of KT
KT is also known by other terms including: implementation science, research utilization and knowledge
management (186). The Canadian Institute of Health Research (CIHR) defines KT as
“a dynamic and iterative process that includes synthesis, dissemination, exchange and ethically-
sound application of knowledge to improve the health of Canadians, provide more effective
health services and products and strengthen the health care system.”(190)
KT is often divided into two types: end of grant KT and integrated KT (190). End of grant KT includes
activities that occur after research is completed. Integrated KT incorporates KT throughout the whole
research process, in an iterative manner. KT is a bidirectional and collaborative process with knowledge
flowing between researchers and knowledge users (186). As part of developing and implementing and
evaluating a KT plan, researchers often use theories, models and frameworks. These assist researchers in
identifying causal pathways, barriers to change and in deciding on appropriate KT interventions. There
are a large number of KT models, theories and frameworks; a recent scoping review identified over 150
different ones used in 596 studies. Most (87%) were used in five or fewer studies (191). Despite
widespread use, KT theories, models and frameworks have been poorly evaluated and their impact is not
well understood (191). One of the KT models, theories and frameworks that can be used across from
planning and development, to implementation, and evaluation is the Knowledge to Action (KTA) cycle
developed by Graham and colleagues (192). They created it through the identification of common
elements in more than 30 planned action theories (186). CIHR, Canada’s health research investment
agency, adopted this model for promoting KT.
16
2.3.4 CIHR KTA Cycle
According to the CIHR KTA cycle, KT consists of four elements: knowledge synthesis, dissemination,
exchange and the ethically-sound application of knowledge (190). The synthesis must be reproducible
and transparent. Dissemination involves identifying the audience and selecting appropriate strategies. In
exchange of knowledge there is sharing between researcher and knowledge user (190). To create
knowledge for translation, researchers synthesize primary data (studies) to form secondary knowledge in
the form of systematic reviews and meta-analyses. The synthesized knowledge is used to create
knowledge tools for dissemination such as clinical practice guidelines and patient decision aids.
Researchers then identify the gaps between knowledge and practice. A gap is a difference between best
(synthesized) evidence and practice. As gaps in practice abound, researchers should have a method to
select the most important areas (193). Considerations include the burden of disease, cost, feasibility and
funding. At this stage, researchers often develop quality indicators to measure gaps and change in quality
or outcomes (193). The National Library of Medicine defines quality indicators as
“norms, criteria, standards and other direct qualitative and quantitative measures used in
determining the quality of health care” (194).
They allow for monitoring of the quality of care and the impact of interventions or in some cases, as
sentinels, triggering follow-up and more investigation (195). To work well, indicators need to be valid,
reliable, modifiable and feasible (186). Once created, indicators can be used across settings to identify
gaps in specific settings. They may need to be adapted, however, to suit other contexts. Quality indicators
have limitations. There is a lack of standardized way to select, develop and evaluate indicators
(186,196,197). Additionally, indicators only assess one small area (usually easily measurable areas) of
quality (197). This may create an incentive to only address that specific area at the expense of others.
After researchers have identified a gap and selected knowledge for translation, they may need to adapt
knowledge for the local context. This may be particularly important when moving between countries and
regions with differing levels of disparity and access to resources (198). Researchers then identify barriers
and facilitators to change. Research has identified hundreds of different types of barriers. Common ones
include lack of knowledge, lack of skills, lack of self-efficacy, low motivation, problems with recall, and
organizational constraints (186). There are a large number of tools available to assist researchers in
determining the barriers to change in their context: a recent systematic review of instruments to assess
organizational readiness for knowledge translation in health care found 26 different instruments (199).
These authors found that the Texas Christian University Organizational Readiness for Change (TCU-
17
ORC) had the highest instrument validity, and a modified version of the TCU-ORC had the highest
reliability scores (199).
In the next step, researchers select and tailor the type of KT intervention to the knowledge gap and
context. There is limited evidence to guide the selection of appropriate KT techniques in most areas and
contexts (186). Therefore, most researchers use one of two approaches when selecting KT interventions:
an exploratory approach using group-based brainstorming or a theory-based approach (200). Throughout
the KT process, researchers should be monitoring and evaluating the process and outcomes, and adjusting
their KT plan as needed. Evaluation should include quantitative and qualitative methodologies (186).
Figure 2.1 Knowledge to action process/cycle (201)*
* Reproduced with permission
18
2.3.5 KT Interventions
KT interventions fall into several categories and have varying degrees of evidence to support their use
(186). Educational interventions are widely used, but, in general, have little impact on behavior. Some
educational interventions are more effective. These include educational sessions with an interactive
portion and sessions that focus on important health outcomes. Outreach educational visits (academic
detailing) are also more likely to be effective (202); in a systematic review they improved adherence to
desired practice by about 5.6%. Opinion leadership is another intervention. Opinion leaders sit at the
centre of communication networks and are able to reach a large number of people (186). They are widely
used in pharmaceutical companies (203). Outcomes are variable, but use of opinion leadership appeared
to lead to a 12% increase in adherence to desired practice in a systematic review (204). Audit and
feedback is a commonly used intervention that has a modest impact on prescribing with significant
variation in outcomes between studies (205). There is currently limited understanding of essential
components (206). Shared-decision making leads to mixed outcomes according to a recent systematic
review; the highest percentage of studies reported positive outcomes for patient affective-cognitive
outcomes (52%) as compared to behavioural (37%) and health outcomes (25%) (207). Financial
incentives have limited evidence to support their use (208–210). Informatics applications can be
effective tools for KT. These include clinical decisions support systems (CDSSs), electronic
communications, and handheld and mobile technologies (186). Many informatics applications lead to
process improvements, but their impact on patient outcomes are unclear (211). Patient-targeted
interventions appear to be promising (186,212). A systematic review found that both simple and complex
interventions had an impact on “patient knowledge, decision-making, communication and behavior”
(213).
2.3.6 Guidelines and Indicators for Safer Opioid Prescribing for CNCP
Numerous organizations have synthesized the data and produced guidelines with recommendations for
practice. Two recent national important guidelines include the 2016 CDC Guideline for Prescribing
Opioids for Chronic Pain and the 2017 Canadian Guideline for Opioids for Chronic Non-Cancer Pain
(12,54). Little work, however, has been done in developing indicators for safer opioid prescribing for
CNCP (214). Two exceptions are the Veterans Affairs (VA) in the U.S. (215) and researchers in New
Zealand (216). The researchers in New Zealand built on the VA indicators and developed new measures
appropriate to the New Zealand context. In Ontario, Health Quality Ontario (HQO), the provincial lead on
19
the quality of health care, produced its own indicators for a recent quality standard to track the quality of
opioid prescribing for CNCP in Ontario (217). The provincial indicators are:
1. Rate of opioid-related deaths, emergency department visits and hospital admissions
2. Rate of people prescribed opioids and rate of opioid prescriptions dispensed
3. Percentage of people with chronic pain with improved quality of life
4. Percentage of people with chronic pain with improved functional outcomes
5. Percentage of people with chronic pain who experience reduced pain
6. Percentage of people who are prescribed opioids for chronic pain and subsequently develop OUD
HQO recommends that these indicators can be used to assess quality of care in Ontario.
2.3.7 Barriers to More Appropriate Opioid Prescribing for CNCP
There appear to be numerous barriers to more appropriate opioid prescribing. Many physicians state that
their training did not adequately prepare them for prescribing opioids for CNCP (21,22) and that they do
not feel confident managing CNCP (23). Some also cite a lack of supports, including poor access to other
pain treatment modalities (24,25). Other barriers include the significant amount of time it takes a
provider to manage CNCP, prescribe opioids safely and screen for misuse and addiction (218).
Additionally, stigma may prevent physicians from asking about opioid misuse and addiction (26). This
may impede provider-patient communication. Physicians and patients report and demonstrate discomfort
in communicating about opioids for CNCP (18–20). Physicians report they have difficulty asking about
aberrant behaviours or assessing for OUD (24). Physicians may also have a fear of angering or
distressing patients if they refuse to provide opioids or recommend tapering to a lower dose (219).
2.3.8 KT Interventions for Opioid Prescribing for CNCP
Various KT interventions show promise in improving the appropriateness of opioid prescribing for
CNCP. A 2018 systematic review of 65 studies looked at interventions to improve appropriate use of
opioids in CNCP (185). The majority of the interventions were targeted at health care providers. As there
were few RCTs and as the studies were heterogeneous and often of low methodological quality, the
authors used a qualitative analysis. The most promising interventions that improved appropriate use of
opioids, to reduce OUD and deaths included:
“…education, clinical practices, collaborations, prescription monitoring programs, public
campaigns, opioid substitution programs, and naloxone distribution.” (185)
20
Going forward, these authors recommend high quality empirical studies to evaluate promising
interventions.
2.3.9 Summary
Therefore, there are a number of KT approaches that may change how providers prescribe opioids for
CNCP. The steps in the KTA cycle model include synthesizing knowledge, identifying gaps, adapting
knowledge to local context (if needed), identifying barriers, selecting appropriate KT approaches,
monitoring and evaluating outcomes (186). In the next section I provide an overview of scoping reviews,
a form of knowledge synthesis that we used in a study on CDSSs for opioid prescribing for CNCP in
primary care settings (chapter 3).
2.4 Scoping Reviews
2.4.1 Introduction
Scoping reviews provide a particular systematic and reproducible approach to assess and summarize the
evidence on a topic (220,221). This section provides an overview of scoping reviews including current
guidance and reporting standards.
2.4.2 History of Reviews
With the rise of RCTs and evidence-based medicine (EBM) in 1990s, researchers saw the need to
systematically and accurately summarize the evidence on a particular topic (222,223). Up to that time,
most reviews were conducted by experts in the field who produced a narrative summary of the evidence
without using scientific methods (224). Studies indicated, however, that these reviews did not accurately
rate quality of evidence (225) and often failed to mention important advances or contained harmful or
unsupported treatment (226). In response, a group of researchers established the Cochrane Collaboration
in 1993 with the stated goal to:
“prepare, maintain and disseminate systematic, up-to-date reviews of RCTs of health care, and,
when RCTs are not available, reviews of the most reliable evidence from other sources”(227) .
High quality systematic reviews enable health care providers to efficiently make evidence-based health
care decisions (221,228).
21
2.4.3 Systematic Reviews and Meta-analyses
In the past twenty years, there has been a proliferation in review types. They vary in process and rigour
(198). Systematic reviews are the prototypical review with a systematic approach. They use a
reproducible method of reviewing, analyzing and displaying the evidence from health care studies.
According to the Cochrane reviews handbook (229) the key characteristics of a systematic review are:
a clearly stated set of objectives with pre-defined eligibility criteria for studies;
an explicit, reproducible methodology;
a systematic search that attempts to identify all studies that would meet the eligibility criteria;
an assessment of the validity of the findings of the included studies, for example through the
assessment of risk of bias; and
a systematic presentation, and synthesis, of the characteristics and findings of the included
studies.
Systematic reviewers often use statistical methods to pool the data to provide a more precise estimate of
effect size using a process called a meta-analysis (229,230). Chalmers and colleagues state that this was
the first main advance in systematic reviews. The second was the development of mechanisms to identify
and address bias (231). Failure to identify all relevant studies—leading to an inaccurate assessment of
effect size—is a major pitfall in systematic reviews. This usually occurs because relevant studies are not
available because of reporting bias. Common types of bias that lead to reporting bias are (229,232):
1) publication bias (non-significant findings and findings not in favour of a sponsor’s intervention are less
likely to be published);
2) time lag bias (more interesting findings or discoveries tend to be published faster)
3) language bias (non-English studies are less likely to be published); and
4) outcome reporting bias (not all pre-specified outcomes are published).
To address publication bias and selective outcome reporting, many journals now require that all
prospective trials are pre-registered (233). Another pitfall in systematic reviews is the inclusion of low
quality studies with high risk of bias. Therefore systematic reviews incorporate methods to detect bias
within studies and mechanisms to address the results (229).
22
2.4.4 Scoping Reviews
Scoping reviews provide a different systematic and reproducible approach to assessing and summarizing
the evidence on a topic (220,221,234). They are used for a variety of reasons including to
“…examine the extent (that is, size), range (variety), and nature (characteristics) of the evidence
on a topic or question; determine the value of undertaking a systematic review; summarize
findings from a body of knowledge that is heterogeneous in methods or discipline; or identify
gaps in the literature to aid the planning and commissioning of future research” (235).
Scoping reviews tend to look at a broad area as opposed to a specific question (as is common with
systematic reviews). They also tend to include a wide variety of study types. They do not quantify effect
sizes. Additionally, in most cases, they do not include a formal assessment of study quality (220)(221).
Scoping reviews are a relatively new type of review with increasing use over time. Colquhoun and
colleagues report that there were less than ten scoping reviews published yearly prior to 2009 and over 80
per annum by 2013 (221). They are useful in both emerging and established fields; in an emerging field
they can assess the extent of the literature; in established fields they can provide a mechanism to map the
“abundance of evidence” (221). They are also frequently conducted to prepare for a systematic review
(220). The field, however, is plagued by variability in terminology and methodological approach (221).
In this guidance article, Colquhoun and colleagues recommend, that going forward, the use of the
following definition:
“A scoping review or scoping study is a form of knowledge synthesis that addresses an
exploratory research question aimed at mapping key concepts, type of evidence, and gaps in
research related to a defined area or field by systematically searching, selecting, and
synthesizing existing knowledge” (221).
Recently a research team used the EQUATOR (Enhancing the QUAlity and Transparency Of health
Research) guidance to adapt reporting guidance for systematic reviews, PRISMA (Preferred Reporting
Items for Systematic reviews and Meta-Analyses), for scoping reviews: PRISMA-ScR (235). In depth
guidance for conducting scoping reviews is available in the frameworks by Arksey and O’Malley (234)
and in the enhancements by Levac et al (236). These frameworks are also used in the Joanna Briggs
Institute (JBI) Reviewers’ Manual 2015 Methodology for JBI Scoping Reviews (220).
23
2.4.5 Summary
Scoping reviews, therefore can provide a systematic way to map evidence in a particular field. They are
often used to map the extent, range and nature of the literature, and can be particularly useful in
summarizing studies across fields or with disparate methodologies. They can be used to identify gaps and
to plan for future research, including a systematic review.
2.5 Clinical Decision Support Systems (CDSSs)
2.5.1 Introduction
There is increasing interest in CDSSs as a mechanism to improve health care processes and patient
outcomes including CDSSs for opioid prescribing for CNCP in primary care settings (237–239). In this
section I will provide an overview of CDSSs, including potential benefits as well as limitations and risks.
2.5.2 Definition, Taxonomy and History
CDSSs are electronic systems that assist health care providers (HCP) in making a clinical decision by
providing patient-specific data at point-of-care. (37,240,241). They are a form of medical informatics or
electronic health (e-health) technologies. CDSSs are intended to work by presenting appropriate
information or conducting calculations, at the time when providers need it, without overloading with
excess information (186). The CDSS may then lead to change or support provider behaviour at the point
of care. CDSSs may support simple or complex behaviours from screening to diagnosis and management
(186). For example, a simple alert may stop a provider from prescribing a medication that puts a patient
at risk of a drug-drug interaction. A more complex CDSS may guide a provider through a number of steps
to monitor and treat congestive heart failure. A CDSS may be integrated into an EMR or a standalone
application, such as a web-based program or an application on a smart phone. Integrated CDSS may
automatically initiate or may depend on a HCP to activate them (39). Some CDSS may require providers
to input patient data and other CDSSs glean patient data from electronic databases, such as the EMR or a
prescription drug database. Others still combine both modes of data acquisition. Common types of CDSSs
include alerts, protocols, dashboards, data repositories, and prognosis calculators (242,243). A third, less
common type, provides diagnostic support. Some of these CDSSs are highly complex and use Bayesian
analytics to provide decision-making support (242).
24
Table 2.1 Common types of CDSSs
Type Description
Reminder Provides a warning to conduct a behavior
Alerts Provides a warning to halt a behaviour
Protocols or algorithms Provides guidance on managing a clinical condition
Dashboards Provides a single location to view key data items to inform provider behaviour
Data repositories Provides access to data to inform provider behaviour
Prognosis calculators Conducts complex calculation to inform provider behaviour
Diagnostic support Uses analytics to provide decision support
CDSSs first came into use in the 1950s with the development of mainframe computers (242). Some of
the earliest systems were prescription modules. Other early CDSS provided diagnostic support. Over
time, however, many of these large systems became difficult to maintain because of the rapid increase and
ever-changing medical knowledge (242). More recently, developers have integrated CDSSs into an
organization’s EMR. This reduces or eliminates the burden of data entry (244).
2.5.3 Evidence of Effectiveness
Systematic reviews show that CDSSs have a modest impact on process measures, such as adherence to
guidelines, preventative care, safer prescribing and in improving the efficiency of health care systems
(27–35). There is less evidence to support CDSS use in chronic disease management; a systematic review
found that in just slightly over half of the studies the CDSS improved the processes of chronic disease
management, such as adherence to diabetes monitoring (29). Impact on patient health outcomes is less
clear. These outcomes less likely to be assessed in studies and outcomes are mixed (27–30). Additionally,
there is little research on the risks of CDSSs (245). For example, a poorly designed CDSS may lead to
inefficiencies through workarounds or an overly restrictive CDSS discourage appropriate medications
(244).
Some researchers have sought to determine why some CDSSs were more likely to achieve pre-defined
outcomes than others. A systematic review by Garg and colleagues in 2005 reported that outcomes were
25
better if the system was integrated and automatically activated instead of requiring provider activation and
if the developers were also the evaluators. A systematic review by Kawamoto published in the same year
supported the first finding and also reported that requiring a reason for over-ride and giving advice to
patients as well as providers increased success (36,37,39). A 2013 meta-regression of 162 randomized
trials found however that CDSS were less likely to be effective when the CDSS was integrated into the
EMR or order entry system (odds ratio (OR) 0.37, 95% confidence interval 0.17 to 0.80) (244). They
noted that their findings conflicted with others, but stated that it was a robust finding based on modeling
techniques not used in the previous studies. They provided possible explanations including competing
alerts, over-sensitivity of alerts and alert fatigue in integrated CDSSs. Interestingly this meta-regression
also found that CDSSs were much more likely to be successful if the system required clinicians to provide
a reason for over-riding an alert (OR 11.23, 1.98 to 63.72), a feature of integrated CDSSs. This meta-
regression also reports that success is more likely when the system provided advice to patients as well as
clinicians (OR 2.77, 1.07 to 7.17); and when the system developers conducted the evaluation (OR 4.35,
1.66 to 11.44). Two more recent systematic reviews indicate that integration leads to improvements in
outcomes. A 2014 systematic review that only included integrated CDSSs, found a reduction in morbidity
(30). A 2015 systematic review of CDSSs for antibiotic prescribing in primary care found that integration
and automatic activation appeared to increase success (39). In summary, systematic reviews consistently
support automatic activation, requiring a reason for over-ride, and advice to patients as well as providers
as leading to better outcomes. Most also support integration of the CDSS into the EMR. The systematic
reviews also demonstrate that when evaluators are the developers, outcomes are better.
2.5.4 Implementation Issues
CDSSs are also plagued by implementation problems. Even when organizations adopt and implement
CDSSs, normalization of CDSSs (integration of the CDSS into health care processes) remains low (40–
43). Clinicians frequently ignore the CDSS and find ways to circumvent its use processes (43–46). This
may be in part due to an overwhelming number of alerts (246). An American study found that PCPs
received an average of 56.4 alerts per day and spent an average of 49 minutes per day processing them
(247). Additional barriers to uptake of CDSSs include problems with usability; a lack of training,
support, fit and integration into work processes; and concerns from providers about loss of autonomy and
medical-legal repercussions (41,42,248–252). Factors that lead to implementation success have been
poorly studied (245):
“We found that despite support from policymakers, there was relatively little empirical evidence
to substantiate many of the claims made in relation to these technologies. Whether the success of
26
those relatively few solutions identified to improve quality and safety would continue if these were
deployed beyond the contexts in which they were originally developed, has yet to be established.
Importantly, best practice guidelines in effective development and deployment strategies are
lacking.”
2.5.5 CDSSs for Opioid Prescribing for CNCP
Some organizations and researchers have created and evaluated CDSSs specifically for opioid prescribing
for CNCP. Some of these have been tested in clinical settings and others are still in the development
stages (237,239). CDSS types include alerts, protocols and dashboards. The researchers theorize that the
CDSSs may lead to more appropriate prescribing by encouraging reduced opioid doses, use of urine drug
testing and opioid prescribing contracts. This in turn may reduce harms and lead to better pain and
function outcomes. We review the studies tested in clinical settings in chapter 3. Other organizations are
also proposing, creating and implementing CDSS for safer opioid prescribing for CNCP (218,253–259).
2.5.6 Prescription Drug Monitoring Programs (PDMPs)
One common type of CDSS for opioid prescribing in use today are prescription drug monitoring
programs (PDMPs) (also called prescription monitoring programs (PMPs)) that contain patient
information on opioid prescriptions. Some jurisdictions (U.S. states, provinces in Canada, and countries
internationally) have made the PDMPs searchable and available to prescribers and pharmacists in real-
time (260,261). Use of the PDMP by prescribers is meant to influence their opioid prescribing behavior
through patient-specific, point-of-care information. The goals of PDMPs are to improve prescribing and
treatment decisions; reduce diversion and serve as a tool for law enforcement; and reduce harms
(260,261). American guidelines for opioid prescribing for CNCP recommend that prescribers check the
PDMPs prior to prescribing opioids, particularly for a new patient (12). Rates of use by prescribers,
however, are often low (262). As a result, some states have mandated use of PDMPs prior to prescribing
for a new patient or for dose changes (263). However, impact on opioid prescribing is not clear (255). A
systematic review found that mandatory review may lead to more of a positive association with reduction
in opioid-related mortality, but the evidence was of “low-strength” (264). Few studies have evaluated the
impact of PDMPs specifically on the primary care prescribing of opioids for patients with CNCP. One
study conducted in the emergency department found that use of a PDMP altered opioid prescribing
behaviour for chronic pain (265). We examine studies on PDMP CDSSs in primary care settings in
further detail in chapter 3.
27
2.5.7 CDSSs in Primary Care Settings
Many of the major systematic reviews on CDSSs include a number of primary care studies (27,35). A
systematic review with 17 studies on CDSSs in primary care settings reported positive or variable results
in 76% and no significant effect in 24% (266). Study outcomes were mainly prescribing outcomes, and
adherence to disease screening and management guidelines. A systematic review on CDSS for
appropriate antibiotic prescribing in primary care found that in five of the seven trials the CDSSs were
marginally to moderately effective at improving antibiotic prescribing behavior (39). A systematic review
of e-health interventions to improve medication safety in primary care, included five studies on CDSSs
and found that CDSSs appeared to be effective if they targeted a limited set of potentially inappropriate
drugs (267). Additionally, the authors reported that CDSSs that targeted initiation were more effective
than those targeting discontinuation. Primary care settings may have additional barriers to implementation
and normalization of CDSSs that have received little attention:
“Most barrier studies have focused on CDSSs that are aimed at a limited number of decision
points… rather than on multiple-domain covering CDSSs targeting multiple groups of users.”
(268)
Providers in primary care settings care for patients with a variety of health issues. As a result they receive
multiple alerts across many areas of care all competing for attention (269). Additionally, many PCPs
work in private clinics in Canada and the U.S. that are not part of larger networks, or even when they are,
may not share the same EMR (270–274). Diffusion, uptake and normalization of a CDSS may be more
difficult to achieve than in a centralized academic setting.
2.5.8 Summary
CDSSs appear to have promise, but also significant limitations and potential risks. Their use and impact on
opioid prescribing for CNCP in primary care setting is not well-understood. To gain a better understanding
of the extent, and range of current research on CDSSs for opioid prescribing for CNCP in primary care
clinical, we conducted a scoping review (chapter 3). In the next chapter, I explore current guidance for the
development and evaluation of complex interventions.
28
2.6 Evaluation of Complex Interventions
2.6.1 Introduction
CDSS are complex interventions. Therefore it can be challenging to determine why a CDSS worked or
failed, to identify the essential components and causal pathways, as well as the impact of the
implementation approach and the context (47). In this section I describe complex interventions and
review current guidance for development and evaluation including the use of process evaluations and
theory.
2.6.2 Description of Complex Interventions
Complex interventions contain multiple interacting parts and causal pathways and are “characterized by
unpredictability, emergence and non-linear outcomes” (275). The outcomes can be the result of recursive
causality (reinforcing loops) and “disproportionate relationships (where at critical levels, a small change
can make a big difference — a `tipping point')” (276). We often think about a complex intervention as
one containing multiple components. However, the interaction between the intervention and the context is
what often makes an intervention complex (276–278). Typically, this is an intervention that seeks to alter
functioning of a complex adaptive system (e.g. a hospital or primary care clinic). There is a myriad of
contextual factors (work processes, patient populations, physician attitudes, and managerial support) that
can lead to unpredictable outcomes. Outcomes also depend on implementation factors, like information
technology support, promotion by local champions and education sessions. As a result, adding a CDSS,
even a simple one, to a primary care clinic would be considered a complex intervention.
Table 2.2 What makes an intervention complex? (277)*
Number of interacting components within the experimental and control interventions
Number and difficulty of behaviours required by those delivering or receiving the
intervention
Number of groups or organisational levels targeted by the intervention
Number and variability of outcomes
Degree of flexibility or tailoring of the intervention permitted
* Reproduced with permission
29
2.6.3 Evaluation of Complex Interventions
RCTs are the gold standard in determining the effectiveness of an intervention. However, RCTs have
some significant limitations when used to evaluate a complex intervention. RCTs (except those with a
pragmatic design (279)) are conducted in a highly controlled environment to ensure internal validity (i.e.
results that are accurate and reproducible). These RCTs can only tell us if a complex intervention works
or fails in this specific environment and not if results can be translated to settings outside of the study
environment (275). Additionally, RCTs, including ones with a pragmatic design, can only explore a
limited number of variables at a time, and therefore, without hundreds or thousands of trials, cannot help
us understand why a complex intervention worked or failed. For example, failure could be due to the
CDSS itself, the implementation process, or factors in the particular context (29). Success could be due to
one or all of the intervention components or a particular interaction between some or all components. As
stated in the MRC guidance document “if only aggregate outcomes are presented, all we can know is
whether an intervention package did more good than harm, in terms of pre-specified outcomes, in a
specific context” (275).
2.6.4 Current Guidance for the Evaluation of Complex Interventions
The Medical Research Council in the United Kingdom (UK), therefore, recommends researchers evaluate
a complex intervention through a carefully staged, series of exploratory studies targeting key uncertainties
and then conduct a definitive evaluation (277). These phases may not occur in linear manner and the
process is often iterative (47). Moore and colleagues define exploratory studies as
…studies intended to generate evidence needed to decide whether and how to proceed with a full-
scale effectiveness study. They do this by optimising or assessing the feasibility of the intervention
and/or evaluation design that the effectiveness study would use. (280)
(Moore and colleagues state that this definition of exploratory studies includes the terms pilot and
feasibility testing or studies.) In the development phase, researchers look for evidence that an intervention
might succeed. This often comes from empirical evidence in the literature or from theory (277). They
model processes and outcomes. In the exploratory stage, researchers conduct a series of pilot and
feasibility studies testing key uncertainties in the intervention. They seek to estimate sample sizes and the
feasibility of recruitment and retention. This can be done with simulations, preliminary surveys, focus
groups or interviews. The data are used to modify the intervention and plan to implementation process. It
is also used to determine what outcome factors to measure including patient health outcomes and system
outcomes as well as determine costs and feasibility. It may be appropriate to consider a variety of
30
outcomes as well as test for unintended consequences (47). Unintended consequences are common
because of the unpredictability, feedback loops and disproportionate relationships. For example, a recent
systematic review reported that in one study, overdose deaths from heroin increased after the introduction
of a PDMP (264). Although it is not clear if the relationship was causal, the authors hypothesized the use
of the PDMP may have led restricted access to prescription opioids and led patients to seek out illicit
opioids, a far riskier option (174–176). In addition to helping answer how and why an intervention
worked, use of exploratory studies will also conserve resources (280). Large evaluative trials are often
expensive and time-consuming, so it is essential to optimize an intervention before evaluating it in a
definitive trial. In the evaluation phase, researchers initially conduct small studies to test effectiveness and
cost-effectiveness. This is often done with different versions of the intervention or variations on dose and
intensity or other implementation factors. The data are used to modify the intervention and plan the
implementation process for an RCT. Researchers then run the definitive RCT (if appropriate). For many
interventions, a cluster RCT is the best approach, as complex interventions often cannot be randomized at
the patient level. In some cases, an RCT may not be feasible and other approaches to gaining knowledge,
such as time-series analyses and simulations, along with qualitative methods are preferred (47,281). In
the implementation phase, researchers seek wide spread uptake and normalization of the intervention.
This stage also includes ongoing surveillance, monitoring and adjusting of the intervention; particularly if
the intervention is implemented in new contexts or if the context changes. There is a move to integrate
effectiveness and implementation stages to allow for evaluation of multiple implementation approaches
(282).
2.6.5 Process Evaluations
The UK Medical Research Council also recommends that all stages of the stepped evaluation should
include process evaluations (275). Process evaluations assist in identifying why an intervention worked
or did not work; and ensuring success when translating to new environments by identifying key
components and mechanisms of the CDSS, as well as determining the impact of context and
implementation process (275). Currently there is no internationally accepted methodology for conducting
a process evaluation. Two well-cited sources that provide guidance are Linnan and Steckler’s text
“Process Evaluation for Public Health Interventions and Research” and the Saunders and colleagues
article “Developing a Process-Evaluation Plan for Assessing Health Promotion Program
Implementation: A How-To Guide” (283,284). However, the UK Medical Research Council recognized a
need for further guidance on process evaluations (285). In particular there was a need for guidance on the
integration of process evaluations in all stages of the design, implementation and evaluation of a complex
31
intervention, not just in the evaluation stage. In 2014, they produced a document called “Process
Evaluations of Complex Interventions: UK Medical Research Council (MRC) Guidance” (275). The
focus is on public health interventions, but—according to authors—also relevant to other domains. The
accompanying summary of the 2015 BMJ article has been cited over 900 times (285).
2.6.6 Process Evaluation Terminology
Implementation: Implementation is defined as “the structures, resources and processes” through which
an intervention is delivered (275). The current research emphasis is on understanding how
implementation occurred. This may include mechanisms like training, peer support, local champions and
communication links. Assessment of implementation also includes a measurement of the quality and
quantity of what was delivered (275,283,284,286). Quality or fidelity usually defined as the extent to
which the intervention was delivered as planned. Quantity is about reach (what proportion of the target
population came into contact with the intervention) and dose. There are several commonly used
frameworks that assist with the assessment of implementation quality and quantity (287,288).
Context: Context is anything external to the implementation that may affect the outcome through an
effect on the intervention or the implementation (275). This includes wider social, political and economic
considerations (284). These are often referred to as barriers and facilitators. Not only does context affect
implementation and intervention, intervention will also affect the context (47). This may lead to feedback
loops with unpredictable and emergent outcomes. For example, a CDSS that advises PCPs to conduct a
urine drug test on patients who are taking opioids for CNCP may lead to the increased identification of
people with OUD. This could lead the clinic to arrange services to treat people with OUD on site (a
change in the context). The interactions between the PCPs and those caring for people with OUD, could
lead to more identification and treatment of those with OUD and eventually lower overdose death rate.
This is a feedback loop with an emergent outcome.
Mechanisms of impact: The mechanism of impact is how the intervention and its implementation impact
the context and lead to change. The mechanisms can be very difficult to determine because of multiple
causal pathways in a complex intervention. However, understanding the mechanism of impact allows
researchers to identify the essential components of the intervention. This will allow for the right balance
between fidelity and adaption to new contexts (286).
32
2.6.7 Conducting a Process Evaluation
The MRC guidance document breaks down the guidance into four sections: planning, design and conduct,
analysis and reporting (275). In the planning stage the process evaluators should define the relationship
with the intervention developers or implementers to have an effective working relationship balanced with
independence. The process evaluators should ensure their team has the correct expertise and should define
their relationship with outcome evaluation team so the integration plan is defined from the start. In the
design and conducting stage, the process evaluators should describe the intervention, implementation,
causal pathways, and outcomes. They should identify uncertainties and key research questions to address.
They should select appropriate methods for evaluation. Quantitative methods should quantify process
variables, such as delivery dose and fidelity. Quantitative methods can also assess the impact of key
variables, different contexts and implementation approaches; and determine the active ingredients.
Qualitative methods can address satisfaction, illuminate possible causal pathways, identify barriers, and
create new theory. In the data analysis phase, the researchers should combine and report on the
quantitative and qualitative data. The process and outcome data should be integrated. In the reporting
stage, evaluators should use appropriate reporting guidance for the study design.
Table 2.3 Key recommendations for process evaluations (285)*
Planning
Carefully define the parameters of relationships with intervention developers or implementers
Balance the need for sufficiently good working relationships to allow close observation, against
the need to remain credible as independent evaluators
Agree whether evaluators will take an active role in communicating findings as they emerge
(and helping correct implementation challenges) or have a more passive role
Ensure that the research team has the correct expertise. This may require:
Expertise in qualitative and quantitative research methods
Appropriate interdisciplinary theoretical expertise
Decide the degree of separation or integration between process and outcome evaluation teams
Ensure effective oversight by a principal investigator who values all evaluation components
Develop good communication systems to minimize duplication and conflict between process
and outcomes evaluations
Ensure that plans for integration of process and outcome data are agreed from the outset
33
Design and conduct
Clearly describe the intervention and clarify causal assumptions (in relation to how it will be
implemented, and the mechanisms through which it will produce change, in a specific context)
Identify key uncertainties and systematically select the most important questions to address
Identify potential questions by considering the assumptions represented by the intervention
Agree on scientific and policy priority questions by considering the evidence for intervention
assumptions and consulting the evaluation team and policy or practice stakeholders
Identify previous process evaluations of similar interventions and consider whether it is
appropriate to replicate aspects of them and build on their findings
Select a combination of methods appropriate to the research questions:
Use quantitative methods to measure key process variables and allow testing of pre-
hypothesized mechanisms of impact and contextual moderators
Use qualitative methods to capture emerging changes in implementation, experiences of the
intervention and unanticipated or complex causal pathways, and to generate new theory
Balance collection of data on key process variables from all sites or participants with detailed
data from smaller, purposively selected samples
Consider data collection at multiple time points to capture changes to the intervention over time
Analysis
Provide descriptive quantitative information on fidelity, dose, and reach
Consider more detailed modelling of variations between participants or sites in terms of factors
such as fidelity or reach (eg, are there socioeconomic biases in who received the intervention?)
Integrate quantitative process data into outcomes datasets to examine whether effects differ by
implementation or pre-specified contextual moderators, and test hypothesized mediators
Collect and analyze qualitative data iteratively so that themes that emerge in early interviews can
be explored in later ones
Ensure that quantitative and qualitative analyses build upon one another (eg, qualitative data used
to explain quantitative findings or quantitative data used to test hypotheses generated by qualitative
data)
Where possible, initially analyze and report process data before trial outcomes are known to avoid
biased interpretation
Transparently report whether process data are being used to generate hypotheses (analysis blind to
trial outcomes), or for post-hoc explanation (analysis after trial outcomes are known)
Reporting
Identify existing reporting guidance specific to the methods adopted
Report the logic model or intervention theory and clarify how it was used to guide selection of
research questions and methods
Disseminate findings to policy and practice stakeholders
If multiple journal articles are published from the same process evaluation ensure that each article
makes clear its context within the evaluation as a whole:
Publish a full report comprising all evaluation components or a protocol paper describing the whole
evaluation, to which reference should be made in all articles
Emphasize contributions to intervention theory or methods development to enhance interest to a
readership beyond the specific intervention in question
*Reproduced under Creative Commons Licence
34
2.6.8 Role of Theory in Process Evaluations
The guidance document also recommends incorporating the use of theory (275). When developing and
evaluating an intervention, researchers often create a logic model to explain the causes of a problem
within a context, possible solutions, mechanism of action of solutions (275,289). The causal assumptions
may come from clinical experience, common sense, literature and theoretical models in medicine and
other disciplines. Simple logic models, however, have significant limitations. They rarely reflect the
multiple causal pathways and are often unable to explicate outcomes of a complex intervention (289). For
example, many models do not describe the recursive loops and the tipping points that lead to emergent
outcomes (276). Additionally, even if logic models incorporate these loops, the effects and emergent
outcomes are rarely predictable. Perhaps more importantly, when researchers rely solely on their own
logic model, they are missing the opportunity to build on existing theory. Theory allows researchers to
use previous work to determine the multiple possible causal pathways and feedback loops and to identify
possible barriers and facilitators to implementation. Researchers can also add to what is already known,
refining existing theory by determining what is generalizable across contexts (290–293). However,
researchers should ensure that the theory fits the context and the intervention. For example, individual
behaviour change theory is unlikely to explicate the embedding of a complex intervention in a health care
system (275,294).
There are a large number of theories, models and frameworks that seek to explain how complex
interventions achieve, or do not achieve, outcomes. Nilsen defines implementation theories, models and
frameworks in his 2015 article “Making sense of implementation theories, models and frameworks.” A
theory provides a system of principles, hypotheses and relationships that provide an explanation for a
phenomenon or set of phenomena (295). A model tends to be more descriptive than explanatory and
often has a narrower, context-specific focus. A framework provides a structure to describe phenomena.
One well-known theory is the “Diffusion of Innovations” theory (Rogers 2003). It was developed by
Everett Rogers in the 1950s to explain how new technologies and ideas spread. The theory states there are
four factors that affect the spread and ongoing adoption: the innovation, communication channels, time
and the social system. Greenhalgh and colleagues modified the theory into a model for healthcare settings
(297). Adoption however, is not the end point in health care settings. The end point is understanding what
leads to use or embedding within the complex system (298,299). The Actor-Network theory is another
widely used theory to explain how infrastructure and relationships influence the development of
technological advances (300). It assigns a role of “actor”, and gives agency to, humans and non-human
entities in the network. The network is fluid and constantly shifting. If a new actor is introduced into the
35
system, such as a health care innovation, the whole system shifts and adjusts. The theory may be more
helpful in providing a description than in identifying barriers, causal mechanisms and outcomes (301–
303). The Technology Acceptance Model (TAM)—with origins in information technology—is a theory
commonly to predict and explain end-users response and use to heath care technologies (298). The TAM
focuses on the individuals and how they perceive the new technology. This allows for an assessment
intention to use, predicting actual system use. Damschroder and colleagues created a meta-theoretical
framework to address the large number of competing implementation theories with overlapping constructs
and differing terminology (290). They used Greenhalgh’s conceptual model of the factors affecting
diffusion, dissemination and implementation as a starting point (297). They state that the framework fills
in gaps, provides consistent terminology and an overarching typology. The framework is limited by the
underlying theories that focus on “what works” but provide little insight into why (290). The NPT is a
theory that seeks to explain how new technologies become imbedded in practice (303). It does not
address diffusion or adoption. The NPT examines the collection of social actions that lead to the
normalization of new technologies. The theory developers state that:
“This distinguishes it from theories of the cultural transmission of innovations such as Diffusion
of Innovations Theory that seek to explain how innovations spread; theories of collective and
individual learning and expertise that seek to explain how innovations are internalized; and
theories of the relationships between individual attitudes and intentions and behavioural
outcomes.” (303)
Importantly, the NPT can be used to understand why an intervention succeeded or failed and to determine
its mechanism of action. It seeks to answer the “why” and “how” not just “what worked” (303–305). I
review the NPT in further detail in the next section, including the NPT development and use of the NPT
in practice.
2.6.9 Use of Theory to Guide the Implementation Process of CDSSs
Some evaluations of CDSSs incorporate theory into the process evaluations (306). However, there is no
consensus on what framework or theory should underpin the process evaluation in particular settings. A
recent review on process evaluations of CDSSs found that in the 16 studies (including three in primary
care) that met their inclusion criteria, researchers used 15 different theoretical frameworks, some of which
were hybrids of different theories and concepts (307). The most frequently used theory was the
Technology Assessment Model. Another recent systematic review looked more broadly at health
information system evaluation frameworks and theories (308). The authors identified 20 different
36
frameworks and theories. The authors reported there was a lack of consistency in the approach to
evaluation and many did not adequately address context.
2.6.10 Summary
Developers and evaluators of CDSSs for opioid prescribing for CNCP in primary care settings are likely
to benefit from following guidance for complex interventions including process evaluations and use of
theory. In our scoping review study in chapter 2, we assess if they are following this guidance. In the next
section I review one theory in more detail, the NPT, and provide justification for using it to guide our
study describing the normalization process of a specific CDSS for opioid prescribing for CNCP (chapter
3).
2.7 The Normalization Process Theory (NPT)
2.7.1 Introduction
The NPT seeks to explain how new technologies become embedded in everyday practice—the
normalization process (303). It has been used successfully to guide the development, implementation and
evaluation of new technologies (305). In the last section I introduced several theories that have been used
to underpin process evaluations. In this chapter, I will review one of the theories, the NPT, in more depth.
I will also describe how the NPT has been used in practice, including in the evaluation of CDSSs.
2.7.2 Theory
Theory is defined as a system of principles, hypotheses, and relationships to describe and explain a
phenomenon or set of phenomena (295). It is a well-tested system that can be used to provide a clear
explanation of how the relationships lead to outcomes. Sociological theory focuses on the social context
of human interactions phenomena (295). Theories can range from a concise explanation of a simple
social phenomena to a large complex hypothesis about how the world functions. The latter is often called
“grand theory” (309). Most theory used today in health care is “middle range theory” as defined by
Robert Merton (309). Middle-range theory is constructed using empirical data to provide an explanation
for a social phenomenon. It does not attempt to find an overarching explanation that unites multiple
phenomena and explains how society functions. Middle range theory provides a deeper understanding of
phenomena through description and systematic explanation and allows the prediction of future events
(knowledge claims) (304,309). The NPT is a middle range theory.
37
2.7.3 Development of the NPT
Carl May and colleagues developed the NPT between 1998 and 2008 (310). NPT arose from the need to
understand why widely supported, diffused and adopted interventions did not become normalized. For
example, telemedicine is a widely diffused and adopted innovation, however, it has failed to be imbedded
in practice (311). May and colleagues empirically derived the NPT from implementation studies rather
than from constructs in other theories (312). In the first step, data from implementation studies were
analyzed to create a set of generalizations about the normalization of healthcare innovations. These
generalizations were observational, not explanatory and were often context-specific. In the next step, the
team used grounded theory-building techniques developed by Glaser and Straus (313) to create the
Normalization Process Model (310). Grounded theory constructs new theory through the systematic
collection and inductive analysis of data. Researchers use a constant comparative method to achieve
saturation and verification of concepts. They create theory from the data through sorting of theoretical
concepts (314). May and colleagues’ goal was to identify and explain factors that promoted or inhibited
the collective action that led to normalization of interventions (310). They tested the model in trials to
assess its usefulness as an analytic tool and found it helped explain the factors that promoted or inhibited
collective action (310). In the final step, May and colleagues created the NPT from the normalization
process model. They identified generic properties as well as causal pathways in the phenomena that were
not context specific. As a result, the NPT describes the normalization process and provides a systematic
explanation the processes of implementation and of causal mechanisms. It can be used to understand the
normalization of new technologies (310).
2.7.4 Goals of the NPT
The NPT seeks to provide an understanding of the processes that assist in or impede the normalization of
a health care intervention, as well as how the intervention achieves change (mechanism of action) (310).
The NPT focuses on three main problems:
“ 1) Implementation, by which we mean the social organization of bringing a practice or
practices into action.
2) Embedding, by which we mean the processes through which a practice or practices become,
(or do not become), routinely incorporated in everyday work of individuals and groups.
3) Integration, by which we mean the processes by which a practice or practices are reproduced
and sustained among the social matrices of an organization or institution” (310).
38
Normalization is not a given. An innovation may be adopted by an organization, but may not become
imbedded in practice. It may also be rejected—individuals find ways to avoid or circumvent use (315).
Additionally, normalization does not mean effectiveness or good quality (315). For example, providers
may use a new electronic system to refer patients to appropriate resources, but long wait-lists mean
patients do not get a timely consultation.
2.7.5 Description of the NPT
The NPT is a theory that focuses on what people and groups of people do, not what they think or intend to
do (305). The NPT postulates that innovations become part of practice through the collective action of
individuals and groups (304,310). The work is enabled or inhibited by “generative mechanisms.” The
collective social action of maintaining an innovation requires ongoing investment by participants. The
theory contains three components: the actors who are the individuals and groups of people in a settings;
the objects which are the means by which knowledge and practice are enacted (examples include
procedures, protocols and EMRs); the context which includes the social structures that enable or constrain
the activities that lead to change. The collective action occurs through interaction between these
components.
The theory consists of four constructs: the “generative mechanisms” (or agents) of the normalization
process: coherence (sense-making work), cognitive participation (relationship work), collective action
(enacting work) and reflexive monitoring (appraisal work) (304). These mechanisms can be constrained
or enhanced by the social norms and conventions. The constructs are interdependent and occur
simultaneously. Coherence or sense-making work defines the components of an innovation. It includes an
understanding of how the innovation differs from existing practice; an understanding of the objective of
the innovation and value of the innovation; an understanding of what this means for the work of
individuals. Collective action or relationship work defines the work needed to enact an innovation. This
includes the impact on roles and responsibilities including need for training; organizational support for the
work; impact on operational problems; and finally, confidence in the innovation. Cognitive participation
or enacting work defines the people involved in the innovation. This includes key individuals to drive the
process, buy-in to the innovation, and individuals’ ability to sustain involvement. Reflexive monitoring
defines the assessment of the outcomes of the innovation. This includes an understanding of the effects of
the innovation and an appraisal of the impact of the innovation on practice and reconfiguration to change
how the innovation is enacted.
39
2.7.6 NPT in Practice
The NPT can be used to assist in the design, implementation and evaluation of new innovations (292).
Researchers can use it to understand causal mechanisms; define research questions; identify barriers and
facilitators to implementation; plan data collection and analysis; and explicate outcomes (305). May and
colleagues have created a number of tools to assist with the use of the NPT in practice. These include a
set of questions that can be used to direct qualitative research on e-health implementation (316) and a
toolkit to support researchers in planning the implementation process (317).
McEvoy and colleagues conducted a systematic review with a qualitative analysis to determine where and
how the NPT was being operationalized (291). They found 29 articles that met the inclusion criteria and
analyzed them using a framework analysis approach. In a framework analysis, predefined codes and
categories are used to conduct a qualitative analysis, a deductive approach (318). They found that the
NPT constructs were stable (researchers were able to apply and use them without difficulty) across a
variety of health care settings. They recommend that researchers justify the use of the NPT and include
multiple stakeholders in the analysis to assess implementation from a broad range of perspectives.
More recently, May and colleagues conducted a systematic review with a qualitative analysis to assess the
use of the NPT in research on implementation of healthcare interventions (305). The authors found 130
articles that met their inclusion criteria. The NPT was used in a broad range of study types including a
large number of e-health and telemedicine studies (19.4%). Most were pilot or feasibility studies. May
and colleagues reported that a diverse range of researchers were able to apply the NPT in a consistent way
across a variety of studies, which highlights the flexibility in the use of the NPT. Additionally, the NPT
appeared to adequately describe the implementation processes and assist in providing an explanation of
the success or failure of the interventions. Critiques mostly focused on terminology, in particular the
technical nature of the vocabulary.
2.7.7 Use of the NPT in CDSS Implementation Studies
Several studies have used the NPT to assess CDSSs. Pope and colleagues conducted an ethnographic
analysis of a CDSS designed to assist with calls to emergent and urgent care services (319). The authors
used the NPT to provide a framework for the analysis of the interviews and observations. They selected
the NPT because it examined the work of embedding of new technologies in practice within a context and
the constraints of the system. They reported that a “huge effort” was required to implement and maintain
the routine use of the technology. Henderson and Rubin evaluated an online diagnostic decision support
40
system—developed for secondary medical care— in primary care settings using the NPT (320). The
researchers conducted focus groups with the general practitioners who agreed to pilot the system at their
clinics. The researchers used content analysis and coded the data inductively. The authors stated:
“Normalization process theory (NPT) was used as a theoretical framework to assess whether and
how well the system had been embedded in everyday practice, based on the evidence available
from the focus groups and post-use survey.”
However, they did not provide information as to why they selected the NPT or detail on how it was used.
Elwyn and colleagues assessed the normalization process model (the model used to create the NPT) to see
if it was able to explain why it was so difficult to imbed decision support technologies (DSTs) in practice
(321). DTSs differ from CDSS as they are directed at patients, not providers. They found that the NPT
was able to provide insight into areas of difficulty with implementation that had received little attention in
the literature. These included division of labour and health care, as well as the health care contexts. They
recommended that the implementation of new DTSs consider the structure of the context, in particular
imbalances in knowledge and power.
2.7.8 Summary
The NPT appears to be a useful and robust theory that can assist in explaining how new technologies
become embedded in everyday practice—the normalization process (310). It has been used to assist in
describing the normalization process of CDSSs in a number of studies. Therefore, we employed it in our
study to describe how the normalization process applies to a specific CDSS for safer opioid prescribing
for CNCP in a primary care setting (chapter 4).
41
Clinical Decision Support Systems for Opioid
Prescribing for Chronic Non-cancer Pain in Primary
Care Settings: a Scoping Review
3.1 Focused Introduction
Although opioid prescribing has leveled off in the last five years, it remains about three times higher than
in the 1990s (2,3,141,142). Prescribed opioids do not improve pain and function outcomes and place
people at risk for harms including overdoses and deaths (6–9). Prescription opioids also spawned the
current opioid crisis (10,11). To address the individual and population level harms, the 2016 CDC
Guideline for Prescribing Opioids for Chronic Pain and the 2017 Canadian Guideline for Opioids for
Chronic Non-Cancer Pain released guidelines with recommendations for providers about opioid
prescribing for CNCP (1,12).
CDSSs may assist providers in adhering to these opioid prescribing guidelines. CDSSs have a modest
impact on process outcomes, such as safer prescribing (27–35). The impact on patient outcomes is less
clear with mixed outcomes and low quality of evidence (27–30). CDSSs, however, often fail to become
integrated into healthcare processes (40–46) Additionally, since CDSSs are complex interventions, they
can be difficult to evaluate as the factors and pathways that led to success or failure in a particular setting
are not always clear (47,277). The Medical Research Council in the United Kingdom (UK) recommends
researchers evaluate a complex intervention through a carefully staged, series of exploratory studies and
then a definitive evaluation (47,277).The phases should all include process evaluations and, because of
the unpredictability, include assessments for unintended consequences (322).
Evidence from RCTs and systematic reviews indicates that some factors improve the success of a CDSS:
requiring clinicians to provide a reason for an over-ride; providing advice to patients as well as clinicians;
automatic activation, and integrating the CDSS into the EMR (30,36,37,39,244) (although one meta-
regression contradicts the benefit of integration (244)). Studies in which the evaluators of the CDSS are
also the CDSS developers—a conflict of interest— are more likely to show positive outcomes (36,244).
42
Researchers have evaluated CDSSs specifically for opioid prescribing for CNCP (237–239). Other
organizations and groups of researchers are also proposing, creating and implementing CDSS for safer
opioid prescribing for CNCP (218,253–259). The researchers theorize that the CDSSs may lead to more
appropriate prescribing by encouraging reduced opioid doses, use of urine drug testing and opioid
prescribing contracts. This in turn may improve pain and function outcomes and reduce harms.
Therefore, we conducted a scoping review to synthesize the evidence on CDSSs for opioid prescribing for
CNCP in primary care settings (220,221,234). Our purpose was to provide guidance to people who fund,
design, evaluate and implement CDSSs for opioid prescribing for CNCP in primary care settings. We
selected a scoping review rather than a systematic review because a preliminary assessment of the
literature revealed that this is an emerging field with heterogeneous studies. Systematic reviews are most
useful with a large number of studies with homogenous designs that permit meta-analysis (231). Scoping
reviews, however, are well-suited to synthesizing this type of data (221,235). Additionally, a scoping
review enabled us to report on a broad range of outcomes. This is particularly important for CDSSs
because they are complex interventions with outcomes that are highly dependent on implementation and
context (47). An understanding of other factors besides the primary outcome or findings will inform
researchers, clinical and policy-makers about the range of study designs, interventions and
implementation approaches across different contexts. Reporting on the broad range of outcomes also
enabled us to understand the extent to which researchers are following best evidence for CDSS
components and current guidance when developing and evaluating complex interventions.
3.2 Research Questions
Our primary research question was: ‘What is the extent and range of the current research on CDSS for
opioid analgesic prescribing for CNCP in primary care clinical settings?’ Our secondary research
question was: ‘Are researchers following best evidence for CDSS components and current guidance when
developing and evaluating CDSS for opioid prescribing for CNCP in primary care clinical settings?’
3.3 Methods
3.3.1 Overview
We conducted a scoping review using the frameworks by Arksey and O’Malley (234) and by Levac et al
(236) as described by Colquhoun et al (221) and the methods outlined by The Joanna Briggs Institute
43
(220). We followed the reporting guidelines from the PRISMA Extension for Scoping reviews
(PRISMA-ScR) (235) (Appendix 3.1 PRISMA-ScR checklist).
We created an a priori protocol using these frameworks (Appendix 3.2 Protocol Scoping Review). We
used an iterative approach to the scoping review and modified the protocol as needed during the data
collection process. This included adding a secondary research question, refining our database and our
grey literature search strategy, changing the data extraction plan (from two researchers to one researcher
conducting the extraction with results checked by other team members) and including several additional
outputs to answer our secondary research question.
3.3.2 Eligibility Criteria
Study design: To achieve our objective of understanding the extent of the evidence, we included peer
reviewed and non-peer reviewed studies. We included studies that used quantitative, qualitative and
mixed-methods methodologies. We included all systematic reviews, scoping reviews, meta-analyses,
RCTs, non-randomized trials, controlled before-after designs, interrupted time series, case-control
designs, observational study designs and all forms of qualitative studies. We excluded studies that were
non-systematic reviews, letters, opinion articles, narrative reviews, commentaries, analysis articles,
clinical practice guidelines and policy documents. We excluded reports on CDSSs that did not include an
evaluation. We also excluded abstracts that did not provide enough information or did not have a full text
publication after contacting the authors to see if more information was available (323).
Population: Our target population was PCPs (such as family physicians, emergency medicine physicians,
nurse practitioners (NPs) and primary care internists) who prescribe opioids for CNCP in primary care
settings. Primary care settings included family practice, nursing stations, primary care internal medicine
clinics and emergency departments. We included studies where other providers (e.g. patients, nurses,
therapists, pharmacists) input the patient-specific data into the CDSS provided the CDSS was used by the
study population (i.e. PCPs who prescribe opioids in primary care settings) to make a clinical decision at
the point of care. We excluded studies in primary care pediatric clinics and where PCPs worked in
secondary and tertiary settings, such as a pain clinic or addiction clinic. In cases where studies had mixed
populations of providers, we included studies where the study population was at least 50% PCPs working
in a primary care setting.
Intervention: We defined a CDSS as an electronic system that assists HCPs in making a clinical decision
by providing patient-specific data at the point-of-care (37,240,241). We included studies on CDSS that
were designed to assist PCPs in making a clinical decision about opioid prescribing for CNCP in primary
44
care settings. We excluded studies where the use of the CDSS was not specified or where it was used for
addiction, acute pain, cancer pain or palliative care. We included CDSS that were integrated into the
EMR (CDSS launches from the EMR, uses patient data from the EMR and reports output into the
patient’s chart) as well as ones that functioned independently (web-accessed on computer or handheld
device). We included PDMPs that enabled the clinician to make a point-of-care decision about opioid
prescribing. We included studies where the CDSS was part of a complex intervention. Given the
definition of CDSS, we excluded paper tools. We also excluded studies that examined CDSS for opioid
prescribing for palliative or cancer pain or for acute pain. We excluded studies where the CDSS was not
designed to have an impact on PCPs’ prescribing. For example, we excluded studies on PDMPs when
they were used to assess prescribing trends. As we were interested in CDSSs used in clinical settings, we
excluded studies that were not implemented in clinical settings, such as those implemented in simulated
clinical settings for usability testing or studies that described the development of a CDSS. We contacted
all authors of studies that reported on the development of a CDSS or that were usability studies, works in
progress or protocols to determine if follow-up studies meeting our inclusion criteria were available.
3.3.3 Search Strategy
We searched five electronic databases (MEDLINE, EMBASE, CINAHL, CENTRAL and PsycINFO)
from January 1st 2008 to April 30th 2018. We selected this date range because there have been significant
technological advances in the last ten years and many CDSS developed prior to this period are likely to
have evolved or be obsolete (324). We built a comprehensive search strategy including the concepts
“opioid,” and “clinical decision support systems” (Appendix 3.3 Medline Search Strategy). We searched
International Pharmaceutical Abstracts via OVIDSP for the last ten years for conference abstracts that
appeared to meet our inclusion criteria. We searched trial registries (ClinicalTrials.gov, World Health
Organization International Clinical Trials Registry Platform (WHO ICTRP)) for studies from January 1st
2008 to August 20th 2018. We also searched the reference lists of eligible studies for additional studies.
We used the approach recommended by the Canadian Agency for Drugs and Technologies (CADTH) to
our grey literature search (325) (Appendix 3.4 Grey literature search). We searched grey literature
databases and relevant institutional repositories and websites of organizations that are involved in health
care information technology. We also conducted an advanced Google search using our key concepts.
3.3.4 Study Selection Process
We imported our database search results into the software program “Covidence” (326). Two researchers
independently screened the abstracts and determined if they appeared to meet the inclusion criteria. Two
researchers then independently screened the full-text of all relevant articles, as well as the full-text of
45
articles identified through screening of the grey literature. SMS screened the complete set of abstracts and
full-text articles. The other reviewers (MAO, SM, AS, QW) divided up the abstracts and full-text articles.
We conducted reviewer meetings prior to screening. To ensure reliability between reviewers, we checked
the percent agreement between reviewers after each reviewer had screened 10 to 15 articles. If inter-rater
agreement was greater than 80%, reviewers continued to screen. If not, the reviewers reviewed the
inclusion and exclusion criteria and then screened another 10 to 15 to recheck agreement prior to
continuing to screen. There was only one episode at full-text stage where two reviewers had lower than
80% agreement and had to re-check agreement. After the re-check, the agreement was over 80%. We
contacted authors for more information when it appeared that the study met our inclusion criteria, but full
text was not available. When there was disagreement between reviewers that was not resolved through
discussion, a third researcher (MAO) assisted in making the final decision.
3.3.5 Data Extraction and Outputs
We created a data extraction form for quantitative data with space to record contextual and process-
oriented data. Two researchers piloted the form with three studies and then, in discussion with the team,
modified the form (Appendix 3.5 Data Extraction Form). Then one researcher, SMS, extracted the data
from the studies. Three other researchers reviewed her work (MAO, SM, SH). Data charting was an
ongoing iterative process where the reviewers continued to modify the form and the process, as needed.
We recorded the following outputs: study population and setting, a description of the intervention and
implementation process, description of the CDSS, details about the evidence-based components of the
CDSS, study aims, methodology and design, study outcomes, study funding information and conflicts of
interest (evaluators were also the developers), and adherence to guidance for complex interventions. To
determine if developers were following best evidence for the design of the CDSS we assessed for four
evidence-based components: integration into the EMR, automatic activation, requiring a reason for over-
ride and provision of advice to patients as well as providers. To determine if investigators were following
guidance for evaluating complex interventions we assessed if the study was part of a stepped approach to
development and evaluation; if there was a plan to assess for unintended consequences; if there was a
planned process evaluation; if the study included both process and outcome measures; if the study used
theory to guide implementation and/or evaluation.
3.3.6 Data Synthesis
We used the preferred reporting items for systematic reviews and meta-analyses (PRISMA-ScR) flow
diagram to report on the steps in our methods (235). We reported on the quantitative data using a tabular
format. We used the contextual and process-oriented data to write a narrative summary. We did not
46
conduct an assessment of study quality, assess for reporting bias, or risk of bias in the included articles.
This is consistent with the current framework and guidance on scoping reviews (220,221,235).
3.4 Results
3.4.1 Search Results
Our initial literature search of the electronic databases provided 8281 articles (Figure 3.1 PRISMA flow
diagram). We also identified 74 articles for full-text screening through a search of the grey literature and
studies from a search of the reference lists from eligible studies for our full text screen. After removing
duplicates, we screened the titles and abstracts of 4468 articles to determine if they met our inclusion
criteria. We included 303 articles for full-text screening and 11 were included in the scoping review.
Reasons for exclusion are in Figure 3.1 PRISMA flow diagram.
47
Figure 3.1 PRISMA Flow Diagram
From: Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group (2009). Preferred Reporting Items for Systematic Reviews and Meta-
Analyses: The PRISMA Statement. PLoS Med 6(7): e1000097. doi:10.1371/journal.pmed1000097
Formoreinformation,visitwww.prisma-statement.org.
PRISMA2009FlowDiagram
Recordsidentifiedthrough
databasesearching(n=8281)
Screen
ing
Includ
ed
Eligibility
Iden
tification
Additionalrecordsidentified
throughothersources(n=74)
Recordsafterduplicatesremoved
(n=4468)
Recordsscreened
(n=4468)
Recordsexcluded
(n=4165)
Full-textarticlesassessedforeligibility(n=303)
Full-textarticlesexcluded,withreasons(n=292)
· 67notastudy
· 186wrongintervention
· 34wrongpopulation
· 6duplicates
· 10couldnotlocatefull-text
Studiesincludedin
scopingreview(n=11)
48
3.4.2 Description of Study Settings, Population, Intervention and
Description of the CDSS
Of the studies we included in the review, five concerned CDSSs that were used locally within a specific
health centre, health system or clinic (237,327–330) and six examined CDSSs that were PDMPs: state-
run, web-based, central CDSSs (265,331–335). We displayed the results using these two typologies. All
studies were set in the United States. Of the local CDSS 4/5 were in primary care clinic settings and 1/5
were in the Emergency Department. Of the PDMP CDSS, 3/6 were in primary care clinic settings and 2/6
were in the Emergency Department and for one study the setting was not provided. There were several
different types of local CDSS: two protocols in the EMR, one dashboard on an intranet, one alert in the
EMR and one web-based clinical tool. More details on the settings, population and intervention are in
Table 3.1 and 3.3. Of the local CDSSs, 3/5 were integrated into the EMR and two of these automatically
activated. None of the studies on PDMP CDSS stated they were integrated. Additional information on the
CDSS components is found in Table 3.4. All of the studies on local CDSSs described the implementation
process. Only one study provided a description of the four training sessions and the ongoing education
sessions (327). The other studies provided little information. One study reported only that the providers
were “trained” in the use of the CDSS (328). In the other four, the authors reported that the CDSSs were
introduced at educational sessions, such as grand rounds or provider meetings, and in one case (329),
providers were also offered the option of a one-to-one meeting with a study principal investigator.
However, none of the studies provided details such as duration of sessions or training and how many (or
what percentage of) participants attended. None of the PDMP CDSS studies described implementation
process.
3.4.3 Description of Study Aims, Methodologies, Methods, Findings and
Adherence to Guidance
Of the studies examining the local CDSSs, all used quantitative methodology. Of the studies examining
the PDMP CDSSs, methodologies and methods varied. The local CDSSs studies and two of the PDMP
CDSS studies (265) aimed to determine if a CDSS led to reductions in opioid prescribing, risk scores or
improved adherence to guidelines. The local CDSS studies all reported positive findings. The two PDMP
CDSSs studies reported mixed or null findings. More details on aims, methodologies, methods and
findings are in Table 3.2. and 3.3.
The remaining studies were all on PDMP CDSSs. One PDMP CDSS study found that an educational
intervention increased use of a PDMP (332). The remaining PDMP CDSS studies aimed to determine
49
providers’ views on PDMPs, use of PDMPs and barriers and facilitators to use. The interview study
reported favourable views towards PDMPs (333); the chart review found most providers documented
accessing a PDMP (334); and the survey found that most providers reported using a PDMP before
prescribing opioids to a new patient with CNCP (335).
None of the studies stated they were a part of a stepped approach to a complex intervention, and only one
study assessed both process and outcome measures. Additional information on adherence to guidance for
development and evaluation of complex interventions is in Table 3.5.
3.4.4 Description of Funding Sources and Conflicts of Interest
Of the local CDSS, 4/5 reported on funding; none of the PDMP CDSS studies reported on funding. For
5/5 studies on the local CDSS, the developers were also the evaluators or the relationship was unclear or
not stated. No evaluators of PDMPs provided information on their relationship to the PDMP developer.
50
3.4.5 Study Tables
Table 3.1 Study overview: settings, population, intervention and description of CDSS
Study
Study setting Study population Intervention or
Independent variable
Description of
type of CDSS
Local CDSS
Anderson
2015
Primary care; United States 201 PCPs (nurses, NPs, physicians
and medical assistants)
CDSS and summary
reports
Dashboard on
intranet
Canada
2014
Primary care; United States 27 PCPs (physicians, NPs, resident
physicians)
CDSS and monetary
incentive
Protocol in EMR
Gugelmann
2013
Emergency department;
United States
* PCPs (physicians, resident
physicians, nurses, NPs)
CDSS and educational
sessions
Alert in EMR
Liebschutz
2017
Primary care; United States 53 PCPs (48 physicians, 5 NPs) Intervention: Multi-
component
Control: CDSS
Web-based
clinical tools
Patel
2018
Primary care and specialist
pain clinics; United States
*PCPs
*Pain medicine providers
CDSS
Protocol in EMR
PDMP
CDSS
Baehren
2009
Emergency department;
United States
18 PCPs (17 physicians, 1 NP) CDSS PDMP
Chaudhary
2017
Primary care; United States 168 PCPs (168 NPs) N/A PDMP
Click 2018 Primary care; United States 28 PCPs (physicians)
2 clinic directors
2 pharmacists
N/A PDMP
Coleman
2016
Primary care; United States 7 PCPs (3 NPs, 3 physicians, 1
physician assistant)
N/A PDMP
Kohlbeck
2018
Emergency department,
Physical Medicine and
Rehabilitation,
Hematology/Oncology;
United States
89 PCPs (physicians, APPs,
residents and students) (initial
survey)
**8 (focus group)
**108 (assessment survey)
**100 (evaluation survey)
**8 (follow-up survey)
Educational module PDMP
Lin 2018 Not provided; United States *PCPs (physicians)
*other prescribers
N/A PDMP
*Number not provided
**Clinician type not provided
Abbreviations: CDSS = Clinical Decision Support System; PDMP = Prescription Drug Monitoring Program; EMR
= Electronic Medical Record; PCPs = Primary Care Provider; NP = Nurse Practitioner; APP = Advance Practice
Provider; N/A = Not Applicable
51
Table 3.2 Study overview: aims, design and summary of relevant findings
Citation Study aim or question Study design Summary of findings related to CDSS
Local
CDSSs
Anderson
2015
“To evaluate the impact of a clinical
dashboard for opioid analgesic
management on opioid prescribing
and adherence to opioid practice
guidelines in primary care.”
Quantitative:
pre-post
“Adherence to several opioid prescribing guidelines
improved in the postimplementation year compared
with the pre-implementation year: (1) the proportions
of COT patients with a signed opioid treatment
agreement and urine drug testing increased from 49%
to 63% and 66% to 86%, respectively. The proportion
of COT patients with a documented assessment of
functional status increased from 33% to 46% and
those with a behavioral health visit increased from
24% to 28%. However, there was a small decline in
the proportion of patients prescribed COT from 3.4%
to 3.1%.”
Canada
2014
“Our objective was to evaluate
provider adherence to this protocol,
attitudes toward the management of
these patients, and knowledge of
opioid prescribing.”
Quantitative:
pre-post and
survey
“Provider adherence to the protocol significantly
improved measured outcomes. The number of UDSs
ordered increased by 145%, and the diagnosis of
chronic pain on the problem list increased by 424%.
There was a statistically significant improvement in
providers’ role adequacy, role support, and job
satisfaction/role-related self-esteem when working
with patients taking opioids. In addition, provider
knowledge of proper management of these patients
improved significantly. Eighty-nine percent of our
physicians attained the monetary incentive.”
Gugelmann
2013
“Can a staged, multidisciplinary
educational and computerized
physician order entry (CPOE)-based
intervention decrease opioid
discharge pack use in patients
treated and released from the ED
and especially in patients at risk for
dependence?”
Quantitative:
pre-post
“There was a significant reduction in the number of
opioid discharge packs ordered in the post-
intervention period […] from 23.7% to 15.1% among
patients with a chronic pain condition.”
Liebschutz
2017
“To determine whether a
multicomponent intervention […]
improves guideline adherence while
decreasing opioid misuse risk.”
Quantitative:
cluster
randomized
control trial
“At 1 year, intervention patients were more likely than
controls to receive guideline-concordant care (65.9%
vs 37.8%; P < .001; adjusted odds ratio [AOR], 6.0;
95% CI, 3.6-10.2), to have a patient-PCC agreement
(of the 376 without an agreement at baseline, 53.8% vs
6.0%; P < .001; AOR, 11.9; 95% CI, 4.4-32.2), and to
undergo at least 1 UDT (74.6% vs 57.9%; P < .001;
AOR, 3.0; 95% CI, 1.8-5.0). There was no difference
in odds of early refill receipt between groups (20.7%
vs 20.1%; AOR, 1.1; 95% CI, 0.7-1.8). Intervention
patients were more likely than controls to have either
a 10% dose reduction or opioid treatment
discontinuation (AOR, 1.6; 95% CI, 1.3-2.1; P < .001).
In adjusted analyses, intervention patients had a mean
(SE) MEDD 6.8 (1.6) mg lower than controls
(P < .001).”
52
Patel 2018 “To determine COT-CR impact on
reducing morphine equivalent
monthly dose (MEMD) and risk
index for overdose or serious
prescription opioid-induced
respiratory depression (RIOSORD)
values in veterans receiving chronic
opioids.”
Quantitative:
cohort
“After matching, 3801 patients were included in the
complete and incomplete COT-CR groups,
respectively. Greater average reduction in MEMD
(−11.6 MEMD; 95% CI = −0.97 to −22.25 MEMD;
P= 0.032) and RIOSORD index score (−0.53
RIOSORD index score; 95% CI = −1.00, −0.05
RIOSORD index score; P = 0.030) was observed in
patients with a complete COT-CR compared to
patients with an incomplete COT-CR. Differences in
RIOSORD risk class were insignificant.”
PDMP
CDSS
Baehren
2009
“To identify the influence of OARRS
data on clinical management of
emergency department (ED) patients
with painful condition.”
Quantitative:
quasi-
experimental
“After review of the OARRS data, providers changed
the clinical management in 41% (N 74) of cases. In
cases of altered management, the majority (61%; N
45) resulted in fewer or no opioid medications
prescribed than originally planned, whereas 39% (N
29) resulted in more opioid medication than previously
planned.”
Chaudhary
2017
“…to describe FNP opioid
prescription patterns and determine
the extent to which FNPs implement
specific RMPs when treating CNMP
patients.”
Quantitative:
convenience
survey
“Many of the opioid-prescribing FNPs (54.9%)
reported that they always consult state prescription
monitoring programs before starting CNMP patients
on opioid therapy. Another 29.3% said that they
consult state prescription monitoring programs for
CNMP patients stratified as high risk. Approximately
7% of respondents were unsure if their state had such
a program.”
Click 2018 “… to learn more about what factors
lead to physicians’ prescribing
control drugs for non-cancer pain
through the use of focus group.”
Qualitative:
interview study
“Prescription Drug Monitoring Programs, while a
relatively new tool, elicited mostly positive comments.
While most prescribers have welcomed the tool and
found it generally helpful, several stated that
numerous issues were preventing the PDMP from
being fully integrated into a patient’s visit.”
Coleman
2016
“…to determine if providers are
accessing the PDMP and utilizing
evidence-based guidelines to
minimize opioid pain medication
misuse among patients with chronic
pain in a primary care setting.”
Quantitative:
observational,
chart review
“The PDMP was documented to be accessed on 5 of
the 7 records (n=5; 71.4%).”
Kohlbeck
2018
“… to evaluate provider knowledge,
attitudes and behaviours regarding
the Wisconsin PDMP before and
after study interventions.”
Mixed methods:
experimental,
focus group and
surveys
“Seventy-five percent of respondents reported that
they either “completely agree” or “agree” with the
statement, “As part of my clinical practice, I check the
PDMP more consistently than I did prior [to the
educational session].”
Lin 2018 “This study examined whether
PDMP implementation and different
levels of PDMP requirements were
associated with physicians' patterns
of prescribing opioid analgesics for
patients with non-cancer chronic
pain.”
Quantitative:
observational,
cross-sectional
study
“State PDMP implementation status and requirement
levels were not associated with physician opioid
prescribing for non-cancer chronic pain treatment (p's
ranged 0.30 to 0.32).”
53
Abbreviations: CDSS = Clinical Decision Support System; PDMP = Prescription Drug Monitoring Program; COT =
chronic opioid therapy; UDS = urine drug screen; AOR = adjusted odds ratio; CI = confidence interval; CPOE =
computerized physician order entry; ED = emergency department; UDT = urine drug testing; COT-CR = study local
CDSS; MEMD = morphine equivalent monthly dose; RIOSORD = risk index for overdose or serious prescription
opioid-induced respiratory depression; OARSS = study PDMP CDSS; FNP = family nurse practitioner; RNP = risk
mitigation procedures; CNMP = chronic non-malignant pain.
Table 3.3 Summary study characteristics
Characteristic Local CDSS
Count (%)
PDMP CDSS
Count (%)
Study setting and population
Country United States
5/5 (100%) 6/6 (100%)
Practice settings Primary Care clinic
Emergency Department
Not provided
4/5 (80%)
1/5 (20%)
0/5
3/6 (50%)
2/6 (33%)
1/6 (17%)
Types of PCPs Physicians (including
resident physicians)
NPs
Other clinicians
Not described
4/5 (80%)
4/5 (80%)
2/5 (40%)
1/5 (20%)
4/6 (67%)
2/6 (33%)
1/6 (17%)
0/6
Study design
Methodology Quantitative
interventional
Quantitative
observational
Quantitative survey
Qualitative
Mixed-methods
4/5 (80%)
1/5 (20%)
0
0
0
1/6 (17%)
2/6 (33%)
1/6 (15%)
1/6 (15%)
1/6 (15%)
Funding and transparency
Funding for CDSS
development
Public/Non-profit
Industry
Both
Not sponsored
Funding not reported
4/5 (80%)
0
0
0
1/5 (20%)
0
0
0
0
6/6 (100%)
Funding for evaluation Public/Non-profit
Industry
Both
Unclear type of funding
Not sponsored
Funding not reported
3/5 (60%)
0
0
1/5
0
1/5 (20%)
3/6 (50%)
0/6
0/6
0/6
2/6 (33%)
1/6 (17%)
Relationship between
developers and evaluators
Same person, group or
organization
Different group or
organization
Not stated/not clear
3/5 (60%)
0/5
2/5 (40%)
0
0
6/6 (100%)
Abbreviations: CDSS = Clinical Decision Support System; PDMP = Prescription Drug Monitoring Program; N/A =
Not Applicable.
54
Table 3.4 Inclusion of evidence-based components*
Component Local CDSS
Count (%)
PDMP CDSS*
Count (%)
Was CDSS integrated into EMR? Yes
No
Not clear/ Not stated
3/5 (60%)
2/5 (40%)
0/5
0/3
0/3
3/3 (100%)
If integrated, did CDSS automatically activate? Yes
No
Not clear/ Not stated
2/3 (67%)
1/3 (33%)
0/3
0/3
0/3
3/3 (100%)
If integrated, did CDSS require a reason for
override?
Yes
No
Not clear/ Not stated
0/3
2/3 (66%)
1/3 (33%)
0/3
0/3
3/3 (100%)
Did CDSS include advice for patients? Yes
No
Not clear/ Not stated
0/5
4/5 (80%)
1/5 (20%)
0/3
0/3
3/3 (100%)
*We excluded 3 studies from this table because they included multiple PDMP CDSS, not a specific CDSS
(331,333,335)
Abbreviations: Clinical Decision Support System; PDMP = Prescription Drug Monitoring Program; N/A = Not
Applicable.
Table 3.5 Adherence to guidance for development and evaluation of complex interventions
Guidance Local CDSS
Count (%)
PDMP CDSS
Count (%)
Did authors state the study was part of a stepped
approach to development and evaluation?
Yes
No
0/5
5/5 (100%)
0/3
3/3 (100%)
Was there a stated plan to assess for unintended
consequences?
Yes
No
0/5
5/5 (100%)
0/3
3/3 (100%)
Did authors state they were conducting a planned process
evaluation to complement the outcome evaluation of the
CDSS in this study or linked study?
Yes
No
0/5
5/5 (100%)
0/3
3/3 (100%)
Did authors include process measure and outcome
measures in this study?
Outcomes only
Processes only
Both
3/5 (60%)
1/5 (20%)
1/5 (20%)
1/3 (33%)
2/3 (67%)
0/3
Did the study use theory to guide implementation and/or
evaluation?
Yes
No
1/5 (20%)
4/5 (80%)
0/3
3/3 (100%)
**We excluded 3 studies from this table because they included multiple PDMP CDSS, not a specific CDSS
(331,333,335)
Abbreviations: Clinical Decision Support System; PDMP = Prescription Drug Monitoring Program; N/A = Not
Applicable.
55
3.5 Discussion
We identified 11 studies that examined CDSS for opioid prescribing for CNCP in primary care clinical
settings. About half of studies examined CDSSs that were used locally in a clinic (local CDSSs) and
about half examined PDMP type of CDSS. Studies on PDMP CDSSs reported mixed outcomes or lack of
association between CDSSs and prescribing. PCPs appear to have positive attitudes towards PDMPs and
use them in practice. Studies on local CDSSs reported that use of CDSSs was associated with reductions
in prescribing and improved adherence to guidelines. However, we identified significant limitations that
indicate that these results should be interpreted with caution. The studies are few in number and did not
assess patient outcomes or look for unintended consequences. Most of the studies that assessed opioid
prescribing outcomes employed designs that tend to lead to lower quality of evidence (336): four used
pre-post design, two were observational and only one was a cluster RCT. CDSSs were often part of multi-
faceted intervention and the impact of the CDSS alone is unclear. (For example, the cluster RCT had a
multi-faceted intervention and included the CDSS in both the intervention and control groups). We also
determined that the CDSS developers do not appear to be building on research on CDSSs in other fields
of medicine. Additionally, few of the studies appeared to adhere to current guidance for development and
evaluation of complex interventions and none assessed for unintended consequences (277). And finally,
many of the evaluators of the CDSSs were also the developers. However, no study addressed this conflict
of interest (36,244).
3.6 Conclusion
In summary, there are few studies examining CDSSs for opioid prescribing for CNCP in primary care
settings. They mostly use lower quality study designs and many have conflicts of interest. None examine
patient outcomes or assess for unintended consequences. Few incorporate evidence-based components
and no studies appear to be following current guidance for development and evaluation of complex
interventions.
56
A Description of the Normalization Process of a
Clinical Decisions Support System for Safer Opioid
Prescribing for Chronic Non-cancer Pain into Primary
Care Settings: an Exploratory Qualitative Study
4.1 Focused Introduction
The CDC and the Canadian National Pain Centre have both recently released guidelines to address the
individual and population level harms from opioid prescribing for CNCP (1,12). These guidelines make a
large number of recommendations, including restricting the opioid dose, tapering patients to lower opioid
doses, encouraging patients to use non-pharmacological pain modalities, providing overdose prevention
education, prescribing naloxone, and identifying behaviours that may indicate OUD.
CDSSs may assist providers in following these guidelines. Systematic reviews show that CDSSs have an
impact on process measures like improving prescribing; their impact on patient outcomes is less clear
(27–35). Normalization of CDSSs into clinical settings, however, often remains low (40–42,337).
Furthermore, as CDSSs are complex interventions they can be difficult to evaluate (47). They have
multiple interacting parts and causal pathways and “unpredictability, emergence and non-linear
outcomes” (47,277). The implementation, intervention and contextual factors and pathways that led to
success or failure in a particular setting are not always clear. The Medical Research Council in the United
Kingdom (UK), therefore, recommends researchers evaluate a complex intervention through a carefully
staged, series of pilot studies targeting key uncertainties, exploratory studies and a definitive evaluation
(47,277). The phases should all include process evaluations (322).
Current guidance also recommends the use of an implementation theory during the development and
evaluation of a complex intervention. A theoretical underpinning may assist implementation and in
interpreting and comparing outcomes (338). The NPT (303) is an explanatory theory to assess healthcare
innovations. It goes beyond diffusion and adoption models and theories (290,296,297). It seeks to
explain how and why new technology becomes imbedded in health care processes (i.e. becomes part of
57
everyday work). This is the end-point of implementation (298,299). The NPT postulates that innovations
become part of practice through the collective action of individuals and groups (304,310). The theory
consists of four constructs: the “generative mechanisms” (or agents) of the normalization process:
coherence (sense-making work), cognitive participation (relationship work), collective action (enacting
work) and reflexive monitoring (appraisal work) (304). The constructs are interdependent and occur
simultaneously. (See chapter 2 for more detail on the NPT.) The NPT has been tested and found to be
stable across contexts with face validity and “excellent descriptive power” (303,317).
Some organizations have created and evaluated CDSS specifically for opioid prescribing for CNCP.
These studies report that the CDSS was associated with improvements in adherence to a guidelines or
improvements in opioid prescribing (237–239). However, our scoping review (see chapter 3) reveals
limitations in the studies. Additionally, none of the studies appear to have conducted a rigorous
assessment of implementation and normalization of the CDSS in practice.
4.2 Aim and Objective
Our aim in this study, therefore, was to gain a better understanding of the barriers and facilitators to the
normalization of a CDSS for opioid prescribing for CNCP into primary care settings. Our study objective
was to describe the normalization process of a specific CDSS—the PCI— for more appropriate opioid
prescribing for CNCP into primary care settings in an exploratory study.
4.3 Methods
4.3.1 Overview
In this exploratory study, we interviewed six PCPs (all physicians). They worked in Southern Ontario and
had used the CDSS in practice. We used thematic analysis (339) as well as an explanatory theory, the
NPT (304), to analyze the data. Research ethics approval was granted by the University of Toronto.
This study was part of a plan to conduct a stepped evaluation of the PCI using guidance for evaluating
complex interventions (47). Information on the development of the PCI and usability testing is available
(340). We used guidance by Moore and colleagues for process evaluations (285). Steps in the guide
include: defining the relationship with the organization that developed the intervention, identifying key
uncertainties, selecting appropriate methods, analyzing data iteratively, reporting using an implementation
theory and disseminating findings to policy and practice stakeholders (Table 2.2).
58
4.3.2 Population of Interest and Sampling Methods
Our target population was PCPs who prescribed opioids for CNCP at sites that used the EMR Telus
Practice Solutions (PSS). The PCI is only available to Telus PSS users who have tablets from the private
software company that created the PCI integrated into their patient practices. We opted to recruit from
sites that were registered with the University of Toronto Practice-Based Research Network (UTOPIAN)
(341). These 14 academic sites in Southern Ontario, most within the Greater Toronto Area, have over
1400 affiliated family physicians. The sites have the infrastructure in place to participate in practice-based
research and, with a few exceptions, are covered under the University of Toronto Research Ethics Board
(REB). UTOPIAN sites are also registered with CPCSSN, a Canadian multi-disease surveillance system
based on primary care EMR data.
Three months after the PCI was made available for download, we sent a recruitment email to all
UTOPIAN sites (n=14, with an estimated 1400 physicians) to determine if PCPs at the sites used the PCI
in practice and were interested in participating in the study. Based on past UTOPIAN research projects,
we knew that at least seven sites had the tablets and software that would allow the site to use the PCI in
practice. Three of those seven sites responded to our recruitment emails. We followed-up with a phone
call to site leads. Site PCPs who responded to the recruitment email were evaluated for eligibility through
a phone call or site visit. To be eligible, PCPs must have used the PCI at least once, be willing to
participate in an interview and agree to allow access to basic anonymized quantitative data collected by
CPCSSN: practice size, number of patients prescribed opioids, and practice location. To ensure that the
prescribing rates reflected prescribing for CNCP, we excluded PCPs who had a focused practice in
palliative care.
We planned to recruit enough PCPs to achieve thematic saturation. Thematic saturation is a concept
stemming from Grounded Theory that has been applied to thematic analysis (313,342). With thematic
saturation, subsequent data do not add additional concepts and only lead to minor modifications in the
codes and categories. To estimate the sample size we would need to reach thematic saturation, we used
the information power model by Malterud and colleagues (343). In the model, the information power of
an interview sample “is determined by items such as study aim, sample specificity, use of established
theory, quality of dialogue, and analysis strategy.” This approach is in keeping with recommendations
from other qualitative researchers (342,344,345). We therefore estimated that it would take a small
number of participants (8 to 12) to achieve thematic saturation because our study is exploratory; the study
aim is narrow (description of the normalization process of one CDSS in a specific primary care setting);
59
there is established theory on the normalization of new technologies (the NPT); and we employed a
highly-qualified qualitative interviewer with experience interviewing PCPs about opioid prescribing (LC).
We had initially planned to use theoretical sampling to achieve thematic saturation: data from already
gathered sources determines further recruitment plan and sampling (346,347). However, despite
contacting all 14 UTOPIAN sites, we were only able to recruit six participants from three sites (two from
each site). We therefore enrolled all participants who agreed to participate (convenience sampling (348)).
Thematic analysis can be conducted with convenience sampling as well, particularly for exploratory
studies (339).
4.3.3 Data Collection and Preparation
Demographic information: We collected practice demographic and study information from CPCSSN
(practice size, number of patients prescribed opioids). We recorded the participant identifier number (not
the participant’s name) on the CPCSSN data collection form and used that number to link to the
interview. For each PCP who agreed to participate, CPCSSN provided us with the number of patients
prescribed opioids in the practice and the practice size for the twelve months prior to study participation.
Interview data: SMS created the initial interview guide using the NPT for guidance (Appendix 4.1
Physician Interview Guide), which allowed us to build on previous work in the field of normalization of
new technologies. LC and FS reviewed the guide and provided feedback. LC pilot-tested the questions
with two physicians. LC is a medical anthropologist who has expertise in conducting qualitative research
on health service delivery and cultural context. She has participated in studies assessing family
physicians’ experience with opioid prescribing for CNCP. LC and SMS, with the input of FS and MAO,
modified the questions in the interview guide based on the pilot-testing to improve flow and
understandability. After obtaining consent, LC conducted the semi-structured interviews by telephone.
We continued to modify the interview guide and our approach as needed, during the data analysis process.
As described above, we were only able to recruit six PCPs in addition to the two physicians who
participated in the pilot interviews. Analysis of the data revealed that we had not achieved thematic
saturation. To enrich the data, we added a longitudinal component by conducting follow-up interviews
approximately one year later, to further explore the topics in which there was disagreement, to delve
further into the barriers to normalization and to ask about ongoing use of the PCI. The interview questions
were modified slightly for each individual physician to reflect what they said in the initial interview (e.g.
to avoid redundancy and to get as much depth as possible from those with additional expertise and
knowledge). Research ethics approval for the follow-up interviews was granted by the University of
Toronto. We interviewed all six PCPs approximately one year after their first interview.
60
4.3.4 Description of the PCI
The PCI (Appendix 4.2 Patient Check-In (PCI)) was created by the Institute for Safe Medication Practices
(ISMP) Canada, a non-profit organization that seeks the “advancement of medication safety in all health
care settings.” The funding agency was Health Canada, Drug Strategy Community Initiatives Fund. A
private, for-profit software company developed the CDSS. The intellectual content of the tool is in the
public domain. However, the PCI can only be used by sites that licence software and tablets from the for-
profit company. ISMP Canada states that the outcome goal of the PCI “is to improve adherence to
clinical practice guidelines when opioids are prescribed to patients. This may support efforts to
reduce the misuse and/or abuse of opioids in Canada” (340). ISMP Canada states that the PCI
will achieve this goal by providing a structured approach and allowing longitudinal evaluation;
improving communication and engaging patients in collaborative decision-making; allowing for
comparisons within and between practices; and permitting self-audit of opioid prescribing
practices.
The PCI is integrated into the EMR, but does not automatically identify appropriate patients.
Instead, physicians identify appropriate patients through searching their appointment list or
running a search within the EMR. Providers then flag appropriate patients within the EMR.
When flagged patients arrive for an appointment, they check-in using a tablet (the typical process
for the clinics) and the PCI automatically initializes on the tablet. The PCI collects data from the
patients while they wait to see the physician. The PCI contains several components:
1) Confirmation of opioids and benzodiazepines,
2) Question about side effects or problems,
3) Current opioid misuse measure (COMM),
4) Brief Pain Index (BPI),
5) Depression screen (PHQ9),
6) Questions about a visit to a pain specialist,
7) Other comments from patient.
The PCI scores the standardized tools, places the information into a note in the patient’s chart
and flags discussion points. The information collected is also presented in a table form and can
be tracked over time. All information can be edited within the chart. The PCI does not provide a
patient handout or advice.
61
4.3.5 Implementation of the PCI in Practice
The for-profit company that developed the PCI added it to their suite of CDSSs. It was accompanied by a
tool called the AUDIT tool that allowed PCPs to run searches to identify appropriate patients for the PCI
(Appendix 4.2 Patient Check-in (PCI)). In their monthly batch email, they notified all clinics (that used
their software and tablets) that the PCI and the AUDIT tool was available for download. The company
was not able to provide us with information on number of downloads of the CDSS to clinics. ISMP
Canada did not do any additional knowledge translation for the PCI.
4.3.6 Ethics, Privacy and Conflicts of Interest
Participation in the study was voluntary. PCPs were compensated at a rate similar to their hourly earning
potential. Their decision to participate or not, or to leave the study, was confidential and did not affect
their current or future employment or their ability to continue using resources from organizations
involved in the development of the PCI. Benefits to participants included opportunities to improve patient
care and compensation for their time. Risks included discomfort from scrutiny of their opioid prescribing
habits. PCPs may have also found the interviews emotionally intense if prescribing opioids had been a
challenging experience. Drawbacks also included time spent in the interview. The risk of harm to patients
from physician’s interviews about the CDSS is very low. It is possible that some patients who may
benefit from opioids are denied opioids. However, given the overwhelming evidence, patients are likely
to benefit from safer opioid prescribing both in improvements in pain and function, and a reduction in
harms. Data were not shared outside of the research team and staff. The interview transcript was not
associated with the interviewee’s name, but was identified only by a participant number. The interview
audio file was kept secure during transcription and was deleted after all analysis was completed and prior
to study termination. The transcriptionist signed a confidentiality agreement which was submitted to the
Research Ethics Board (REB). All computerized study information is kept on a secure server and
password-protected, and therefore is inaccessible to anyone outside of the research team. All study
information, with the exception of the audio files, will be kept for 10 years after the end of the study and
then securely destroyed. Any information that reveals a participant’s identity will not be released without
their consent, unless required by law. Responses to the interview questions do not contain identifying
information and so cannot be linked back to participants in any publication or presentation. In particular,
direct quotations do not include any identifying information or details to attribute the quotations to a
particular individual.
In this study, I was the evaluator and was a co-investigator on the grant application with ISMP Canada.
No other study members had any role in the development of the PCI. My role in the development was to
62
provide expertise in two sessions where we identified gaps in adherence to guidelines for opioid
prescribing for CNCP in primary care settings. I also participated in a usability testing session. I was not
involved in selecting the type of KT intervention (a CDSS), nor in design or creation of the CDSS. I do
not have a financial stake in the PCI or in the private company that developed the PCI. My role as
developer and evaluator does present a conflict of interest. To minimize risk of bias, another investigator
(LC) conducted the interviews. She also independently analyzed the first four interviews and checked my
work for the last two interviews.
4.3.7 Analysis
Using CPCSSN data, we reported on the provider’s basic demograpics and prescribing information (high,
medium or low prescriber) to provide context for the qualitative data. We calculated if participants were
high, medium or low prescribers in the 12 months prior to recruitment using CPCSSN data. We produced
an anonymized chart of all providers who had registered with UTOPIAN. We excluded those with a
focused practice in palliative care using billing codes for a palliative care consultation. We then listed the
remaining physicians by number of patients prescribed opioids divided by the practice size for the twelve
months prior to participation in the study. We then divided the sample into thirds: low, medium and high
prescribing. Once a PCP agreed to participate, we located the participant in the data set and determined if
the participant’s prescribing fell in the high, medium or low prescribing category.
We conducted a thematic analysis with an inductive approach to identify codes and categories and then
mapped the categories to constructs in the NPT to assist in generating themes. (This is a similar approach
to several other studies that have assessed a health innovation and used the NPT framework (349,350)).
Thematic analysis is a flexible and widely used qualitative analytic method that is not bound to a
particular theoretical and epistemological approach. As a result, it can be use within many theoretical
frameworks (339). In a thematic analysis, researchers search for repeated patterns of meaning across a
dataset. A theme is something that “captures something important in relation to the overall research
question”(339). In the first step, researchers LC and SMS familiarized themselves with the data by
reading through the transcripts. SMS also listened to recordings as she had not participated in the
interviews. LC and SMS independently coded the first two transcripts in a text document and identified
preliminary categories, while comparing within and between transcripts. We used a constant comparative
method (developed for grounded theory but can be useful for other approaches) (346,351,352) to
develop codes and categories; identify divergent data; and discover patterns. In the constant comparative
method, each new data item was compared and contrasted with the existing dataset. It also informed and
directed the next step of data gathering in an iterative manner. LC and SMS, together with FS and MAO,
63
reviewed, collated and modified the codes and categories as well as adjusted the interview guide for flow,
clarity and data gathering purposes. SMS created a preliminary chart by putting the collated codes into
categories. LC and SMS then independently applied the codes to the next two interviews. They met to
review, adjust and refine codes and categories. SMS coded and categorized the last two interviews and
LC reviewed her work. After creating the categories using the inductive analysis, SMS mapped the
categories to the domains in the NPT (Table 4.1 NPT coding framework) (Appendix 4.3 Mapping
categories to NPT constructs) and reviewed with LC, MAO and FS. We assessed the extent to which the
data from the inductive analysis aligned with the domains in the NPT. We also looked for categories from
the inductive analysis that did not map to the NPT. SMS then used the data in the NPT domains to
generate themes and reviewed these with the team. In the final stage, SMS checked the themes across the
entire dataset. She reviewed this with LC, MAO and FS. We then reported on these themes. In all cases,
disagreements were resolved with discussion.
Table 4.1 NPT coding framework (316)*
Coherence
(Sense-making work)
Cognitive participation
(Relationship work)
Collective action
(Enacting work)
Reflexive monitoring
(Appraisal work)
Differentiation
An important element of
sense-making work is to
understand how a set of
practices and their objects are
different from each other.
Enrolment Do individuals “buy into”
the idea of the e-health
service?
Skill set workability How does the innovation
affect roles and
responsibilities or training
needs?
Reconfiguration
Do individuals try to alter the
new service?
Communal specification Do individuals have a shared
understanding of the aims,
objectives and expected
benefits of the e-health
service?
Activation
Can individuals sustain
involvement?
Contextual Integration
Is there organizational
support?
Communal appraisal How do groups judge the
value of the e-health service?
Individual specification Do individuals have a clear
understanding of their specific
tasks and responsibilities in
the implementation of an e-
health service?
Initiation
Are key individuals willing
to drive the
implementation?
Interactional workability
Does the e-health service
make people’s work easier?
Individual appraisal
How do individuals appraise
the effects on them and their
work environment?
Internalization
Do individuals understand the
value, benefits and importance
of the e-health service?
Legitimation
Do individuals believe it is
right for them to be
involved?
Relational integration
Do individuals have
confidence in the new
system?
Systematization
How are benefits or problems
identified or measured?
*Reproduced under the Creative Commons licence
64
4.3.8 Rigour
We ensured rigour through several mechanisms. Two of the team members, LC and MAO are
experienced qualitative researchers who provided oversight and direction for the project. LC and SMS
independently coded and categorized the interview data from the first four interviews. LC checked SMS’s
work for the fifth and sixth interviews. We used a constant comparative method to ensure we identified
and explored divergent and disconfirming data. We allowed for triangulation by interviewing participants
from different sites and at two points in time. We used existing theory to assist in describing the
normalization process. And finally, to allow for transparency and to provide context, SMS kept field notes
and memos to document decisions, discussion and disagreements.
4.4 Results
We recruited six PCPs (all physicians), two women and four men, from three UTOPIAN practice sites for
interviews. Two were high prescribers, one a medium prescriber and three low prescribers. All PCPs had
used the PCI at least once prior to the initial interview. We conducted the first interviews about six
months after the PCI was released. At the time of the one-year follow-up interview, all PCPs had stopped
using the PCI.
We found that data from our interviews aligned with the four NPT constructs, which helped us identify
four main themes that allowed us to better describe the lack of normalization of the PCI.
Theme 1: “Always a problem”
The first theme aligned with the NPT “Coherence” (sense-making) construct. PCPs were dissatisfied with
their current approach to chronic pain and opioid prescribing. They described it as “difficult” and “a
struggle.” They felt that chronic pain had a huge impact on some patients’ lives and was difficult to treat.
They had received little training during formal medical education on chronic pain and felt they had
limited ability to provide access to treatment modalities, other than medication. Many of the difficulties
seemed to come from prescribing opioids. PCPs were aware of the guidelines for opioid prescribing for
CNCP. However, they found it difficult to follow them and were dissatisfied with their approach to opioid
prescribing. They were “winging it.”
65
“So, it’s very frustrating, you don’t make them better, you don’t get them back to work, you don’t
really do them any good but now they’re hooked on pain medication too.” (004M)
Theme 2: Technology can help
The second theme also aligned with the NPT “Coherence” (sense-making) construct. PCPs did see it as
their role to make changes in their opioid prescribing practices and felt that technology could help. They
understood that the goal of the PCI was to make opioid prescribing safer and to improve patient
outcomes. They felt the PCI might help them apply guidelines more consistently and move them from a
reactive toward a proactive approach to opioid prescribing for CNCP. They also felt that it would help
them meet medical-legal requirements.
“And I think it can help catch people that are going to run into trouble sooner so you can help
them so I think from that perspective, it will be useful.” (007A)
Most were initially positive about the PCI and liked that the PCI was able to standardize and track
information, such as pain and function score. They stated that it was a good starting point for discussion
and identified issues they may have otherwise been missed.
“Now they [patients] will have something that’s a little bit more numeric to compare back
against later on.” (002D)
“The benefit, I like the idea of scores over time, that’s a really significant benefit.” (004M)
“It’s meant as a springboard for further discussion. So, you’re looking at, oh you have been
suicidal on the PHQ-9, what’s going on there? Or, you have got [pain that] completely interferes
with your function here, okay well maybe we should look at that.” (006L)
PCPs liked that the PCI was integrated into the EMR. PCPs stated that this integration was essential.
“… it’s got to work seamlessly with our electronic records for us to be using it within the office.”
(002D)
Providers also liked that once the patients were flagged, the process proceeded automatically with
minimal assistance from them. They did not have to change roles (become a technician or take on more
responsibility).
“I want to be able to walk in the room and be the doctor, not have to be the techie.” (002D)
66
“And I didn’t give [the patients] very much instruction [on completing the PCI], because I
thought usually, I’m not even going to be there, it’s going to be in the waiting room or I’ll send it
to them.” (005-K)
Theme 3: An irritant and a barrier to communication
The third theme aligned with the NPT construct “Cognitive participation” (relationship work). Concerns
about the potential negative impact of the PCI on the provider-patient communication and relationship for
some of the PCPs may have led to incomplete buy-in. PCPs also felt that a CDSS was not always helpful
and that the “click, clicks” interfered with communication.
“I want to ask the questions. I want to know for myself, to get the nuance from either the
language that the patient says or the information exactly.” (003B)
“Some patients might find the question offensive, why am I being asked about double doctoring
or getting a high or a buzz? These questions are highly offensive and because of the nature of the
custom form being very short, there is no way to soften the way that the questions are asked. It’s
very to the point, do they get high or don’t they, whereas there might be ways to soften it if I was
verbally asking it myself.” (006L)
Several PCPs were concerned that the PCI might cause patients irritation or distress and affect their
relationship with the patient. These concerns appeared to be centred around the COMM, the component
of the PCI that asked patients questions about aberrant behaviours that may indicate an OUD.
“[A CDSS] is already is a very polarizing method to interact with patients and then opiate
prescriptions and pain visits in general are very emotional for people. They feel like they are
getting audited or checked or, [006L] doesn’t like me.” (006L)
“So, I really have to watch my patients and see who might be okay with [the CDSS] and consider
it convenient versus for some people it might be an added irritant.” (006L)
“But I think there was an affect [sic], certainly for a couple of my patients at least, that once
you’re giving them this questionnaire, they almost felt like they were being accused of
something.” (002D)
One PCP had a patient who refused to complete the PCI, because the patient felt the questions were ‘too
personal’:
67
“But I had one, for example, this was a patient who pretty insistently over the last six months has
told me that I don’t give him enough pain medication and he wants something stronger and he
wants more of it. So, he started this but he said the questions were too personal and he refused to
continue, which I thought was an odd answer in a doctor’s office.” (004M)
However, two PCPs were certain that patients would not have a negative reaction to the PCI. When asked
in the one-year follow-up interview if the PCI could be offensive to patients, one PCP responded:
“Not at all. I think once you explain to patients the reporting requirement, almost all of them
understand. And whether you’re asking them a question individually or they’re answering the
question directly so that it saves you some time to talk about what they really want to talk about,
they’re happy with it. But I’m not seeing a negative impact with that.” (007A-F)
All the PCPs did acknowledge, however, that new technologies were not going away and were likely to
transform how we practice.
Theme 4: More work and an interrupted workflow.
The fourth theme aligned with the NPT construct “Collective action” (enacting work). The PCI was
designed so that patients completed the PCI while they were waiting to see their provider for their
appointment. However, our participants reported that often the patient had not completed the questions by
the time the PCP was ready to see them; as a result, the PCP ended up waiting or working with the
patient, and slowing, rather than improving workflow.
“I know it kind of messes up the schedule. So, if a patient is already in a room and I think, okay,
let’s do this, where is that 15 minutes going to come from? That’s a challenge, so we need to
figure out ways to do this.” (005K)
In follow-up interviews, most PCPs cited the interrupted workflow as the main reasons they were no
longer using the PCI.
“So, we didn’t always have that 15 minutes or so for them to be sitting in the waiting room with
the iPad or with the tablet…” (002D-F)
“Sometimes, they’re there long enough to complete a form, but not always, and it just doesn’t
seem to have been built into my workflow to have them wait around long enough to do a form.”
(004M-F)
68
To address this, some PCPs had the front desk staff call patients to come in early. This however added
work to front desk staff.
One PCP felt that the PCI did not interrupt workflow. This PCP was using paper tools to help manage
opioid prescribing for CNCP and had adjusted his workflow already for the paper tools. He felt the PCI
would be an improvement.
“Because now, the nurse prints out paper that she gives to the patient that I review that then has
to go in and get scanned. That’s how it goes, and so I see it when I’m with the patient because it’s
at the time of the visit. With this, I’d walk into the room, and this would already be in the chart,
and it evolved straightforward.” (007A-M)
The PCI also placed additional responsibility and work for the PCPs. Providers had to look ahead to their
schedule for the day or next few days to flag or run a search program (AUDIT Tools (Appendix 4.2
Patient Check-in (PCI)) to find and flag appropriate patients. (The AUDIT tools appeared to be difficult
to run, as none of the PCPs used them to identify patients.) If the PCP neglected to identify the patient
ahead of time, the patient would have to complete the PCI in the exam room. This could take an
additional 10-15 minutes. Another issue may have been the extra work placed on the PCP who was the
identified “lead for anything electronic” at each site. This individual did the implementation of the PCI
and often also ran the searches to help colleagues identify appropriate patients, or helped administrative
staff and medical colleagues to use their patient list to identify such patients for the PCI. This also meant,
however, that some PCPs did not identify and flag patients on their own.
“I am totally and absolutely ignorant because Dr. X did all that.” (003B)
“[Another PCP] been quite helpful and encouraging the front desk to hand it out without even
asking me.” (004M)
Theme 5: “It should be customizable.” The fifth theme aligned with the NPT construct “Reflexive
monitoring” (appraisal work). PCPs identified ways to improve the PCI. Many expressed a desire to
customize the components.
“…the question would be if it would be possible to modify what’s on the tablet, maybe more
customize it.” (006L)
For some this meant including additional components, such as a function that would calculate the
prescribed opioid dose in morphine equivalents (MEQs).
69
“But I don’t know what his MEQs are. I don’t know what his average total amount of narcotic
use is. There’s nothing in this tool that tells me that.” (005K)
For others, it meant removing or modifying components that might negatively impact PCP-patient
relationship. In particular, some PCPs were concerned about sections that asked about opioid misuse.
“…there’s no question that at least for a few patients, the attitude was, well, you’ve been
prescribing me these medications, I’ve been taking them for so long and now you’re accusing me
of doing something wrong with them?” (002D)
PCPs also identified how they would assess the benefits of the PCI. They looked for measures like
increased patient satisfaction, better workflow and changes in prescribing (e.g. reduced opioid dose).
4.5 Discussion
Our study generated a number of themes through describing the normalization of a specific CDSS for
opioid prescribing for CNCP, the PCI, in primary care practice. The NPT assisted us in generating these
themes by focusing our interview questions and organizing our analysis. We found that the PCPs in our
study struggled with their current approach to opioid prescribing for CNCP—“always a problem”— and
saw a potential benefit to using a CDSS. The PCPs reported they felt a need to change their approach and
viewed medical informatics, including CDSSs, as a possible solution. They all adopted and starting using
the PCI within the first six months after its release. However, the PCI, did not appear to meet their needs.
After initial use of the PCI, all PCPs had abandoned the tool by the follow-up interviews (held
approximately one year after the first set of interviews). There appeared to be several barriers to
normalization of the PCI: the PCPs reported that the PCI created more work and interrupted the
workflow. This aligns with the NPT construct of ‘collective action.’ The PCI was designed to gather
information from patients while they were in the waiting room. However, PCPs reported that the data
gathering often took too long, delayed the start of the appointment and interrupted the PCP’s workflow.
In initial and follow-up interviews, interrupted workflow was a recurring theme and, according to the
PCPs, a major reason they discontinued using the PCI. The PCI also created extra work; it did not
automatically identify appropriate patients, so PCPs had to search their patient rosters or list of upcoming
appointments to identify them. Additionally, PCPs had to remember to do this prior to a patient’s
appointment, so the patient could complete the PCI while waiting to see the PCP for an appointment. The
PCI also appeared to have a major barrier in the construct of “cognitive participation” with incomplete
buy-in to the use of the PCI. A number of the PCPs were concerned that the use of the PCI to gather data
from the patient might interfere with provider-patient communication and the provider-patient
70
relationship. The PCPs reported that the use of opioids for CNCP was a highly-charged topic that required
sensitivity, empathy and sophisticated interviewing skills. The use of a CDSS to gather information might
lead the provider to miss subtleties and nuances that could be picked up in a face-to-face conversation.
Additionally, some PCPs were concerned that some of the questions in the PCI could cause a negative
reaction (irritation or offense) in the patient and drive a wedge in the patient-provider relationship.
Providers were particularly concerned about the questions about aberrant behaviours contained in the
COMM. PCPs suggested modifications of the PCI to meet their needs, and had ideas about how they
could measure its success. In its present form, however, the PCI appears unlikely to be normalized within
practice settings.
4.6 Conclusions
The specific CDSS for opioid prescribing for CNCP that we assessed did not become normalized in the
practices of the six PCPs in this study. The NPT assisted us in understanding the barriers to
normalization. One barrier was the disruption to workflow and the additional work to identify patients.
Another major barrier was PCPs’ concerns that the PCI may negatively impact communication and the
provider-patient relationship. The reason appears to be how the PCI gathers sensitive information about
opioid use in CNCP directly from the patients. This exploratory study is valuable in understanding why
new technology, in this case a particular type of CDSS that gathers sensitive information directly from
patients, may not be accepted in medical practices.
71
General Discussion
5.1 Discussion
Prescribed opioids for CNCP lead to substantial harms and spawned the current opioid crisis in Canada
and the U.S. (6–11). Recent Canadian and American guidelines recommend that prescribers make
changes to prescribe more safely and appropriately (12,54). Adherence to the guidelines may be difficult
to achieve (13–17,180,353), as complex behavior changes are difficult (188,189). CDSSs, a form of e-
health technologies, may be a solution. Research indicates that CDSSs lead to improved prescribing in
other areas of medicine (27–35). However, impact on patient outcomes is not clear (27–30), and CDSSs
are plagued by implementation issues (40–43,45,46). Additionally, CDSSs can be difficult to develop
and evaluate as they are complex interventions—they seek to change the functioning of a complex
adaptive system and have multiple interacting parts and causal pathways (47,277). Therefore, the aim of
my thesis was to gain an understanding of the potential benefits and limitations of CDSS for opioid
prescribing for CNCP in primary care clinical settings, including an understanding of the gaps in
knowledge. We conducted two studies to address this aim. In the first scoping review study, my objective
was to report on the range and extent of current research on CDSS for opioid prescribing for CNCP in
primary care clinical settings and the extent to which researchers are following best evidence for CDSS
components and current guidance for complex interventions. In the exploratory qualitative study, my
objective was to describe the normalization process of a specific CDSS for more appropriate opioid
prescribing for CNCP in primary care settings.
The studies helped address our research aim and specific objectives. Our research indicates that there is
little evidence of benefit in using CDSSs for opioid prescribing for CNCP in primary care settings.
Although they appeared acceptable to PCPs, and some studies in the scoping review reported CDSSs led
to more appropriate prescribing, we found important limitations. First of all, our research demonstrated
that there are substantial gaps in the literature. In our scoping review, we found few studies examining
CDSSs for opioid prescribing for CNCP in primary care settings. Additionally, the studies used designs
that tend to lead to lower quality of evidence (336); did not examine important patient outcomes; nor
assessed for unintended consequences. We also found that developers did not incorporate best evidence
for CDSS components into the CDSSs. Investigators failed to follow current guidance for development
and evaluation of complex interventions and as a result paid little attention to implementation and
72
normalization. Our exploratory qualitative study revealed that there were significant barriers to
normalization of a specific CDSS for opioid prescribing for CNCP in primary care.
Our scoping review indicated that PCPs may have positive attitudes towards PDMP CDSS and that many
PCPs use them in their practices. Our findings in the exploratory qualitative study added support to this.
This may be because PCPs are unhappy with the current approach to opioid prescribing for CNCP. In our
exploratory qualitative study, PCPs reported that they struggle with the management of opioid prescribing
for CNCP. This is consistent with results in many other studies (21,24,218,354). PCPs saw CDSSs as a
possible improvement over the current approach. The medical literature on PCPs’ views on CDSSs is
somewhat more mixed. In pre-implementation studies on the potential usefulness of a CDSS for opioid
prescribing for CNCP, providers had many concerns about potential negative impact on work and
workflow, and access to opioids for patients who really needed them (253,255,355). Studies on attitudes
towards CDSSs in other fields show many provider concerns about the use of CDSSs: competing
demands on the provider, an overwhelming number of alerts, problems with CDSS usability; a lack of
training, support, and integration into work processes; and concerns from providers about loss of
autonomy and medical-legal repercussions (41,251,252,356). The PCPs in our study may have more
positive views than those in other studies because of differences in sampling. Because of problems with
recruitment, we used a convenience sampling approach, interviewing all users of the CDSS who agreed to
participate. These were early adopters of technology (they had all adopted and started using the PCI
within six months of its release)—people who tend to have more positive views of technology (357).
Several studies in our scoping review reported that CDSSs, or CDSSs as part of a multi-faceted
intervention, in primary care led to more appropriate opioid prescribing for CNCP. This aligns with the
research in other fields that demonstrates that CDSSs have an impact on process outcomes, like
prescribing systems (27–35). However, it contrasts with a recent scoping review on PDMPs by Finley
and colleagues, that reported mixed results of PDMPs’ impact on opioid prescribing rates (358). Four of
the five studies in the review looked at the state population-level impact of the PDMP and one at
prescribing at a dental clinic. Only two studies reported positive outcomes. The difference in outcomes
between our scoping review and the study by Finley and colleagues may be due to the different settings
(primary care vs. dental clinic or population level). It is also possible that PDMP CDSSs are less effective
than local CDSSs in primary care at leading to changes in prescribing and adherence to guidelines.
However, there are indications that we should be cautious in how we interpret the positive findings in our
scoping review.
73
Of concern, we identified substantial gaps in the literature on CDSSs for primary care opioid prescribing
for CNCP. Our scoping review demonstrated that there are few studies. There may be several contributing
factors. The prescription opioid crisis only gained widespread attention in the last ten years (359), and it
takes time to develop a complex intervention like a CDSS (47). Additionally, it is possible that some
CDSSs failed to show promise early on and development stalled or was halted. Accordingly, there are a
number of reports on the development of a CDSSs for opioid prescribing for CNCP where clinical
outcomes have not been reported yet (218,253,258,259). Another limitation is that most of the studies in
our scoping review used pre-post, non-randomized control or observational designs; only one used a
cluster randomized control design. Although we did not do a quality assessment, these types of study
design are more likely to lead to lower quality evidence (336). As a result, it is possible that the positive
impact on prescribing outcomes in our scoping review studies may be due to other factors. Additionally,
in several of the studies in the scoping review, the CDSS was part of a multifaceted intervention and it is
unclear if impact on prescribing is from the CDSS or other components of the intervention (327,360).
These findings demonstrate a need for more studies with high-quality study designs to determine if
CDSSs lead to safer and more appropriate prescribing.
Another limitation may be a biased literature base. In the scoping review, we found that most local CDSS
developers were also evaluators. (No study on PDMP CDSS stated the relationship between developers
and evaluators, likely because authors of study felt it was clear that they were separate group.) Systematic
reviews demonstrate that when the CDSS evaluator is also the developer, outcomes are better (36,244).
The explanation is unclear. It is possible that developers are able to achieve better outcomes because they
understand the CDSS better and can design a more effective implementation plan (244). It is also
possible, however, that because of the conflict of interest, outcomes are biased (244). There is some
evidence to support latter hypothesis. Researchers have demonstrated that when the pharmaceutical
industry evaluates its own drugs, outcomes are also better (361–364). The pharmaceutical industry
achieves this through several techniques: they design studies that are more likely to show positive
outcomes (e.g. by comparing to placebo, or to non-standard of care drug or to a sub-therapeutic dose of
another drug); they selectively report outcomes; and they avoid or delay publishing studies with negative
outcomes (361,362,365–367). The pharmaceutical industry is strongly motivated by financial gain and
the bias is likely deliberate (368). However, conflicts of interest can also lead to unintentional and
subconscious bias (369). Therefore, it is essential that all studies report on the relationship between the
evaluator and developer, and clearly reporting all roles, as well as any financial conflicts of interest. As
the development and exploratory evaluation of a complex intervention should be entwined (47), this
conflict of interest is often impossible to avoid in the early stages. The developers’ intimate knowledge of
74
the CDSS also may be essential in the evaluation. The evaluators should, however, explore the potential
impact of the relationship and include plans to mitigate bias (370). Definitive evaluations, ideally, should
be led by a disinterested third party (244). If this is not feasible, the evaluators should have a third party
review the study protocol to determine where the conflict of interest could have an impact on outcomes.
Then they should take measures to address, and transparently report on, the conflict of interest. To reduce
selective publication and outcome reporting, evaluators should register trial protocols and ensure that they
publish all studies, including pilot and feasibility studies, expediently as possible even where outcomes
are negative (371,372).
Another major limitation with the literature base is lack of evaluation of important patient outcomes (as
described in chapter 2). None of the studies in our scoping review reported on any of these or similar
patient-related outcomes. Instead these studies reported on prescribing rates or adherence to a protocol,
including items like urine drugs testing and recording of diagnoses. These are called surrogate endpoints
and are used because they are likely to reflect patient outcomes (373). Researchers often use them rather
than patient outcomes because the endpoints occur sooner and more frequently, thus allowing for smaller
and shorter trials (374). There may be an additional incentive for researchers investigating CDSSs to use
surrogate endpoints: systematic reviews show that studies that examine patient outcomes have mixed
results (27–30). Although they can be useful, surrogate endpoints have substantial limitations. They may
not accurately reflect important patients outcomes and can lead investigators to conclude that an
intervention will benefit patients when it does not (373). For example, although initial studies showed
that the two anti-arrhythmic drugs, encainide and flecainide, suppressed arrhythmias effectively, follow-
up studies revealed they actually increased mortality (373). Therefore researchers, clinician and policy-
makers should be cautious when interpreting a reduction in prescribing as it may not have a positive
impact on patient outcomes, such as improved quality of life or reduction in overdose deaths. Future
studies on CDSSs for opioid prescribing for CNCP in primary care settings should seek to include patient
outcomes.
Additionally, our scoping review indicated that investigators are not assessing for unintended outcomes.
As CDSSs are complex interventions, they are likely to cause unexpected, unintended consequences as
the result of feedback loops, disproportionate effects, and emergent outcomes (47). The unintended
consequences may be particularly problematic in this area of medicine. Studies have shown that when
patients have reduced access to opioids, they may use illicit opioids (173,375). Use of illicit opioids is
riskier than use of prescription opioids because of the lack of quality control (174–176). The increase in
deaths in the US and Canada is no longer from prescription opioids but from fentanyl, much of which
appears to be illicit (169,174,176,179). There is evidence that use of PDMP CDSSs may lead to this
75
unintended consequence. A recent systematic review reported that three of the six studies that examined
the impact of a PDMP on heroin overdoses found an increase in heroin overdoses after implementation of
the PDMP (264). It is not clear if the relationship was causal and other factors may have led to the
increase in deaths, such as the influx of cheap illicit opioids and the release of tamper-resistant oxycontin
(376). However, the authors of the review hypothesize that use of the PDMP may have restricted access
to prescription opioids, and led patients to seek out riskier illicit opioids. Therefore, all studies evaluating
CDSSs, particularly ones that seek to change opioid prescribing, should assess for unintended
consequences. Researchers should use the medical literature, as well as their theoretical model to
determine possible unintended consequences, and develop plans on how to address them.
We also found in our scoping review that few CDSS developers were incorporating evidence into the
design components of the CDSS. Only three of the CDSSs in our scoping review were integrated into the
EMR and two of those automatically activated. None required a reason for over-ride or provided advice to
patients. Developers may be ignoring the information in the systematic review because they feel that the
evidence is not strong enough. However, the evidence supporting automatic activation, patient advice and
reason for over-ride is consistent in systematic reviews (30,36,37,39,244). They may also feel that the
evidence does not apply to this particular sub-specialty or setting (316). Another reason may be a general
excitement and overconfidence in e-health technologies (245). Funders and developers may be too eager
to solve the problem of unsafe opioid prescribing using e-health technologies and are not ensuring that
developers are building on information from the medical literature (245). PDMPs in our study did not
appear to have any of the evidence-based components. This may be changing. There is a move in many
U.S. states to integrate PDMP CDSSs into local EMRs. Between 2012 and 2016, the Substance Abuse
and Mental Health Services Administration funded nine projects to integrate PDMP data into EMRs
(377). The developers working on the integration of PDMP data should also consider incorporating other
evidence-based components. Going forward, when creating a new CDSS, developers should provide
design justification for their CDSSs, including why they are following or not following current evidence
from systematic reviews, based on their hypothesized causal pathways and mechanisms of action.
Another limitation of the knowledge base is that the studies provided poor insight into why or how the
CDSS for opioid prescribing for CNCP worked in primary care settings. In all the studies that examined
more appropriate opioid prescribing for CNCP, we only know that the intervention worked or did not
work. This is because the studies in our scoping study are not following guidance for development and
evaluation of complex interventions. Guidance recommends a series of carefully staged exploratory
studies and a definitive evaluation, as well as complementary planned process evaluations at each step
(47,277,322). Process evaluations can assist in identifying why an intervention worked or did not work;
76
ensure success when translating to new environments by identifying key components and mechanisms of
the CDSS, as well as determine the impact of context and implementation process (275). Additionally,
most of the experimental studies in the scoping review provided minimal information on implementation,
making it impossible for other researchers to replicate the process. As a result, it would be very difficult
to those seeking to replicate the CDSS in other setting to create an effective implementation plan or
predict if they could expect the same outcomes. The explanation for the failure to follow guidance is
unclear, but the problem is not unique to CDSSs for opioid prescribing for CNCP. Although more
complex interventions go through a series of evaluations with pilot and feasibility testing than in the past,
a recent systematic review found few published studies (371). Additionally investigators rarely provide
sufficient information about the intervention and implementation for others to replicate the study
(378,379). Although poorly studied, it appears that many evaluations of complex interventions also lack
quality process evaluations (380,381). This is problematic. If researchers run a trial without testing
components, possible causal pathways, uncertainties, contextual factors, implementation approaches, they
a risk wasting resources on an expensive trial and perhaps causing harm (277,322,370).
Our exploratory qualitative study provided additional support for conducting process evaluations when
evaluating a CDSS for opioid prescribing for CNCP in primary care settings. Although the PCPs had
favourable views of CDSSs and had used the PCI at least once in practice, all had stopped using it by the
follow-up interviews. Without those interviews, we would have had little insight into the reasons why or
how to go about improving the CDSS. From the interviews, we identified several possible barriers to
normalization. The PCPs in our study found that the CDSS increased workload and interfered with
workflow. (This is in the NPT construct of ‘collective action.’) PCPs reported that data gathering often
took too long and interrupted the PCP’s workflow. Additionally, the PCI did not automatically identify
appropriate patients, but relied on PCPs to do this in advance of an appointment, adding extra work.
These are common issues with CDSSs. Studies report that workload and workflow are reasons that
CDSSs fail to become integrated into health care practices. For example, in a pre-implementation study
on the potential usefulness of a CDSS in a military health system, providers had many concerns including
impact on work and workflow (255). In another pre-implementation study, providers also indicated that
any increased workload would be a barrier to the proposed CDSS (382). Extra work may be particularly
problematic in primary care settings as providers have many competing demands for a wide range of
health conditions (268). If a CDSS does not automatically spur the provider to action, it is unlikely the
provider will remember to use it on their own. This may be why a recent systematic review of CDSS for
antibiotic prescribing in primary care found that automatic activation led to better outcomes than provider
initiation (39). Additionally, work and workflow may be more of an issue for opioid prescribing for
77
CNCP than for many other conditions. Prescribing opioids for CNCP requires a lot of time, effort and
additional skills. This includes:
1. Time to conduct shared-decision making with patients who may not buy-in to the changes (18).
2. Time to secure access to other treatment modalities for patients in a timely manner. Poor access
to other treatment modalities is an issue that providers frequently raise as a barrier to better
management of CNCP (25).
3. Skills to manage aberrant behaviours and OUD (24).
Therefore, when designing, implementing and evaluating CDSS, the immediate impact on work and
workflow should be considered, as well as also the extra work generated by the CDSS in changing the
management of opioid prescribing for CNCP.
Our exploratory qualitative study also identified an additional barrier to normalization that to our
knowledge has not been identified in previous studies. In our exploratory qualitative study, some of the
providers were concerned that the CDSS may have a negative impact on provider-patient communication
and on their relationship with patients. As a result, they had incomplete buy-in to the PCI. (This aligned
with the NPT construct of “cognitive participation.”) This appeared to be the result of two interactive
factors. First, the CDSS gathers information directly from the patients and secondly, it is highly sensitive
information. The PCPs reported that the use of opioids for CNCP was a highly charged topic that required
sensitivity, empathy and sophisticated interviewing skills. The use of a CDSS to gather information might
lead the provider to miss subtleties and nuances that could be picked up during a face-to-face
conversation. Additionally, some PCPs were concerned that the questions about opioid misuse on the
tablet could cause a negative patient reaction (irritation or offense) and drive a wedge in the patient-
provider relationship. The literature indicates that patients find use of a tablet to gather patient
information acceptable in general. Studies have used it for recruitment for research, sharing health-related
information and clinical data without any apparent issue (383–386). Several studies have directly sought
patient’s views and found the approach acceptable (387–389). A study by Harle and colleagues assessed
the acceptability of patient entered data on CNCP (390). The authors reported that there was no evidence
that the gathering data on patient reported outcomes using a tablet caused dissatisfaction. However, none
of the questions were about opioid prescribing or aberrant behaviors that may indicate an addiction. A
study by Goodyear-Smith and colleagues assessed the acceptability of a paper tool, the Case-finding and
Help Assessment tool (CHAT), to gather information of lifestyle and mental health conditions on 2543
patients (391). Patients and providers found the CHAT acceptable and most providers stated they would
use the tool. Although the CHAT asked sensitive questions about depression, alcohol use and smoking, it
78
did not inquire about opioids for CNCP. It is likely that questions about opioid prescribing for CNCP and
aberrant behaviours are less acceptable to patients and providers. Many studies have also found that
providers and patients report discomfort in communications around opioid prescribing for CNCP,
particularly around aberrant behaviours, and providers have concerns about patient dissatisfaction
(18,19,24,219). It is not clear if this concern will affect CDSSs for opioid prescribing for CNCP that do
not gather information from the patient and instead use information from the EMR or another database.
No studies in our scoping review mentioned this as an issue. However only one study interviewed
providers about their views on CDSS (a study on PDMP CDSS (333)) and only one conducted a
satisfaction survey about a CDSS (a local CDSS (327).
5.2 Implications
Our research has implications for clinicians, administrators, researchers and funders. As there are
significant limitations and gaps in the literature, it is unclear if CDSSs are an appropriate KT intervention
to encourage more appropriate opioid prescribing for CNCP in primary care settings. Additionally,
although CDSSs have an impact on prescribing in other areas of medicine, opioid prescribing appears to
have unique considerations: a CDSS may not meet the need of providers who report that lack of supports
as well as communication difficulties are barriers to change.
Therefore, clinicians and administrators should be cautious when deciding to implement or use a CDSS
for opioid prescribing for CNCP in their primary care setting. They should be aware that there are few
studies, and that none examined important patient outcomes or assessed for unintended consequences.
They should also be aware that few studies have included process evaluations, and as a result, there is
little understanding of the essential components of a CDSS, and the role of its implementation and
context. Additionally, there are likely to be significant barriers to normalization that are not yet well
understood. If they decide to proceed with a CDSS, they should ensure that the CDSS has undergone
rigorous outcome as well as process evaluations before implementation to increase the likelihood that the
CDSS will achieve the desired outcome in their setting. They should ensure that there is ongoing
monitoring of implementation, outcomes and unintended consequences. They should be aware that bias
may result if the evaluators are also the CDSS developers. Researchers should ensure that going forward,
they use current evidence on CDSSs to design the components of the CDSS. They should also be
following guidance to develop and evaluate the CDSS. Researchers should have plans in place to address
successes and failures at each stage of development and evaluation, as well as plans to assess for
unintended consequences. They should share the results of all steps of development and evaluation so
other researchers can learn from their processes. They should follow guidelines for randomized pilot and
79
feasibility studies (392). They should publish a protocol for all experimental studies. Granting agencies
should consider these factors before providing funding.
Our exploratory qualitative study has particular implications for those who implement the PCI in their
practice. Our research indicates that there may be barriers to normalization, resulting from extra workload
and interrupted workflow. Our research also indicates that the PCI may not get buy-in from PCPs,
because of concerns that it might create a barrier to communication and might harm the patient-provider
relationship. The CDSS from our exploratory qualitative study needs further refinement and evaluation to
ensure it is acceptable to providers and patients. Patients, PCPs and other end-users should be involved in
the design and testing from the outset. Patients and providers both must be comfortable with the interface
and understand its aims. Our results have broader implications, indicating that if providers perceive that
the CDSS is not acceptable to patients, they are less likely to use it. More research is needed to determine
the acceptability of using a patient interface to gather data about opioid prescribing for CNCP, and how
such a CDSS works best for both patient care and for effectiveness. Researchers who design CDSSs for
opioid prescribing should be aware of this potential barrier, particularly if there is a patient interface.
5.3 Strengths
Our two studies have a number of strengths. Our scoping study was rigorous. We followed current
guidance in conducting our review, adhering to all points in reporting guidance document (235). We will
continue to contact experts in field to identify any pending studies prior to a peer-reviewed publication.
To our knowledge this is the first scoping review looking at CDSS for opioid prescribing for CNCP in
primary care settings. We were able to report and synthesize the current extent and range of the evidence.
We identified substantial gaps in knowledge, as well as in the development and evaluation processes.
These findings should provide guidance to researchers, administrators and funders, and we hope, will
encourage more rigor in designing, implementing, and evaluating new CDSSs. Our qualitative study
provided an in-depth description of the normalization process of a specific CDSS for safer opioid
prescribing for CNCP into primary care settings. The failure to normalize provided an opportunity to
learn about barriers. We reported on a novel finding related to the patient information gathering
component that may be a significant barrier for this type of CDSS for opioid prescribing for CNCP.
Another strength is the rigour of our approach. We had two team members with expertise in qualitative
research. Two researchers independently analyzed the data and then worked together with the team to
complete the analysis. We used a well-developed normalization theory that assisted us in our analysis. To
enrich our data, we conducted interviews with PCPs at different sites and at two times points, allowing us
also to assess ongoing use. Combining the studies provided both a broad overview of CDSSs for opioid
80
prescribing for CNCP in primary care settings and an in-depth description of the normalization process of
a specific CDSS.
5.4 Limitations
There are a number of limitations in our studies. In the scoping review we may have missed non-English
language studies that were not peer-reviewed (grey literature). We conducted our searches in English so
studies not published in English would unlikely to show up. Another limitation was that several of the
studies included both PCPs and other provider types (we excluded those with less than 50% PCPs), and as
these studies only reported aggregate outcomes they may not accurately reflect the PCP population. The
qualitative study had a number of limitations as well. One limitation is that the developer, ISMP Canada,
decided not to proceed with further development and evaluation of the PCI. As a result, our study is no
longer part of a planned series of exploratory studies and process evaluations. Our study, however, will
add to the general knowledge of CDSSs for opioid prescribing for CNCP in primary care settings.
Another limitation is that I was also a content expert on the grant to create the PCI, creating a conflict of
interest. Although I have no financial stake in the product, I may still be affected by my role in providing
advice to those creating the PCI. Another limitation of the qualitative study is our small sample size.
Despite contacting all 14 UTOPIAN sites (with an estimated 1400 affiliated physicians) we were only
able to recruit six physicians to participate. We were hampered by an inability to access information from
the private software company about what sites were using their software and had downloaded the PCI. As
a result of the small sample size we were unable to reach thematic saturation, the major limitation in our
study. To partially mitigate this, we conducted a second round of interviews with our participants and
thus were able to add a longitudinal component to the study and to further explore the themes we
identified.
5.5 Conclusion
Our research indicates that there is little evidence to support use of CDSSs for opioid prescribing for
CNCP in primary care settings. Although CDSSs appeared acceptable to PCPs and, some studies in the
scoping review reported CDSSs led to more appropriate prescribing, we found important limitations.
Studies were few in number and used designs that tend to lead to lower quality of evidence. Studies did
not examine the impact on patient outcomes or assess for unintended consequences. In many studies the
evaluators were also the CDSS developers, but the authors did not address this conflict of interest or
report on steps to mitigate potential bias. Additionally, developers and evaluators did not appear to be
incorporating best evidence for CDSS design. They are also not using current guidance for developing
81
and evaluating a complex intervention, so there is little insight into how or why a CDSS works or does
not work. Additionally, our research indicates that there may be barriers to normalization for the specific
CDSS we assessed, related to the sensitive nature of opioid prescribing for CNCP. To encourage
appropriate opioid prescribing and to improve patient outcomes, investigators, and funders should be
more rigorous in planning, funding, developing and evaluating CDSSs for opioid prescribing for CNCP in
primary care settings.
5.6 Future Directions
Our findings point to several areas that need more exploration. With respect to the specific CDSS we
assessed in our exploratory qualitative study, if the developer was planning further development and
evaluation, there are several next steps to address the possible barriers to normalization we reported on in
our study. First, researchers should conduct additional interviews to reach thematic saturation. Then they
should engage providers and patients to redesign and test the patient interface and data gathering
approach. They should also design the CDSS to address the concerns around work and workflow and
around the CDSS creating a barrier to communication and damaging the provider patient relationship.
Some possible options include:
1. Modifying the CDSS so patients can complete it at home prior to their appointment using an
online email link.
2. Modifying the CDSS so patients can complete it quickly.
3. Programming the CDSS to automatically identify and flag appropriate patients to complete the
CDSS instead of relying on providers to perform this step.
4. Modifying PCI questions that are acceptable to both patients and providers. The Case-finding and
Help Assessment Tool (CHAT) developed in New Zealand, asked sensitive questions about
lifestyle and mental health, but achieved high acceptability ratings (391).
5. Providing an explanation and placing sensitive questions in context to increase patients’
participation in the CDSS.
To follow best evidence for the PCI, researchers should incorporate advice to patients as well as providers
and should require an over-ride with a reason if the PCP chooses to ignore the CDSS. With each of these
potential solutions, the PCI should then be tested with small groups of patients and providers in a
simulated setting to access usability and acceptability as well as workload and workflow. Once
researchers are satisfied with the CDSS, they should conduct a series of exploratory studies to test key
processes and outcomes along with process evaluations. However, given that the developer of the PCI
does not intend to further develop and evaluate this CDSS, and that a private company owns the software
82
code and platform, it is unlikely that this specific CDSS will be used or modified. However, these
findings may be of use to developers who create a similar CDSS for opioid prescribing for CNCP.
Our research identified some other areas that need further exploration. With respect to the scoping review,
researchers could consider conducting a formal assessment of quality of the studies. However, given that
three of the five studies that report a positive outcome use a pre-post design, and the sole cluster RCT has
the CDSS in the intervention and control arm, even if the studies are of high quality, the results will not
change the conclusions of this thesis. Instead, researchers could consider first taking a step back to
determine if CDSSs are likely to meet the needs of providers in leading to behavior changes. Qualitative
studies or modelling studies may indicate if a CDSS is appropriate or if other KT interventions would be
more useful. As mentioned in chapter 2, there is limited evidence to guide the selection of appropriate KT
techniques in most areas and contexts (186). Therefore, throughout the KT process, researchers should be
monitoring and evaluating the process and outcomes, and adjusting their KT plan as needed
(186,200)(186).
There is a need for more understanding of what is happening in the development stages of CDSSs for
opioid prescribing for CNCP in primary care settings. We identified many studies in this stage during our
scoping review CNCP (218,253–259). A scoping review or qualitative analysis of these studies would
give us a better understanding of the development process and where projects run into problems or get
stalled. It would allow us to determine if they are following evidence for CDSS components and guidance
for complex interventions.
There is also a need to further explore how conflicts of interest affect outcomes (when developers are also
the CDSS evaluators). Researchers could conduct a study to compare methods, publication rates and
reporting of CDSSs between studies where developers are also evaluators and where they are not. This
may provide insight as to why studies where the developers are the evaluators have better outcomes
(36,244). Lexchin and colleagues conducted a similar study where they compared trials funded by the
pharmaceutical industry to trials funded by other sources (362). The difference appeared to be related to
publication bias; research funded by pharmaceutical companies was less likely to be published when
negative. As a next step, researchers could conduct a study with an assessment of bias in studies where
the evaluators where also the developers (229).
Our research indicates a need to better understand the possible unintended consequences of CDSSs for
opioid prescribing for CNCP. This may prompt developers and evaluators to focus on this issue.
Observational data may help provide answers. For example, an observational study could examine what
happens when PCPs obtain PDMP data that shows that a patient sought opioids from multiple providers
83
or when a local CDSS recommends a tapering of opioids. Researchers could use secondary data sources
to determine if this leads to potentially harmful outcomes like abrupt discontinuation of prescribed
opioids. And finally, our research indicates a further need to understand why investigators are not
following guidance for evaluating complex interventions such as conducting complementary planned
process evaluations alongside outcome evaluations for CDSSs. Reasons are unclear but may include a
lack of knowledge, skills or resources (380,381). A qualitative study could investigate and further
describe the barriers. A diagram that identifies all the key stakeholders (developers, clinicians,
researchers, patients, funders, administrators) during the stages of development and evaluation of
complex interventions may help plan sampling for the qualitative study. This diagram could also have
many uses throughout the development and evaluation of complex interventions.
84
References
1. Busse J. The 2017 Canadian Guideline for Opioids for Chronic Non-Cancer Pain
[Internet]. National pain center; 2017. Available from:
http://www.cmaj.ca/content/suppl/2017/05/03/189.18.E659.DC1/170363-guide-1-at-
updated.pdf
2. Guy GP. Vital Signs: Changes in Opioid Prescribing in the United States, 2006–2015.
MMWR Morb Mortal Wkly Rep [Internet]. 2017 [cited 2018 Mar 13];66. Available from:
https://www.cdc.gov/mmwr/volumes/66/wr/mm6626a4.htm
3. Levy B, Paulozzi L, Mack KA, Jones CM. Trends in Opioid Analgesic-Prescribing Rates
by Specialty, U.S., 2007-2012. Am J Prev Med. 2015 Sep;49(3):409–13.
4. Volkow ND, McLellan TA, Cotto JH, Karithanom M, Weiss SRB. Characteristics of
Opioid Prescriptions in 2009. JAMA. 2011 Apr 6;305(13):1299–301.
5. Health Quality Ontario. Starting on Opioids in Ontario [Internet]. 2017 [cited 2018 Jul
12]. Available from: http://startingonopioids.hqontario.ca/
6. Dunn KM, Saunders KW, Rutter CM, Banta-Green CJ, Merrill JO, Sullivan MD, et al.
Overdose and prescribed opioids: Associations among chronic non-cancer pain patients.
Ann Intern Med. 2010 Jan 19;152(2):85–92.
7. Bohnert AB, Valenstein M, Bair MJ, et al. Association between opioid prescribing
patterns and opioid overdose-related deaths. JAMA. 2011 Apr 6;305(13):1315–21.
8. Gomes T, Mamdani MM, Dhalla IA, Paterson JM, Juurlink DN. Opioid dose and drug-
related mortality in patients with nonmalignant pain. Arch Intern Med. 2011 Apr
11;171(7):686–91.
9. Chou R, Turner JA, Devine EB, Hansen RN, Sullivan SD, Blazina I, et al. The
effectiveness and risks of long-term opioid therapy for chronic pain: a systematic review
for a National Institutes of Health Pathways to Prevention Workshop. Ann Intern Med.
2015 Feb 17;162(4):276–86.
10. Dhalla IA, Persaud N, Juurlink DN. Facing up to the prescription opioid crisis. BMJ.
2011;343:d5142.
11. Okie S. A Flood of Opioids, a Rising Tide of Deaths. N Engl J Med. 2010;363(21):1981–
5.
12. Dowell D, Haegerich TM, Chou R. CDC Guideline for Prescribing Opioids for Chronic
Pain--United States, 2016. JAMA. 2016 Apr 19;315(15):1624–45.
85
13. Glajchen M. Chronic pain: treatment barriers and strategies for clinical practice. J Am
Board Fam Pract. 2001 May 1;14(3):211–8.
14. Khalid L, Liebschutz JM, Xuan Z, Dossabhoy S, Kim Y, Crooks D, et al. Adherence to
Prescription Opioid Monitoring Guidelines among Residents and Attending Physicians in
the Primary Care Setting. Pain Med. 2015 Mar 1;16(3):480–7.
15. McCracken LM, Boichat C, Eccleston C. Training for General Practitioners in Opioid
Prescribing for Chronic Pain Based on Practice Guidelines: A Randomized Pilot and
Feasibility Trial. J Pain. 2012 Jan 1;13(1):32–40.
16. Morasco BJ, Duckart JP, Carr TP, Deyo RA, Dobscha SK. Clinical characteristics of
veterans prescribed high doses of opioid medications for chronic non-cancer pain. Pain.
2010 Dec;151(3):625–32.
17. Starrels JL, Becker WC, Alford DP, Kapoor A, Williams AR, Turner BJ. Systematic
review: treatment agreements and urine drug testing to reduce opioid misuse in patients
with chronic pain. Ann Intern Med. 2010 Jun 1;152(11):712–20.
18. Frank JW, Levy C, Matlock DD, Calcaterra SL, Mueller SR, Koester S, et al. Patients’
Perspectives on Tapering of Chronic Opioid Therapy: A Qualitative Study. Pain Med
Malden Mass. 2016;17(10):1838–47.
19. Hughes HK, Korthuis PT, Saha S, Eggly S, Sharp V, Cohn J, et al. A Mixed Methods
Study of Patient-Provider Communication about Opioid Analgesics. Patient Educ Couns.
2015 Apr;98(4):453–61.
20. Becker WC, Merlin JS, Manhapra A, Edens EL. Management of patients with issues
related to opioid safety, efficacy and/or misuse: a case series from an integrated,
interdisciplinary clinic. Addict Sci Clin Pract [Internet]. 2016 [cited 2018 Jul 20];11.
Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4730605/
21. Keller CE, Ashrafioun L, Neumann AM, Van Klein J, Fox CH, Blondell RD. Practices,
perceptions, and concerns of primary care physicians about opioid dependence associated
with the treatment of chronic pain. Subst Abuse Off Publ Assoc Med Educ Res Subst
Abuse. 2012;33(2):103–13.
22. Jamison RN, Sheehan KA, Scanlan E, Matthews M, Ross EL. Beliefs and attitudes about
opioid prescribing and chronic pain management: survey of primary care providers. J
Opioid Manag. 2014 Dec;10(6):375–82.
23. Pearson AC, Moman RN, Moeschler SM, Eldrige JS, Hooten WM. Provider confidence in
opioid prescribing and chronic pain management: results of the Opioid Therapy Provider
Survey. J Pain Res. 2017 Jun 7;10:1395–400.
24. Roy É, Côté RJ, Hamel D, Dubé P-A, Langlois É, Labesse ME, et al. Opioid Prescribing
Practices and Training Needs of Québec Family Physicians for Chronic Noncancer Pain.
Pain Res Manag [Internet]. 2017 [cited 2018 Jul 20];2017. Available from:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5555017/
86
25. Giannitrapani KF, Ahluwalia SC, McCaa M, Pisciotta M, Dobscha S, Lorenz KA. Barriers
to Using Nonpharmacologic Approaches and Reducing Opioid Use in Primary Care. Pain
Med Malden Mass. 2017 Oct 20;
26. Kavukcu E, Akdeniz M, Avci HH, Altuğ M, Öner M. Chronic noncancer pain
management in primary care: family medicine physicians’ risk assessment of opioid
misuse. Postgrad Med. 2015 Jan;127(1):22–6.
27. Jia P, Zhang L, Chen J, Zhao P, Zhang M. The Effects of Clinical Decision Support
Systems on Medication Safety: An Overview. PLOS ONE. 2016 Dec
15;11(12):e0167683.
28. Sahota N, Lloyd R, Ramakrishna A, Mackay JA, Prorok JC, Weise-Kelly L, et al.
Computerized clinical decision support systems for acute care management: A decision-
maker-researcher partnership systematic review of effects on process of care and patient
outcomes. Implement Sci IS. 2011 Aug 3;6:91.
29. Roshanov PS, Misra S, Gerstein HC, Garg AX, Sebaldt RJ, Mackay JA, et al.
Computerized clinical decision support systems for chronic disease management: A
decision-maker-researcher partnership systematic review. Implement Sci IS. 2011 Aug
3;6:92.
30. Moja L, Kwag KH, Lytras T, Bertizzolo L, Brandt L, Pecoraro V, et al. Effectiveness of
Computerized Decision Support Systems Linked to Electronic Health Records: A
Systematic Review and Meta-Analysis. Am J Public Health. 2014 Dec;104(12):e12–22.
31. Jaspers MWM, Smeulers M, Vermeulen H, Peute LW. Effects of clinical decision-support
systems on practitioner performance and patient outcomes: a synthesis of high-quality
systematic review findings. J Am Med Inform Assoc JAMIA. 2011;18(3):327–34.
32. Nieuwlaat R, Connolly SJ, Mackay JA, Weise-Kelly L, Navarro T, Wilczynski NL, et al.
Computerized clinical decision support systems for therapeutic drug monitoring and
dosing: A decision-maker-researcher partnership systematic review. Implement Sci IS.
2011 Aug 3;6:90.
33. Souza NM, Sebaldt RJ, Mackay JA, Prorok JC, Weise-Kelly L, Navarro T, et al.
Computerized clinical decision support systems for primary preventive care: A decision-
maker-researcher partnership systematic review of effects on process of care and patient
outcomes. Implement Sci IS. 2011 Aug 3;6:87.
34. Hemens BJ, Holbrook A, Tonkin M, Mackay JA, Weise-Kelly L, Navarro T, et al.
Computerized clinical decision support systems for drug prescribing and management: a
decision-maker-researcher partnership systematic review. Implement Sci IS. 2011;6:89.
35. Bright TJ, Wong A, Dhurjati R, Bristow E, Bastian L, Coeytaux RR, et al. Effect of
clinical decision-support systems: a systematic review. Ann Intern Med. 2012 Jul
3;157(1):29–43.
87
36. Garg AX, Adhikari NKJ, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J, et
al. Effects of computerized clinical decision support systems on practitioner performance
and patient outcomes: a systematic review. JAMA. 2005 Mar 9;293(10):1223–38.
37. Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving clinical practice using
clinical decision support systems: a systematic review of trials to identify features critical
to success. BMJ. 2005 Apr 2;330(7494):765.
38. Roshanov PS, Fernandes N, Wilczynski JM, Hemens BJ, You JJ, Handler SM, et al.
Features of effective computerised clinical decision support systems: meta-regression of
162 randomised trials. BMJ. 2013 Feb 14;346:f657.
39. Holstiege J, Mathes T, Pieper D. Effects of computer-aided clinical decision support
systems in improving antibiotic prescribing by primary care providers: a systematic
review. J Am Med Inform Assoc JAMIA. 2015 Jan;22(1):236–42.
40. Stultz JS, Nahata MC. Computerized clinical decision support for medication prescribing
and utilization in pediatrics. J Am Med Inform Assoc JAMIA. 2012;19(6):942–53.
41. Moxey A, Robertson J, Newby D, Hains I, Williamson M, Pearson S-A. Computerized
clinical decision support for prescribing: provision does not guarantee uptake. J Am Med
Inform Assoc JAMIA. 2010;17(1):25–33.
42. Patterson ES, Doebbeling BN, Fung CH, Militello L, Anders S, Asch SM. Identifying
barriers to the effective use of clinical reminders: bootstrapping multiple methods. J
Biomed Inform. 2005 Jun;38(3):189–99.
43. Arts DL, Medlock SK, van Weert HCPM, Wyatt JC, Abu-Hanna A. Acceptance and
barriers pertaining to a general practice decision support system for multiple clinical
conditions: A mixed methods evaluation. PLoS ONE [Internet]. 2018 Apr 19 [cited 2018
Jul 19];13(4). Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5908177/
44. Nanji KC, Slight SP, Seger DL, Cho I, Fiskio JM, Redden LM, et al. Overrides of
medication-related clinical decision support alerts in outpatients. J Am Med Inform Assoc
JAMIA. 2014 Jun;21(3):487–91.
45. Dalleur O, Seger DL, Slight SP, Amato M, Eguale T, Nanji KC, et al. Inappropriate
overrides of age-related alerts in prescriber order entry. J Gen Intern Med. 2015
Apr;2):S189–90.
46. Slight SP, Beeler PE, Seger DL, Dalleur O, Amato M, Eguale T, et al. An evaluation of
drug-allergy interaction alert overrides in inpatients. J Gen Intern Med. 2015 Apr;2):S81.
47. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and
evaluating complex interventions: The new Medical Research Council guidance. Int J
Nurs Stud. 2013 May 1;50(5):587–92.
88
48. Andrew R, Derry S, Taylor RS, Straube S, Phillips CJ. The Costs and Consequences of
Adequately Managed Chronic Non-Cancer Pain and Chronic Neuropathic Pain. Pain
Pract. 2014 Jan 1;14(1):79–94.
49. Lynch ME. The need for a Canadian pain strategy. Pain Res Manag J Can Pain Soc.
2011;16(2):77–80.
50. Gaskin DJ, Richard P. The Economic Costs of Pain in the United States [Internet].
National Academies Press (US); 2011 [cited 2018 May 31]. Available from:
http://www.ncbi.nlm.nih.gov/books/NBK92521/
51. Treede R-D, Rief W, Barke A, Aziz Q, Bennett MI, Benoliel R, et al. A classification of
chronic pain for ICD-11. Pain. 2015 Jun;156(6):1003–7.
52. IASP. Classification of chronic pain, description of chronic pain syndromes and definition
of pain terms (2nd edition). H. Merskey, N. Bogduk, Eds. Washington: IASP Press; 1994.
53. Geneen LJ, Moore RA, Clarke C, Martin D, Colvin LA, Smith BH. Physical activity and
exercise for chronic pain in adults: an overview of Cochrane Reviews. Cochrane Database
Syst Rev. 2017 14;1:CD011279.
54. Busse JW, Craigie S, Juurlink DN, Buckley DN, Li W, Couban RJ, et al. Guideline for
opioid therapy and chronic noncancer pain. CMAJ Can Med Assoc J. 2017;189:E659–66.
55. Johannes CB, Le TK, Zhou X, Johnston JA, Dworkin RH. The prevalence of chronic pain
in United States adults: results of an Internet-based survey. J Pain Off J Am Pain Soc.
2010 Nov;11(11):1230–9.
56. Nickel R, Raspe HH. [Chronic pain: epidemiology and health care utilization].
Nervenarzt. 2001 Dec;72(12):897–906.
57. Breivik H, Collett B, Ventafridda V, Cohen R, Gallacher D. Survey of chronic pain in
Europe: Prevalence, impact on daily life, and treatment. Eur J Pain. 2006 May
1;10(4):287–287.
58. Fayaz A, Croft P, Langford RM, Donaldson LJ, Jones GT. Prevalence of chronic pain in
the UK: a systematic review and meta-analysis of population studies. BMJ Open
[Internet]. 2016 May 25 [cited 2018 May 18];6(6). Available from:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4932255/
59. Colloca L, Ludman T, Bouhassira D, Baron R, Dickenson AH, Yarnitsky D, et al.
Neuropathic pain. Nat Rev Dis Primer. 2017 Feb 16;3:17002.
60. Reid KJ, Harker J, Bala MM, Truyers C, Kellen E, Bekkering GE, et al. Epidemiology of
chronic non-cancer pain in Europe: narrative review of prevalence, pain treatments and
pain impact. Curr Med Res Opin. 2011 Feb 1;27(2):449–62.
61. Eriksen J, Jensen MK, Sjøgren P, Ekholm O, Rasmussen NK. Epidemiology of chronic
non-malignant pain in Denmark. Pain. 2003 Dec;106(3):221–8.
89
62. Steingrímsdóttir ÓA, Landmark T, Macfarlane GJ, Nielsen CS. Defining chronic pain in
epidemiological studies: a systematic review and meta-analysis. Pain. 2017
Nov;158(11):2092–107.
63. Rashiq S, Dick BD. Factors associated with chronic noncancer pain in the Canadian
population. Pain Res Manag J Can Pain Soc. 2009;14(6):454–60.
64. Currow DC, Agar M, Plummer JL, Blyth FM, Abernethy AP. Chronic pain in South
Australia – population levels that interfere extremely with activities of daily living. Aust N
Z J Public Health. 2010 Jun 1;34(3):232–9.
65. Abdulla A, Adams N, Bone M, Elliott AM, Gaffin J, Jones D, et al. Guidance on the
management of pain in older people. Age Ageing. 2013 Mar;42 Suppl 1:i1-57.
66. Schofield P, Clarke A, Jones D, Martin D, McNamee P, Smith B. Chronic pain in later
life: a review of current issues and challenges. Aging Health. 2011 Aug 1;7(4):551–6.
67. Elliott AM, Smith BH, Penny KI, Smith WC, Chambers WA. The epidemiology of
chronic pain in the community. Lancet Lond Engl. 1999 Oct 9;354(9186):1248–52.
68. van Hecke O, Torrance N, Smith BH. Chronic pain epidemiology and its clinical
relevance. Br J Anaesth. 2013 Jul;111(1):13–8.
69. Schopflocher D, Taenzer P, Jovey R. The prevalence of chronic pain in Canada. Pain Res
Manag J Can Pain Soc. 2011;16(6):445–50.
70. Elzahaf RA, Tashani OA, Unsworth BA, Johnson MI. The prevalence of chronic pain with
an analysis of countries with a Human Development Index less than 0.9: a systematic
review without meta-analysis. Curr Med Res Opin. 2012 Jul 1;28(7):1221–9.
71. Mansfield KE, Sim J, Jordan JL, Jordan KP. A systematic review and meta-analysis of the
prevalence of chronic widespread pain in the general population. Pain. 2016
Jan;157(1):55–64.
72. Reitsma, ML. The prevalence of chronic pain and pain-related interference in the
Canadian population from 1994 to 2008 - Chronic Diseases and Injuries in Canada -
Public Health Agency of Canada [Internet]. 2011 [cited 2016 Oct 30]. Available from:
http://www.phac-aspc.gc.ca/publicat/hpcdp-pspmc/31-4/ar-04-eng.php
73. Pergolizzi J, Ahlbeck K, Aldington D, Alon E, Coluzzi F, Dahan A, et al. The
development of chronic pain: physiological CHANGE necessitates a multidisciplinary
approach to treatment. Curr Med Res Opin. 2013 Sep;29(9):1127–35.
74. Anwar K. Pathophysiology of pain. Dis--Mon DM. 2016 Sep;62(9):324–9.
75. Feizerfan A, Sheh G. Transition from acute to chronic pain. Contin Educ Anaesth Crit
Care Pain. 2015 Apr 1;15(2):98–102.
90
76. Gatchel RJ, Bevers K, Licciardone JC, Su J, Du Y, Brotto M. Transitioning from Acute to
Chronic Pain: An Examination of Different Trajectories of Low-Back Pain. Healthc Basel
Switz. 2018 May 17;6(2).
77. Teasell RW. Pathophysiology of Chronic Pain Disorders. Clin J Pain. 2001 Dec;17(4):S8.
78. Leadley RM, Armstrong N, Reid KJ, Allen A, Misso KV, Kleijnen J. Healthy aging in
relation to chronic pain and quality of life in Europe. Pain Pract Off J World Inst Pain.
2014 Jul;14(6):547–58.
79. McDermott AM, Toelle TR, Rowbotham DJ, Schaefer CP, Dukes EM. The burden of
neuropathic pain: results from a cross-sectional survey. Eur J Pain Lond Engl. 2006
Feb;10(2):127–35.
80. Spenkelink CD, Hutten MM, Hermens HJ, Greitemann BO. Assessment of activities of
daily living with an ambulatory monitoring system: a comparative study in patients with
chronic low back pain and nonsymptomatic controls. Clin Rehabil. 2002 Feb 1;16(1):16–
26.
81. Hogan M-E, Taddio A, Katz J, Shah V, Krahn M. Incremental health care costs for
chronic pain in Ontario, Canada: a population-based matched cohort study of adolescents
and adults using administrative data. Pain. 2016;157(8):1626–33.
82. Krebs EE, Gravely A, Nugent S, Jensen AC, DeRonne B, Goldsmith ES, et al. Effect of
Opioid vs Nonopioid Medications on Pain-Related Function in Patients With Chronic
Back Pain or Hip or Knee Osteoarthritis Pain: The SPACE Randomized Clinical Trial.
JAMA. 2018 Mar 6;319(9):872–82.
83. Enke O, New HA, New CH, Mathieson S, McLachlan AJ, Latimer J, et al.
Anticonvulsants in the treatment of low back pain and lumbar radicular pain: a systematic
review and meta-analysis. CMAJ Can Med Assoc J J Assoc Medicale Can. 2018 Jul
3;190(26):E786–93.
84. Whiting PF, Wolff RF, Deshpande S, Nisio MD, Duffy S, Hernandez AV, et al.
Cannabinoids for Medical Use: A Systematic Review and Meta-analysis. JAMA. 2015 Jun
23;313(24):2456–73.
85. Allan GM, Ramji J, Perry D, Ton J, Beahm NP, Crisp N, et al. Simplified guideline for
prescribing medical cannabinoids in primary care. Can Fam Physician. 2018
Feb;64(2):111–20.
86. Riediger C, Schuster T, Barlinn K, Maier S, Weitz J, Siepmann T. Adverse Effects of
Antidepressants for Chronic Pain: A Systematic Review and Meta-analysis. Front Neurol
[Internet]. 2017 Jul 14 [cited 2018 May 18];8. Available from:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5510574/
87. Chou R, Deyo R, Friedly J, Skelly A, Weimer M, Fu R, et al. Systemic Pharmacologic
Therapies for Low Back Pain: A Systematic Review for an American College of
Physicians Clinical Practice Guideline. Ann Intern Med. 2017 Apr 4;166(7):480–92.
91
88. Enthoven WTM, Roelofs PDDM, Deyo RA, van Tulder MW, Koes BW. Non-steroidal
anti-inflammatory drugs for chronic low back pain. Cochrane Database Syst Rev. 2016
Feb 10;2:CD012087.
89. Shanthanna H, Gilron I, Rajarathinam M, AlAmri R, Kamath S, Thabane L, et al. Benefits
and safety of gabapentinoids in chronic low back pain: A systematic review and meta-
analysis of randomized controlled trials. PLoS Med. 2017 Aug;14(8):e1002369.
90. Ennis ZN, Dideriksen D, Vaegter HB, Handberg G, Pottegård A. Acetaminophen for
Chronic Pain: A Systematic Review on Efficacy. Basic Clin Pharmacol Toxicol. 2016
Mar;118(3):184–9.
91. Machado GC, Maher CG, Ferreira PH, Pinheiro MB, Lin C-WC, Day RO, et al. Efficacy
and safety of paracetamol for spinal pain and osteoarthritis: systematic review and meta-
analysis of randomised placebo controlled trials. BMJ. 2015 Mar 31;350:h1225.
92. Finnerup NB, Attal N, Haroutounian S, McNicol E, Baron R, Dworkin RH, et al.
Pharmacotherapy for neuropathic pain in adults: systematic review, meta-analysis and
updated NeuPSIG recommendations. Lancet Neurol. 2015 Feb;14(2):162–73.
93. Moulin D, Boulanger A, Clark A, Clarke H, Dao T, Finley G, et al. Pharmacological
management of chronic neuropathic pain: Revised consensus statement from the Canadian
Pain Society. Pain Res Manag J Can Pain Soc. 2014;19(6):328–35.
94. Duehmke RM, Derry S, Wiffen PJ, Bell RF, Aldington D, Moore RA. Tramadol for
neuropathic pain in adults. Cochrane Database Syst Rev. 2017 15;6:CD003726.
95. Nugent SM, Morasco BJ, O’Neil ME, Freeman M, Low A, Kondo K, et al. The Effects of
Cannabis Among Adults With Chronic Pain and an Overview of General Harms: A
Systematic Review. Ann Intern Med. 2017 Sep 5;167(5):319.
96. Dosenovic S, Jelicic Kadic A, Miljanović M, Biočić M, Boric K, Cavar M, et al.
Interventions for Neuropathic Pain: An Overview of Systematic Reviews. Anesth Analg.
2017 Aug 1;125:643–52.
97. Parr AT, Manchikanti L, Hameed H, Conn A, Manchikanti KN, Benyamin RM, et al.
Caudal epidural injections in the management of chronic low back pain: a systematic
appraisal of the literature. Pain Physician. 2012 Jun;15(3):E159-198.
98. Kaye AD, Manchikanti L, Abdi S, Atluri S, Bakshi S, Benyamin R, et al. Efficacy of
Epidural Injections in Managing Chronic Spinal Pain: A Best Evidence Synthesis. Pain
Physician. 2015 Nov;18(6):E939-1004.
99. Hoffman EM, Watson JC, St Sauver J, Staff NP, Klein CJ. Association of Long-term
Opioid Therapy With Functional Status, Adverse Outcomes, and Mortality Among
Patients With Polyneuropathy. JAMA Neurol. 2017 Jul 1;74(7):773–9.
92
100. Eccleston C, Crombez G. Advancing psychological therapies for chronic pain.
F1000Research [Internet]. 2017 Apr 11 [cited 2018 Jun 6];6. Available from:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5389407/
101. Pike A, Hearn L, Williams AC de C. Effectiveness of psychological interventions for
chronic pain on health care use and work absence: systematic review and meta-analysis.
Pain. 2016 Apr;157(4):777–85.
102. Hilton L, Hempel S, Ewing BA, Apaydin E, Xenakis L, Newberry S, et al. Mindfulness
Meditation for Chronic Pain: Systematic Review and Meta-analysis. Ann Behav Med Publ
Soc Behav Med. 2017 Apr;51(2):199–213.
103. Hughes LS, Clark J, Colclough JA, Dale E, McMillan D. Acceptance and Commitment
Therapy (ACT) for Chronic Pain: A Systematic Review and Meta-Analyses. Clin J Pain.
2017 Jun;33(6):552–68.
104. Veehof MM, Trompetter HR, Bohlmeijer ET, Schreurs KMG. Acceptance- and
mindfulness-based interventions for the treatment of chronic pain: a meta-analytic review.
Cogn Behav Ther. 2016 Jan 2;45(1):5–31.
105. Eccleston C, Fisher E, Craig L, Duggan GB, Rosser BA, Keogh E. Psychological
therapies (Internet-delivered) for the management of chronic pain in adults. Cochrane
Database Syst Rev. 2014 Feb 26;(2):CD010152.
106. Eccleston C, Hearn L, Williams AC de C. Psychological therapies for the management of
chronic neuropathic pain in adults. Cochrane Database Syst Rev. 2015 Oct
29;(10):CD011259.
107. Chou R, Deyo R, Friedly J, Skelly A, Hashimoto R, Weimer M, et al. Nonpharmacologic
Therapies for Low Back Pain: A Systematic Review for an American College of
Physicians Clinical Practice Guideline. Ann Intern Med. 2017 Apr 4;166(7):493–505.
108. Jordan JL, Holden MA, Mason EE, Foster NE. Interventions to improve adherence to
exercise for chronic musculoskeletal pain in adults. Cochrane Database Syst Rev
[Internet]. 2010 [cited 2018 Oct 5];(1). Available from:
http://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD005956.pub2/abstract
109. Vickers AJ, Linde K. Acupuncture for chronic pain. JAMA J Am Med Assoc. 2014 Mar
5;311(9):955–6.
110. Kamper SJ, Apeldoorn AT, Chiarotto A, Smeets RJEM, Ostelo RWJG, Guzman J, et al.
Multidisciplinary biopsychosocial rehabilitation for chronic low back pain: Cochrane
systematic review and meta-analysis. BMJ. 2015 Feb 18;350:h444.
111. Upshur CC, Luckmann RS, Savageau JA. Primary Care Provider Concerns about
Management of Chronic Pain in Community Clinic Populations. J Gen Intern Med. 2006
Jun;21(6):652–5.
93
112. Frantsve LME, Kerns RD. Patient–Provider Interactions in the Management of Chronic
Pain: Current Findings within the Context of Shared Medical Decision Making. Pain Med.
2007 Jan 1;8(1):25–35.
113. Green CR, Wheeler JRC, LaPorte F, Marchant B, Guerrero E. How well is chronic pain
managed? Who does it well? Pain Med Malden Mass. 2002 Mar;3(1):56–65.
114. The Canadian Pain Coalition. THE PAINFUL TRUTH The State of Pain Management in
Canada [Internet]. 2014. Available from: https://s3.amazonaws.com/tld-
documents.llnassets.com/0004000/4657/the%20painful%20truth.pdf
115. Breuer B, Cruciani R, Portenoy RK. Pain management by primary care physicians, pain
physicians, chiropractors, and acupuncturists: a national survey. South Med J. 2010
Aug;103(8):738–47.
116. Ponte CD, Johnson-Tribino J. Attitudes and knowledge about pain: an assessment of West
Virginia family physicians. Fam Med. 2005 Aug;37(7):477–80.
117. Lalonde L, Leroux-Lapointe V, Choinière M, Martin E, Lussier D, Berbiche D, et al.
Knowledge, attitudes and beliefs about chronic noncancer pain in primary care: A
Canadian survey of physicians and pharmacists. Pain Res Manag J Can Pain Soc.
2014;19(5):241–50.
118. Mezei L, Murinson BB, Johns Hopkins Pain Curriculum Development Team. Pain
education in North American medical schools. J Pain Off J Am Pain Soc. 2011
Dec;12(12):1199–208.
119. Shipton EE, Bate F, Garrick R, Steketee C, Shipton EA, Visser EJ. Systematic Review of
Pain Medicine Content, Teaching, and Assessment in Medical School Curricula
Internationally. Pain Ther. 2018 Jul 30;
120. Watt-Watson J, McGillion M, Hunter J, Choiniere M, Clark AJ, Dewar A, et al. A survey
of prelicensure pain curricula in health science faculties in Canadian universities. Pain Res
Manag. 2009 Dec;14(6):439–44.
121. Briggs EV, Battelli D, Gordon D, Kopf A, Ribeiro S, Puig MM, et al. Current pain
education within undergraduate medical studies across Europe: Advancing the Provision
of Pain Education and Learning (APPEAL) study. BMJ Open. 2015 Aug 10;5(8):e006984.
122. Seal K, Becker W, Tighe J, Li Y, Rife T. Managing Chronic Pain in Primary Care: It
Really Does Take a Village. J Gen Intern Med. 2017 Aug 1;32(8):931–4.
123. Lalonde L, Choinière M, Martin E, Lévesque L, Hudon E, Bélanger D, et al. Priority
interventions to improve the management of chronic non-cancer pain in primary care: a
participatory research of the ACCORD program. J Pain Res. 2015 Apr 30;8:203–15.
124. Hadi MA, Alldred DP, Briggs M, Marczewski K, Closs SJ. “Treated as a number, not
treated as a person”: a qualitative exploration of the perceived barriers to effective pain
management of patients with chronic pain. BMJ Open. 2017 Jun 12;7(6):e016454.
94
125. Becker WC, Dorflinger L, Edmond SN, Islam L, Heapy AA, Fraenkel L. Barriers and
facilitators to use of non-pharmacological treatments in chronic pain. BMC Fam Pract
[Internet]. 2017 Mar 20 [cited 2018 Oct 3];18. Available from:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5359906/
126. Yanni LM, McKinney-Ketchum JL, Harrington SB, Huynh C, Amin, BS S, Matsuyama R,
et al. Preparation, Confidence, and Attitudes About Chronic Noncancer Pain in Graduate
Medical Education. J Grad Med Educ. 2010 Jun;2(2):260–8.
127. Van Zee A. The Promotion and Marketing of OxyContin: Commercial Triumph, Public
Health Tragedy. Am J Public Health. 2009 Feb;99(2):221–7.
128. Pathan H, Williams J. Basic opioid pharmacology: an update. Br J Pain. 2012
Feb;6(1):11–6.
129. National Academies of Sciences E, Division H and M, Policy B on HS, Abuse C on PM
and RS to APO, Phillips JK, Ford MA, et al. Pain Management and the Intersection of
Pain and Opioid Use Disorder [Internet]. National Academies Press (US); 2017 [cited
2018 Oct 5]. Available from: http://www.ncbi.nlm.nih.gov/books/NBK458655/
130. Kidner CL, Mayer TG, Gatchel RJ. Higher opioid doses predict poorer functional outcome
in patients with chronic disabling occupational musculoskeletal disorders. J Bone Joint
Surg Am. 2009 Apr;91(4):919–27.
131. Franklin GM, Stover BD, Turner JA, Fulton-Kehoe D, Wickizer TM, Disability Risk
Identification Study Cohort. Early opioid prescription and subsequent disability among
workers with back injuries: the Disability Risk Identification Study Cohort. Spine. 2008
Jan 15;33(2):199–204.
132. Busse JW, Ebrahim S, Heels-Ansdell D, Wang L, Couban R, Walter SD. Association of
worker characteristics and early reimbursement for physical therapy, chiropractic and
opioid prescriptions with workers’ compensation claim duration, for cases of acute low
back pain: an observational cohort study. BMJ Open. 2015 Aug 1;5(8):e007836.
133. Rivat C, Ballantyne J. The dark side of opioids in pain management: basic science
explains clinical observation. Pain Rep [Internet]. 2016 Sep 8 [cited 2018 Jul 12];1(2).
Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5741356/
134. Baron MJ, McDonald PW. Significant pain reduction in chronic pain patients after
detoxification from high-dose opioids. J Opioid Manag. 2006 Oct;2(5):277–82.
135. Vorobeychik Y, Chen L, Bush MC, Mao J. Improved opioid analgesic effect following
opioid dose reduction. Pain Med Malden Mass. 2008 Sep;9(6):724–7.
136. Hooten WM, Townsend CO, Sletten CD, Bruce BK, Rome JD. Treatment outcomes after
multidisciplinary pain rehabilitation with analgesic medication withdrawal for patients
with fibromyalgia. Pain Med Malden Mass. 2007 Feb;8(1):8–16.
95
137. Townsend CO, Kerkvliet JL, Bruce BK, Rome JD, Hooten WM, Luedtke CA, et al. A
longitudinal study of the efficacy of a comprehensive pain rehabilitation program with
opioid withdrawal: comparison of treatment outcomes based on opioid use status at
admission. Pain. 2008 Nov 15;140(1):177–89.
138. Sullivan MD, Edlund MJ, Fan M-Y, DeVries A, Braden JB, Martin BC. Trends in use of
opioids for non-cancer pain conditions 2000-2005 in commercial and Medicaid insurance
plans: The TROUP Study. Pain. 2008 Aug 31;138(2):440–9.
139. Chang H-Y, Daubresse M, Kruszewski SP, Alexander GC. Prevalence and treatment of
pain in EDs in the United States, 2000 to 2010. Am J Emerg Med. 2014 May 1;32(5):421–
31.
140. Mazer-Amirshahi M, Mullins PM, Rasooly I, van den Anker J, Pines JM. Rising opioid
prescribing in adult U.S. emergency department visits: 2001-2010. Acad Emerg Med Off J
Soc Acad Emerg Med. 2014 Mar;21(3):236–43.
141. Pezalla EJ, Rosen D, Erensen JG, Haddox JD, Mayne TJ. Secular trends in opioid
prescribing in the USA. J Pain Res. 2017;10:383–7.
142. Canadian Institute for Health Information. Pan-Canadian Trends in the Prescribing of
Opioids, 2012 to 2016 [Internet]. 2017. Available from:
https://secure.cihi.ca/free_products/pan-canadian-trends-opioid-prescribing-2017-en-
web.pdf
143. Barnett ML, Olenski AR, Jena AB. Opioid-Prescribing Patterns of Emergency Physicians
and Risk of Long-Term Use. N Engl J Med. 2017 Feb 16;376(7):663–73.
144. Tamayo-Sarver JH, Dawson NV, Cydulka RK, Wigton RS, Baker DW. Variability in
emergency physician decision making about prescribing opioid analgesics. Ann Emerg
Med. 2004 Apr;43(4):483–93.
145. Hoppe JA, McStay C, Sun BC, Capp R. Emergency Department Attending Physician
Variation in Opioid Prescribing in Low Acuity Back Pain. West J Emerg Med. 2017
Oct;18(6):1135–42.
146. Dhalla IA, Mamdani MM, Gomes T, Juurlink DN. Clustering of opioid prescribing and
opioid-related mortality among family physicians in Ontario. Can Fam Physician. 2011
Mar 1;57(3):e92–6.
147. Gomes T, Mamdani MM, Paterson JM, Dhalla IA, Juurlink DN. Trends in high-dose
opioid prescribing in Canada. Can Fam Physician. 2014 Sep 1;60(9):826–32.
148. Zin C, Chen L-C, Knaggs R. Changes in trends and pattern of strong opioid prescribing in
primary care. Eur J Pain Lond Engl. 2014 Oct;18(9):1343–51.
149. Schubert I, Ihle P, Sabatowski R. Increase in opiate prescription in Germany between
2000 and 2010: a study based on insurance data. Dtsch Arzteblatt Int. 2013 Jan;110(4):45–
51.
96
150. Hamunen K, Paakkari P, Kalso E. Trends in opioid consumption in the Nordic countries
2002-2006. Eur J Pain Lond Engl. 2009 Oct;13(9):954–62.
151. Hider-Mlynarz K, Cavalié P, Maison P. Trends in analgesic consumption in France over
the last 10 years and comparison of patterns across Europe. Br J Clin Pharmacol. 2018
Jun;84(6):1324–34.
152. Shei A, Hirst M, Kirson NY, Enloe CJ, Birnbaum HG, Dunlop WCN. Estimating the
health care burden of prescription opioid abuse in five European countries. Clin Outcomes
Res CEOR. 2015 Sep 15;7:477–88.
153. Blanch B, Pearson S-A, Haber PS. An overview of the patterns of prescription opioid use,
costs and related harms in Australia. Br J Clin Pharmacol. 2014 Nov;78(5):1159–66.
154. Fischer B, Keates A, Bühringer G, Reimer J, Rehm J. Non-medical use of prescription
opioids and prescription opioid-related harms: why so markedly higher in North America
compared to the rest of the world? Addict Abingdon Engl. 2014 Feb;109(2):177–81.
155. International Narcotic Control Board. Narcotic Drugs Technical report: Estimated World
Requirements for 2017 – Statistics for 2015. 2017.
156. Dhalla IA, Mamdani MM, Sivilotti MLA, Kopp A, Qureshi O, Juurlink DN. Prescribing
of opioid analgesics and related mortality before and after the introduction of long-acting
oxycodone. CMAJ Can Med Assoc J. 2009 Dec 8;181(12):891–6.
157. Morgan JP. American opiophobia: customary underutilization of opioid analgesics. Adv
Alcohol Subst Abuse. 1985 Fall-1986 Winter;5(1–2):163–73.
158. Terry CE. THE HARRISON ANTI-NARCOTIC ACT. Am J Public Health N Y N 1912.
1915 Jun;5(6):518.
159. Keefe PR. The Family That Built an Empire of Pain. The New Yorker [Internet]. 2017 Oct
23 [cited 2017 Oct 23]; Available from:
https://www.newyorker.com/magazine/2017/10/30/the-family-that-built-an-empire-of-
pain
160. United States General Accounting Office. Report to Congressional Requesters.
Prescription Drugs: OxyContin Abuse and Diversion and Efforts to Address the Problem.
Washington, DC: Publication GAO-04-110; 2003.
161. Meier B. Origins of an Epidemic: Purdue Pharma Knew Its Opioids Were Widely Abused.
The New York Times [Internet]. 2018 May 29 [cited 2018 Jun 6]; Available from:
https://www.nytimes.com/2018/05/29/health/purdue-opioids-oxycontin.html
162. The Ontario Drug Policy Research Network. Opioid Use and Related Adverse Events in
Ontario [Internet]. 2016 [cited 2018 Jul 16]. Available from: http://odprn.ca/wp-
content/uploads/2016/11/ODPRN-Opioid-Use-and-Related-Adverse-Events-Nov-
2016.pdf
97
163. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders
[Internet]. American Psychiatric Association; 2013 [cited 2014 Dec 8]. 1. (DSM Library).
Available from:
http://dsm.psychiatryonline.org/doi/book/10.1176/appi.books.9780890425596
164. Edlund MJ, Martin BC, Russo JE, DeVries A, Braden JB, Sullivan MD. The role of opioid
prescription in incident opioid abuse and dependence among individuals with chronic
noncancer pain: the role of opioid prescription. Clin J Pain. 2014 Jul;30(7):557–64.
165. Bohnert ASB, Valenstein M, Bair MJ, Ganoczy D, McCarthy JF, Ilgen MA, et al.
Association between opioid prescribing patterns and opioid overdose-related deaths.
JAMA. 2011 Apr 6;305(13):1315–21.
166. Kaplovitch E, Gomes T, Camacho X, Dhalla IA, Mamdani MM, Juurlink DN. Sex
Differences in Dose Escalation and Overdose Death during Chronic Opioid Therapy: A
Population-Based Cohort Study. PLoS ONE. 2015 Aug 20;10(8):e0134550.
167. Seth P. Overdose Deaths Involving Opioids, Cocaine, and Psychostimulants — United
States, 2015–2016. MMWR Morb Mortal Wkly Rep [Internet]. 2018 [cited 2018 Apr
23];67. Available from: https://www.cdc.gov/mmwr/volumes/67/wr/mm6712a1.htm
168. Rudd RA, Seth P, David F, Scholl L. Increases in Drug and Opioid-Involved Overdose
Deaths - United States, 2010-2015. MMWR Morb Mortal Wkly Rep. 2016 Dec
30;65(5051):1445–52.
169. Government of Canada. National Report: Apparent opioid-related deaths in Canada
[Internet]. 2018 [cited 2018 Apr 23]. Available from: www.canada.ca/en/public-
health/services/publications/healthy-living/national-report-apparent-opioid-related-deaths-
released-march-2018.html
170. Han B, Compton WM, Jones CM, Cai R. Nonmedical Prescription Opioid Use and Use
Disorders Among Adults Aged 18 Through 64 Years in the United States, 2003-2013.
JAMA. 2015 Oct 13;314(14):1468–78.
171. American Society of Addiction Medicine. Opioid Addiction 2016 Facts and Figures
[Internet]. [cited 2018 Apr 23]. Available from: https://www.asam.org/docs/default-
source/advocacy/opioid-addiction-disease-facts-figures.pdf
172. Dasgupta N, Beletsky L, Ciccarone D. Opioid Crisis: No Easy Fix to Its Social and
Economic Determinants. Am J Public Health. 2018 Feb;108(2):182–6.
173. Gomes T, Khuu W, Martins D, Tadrous M, Mamdani MM, Paterson JM, et al.
Contributions of prescribed and non-prescribed opioids to opioid related deaths:
population based cohort study in Ontario, Canada. BMJ. 2018 Aug 29;362:k3207.
174. Rubin R. Illicit Fentanyl Driving Opioid Overdose Deaths. JAMA. 2017 Dec
12;318(22):2174–2174.
98
175. Ciccarone D. Fentanyl in the US heroin supply: A rapidly changing risk environment. Int J
Drug Policy. 2017 Aug 1;46:107–11.
176. Baldwin N, Gray R, Goel A, Wood E, Buxton JA, Rieb LM. Fentanyl and heroin
contained in seized illicit drugs and overdose-related deaths in British Columbia, Canada:
An observational analysis. Drug Alcohol Depend. 2018 Apr 1;185:322–7.
177. Dowell D, Noonan RK, Houry D. Underlying Factors in Drug Overdose Deaths. JAMA.
2017 Dec 19;318(23):2295–6.
178. Prekupec MP, Mansky PA, Baumann MH. Misuse of Novel Synthetic Opioids: A Deadly
New Trend. J Addict Med. 2017 Jul;11(4):256–65.
179. O’Donnell J, Gladden RM, Seth P. Trends in Deaths Involving Heroin and Synthetic
Opioids Excluding Methadone, and Law Enforcement Drug Product Reports, by Census
Region — United States, 2006–2015 [Internet]. 2017 [cited 2018 Oct 1]. Available from:
https://www-cdc-
gov.myaccess.library.utoronto.ca/mmwr/volumes/66/wr/mm6634a2.htm?s_cid=mm6634a
2_e
180. Tournebize J, Gibaja V, Muszczak A, Kahn J-P. Are Physicians Safely Prescribing
Opioids for Chronic Noncancer Pain? A Systematic Review of Current Evidence. Pain
Pract. 2016 Mar 1;16(3):370–83.
181. Morasco BJ, Duckart JP, Dobscha SK. Adherence to Clinical Guidelines for Opioid
Therapy for Chronic Pain in Patients with Substance Use Disorder. J Gen Intern Med.
2011 May 12;26(9):965.
182. Davis D, Davis ME, Jadad A, Perrier L, Rath D, Ryan D, et al. The case for knowledge
translation: shortening the journey from evidence to effect. BMJ. 2003 Jul
3;327(7405):33–5.
183. Grimshaw JM, Shirran L, Thomas R, Mowatt G, Fraser C, Bero L, et al. Changing
provider behavior: an overview of systematic reviews of interventions. Med Care. 2001
Aug;39(8 Suppl 2):II2-45.
184. Kahan M, Wilson L, Midmer D, Borsoi D, Martin D. Randomized controlled trial on the
effects of a skills-based workshop on medical students’ management of problem drinking
and alcohol dependence. Subst Abuse Off Publ Assoc Med Educ Res Subst Abuse. 2003
Mar;24(1):5–16.
185. Furlan AD, Carnide N, Irvin E, Eerd DV, Munhall C, Kim J, et al. A Systematic Review
of Strategies to Improve Appropriate use of Opioids and to Reduce Opioid use Disorder
and Deaths from Prescription Opioids. Can J Pain. 2018 Jun 18;0(ja):null.
186. Straus SE, Tetroe J, Graham ID. Knowledge Translation in Health Care. Second. Wiley
Blackwell BMJIbooks; 2013.
99
187. McGlynn EA, Asch SM, Adams J, Keesey J, Hicks J, DeCristofaro A, et al. The quality of
health care delivered to adults in the United States. N Engl J Med. 2003 Jun
26;348(26):2635–45.
188. Grol R, Grimshaw J. From best evidence to best practice: effective implementation of
change in patients’ care. The Lancet. 2003 Oct 11;362(9391):1225–30.
189. Grimshaw JM, Eccles MP, Lavis JN, Hill SJ, Squires JE. Knowledge translation of
research findings. Implement Sci IS. 2012 May 31;7:50.
190. Government of Canada CI of HR. Knowledge Translation at CIHR [Internet]. 2005 [cited
2018 Jul 19]. Available from: http://www.cihr-irsc.gc.ca/e/29418.html
191. Strifler L, Cardoso R, McGowan J, Cogo E, Nincic V, Khan PA, et al. Scoping review
identifies significant number of knowledge translation theories, models, and frameworks
with limited use. J Clin Epidemiol. 2018 Aug;100:92–102.
192. Graham ID, Logan J, Harrison MB, Straus SE, Tetroe J, Caswell W, et al. Lost in
knowledge translation: Time for a map?: J Contin Educ Health Prof. 2006;26(1):13–24.
193. Kitson A, Straus SE. The knowledge-to-action cycle: identifying the gaps. CMAJ Can
Med Assoc J. 2010 Feb 9;182(2):E73–7.
194. U.S. National Library of Medicine. Medical subject headings. National Institutes of
Health. 2012.
195. Mainz J. Defining and classifying clinical indicators for quality improvement. Int J Qual
Health Care J Int Soc Qual Health Care. 2003 Dec;15(6):523–30.
196. Stelfox HT, Straus SE. Measuring quality of care: considering measurement frameworks
and needs assessment to guide quality indicator development. J Clin Epidemiol. 2013 Dec
1;66(12):1320–7.
197. Perera R, Dowell T, Crampton P, Kearns R. Panning for gold: an evidence-based tool for
assessment of performance indicators in primary health care. Health Policy Amst Neth.
2007 Feb;80(2):314–27.
198. Orem JN, Mafigiri DK, Marchal B, Ssengooba F, Macq J, Criel B. Research, evidence and
policymaking: the perspectives of policy actors on improving uptake of evidence in health
policy development and implementation in Uganda. BMC Public Health. 2012 Feb
9;12:109.
199. Gagnon M-P, Attieh R, Ghandour EK, Légaré F, Ouimet M, Estabrooks CA, et al. A
Systematic Review of Instruments to Assess Organizational Readiness for Knowledge
Translation in Health Care. PLoS ONE [Internet]. 2014 Dec 4 [cited 2016 Jul 4];9(12).
Available from: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4256226/
200. Wensing M, Bosch M, Grol R. Developing and selecting interventions for translating
knowledge to action. CMAJ Can Med Assoc J. 2010 Feb 9;182(2):E85–8.
100
201. Straus SE, Tetroe J, Graham I. Defining knowledge translation. CMAJ Can Med Assoc J.
2009 Aug 4;181(3–4):165–8.
202. O’Brien MA, Rogers S, Jamtvedt G, Oxman AD, Odgaard‐Jensen J, Kristoffersen DT, et
al. Educational outreach visits: effects on professional practice and health care outcomes.
In: The Cochrane Library [Internet]. John Wiley & Sons, Ltd; 2007 [cited 2018 Jul 20].
Available from:
http://cochranelibrary.wiley.com/doi/10.1002/14651858.CD000409.pub2/full
203. Sismondo S. How to make opinion leaders and influence people. CMAJ Can Med Assoc J.
2015 Jul 14;187(10):759–60.
204. Flodgren G, Parmelli E, Doumit G, Gattellari M, O’Brien MA, Grimshaw J, et al. Local
opinion leaders: effects on professional practice and health care outcomes. Cochrane
Database Syst Rev. 2011 Aug 10;(8):CD000125.
205. Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, et al. Audit and
feedback: effects on professional practice and healthcare outcomes. In: Cochrane Database
of Systematic Reviews [Internet]. John Wiley & Sons, Ltd; 2012 [cited 2016 Mar 17].
Available from:
http://onlinelibrary.wiley.com.myaccess.library.utoronto.ca/doi/10.1002/14651858.CD000
259.pub3/abstract
206. Ivers NM, Sales A, Colquhoun H, Michie S, Foy R, Francis JJ, et al. No more ‘business as
usual’ with audit and feedback interventions: towards an agenda for a reinvigorated
intervention. Implement Sci. 2014;9:14.
207. Shay LA, Lafata JE. Where is the evidence? A systematic review of shared decision
making and patient outcomes. Med Decis Mak Int J Soc Med Decis Mak. 2015
Jan;35(1):114–31.
208. Langdown C, Peckham S. The use of financial incentives to help improve health
outcomes: is the quality and outcomes framework fit for purpose? A systematic review. J
Public Health Oxf Engl. 2014 Jun;36(2):251–8.
209. Scott A, Sivey P, Ait Ouakrim D, Willenberg L, Naccarella L, Furler J, et al. The effect of
financial incentives on the quality of health care provided by primary care physicians.
Cochrane Database Syst Rev. 2011 Sep 7;(9):CD008451.
210. Mandavia R, Mehta N, Schilder A, Mossialos E. Effectiveness of UK provider financial
incentives on quality of care: a systematic review. Br J Gen Pract J R Coll Gen Pract. 2017
Nov;67(664):e800–15.
211. Black AD, Car J, Pagliari C, Anandan C, Cresswell K, Bokun T, et al. The Impact of
eHealth on the Quality and Safety of Health Care: A Systematic Overview. PLoS Med
[Internet]. 2011 Jan 18 [cited 2018 Jul 18];8(1). Available from:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3022523/
101
212. Vernooij RWM, Willson M, Gagliardi AR. Characterizing patient-oriented tools that
could be packaged with guidelines to promote self-management and guideline adoption: a
meta-review. Implement Sci. 2016 Apr 14;11(1):52.
213. Gagliardi AR, Légaré F, Brouwers MC, Webster F, Badley E, Straus S. Patient-mediated
knowledge translation (PKT) interventions for clinical encounters: a systematic review.
Implement Sci IS [Internet]. 2016 Feb 29 [cited 2018 Jul 19];11. Available from:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4770686/
214. Leece P, Buchman DZ, Hamilton M, Timmings C, Shantharam Y, Moore J, et al.
Improving opioid safety practices in primary care: protocol for the development and
evaluation of a multifaceted, theory-informed pilot intervention for healthcare providers.
BMJ Open [Internet]. 2017 Apr 26 [cited 2018 Jul 20];7(4). Available from:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5719659/
215. Midboe AM, Lewis ET, Paik MC, Gallagher RM, Rosenberg JM, Goodman F, et al.
Measurement of adherence to clinical practice guidelines for opioid therapy for chronic
pain. Transl Behav Med. 2012 Mar;2(1):57–64.
216. Perera R, Moriarty H. Prescription Safety Pilot Project Report to the Health Quality and
Safety Commission [Internet]. Wellington School of Medicine and Health Sciences; 2012.
Available from: http://www.hqsc.govt.nz/assets/Other-Topics/QS-challenge-reports/A-
prescription-safety-pilot-project-Final-Report.pdf
217. Health Quality Ontario. Recommendations for Adoption: Opioid Prescribing for Chronic
Pain [Internet]. 2018 [cited 2018 Jul 20]. Available from:
http://www.hqontario.ca/Portals/0/documents/evidence/quality-standards/qs-opioid-
chronic-pain-recommendations-for-adoption-en.pdf
218. Harle CA, Bauer SE, Hoang HQ, Cook RL, Hurley RW, Fillingim RB. Decision support
for chronic pain care: how do primary care physicians decide when to prescribe opioids? a
qualitative study. BMC Fam Pract. 2015;16:48.
219. Lembke A. Why Doctors Prescribe Opioids to Known Opioid Abusers. N Engl J Med.
2012;367(17):1580–1.
220. Peters M, Godfrey C, McInerney P, Soares C, Khalil H, Parker D. The Joanna Briggs
Institute Reviewers’ Manual 2015: Methodology for JBI Scoping Reviews. 2015 Jan 1
[cited 2018 Feb 7]; Available from: https://espace.library.uq.edu.au/view/UQ:371443
221. Colquhoun HL, Levac D, O’Brien KK, Straus S, Tricco AC, Perrier L, et al. Scoping
reviews: time for clarity in definition, methods, and reporting. J Clin Epidemiol. 2014
Dec;67(12):1291–4.
222. Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated
methodologies. Health Inf Libr J. 2009 Jun;26(2):91–108.
102
223. Clarke M, Chalmers I. Discussion sections in reports of controlled trials published in
general medical journals: islands in search of continents? JAMA. 1998 Jul 15;280(3):280–
2.
224. Mulrow CD. The medical review article: state of the science. Ann Intern Med. 1987
Mar;106(3):485–8.
225. Oxman AD, Guyatt GH. The science of reviewing research. Ann N Y Acad Sci. 1993 Dec
31;703:125–33; discussion 133-134.
226. Antman EM, Lau J, Kupelnick B, Mosteller F, Chalmers TC. A comparison of results of
meta-analyses of randomized control trials and recommendations of clinical experts.
Treatments for myocardial infarction. JAMA. 1992 Jul 8;268(2):240–8.
227. Chalmers I. The Cochrane collaboration: preparing, maintaining, and disseminating
systematic reviews of the effects of health care. Ann N Y Acad Sci. 1993 Dec
31;703:156–63; discussion 163-165.
228. Montori VM, Saha S, Clarke M. A Call for Systematic Reviews. J Gen Intern Med. 2004
Dec;19(12):1240–1.
229. Higgins, Julian G Sally. Cochrane Handbook for Systematic Reviews of Interventions
Version 5.1.0 [Internet]. Cochrane Reviews; 2011 [cited 2018 Jun 8]. Available from:
http://handbook-5-1.cochrane.org/
230. Glass GV. Primary, Secondary, and Meta-Analysis of Research. Educ Res. 1976;5(10):3–
8.
231. Chalmers I, Hedges LV, Cooper H. A Brief History of Research Synthesis. Eval Health
Prof. 2002 Mar 1;25(1):12–37.
232. Chalmers I. Current Controlled Trials: an opportunity to help improve the quality of
clinical research. Curr Control Trials Cardiovasc Med. 2000;1(1):3–8.
233. Zarin DA, Tse T, Williams RJ, Rajakannan T. The Status of Trial Registration Eleven
Years after the ICMJE Policy. N Engl J Med. 2017 Jan 26;376(4):383–91.
234. Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc
Res Methodol. 2005 Feb 1;8(1):19–32.
235. Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, et al. PRISMA
Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. Ann Intern
Med [Internet]. 2018 Sep 4 [cited 2018 Sep 21]; Available from:
http://annals.org/article.aspx?doi=10.7326/M18-0850
236. Levac D, Colquhoun H, O’Brien KK. Scoping studies: advancing the methodology.
Implement Sci. 2010 Sep 20;5:69.
103
237. Anderson D, Zlateva I, Khatri K, Ciaburri N. Using health information technology to
improve adherence to opioid prescribing guidelines in primary care. Clin J Pain.
2015;31(6):573–9.
238. Malte CA, Berger D, Saxon AJ, Hagedorn HJ, Achtmeyer CE, Mariano AJ, et al.
Electronic Medical Record Alert Associated With Reduced Opioid and Benzodiazepine
Coprescribing in High-risk Veteran Patients. Med Care. 2018;56(2):171–8.
239. Patel S, Carmichael JM, Taylor JM, Bounthavong M, Higgins DT. Evaluating the Impact
of a Clinical Decision Support Tool to Reduce Chronic Opioid Dose and Decrease Risk
Classification in a Veteran Population. Ann Pharmacother. 2017 Oct
1;1060028017739388.
240. Sim I, Gorman P, Greenes RA, Haynes RB, Kaplan B, Lehmann H, et al. Clinical
Decision Support Systems for the Practice of Evidence-based Medicine. J Am Med Inform
Assoc JAMIA. 2001;8(6):527–34.
241. Haynes RB, Wilczynski NL. Effects of computerized clinical decision support systems on
practitioner performance and patient outcomes: Methods of a decision-maker-researcher
partnership systematic review. Implement Sci IS. 2010 Feb 5;5:12.
242. Moore M, Loper KA. An Introduction to Clinical Decision Support Systems. J Electron
Resour Med Libr. 2011 Oct 1;8(4):348–66.
243. Bates DW, Kuperman GJ, Wang S, Gandhi T, Kittler A, Volk L, et al. Ten
Commandments for Effective Clinical Decision Support: Making the Practice of
Evidence-based Medicine a Reality. J Am Med Inform Assoc JAMIA. 2003;10(6):523–
30.
244. Roshanov PS, Fernandes N, Wilczynski JM, Hemens BJ, You JJ, Handler SM, et al.
Features of effective computerised clinical decision support systems: meta-regression of
162 randomised trials. BMJ. 2013 Feb 14;346:f657.
245. Black AD, Car J, Pagliari C, Anandan C, Cresswell K, Bokun T, et al. The Impact of
eHealth on the Quality and Safety of Health Care: A Systematic Overview. PLoS Med
[Internet]. 2011 Jan 18 [cited 2018 Jul 18];8(1). Available from:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3022523/
246. Lugtenberg M, Pasveer D, van der Weijden T, Westert GP, Kool RB. Exposure to and
experiences with a computerized decision support intervention in primary care: results
from a process evaluation. BMC Fam Pract [Internet]. 2015 Oct 16 [cited 2018 Jun 20];16.
Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4608282/
247. Murphy DR, Reis B, Sittig DF, Singh H. Notifications received by primary care
practitioners in electronic health records: a taxonomy and time analysis. Am J Med. 2012
Feb;125(2):209.e1-7.
248. Liberati EG, Ruggiero F, Galuppo L, Gorli M, González-Lorenzo M, Maraldi M, et al.
What hinders the uptake of computerized decision support systems in hospitals? A
104
qualitative study and framework for implementation. Implement Sci. 2017 Sep
15;12(1):113.
249. Murphy EV. Clinical Decision Support: Effectiveness in Improving Quality Processes and
Clinical Outcomes and Factors That May Influence Success. Yale J Biol Med. 2014 Jun
6;87(2):187–97.
250. Greenhalgh T, Wherton J, Papoutsi C, Lynch J, Hughes G, A’Court C, et al. Beyond
Adoption: A New Framework for Theorizing and Evaluating Nonadoption, Abandonment,
and Challenges to the Scale-Up, Spread, and Sustainability of Health and Care
Technologies. J Med Internet Res. 2017;19(11):e367.
251. Ross J, Stevenson F, Lau R, Murray E. Factors that influence the implementation of e-
health: a systematic review of systematic reviews (an update). Implement Sci IS [Internet].
2016 Oct 26 [cited 2018 Jul 18];11. Available from:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5080780/
252. Ash JS, Sittig DF, Guappone KP, Dykstra RH, Richardson J, Wright A, et al.
Recommended practices for computerized clinical decision support and knowledge
management in community settings: a qualitative study. BMC Med Inform Decis Mak.
2012 Feb 14;12:6.
253. Trafton J, Martins S, Michel M, Lewis E, Wang D, Combs A, et al. Evaluation of the
acceptability and usability of a decision support system to encourage safe and effective
use of opioid therapy for chronic, noncancer pain by primary care providers. Pain Med.
2010;11:575–85.
254. Sinha S, Jensen M, Mullin S, Elkin PL. Safe Opioid Prescription: A SMART on FHIR
Approach to Clinical Decision Support. Online J Public Health Inform. 2017;9:e193.
255. Finley EP, Schneegans S, Tami C, Pugh MJ, McGeary D, Penney L, et al. Implementing
prescription drug monitoring and other clinical decision support for opioid risk mitigation
in a military health care setting: a qualitative feasibility study. J Am Med Inform Assoc
JAMIA. 2017 Aug 21;
256. Harle C, Cook R, Fillingim R. Informing clinical decision support for chronic pain in
primary care: How do physicians decide when to initiate opioids? J Pain. 2014
Apr;1):S87.
257. Lapoint J, Perrone J, Nelson LS. Electronic Pharmacopoeia: A Missed Opportunity for
Safe Opioid Prescribing Information? J Med Toxicol. 2014 Mar;10(1):15–8.
258. Furlan AD, Reardon R, Salach L. The opioid manager: a point-of-care tool to facilitate the
use of the Canadian Opioid Guideline. J Opioid Manag. 2012;8(1):57–61.
259. Midboe AM, Lewis ET, Cronkite RC, Chambers D, Goldstein MK, Kerns RD, et al.
Behavioral medicine perspectives on the design of health information technology to
improve decision-making, guideline adherence, and care coordination in chronic pain
management. Transl Behav Med. 2011;1:35–44.
105
260. Sproule BA. Prescription Monitoring Programs in Canada: Best Practice and Program
Review [Internet]. Canadian Centre on Substance Abuse, Ottawa, ON; 2015 [cited 2018
Jun 14]. Available from: http://www.ccsa.ca/Resource%20Library/CCSA-Prescription-
Monitoring-Programs-in-Canada-Report-2015-en.pdf
261. Rutkow L, Smith KC, Lai AY, Vernick JS, Davis CS, Alexander GC. Prescription drug
monitoring program design and function: A qualitative analysis. Drug Alcohol Depend.
2017;180:395–400.
262. Garcia AA, Rosen KD, Finley E, Potter JS. A systematic review of barriers and facilitators
to implementing a prescription drug monitoring program. Drug Alcohol Depend. 2017 Feb
1;171:e69.
263. Haffajee RL, Jena AB, Weiner SG. Mandatory Use of Prescription Drug Monitoring
Programs. JAMA. 2015 Mar 3;313(9):891–2.
264. Fink DS, Schleimer JP, Sarvet A, Grover KK, Delcher C, Castillo-Carniglia A, et al.
Association Between Prescription Drug Monitoring Programs and Nonfatal and Fatal
Drug Overdoses: A Systematic Review. Ann Intern Med. 2018 Jun 5;168(11):783.
265. Baehren DF, Marco CA, Droz DE, Sinha S, Callan EM, Akpunonu P. A statewide
prescription monitoring program affects emergency department prescribing behaviors.
Ann Emerg Med. 2010 Jul;56(1):19-23.e1-3.
266. Bryan C, Boren SA. The use and effectiveness of electronic clinical decision support tools
in the ambulatory/primary care setting: a systematic review of the literature. Inform Prim
Care. 2008;16(2):79–91.
267. Lainer M, Mann E, Sönnichsen A. Information technology interventions to improve
medication safety in primary care: a systematic review. Int J Qual Health Care. 2013 Oct
1;25(5):590–8.
268. Lugtenberg M, Weenink J-W, van der Weijden T, Westert GP, Kool RB. Implementation
of multiple-domain covering computerized decision support systems in primary care: a
focus group study on perceived barriers. BMC Med Inform Decis Mak [Internet]. 2015
Oct 12 [cited 2018 Jul 19];15. Available from:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4603732/
269. Lugtenberg M, Westert GP, Pasveer D, van der Weijden T, Kool RB. Evaluating the
uptake and effects of the computerized decision support system NHGDoc on quality of
primary care: protocol for a large-scale cluster randomized controlled trial. Implement Sci
IS [Internet]. 2014 Oct 17 [cited 2018 Jul 19];9. Available from:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4205280/
270. Levesque J-F, Haggerty JL, Hogg W, Burge F, Wong ST, Katz A, et al. Barriers and
Facilitators for Primary Care Reform in Canada: Results from a Deliberative Synthesis
across Five Provinces. Healthc Policy. 2015 Nov;11(2):44–57.
106
271. Hutchison B, Levesque J-F, Strumpf E, Coyle N. Primary Health Care in Canada: Systems
in Motion. Milbank Q. 2011 Jun;89(2):256–88.
272. Chang F, Gupta N. Progress in electronic medical record adoption in Canada. Can Fam
Physician. 2015 Dec;61(12):1076–84.
273. Stange KC. The Problem of Fragmentation and the Need for Integrative Solutions. Ann
Fam Med. 2009 Mar;7(2):100–3.
274. Green LA, Klinkman M. Perspectives in Primary Care: The Foundational Urgent
Importance of a Shared Primary Care Data Model. Ann Fam Med. 2015 Jul;13(4):303–11.
275. Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process
evaluations of complex interventions: UK Medical Research Council (MRC) guidance
[Internet]. MRC Population Health Science Research Network; 2014 [cited 2018 Jul 23].
Available from: https://mrc.ukri.org/documents/pdf/mrc-phsrn-process-evaluation-
guidance-final/
276. Rogers PJ. Using Programme Theory to Evaluate Complicated and Complex Aspects of
Interventions. Evaluation. 2008 Jan 1;14(1):29–48.
277. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and
evaluating complex interventions: the new Medical Research Council guidance. The BMJ
[Internet]. 2008 Sep 29 [cited 2018 Jul 26];337. Available from:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2769032/
278. Shiell A, Hawe P, Gold L. Complex interventions or complex systems? Implications for
health economic evaluation. BMJ. 2008 Jun 5;336(7656):1281–3.
279. Roland M, Torgerson DJ. Understanding controlled trials: What are pragmatic trials?
BMJ. 1998 Jan 24;316(7127):285.
280. Moore L, Hallingberg B, Wight D, Turley R, Segrott J, Craig P, et al. Exploratory studies
to inform full-scale evaluations of complex public health interventions: the need for
guidance. J Epidemiol Community Health. 2018 Oct 1;72(10):865–6.
281. Berwick DM. The Science of Improvement. JAMA. 2008 Mar 12;299(10):1182–4.
282. Brown CH, Curran G, Palinkas LA, Aarons GA, Wells KB, Jones L, et al. An Overview
of Research and Evaluation Designs for Dissemination and Implementation. Annu Rev
Public Health. 2017 Mar 20;38:1–22.
283. Saunders R, Evans M, Joshi P. Developing a Process-Evaluation Plan for Assessing
Health Promotion Program Implementation: A How-To Guide. Health Promot Pract.
2005;6(2):134–47.
284. Steckler A, Linnan L. Process Evaluation for Public Health Interventions and Research
[Internet]. Wiley; 2002 [cited 2018 Jul 28]. 440 p. Available from:
https://pdfs.semanticscholar.org/d833/108331e1fd9762850819b872253a626efc4d.pdf
107
285. Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process
evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015 Mar
19;350:h1258.
286. Durlak JA, DuPre EP. Implementation Matters: A Review of Research on the Influence of
Implementation on Program Outcomes and the Factors Affecting Implementation. Am J
Community Psychol. 2008;41(3–4):327.
287. Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health
promotion interventions: the RE-AIM framework. Am J Public Health. 1999
Sep;89(9):1322–7.
288. Montgomery P, Underhill K, Gardner F, Operario D, Mayo-Wilson E. The Oxford
Implementation Index: A new tool for incorporating implementation data into systematic
reviews and meta-analyses. J Clin Epidemiol. 2013 Aug;66(8):874–82.
289. Pawson R, Greenhalgh T, Harvey G, Walshe K. Realist review--a new method of
systematic review designed for complex policy interventions. J Health Serv Res Policy.
2005 Jul;10 Suppl 1:21–34.
290. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering
implementation of health services research findings into practice: a consolidated
framework for advancing implementation science. Implement Sci IS. 2009 Aug 7;4:50.
291. McEvoy R, Ballini L, Maltoni S, O’Donnell CA, Mair FS, MacFarlane A. A qualitative
systematic review of studies using the normalization process theory to research
implementation processes. Implement Sci IS. 2014 Jan 2;9:2.
292. Murray E, Treweek S, Pope C, MacFarlane A, Ballini L, Dowrick C, et al. Normalisation
process theory: a framework for developing, evaluating and implementing complex
interventions. BMC Med. 2010 Oct 20;8:63.
293. Nilsen P. Making sense of implementation theories, models and frameworks. Implement
Sci IS [Internet]. 2015 Apr 21 [cited 2017 Sep 15];10. Available from:
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4406164/
294. Hawe P, Shiell A, Riley T. Theorising interventions as events in systems. Am J
Community Psychol. 2009 Jun;43(3–4):267–76.
295. Willis K, Daly J, Kealy M, Small R, Koutroulis G, Green J, et al. The essential role of
social theory in qualitative public health research. Aust N Z J Public Health. 2007 Oct
1;31(5):438–43.
296. Rogers E. Diffusion of innovations. – Digital Journal [Internet]. 2003 [cited 2018 Jul 27].
Available from: https://kinasevych.ca/2010/03/15/rogers-2003-diffusion-of-innovations/
297. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in
service organizations: systematic review and recommendations. Milbank Q.
2004;82(4):581–629.
108
298. Holden R J, Karsh B-T. The Technology Acceptance Model: Its past and future in health
care. J Biomed Inform. 2010 Feb;43(1):159.
299. Simon SR, Kaushal R, Cleary PD, Jenter CA, Volk LA, Orav EJ, et al. Physicians and
Electronic Health Records: A Statewide Survey. Arch Intern Med. 2007 Mar
12;167(5):507–12.
300. Latour B. Reassembling the Social: An Introduction to Actor Network Theory. Oxford
University Press; 2005.
301. Cresswell KM, Worth A, Sheikh A. Actor-Network Theory and its role in understanding
the implementation of information technology developments in healthcare. BMC Med
Inform Decis Mak. 2010 Nov 1;10:67.
302. Pope C, Halford S, Turnbull J, Prichard J, Calestani M, May C. Using computer decision
support systems in NHS emergency and urgent care: ethnographic study using
normalisation process theory. BMC Health Serv Res. 2013 Mar 23;13:111.
303. May CR, Mair F, Finch T, MacFarlane A, Dowrick C, Treweek S, et al. Development of a
theory of implementation and integration: Normalization Process Theory. Implement Sci
IS. 2009 May 21;4:29.
304. May C, Finch T. Implementing, Embedding, and Integrating Practices: An Outline of
Normalization Process Theory. Sociology. 2009 Jun 1;43(3):535–54.
305. May CR, Cummings A, Girling M, Bracher M, Mair FS, May CM, et al. Using
Normalization Process Theory in feasibility studies and process evaluations of complex
healthcare interventions: a systematic review. Implement Sci IS [Internet]. 2018 Jun 7
[cited 2018 Jun 27];13. Available from:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5992634/
306. Bright TJ, Wong A, Dhurjati R, Bristow E, Bastian L, Coeytaux RR, et al. Effect of
clinical decision-support systems: a systematic review. Ann Intern Med. 2012 Jul
3;157(1):29–43.
307. Khong PCB, Holroyd E, Wang W. A Critical Review of the Theoretical Frameworks and
the Conceptual Factors in the Adoption of Clinical Decision Support Systems. Comput
Inform Nurs CIN. 2015 Dec;33(12):555–70.
308. Eslami Andargoli A, Scheepers H, Rajendran D, Sohal A. Health information systems
evaluation frameworks: A systematic review. Int J Med Inf. 2017 Jan 1;97:195–209.
309. Bluedorn AC, Evered R. Middle Range Theory and the Strategies of Theory Construction.
In: Middle Range Theory and the Study of Organizations [Internet]. Springer, Dordrecht;
1980 [cited 2018 Aug 1]. p. 19–32. Available from:
http://link.springer.com/chapter/10.1007/978-94-009-8733-3_2
109
310. May CR, Mair F, Finch T, MacFarlane A, Dowrick C, Treweek S, et al. Development of a
theory of implementation and integration: Normalization Process Theory. Implement Sci
IS. 2009 May 21;4:29.
311. May C, Harrison R, Finch T, MacFarlane A, Mair F, Wallace P. Understanding the
Normalization of Telemedicine Services through Qualitative Evaluation. J Am Med
Inform Assoc JAMIA. 2003;10(6):596–604.
312. May C. A rational model for assessing and evaluating complex interventions in health
care. BMC Health Serv Res. 2006 Jul 7;6:86.
313. Glaser B, Strauss A. The discovery of grounded theory: strategies for qualitative research.
Chicago: Aldine Publishing Company, Hawthorne, NY; 1967.
314. Holton J. Grounded theory as a general reserach methodology [Internet]. Grounded
Theory Review; 2008 [cited 2018 Jul 30]. Available from:
http://groundedtheoryreview.com/2008/06/30/grounded-theory-as-a-general-research-
methodology/
315. May C, Finch T, Mair F, Ballini L, Dowrick C, Eccles M, et al. Understanding the
implementation of complex interventions in health care: the normalization process model.
BMC Health Serv Res. 2007;7:148.
316. Mair FS, May C, O’Donnell C, Finch T, Sullivan F, Murray E. Factors that promote or
inhibit the implementation of e-health systems: an explanatory systematic review. Bull
World Health Organ. 2012 May 1;90(5):357–64.
317. Finch TL, Rapley T, Girling M, Mair FS, Murray E, Treweek S, et al. Improving the
normalization of complex interventions: measure development based on normalization
process theory (NoMAD): study protocol. Implement Sci IS. 2013 Apr 11;8:43.
318. Gale NK, Heath G, Cameron E, Rashid S, Redwood S. Using the framework method for
the analysis of qualitative data in multi-disciplinary health research. BMC Med Res
Methodol. 2013;13:117.
319. Pope C, Halford S, Turnbull J, Prichard J, Calestani M, May C. Using computer decision
support systems in NHS emergency and urgent care: ethnographic study using
normalisation process theory. BMC Health Serv Res. 2013 Mar 23;13(1):111.
320. Henderson EJ, Rubin GP. The utility of an online diagnostic decision support system
(Isabel) in general practice: a process evaluation. JRSM Short Rep. 2013 May 1;4(5):1–
11.
321. Elwyn G, Légaré F, Weijden T van der, Edwards A, May C. Arduous implementation:
Does the Normalisation Process Model explain why it’s so difficult to embed decision
support technologies for patients in routine clinical practice. Implement Sci IS. 2008 Dec
31;3:57.
110
322. Catwell L, Sheikh A. Evaluating eHealth Interventions: The Need for Continuous
Systemic Evaluation. PLoS Med [Internet]. 2009 Aug 18 [cited 2018 Sep 24];6(8).
Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2719100/
323. Li G, Abbade LPF, Nwosu I, Jin Y, Leenus A, Maaz M, et al. A scoping review of
comparisons between abstracts and full reports in primary biomedical research. BMC Med
Res Methodol. 2017 Dec 29;17(1):181.
324. Main C, Moxham T, Wyatt JC, Kay J, Anderson R, Stein K. Computerised decision
support systems in order communication for diagnostic, screening or monitoring test
ordering: systematic reviews of the effects and cost-effectiveness of systems [Internet].
NIHR Journals Library; 2010 [cited 2018 Jan 7]. Available from:
https://www.ncbi.nlm.nih.gov/books/NBK56829/
325. Grey Matters: a practical tool for searching health-related grey literature | CADTH.ca
[Internet]. [cited 2018 Sep 24]. Available from: https://www.cadth.ca/resources/finding-
evidence/grey-matters
326. Covidence - Better systematic review management [Internet]. [cited 2018 Sep 21].
Available from: https://www.covidence.org/about-us
327. Canada RE, DiRocco D, Day S. A better approach to opioid prescribing in primary care. J
Fam Pract. 2014;63(6):E1-8.
328. Patel S, Carmichael JM, Taylor JM, Bounthavong M, Higgins DT. Evaluating the Impact
of a Clinical Decision Support Tool to Reduce Chronic Opioid Dose and Decrease Risk
Classification in a Veteran Population. Ann Pharmacother. 2018;52:325–31.
329. Liebschutz JM, Xuan Z, Shanahan CW, LaRochelle M, Keosaian J, Beers D, et al.
Improving Adherence to Long-term Opioid Therapy Guidelines to Reduce Opioid Misuse
in Primary Care: A Cluster-Randomized Clinical Trial. JAMA Intern Med.
2017;177:1265–72.
330. Gugelmann H, Shofer FS, Meisel ZF, Perrone J. Multidisciplinary intervention decreases
the use of opioid medication discharge packs from 2 urban EDs. Am J Emerg Med. 2013
Sep 1;31(9):1343–8.
331. Lin HC, Wang Z, Boyd C, Simoni-Wastila L, Buu A. Associations between statewide
prescription drug monitoring program (PDMP) requirement and physician patterns of
prescribing opioid analgesics for patients with non-cancer chronic pain. Addict Behav.
2018;76:348–54.
332. Kohlbeck S, Akert B, Pace C, Zosel A. A multistep approach to address clinician
knowledge, attitudes, and behavior around opioid prescribing. Wis Med J. 2018;117:38–
41.
333. Click IA, Basden JA, Bohannon JM, Anderson H, Tudiver F. Opioid Prescribing in Rural
Family Practices: A Qualitative Study. Subst Use Misuse. 2018 Mar 21;53(4):533–40.
111
334. Coleman CD. Evidence-Based Strategies To Minimize Risk For Opioid Pain Medication
Misuse Among Patients With Chronic Pain In A Primary Care Setting. Diss Abstr Int Sect
B Sci Eng. 2016;64.
335. Chaudhary S, Compton P. Use of risk mitigation practices by family nurse practitioners
prescribing opioids for the management of chronic nonmalignant pain. Subst Abuse. 2017
Mar;38(1):95–104.
336. Grading quality of evidence and strength of recommendations. BMJ. 2004 Jun
19;328(7454):1490.
337. Arts DL, Abu-Hanna A, Medlock SK, van Weert HCPM. Effectiveness and usage of a
decision support system to improve stroke prevention in general practice: A cluster
randomized controlled trial. PLoS ONE [Internet]. 2017 Feb 28 [cited 2018 Jul 19];12(2).
Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5330455/
338. Nilsen P. Making sense of implementation theories, models and frameworks. Implement
Sci IS [Internet]. 2015 Apr 21 [cited 2018 Jan 7];10. Available from:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4406164/
339. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006 Jan
1;3(2):77–101.
340. Institute for Safe Medication Practices Canada. Safer Decisions Save Lives: Usability,
Functionality, and UtilityTesting of a Prototype Clinical Decision Support System to
Enhance Opioid Prescribing [Internet]. 2017. Available from: https://www.ismp-
canada.org/download/OpioidStewardship/SDSL.PCI.UsabilityTesting.pdf
341. About UTOPIAN [Internet]. Department of Family & Community Medicine. 2016 [cited
2018 Sep 18]. Available from: https://www.dfcm.utoronto.ca/about-utopian
342. Ando H, Cousins R, Young C. Achieving Saturation in Thematic Analysis: Development
and Refinement of a Codebook. Compr Psychol. 2014 Jan 1;3:03.CP.3.4.
343. Malterud K, Siersma VD, Guassora AD. Sample Size in Qualitative Interview Studies:
Guided by Information Power. Qual Health Res. 2016 Nov 1;26(13):1753–60.
344. Morse JM. Determining Sample Size. Qual Health Res. 2000 Jan 1;10(1):3–5.
345. Saunders B, Sim J, Kingstone T, Baker S, Waterfield J, Bartlam B, et al. Saturation in
qualitative research: exploring its conceptualization and operationalization. Qual Quant.
2017 Sep 14;1–15.
346. Boeije H. A Purposeful Approach to the Constant Comparative Method in the Analysis of
Qualitative Interviews. Qual Quant. 2002 Nov 1;36(4):391–409.
347. Coyne IT. Sampling in qualitative research. Purposeful and theoretical sampling; merging
or clear boundaries? J Adv Nurs. 1997 Sep 1;26(3):623–30.
112
348. Etikan I. Comparison of Convenience Sampling and Purposive Sampling. Am J Theor
Appl Stat. 2016;5(1):1.
349. Holtrop JS, Potworowski G, Fitzpatrick L, Kowalk A, Green LA. Effect of care
management program structure on implementation: a normalization process theory
analysis. BMC Health Serv Res. 2016 Aug 15;16:386.
350. Jones CHD, Glogowska M, Locock L, Lasserson DS. Embedding new technologies in
practice – a normalization process theory study of point of care testing. BMC Health Serv
Res [Internet]. 2016 Oct 19 [cited 2017 Jul 21];16. Available from:
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC5070078/
351. Fram SM. The Constant Comparative Analysis Method Outside of Grounded Theory.
Qual Rep Ft Lauderdale. 2013 Jan;18(1):1–25.
352. Thorne S. Data analysis in qualitative research. Evid Based Nurs. 2000 Jul 1;3(3):68–70.
353. McJunkin B, Riley MA, Lilly JK, Casto A, Bowe A. Approach to pain management in a
large outpatient clinic population. W V Med J. 2010;106(4 Spec No):72–8.
354. Wenghofer EF, Wilson L, Kahan M, Sheehan C, Srivastava A, Rubin A, et al. Survey of
Ontario primary care physicians’ experiences with opioid prescribing. Can Fam Physician
Med Fam Can. 2011 Mar;57(3):324–32.
355. Trafton JA, Martins SB, Michel MC, Wang D, Tu SW, Clark DJ, et al. Designing an
automated clinical decision support system to match clinical practice guidelines for opioid
therapy for chronic pain. Implement Sci IS. 2010 Apr 12;5:26.
356. Liberati EG, Ruggiero F, Galuppo L, Gorli M, González-Lorenzo M, Maraldi M, et al.
What hinders the uptake of computerized decision support systems in hospitals? A
qualitative study and framework for implementation. Implement Sci IS [Internet]. 2017
Sep 15 [cited 2017 Dec 4];12. Available from:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5602839/
357. Jamoom EW, Patel V, Furukawa MF, King J. EHR adopters vs. non-adopters: Impacts of,
barriers to, and federal initiatives for EHR adoption. Healthc Amst Neth. 2014
Mar;2(1):33–9.
358. Finley EP, Garcia A, Rosen K, McGeary D, Pugh MJ, Potter JS. Evaluating the impact of
prescription drug monitoring program implementation: a scoping review. BMC Health
Serv Res [Internet]. 2017 Jun 20 [cited 2018 Jun 14];17. Available from:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5477729/
359. McCarthy M. Containing the opioid overdose epidemic. BMJ. 2012 Dec 14;345(dec14
2):e8340–e8340.
360. Anderson D, Zlateva I, Khatri K, Ciaburri N. Using health information technology to
improve adherence to opioid prescribing guidelines in primary care. Clin J Pain. 2015
Jun;31(6):573–9.
113
361. Heres S, Davis J, Maino K, Jetzinger E, Kissling W, Leucht S. Why Olanzapine Beats
Risperidone, Risperidone Beats Quetiapine, and Quetiapine Beats Olanzapine: An
Exploratory Analysis of Head-to-Head Comparison Studies of Second-Generation
Antipsychotics. Am J Psychiatry. 2006 Feb 1;163(2):185–94.
362. Lexchin J, Bero LA, Djulbegovic B, Clark O. Pharmaceutical industry sponsorship and
research outcome and quality: systematic review. BMJ. 2003 May 31;326(7400):1167.
363. Lesser LI, Ebbeling CB, Goozner M, Wypij D, Ludwig DS. Relationship between funding
source and conclusion among nutrition-related scientific articles. PLoS Med. 2007
Jan;4(1):e5.
364. Procyshyn RM, Chau A, Fortin P, Jenkins W. Prevalence and outcomes of pharmaceutical
industry-sponsored clinical trials involving clozapine, risperidone, or olanzapine. Can J
Psychiatry Rev Can Psychiatr. 2004 Sep;49(9):601–6.
365. Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R. Selective Publication of
Antidepressant Trials and Its Influence on Apparent Efficacy. N Engl J Med. 2008 Jan
17;358(3):252–60.
366. Doshi P, Dickersin K, Healy D, Vedula SS, Jefferson T. Restoring invisible and
abandoned trials: a call for people to publish the findings. BMJ. 2013 Jun 13;346(jun13
2):f2865–f2865.
367. Chan A-W, Hróbjartsson A, Haahr MT, Gøtzsche PC, Altman DG. Empirical Evidence
for Selective Reporting of Outcomes in Randomized Trials: Comparison of Protocols to
Published Articles. JAMA. 2004 May 26;291(20):2457–65.
368. Goldacre B. Bad Pharma: How Drug Companies Mislead Doctors and Harm Patients.
Reprint edition. New York: Farrar, Straus and Giroux; 2013. 448 p.
369. Dana J, Loewenstein G. A social science perspective on gifts to physicians from industry.
JAMA J Am Med Assoc. 2003 Jul 9;290(2):252–5.
370. Greenhalgh T, Russell J. Why Do Evaluations of eHealth Programs Fail? An Alternative
Set of Guiding Principles. PLoS Med [Internet]. 2010 Nov 2 [cited 2018 Sep 24];7(11).
Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2970573/
371. Chan CL, Leyrat C, Eldridge SM. Quality of reporting of pilot and feasibility cluster
randomised trials: a systematic review. BMJ Open. 2017 Nov 8;7(11):e016970.
372. Hopewell S, Loudon K, Clarke MJ, Oxman AD, Dickersin K. Publication bias in clinical
trials due to statistical significance or direction of trial results. Cochrane Database Syst
Rev. 2009 Jan 21;(1):MR000006.
373. Ciani O, Buyse M, Drummond M, Rasi G, Saad ED, Taylor RS. Time to Review the Role
of Surrogate End Points in Health Policy: State of the Art and the Way Forward. Value
Health. 2017 Mar 1;20(3):487–95.
114
374. Wittes J, Lakatos E, Probstfield J. Surrogate endpoints in clinical trials: Cardiovascular
diseases. Stat Med. 1989 Apr 1;8(4):415–25.
375. Mars SG, Bourgois P, Karandinos G, Montero F, Ciccarone D. “Every ‘never’ I ever said
came true”: transitions from opioid pills to heroin injecting. Int J Drug Policy. 2014
Mar;25(2):257–66.
376. Dasgupta N, Beletsky L, Ciccarone D. Opioid Crisis: No Easy Fix to Its Social and
Economic Determinants. Am J Public Health. 2018 Feb;108(2):182–6.
377. National Center for Injury Prevention and Control. Integrating & Expanding Prescription
Drug Monitoring Program Data: Lessons from Nine States. Centre for Disease Control
and Prevention; 2017.
378. Glasziou P, Meats E, Heneghan C, Shepperd S. What is missing from descriptions of
treatment in trials and reviews? BMJ. 2008 Jun 26;336(7659):1472–4.
379. Datta J, Petticrew M. Challenges to evaluating complex interventions: a content analysis
of published papers. BMC Public Health. 2013 Jun 11;13:568.
380. Liu H, Muhunthan J, Hayek A, Hackett M, Laba T-L, Peiris D, et al. Examining the use of
process evaluations of randomised controlled trials of complex interventions addressing
chronic disease in primary health care—a systematic review protocol. Syst Rev [Internet].
2016 Aug 15 [cited 2018 Sep 24];5. Available from:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4986376/
381. Oakley A, Strange V, Bonell C, Allen E, Stephenson J. Process evaluation in randomised
controlled trials of complex interventions. BMJ. 2006 Feb 16;332(7538):413–6.
382. Trafton J, Martins S, Michel M, Lewis E, Wang D, Combs A, et al. Evaluation of the
acceptability and usability of a decision support system to encourage safe and effective
use of opioid therapy for chronic, noncancer pain by primary care providers. Pain Med.
2010;11:575–85.
383. Patel V, Hale TM, Palakodeti S, Kvedar JC, Jethwani K. Prescription Tablets in the
Digital Age: A Cross-Sectional Study Exploring Patient and Physician Attitudes Toward
the Use of Tablets for Clinic-Based Personalized Health Care Information Exchange.
JMIR Res Protoc. 2015 Oct 19;4(4):e116.
384. Stribling JC, Richardson JE. Placing wireless tablets in clinical settings for patient
education. J Med Libr Assoc JMLA. 2016 Apr;104(2):159–64.
385. Jensen RE, Rothrock NE, DeWitt EM, Spiegel B, Tucker CA, Crane HM, et al. The Role
of Technical Advances in the Adoption and Integration of Patient-Reported Outcomes in
Clinical Care. Med Care. 2015 Feb;53(2):153–9.
386. Anand V, McKee S, Dugan TM, Downs SM. Leveraging Electronic Tablets for General
Pediatric Care. Appl Clin Inform. 2015 Jan 7;6(1):1–15.
115
387. Kastner M, Li J, Lottridge D, Marquez C, Newton D, Straus SE. Development of a
prototype clinical decision support tool for osteoporosis disease management: a qualitative
study of focus groups. BMC Med Inform Decis Mak. 2010 Jul 22;10:40.
388. Coathup V, Finlay T, Teare HJ, Kaye J, South M, Watt FE, et al. Making the most of the
waiting room: Electronic patient engagement, a mixed methods study. Digit Health
[Internet]. 2018 Jan 10 [cited 2018 Sep 18];4. Available from:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6001187/
389. Lyles CR, Altschuler A, Chawla N, Kowalski C, McQuillan D, Bayliss E, et al. User-
Centered Design of a Tablet Waiting Room Tool for Complex Patients to Prioritize
Discussion Topics for Primary Care Visits. JMIR MHealth UHealth [Internet]. 2016 Sep
14 [cited 2018 Sep 18];4(3). Available from:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5040865/
390. Harle CA, Marlow NM, Schmidt SOF, Shuster JJ, Listhaus A, Fillingim RB, et al. The
Effect of EHR-Integrated Patient Reported Outcomes on Satisfaction with Chronic Pain
Care. Am J Manag Care. 2016 Dec 1;22(12):e403.
391. Elley CR, Dawes D, Dawes M, Price M, Draper H, Goodyear-Smith F. Screening for
lifestyle and mental health risk factors in the waiting room: feasibility study of the Case-
finding Health Assessment Tool. Can Fam Physician Med Fam Can. 2014
Nov;60(11):e527-534.
392. Eldridge SM, Chan CL, Campbell MJ, Bond CM, Hopewell S, Thabane L, et al.
CONSORT 2010 statement: extension to randomised pilot and feasibility trials. Pilot
Feasibility Stud [Internet]. 2016 Oct 21 [cited 2018 Oct 14];2. Available from:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5154046/
116
Appendix 3.1 PRISMA-ScR Checklist
117
Appendix 3.2 Protocol Scoping Review
Clinical decision support systems for opioid analgesic prescribing for chronic non-cancer
pain: a scoping review
Review question
Our research question is: ‘What is the extent of the current research on CDSS for opioid
analgesic prescribing for CNCP in primary care settings?’ We will report on the following
outputs: study source (e.g. peer reviewed or grey literature), type of study (e.g. review.
randomised controlled trial, quasi-observational or observational study), study funding source,
study aims, study population and setting, description of CDSS, implementation process, and
study outcomes.
Methods
There is no international standard for scoping reviews. We will use the scoping review
frameworks by Arksey and O’Malley (234) and by Levac et al. (236) as described in the article
by Colquhoun et al (221) and the methods outlined by The Joanna Briggs Institute (220).
Search strategy
We will search MEDLINE, EMBASE, CINAHL, CENTRAL and PsycINFO databases from
2008 to present. This time period was selected because there have been significant technological
advances in the last ten years and many CDSS developed prior to this period are likely to have
evolved or be obsolete(324). We will build a comprehensive search strategy including the
118
concepts “opioid prescribing,” and “clinical decision support systems.” We will also search
International Pharmaceutical Abstracts via OVIDSP for the last ten years for conference
abstracts that appear to meet our inclusion criteria.
We will search trial registries (ClinicalTrials.gov, World Health Organization International
Clinical Trials Registry Platform (WHO ICTRP)) for studies from 2008 to present. We will
search Health Management Information Consortium, Open Sigle, Grey literature report and
OpenGrey for grey literature. We will search websites from organizations involved with health
care information technology. We will also conduct an advanced google search using our key
concepts.
We will also search the reference lists of eligible studies for additional studies. And, as a final
step, will circulate our list of studies to experts in the field to ensure relevant studies have been
included.
Study types
We will include all systematic reviews, scoping reviews, meta-analyses, randomized controlled
trials, non-randomized trials, controlled before-after designs, interrupted time series, case studies
and observational study designs. We will include studies that use quantitative, qualitative and
mixed-methods methodologies.
We will exclude letters, opinion articles, narrative reviews, commentaries, analysis articles,
clinical practice guidelines and policy documents. We will exclude reports on CDSS that do not
include some kind of evaluation.
119
Population and setting
Our target population is primary care providers (PCPs) (such as family physician, nurse
practitioners and primary care gynaecologists) who prescribe analgesic opioids for CNCP in
primary care settings. These settings include family practice, nursing stations, primary care
obstetrical and gynecological clinics, primary care internal medicine clinics and emergency
departments. We will include studies where others input patient-specific data (e.g. patients,
nurses, therapists, pharmacists) as long at the CDSS is designed to have an impact on our study
population: PCPs who prescribe opioids in primary care settings.
We will exclude studies where primary care providers are working in a secondary and tertiary
settings such as a pain clinic or addiction clinic. We will exclude primary care pediatric clinics.
Intervention(s), exposure(s)
We defined a CDSS as a clinical electronic informatics tool that provide point-of-care, patient-
specific information to assist a health care providers in making a clinical decision. We will
include studies on CDSS that were designed to assist PCPs in making a clinical decision about
opioid prescribing for CNCP in primary care settings. We will include CDSS that are integrated
into the EMR (CDSS launches from the EMR, uses patient data from the EMR and reports
output into the patient’s chart) as well as those that function independently (web-accessed on
computer or handheld device) where health care professionals have to enter patient specific data.
We will include prescription drug monitoring programs (PDMPs)/prescription monitoring
programs (PMPs) that enable the clinician to make a point of care decision about opioid
prescribing. We will include studies where the CDSS is part of a complex intervention.
We will exclude studies that examine CDSS for opioid prescribing for palliative or cancer pain.
We will exclude studies where the output is not designed to have an impact on PCPs’
prescribing. For example, we will exclude studies on PMPs/PDMPs when they are used for a
120
reason other than to assist a clinician in making a decision about opioid prescribing. We will also
exclude studies that were not implemented in clinical settings (e.g. simulated clinical settings).
We will exclude paper tools.
Study selection process
Two researchers will independently screen all the relevant articles and determine if they meet the
inclusion and exclusion criteria. We will contact authors for more information if it appears that
the study meets our inclusion criteria and full text is not available. If there is disagreement a third
researcher will assist in making the final decision. We will conduct reviewer meetings prior to
starting study selection and at other points as needed.
Data extraction process
Two researchers will pilot a data charting form on three to four studies. The form will contain
space to record contextual or process oriented data. After reviewing the results with the whole
team and modifying the form as needed, two researchers will continue to independently extract
data. Data charting will be an ongoing iterative process where the reviewers continue to modify
the form and the process as needed.
Data synthesis and analysis
We will report on the following outputs: study source (e.g. peer reviewed or grey literature), type
of study (e.g. review. randomised controlled trial, quasi-observational or observational study),
study funding source, study aims, study population and setting, description of CDSS,
implementation process, and study outcomes.
121
We will report mapping descriptively and prepare it for journal publication. We will use the
contextual and process-oriented data to write a narrative summary. We will report
inclusion/exclusion results using a preferred reporting items for systematic reviews and meta-
analyses (PRISMA) diagram.
Team members
Sheryl Spithoff MD CCFP, Staff Physician, Women’s College Hospital Toronto; Lecturer,
Department of Family and Community Medicine, University of Toronto; Mary Ann O’Brien
PhD, Assistant Professor, Department of Family and Community Medicine, University of
Toronto; Scientific Associate, Knowledge Translation Research Network, Health Services
Research Program, Ontario Institute for Cancer Research; Stephanie Mathieson PhD, Research
Fellow, Musculoskeletal Health Sydney, School of Public Health, University of Sydney; Frank
Sullivan MBChB PhD, Professor of Primary Care Medicine, University of St Andrews;
Professor, Department of Family and Community Medicine, University of Toronto (supervisor);
Susan Hum, MSc, Research Associate, Department of Family and Community Medicine,
Women’s College Hospital; Qi Guan MSc, Doctoral Student, Institute of Health Policy,
Management and Evaluation, University of Toronto; Abhimanyu Sud, MD CCFP, Academic
Director, Safer Opioid Prescribing, Continuing Professional Development, Faculty of Medicine,
University of Toronto.
122
123
Appendix 3.3 Medline Search Strategy
Database(s): Ovid MEDLINE: Epub Ahead of Print, In-Process & Other Non-Indexed Citations,
Ovid MEDLINE® Daily and Ovid MEDLINE® 1946-Present
Search Strategy:
# Searches Results
1
narcotics/ or analgesics, opioid/ or alfentanil/ or alphaprodine/ or buprenorphine/ or
buprenorphine, naloxone drug combination/ or butorphanol/ or codeine/ or
dextromoramide/ or dextropropoxyphene/ or dihydromorphine/ or diphenoxylate/ or
"enkephalin, ala(2)-mephe(4)-gly(5)-"/ or "enkephalin, d-penicillamine (2,5)-"/ or
ethylketocyclazocine/ or ethylmorphine/ or etorphine/ or fentanyl/ or heroin/ or
hydrocodone/ or hydromorphone/ or levorphanol/ or meperidine/ or meptazinol/ or
methadone/ or methadyl acetate/ or morphine/ or nalbuphine/ or opiate alkaloids/ or
oxycodone/ or oxymorphone/ or pentazocine/ or phenazocine/ or phenoperidine/ or
pirinitramide/ or promedol/ or sufentanil/ or tilidine/ or tramadol/
110562
2
(analgesic* or opioid* or opiate* or narcotic* or alfentanil or alphaprodine or
buprenorphine or (buprenorphine adj2 naloxone) or suboxone or subutex or
butorphanol or codeine or dihydrocodeine or dextromoramide or
dextropropoxyphene or dihydromorphine or diphenoxylate or ethylketocyclazocine
or ethylmorphine or etorphine or fentanyl or duragesic or hydrocodone or
hydromorphone or levorphanol or meperidine or meptazinol or methadone or
methadyl acetate or morphine or nalbuphine or oxycodone or oxymorphone or
pentazocine or phenazocine or phenoperidine or pirinitramide or promedol or
sufentanil or tilidine or tramadol or dilaudid or OPANA or targin or tapendatol or
nalbuphine or trama*).kf,tw.
209523
3 1 or 2 231175
124
4 hospital information systems/ or ambulatory care information systems/ or medical
order entry systems/ or point-of-care systems/ 23977
5 Prescription Drug Monitoring Programs/ 28
6
medical informatics/ or health information exchange/ or medical informatics
applications/ or decision making, computer-assisted/ or diagnosis, computer-
assisted/ or therapy, computer-assisted/ or drug therapy, computer-assisted/ or
decision support techniques/ or "information storage and retrieval"/ or data mining/
or health information interoperability/ or information systems/ or community
networks/ or decision support systems, clinical/ or health information systems/ or
integrated advanced information management systems/ or management information
systems/ or clinical pharmacy information systems/ or database management
systems/ or decision support systems, management/ or medical order entry systems/
or reminder systems/ or medical informatics computing/ or nursing informatics/
125955
7 microcomputers/ or computers, handheld/ or smartphone/ 19013
8 prescription drug misuse/ or prescription drug overuse/ 1405
9 "Drug and Narcotic Control"/mt, og, pc, sn, td [Methods, Organization &
Administration, Prevention & Control, Statistics & Numerical Data, Trends] 1083
10 Drug Monitoring/mt, sn [Methods, Statistics & Numerical Data] 7965
11 4 or 5 or 6 or 7 or 8 or 9 or 10 170667
12 ((health* or clinical or medic*) adj3 informatic*).kw,tw. 4610
13 ((electronic* or computer*) adj3 order entr*).kw,tw. 1665
14 ((phone* or cellphone* or smartphone* or handheld) adj4 (app$1 or
application*)).kw,tw. 3173
125
15 ((decision* or reminder* or point of care or point-of-care or alert*) adj3 (tool* or
system* or electronic* or computer-assisted)).kw,tw. 16630
16 (PDMP or PDMPs).kw,tw. 455
17 (PMP or PMPs).kw,tw. 2626
18 ((prescri* or drug* or narcotic*) adj3 monitor*).tw,kw. 16166
19 12 or 13 or 14 or 15 or 16 or 17 or 18 44403
20 11 or 19 201546
21 3 and 20 2683
126
127
Appendix 3.4 Grey Literature Search
Date URL Database Search strategy # items
retrieved
# screened
(uploaded to
citation
manager)
August 20 ClinicalTrials.gov (Informatics OR order entry OR phone OR handheld OR
application OR reminder OR alert OR electronic OR tool
OR prescription drug monitoring database OR decision
support) AND opioids
82 5
August 20 WHO ICTRP Opioid AND decision support
Opioid AND electronic
Opioid AND informatics
Opioid AND reminder
Opioid AND prescription drug monitoring program
Opioid AND alert
Opioid AND application
Opioid AND tool
Opioid AND phone
Opioid AND handheld
4
8
0
0
0
0
0
0
19
7
2
1
0
0
0
0
0
0
0
1
128
Databases grey literature
Date URL Database Search strategy # items
retrieved
# screened
(uploaded to
citation
manager)
August 20 Open Grey or
Open Sigle
www.open grey.eu
(double check)
(Informatics OR order entry OR phone OR handheld OR
application OR reminder OR alert OR electronic OR tool
OR prescription drug monitoring database OR decision
support) AND opioid
Opioid AND “decision support”
13 0
September 5,
2018
Grey literature
Report
Opioid 42 0
Website searching—if greater than 150, searched first 5 pages
Date Organization name URL Search strategy # items retrieved # screened
(uploaded to
citation
manager)
August 15,
2018
Department of
Veterans Affairs
www.va.gov
used google
advanced search bc
search engine was
Opioid “decision support”
Opioid “electronic”
Opioid “reminder”
81
517 (first 5 pages)
168 (first 5 pages)
0
1
0
129
not retrieving any
appropriate results
Opioid “informatics”
Opioid “prescription drug monitoring program”
Opioid “alert”
Opioid ”application”
Opioid ”tool”
Opioid ”phone”
Opioid ”handheld”
Opioid “dashboard”
Opioid “toolbar”
114
9
302 (first 5 pages)
913 (first 5 pages)
306
5
6
71
6
1
0
0
0
0
0
0
0
1
August 20 Regenstrief Institute
Regenstrief.org
used google
advanced search bc
search engine was
not retrieving any
appropriate results
Opioid “decision support”
Opioid “electronic”
Opioid “reminder”
Opioid “informatics”
Opioid “prescription drug monitoring program”
Opioid “alert”
Opioid ”application”
Opioid ”tool”
Opioid ”phone”
Opioid ”handheld”
42
59
4
202
35
0
4
10
4
0
2
0
1
0
1
0
0
0
0
0
130
August 20 CADTH CADTH search
Used search on site
Opioid AND decision support
Opioid AND electronic
Opioid AND informatics
Opioid AND reminder
Opioid AND prescription drug monitoring
program
Opioid AND alert
Opioid AND application
Opioid AND tool
Opioid AND phone
Opioid AND handheld
99
163
0
2
17
28
156
70
6
1
0
0
0
0
0
0
0
0
1
0
August 15,
2018
IHE http://www.ihe.ca/i
ndex.php?/publicatio
ns
Opioid 2 0
August 15,
2018
Pan-Canadian HTA
collaborative
http://www.crd.yor
k.ac.uk/PanHTA/
Opioid 9 0
August 15,
2018
Programs for
Assessment of
Technology in Health
(Canada)
Google advanced
search
opioid 0 0
131
August 15,
2018
INATHA http://www.inahta.
org/publications/
Opioid 11 0
August 17,
2018
Agency for
Healthcare Research and
Quality (AHRQ)
https://search.ahrq.
gov/search?q=opioid
+&search_icon.x=0&
search_icon.y=0
Used google
advanced search
Opioid “decision support”
Opioid “electronic”
Opioid “informatics”
Opioid “reminder ”
Opioid “prescription drug monitoring program”
Opioid “alert”
Opioid ”application”
Opioid ”tool”
Opioid ”phone”
Opioid ”handheld”
First 5 pages
First 5 pages
First 5 pages
First 5 pages
First 5 pages
First 5 pages
First 5 pages
First 5 pages
First 5 pages
First 5 pages
4
0
0
0
0
0
0
0
0
0
CDC CDC site
Site search
Opioid “decision support”
Opioid “electronic tool”
Opioid “reminder system”
Opioid “informatics”
Opioid “prescription drug monitoring program”
Opioid “alert”
151
9
17
243 (first 5 pages)
385
945 (first 5 pages)
0
0
0
0
0
0
132
STOPPED search as no relevant results
Opioid ”application”
Opioid ”tool”
Opioid ”phone”
Opioid ”handheld”
Health IT site https://www.health
it.gov/
Used advanced
google search (bc site
used bing)
Opioid “decision support”
Opioid “electronic”
Opioid “reminder”
Opioid “informatics”
Opioid “prescription drug monitoring program”
Opioid “alert”
Opioid ”application”
Opioid ”tool”
Opioid ”phone”
Opioid ”handheld”
93
156
26
59
37
43
83
87
55
3
0
0
0
0
1
1
1
1
0
0
HIMSS (non-profit)
but run by for-profit
EMR /IT companies
HIMSS site search Opioid
116 5
OntarioMD Ontario MD site
search
Opioid 16 0
Health ITanalytics
(for-profit org)
Site search is
faulty (only works
for page 1), used
Opioid “decision support system”
Opioid “electronic tool”
4
127
0
0
133
google advanced
search
Opioid “reminder system”
Opioid “informatics”
Opioid “prescription drug monitoring program”
Opioid “alert”
Stopped search as none relevant
Opioid ”application”
Opioid ”tool”
Opioid ”phone”
Opioid ”handheld”
4
135
42
19
0
0
0
0
National technical
information service
https://www.ntis.g
ov/
ntis site search
Opioid
0
0
Search engine searching
Date Search engine Search strategy # screened
(uploaded to
citation
manager)
134
August 23
2018
Google advanced
search
opioid decision support OR electronic OR reminder OR informatics
OR alert OR application OR tool OR phone OR handheld
First 15 pages 21
Google advanced
search
Opioid AND “prescription drug monitoring program” AND “chronic
pain”
First 10 pages 5
135
Appendix 4.1 Physician Interview Guide
Qualitative Interview Guide
This Interview guide is for primary care physicians (PCPs) who responded to email
recruitment script, and signed a consent form agreeing to participate in the interview.
Participants: will include 6 to 12 PCPs who have used the Institute for Safer Medication
Practices (ISMP) Canada tools for safer opioid prescribing: Patient Pain Check-In (PCI) and
the Audit Package.
Interviewers: Study qualitative researcher, Dr. Leslie Carlin
Please Note:
This guide only represents the main themes to be discussed with the participants and as such
does not include the various probes that may also be used.
Non-leading prompts (in italics) will also be used, such as “Can you please tell me a little bit
more about that?” and “What does that look like for you?” when probing a vague statement
such as “I was active.”
Concrete examples will be asked for, regarding general descriptive statements that are made.
Introduction
Thank you for agreeing to participate in this interview. We are interviewing you to better
understand your experience with the clinical tools for safer opioid prescribing for chronic non-
cancer pain. The tools were developed to improve the structure and approach to pain related
136
visits, and to improve pain and related symptom assessment, as well as identify problematic
opioid use. One tool is the Patient Pain Check-In (PCI) and it is accompanied by a package of
audit tools that can be used to identify appropriate patients for the PCI, the Audit Package. The
purpose of the interview is to give you a chance to tell us what you think about these clinical
tools. We would like to hear about how these clinical tools affected (or did not affect) your
practice, your interactions with patients and any impact you feel the tools had on patient
outcomes. We would like to find out how you think we can improve these tools to make them
more useful and likely to be used in practice. Participation in this interview is voluntary. The
interview should take approximately 30 to 45 minutes. I will audio record the interview for
future data analysis. All responses will be kept confidential and will only be shared with research
team members. No identifying information will be included in our report. You may decline to
answer any question or stop the interview at any time, or take a break for any reason.
Are there any questions about what I have just explained?
Introduction:
1. To start, can you tell me a bit about your practice?
2. Can you tell me about your experiences caring for patients with chronic non-cancer pain?
3. What have your experiences been like in caring for patients with chronic non-cancer pain
who are prescribed opioids?
a. What are the challenges you face in providing care for this population?
b. How do you overcome these challenges in your practice?
4. Have you had experience with electronic clinical tools that assist in managing chronic health
conditions?
i. Can you tell me about those experiences?
b. Have you used tools to assist with opioid prescribing in patients with chronic non-
cancer pain?
i. Can you tell me about those experiences?
137
The new clinical tools:
Patient Pain Check-In (PCI)
5. Can you tell me about your experience(s) using the PCI with patients in your practice?
a. How many patients have used the tool?
b. How did you use the information the PCI gathered?
6. Was the interaction with patients different than past interactions with patients taking opioids
for chronic non-cancer pain?
a. If so, in what way?
b. Did it impact the challenges you mentioned in caring for patients prescribed opioids
for chronic non-cancer pain? If so, in what way?
7. What criteria did you use to select patients for the PCI?
8. What process did you use to get them to complete the PCI?
9. How did patient(s) respond to use of the PCI?
10. What do you think are the benefits of the PCI?
11. What are the challenges?
12. How did the PCI affect your workload? What about the workload of other staff?
a. How did it affect your workflow when seeing a patient?
13. Is the PCI something you will use ongoing?
a. Why or why not?
b. How will you decide when to use it?
14. What do other physicians or staff think about the PCI?
15. What about the physician lead at your site?
16. Have you modified how you use the PCI compared to the first time [if only physician has
only used it with one patient, ask: hypothetically, how might you modify it?]
a. If yes, in what way?
17. Have you modified it compared to how others use it?
18. How do you think the PCI might affect patient health outcomes?
138
a. In what way?
19. What is your desired impact from implementing the PCI tool in your practice ?
20. How would you determine (measure) if this impact has occurred?
21. How could we improve the PCI?
22. How could we improve how we implement it?
23. Is there anything else you would like to tell me about the PCI or how it has been
implemented at your site?
The Audit Package
24. Can you tell me about your experience(s) using the Audit Package?
a. Did you run all of them or some of them?
i. If only some: Why did you select only some of them?
25. What did you do with the information you found out?
a. Did you make any changes to what you would typically do?
b. Did it lead you to do anything different in patient care?
26. What are the positives of using the Audit Package?
27. What were the challenges using the Audit Package?
28. How did the Audit Package affect your workload? What about the workload of other staff?
29. Will you continue to use the Audit Package?
a. Why or why not?
30. Have you modified how you use the Audit Package compared to the first time you used it or
compared to how others use it?
a. If so, in what way and why?
31. What kind of impact might the Audit Package have on patient health outcomes?
32. What is the desired impact that you want from implementing using the AUDIT tools in your
practice?
33. How would you determine this impact has occurred?
34. How do other physicians or staff find the Audit Package?
a. Do they find it useful? Do they recommend it to others?
35. What about the physician lead at your site?
139
a. Does he/she support its use?
36. How could we improve the Audit Package?
37. How could we improve how we implement it?
38. Is there anything else you would like to tell me about the Audit Package or how it has been
implemented at your site?
Thank you for participating in this interview. We appreciate your time and your feedback!
140
Appendix 4.3 Mapping Categories to NPT Constructs
Category from inductive
analysis
Coherence
(Sense-making
work)
Cognitive
participation
(Relationship
work)
Collective
action
(Enacting work)
Reflexive
monitoring
(Appraisal work)
CNCP x
Opioid prescribing for
CNCP
x
Technology x
How does CDSS change
approach
x x x
Physician time and
workflow
x x
Physician view of staff
time and workflow
x
Set-up work x x
Impact or benefits of
CDSS
x x
Limitations of problems
with CDSS
x x x
Other physicians’ views x x
141
How CDSS was modified x
How to improve CDSS x x
Other
142
Copyright Acknowledgements
Table 2.1 Knowledge to action process/cycle
This Agreement between Dr. Sheryl Spithoff ("You") and Elsevier ("Elsevier") consists of
your license details and the terms and conditions provided by Elsevier and Copyright
Clearance Center.
License Number 4454270321142
License date Oct 22, 2018
Licensed Content Publisher Elsevier
Licensed Content Publication Journal of Clinical Epidemiology
Licensed Content Title Knowledge translation is the use of knowledge in health
care decision making
Licensed Content Author Sharon E. Straus,Jacqueline M. Tetroe,Ian D. Graham
Licensed Content Date Jan 1, 2011
Licensed Content Volume 64
Licensed Content Issue 1
Licensed Content Pages 5
Start Page 6
End Page 10
143
Type of Use reuse in a thesis/dissertation
Intended publisher of new work other
Portion figures/tables/illustrations
Number of
figures/tables/illustrations 1
Format both print and electronic
Are you the author of this
Elsevier article? No
Will you be translating? No
Original figure numbers Fig. 1. The knowledge-to-action framework.
Title of your thesis/dissertation Clinical Decision Support Systems for opioid prescribing
for chronic non-cancer pain in primary care settings
Expected completion date Jan 2019
Estimated size (number of pages) 100
Requestor Location
Dr. Sheryl Spithoff
84 Bleecker Street
Toronto, ON M4X 1L8
Canada
Attn: Dr. Sheryl Spithoff
Publisher Tax ID GB 494 6272 12
144
Total 0.00 USD
Terms and Conditions
Table 2.2 What makes an intervention complex
This Agreement between Dr. Sheryl Spithoff ("You") and BMJ Publishing Group Ltd.
("BMJ Publishing Group Ltd.") consists of your license details and the terms and conditions
provided by BMJ Publishing Group Ltd. and Copyright Clearance Center.
License Number 4454260150326
License date Oct 22, 2018
Licensed Content
Publisher BMJ Publishing Group Ltd.
Licensed Content
Publication The BMJ
Licensed Content Title Developing and evaluating complex interventions: the new
Medical Research Council guidance
Licensed Content
Author
Peter Craig, Paul Dieppe, Sally Macintyre, Susan Michie, Irwin
Nazareth, Mark Petticrew
Licensed Content Date Sep 29, 2008
Licensed Content
Volume 337
Type of Use Dissertation/Thesis
145
Requestor type Individual
Format Print and electronic
Portion Figure/table/extract
Terms and Conditions