better improvement research resources download from:
Post on 28-Dec-2015
213 Views
Preview:
TRANSCRIPT
Better Improvement Research
Resources download from:http://homepage.mac.com/johnovr/FileSharing2.html
1
John Øvretveit,Director of Research, Professor, Karolinska Medical
Management Centre Sweden and Professor of Health Management, Faculty of Medicine, Bergen University
04/19/23
Recognition of AHRQ & researchersYou are making a difference…
Just some achievements : Shojania ed 2001 700 page review of safety interventions
Quality and safety indicators
Culture survey
Team STEPS & other tools
Innovations exchange
2
Achievements
Notable research funded by AHRQ:
Closing the quality gap series http://www.ahrq.gov/clinic/epc/qgapfact.htm
Henriksen K, Battles JB, Marks ES, Lewin DI, editors. Advances in patient safety: from research to implementation. Vol. 1, Vol2. Vol 3 implement Vol4 AHRQ Publication No. 05-0021-1. Rockville, MD: Agency for Healthcare Research and Quality; Feb. 2005.
http://www.ncbi.nlm.nih.gov/books/bv.fcgi?rid=aps.part.1
Partnerships in Implementing Patient Safety (PIPS) grants
REAIM studies (eg Magid et al 2008)
3
Shown excellence, but now challenges# 1: Is it effective? (for many types of QSI)
# 2: Why?: causal model
# 3: Who cares anyway? - More useful to research users
# 3a: How to implement it?
# 3b: Researcher-user interaction: use knowledge translation res/K to shape question and enable users to use
= Exciting opportunity for research innovation
But silos6
My subject interventions to providers/organisations, not patientsevaluating non-standardisable complex interventions and implementation
strategies
Not Treatments: BBs after AMI (beta-blockers after myo cardial infarc tion)
But Intervention to get BBs given appropriately (eg Education,
guidelines, CDS, audit) Intervention to spread CDM eg Breakthrough collaborative RRT (or CRM) Development programme to lead improvement P4P for QS Accreditation: benefits for costs compared to alt?
7
Distinguish
8
INTERVENTION
Seed
IMPLEMENTATION STRATEGIESPlanting
CONTEXT
Soil and climate
Clinical QSI (eg prescribe BBs)
EducationGuidelinesAudit and feedbackAcademic detailing
Organisational structureCultureSystemsFinancial system?
Organisational QSI (eg care management; RRT)
Breakthrough collaborative
Which effective for which intervention?Classification of strategies?
Which features help and hinder which strateges/support which interventions?
Themes
Horses for courses Match method to question and type of QSI More flexibility and innovation
Its not the camera, but what’s behind and in front that makes a quality picture
Its not the intervention, but the context and the beneficiaries that makes the impact
9
More complex
=
more
dependent
on context for i
mplementation
How
10
Evaluation Method > How context dependent is the intervention? More complex = more dependent on context for implementation
Next: 4 challenges and resolutions
Useful research Efficacy Effectiveness/generalisation Translation
Examples: RRT; CRM; Transition interventions; Accreditation.
1104/19/23
Summary
1204/19/23
Challenge Resolutions
1 Decision makers information needs
Their hierarachy of evidence
2 Proof of efficacy
RCT/CT priceless...For all else, strengthen observational studiesParallel process evaluationReporting
3 Effectiveness research for generalisation
Pragmatic trials – variationsCase studyTheory-based researchAction evaluation learning cycle
4 Faster wider use
Content; process; structure; culture? Silos?
#1 challenge: decision makers information needs Go/not go decision – pilot, full-scale?
Implementer’s guidance: adapt and progress it?
Install update?
Needs: useful credible information, now!, about: Costs, savings, benefits, risks – for our organisation
Implementation to maximise success
Don’t even think about it unless….
Utility not purity: “Good enough validity” &some attention to bias
Researcher response? No compromise – publication and promotion 1
304/19/23
#1 challenge: decision makers information needs
"Many QIs have small to moderate effect" Research design limitations?
Does quantitative RCT/CT design a) fail to measure enough intermediate or ultimate
outcomes? b) obscure extremes, where context important? c) require prescribed implementation, when iterative
adaption necessary?
1404/19/23
#1 challenge: decision maker’s information needs
Resolution by decision-maker’s :
Hierarchy of evidence:
1)Face validity/make sense? - Try it on a small scale
2)Steve or Jane’s experience in Kansas
3)IHI practitioner reports: O1 > I > O2 data (Before>Intervention>After)
4)Published practitioner-scientist study
5)High-church medical journal publication
Proportionality of proof – cost/ease, risk, benefit1504/19/23
#2 Challenge: Efficacy proof Does it work – anywhere?
Maximise certainty of attribution of outcomes to intervention
Causal assumptions: why/how does it work?
Resolutions: Paradigm: O1 > I > O2 quantitative experimental black box
Is there are difference? Better:
O1 > I > O2 Bigger difference?
O1 > ? > O2
Other explanations for difference?
Control, randomise, compare, hygiene to avoid contamination by confounders1604/19/23
#2 Resolutions to increase proof of Efficacy Strengths
√ specifiable, controllable interventions like drug
= √ Unchanging, control known confounders and randomise-out others, 2/3 measures all you need
Limitations Absence of above. Works for whom? - Multiple perspectives.
Unintended consquences – study more outcomes
Decision makers translation – info they need in addition
1804/19/23
#2 Resolutions to increase proof of Efficacy
Strengthening Parallel process evaluation
Reporting ("SQUIRE" etc) (labels for what implemented, not the brand)
Attribution steriods for observational studies (sensitivity analyses to assess results Propensity score (Johnson et al 2006)
and instrumental variable (Harless and Mark 2006) methods
1904/19/23
#3 Challenge: effectiveness research for generalisation Effectiveness in different situations?
Issues: Many interventions sensitive to context
Implementable only if changed to suit context
Evolve in interaction with changing context - journey/story
Ie efficacy guarantee violated by user adaption of some interventions
For others: guarantee failure if you do not adapt
Or buy installation and 3 year guarantee
2004/19/23
#3 Resolutions: generalisable effectiveness research R1: Maintain paradigm: “Pragmatic trials”
Minimise loss of attribution with Time series, Step-wise wedge, SPC (but increase cost and time)
Some √ for routine practice feedback
Generalisable to similar situations and interventions
Add more situations and variations of the intervention
Compare many pragmatic trials and assess what works best where
Invite trails in X situations?
Improve reporting (standardise and details)
- ve: no answer to why? – explanation helps adapt, and contributes to science
. .
2104/19/23
#3 Resolutions: context sensitive generalisable effectiveness research
R2: Case study research
√ Describes intervention as it evolves & context helpers and hinderers
√ Assesses intermediate changes
√ Links these to ultimate patient/cost outcomes, if possible
Multiple case study in selected situations (eg Dopson 2002)
NEXT: What we have learned in doing this research
2204/19/23
What we have learned in doing this research
The research:
12 Action evaluation case studies of innovation implementation in Swedish health care
& variety of “research into practice” implementation and change studies
2304/19/23
L2: Distinguish Safer clinical practices
Changed providers behaviour = reduce adverse events?
Safer organisation and processes support changes in provider behaviour and address latent
causes
Implementation actions to achieve the above at team, organisation, system and national levels
External context helpers and hinders
(is a MET/RRT a safe clinical practice or a "safer organisation or process" change, or both?)
2404/19/23
The seed
Planting
Soil & climate
04/19/23 251994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009
Sodertalje innovation established 1996-1999
Actions: planning and pre- implementation
Result: Innovation content – three shared rehabilitation units with common decision making forum
Planning Establishment Further actions and types of coordination created
Combined client/patient care planning system Development of systemsprocedures and sub structures>>>>>>>>>>>>
One management group with representatives from the county council and municipal care
Context factors help and hinder implementation at different timesGovernment policy helps planning
L3: Theory essential - of intervention pathway to outcomes
To decide which data to gather To provide explanations to test To give implementers to help them adapt.
(Program theory Weiss 1972, 1997 Rog&Fournier 1997; Logic Model Wholey 1979; Theory-driven evaluation Chen 1990, Sidani & Braden 1998; realist evaluation Henry et al 1998, Pawson & Tilley 1997; Theories Grol et al 2007)
2604/19/23
L4: Action evaluation learning cycle
Feedback findings during implementation + and – for science
Assess effect of researcher on implementation and results
Helps develop intervention during the implementation journey
Increases cooperation and access to data
Partnership, but distinct roles
Study how implementers use knowledge and help use more
2704/19/23
#4 Challenge: use – faster, wider
Demand? - Real men don’t need research
Supply? - Real researchers don’t write exec summaries Make sure unusable and ”throw over the fence”
delivery
Closing the research/practice gap
2804/19/23
Translation in QSI HSR
Evidence>Test>Package User>Adapt>Implement/Adjust
Development Translation 1 Implementation Translation 2
(intervention development and testing) (adoption/ spread)
What is the intervention?
Where do you draw the boundary?
29
#4 Resolutions – our experience Use KT/KM literature – what works? Content: accessibility and relevance
Service implications; many examples; 3:20:Appx reports; ghost writers and mediator authors;
Engage emotionally: patient describes experience or video
Process: interact with users at each stage Structure: forums, networks, joint appointments,
brokers
3004/19/23
Summary
3104/19/23
Challenge Resolutions
Decision makers information needs
Their hierarachy of evidence
Proof of efficacy RCT/CT priceless...For all else, strengthen observational studiesParrallel process evaluationReporting
Effectiveness research for generalisation
Pragmatic trials – variationsCase studyTheory-based researchAction evaluation learning cycle
Faster wider use Content; process; structure; culture? Silos?
Questions Efficacy and causality
System thinking in research - causality explanations and data gathering
Always trade off between internal/external validity? Generalisable effectiveness research
Journey/story approach – unique? Use: faster, quicker
Extend researcher role? Increase demand? Effect of action role? 3
204/19/23
top related