how evidence is used: implications for thinking about quality
DESCRIPTION
How evidence is used: implications for thinking about quality. Professor Sandra.Nutley University of Edinburgh Research Unit for Research Utilisation (RURU). Different ways evidence is used What quality means to users of research Implications for increasing research use/ impact. Minister. - PowerPoint PPT PresentationTRANSCRIPT
How evidence is used: implications for thinking about
qualityProfessor Sandra.Nutley
University of EdinburghResearch Unit for Research Utilisation (RURU)
Different ways evidence is
used
What quality means to users
of research
Implications for increasing
research use/ impact
Minister
Using Evidence: How research can inform public services
(Nutley, Walter and Davies, Policy Press, 2007)
“Anyone who has ever written or utter the words ‘evidence-based policy’ should read this outstanding book.” (Carol Weiss, Harvard)
"This book is a major contribution to the literature: clear, thoughtful, relevant and evidence-informed.” (Tom Rundall, UC Berkeley)
Diverse forms of knowledge and evidence
• Research & evaluation reports
• Audit & inspection findings/data
• Routine monitoring data/KPIs
• Local & international exemplars
• Costings data
• Client & user experience data
• Expert views & insider knowledge
• Opinion polls & stakeholder consults
• System capacity & implementation issues
• Models & forecasts
Research and evidence
Researchers offers more than “evidence”
• Analytical capacity
• Theoretical insight
• Critique and challenge
Research only one form of evidence
But…
1. Different ways evidence is used
Evidence helps to address:
• Know-about (problems)
• Know-what (works)
• Know-how (to put into practice)
• Know-who (to involve)
• Know-why (requirements of action)
An evidence use continuum:
Awareness
Knowledge
Changing attitudes, perceptions, ideas
Knowledge & understanding
Persuasion
Practice & policy adaptation
Decision Implementation
Confirmation
MORE CONCEPTUAL USES INSTRUMENTAL USES
The “enlightenment” role of research (Weiss)
PROBLEM REFRAMING
Importance of
informal carers…Decarceration
policies…
Patient safety…
Harm reduction in substance misuse…
Service user
engagement…
Enhancing self-care…
The happiness and well-
being agenda…
“Enlightenment use”: promoting new ways of thinking…
Knowledge impels action
Many different ways of ‘using’ evidence:
Stocks and flows of knowledge
Policy, organisational and professional environments, and
society at large
Percolation
Knowledge grabbing
Problem solving or tactical
Interaction
Co-productio
n of knowledg
e
Researchers not disinterested in all this
• The consensual approach – working with the grain of current policy
• The contentious approach – “keeping the system honest”
• Paradigm challenging – subverting current thinkingand perhaps proposing new principles for action
Evidence use is complex – because ‘policy making’ is complex
SOMETIMES:
• clearly defined event
• explicit decisions
• conscious deliberation
• defined policies
• policy fixed at implementation
OFTENTIMES:
• ongoing process
• piecemeal: no single decision
• muddling through
• policies emerge and accrete
• shaped through implementation
Role of evidence varies:Engineered vs Emergent solutions impacts
2. What quality means to users of research
Policy Makers’ Hierarchy of Evidence
• ‘Experts’ evidence (incl. consultants and think tanks)
• Opinion-based evidence (incl. lobbyists/pressure groups)
• Ideological ‘evidence’ (party think tanks, manifestos)
• Media evidence
• Internet evidence
• Lay evidence (constituents’, citizens’ experiences)
• ‘Street’ evidence (urban myths, conventional wisdom)
• Cabbie’s evidence
• Research Evidence
Source: Phil Davies, 2007
Attention IS paid to research when:
• Research is timely, evidence is clear and relevant, and the methodology is relatively uncontested.
• Results support existing ideologies, are convenient and uncontentious to the powerful.
• Policy makers believe in evidence as an important counterbalance to expert opinion: and act accordingly.
• Research findings have strong advocates and are endorsed by opinion leaders (personal contact is most effective).
• Research users are partners in the generation of evidence.
• The results are robust in implementation and implementation is reversible if need be.
Users’ perspectives on quality
• Fitness for purpose is key (inc. timeliness, accessibility and relevance to issue at hand)
• Source a proxy for quality (attention more likely to be paid to evidence from trusted sources)
• Persuasiveness not necessarily a function of methodology (e.g. clear implications, a good story, rhetorical presentation)
3. Implications for increasing research use/ impact
Yes, it’s quite a noise – but are we having any impact?
Improving research use: addressing supply, demand, and that in between
Improving stocks or reservoirs of research knowledge (nb beyond
methods)
Increasing demand in political and professional worlds, and wider
society
intermediation
Increasing
Knowledge
exchange
Improving research use: addressing supply, demand, and that in between
Improving stocks or reservoirs of research knowledge
Increasing demand in political and professional worlds, and wider
society
Research translation, knowledge management and knowledge pools, research brokering and boundary
spanning, co-location, secondments and role cycling, partnerships of all
kinds, sustained interactivity…
Challenges: many active players in complex policy networks
Politicians Civil servants
Political advisors
Professional groups
The media
Lobbyists and pressure groups
Audit, inspection & scrutiny regimes
Government analysts
Opening up to polyphony, loosening control,tolerating diverse views on ‘evidence’… FLUIDITY
University researchers
Independent researchers and
evaluators
Think tanks/charities
Some conclusions• Many sources and forms of evidence
that serve diverse purposes, and are used in different ways
• Quality for research users is more about usefulness than methodological rigour
• Interactive models of research use are helpful
Interactive models of research use
• The importance of context;• Interaction with other types of
knowledge (tacit; experiential);• Multi-voiced iterative dialogue;• ‘Use’ as a process not an event.
Moving away from ideas of ‘packaging’ knowledge and enabling knowledge transfer – recognising instead: