The Myth of Evidence Based Policy

Robin DavidsonSSA. York 2013

Thanks

• Staff of ALRUK.• Particularly James Nicholls• Willm Mistral

• Davidson, R (2013 ) Evidenced based alcohol policy : not as simple as it sounds. In Mistral, W (ed). Substance Misuse; Emerging Perspectives. Wiley Blackwell. London

Definition of EBP

• Linear and direct selection, synthesis and critical evaluation of best research evidence to answer a policy problem and which can then be implemented

Greenlagh and Russell (2009)

To begin with

• Evidence based practice?• Dearth of evidence?• Introverted scientists?• The sabotaging industry?

How to invent a new therapy

1. Start with an established psychological process of change 2. Add functionally trivial bells and whistles

3. Demonstrate using one RCT that the “new” treatment is statistically (P< .05) better than no treatment or TAU

4. Give “new” treatment a name and acronym

5. Patent it

6. Set up accreditation process and retire to the South of France.

Davidson (2008)“The central goal of psychotherapy research should be to achieve an

understanding of the psychological mechanisms or processes of change and not focus on brand name treatments”

In the last year…….• Office of National Statistics• Health and Social Care Information Centre • Health Survey England• Chief Medical Officer• World Health Organisation• EU Alcohol Health Forum• OECD• Department of Health• Home Office• HMRC• Scottish Government• NHS Scotland / MESAS• British Beer and Pub Association• Institute of Fiscal Studies• Centre for Economic and Business Research• Scottish Health Action on Alcohol Problems• Alcohol Focus Scotland• Balance North East• Alcohol Research UK• AMPHORA• EUCAM• Eurocare• Institute of Alcohol Studies• Local Government Association• Greater London Authority• Alcohol Concern• University of Stirling / Alcohol Health Alliance• Ofcom• RAND Europe• The Adam Smith Institute• Demos• Mentor UK•

The Statisticians TaleHome Office Data Analyst.

:

I could call up data that could prove anything I wanted to. If I want to make a case that is really good ....using proper data I can manipulate it to make it look pretty. Similarly, if I wanted to placepressure on the Home Office to support us with an initiative I could put my hands on data that would do just that. So numbers do whatever you want. ...

Don’t all partnerships do that.?

The Journalists TaleA.A. Gill. Sun Times. May 24th 2009

There is an academic assumption that all research is a good thing and that all

knowledge gleaned from research is of equal importance as every shard in a

mosaic. This is of course empirical bollocks. Most research is forgotten and an awful lot is utterly pointless and has more to do with the search for funding, career building and hierarchies than it

does with uncovering and thereby righting the world

Ducks in a row in 1997

• New Labour elected.• Tony Blairs public commitment to “New Public

Management” and that “evidence based solutions should drive policy”.

• David Blunket “information and knowledge at the heart of policy making and delivery”.

• Gus O’Donnell appointed cabinet secretary

Maybe four reasons for the Myth

• The structure of government

• The psychology of decision makers

• The assumptions of “naïve rationalism”

• The spirit of the times

The structure of Government

• Departmental Pluralism resulting in conflicting and multiple outcomes

• Short term, adversarial, ideological split yet marketing pragmatists,

• Yes minister• Career politicians, ambitious,

Cognitive dysfunction in politicians

• Primacy effect

• Confirmation bias (favour of evidence that confirms beliefs)

• Belief perseverance (beliefs persist after evidence for them has shown to be false)

And what about civil servants…………

Naïve rationalism

• What works trumps desirable ends and appropriate means?

• Values - whose likely benefits worth whose potential losses

• What about embodied knowledge (Praxis) • There is no such thing as a body of evidence. Rather

competing constructions able to support any position.• Collective deliberation. policy is language,

argumentation, discourse and incremental decision making

Spirit of the times

• Beerhouse act-1830• Industrial revolution- trading imperatives• mid 19thC -moral issues• Late Victorian-social reform• 1915 – national industrial efficiency.• 1960s - health• 1990s – tourism• Now- youthful public disorder (MUP)

Not new-Aristotle

• Logos; the argument• Ethos; the speaker• Pathos; emotional• Coming from; understanding the audience.

Recent Schemes

• March 2013 What Works Evidence Centres• Out of 70 programmes implemented by the DoE 2

were evaluated. Lead to the Education Endowment Foundation.

• The Behavioural Insights Team• Trial Design Advisory Panel – Ben Goldacre.• Project Oracle leading to a standard of evidence

framework.• Policy Skills Framework launched by the institute for

Government

The End

robindavidson@hotmail.co.uk

Alcohol Research UKWillow House, 4th Floor17 - 23 Willow PlaceLONDON SW1P 1JH

Tel 020 7821 7880