Knowing what or understanding how:

The role of RCTs in changing

clinical practice

Ivan Eisler

Reader in Family Therapy

Institute of Psychiatry, Kings College London

EFTA/AFT Congress4th - 6th October, 2007.

Glasgow

All who drink this remedy recover in a short time except those whom it does not help who all die.

Therefore it is obvious that it only fails in incurable cases.

Attributed to Galen 2nd century A.D.

Our 2 psychology colleagues maintain that the RCTs all point to CBT and there is little or no RCT research evidence for the value of systems or psychoanalytical family work.   Any advice or pointers that you could offer would be appreciated, as my colleagues here and I use psychodynamic and systems approaches because we are committed to them and believe that they work

Smith & Pell 2003 BMJ;327;1459-1461

One of the major weaknesses of observational data is the possibility of bias, including selection bias and reporting bias, which can be obviated largely by using randomised controlled trials. The relevance to the procedure under discussion is that individuals using it are likely to have a high prevalence of pre-existing psychiatric morbidity. Individuals who do not use it are likely to have less psychiatric morbidity and may also differ in key demographic factors, such as income and cigarette use. It follows, therefore, that the apparent protective effect may be merely an example of the “healthy cohort” effect.

Smith & Pell 2003 Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials BMJ;327;1459-1461

One of the major weaknesses of observational data is the possibility of bias, including selection bias and reporting bias, which can be obviated largely by using randomised controlled trials. The relevance to parachute use is that individuals jumping from aircraft without the help of a parachute are likely to have a high prevalence of pre-existing psychiatric morbidity. Individuals who use parachutes are likely to have less psychiatric morbidity and may also differ in key demographic factors, such as income and cigarette use. It follows, therefore, that the apparent protective effect of parachutes may be merely an example of the “healthy cohort” effect.

Debriefing

“the experiences of 700 CISM teams in more than

40,000 debriefings cannot be ignored,

especially so when the overwhelming majority

of reports are extremely positive”

Mitchell & Everly, 2003

Debriefing

• Review of 15 RCTs of single session debriefing

showed no short term difference between

debriefing and control

• 2 RCTs with long term follow-up showed worse

outcome following debriefing (particularly in

those with worse initial trauma)

The development of 'empirically validated treatments'

Open testing of treatment

Randomised clinical trial

Abandon treatment

-ve result

Clinical observations

Theoretical model

Development of new treatment

Adopt treatment as standard

practice

+ve result

Limitations of randomized treatment trials

Subjects selectivity classification dropouts

Nature of treatments restricted nature treatments time limitations common factors

Evaluation of outcome efficacy v effectiveness short term v long term outcome clinical v measurable outcome outcome from whose perspective

The evolution of clinical practice

Conceptualization of therapy and model of change

Explanatory models of disorders

Professional consensus views

Evolution of clinical practice

Clinical practice Randomised clinical trial

Evidence-based medicine is the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients.

Sackett et al (1996) Evidence based medicine: what it is and what it isn't. BMJ. 312, 71-2,

David Sackett

The practice of EBM means integrating individual clinical expertise with the best available external clinical evidence from systematic research

Sackett et al (1996) Evidence based medicine: what it is and what it isn't. BMJ. 312, 71-2,

David Sackett

Conclusions

• It is important to distinguish the role of evidence in making “categorical” decision and ongoing “process” decisions

• The RCT paradigm provides a useful but limited test of the efficacy of treatments

• RCTs (cumulatively) can have an important effect on professional consensus which influences the allocation of resources, training as well as clinical practice

• RCTs, by limiting clinical autonomy, will often lead to unexpected results which challenge beliefs about mechanisms of change and conceptualization of treatment

• Changes in clinical practice require a change in the understanding of how treatments work which requires a mixture of inductive and deductive reasoning