Introduction to

Outcomes Informed Care

What is “Outcomes Informed Care”?

Routine use of patient self report outcome and therapeutic alliance questionnaires to inform the treatment process.

Questionnaires are administered a frequent intervals throughout an episode of care.

Clinicians given access to continuous feedback on patients improvement as measured by the outcome questionnaires.

Who benefits?

Strong evidence from controlled studies and real world applications that patients benefit.Higher probability of improvementFewer treatment failures

Clinicians

Increasing market demand for clinicians with strong practice based evidence of effectiveness

Employers & payersGreater productivity gains Higher “return on investment”

What about the research?

Evidence of patient benefit from use of questionnaires and clinician feedback: References 1 - 7

Evidence for role of common factors: References 8-14

Evidence for importance of the clinician: References 15-25

Evidence for importance of working alliance: References 26-32

Tell me about the questionnaires

Items written to 4th grade reading level

Simple to understand frequency anchors

Never, Hardly Ever, Sometimes, Often, Very Often

Common sentence structure aids rapid completion

How often in the past two weeks did you

…feel unhappy or sad?…have little or no energy?

Reliability & validity

Reliability Coefficient alpha =>.90

Validity Strong correlations with other measures of global

distressCorrelations of Concurrent Validity

Beck Depression Inventory =.78PHQ9=.81

What is global distress?

All measures commonly used in mental health research appear to load correlate strong with a common factor, common called “global distress”

Global Factor includes items assessing:Symptoms of depression & anxietyAttention and concentration problemsFamily and interpersonal relationsWork place productivity & functionality

Item Response Theory

Permits us to evaluate how well items works for clients at varying levels of distress

Identify items sensitive for clients with very high levels of global distress scores

Thoughts of suicide; WorthlessOutpatient questionnaires ideally

targeted for moderate to severe cases with a mix of items

Client registration form

Completed at start of treatmentUsed to collect case mix variables such

as:Age Prior treatment history

Sex Language & ethnicityDiagnosis Health statusHistory of physical and/or sexual abuse

How is outcome measured?

Treatment outcome is measured as the improvement between intake scores and scores at the last encounter in a treatment episode

Multivariate modeling used to adjust for difference in case mix

Severity Adjusted Effect Size calculated for all patients with intake scores in the clinical range on the outcome questionnaire

How will I get feedback?

Clinician’s Decision Support ToolkitData available within 24 hours of

submissionSummary statisticsCase level dataGraph results, monitor improvement

Relationship building is an Evidence Based Practice!

Practitioners are encouraged to routinely monitor patients’ responses to the therapy relationship and ongoing treatment. Such monitoring leads to increased opportunities to repair alliance ruptures, improve the relationship, modify technical strategies, and avoid premature termination.

- Norcross & Lambert (2006) in Evidence-Based Practices in Mental Health, Norcross, Beutler & Levant (Eds), p. 218

Concept of Therapeutic Alliance

Three Components:

Tasks: Behaviors and processes within the therapy session that constitute the actual work of therapy

Bonds: The positive interpersonal attachment between therapist and client of mutual trust, confidence, and acceptance

Goals: Objectives of therapy that both client and therapist endorse

Therapeutic Alliance Scale

Sample items

I felt like we talked about the things that were important to me

I felt like the therapist liked and understood me

I felt the session was helpful.I felt confident that the therapist and I

worked well togetherDid you feel that the clinician understood

what it was like to be you

Alliance Scale Psychometrics

Items are heavily skewed in positive direction.

Scale scores are not normally distributed.Cannot calculate reliability & validity

using parametric statistics that assume normality of distribution

Items are only as “valid” as clinician’s ability to illicit honest and frank responses!

Alliance ResultsMeasurement make a difference

0

0.2

0.4

0.6

0.8

1

1.2

Alliance items completedat start of treatment

(n=1924)

No items alliance at startof treament (n=1192)

Eff

ect

Siz

e Effective Range

Highly effective range

High praise for cliniciansAlliance scores at start of treatment

39%

51%

10%

Perfect (Alliance=0)

Almost Perfect(Alliance<1)

Room for improvement(Alliance=>1)

Alliance changes in treatment

8%

45%

47%

Alliance Change forBetter

No Change

Alliance Change forWorse

Alliance change & outcome

0.00

0.20

0.40

0.60

0.80

1.00

1.20

Alliance Change forWorse

No Change Alliance Change forBetter

Eff

ect

Siz

e Effective Range

Highly effective range

An outcomes informed clinician…

Uses the best available data on treatment outcomes to inform the treatment for each client/patient

Recognizes the importance of clinician skill in providing effective treatments.

Accepts personal responsibility for evaluating and improving his or her outcomes.

What if my outcomes aren’t “good enough”?

Many clinicians experience anxiety at the thought of “being evaluated”

Overall outcomes tend to be very good, with a very large percentage of clinicians demonstrating effectiveness

Clinicians with lower effect sizes at first usually show substantial improvement over time

The result: Clinicians find that outcomes informed care give them a sense of confidence in their ability to help patients.

Who do I call for help?

Questions about missing or incorrect data

Audra Day; Data Center Coordinator

mailto:datacenter@clinical-informatics.com

801 554-4340

Clinician to clinician consultation & help

Joanne Cameron, PhD

mailto:joanne@clinical-informatics.com

801 739-6268

Technical questions

Jeb Brown PhD

mailto:jebbrown@clinical-informatics.com

References

1. Lueger RJ. 1998. Using feedback on patient progress to predict the outcome of psychotherapy. J Clin Psychol 54:383-93.

2. Lambert MJ, Whipple JL, Smart DW, et al. 2001. The effects of providing therapists with feedback on patient progress during psychotherapy: Are outcomes enhanced? Psychother Res 11(1):49-68.

3. Hannan C, Lambert MJ, Harmon C et al. 2005. A lab test and algorithms for identifying clients at risk for treatment failure. J Clin Psychol 61(2):155-63.

4. Lambert MJ, Harmon C, Slade K et al. 2005. Providing feedback to psychotherapists on their patients progress: Clinical results and practice suggestions J Clin Psychol 61(2):165-74.

5. Harmon C, Hawkins, Lambert MJ et al. 2005. Improving outcomes for poorly responding clients: The use of clinical support tools and feedback to clients. J Clin Psychol 61(2):175-85.

6. Brown GS, Jones DR. 2005. Implementation of a feedback system in a managed care environment: What are patients teaching us? J Clin Psychol 61(2):187-98. Claiborn CD, Goodyear EK. 2005. Feedback in psychotherapy. J Clin Psychol 61(2):209-21.

References (continued)

7. Azocar, F., Cuffel, B, McCulloch, J., McCabe, J., Tani, S., Brodey, B. (2007) Monitoring patient improvement and treatment outcomes in managed behavioral healthcare. Journal for Healthcare Quality (March/April)

8. Rosenzweig S. 1936. Some implicit common factors in diverse methods of psychotherapy: “At last the Dodo said, ‘Everybody has won and all must have prizes.’” Am J Orthopsychiatry 6:412-5.

9. Shapiro DA & Shapiro D. 1982. Meta-analysis of comparative therapy outcome studies: A replication and refinement. Psychol Bull 92:581-604.

10. Robinson LA, Berman JS, Neimeyer RA. 1990. Psychotherapy for treatment of depression: A comprehensive review of controlled outcome research. Psychol Bull 108:30-49.

11. Wampold BE, Mondin GW, Moody M, et al. 1997. A meta-analysis of outcome studies comparing bona fide psychotherapies: Empirically, “All must have prizes.” Psychol Bull 122:203-15.

12. Ahn H, Wampold BE. 2001. Where oh where are the specific ingredients? A meta-analysis of component studies in counseling and psychotherapy. J Counsel Psychol 48:251-7.

References (continued)

13. Chambless DL, Ollendick TH. 2001. Empirically supported psychological interventions: Controversies and evidence. Annual Rev Psychol 52:685-716.

14. Luborsky, L., Rosenthal, R., Diguer, L., et al. 2002. The dodo bird verdict is alive and well--mostly. J. Psychotherapy Integration Vol 12(1) 32-57

15. Luborsky L, Crits-Christoph P, McLellan T, et al. 1986. Do therapists vary much in their success? Findings from four outcome studies. Am J Orthopsychiatry 56:501-12.

16. Crits-Christoph P, Baranackie K, Kurcias JS, et al. 1991. Meta-analysis of therapist effects in psychotherapy outcome studies. Psychother Res 1:81-91.

17. Crits-Christoph P, Mintz J. 1991. Implications of therapist effects for the design and analysis of comparative studies of psychotherapies. J Consul Clin Psychol 59:20-6.

18. Wampold BE. 1997. Methodological problems in identifying efficacious psychotherapies. Psychother Res 7:21-43.

19. Elkin I. 1999. A major dilemma in psychotherapy outcome research: Disentangling therapists from therapies. Clin Psychol Sci Prac 6:10- 32.

References (continued)

20. Wampold BE, Serlin RC. 2000. The consequences of ignoring a nested factor on measures of effect size in analysis of variance designs. Psychol Methods 4:425-33.

21. Huppert JD, Bufka LF, Barlow DH, et al. 2001. Therapists, therapist variables, and cognitive-behavioral therapy outcomes in a multicenter trial for panic disorder. J Consul Clin Psychol 69:747-55.

22. Okiishi J, Lambert MJ, Nielsen SL, et al. 2003. Waiting for supershrink: An empirical analysis of therapist effects. Clin Psychol Psychother 10:361-73.

23. Kim DM, Wampold BE, Bolt DM. 2006. Therapist effects and treatment effects in psychotherapy: Analysis of the National Institute of Mental Health Treatment of Depression Collaborative Research Program. Psychother Res. 16(2): 161-172.

24. McKay, K. M., Imel, Z. E., & Wampold, B. E. (2006). Psychiatrist effects in the psychopharmacological treatment of depression. Journal of Affective Disorders, 92, 287-290.

References (continued)

25. Wampold BE, Brown GS. 2005. Estimating variability in outcomes due to the therapist: A naturalistic study of outcomes in managed care. J Consul Clin Psychol. 73(5): 914-923.

26. Bachelor, A., & Horvath, A. (1999). The therapeutic relationship. In M.A. Hubble, B.L. Duncan, and S.D. Miller (eds.). The Heart and Soul of Change: What Works in Therapy. Washington, D.C.: APA Press, 133-178.

27. Blatt, S. J., Zuroff, D.C., Quinlan, D.M., & Pilkonis, P. (1996). Interpersonal factors in brief treatment of depression: Further analyses of the NIMH Treatment of Depression Collaborative Research Program. J Consul Clin Psychol. 64, 162-171.

28. Bordin, E. S. (1979). The generalizability of the psychoanalytic concept of the working alliance. Psychotherapy: Theory, Research, and Practice, 16, 252-260.

29. Burns, D., & Nolen-Hoeksema, S.  (1992). Therapeutic empathy and recovery from depression in cognitive-behavioral therapy: A structural equation model. J Consul Clin Psychol. 60, 441-449.

References (continued)

30. Connors, GJ, DiClemente, CC., Carroll, KM, et al. 1997 The therapeutic alliance and its relationship to alcoholism treatment participation and outcome. J Consul Clin Psychol, 65(4), 588-598.

31. Horvath, A. O., & Symonds, B. D. (1991). Relation between working alliance and outcome in psychotherapy: A meta-analysis. J Consul Clin Psychol. 38, 139-149.

32. Krupnick, J., Sotsky, SM, Simmens, S et al. (1996) The role of the therapeutic alliance in psychotherapy and pharmacotherapy outcome: Findings in the National Institute of Mental Health Treatment of Depression Collaborative Research Project. J Consul Clin Psychol. , 64, 532-539.