1 performance and outcomes that demonstrate effectiveness of criminal justice fadaa/fccmh annual...

1

Performance and Outcomes that Demonstrate

Effectiveness of Criminal Justice

FADAA/FCCMH Annual ConferenceAugust 7 & 8, 2013

Neil Gaer, MPA VP, Senior Director Phoenix Houses of New England

Jack M. Feinberg, LMHC, CAP VP, Clinical Director Phoenix Houses of Florida

I. Introduction

II. Project overview – what we set out to do

III. Demonstration of the integrated CEST assessment and EHR

IV. Reports, data, and how to use it

V. Discussion and Q&A2

Outline of workshop

I. Introductions

3

II. Project Overview

4

What we set out to do

Context: Organization was implementing an EHR nationallyAim: Standardize a process where every client in the NE region

* receives a CEST assessment at intake

* and gets re-assessed at periodic intervals (depending on length of stay) during treatment

5

Why do we want to do this?

NIATx promising practice –

Enhance Motivation• Tailor treatment to each client’s

circumstances and needs• Offer motivation groups for clients to enhance

treatment engagement• Identify clients at risk for dropping out

and intervene• Map ongoing care at regular intervals

(treatment plan reviews using assessment results)

6

Why CEST and what is it?

• Client Evaluation of Self and Treatment

• Developed by the Institute of Behavioral Research at TCU– decades of research, federal outcome studies– studies of treatment processes (what

happens inside treatment, what are the elements of effective treatment)

7

8

Follow-upOutcomes

•Drug use•Crime•Social Functioning

Follow-upOutcomes

• Drug use• Crime• Social

Functioning

Client FactorsProblem Severity

& Treatment Readiness

Client FactorsProblem Severity

& Treatment Readiness

TCU Treatment Process Model

what’s inside the “black box”?

Early Engagement•

Participatio

n•

Therapeutic

Relationship Early

RecoveryChanges in

• Thinking• Behavior

RetentionThreshold

Tailoring treatment ReadinessInterventions

what’s inside the “black box”?

BehavioralInterventions

CognitiveInterventions

Social SkillsTraining

Recovery SkillsTraining

Aftercare Services

& Support Networks

Recovery Process

CEST assessments

• Client self-report• 4 modules, about 30 questions each• Measures client changes in thinking

and behavior – which may lead to more long term outcomes

in housing, employment, legal involvement

9

Motivation

Problem RecognitionDesire for HelpTreatment ReadinessPressures for Treatment

Psychosocial Functioning

Depression AnxietySelf Esteem Decision

MakingHostility/Anger Risk TakingSocial Support

Engagement

Treatment ParticipationTreatment SatisfactionCounselor RapportPeer Support

CEST Scales measuring...

11

PSYCHOLOGICAL FUNCTIONING SCALES E. Self Esteem (SE) 17. You have much to be proud of. 59. You feel like a failure. ® 71. You wish you had more respect for yourself. ® 88. You feel you are basically no good. ® 100. In general, you are satisfied with yourself. 109. You feel you are unimportant to others. ® F. Depression (DP) 57. You feel interested in life. ® 74. You feel sad or depressed. 76. You feel extra tired or run down. 90. You worry or brood a lot. 97. You feel hopeless about the future. 123. You feel lonely.

sample questions

Adaptive Interventions

Recap of TCU’s Adaptive Treatment Model

– manualized curricula for identified problem areas

– evidence-based (listed on NREPP)– it’s FREE – download CEST assessments, curriculum

manuals at www.ibr.tcu.edu

13

http://www.ibr.tcu.edu/

14

How did Phoenix House New England come upon and utilize the

CEST• Searching for evidenced based

practices– NREPP

• Need for outcome studies for funding sources

• Monitor Quality Performance Improvement (QPI) as part of management reporting – And it sounded like a good idea at the time

15

• Effectiveness – measures the quality of care through measuring change over time.

• Efficiency – measures how well resources are used to accomplish agency program goals.

• Satisfaction – measures oriented toward the person served, family members, personnel, and funding sources.

Meeting CARF definitions of

III. The integrated CEST and EHR

16

What was the plan?

• Build the assessments into EHR. Have automatic date triggers. Automate real-time scoring.

• Build reports to look at change over time on aggregate level.

• Create a Quality Performance Improvement (QPI) system to check how we are doing (efficiency & effectiveness).

17

The old way...

18

21

Example of On Screen

Close up of Scoring Summary

23

25

CEST in New England

• Residential – Adult– Adolescent – Corrections

• Outpatient• Community Corrections

26

Welligent and CEST

• Integration• Transparency• Individualized• Facility Averages• National Norms• Graph All Client Scores Over Time• REAL TIME

So how did we do

27

How many clients were receiving comprehensive

assessments?

• At admission/intake for initial treatment planning– many programs were using the ASI, some

were using other tools

• During treatment for ongoing monitoring of progress– none

28

29

Setting due date triggers

Assessment Module

Admission(Days post Intake)

Recurrence Discharge (Days before Discharge)

Motivation 14 days - -

Engagement 30 days every 30 days within 30 days

Psychological Functioning 14 days every 30 days within 30 days

Social Functioning 14 days every 30 days within 30 days

Criminal Thinking Scales 14 days every 90 days within 30 days

Trauma 14 days every 90 days within 30 days

example: Adolescent Residential Programs

Efficiency Benchmarks

Click on below LINK

to go to SharePoint

Indicator 3 - CEST Psychological Functioning

GOAL: 70% of clients complete BASELINE CEST surveys

http://phintranet/Foundation/Clinical/performancemonitoring/Lists/Performance%20Monitoring%20Reports/AllItems.aspx

Total number of client admissions scheduled to take Baseline survey

Total number of clients completing Baseline survey

% of completed Baseline surveys this quarter #DIV/0!

% of completed Baseline surveys last quarter

The standard is met by the program Y N

Corrective Action Plan (if needed or in red): Include Goal(s) & Measurable Objectives

Click on below LINK

to go to SharePoint


GOAL: 70% of clients complete DISCHARGE CEST surveys


Total number of client admissions scheduled to take Discharge survey

Total number of clients completing Discharge survey

% of completed Discharge surveys this quarter #DIV/0!

% of completed Discharge surveys last quarter



















Let’s focus on the New England Adult

Residential programs

31

2011 2012 20130%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

67%

86%

95%

53%

64%67%

41%

55%

64%

% who stayed at least 14 days and received baseline CEST% with another assessment before end of 2nd month% of discharges with assessment within 30 days of DC

33

III. Reports, Data, and How to Use it

34

So what does this all mean?

35

• 44 y.o. male • crack (age 23), marijuana/alcohol (age 11)• referred by parole – was incarcerated 1 year• 5 prior treatment episodes• H.S. diploma, unemployed• depression, on medication• parole officer and client felt he needed

treatment to maintain sobriety while transitioning back to community

Individual case example“John”

36

Assessing client motivation levels

37

Assessing client risks and needs

38

Monitoringprogress in treatment

39

• Reports can be reviewed with clients in clear, simple language

“I see that your scores on feelings of anger have increased quite a bit (peer support decreased, treatment satisfaction decreased) since last month. Can you tell me what’s been going on?”• Opportunities for therapeutic intervention

“So you’re having problems with your roommate. Have you and [roommate] talked about it? What skills might you use from your “Ideas for Better Communication” group?• Collaborative treatment planning

“Last time we met you talked about how your feelings of anger sometimes get in the way of your relationships with others. What would you think about participating in the “Emotional Cartography” (or Anger Management) group?

Real time feedback

monitoring progress and defining treatment goals

40

Aggregate data

41

Our goal within the course of treatment is to

increase psycho-social strengths scores in Self Esteem Decision Making Social Support Expectancy for Recovery

and to decrease psycho-social problems scores in- Depression- Anxiety- Anger/Hostility- Risk Taking

Monitoring program deliverables

(and adjusting evidence-base practices)

42

Increase in scores on Self Esteem, Decision Making

43

Increases in Social Support, Recovery Expectations

44

Decrease in scores on Depression, Anxiety scales

45

Decrease in scores for Hostility/Anger, Risk taking

What was the plan?

Build the assessments into EHR. Have automatic date triggers. Automate real-time scoring.Build reports to look at change over time on aggregate level.Create a QPI system to check how we are doing (efficiency & effectiveness).

46

Interactive reports

47

Efficiency Benchmarks

Click on below LINK

to go to SharePoint


GOAL: 70% of clients complete BASELINE CEST surveys


Total number of client admissions scheduled to take Baseline survey

Total number of clients completing Baseline survey

% of completed Baseline surveys this quarter #DIV/0!

% of completed Baseline surveys last quarter











Interactive reports on SharePoint

49

Report filters

50

51

What’s in this report

- scales- averages- instrument norms- problem “flags”

Quality Improvement Indicators

Indicator 3C CEST - Increase psychological Strength scores over course of treatment

Indicator 4D CEST - Decrease psychological Problem score over course of

treatment

Instrument Norm = 3.8



Instrument Norm = N/A





Self-EsteemDecision

Making SkillsSocial

Support

Has High Expectations for Recovery Anxiety Depression

Anger/ Hostility Risk Taking

Discharge Discharge Discharge Discharge Discharge Discharge Discharge Discharge

Adolescent Academies

3.5 3.3 4.0 3.6 3.1 2.7 3.1 3.6

3.6 3.6 4.0 3.9 2.7 2.4 2.7 3.2

Adult Residential

3.2 3.5 4.1 3.8 3.2 2.9 2.6 3.5

3.7 3.8 4.2 4.2 2.7 2.4 2.5 2.9

Community Residential

3.0 3.4 4.0 3.5 3.3 3.0 2.6 3.5

3.6 3.7 4.3 3.9 2.9 2.5 2.4 3.3

Supportive Living

3.7 3.9 3.9 4.1 3.1 2.6 2.5 3.3

3.7 4.1 4.3 4.1 2.6 2.2 2.4 2.9

Outpatient Corrections

3.7 3.7 4.1 4.1 2.7 2.4 2.4 3.1

3.8 3.9 4.1 4.1 2.6 2.2 2.2 2.8 Outpatient Adolescent

& Adult

3.7 3.8 4.2 3.9 2.8 2.5 2.4 3.1

4.1 4.1 4.3 4.1 2.2 2.0 1.6 2.6

53

Indicator 3C CEST - Increase psychological Strength scores over course of treatment

Indicator 4D CEST - Decrease psychological Problem score over course of

treatment




Instrument Norm = N/A





Self-EsteemDecision

Making SkillsSocial

Support

Has High Expectations for Recovery Anxiety Depression

Anger/ Hostility Risk Taking

Discharge Discharge Discharge Discharge Discharge Discharge Discharge Discharge

Adolescent Academies

Adult Residential

Community Residential

Supportive Living

Outpatient Corrections Outpatient Adolescent

& Adult

Measuring Effectiveness

Other trends

54

New England Adult Residential Programs

treatment completion rates

552006 2007 2008 2009 2010 TEDS 2011 2012 20130%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

32%29%

40%

35%

27%

46%

42% 43%46%

Lessons learned and next steps...

56

57

Next Steps

• Continue staff training • Integration into treatment planning• Integration of mapping curriculum• Balance individual treatment /

facility service delivery based on the aggregate data – Retention rates– Completion rates

Discussion, Q & A

58

Resources

• TCU Institute of Behavioral Research http://www.ibr.tcu.edu/

• TCU Mapping Enhanced Counseling on SAMHSA NREPP http://www.nrepp.samhsa.gov/ViewIntervention.aspx?id=161

• Phoenix House www.phoenixhouse.org

• Neil Gaer [email protected]

• Jack Feinberg [email protected]

59

http://www.ibr.tcu.edu/

http://www.nrepp.samhsa.gov/ViewIntervention.aspx?id=161



http://www.phoenixhouse.org/

mailto:[email protected]

mailto:[email protected]

Thank you!

60

1 performance and outcomes that demonstrate effectiveness of criminal justice fadaa/fccmh annual...

Documents