using outcomes in the quality improvement process:

Using Outcomes in the Quality Improvement Process:

Setting System Level Standards

Erik R. Stewart, Ph.D. Vice President, System Performance

Eriks@hamilton.mh.state.oh.us

Renee Kopache, M.S.Recovery Coordinator

Reneek@hamilton.mh.state.oh.us

History/Purpose of Outcomes

• Ohio Mental Health Outcomes Task Force (OTF) – 1996– Charge

• Develop a statewide approach to measuring consumer outcomes in Ohio’s publicly-supported mental health system

Vital Signs (1998)

• Outcomes Implementation Pilot Coordinating Group– Final recommendations were made in 1999.

Adult Outcomes Domains

• Clinical Status– looks at symptoms that a person may experience

from their illness and how much they interfere with their daily living.

• Quality of Life– examines how “good” a person’s life is, and if their

needs are being met. Also examines how much control a person has over the events in their life (empowerment).

Adult Outcomes Domains

• Functional Status– This domain identifies how well a person is

doing in the community including areas such as work, school and social relationships.

• Safety & Health – Addresses how a person is doing physically

and the amount of freedom they have from psychological harm from self and others.

Adult Outcomes: Instruments

• Adult Consumer Form A

– This form is used by consumers with a severe mental illness. (case management)

– The survey consists of four parts: Quality of Life, Safety and Health, Symptom Distress and Empowerment.

• Adult Provider Form A – consists of two parts: Functional Status and

Safety & Health.

– Provider’s observations and clinical judgments about the consumer’s social & role functioning, housing status, activities of daily living, criminal justice involvement, harmful behavior and victimization.

• Adult Consumer Form B– Consists of three parts: Quality of Life

(excluding empowerment, Safety & Health and Symptom Distress.

– This shorter version of the Adult A form is used for adult outpatient clients. Providers of non-SMD consumers are not required to complete a provider survey.

Youth Outcomes Domains

• Four primary areas or domains of assessment were selected for Ohio Scales:

– Problem severity– Functioning– Hopefulness– Satisfaction with behavioral health

services

Youth Outcomes: Instruments

• Ohio Scales

Three parallel forms: – Y-form is completed by the youth (self-

report for ages 12 and older).

– P-form is completed by the youth’s parent (or primary caretaker).

– W-form is completed by the youth’s agency worker/case manager.

Administration Periods

• Initial• 6 Month• 12 Month• Annually thereafter

(Adult Consumer Form B: Initial and Discharge administrations only)

OCO History in Hamilton County

• Data collection begins – September, 2001• Regular and varied production reporting begins

at same time• Production performance abysmal, though

consistent with other participating areas of State

HCCMHB surveys agencies to discern impediments to production

Agencies React

HCCMHB representatives visit agencies to discern impediments to

production

Impediments (in no particular order):

1) Excuses:

• POV boxes don’t turn on (batteries)• Staff refuse to complete or administer• Multi-million $ agency has only 1 printer• POV box times out too quickly

2) Collection technology:

• POV data system interface with other agency data system(s)

3) Lack of agency/staff “buy-in”

• We know we do good work• We don’t care about consumer outcomes

4) Lack of products/information to staff following administrations (initially)

• Staff frustration with initial software• Introduction of ARROW with use limited to POV• Immediate red flags printout• Weekly reports to staff• Consistent supervisor response to problems• Use of data in “assessment update printout”

5) Diffusion of responsibility (a social phenomenon that

occurs in groups of people when responsibility is not explicitly assigned)

• Request made by QA/Outcomes member to resurrect regular IT/IS staff meetings at HCCMHB. Solicitation made. ONE response.

• Lack of departmental ownership

6) Agency-specific challenges

• Central Point of Access• New administration at agency intake?• Paper and pencil administration• Program support staff enter data

QA Approach Introduced

• Quarterly indicator measuring Initial Adult Provider compliance added to existing indicators

ODMH Introduces Missing Data Reports - Summer, 2003

• Hamilton County Performance– 23% Adult OR/OE– 34% Youth OR/OE

October, 2003 - HCCMHB and Agencies Form PI Sub-committee

to Develop Production Logic

February, 2004 – Ohio Department of Mental Health Presents State

Plan for Upcoming PI Requirements

PDCA Cycle

April, 2004 – Proposed Logic for OCO Production Measures Shared

With Membership

Initial Adult Consumer Production Analysis

• Claims data compiled for quarter under review

UCI First Svc. Date Min Age UPID Service Type

1000001 7/10/2005 42 10001 CPST

1000001 8/1/2005 42 10002 Counseling

1005002 9/2/2005 28 10001 CPST

2052520 7/20/2005 19 10005 Med Som

2052520 7/25/2005 19 10006 CPST

• “Pre-quarter period” is determined by subtracting 120 days from first date of service in period

UCI First Svc. Date Pre-Quarter Period

1000001 7/10/2005 3/12/2005

1000001 8/1/2005 4/3/2005

1005002 9/2/2005 5/5/2005

2052520 7/20/2005 3/22/2005

2052520 7/25/2005 3/27/2005

• Review of claims during pre-quarter period is conducted. Those exhibiting claims are removed

UCI First Svc. Date Pre-Quarter Period Pre-Quarter Claims

1000001 7/10/2005 3/12/2005 No

1000001 8/1/2005 4/3/2005 No

1005002 9/2/2005 5/5/2005 Yes

2052520 7/20/2005 3/22/2005 No

2052520 7/25/2005 3/27/2005 No

• Unduplicate list by selecting case with highest order service

UCIFirst Svc. Date

Pre-Quarter Period

Min Age UPID

Service Type

1000001 7/10/2005 3/12/2005 42 10001 CPST

1000001 8/1/2005 4/3/2005 42 10002 Counseling

2052520 7/20/2005 3/22/2005 19 10005 Med Som

2052520 7/25/2005 3/27/2005 19 10006 CPST

• Create date range for acceptable OCO administrations

UCI First Svc. Date 1st Svc. Minus 60 1st Svc. Plus 44

1000001 7/10/2005 5/11/2005 8/23/2005

2052520 7/25/2005 5/26/2005 9/7/2005

• Merge Adult Consumer A and Adult Consumer B OCO administrations

• Match by date when acceptable date exists

• Determine production ratio/percentage

January 2005 February 2005 March 2005