OverviewThe Importance of

Data: As easy as balancing your

checkbook

(Title)

Name(s) of presenter(s)

Organizational Affiliation

What do you need to know?

• 2 Categories– Deposits– Expenditures

How does this knowledge help?• Deposits

• Number of Deposits• Total Amount In

• Expenditures• Number of Checks Written• Total Amount Out

What can you do with the knowledge?

• Net Profit = Deposits - ExpendituresHit Jackpot

Broke the Bank

What can you do with the knowledge?

• Ask Questions–What type of expenditures?–When do they occur?–Why did I experience a loss? Profit?–What happened that was different

this month?

What is the rest of the story?

1. Deposits2. Expenditures 1. Total Deposits

2. Number of Deposits3. Total Expenditures4. Number of Expenditures5. Net Profit or Loss

Power of Two

How does that translate to measuring the impact of change?

1. Number of First Contacts2. Number of Assessments3. Elapsed Time4. Assessment Conversion

1. Date of First Contact2. Date of Assessment

How does that translate to measuring the impact of change?

1. Number of Clients w/First Treatment Session2. Number of Clients w/Fourth Treatment Session3. Elapsed Time between First and Fourth Treatment Sessions4. % of Clients w/1st Treatment Session who get a 4th Treatment Session

1. Date of First Treatment Session2. Date of Fourth Treatment Session

How does that translate to measuring the impact of change?

1. Three Counts2. Two Conversion Numbers3. Three Elapsed Time Figures

1. Date of First Contact2. Date of Assessment3. Date of Admissions

WHY IS THIS IMPORTANT?

Principle #5 Rapid Cycle Testing

• Start by asking 3 questions– What are we trying to accomplish?– How will we know the change is an

improvement?– What changes can we test that will result in

an improvement?

Model for ImprovementReference: Langley, Nolan, Nolan, Norman, & Provost. The Improvement Guide, San Francisco, Jossey-Bass Publishers, 1996

7 Simple Rules of the Road• Define measures

• Collect baseline data

• Establish a clear aim

• Consistent collection

• Avoid common pitfalls

• Report and Chart progress

• Ask questions

–Use of Data

• 2X2 Tables

PreRequest to Service

PostRequest to Service

PreAdmissions

PostAdmissions

Rule 1: Define Measures

• Establish clear definitions

• Clarify project aims

• Agreed upon by key stakeholders

Establishing Clear Definitions• Example suggested measures

– Time from First Contact to Assessment

– # of clients moving to the next level of care

– % of clients attending their 1st four sessions

• Measure definition– Elapsed Time from Date of Assessment – Date of 1st Contact

– # of admissions to next LOC/# of discharges from higher LOC

– # of clients with four sessions/# of admissions

Rule 2: Establish a Baseline

• Never start a project without it

• Define a clear starting point

• Use agreed-upon definition

Start Finish

Baseline Data• Serve as a meaningful “road map”

• Helps answer the question: “How will we know a change is an improvement?”

• Use tools to collect a adequate baseline information

• The time period for the baseline will vary by– Measure– Agency’s size– Ease of Collection

• Preferred Sample Size is at least 40

Examples of Baseline Data

• Average time from 1st Contact to Assessment is 35 Days

• 40% of our assessments are no-shows

• Only 30% of our clients receive 4 units of service in 30 days

• Only 25% of Detoxification Discharges connect with the next level of care

Rule 3: Establish an aim

• Be flexible– Information suggests changing the

aim, change it

– Aim is too ambitious, set a realistic aim that still challenges the agency to improve

– Aim is easily achieved, set a more ambitious aim that stretches the agency’s capacity to improve

Examples – Defining the Aims• No: ↑ % of Res Discharges to OP by 20%• Yes: ↑ Successful Res Discharges to OP from 40 to

48%

• No: ↓ Assessment No-Shows by 40%• Yes: ↓ Assessment No-Shows from 60 to 36%

• No: ↓ Time from 1st Contact to Assessment by 20 days

• Yes: ↓ Time from 1st Contact to Assessment from 28 to 7 days

Rule 4: Consistently Collect Data• The length of time necessary to test a change will vary depending on an

agency’s size.

• Devise ways to collect information but remember KEEP IT SIMPLE

• Collect small samples over short time periods– Next 10 clients– Next 15 Phone Calls– Measure impact in days not weeks or months– Preferred Sample Size is at least 40– Pilots should not last more than a month

• Once change is successful, collect larger samples to verify progress– Track clients admitted next week– Examine data for one month

Rule 5: Avoid Common Pitfalls

1. Events occurring before their time

2. Sequential events occurring in order but with long lag times

3. Sequential events occurring out of order

4. Missing dates

5. Cell contains characters

6. Incorrect recording of dates

Client #

Date of 1st

Contact

Date of clinical

assessmentDate of

ISPISP

completedclient called

Date of orientation

Client called

Date of admission

(a) (b) ( d) ( e) ( f) (g) (h) (i)9 01/23/07 1/1/29/07 02/05/07 A/M N/S L/M

24 03/21/14 02/06/07 02/09/07 seen MBK 02/12/07 01/01/0054 04/10/14 02/19/07 02/20/07 seen 01/02/00 02/22/07 02/22/07

102 01/18/07 03/07/07 04/24/07 seen ok 03/12/07164 03/24/07 03/29/07 04/03/07 left by 4 ok 04/02/07 phone off N/S4/3/07169 01/06/07 01/08/07 no ISP 03/28/07 03/28/07196 04/09/07 04/11/07 ?264 04/30/08 05/09/07 05/17/07 ok 5/25/2006 ok/ns289 11/09/06 11/13/06 ? ? ? 05/16/07 OK 05/16/07

Dates before their time Dates out of sequence

Long lag times

Stay out of the Quicksand• Don’t collect to much

data

• Don’t focus on too many measures

• Don’t get trapped in analysis paralysis

• Ask a “yes or no” question of your data

Rule 6: Report and Chart Progress

• A Simple Axiom: One chart, one message

• Charts can be used to: – Highlight the baseline (pre-change) data– Identify when a change was introduced– Visually represent the impact of individual changes

over time, and – Inform your agency about sustaining change over time.

Example ChartTime from First Contact to First Treatment

0.00

2.00

4.00

6.00

8.00

10.00

12.00

Jun-03

Jul-03

Aug-03

Sep-03

Oct-03

Nov-03

Dec-03

Jan-04

Feb-04

Mar-04

Apr-04

May-04

Jun-04

Jul-04

Aug-04

Sep-04

Oct-04

Nov-04

Changes Implemented ChangesSustained

Use TrendlinesTime from 1st Contact to 1st Treatment

9.6

7.8 7.6

9.1

5.2 5.6

2.2

4

0

2

4

6

8

10

12

Jun-03 Jul-03 Aug-03 Sep-03 Oct-03 Nov-03 Dec-03 Jan-04 Feb-04

Months

Day

s

Effect of STAR on Access to Assessments

0

5

10

15

20

Days

Ave Number of Days To Assessment

Baseline Overbooking Collapse Forms Walk-ins Shorten Walk-in Times

Rule 7: Ask Questions• Do not accept results at face value

• Do the results look right?

• What is the data telling us?

• Unsuccessful changes afford the opportunity to ask Why?

What do you know with more information?(e.g. Gender, Age, Drug of Choice)

1. Number of First Contacts2. Number of Assessments3. Elapsed Time4. Assessment Conversion

1. Date of First Contact2. Date of Assessment

Summary• Determine what you will measure• Establish a system to collect key data elements• Gather your baseline• Define your aim• Collect data often and consistently – who’s job

is this?!• Evaluate the impact

– One chart, one message– Ask questions

QUESTIONS