The Management and Flow of Data in Drug Treatment Agencies

Jennifer P. Wisdom, PhD MPHJames H. Ford II, PhD

Using Performance and Outcomes Measures to Improve Treatment

March 21, 2008

Improving the quality of care requires… Identifying problem areas Creative interventions See if the intervention worked

In other words, you have to capture and track essential measures of performance and quality … which requires staff (or someone*) to collect, analyze, and learn from data

Drug treatment agencies have extensive reporting requirements … but few agencies have access to well-developed information systems.

Agencies spend enormous resources collecting data … but very little data is used for clinical decision-making or program planning

For most agencies, it’s just “paperwork.”

Research Plan

Step 1: How to drug treatment agencies learn to use data?

Step 1, revised: What do drug treatment agencies need to improve their capacity to collect data?

Step 1, take 3: What on earth are drug treatment agencies doing with data?

How do drug treatment agencies learn to use data? In context of NIATx, we defined success in “using

data” as: Successful use of data in process improvement

decisions Data collection appropriate to identified goal and

interpret it correctly Data reported are internally consistent and reliable Agency staff have expertise to resolve data

inconsistencies and to explain results to others

How do drug treatment agencies learn to use data? Factors associated with success in improving

timeliness and retention Agency leadership values data Resources committed to infrastructure Training on data issues Sharing of change results Success making data-driven decisions

How do drug treatment agencies learn to use data? Barriers to implementing process-focused

data systems

Time and moneySystem complexityExpertiseResistance*

What do drug treatment agencies need to improve their capacity to collect data?

Met with stakeholders to inquire about what is needed Agency staff = 10 Researchers/evaluators = 6 Funding agency staff = 1 An additional funding agency staff provided

perspectives.

What do drug treatment agencies need to improve their capacity to collect data? Attendees were asked to consider three main

questions: How can policy makers and funding agencies

promote data education and support? How can agencies that value data help other

agency leaders promote and value data and tie data to financial outcomes?

How can educational bodies, licensing boards, and agency continuing education programs foster the value of data?

What do drug treatment agencies need to improve their capacity to collect data?

The discussion revealed areas of conflict: Even though treatment agencies, researchers, and

funders agree on the importance of data-reporting and accountability, it can be difficult for addiction treatment agencies to obtain and analyze data to meet those goals.

Infrastructure (including technology and training) is needed, but support of agency infrastructure may not fall within the funder’s / state’s strategic objectives.

Educational bodies teach to licensing requirements, which are difficulty to modify.

And what data do we really need to collect anyway?

What are drug treatment agencies doing with data? Robert Wood Johnson Foundation funded

examination of the flow and management of data in drug treatment agencies

8 drug treatment agencies in 4 states Staff interviews (n=55) Flow charts of how information moves Review of forms

What are drug treatment agencies doing with data? All agencies had at least two levels of care Sampled broadly for varied characteristics

(urban/rural, large/small) Examined 3 processes

Intake Transfer within agency Discharge (or transfer outside of agency)

What we’ve learned…

What influences agency sophistication The state data tracking system

Including sophistication of data system Emphasis of state on tracking/using data Whether state gives data access to agency

Executive leader recognition of the importance of data Influences resource allocation

Grant participation

Factors that influence what data are collected and how

Always done it this way State reporting requirements Financial contracts/referral sources Level of Care Informal Agency Protocols

Few agencies had written protocols that Describe the flow of paperwork,

Govern the use of information or

Outline procedures for error checking to ensure data accuracy

Flow of information in the agency

Disparity in the amount and type of information collected at intake vs. any other time (discharge or transfer)

How information is shared across the agency depends on the level of sophistication (Electronic distribution vs. written documentation vs. verbal communication)

The Issue of Paper…

Despite technology, paper is king! Even with electronic systems, staff like paper.

It appears many forms are completed, filed, and never looked at again.

One agency was attempting to move to a paperless system; staff were challenged to let go of the paper…

Implications

There are opportunities to increase efficiency in how drug treatment agencies manage information. Processes for information collection

Reduce amount of data Simplify processes

Processes for using information Increase use of data Increase focus on outcomes

But there is a See Saw Battle

+ -Agencies can benefit from training to help them learn how to use data (e.g., NIATx).

Training is often hard to obtain due to time, finances, availability or technological barriers

When they learn how to use data, they improve their infrastructures to the level of sophistication desired.

No significant funding to drive these infrastructure changes such support may not fall within the funder’s / state’s strategic objectives.

Data reporting & accountability is important

Difficult to obtain and analyze data to meet goals

And then there is…

Next Steps

We need research that Identifies how the role of data sophistication and the

ability to use data impacts an agencies ability to improve and impact outcomes

Verifies organizational “factors” influencing data use and offers advice for improvement

Determines what type of organizational feedback is most useful to help drug treatment agencies use process and outcome measures to enhance quality of care, and

Allows drug treatment agencies to assess their own organizational data readiness

Finagle Law of Health Information

The information you have

is not the information you want,

The information you want

is not the information you need,

The information you need

is usually not available,

And the Information you can get

costs more than you want to pay.

Contact:Jennifer Wisdom, PhD MPHjwisdom@pi.cpmc.columbia.edu

Appreciation to:

Robert Wood Johnson Foundation

Elaine Cassidy PhD Victor Capoccia, PhD

Deirdre Mackey, BA Meg Wise, PhD Dennis McCarty, PhD

Extra slides if there’s time…

How can policy makers & funding agencies promote

data education and support?

Universal agreement that drug treatment agencies generally are not well-skilled at collecting or interpreting data, and that it would be beneficial for them to increase their ability to do so.

Barriers to system development hamper progress: unsophisticated state and agency data systems lack of computer skills of drug treatment staff general lack of understanding by all parties about common

goals and metrics Stakeholders suggested policy makers use both

incentives (such as providing higher reimbursement rates for demonstrated performance) and

penalties (such as making funding contingent on data collection).

How can agencies that value data help other agency leaders promote and value data and tie data to financial outcomes?

Agencies can assist each other by: promoting the benefits of data collection (e.g.,

increasing the likelihood of funding) sharing evidence of how using data to make

decisions has improved their financial bottom line

sharing strategies for increasing infrastructure capability and staff expertise.

How can educational bodies, licensing boards, and agency continuing education programs foster the value

of data?

Stakeholders suggested increasing communication between drug treatment agencies and educational bodies and licensing boards to partner in increasing quality and quantity of training.

Training should specifically focus on information will be directly applicable to graduates’ duties in drug treatment agencies.