Evaluation of HCHDcommunity behavior health program 2005 - 2006

Evaluators:- Charles Begley- Scott Hickey- Britta Ostermeyer- Ann Teske

- Thien Vu

- Julia Wolf

- Mark Kunik

Outline

Background Objectives Evaluation framework and methods Results Discussion and recommendation

Background HCHD:- serves around 300,000

individuals/year, most are low income and/or underinsured

- Operates BTGH, Lyndon Johnson GH, Quentin Mease Community Hospital, 11 CHCs, 7 school-based clinics, a healthcare program for the homeless, a center for HIV/AIDS (Thomas st.) and a dental center

CBHP:- First launched by HCHD in

3 CHCs- Officially created in July

2005,- CBHP team includes

psychiatrists, psychotherapists, counselors, residents and med students in 11 HCHD CHCs, 5 partner centers, 2 school-based clinics

Background

Why CBHP?- Estimates: 20,000 children and

84,000 adults in Harris County needed mental health services in 2004

- There was only one District’s outpatient clinic at BTGH at the time

- Only 8,800 adults and 1,700 children were served

- Average appointment time: 6 months

CBHP objectives:1. to redirect BH patients to

community clinics (integrated care)

2. to provide specific BH services 3. to develop & provide

educational services for PCPs4. to consult and coordinate with

primary care providers

Evaluation framework

Evaluation objectives

Process evaluation• to describe major resources and features of the

program• to describe services provided and patients served

Preliminary impact evaluation• to evaluate initial impact on access, BH outcomes,

provider satisfaction and costs

MethodsVariables Measurement

methodData sources

Program resources and features:- Number and type of CBHP staff- Resources used

- simple calculation- aggregated

- Project documents- Monitoring report

Amount and type of services provided & characteristics of patient served: - # of patient seen by provider & patient types- demographic characteristics- types of counseling sessions by individual/group/family/total

- aggregated (using patient-coded medical records)

- CBHP integrated database

Initial impacts of the program:- Provider satisfaction

- Accessibility- Health outcomes

- Provider survey

- Pre-post analyses- BASIS-24 analysis

- 100 questionnaires PCPs, psychiatrists and BH therapists- HCHD database- patient’s BASIS-24 assessments

ResultsProgram resources and featuresProgram resources and features

Originally proposed:- hire 1 Prog. Director, 4

licensed Social workers & 1 psychiatrist

- Involve 4 HCHD’s CHC and 3 private community clinics

Actually implemented- As proposed- Additional: 1 project

coordinator, 6 social workers and 10 part time psychiatrists

- Involve all 11 HCHD’s CHC, 4 private community clinics

- Additional education sessions: teleconference lectures, psychotherapy referrals, DVD and audio tape lectures

2,895 patients seen - 2075 females and 820 males - 801 by psychiatrists, 1,824 by

BH therapists- 2,363 MH patients, 6 substance

abuse, 336 both- 34 referred by project Insight,

12 referred by Council on Alcohol & Drug Houston

- 75 < 6 years; 137 ages 7-12; 169 from 13-18; 2,376 from 19-64; 128 > 65

- 835 African American, 58 Asian, 754 Caucasian, 1225 Hispanic; 3 American Indian; 17 others

- 55 Katrina and 10 Rita victims

7,392 counseling sessions:- 1,696 psychiatry; 3,342

individual counseling, 562 group counseling, 830 families and 95 phone sessions

ResultsServices provided & patient characteristicsServices provided & patient characteristics

Timeframe: July 2005 – May 2006

ResultsInitial impact on patient’s health BASIS-24:- 24 questions- domains are psychiatric and

substance abuse functioning: depression, relationship, self-harm, emotional ability, psychosis and substance abuse

- administered at least twice in the study period with the follow-up assessment at least 30 days after the first

- responses scored using weighted average algorithm that give overall score for each assessment

Results:- Significant improvement

detected in overall score (p <.000) and 4 out of 6 domain scores: depression, self-harm; emotional lability and substance abuse (p<.000)

- Average improvement percent change: 26% in overall score, 30% in depression, 75% in self-harm, 37% in emotional lability and 72% in substance abuse scores

ResultsProvider’s satisfaction

BHS + Psychiatrist PCPs

Variables Obs Mean Obs Mean

Accessibility 18 4.58 27 4.18

General quality improvement of PHC 18 4.68 27 4.53

Common understanding about CBHP 18 4.69 27 4.10

Time flexibility 18 3.25 27 3.78

Interaction between PCPs and BH 18 4.00 24 3.92

PCP education 17 3.34

General satisfaction scores by areaGeneral satisfaction scores by area

ResultsProvider satisfaction – service accessibilityProvider satisfaction – service accessibility

VariableCombined

score BH + Psy PCPs

Poor accessibility prior to CBHP 4.42 4.56 4.33

Improved accessibility by CBHP 4.47 4.78 4.26

CBHP reduced ER visit 4.40 4.61 4.26

CBHP reduced length of time for accessing

BH services 4.16 4.44 3.96

CBHP helps improve access care in

appropriate time frame 4.24 4.50 4.07

Satisfaction scores on accessibility by provider type

ResultsProvider satisfaction - quality of careProvider satisfaction - quality of care

Variable BHS + Psy PCPs

BHC necessary for PHC 4.94 4.70

BHC improve adherence to treatment 4.89 4.67

BHC offered by CBHP improve general quality of PHC 4.72 4.44

CBHP enhances PCPs’ ability to provide BHC 4.44 4.52

BHC enhances quality of the clinics 4.39 4.30

Mean scores on quality of care variables by provider type

ResultsProvider satisfaction – time flexibility and staffingProvider satisfaction – time flexibility and staffing

Mean scores on time flexibility reported by providers

Variable Mean Min Max

Time flexibility of BH physicians 3.25 1 5

Time flexibility of PCPs 3.78 2 5

ResultsProvider satisfaction – interaction between PCP & BH physiciansProvider satisfaction – interaction between PCP & BH physicians

VariableBH

physicians PCP

Effective referral between PCPs and BH Therapists 3.94 4.26

Comfortable with referring patients to PCPs/BH Therapists 4.00 4.16

Awareness of CBHP protocol, roles and functions of members 4.06 3.54

Mean scores on interaction between PCPs and BH physicians

ResultsProvider satisfaction – Educational activities and materials for PCPsProvider satisfaction – Educational activities and materials for PCPs

Purpose: to improve PCP’s capacity to provide BH at CHC through on-going trainings and on-site consultation

Materials: teleconference lectures, DVD + audio tapes Result: negative • 52% of PCPs have not received any educational

materials• mean score on effectiveness of PCP education

session was 3.34, lower than cut point

ResultsProvider satisfaction – Common vision, understanding and overall satisfactionProvider satisfaction – Common vision, understanding and overall satisfaction

Variable Combined BH physicians PCP

Share sense of responsibility 4.40 4.78 4.15

Common treatment goals 4.40 4.67 4.22

Understanding of roles and responsibility 4.33 4.67 4.11

Formal and informal interaction 4.20 4.72 3.85

Share knowledge 4.38 4.78 4.11

Common vision/philosophy of CBHP 4.19

Total (combined score) 4.31 4.50 4.10

Mean scores on common vision, understanding by provider type

Results Service patternsService patterns

Results Service patternsService patterns

Results Service patternsService patterns

Conclusions & recommendations CBHP has achieved many

of its implementation objectives

Impacts of CBHP on patients’ health, accessibility to BH services were positive

Patient flow for BH was initially re-directed to lower cost and more convenient settings

BH providers’ working schedules should be more flexible

Interaction between PCPs and BH providers should be further facilitated for smoother operation of CBHP

More BH staff is needed More rigorous evaluation

plan should be developed Educational efforts should

be improved