early intervention program & early family support services: analyzing program outcomes with the...
TRANSCRIPT
Early Intervention Program & Early Family Support Services: Analyzing Program Outcomes with the Omaha System of Documentation
Presented to: Minnesota Omaha System Users GroupFebruary 1, 2012
Project Background
Since 1989 Public Health - Seattle and King County (PHSKC) has provided public health nursing (PHN) services for maltreated children through the Early Intervention Program (EIP) and Early Family Support Services Program (EFSS).
The EIP and EFSS programs serve children ages birth to eighteen years of age, who have been reported to Washington State Children’s Administration (CA) for child maltreatment concerns.
Although EIP and EFSS program data has been collected over the years, there was not a mechanism to evaluate the overall program benefits and outcomes.
Choosing the Omaha SystemA statewide workgroup, which consisted of the EIP and EFSS program staff and Children’s Administration’s Program Director, was convened to choose an assessment/evaluation tool. The Omaha System was selected as its KBS rating system was easy to use, the system could be utilized by nurses, social workers, and community outreach workers, and it could demonstrate program outcomes. Key Omaha System problems were selected to be used to evaluate program outcomes for this vulnerable population: Abuse (Child Abuse and Adult Domestic Violence)Caretaking and Parenting Growth and Development Neglect (Child)Health Care Supervision (Child)Income Mental HealthResidence Substance Use (Drugs and Alcohol; Tobacco)
Omaha System Tools and Guidelines Development for EIP/EFSS Providers
A KBS rating manual and Omaha care pathways were developed to be consistent with Washington State statutes and local child welfare practice, and to address the specific health conditions and service needs of maltreated children.
Omaha System training materials were also developed to meet the specific practice needs of EIP and EFSS providers
Study Methods:
Providers were asked to record KBS intake and exit ratings for the top three Omaha problems they had opened for families onto excel billing logs.
Those EIP and EFSS families that had received PHN services during October 2009 through January 2011 and had KBS ratings documented at intake and case closure were included into the study sample.
Methods (cont.):
213 Omaha Problems, generated from 123 families, met the study criteria,and were included into the study’s sample for analysis.
Data Analysis
As anticipated, the most frequently opened Omaha System problems were for Caretaking and Parenting (37% of opened problems) and Child Growth and Development (17% of opened problems).
The next most selected problem was Residence (10% of opened problems), which is consistent with the numbers of children with homelessness risks and safety issues in their homes.
Omaha System Problem Identified and Opened for EIP/EFSS Clients
Number/%of Clients with the
ProblemAbuse Domestic Violence 10 (5%)
Abuse- Child 10 (5%)
Caretaking/Parenting 79 (37%)
Growth and Development 36 (17%)
Health Care Supervision 9 (4%)
Income 9 (4%)
Mental Health 19 (9%)
Neglect- Child 6 (3%)
Residence 22 (10%)
Substance Use - Drugs & Alcohol 11 (5%)
Total Number of Omaha System Problems 211
Opended Omaha System Problems
Data Analysis (cont.)
EIP/EFSS Omaha System KBS scores were aggregated and compared from case intake to case closure.
Across all the Omaha problems the average intake KBS ratings were at the lower range or a 1-3 rating. This would be expected considering the high risk factors present in this population.
Despite average low intake ratings, the average change in KBS exit ratings across all the Omaha System problems showed that there was a positive change of 0.5 or greater for most ratings.
Data Analysis (cont.)
The most significant positive observed changes across all KBS rating areas were for the Mental Health and Substance Abuse (Drugs & Alcohol) Omaha System problems. This was a surprising finding as it had been anticipated that these would be the problems that would be the most difficult to make change in.
Summary of KBS Rating Changes from Intake to Case Closure
Omaha System Problem
Number of
Clients
Average Change In
Knowledge
Average Change In
BehaviorAverage Change
In Status
Abuse Domestic Violence 10 0.80 0.60 1.00
Abuse- Child 10 0.70 0.30 0.90
Caretaking/Parenting 79 0.75 0.66 0.67
Growth and Development 36 0.83 0.69 0.53
Health Care Supervision 9 0.56 0.67 0.56
Income 9 1.00 0.78 0.89
Mental Health 19 0.95 1.26 1.00
Neglect- Child 6 0.83 0.17 0.83
Residence 22 0.77 0.55 0.77Substance Use Drug &
Alcohol 11 1.18 1.18 1.27
Substance Use Tobacco 2 0.00 0.00 0.00
1.0 Average Change in Mental Health Status Scores
Mental Health Status Intake and Exit Scores
0
2
4
6
8
10
12
14
1 2 3 4 5
Score
Intake
Exit
Mental Health Status Rating
Number Clients With “1” Rating
Number ClientsWith “2” Rating
Number ClientsWith “3” Rating
Number ClientsWith “4” Rating
Number ClientsWith “5” Rating
Total NumberClients with Problem
Rating at Intake 3 13 3 0 0 19Rating at Exit 0 5 9 5 0 19
1.27 Average Change in Substance Abuse Status Scores
Substance Use Status Intake and Exit Scores
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
1 2 3 4 5
Score
Intake
Exit
Substance Abuse – Drugs and Alcohol Status Rating
Number Clients With “1” Rating
Number ClientsWith “2” Rating
Number ClientsWith “3” Rating
Number ClientsWith “4” Rating
Number ClientsWith “5” Rating
Total NumberClients with Problem
Rating at Intake 4 4 2 1 0 11Rating at Exit 1 3 1 4 2 11
Data Analysis (cont.)
In analyzing the KBS intake and exit ratings it was observed that several problems had no changes in KBS ratings or that there were negative changes in the ratings. As some of the children were closed due to out-of-home placements, or lost to follow up this would be anticipated.
These KBS ratings helps to illustrate that the providers were able to report positive as well as negative changes with their clients.
Discussion
The use of the Omaha System easily allows for the aggregation of program data.
The KBS ratings can be shown numerically and graphically in ways that are easy to understand. This is important as the data was shared with a Child Protection Agency outside the health care system.
The use of Omaha System Problems and KBS ratings was an important improvement over previous methods of program evaluation such as interviewing staff/community partners, and reviewing client records/reports.
Discussion (cont.)
Through the use of the Omaha System, we were able to successfully demonstrate improvements in serious problems for maltreated children and their caregivers, after their participation with PHN services.
This data analysis project has contributed to a better understanding the needs of our vulnerable population and the effectiveness of program services.
Public Health Seattle King County EIP/EFSS Program Contacts: Deborah Greenleaf, Program Lead
(206) 263-8375 Rebecca Benson, Program Referral Coordinator
(206) 263-8381 Lois Schipper, Program Supervisor
(206) 263-8378