innovations in measuring outcomes in children’s school performance and social functioning...
TRANSCRIPT
Innovations in Measuring Outcomes in Children’s School Performance and
Social Functioning
Manitoba’s Child Well-Being Windows
Presenters: Linda Burnside and Kathy Kristjanson
2
Introduction
• Like other jurisdictions around the world, Manitoba has become increasingly interested in how we assess and measure the well-being of our children.
• This interest is particularly critical in the child welfare field, where the state has assumed parental responsibility for the care of children and therefore, responsibility for their well-being.
3
Challenges
• The issues that bring children to the attention of child welfare agencies are multi-dimensional, which adds to the complexity of defining and measuring child well-being.
• Child welfare agencies may be reluctant to measure child well-being, given that the issues that result in many children coming to the attention of child welfare are issues that have likely already compromised their well-being.
4
Purpose of Presentation• To share Manitoba’s approach to
measuring and tracking child well-being for children in care, in the following areas:
– School performance• Age/Grade Appropriateness• School Type• Literacy• Learning Disabilities• Speech and Language Development
– Child Functioning• Social Skills• Problem behaviors• Substance Abuse
5
Acknowledgements
• The following individuals have been instrumental in advancing Manitoba’s work on the measurement of child well-being:
– Nico Trocmé– Laurie Olafson– Rhonda Dagg– Dave McGregor
6
Manitoba CFS Context• 6629 (the number of children in care of Manitoba
child and family services agencies at March 31, 2006)
• 7364 (the number of families receiving service from Manitoba child and family services agencies)
• 25 (the number of agencies operating under four child and family services Authorities)
• 133 (the number of child and family services offices operating in Manitoba)
7
Manitoba CFS Context
Winnipeg
Brandon
8
Methodology for Demonstration Project
• Looked at School Performance and Social Skills data for all children in care at one child welfare agency in Manitoba.
• Demonstration data only, to illustrate capacity of the windows, based on one agency.
9
Demonstration Study Sample
Total Number of Children in Care: 115
744
27
19
Attending School Kindergarten/Pre School
Not school aged School aged: Not Attending
Special Needs Program
CFS Western Manitoba2005/06
10
Manitoba’s Approach
11
National Outcome Measures Project
• The National Outcome Measures Project originated with the provincial Directors of Child Welfare across Canada.
• Intent of this Canadian initiative is to measure multi-dimensional child welfare outcomes in four domains (Trocmé, Nutter, MacLaurin, & Fallon, 1999):
– Child safety– Child well being– Permanence– Family and community support
12
Child Welfare Outcome Indicator Matrix*
• Child Safety– Recurrence of Maltreatment– Serious Injuries/Death
• Child Well Being– School Performance– Child Behavior
• Permanence– Moves in Care– Admission to Care– Time to Achieving Permanent Placement
• Family and Community Support– Family Moves– Aboriginal Placement Matching– Parenting Capacity
* Trocmé, Nutter, MacLaurin, & Fallon, 1999
13
Informed by Other Models
• Looking After Children (LAC) is a UK-based approach to working with children in care which measures a child’s developmental progress in seven dimensions:
– health– education– identity– family and social relationships– social presentation skills– emotional and behavioral development– self-care
14
InformationTechnology
• The Child and Family Services Information System (CFSIS) for Manitoba is a comprehensive case tracking system that was designed in 1993 to facilitate the tracking of children and families served, as well as reporting on the services being provided.
• The Intake Module, which began in 2004, functions in conjunction with CFSIS to provide standardized recording of intake information throughout the province as required by legislation.
15
Child Well Being Windows
16
Child Well Being Windows
17
School Information
18
Social Skills
19
Data Collection and Measurement
• The applications create unique person records which may be attached to multiple intakes or CFSIS cases.
• CFSIS has a total of 436 tables which collect data from 3372 separate data fields.
• Any mix of this data can be produced in an ad hoc query.
20
Data Collection and Measurement
• Information related to child well-being is updated annually.
• All well-being windows have 2 print functions. One will print a blank form that workers can print off to have filled out by foster parent, teacher, etc.
• The other print function shows all the data that has been entered.
21
Benefits of Approach• Child-specific data gathered by those who
know the child best.
• Capacity to compare for each unique child with himself/herself over time (facilitates case planning and intervention for children who may need assistance in school performance or social skills development).
• Capacity for aggregate comparisons across agencies, regions, cultural groups, legal statuses, placement type, etc.
22
Limitations of Approach
• Data may be affected by:
– accurate data entry,– consistent definitions of terms,– how long the worker/teacher/foster parent has
worked with the child, – how well the worker/teacher/foster parent
knows the child,– turnover of staff, changes in placement, school
changes,– behavioral/emotional issues that interfere with
child’s expression of abilities.
23
Limitations of Approach
• Measurement may be affected by changes in child-in-care population due to:
– new admissions to care, – discharges from care, – children reaching age of majority.
24
School Performance
25
School Performance for Children in Care
• Several studies have found that youth in care are less likely to perform at age-appropriate grade level than their peers (e.g. Carey, Concannon & Goldschmidt, 1990).
• Research has also shown that children in care are less likely to graduate from high school (e.g. Barth, 2000; Westat, 1994; Roos, Brownell, & MacWilliam, 2007).
26
Manitoba School Performance Measures
• Age/Grade Appropriateness• Literacy• Numeracy• Speech and Language• Learning Disabilities• School Supports• Type of Classroom• Attendance• School Changes
27
School Performance
• A recent study of children in care involved with LAC found that 25% of youth were below an age appropriate grade level (Kufeldt, Simard, Tite & Vachon, 2003)
28
Age-Grade Appropriateness
2005/0615%
85%
Not Age GradeAppropriate
Age GradeAppropriate
CFS Western Manitoba
Children in Care
Attending School
29
School Grade Range2005/06
25
29
20
0
5
10
15
20
25
30
35
Grade 1-6 Grade 7-9 Grade 10-12
30
Age-Grade Appropriateness and
School Type
49
14
5
6
0
10
20
30
40
50
60
Regular Class Special Class
Age Grade Appropriate Not Age Grade Appropriate
31
Age-Grade Appropriateness and School Type/Grade
Range
18 18 13
3 56
0
10
20
30
Grade 1-6 Grade 7-9 Grade 10-12
Regular Class Special Class
32
Not Age-Grade Appropriate and School
Type/Grade Range
02
3
1 4
101
23
45
67
Grade 1-6 Grade 7-9 Grade 10-12
Regular Class Special Class
33
School Performance
• LAC study found that 40% of children ages 10 – 14 stated that they had a learning difficulty.* For youth ages 15 – 19, 27% stated that they had a learning difficulty.*
*Based on youth self-reporting
34
Learning Disability and Grade Range
18 1812
711
8
0
5
10
15
20
25
30
35
Grade 1-6 Grade 7-9 Grade 10-12
No Learning Disability Learning Disability
35
Learning Disability and School Supports
• LAC study found that only 9% of children who self-identified difficulty with learning were receiving extra help.
36
Learning Disability and School Supports
• 26 (35%) of the total 74 children in school were identified with a learning disability.
• 24 (92%) were receiving additional supports in the school.
• 18 (24%) children were not identified as having a learning disability, but were still receiving additional supports in school.
37
Learning Disability and School Supports
For the 63 children who were in an age-appropriate grade:
• 20 were identified with a learning disability and 18 of them were receiving school supports (90%).
• 43 children were not identified with a learning disability. However, 16 of these children were receiving school supports.
38
Learning Disability and School Supports
For the 11 children who were not in an age-appropriate grade:
• 6 children were identified with a learning disability, and all 6 were receiving school supports (100%).
• 5 children were not identified with a learning disability. However, 2 of these children were receiving school supports.
39
Learning Disabilityand School Supports
• Teaching assistant• Resource support• Counselling• Reading assistance• Occupational therapy• Speech/language therapy• Psychology• Audiology• Psychiatry
40
Literacy and School Type
• Shin (2003) found that receiving special education was related to poorer reading skills.
41
Literacy and School Type
• Of the 54 children attending regular class settings, 18 required ongoing assistance with literacy (33%).
• Of the 20 children attending special class settings, 13 required ongoing assistance with literacy (65%).
42
Literacy and Placement Type
• Shin (2003) found that youth in kinship care had higher reading levels than those who were not in kinship care.
43
Literacy andPlacement Type
• Of the 74 children in care, 46 were in a foster home, with 19 (41%) of these children requiring assistance with literacy.
• Of the 15 children placed in kinship settings, 5 (33%) required assistance with literacy.
44
School Changes
• Only 8% of the LAC study children had no school changes.
• 30% of the LAC sample had more than 6 school changes.
45
School Changes
• 57 children of the 74 children in care attending school experienced no change in school (77%).
• 17 children did experience a change in schools (23%).
46
School Changes and Age-Appropriate Grade
• For the 63 children in an age-appropriate grade, 14 experienced a change in school (22%).
• For the 11 children not in an age-appropriate grade, 3 experienced a change in school (27%).
47
Speech and Language
15 17 16
1012
4
0
5
10
15
20
25
30
35
Grade 1-6 Grade 7-9 Grade 10-12
Meets Expectations Needs Ongoing Help
48
Speech and Language and Literacy
• Of the 74 children in school, 48 (65%) meet or exceed expectations in speech and language performance.
• Of those 48, 41 meet or exceed expectations in literacy (85%).
• Of the 26 who require ongoing help in speech and language, only 2 met expectations in literacy (8%) and 24 required ongoing help in literacy (92%).
49
Child Functioning
50
Manitoba Social Skills Measures
• Emotional Maturity
• Making Friends
• School Activities
• Problem Behaviors
• Substance Abuse
51
Emotional Maturity
204
43
70
10
20
30
40
50
60
70
Age Grade Appropriate Not Age Grade Appropriate
Meets/Exceeds Expectations Needs Some/Ongoing Help
52
Making Friends
27
5
36
60
10
20
30
40
50
60
70
Age Grade Appropriate Not Age Grade Appropriate
Meets/Exceeds Expectations Needs Some/Ongoing Help
53
Extracurricular Activities/Educational
Attainment
• Shin (2003) found that active participation in extracurricular activities has a strong association with educational attainment.
54
School Activities/Cognitive Development
• Of the 63 children in an age-appropriate grade, 23 were involved in school activities (37%).
• A total of 37 of the 63 children were meeting school expectations in terms of cognitive development.
55
School Activities/Cognitive Development
• Of the 11 children who were not in an age-appropriate grade, 4 were involved in school activities (36%).
• A total of 6 were meeting school expectations in terms of cognitive development.
56
Behavioral Issues
• Inability to manage own behavior• Inability to interact constructively with
others• Aggression• Inability to adhere to structure and
expectations• Not aware of risk to self/others• Out of control behavior• Illegal activity• Sexually inappropriate acting out
57
Substance Abuse
• Alcohol• Marijuana• Substance not identified• Cocaine• Ecstasy• Prescription drugs
58
Substance Use and Literacy
• Shin (2003) found that youth who were involved in drug use had lower reading skills than youth who had not used drugs.
59
Substance Useand Literacy
• Of the 74 children attending school, 11 were identified as having a substance abuse issues (15%).
• Of those 11 children, 6 required ongoing help with literacy (55%).
• Of the 63 children who did not have substance abuse issues, 24 required ongoing help with literacy (39%).
60
Substance Abuse and Attendance
• School attendance is a concern for 8 of the 11 children with substance abuse issues.
61
Substance Abuse/Speech and Language
• Of the 74 children attending school, 26 require ongoing help with speech and language.
• 5 of these children have an identified substance abuse issue.
62
School Change/Substance Abuse
• The LAC study found that school disruption is highly correlated to the risk of substance use.
63
School Change/Substance Abuse
• A total of 17 children had a change in school. Of those, 6 had a substance abuse issue (35%).
64
Implementation Challenges
• Manitoba is initiating a process to provide agencies with the necessary computer hardware and staff training to allow for the use of CFSIS throughout the province.
• Further, all agencies will use the same information system process for new intakes, which will allow for the uniform collection of data at the onset of service.
65
Opportunities
• Valuable source of information for social workers, agencies, and the government.
• Strategy to further measurement of child well-being, both for National Outcome Measures Project and for the knowledge base of outcome measurement.
• Research opportunities that can be generated from the data collected.
66
Questions?
• For further information, please contact the Manitoba Child Protection Branch, c/o:
Linda BurnsideChild Protection BranchManitoba Department of Family Services
and Housing201 – 114 Garry StreetWinnipeg, ManitobaR3C 4V5 [email protected]