looking for patterns in child outcome data – examples from nys
DESCRIPTION
Looking for Patterns in Child Outcome Data – Examples from NYS. New York State Department of Health Bureau of Early Intervention. New York’s Child & Family Outcomes System. New York’s Early Intervention Program (NYSEIP) is one of the largest in the nation - PowerPoint PPT PresentationTRANSCRIPT
Looking for Patterns in Child Outcome Data – Examples from NYS
New York State Department of Health
Bureau of Early Intervention
July 30, 2010 Measuring Child & Family Outcomes 2
New York’s Child & Family Outcomes System
• New York’s Early Intervention Program (NYSEIP) is one of the largest in the nation– Provides services to nearly 75,000 children and
families each year– 58 local programs administered by 57 county
government agencies and New York City
• Given size and scope of the NYSEIP, a sampling approach has been implemented for collection of child and family outcome data
July 30, 2010 Measuring Child & Family Outcomes 3
New York’s Child & Family Outcomes System
• Child outcomes data are collected using the COSF
• Each year, local programs enroll up to a designated number of children in child outcomes samples
• COSF forms are completed at entry and exit for children enrolled in child outcomes samples
July 30, 2010 Measuring Child & Family Outcomes 4
New York’s Child & Family Outcomes System
• A memorandum of understanding with the University at Buffalo, School of Public Health and Health Professions, Population Health Observatory, to partner with BEI in implementing New York’s child and family outcomes system– Responsible for data entry, quality control, management,
and analyses– Maintains a suballocation with the University At
Binghamton, Institute for Child Development, for materials development, training and technical assistance, and annual family survey
July 30, 2010 Measuring Child & Family Outcomes 5
New York’s Child & Family Outcomes System
• Special thanks to University at Buffalo, PHO staff who collaborated on the analyses and presentation– Li Yan, Ph.D., M.A., Data Manager– Amy Barczykowski, M.S.Ed., Project Coordinator– Randy Carter, Ph.D., Director of PHO
• Our partners at University at Binghamton staff are– Ray Romanczyk, Ph.D., Director, Institute for Child
Development– Emily Huber-Callahan– Jamie Silverio
July 30, 2010 Measuring Child & Family Outcomes 6
Pattern 1a
Children will differ from one another in their entry scores in reasonable ways (few scores at the high and low ends and more in the middle)
July 30, 2010 Measuring Child & Family Outcomes 7
Pattern 1aEntry Scores Across All 3 Outcome Areas
Outcome 1 Positive Social Emotion SkillsOutcome 2 Acquire and Use Knowledge and SkillsOutcome 3 Take Appropriate Action to Meet Needs
July 30, 2010 Measuring Child & Family Outcomes 8
Pattern 1b
Children will differ from one another in their exit scores in reasonable ways
July 30, 2010 Measuring Child & Family Outcomes 9
Pattern 1b
Exit Scores Across all 3 Outcome Areas
Outcome 1 Positive Social Emotion SkillsOutcome 2 Acquire and Use Knowledge and SkillsOutcome 3 Take Appropriate Action to Meet Needs
July 30, 2010 Measuring Child & Family Outcomes 10
Pattern 1c
Children will differ from one another in their OSEP Progress Categories in reasonable ways
July 30, 2010 Measuring Child & Family Outcomes 11
Pattern 1c
OSEP Categories Across all 3 Outcome Areas
a Children who did not improve functioningb Children who improved functioning but not sufficient to move nearer to functioning comparable to same age
peersc Children who improved functioning to a level nearer to same-aged peers but did not reach itd Children who improved functioning to reach a level comparable to same-aged peerse Children who maintained functioning at a level comparable to same-aged peers
July 30, 2010 Measuring Child & Family Outcomes 12
Pattern 2
Functioning in one outcome area will be related to functioning in other outcome areas
July 30, 2010 Measuring Child & Family Outcomes 13
Pattern 2
Comparisons Between Outcome 1 and Outcome 2
July 30, 2010 Measuring Child & Family Outcomes 14
Pattern 2
Comparisons Between Outcome 1 and Outcome 2
Exit
Outcome 1 x Outcome 2
Outcome 1 Outcome 2
1 2 3 4 5 6 7 TOTAL
1 14 3 5 2 1 0 0 252 5 31 18 4 4 0 0 623 2 21 114 35 25 13 0 2104 1 5 41 77 68 17 3 2125 0 8 32 66 173 100 18 3976 1 0 13 28 122 294 89 5477 0 0 1 10 50 143 362 566
TOTAL 23 68 224 222 443 567 472 2019
July 30, 2010 Measuring Child & Family Outcomes 15
Pattern 2 Comparisons Between Outcome 1
and Outcome 2 OSEP CategoriesOutcome 1 and Outcome 2 OSEP Scores
Outcome 1 x Outcome 2
Outcome 1 Outcome 2
a b c d e TOTAL
a 27 19 18 5 2 71
b 17 211 138 43 20 429
c 5 67 253 70 11 406
d 2 31 84 353 68 538
e 6 46 56 220 247 575
TOTAL 57 374 549 691 348 2019
July 30, 2010 Measuring Child & Family Outcomes 16
Pattern 3
Functioning at entry within one outcome area will be related to functioning at exit (children who have higher functioning at entry in an area will be the ones who are high functioning at exit in that area)
July 30, 2010 Measuring Child & Family Outcomes 17
Pattern 3
Comparisons Between Entry Outcome 2 and Exit Outcome 2
Entry and Exit
Entry Outcome 2 by Exit Outcome 2
Entry Outcome 2
Exit Outcome 2
1 2 3 4 5 6 7 TOTAL
1 6 13 25 9 15 11 3 82 2 6 22 66 41 62 39 23 259 3 5 12 80 94 135 103 59 488 4 2 6 26 38 89 109 72 342 5 2 9 19 19 87 169 103 408 6 2 4 5 16 38 106 106 277 7 0 2 3 5 17 30 106 163
TOTAL 23 68 224 222 443 567 472 2019
July 30, 2010 Measuring Child & Family Outcomes 18
Pattern 4
Most children will either hold their developmental trajectories or improve their trajectories from entry to exit
July 30, 2010 Measuring Child & Family Outcomes 19
Pattern 4
Outcome 2Exit Score-Entry Score
Outcome 2 Acquire and Use Knowledge and Skills
July 30, 2010 Measuring Child & Family Outcomes 20
Pattern 5
Entry, exit, and OSEP progress category distributions should be similar from year to year
July 30, 2010 Measuring Child & Family Outcomes 21
Pattern 5
Outcome 2 Entry Scores Across 4 Cohort Years
Outcome 2 Acquire and Use Knowledge and Skills
July 30, 2010 Measuring Child & Family Outcomes 22
Pattern 5
Outcome 2 Exit Scores Across 3 Exiting Periods
Outcome 2 Acquire and Use Knowledge and Skills
July 30, 2010 Measuring Child & Family Outcomes 23
Pattern 5
Outcome 2OSEP Categories Across 3 Entry Cohort Years
a Children who did not improve functioningb Children who improved functioning but not sufficient to move nearer to functioning comparable to same age
peersc Children who improved functioning to a level nearer to same-aged peers but did not reach itd Children who improved functioning to reach a level comparable to same-aged peerse Children who maintained functioning at a level comparable to same-aged peers
July 30, 2010 Measuring Child & Family Outcomes 24
Pattern 7
Entry and exit scores and OSEP categories should be related to the nature of the child’s disability
July 30, 2010 Measuring Child & Family Outcomes 25
Pattern 7
Children with Only a Communication Delay ENTRY Scores for 3 Outcome Areas
Outcome 1 Positive Social Emotion SkillsOutcome 2 Acquire and Use Knowledge and SkillsOutcome 3 Take Appropriate Action to Meet Needs
July 30, 2010 Measuring Child & Family Outcomes 26
Pattern 7
Children with Only a Communication Delay EXIT Scores for 3 Outcome Areas
Outcome 1 Positive Social Emotion SkillsOutcome 2 Acquire and Use Knowledge and SkillsOutcome 3 Take Appropriate Action to Meet Needs
July 30, 2010 Measuring Child & Family Outcomes 27
Pattern 7
OSEP Categories for Children with Only a Communication Delay
a Children who did not improve functioning
b Children who improved functioning but not sufficient to move nearer to functioning comparable to same age peers
c Children who improved functioning to a level nearer to same-aged peers but did not reach it
d Children who improved functioning to reach a level comparable to same-aged peers
e Children who maintained functioning at a level comparable to same-aged peers
July 30, 2010 Measuring Child & Family Outcomes 28
Pattern 8
Scores at entry and exit should not be related to certain characteristics
July 30, 2010 Measuring Child & Family Outcomes 29
Pattern 8
Outcome 2Percentage of Male and Female Entry Scores
Outcome 2 Acquire and Use Knowledge and Skills
July 30, 2010 Measuring Child & Family Outcomes 30
Pattern 8
Outcome 2Medicaid Status and Entry Scores
Outcome 2 Acquire and Use Knowledge and Skills
July 30, 2010 Measuring Child & Family Outcomes 31
Conclusion/Next Steps
• Pattern checking is a useful way to begin the data quality assurance/quality improvement process
• Analyzing the data in these ways has helped NY target areas for improvement
• Planning/designing a webinar to share data with local programs
• Identifying other strategies to provide training and technical assistance to programs, providers, and families