measuring child progress: two state’s journeys barbara jackson, ne beppie shapiro, hi measuring...
TRANSCRIPT
![Page 1: Measuring Child Progress: Two State’s Journeys Barbara Jackson, NE Beppie Shapiro, HI Measuring Child and Family Outcomes Albuquerque, NM April 25, 2006](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f0c5503460f94c1f312/html5/thumbnails/1.jpg)
Measuring Child Progress:Measuring Child Progress:Two State’s JourneysTwo State’s Journeys
Barbara Jackson, NEBarbara Jackson, NE
Beppie Shapiro, HIBeppie Shapiro, HI
Measuring Child and Family Outcomes
Albuquerque, NM
April 25, 2006
![Page 2: Measuring Child Progress: Two State’s Journeys Barbara Jackson, NE Beppie Shapiro, HI Measuring Child and Family Outcomes Albuquerque, NM April 25, 2006](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f0c5503460f94c1f312/html5/thumbnails/2.jpg)
Format of PresentationFormat of Presentation
• Meet each stateMeet each state
• Overview of the 4 key questions to Overview of the 4 key questions to be discussedbe discussed
• For each key question:For each key question:
– Description of each state’s plansDescription of each state’s plans
– Questions and comments from Questions and comments from participantsparticipants
![Page 3: Measuring Child Progress: Two State’s Journeys Barbara Jackson, NE Beppie Shapiro, HI Measuring Child and Family Outcomes Albuquerque, NM April 25, 2006](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f0c5503460f94c1f312/html5/thumbnails/3.jpg)
What Counts:
Measuring Benefits of Early Intervention in Hawai`i
Beppie Shapiro, University of Hawai`[email protected]
![Page 4: Measuring Child Progress: Two State’s Journeys Barbara Jackson, NE Beppie Shapiro, HI Measuring Child and Family Outcomes Albuquerque, NM April 25, 2006](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f0c5503460f94c1f312/html5/thumbnails/4.jpg)
About Hawai`i
About 17,000 births per year
Over 7% of each birth cohort served in Part C
No majority ethnicity
4.1 % of population are immigrants, primarily Asians and Pacific Islanders
Geographic isolation
Five main islands – different access to services
![Page 5: Measuring Child Progress: Two State’s Journeys Barbara Jackson, NE Beppie Shapiro, HI Measuring Child and Family Outcomes Albuquerque, NM April 25, 2006](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f0c5503460f94c1f312/html5/thumbnails/5.jpg)
Special education in Hawai`i
Part C under Department of Health
Part B under Department of Education
Outcomes process differs from C to 619
SEA = 1 LEA
![Page 6: Measuring Child Progress: Two State’s Journeys Barbara Jackson, NE Beppie Shapiro, HI Measuring Child and Family Outcomes Albuquerque, NM April 25, 2006](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f0c5503460f94c1f312/html5/thumbnails/6.jpg)
Part C in Hawai`i, 2006
Broad eligibility
Three DOH agencies provide services statewide
Recent development of statewide IFSP to be used by all programs
No Part C database – 2 agencies have their own
![Page 7: Measuring Child Progress: Two State’s Journeys Barbara Jackson, NE Beppie Shapiro, HI Measuring Child and Family Outcomes Albuquerque, NM April 25, 2006](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f0c5503460f94c1f312/html5/thumbnails/7.jpg)
Standardized IFSP form, process
2005: All Part C programs trained to use new statewide IFSP form– Present Levels of Development– Family-driven Outcomes* and Objectives– Services to support Objectives– Transition
![Page 8: Measuring Child Progress: Two State’s Journeys Barbara Jackson, NE Beppie Shapiro, HI Measuring Child and Family Outcomes Albuquerque, NM April 25, 2006](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f0c5503460f94c1f312/html5/thumbnails/8.jpg)
What Counts?....What Counts?.... Measuring the Outcomes of Measuring the Outcomes of
Early Intervention in NebraskaEarly Intervention in Nebraska
Barbara Jackson, Munroe Meyer Institute
![Page 9: Measuring Child Progress: Two State’s Journeys Barbara Jackson, NE Beppie Shapiro, HI Measuring Child and Family Outcomes Albuquerque, NM April 25, 2006](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f0c5503460f94c1f312/html5/thumbnails/9.jpg)
About NebraskaAbout Nebraska
Birth Mandate State
Co-Leads for Part C: Health and Human Services & Education
Outcome data process will be the same across the birth through 5 age group
![Page 10: Measuring Child Progress: Two State’s Journeys Barbara Jackson, NE Beppie Shapiro, HI Measuring Child and Family Outcomes Albuquerque, NM April 25, 2006](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f0c5503460f94c1f312/html5/thumbnails/10.jpg)
Nebraska (continued)Nebraska (continued)
460 School Districts
Number of children birth to 3: 1303
Number of children ages 3 and 4: 2811
Number of children 3 through 5: 4707
![Page 11: Measuring Child Progress: Two State’s Journeys Barbara Jackson, NE Beppie Shapiro, HI Measuring Child and Family Outcomes Albuquerque, NM April 25, 2006](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f0c5503460f94c1f312/html5/thumbnails/11.jpg)
Key Question…………Key Question…………
1.1. How will the state determine children’s How will the state determine children’s status on each of the outcomes?status on each of the outcomes?
![Page 12: Measuring Child Progress: Two State’s Journeys Barbara Jackson, NE Beppie Shapiro, HI Measuring Child and Family Outcomes Albuquerque, NM April 25, 2006](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f0c5503460f94c1f312/html5/thumbnails/12.jpg)
Assessment in Hawai`i
Part B Section 619, Hawai`i Assessment: Early Brigance
Part C: Since 2004: 3 assessment tools approved for CDE – Hawaii Development Charts (HELP), “Michigan” EIDP, or ASQ
![Page 13: Measuring Child Progress: Two State’s Journeys Barbara Jackson, NE Beppie Shapiro, HI Measuring Child and Family Outcomes Albuquerque, NM April 25, 2006](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f0c5503460f94c1f312/html5/thumbnails/13.jpg)
Hawai`i’s Part C Assessment : Ages and Stages
For babies eligible due to environmental risk
Forms for specific age levelsScores indicate only “Typical”, “Monitor”, “Refer” Referred for CDE and services by program serving DDCare coordination may remain with original program
![Page 14: Measuring Child Progress: Two State’s Journeys Barbara Jackson, NE Beppie Shapiro, HI Measuring Child and Family Outcomes Albuquerque, NM April 25, 2006](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f0c5503460f94c1f312/html5/thumbnails/14.jpg)
1. What Counts Design Team convenes monthly to develop/review progress
2. How can we assign a “score” to each child on each EI Goal?
What Counts: Measuring the Benefits of Early Intervention in Hawai`i:
![Page 15: Measuring Child Progress: Two State’s Journeys Barbara Jackson, NE Beppie Shapiro, HI Measuring Child and Family Outcomes Albuquerque, NM April 25, 2006](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f0c5503460f94c1f312/html5/thumbnails/15.jpg)
Hawai`i’s Part C Outcomes (Goals) Measurement Process
Data Collection Intervals & Rationale– At every IFSP: initial, review, annual
–Starting when child is at least 4 months old
![Page 16: Measuring Child Progress: Two State’s Journeys Barbara Jackson, NE Beppie Shapiro, HI Measuring Child and Family Outcomes Albuquerque, NM April 25, 2006](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f0c5503460f94c1f312/html5/thumbnails/16.jpg)
Hawai`i’s Part C Outcomes (Goals) Measurement Process
Assigning scores: decisions & rationale
WHO: IFSP team
Following “Present levels of Development” description
HOW: all team members reach consensus on rating using modified ECO Child Summary Reporting Form
![Page 17: Measuring Child Progress: Two State’s Journeys Barbara Jackson, NE Beppie Shapiro, HI Measuring Child and Family Outcomes Albuquerque, NM April 25, 2006](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f0c5503460f94c1f312/html5/thumbnails/17.jpg)
Hawai`i’s 619 Outcomes Measurement Process
Family involved in assessment process Family provides input on written form with open-ended questionsA professional assigns rating on each goal Other providers asked for input to ratingRatings assigned at entry and annually
![Page 18: Measuring Child Progress: Two State’s Journeys Barbara Jackson, NE Beppie Shapiro, HI Measuring Child and Family Outcomes Albuquerque, NM April 25, 2006](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f0c5503460f94c1f312/html5/thumbnails/18.jpg)
Nebraska’s ProcessNebraska’s Process
Child Outcome Task Force convened to guide process
![Page 19: Measuring Child Progress: Two State’s Journeys Barbara Jackson, NE Beppie Shapiro, HI Measuring Child and Family Outcomes Albuquerque, NM April 25, 2006](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f0c5503460f94c1f312/html5/thumbnails/19.jpg)
Nebraska calls for Child Assessment Nebraska calls for Child Assessment that….that….
Is based on ongoing observation of children engaged in real activities, with people they know, in natural settings
Engage families and primary caregivers as active participants
Is individualized to address each child’s unique ways of learning
Reflects that development and learning are rooted in culture and supported by the family
Integrates information across settings
![Page 20: Measuring Child Progress: Two State’s Journeys Barbara Jackson, NE Beppie Shapiro, HI Measuring Child and Family Outcomes Albuquerque, NM April 25, 2006](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f0c5503460f94c1f312/html5/thumbnails/20.jpg)
Nebraska’s Assessment ProcessNebraska’s Assessment Process
Which assessments?
AEPS Creative Curriculum High Scope COR for Infants/
Toddlers & Preschoolers
![Page 21: Measuring Child Progress: Two State’s Journeys Barbara Jackson, NE Beppie Shapiro, HI Measuring Child and Family Outcomes Albuquerque, NM April 25, 2006](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f0c5503460f94c1f312/html5/thumbnails/21.jpg)
Why Selected?Why Selected?
Assessment approach parallels other Early Childhood Program assessment processes
Use information from multiple sources (e.g., family, providers) and multiple observations
Curriculum-based Assessment – can be used for multiple purposes
Can build the capacity of our system to support children and their families
![Page 22: Measuring Child Progress: Two State’s Journeys Barbara Jackson, NE Beppie Shapiro, HI Measuring Child and Family Outcomes Albuquerque, NM April 25, 2006](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f0c5503460f94c1f312/html5/thumbnails/22.jpg)
Nebraska’s Nebraska’s Process for parent input?Process for parent input?
Parents provide input during the assessment process
![Page 23: Measuring Child Progress: Two State’s Journeys Barbara Jackson, NE Beppie Shapiro, HI Measuring Child and Family Outcomes Albuquerque, NM April 25, 2006](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f0c5503460f94c1f312/html5/thumbnails/23.jpg)
How is the assessment information How is the assessment information “transformed”?“transformed”?
Working with publishers to determine feasibility of computer-based formulas
Scores will be reviewed by team to assure validity of score
![Page 24: Measuring Child Progress: Two State’s Journeys Barbara Jackson, NE Beppie Shapiro, HI Measuring Child and Family Outcomes Albuquerque, NM April 25, 2006](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f0c5503460f94c1f312/html5/thumbnails/24.jpg)
Nebraska’s Assessment ProcessNebraska’s Assessment Process
Data Collection Schedule Districts will be mandated to report entry
and exit data Entry data will collected within the 45-60
days after IFSP/IEP meeting Exit data collected within 2 months of
leaving the program Districts will be encouraged follow
publishers guidelines for frequency of assessments
![Page 25: Measuring Child Progress: Two State’s Journeys Barbara Jackson, NE Beppie Shapiro, HI Measuring Child and Family Outcomes Albuquerque, NM April 25, 2006](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f0c5503460f94c1f312/html5/thumbnails/25.jpg)
Key Question…………Key Question…………
2. What reporting categories has the 2. What reporting categories has the state chosen to use?state chosen to use?
![Page 26: Measuring Child Progress: Two State’s Journeys Barbara Jackson, NE Beppie Shapiro, HI Measuring Child and Family Outcomes Albuquerque, NM April 25, 2006](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f0c5503460f94c1f312/html5/thumbnails/26.jpg)
Reporting Categories in Hawai`i
What Counts: Measuring Benefits of Early Intervention in Hawai`i
![Page 27: Measuring Child Progress: Two State’s Journeys Barbara Jackson, NE Beppie Shapiro, HI Measuring Child and Family Outcomes Albuquerque, NM April 25, 2006](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f0c5503460f94c1f312/html5/thumbnails/27.jpg)
Reporting Categories in Hawai`i
Part C Decision & RationaleUse ECO 5 CategoriesA. maintained typical functioning
B. made progress to achieve typical functioning
C. moved nearer to typical functioning
D. progress but not enough to move nearer to typical
E. did not make progress
Meaningful, Program Improvement
![Page 28: Measuring Child Progress: Two State’s Journeys Barbara Jackson, NE Beppie Shapiro, HI Measuring Child and Family Outcomes Albuquerque, NM April 25, 2006](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f0c5503460f94c1f312/html5/thumbnails/28.jpg)
Reporting Categories in Hawai`i
619– OSEP Categories
![Page 29: Measuring Child Progress: Two State’s Journeys Barbara Jackson, NE Beppie Shapiro, HI Measuring Child and Family Outcomes Albuquerque, NM April 25, 2006](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f0c5503460f94c1f312/html5/thumbnails/29.jpg)
Reporting Categories in NEReporting Categories in NE
Four Categories OSEP’s 3 categories
Plus % of children who attain typical development
![Page 30: Measuring Child Progress: Two State’s Journeys Barbara Jackson, NE Beppie Shapiro, HI Measuring Child and Family Outcomes Albuquerque, NM April 25, 2006](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f0c5503460f94c1f312/html5/thumbnails/30.jpg)
Key Question…………Key Question…………
3. How and in what form will data get 3. How and in what form will data get from local programs to the state?from local programs to the state?
![Page 31: Measuring Child Progress: Two State’s Journeys Barbara Jackson, NE Beppie Shapiro, HI Measuring Child and Family Outcomes Albuquerque, NM April 25, 2006](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f0c5503460f94c1f312/html5/thumbnails/31.jpg)
Getting Part C Data to the StateIn Hawai`i
Simpler than most states?
More difficult than many states?
Each “Agency” will collect and summarize data from its programs
![Page 32: Measuring Child Progress: Two State’s Journeys Barbara Jackson, NE Beppie Shapiro, HI Measuring Child and Family Outcomes Albuquerque, NM April 25, 2006](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f0c5503460f94c1f312/html5/thumbnails/32.jpg)
Data to explain results
Length of enrollmentAge at enrollmentICD9s/conditions
![Page 33: Measuring Child Progress: Two State’s Journeys Barbara Jackson, NE Beppie Shapiro, HI Measuring Child and Family Outcomes Albuquerque, NM April 25, 2006](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f0c5503460f94c1f312/html5/thumbnails/33.jpg)
Getting Data to the State: Part C
Electronic data entry at local programTransmitted to AgencyAgency calculates number of children in each categoryAgency sends these numbers to State State creates OSEP and cross-Agency reports
![Page 34: Measuring Child Progress: Two State’s Journeys Barbara Jackson, NE Beppie Shapiro, HI Measuring Child and Family Outcomes Albuquerque, NM April 25, 2006](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f0c5503460f94c1f312/html5/thumbnails/34.jpg)
Unresolved data questions
Local programs send “explanatory data” to Agency– How much of this data goes to State?
How?
How to identify potential duplicates
![Page 35: Measuring Child Progress: Two State’s Journeys Barbara Jackson, NE Beppie Shapiro, HI Measuring Child and Family Outcomes Albuquerque, NM April 25, 2006](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f0c5503460f94c1f312/html5/thumbnails/35.jpg)
Getting Data to the State: 619
Teacher enters data into stand-alone EXCEL spreadsheet
Spreadsheets collected at school, sent to State
![Page 36: Measuring Child Progress: Two State’s Journeys Barbara Jackson, NE Beppie Shapiro, HI Measuring Child and Family Outcomes Albuquerque, NM April 25, 2006](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f0c5503460f94c1f312/html5/thumbnails/36.jpg)
NE: Getting Data to the StateNE: Getting Data to the State
Using the publisher’s internet system- State will be the licensed manager
Link with the State Data System
![Page 37: Measuring Child Progress: Two State’s Journeys Barbara Jackson, NE Beppie Shapiro, HI Measuring Child and Family Outcomes Albuquerque, NM April 25, 2006](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f0c5503460f94c1f312/html5/thumbnails/37.jpg)
Key Question…………Key Question…………
4. What will you do to maximize the 4. What will you do to maximize the reliability of the data?reliability of the data?
![Page 38: Measuring Child Progress: Two State’s Journeys Barbara Jackson, NE Beppie Shapiro, HI Measuring Child and Family Outcomes Albuquerque, NM April 25, 2006](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f0c5503460f94c1f312/html5/thumbnails/38.jpg)
Training, Support to Maximize Data Reliability in Hawai`i
Training:Training:
Support:Support:
Web:Web:
7 hrs, all EI programs, by 7 hrs, all EI programs, by communitycommunity
1-2 months post training: 1-2 months post training:
On site? Conference call? On site? Conference call?
FAQs, email listservFAQs, email listserv
![Page 39: Measuring Child Progress: Two State’s Journeys Barbara Jackson, NE Beppie Shapiro, HI Measuring Child and Family Outcomes Albuquerque, NM April 25, 2006](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f0c5503460f94c1f312/html5/thumbnails/39.jpg)
Maximizing Data Reliability in Hawai`i: Evidence for Rating
IFSP teams document evidence used to select rating category
Pilot showed need for training on what makes good evidence
Supervisors provide support and QA
Criteria for Evidence Criteria for Evidence (in progress)(in progress)
Different contextsDifferent contexts
Highest level of Highest level of achievementachievement
Specific to Goal AreaSpecific to Goal Area
![Page 40: Measuring Child Progress: Two State’s Journeys Barbara Jackson, NE Beppie Shapiro, HI Measuring Child and Family Outcomes Albuquerque, NM April 25, 2006](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f0c5503460f94c1f312/html5/thumbnails/40.jpg)
Examining Reliability in Hawai`i
Another measurement of “how well child is doing”
– Compare ratings
Compare ratings for children with different conditions/diagnoses
![Page 41: Measuring Child Progress: Two State’s Journeys Barbara Jackson, NE Beppie Shapiro, HI Measuring Child and Family Outcomes Albuquerque, NM April 25, 2006](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f0c5503460f94c1f312/html5/thumbnails/41.jpg)
Challenges for Hawai`i: Quality Assurance - Part CHow reliable are assessments?
Does every IFSP team have someone knowledgeable about typical child development?
How reliable are summary form ratings?
![Page 42: Measuring Child Progress: Two State’s Journeys Barbara Jackson, NE Beppie Shapiro, HI Measuring Child and Family Outcomes Albuquerque, NM April 25, 2006](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f0c5503460f94c1f312/html5/thumbnails/42.jpg)
Reliability of Data in NE: TrainingReliability of Data in NE: Training
Intensive training on assessments throughout state with consistent trainers
![Page 43: Measuring Child Progress: Two State’s Journeys Barbara Jackson, NE Beppie Shapiro, HI Measuring Child and Family Outcomes Albuquerque, NM April 25, 2006](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f0c5503460f94c1f312/html5/thumbnails/43.jpg)
Reliability of Data in NE: Reliability of Data in NE: ImplementationImplementation
Review of data at the state level
Ongoing questions/answers communication document based on local questions
Team approach to assessment process
Pilot process to determine reliability issues