1

Birth to 3Child Outcomes

Maryland’s Approach to Converting Assessment Data to

OSEP Outcome Categories

August 28, 2007

Deborah Metzger

Dmetzger@msde.state.md.us

Funded by IDEA General Supervision Enhancement Grant (GSEG) CFDA84.326X

2

Outcome Data CollectedWhen the Child Enters and When the Child ExitsExample

Child enters at 20 months

Initial Present Levels of Development

data are extracted fromthe IFSP database

Child exits at 36 months

Present Levels ofDevelopment data at

Exit are extracted from the IFSP database

and compared to entrydata to determine

progress

Status At Entry Data Progress At Exit Data

3

How Do We Get The Data? (for Status At Entry and Progress at Exit)Data Extracted from Present Levels of Development and Electronically Linked to 3 Outcomes to Produce Answers

Is the child’s acquisition and use of knowledge and skills (including

early language/communication) at the level expected for his or her age? 

Are child’s social-emotional skills (including social relationships) at the level expected for his or her age? 

Does the child use appropriate behavior to meet his or her needs

at the level expected for his or her age?

Alignment of broad outcomes to Present Levels of Development

4

Protocols for Linking Age Levels/Age Ranges with Outcomes

For the outcome “acquisition and use of knowledge and skills (including early language/communication),” two domain categories (cognitive and communication) will be used. If both domains have quantitative data, the category that has the lowest range of data will be used.

When an age range has been entered, the midpoint of the range will be used.

5

Rationale for Maryland’s Approach

Decision to align outcome process with the IFSP process

Focus on improving evaluation and assessment practices (Online Tutorial)

Focus on ensuring data is collected in all domains (Monitoring)

Have a data system that collects Present Levels of Development (PLOD)

Could get started by generating electronic reports from data entered into PLOD

Response to local input

6

Most Commonly Used Toolsin Maryland

Early Learning Accomplishment Profile (E-LAP)

Early Intervention Developmental Profile (EIDP)

Hawaii Early Learning Profile (HELP)

Battelle Developmental Inventory II

Ages and Stages

7

How will Local Programs Measure and Report Status at Exit Data

Assessment at exit results will be entered on the IFSP form, Section II: Present Levels of Development (PLOD). The form will be revised so that providers can check off if the PLOD are from an entry, interim, or exit assessment.

The name of the assessment(s) that were used will be documented on the IFSP form.

The results of the assessments at exit will be entered into the new screens in the IFSP database.

8

Need to Validate the DataCompare Domain Results to Functional Results

9

Child Outcomes—ValidationWHY?

Maryland has elected to use the information from domain-specific assessment results to determine the results of functional child outcomes. Therefore it is important that the electronic results be validated.In other words, we are asking the following:

“Are the responses derived from the electronically-extracted domain data consistent with direct responses from providers about a child’s functioning in the three outcomes?”

10

How Will We Validate the Results?

In December 2006, local Infants and Toddlers Programs began completing the Child Outcome Summary Form as soon as possible following initial evaluation and assessment.

Local programs are completing COSFs at exit for children:•Who were referred since December 2006•Who received services for at least six months, and•For whom a COSF was completed at entry

COSF results will be entered into the IFSP database.

11

What if the “electronic results” and “validation results” are different?

During 2007-2008, we will begin to conduct a validation study by comparing electronic results and COSF results, conducting focus groups with local programs, and discussing discrepancies in results with local provider teams.

Based on the validation study results, we will determine how our approach should be modified or changed, whether we will continue our progress reporting by using Present Levels of Development data or through the COSF.

12

Converting Assessment Data to OSEP Outcome Categories

13

Measuring Progress Based on the Rate of Growth Between Entry and Exit

•Working with Evaluation and Assessment Consultant to identify a methodology for measuring developmental gains during participation in early intervention

•Testing child data using two existing indices:

Intervention Efficacy Index

Proportional Change Index

14

Intervention Efficacy Index

Relates changes in child capabilities to time spent in program; describes individual and group progress in terms of developmental gains within and across domains for each month in an intervention program. (Bagnato & Neisworth)

15

Intervention Efficacy Index

IEI = Developmental gain in months Time in intervention in months

IEI = Exit DA - Entry DA Time in Intervention

IEI = 34 months-20 months12 months

IEI = 1.17

16

Proportional Change Index

•Controls for children’s developmental status before intervention

•Calculation is a ratio of a child’s rate of development at pretesting to the rate ofDevelopment during intervention defined atposttesting (Wolery, 1983)

17

Proportional Change Index

PCI = Exit DA-Entry DA / Entry DA

Time in Intervention Entry CA

PCI = 34months-20months/ 20 months 12 months 24 months

PCI = 14 /.83 12

PCI = 1.40

18

Linking Results to OSEP Categories

•Test both Indexes with real-child data

•Determine numerical ranges for linkage toOSEP categories

•Decide which index yields most meaningful results and most accurately matches the OSEP categories

19

Pros, Cons, Challenges!

Pros:

•Existing IFSP data and statewide database

•Use of COSF for validation has generated good discussions of functional performance

•Possibility of more meaningful results using factorssuch as time in intervention and relationship of chronological age to developmental age at entry

20

Pros, Cons, Challenges!

Cons:•Using domain-based assessment results to Measure developmental progress in functionalOutcomes

•Using multiple assessment tools, rather than singleor limited number of tools

Challenges:•Making decisions thoughtfully with regard to Impact on local programs and families, but quicklyenough to ensure meaningful data and analysis

•Sleeping at night without outcome nightmares!

21

Ultimate Goal:Positive Results for Infants, Toddlers, and Families