Head

Child OutcomesBaseline and Targets for Indicator 7 on the State Performance PlanState Advisory Panel for Exceptional ChildrenNovember 12, 2009January 14, 2010Preschool children with IEPs demonstrate improved:Positive social-emotional skillsAcquisition and use of knowledge and skillsUse of appropriate behaviors to meet their needs

As we all know, there is a six-year State Performance Plan with annual performance reports that address 20 indicators for IDEA. Three of the indicators are specific only to preschool:#6- preschool LRE#7 -child outcomes, - transition from Part C.

Child outcomes is a performance measure which means a baseline for the state is established and targets for succeeding years are set. If a target is not met, then an explanation for slippage is required along with improvement strategies. The current SPP/APR are available online at ecuation.ohio.gov using the search word SPP

In the report submitted to OSEP (Feb 2010), baseline was to be established for indicator #7, preschool child outcomes. The baseline data was for the 2008-2009 school year which is the second year for using the Early Childhood Outcomes Summary Form.

In November and January of this school year, preschool child outcomes were discussed with the State Advisory Panel for Exceptional Children (SAPEC as required under IDEA) who approved the targets for the OSEP report.PSE SupervisorsFebruary 2010Office of Early Learning & School Readiness1OutcomesAccountability frameworkPartnership projectOSEP categories a reviewSummary statements and calculationsLocal reportsData errorsProgram evaluation analysisBaseline/targets

In this section, we will review several factors related to child outcomes. There are several layers to understanding this:

The state context of the early childhood accountability framework and how the OSEP measure relates to this framework;The national efforts with the Early Childhood Outcomes Center in partnership with stakeholders and OSEP;How the Early Childhood Outcomes Summary Form ratings translate into the reporting categories set by OSEP;How the data can be meaningful through consistent statements and provide for a powerful message of program effectiveness;The requirement that next year ODE report results for each LEA and marketing messages district may consider;What state aggregate data tells us for comparison of how well local programs are fairing.

There will be an ECO Institute again this May addressing these components in detail. This is a continuation of last Mays Institute on data quality. At this point we know that the summary form is being used and data is being reported. We will continue on our journey addressing what the data means and how it can be used for program improvement and communicating with stakeholders in the community.

The Institute will be the week of May 24- one session will be TW and repeated on RF.

This section will review the meanings of each of the 3 outcomes, the summary form ratings in relationship to the OSEP categories and developmental trajectories.PSE SupervisorsFebruary 2010Office of Early Learning & School Readiness2ACCOUNTABILITYWhere does this performance measure fit in the big picture?

So exactly where do child outcomes fit in the early childhood accountability framework and how do they relate to the other key components for accountability?PSE SupervisorsFebruary 2010Office of Early Learning & School Readiness3Accountability FrameworkEarly ChildhoodProgram Capacity MeasuresCurriculum-Embedded Performance MeasuresChild and Family Outcome MeasuresAligned with P-12 Content Standards

44Our Early Learning Programs fall within an accountability framework measuring outcomes at the child and family levels; the classroom level and at the program level.

There are three cornerstones to the accountability framework:Child and family outcomesProgram capacity Curriculum embedded performance measuresFebruary 2010Office of Early Learning & School ReadinessPSE SupervisorsAccountability FrameworkEarly ChildhoodProgram Capacity MeasuresCurriculum-Embedded Performance MeasuresChild and Family Outcome MeasuresAligned with P-12 Content StandardsLiteracy: GGG and KRA-LEarly Childhood Outcomes SummaryParent SurveyHealth and Developmental ScreeningsTools for Alignment to the Early Learning Content StandardsIEP OutcomesELLCO: Early Language and Literacy Classroom ObservationELLRT: Early Language and Literacy Reflection ToolSelf-Assessment Tools

55In review of specific activities related to each cornerstone:

Child/Family Outcome measures include: GGG, KRALhealth and developmental screenings per the Early Learning Program GuidelinesNote that for preschool special education, developmental screening may be redundant depending upon the evaluation completed for eligibilityNondisabled peers enrolled in a PSE class should be developmentally screenedHealth screenings would be expected for preschool children with disabilities and nondisabled peers unless an extensive health exam was part of the eligibility processA revised sample medical statement for program licensing is available on the Web and it includes the required screenings and information under the guidelines; if a doctor does not complete and states why it was not appropriate, a program would not be held noncompliant; would expect that some follow-up later on would be done to ensure that further or follow-up screening was necessaryFor PSE, indicator 8 requires a survey to preschool parents every year (approximately 3000) to respond to their understanding of their rights and involvement;

Program capacity measures: ELLCO, Self-reflection planning tool (ELLRT)Self assessment tools, such as the one for transition you may be most familiar with (go to educaton.ohio.gov and use the search word preschool transition

Curriculum embedded performance measures: Under licensing requirements, programs are to have a comprehensive curriculum that is aligned with Ohios early learning content standards and encompass all developmental domains. We are working to help identify more curriculum embedded performance measures.

Most commonly used curriculums for preschool = Creative Curriculum & High ScopeMost commonly used PSE = Creative Curriculum & AEPS

February 2010Office of Early Learning & School ReadinessPSE SupervisorsChild Outcomes

Preschool children with IEPs demonstrate improved: Positive social-emotional skillsAcquisition and use of knowledge and skillsUse of appropriate behaviors to meet their needs

Preschool Special Education Data & ReportingSeptember 2008ODE/EL&SR6In the State Performance Plan and the Annual Performance Report, Monitoring Priority: FAPE in the LRE for preschool includesIndicator 7: Percent of preschool children with IEPs who demonstrate improved:Positive social-emotional skills (including social relationships);Acquisition and use of knowledge and skills (including early language/communication and early literacy); andUse of appropriate behaviors to meet their needs.(20 U.S.C. 1416 (a)(3)(A))Measurement (for each: A, B and C): Percent of preschool children who did not improve functioning = [(# of preschool children who did not improve functioning) divided by the (# of preschool children with IEPs assessed)] times 100.Percent of preschool children who improved functioning but not sufficient to move nearer to functioning comparable to same-aged peers = [(# of preschool children who improved functioning but not sufficient to move nearer to functioning comparable to same-aged peers) divided by the (# of preschool children with IEPs assessed)] times 100.Percent of preschool children who improved functioning to a level nearer to same-aged peers but did not reach it = [(# of preschool children who improved functioning to a level nearer to same-aged peers but did not reach it) divided by the (# of preschool children with IEPs assessed)] times 100.Percent of preschool children who improved functioning to reach a level comparable to same-aged peers = [(# of preschool children who improved functioning to reach a level comparable to same-aged peers) divided by the (# of preschool children with IEPs assessed)] times 100.Percent of preschool children who maintained functioning at a level comparable to same-aged peers = [(# of preschool children who maintained functioning at a level comparable to same-aged peers) divided by the (# of preschool children with IEPs assessed)] times 100.If a + b + c + d + e does not sum to 100%, explain the difference.Performance Targets will be provided in the FFY 2008 (2008-2009) SPP, due February 1, 2010Baseline will be FFY 2008 (2008-2009)Resources:Early Childhood Outcome CenterMeasuring Child and Family Outcomes ConferenceResources on the ECO and NECTAC websitesEarly Childhood Outcomes CenterOSEP-funded technical assistance projectDeveloped three summary measuresDeveloped summary form for reporting 38 out of 59 states/territories using this formDeveloped crosswalk translating entry/exit scores to OSEP categoriesDeveloped summary statements reported in Annual Performance Report (APR)Analyzes all APRs submitted annually and provides summary report

The three outcomes were developed with the Early Childhood Outcomes Center which is an OSEP funded project. The outcome measures are appropriate measures for all preschool children, birth through age five. Discussions began with stakeholders about 8 years ago to develop the outcome measures and as time goes on, we continue to learn about the measures, how to give meaning to the measures, and develop more consistency in measurement and reporting.

Once the 3 outcomes were determined, the ECO Center developed the summary form for reporting. 2009-2010 is the third year that Ohio has used the summary form for reporting. The majority of states and territories in the US use the same form. Some states use only one instrument for reporting, like California that has the Desired Results Profile. Some states provide local districts the option of selecting an online curriculum reporting method from an approved list. In Ohio, ODE cannot mandate the curriculum to be used.

The ECO Center developed a cross walk from the summary form ratings to the OSEP categories based upon some basic business rules for 2 scores. On the ECO website, there is a calculator in which data may be entered and the translation from summary ratings to OSEP categories is determined.

The past few years discussions have centered on how to make the 5 OSEP categories meaningful and tell a story about children and programs. Two summary statements have been developed that address the number of children that are essentially increasing their rate of growth and contribute to closing the achievement gap as well as the children who exit preschool functioning comparable to their same aged nondisabled peers.

Ohios data is reviewed in conjunction with national data. The ECO Center summarized all 59 states and territories submission annually to identify trends and issues.PSE SupervisorsFebruary 2010Office of Early Learning & School Readiness7Early Childhood Outcomes CenterOhio one of seven partner states (competitive application) to develop accountability framework for this indicator ECO Center specific work with OhioAnalysis of outcomes dataProfessional development on data accuracy and qualityMeasurement scale appropriate for use across public preschool and special educationRecommendations for data collection

In addition to the general work related to the technical assistance grant with OSEP, the ECO Center put forth a competitive application for states to join in a partnership with the center to work on our states measurement system and assist in developing the measurement framework for other states to model. The seven framework partner states are: California (619), Colorado (Part C and 619), Delaware (Part C and 619), Maine (Part C and 619), Minnesota (Part C and 619), New York (Part C), and Ohio (Part C and 619).The focus of our work together will be on developing and refining an outcomes measurement framework. The framework is seen as a potential blueprint for state outcomes systems development that will provide both a comprehensive description of a states measurement system as well as a method for helping a state to identify components that are well and not so well developed. This framework will then assist all states in reporting data for the Annual Performance Report submitted to OSEP in February each year and assist state and local agencies in analyzing the data, identifying areas of need, and targeting strategies to improve outcomes for young children.Ohio also has a separate contract with the ECO center as one of our targeted improvement strategies. ECO staff analyze Ohios aggregate data in relationship to different variables, disaggregating for trends and patterns, analyzing data errors to increase validity, and provide professional development. Last year, the spring Institutes focused on data quality, accuracy and validity. This years Institutes will focus on the use of data, the story the data tells the public, and provide some insights into marketing messages which will be important since ODE will be reporting local results publically next year.Preschool special education is required to use results of the Get It, Got It, Go and the Ages and Stages Questionnaire: Social-0Emotional as two sources of information for the Early childhood outcomes summary form. Public preschool (aka ECE grant) programs are required to use GGG as well. Beginning next , ECE programs will be also administering the ASQSE. We are working with the ECO center to adjust the ECO rating scale to 9 points so that ratings of 8 and 9 can apply to nondisabled peers for future consistency across all ODE-funded programs.

PSE SupervisorsFebruary 2010Office of Early Learning & School Readiness8The Basics of Quality Data and Target Setting

Early Childhood Outcomes (ECO) CenterNational Early Childhood Technical Assistance Center (NECTAC)

Workshop 1: December 5, 2009

These next several slides have been borrowed from a presentation by the ECO Center at the national early childhood meeting by OSEP. This will be a review of the 5 OSEP categories used for reporting.

Currently, districts report the ratings of 1 to 7 for each preschool child with a disability. ODE than takes the scores and does the calculations for children to be represented in one of five buckets or categories. We have presented these buckets to the supervisors the past three years. It is possible to go the ECO center website and use a calculator to determine the categories for any child in a program.

It is important to understand the 7 point rating scale, how it relates to the 5 OSEP categories, and the intent of the 3 outcome measures in order to related the meaning of the outcome measures as they are reported.

Remember that the three outcomes are birth through five years of age and therefore encompass early intervention/Part C as well. In Ohio, Part C is also using the ECO summary form.February 2010Office of Early Learning & School ReadinessPSE SupervisorsGoal of Early Intervention/Early Childhood Special EducationTo enable young children to be active and successful participants during the early childhood years and in the future in a variety of settings in their homes with their families, in child care, in preschool or school programs, and in the community.

Early Childhood Outcomes Center,http://www.fpg.unc.edu/~eco/pdfs/eco_outcomes_4-13-05.pdf

10Under ECO, stakeholders got to work on the child outcomes, including this overarching goal for EI/ECSE

2004-early 2005: ECO generates discussion and gathers input on child and family outcomes. Stakeholders came up with these three outcomes.

It is important to understand the intended meaning for each of the three outcome areas.

February 2010Office of Early Learning & School ReadinessPSE SupervisorsChild OutcomesOutcome 1: Positive social emotional skills (including positive social relationships)The intended meaning:Using social/emotional, language, cognitive (etc.) skills to have positive social relationshipsEarly Childhood Outcomes Center

ReadFebruary 2010Office of Early Learning & School ReadinessPSE SupervisorsChild OutcomesOutcome 2: Acquisition and use of knowledge and skills (including early language/ communication [and early literacy])The intended meaning:Using cognitive, language (etc.) skills to reason, problem solve and understand their physical and social worldsEarly Childhood Outcomes Center

Child OutcomesOutcome 3: Use of appropriate behaviors to meet their needsThe intended meaning:Using motor, language, social, cognitive (etc.) skills to successfully take care of themselves and their needsEarly Childhood Outcomes Center

Note that in each of these outcomes, the intended meaning does not focus on one developmental domain but emphasizes how multiple domains and skills are involved to achieve functional outcomes. Functionality addresses the childs ability to physically, socially, linguistically navigate the physical world and social interactions.PSE SupervisorsFebruary 2010Office of Early Learning & School Readiness13OSEP Reporting CategoriesPercentage of children who: a.Did not improve functioningb.Improved functioning, but not sufficient to move nearer to functioning comparable to same-aged peers c.Improved functioning to level nearer to same-aged peers but did not reach itd.Improved functioning to reach level comparable to same-aged peersMaintained functioning at level comparable to same-aged peersEarly Childhood Outcomes Center

OSEP came up with these 5 categories for reporting childrens progress in the three outcome areas. (Summer 2005: OSEP announces the child and family outcomes States must report on through their SPP/APRs. Progress categories at first were limited to three , moved to four and finally evolved to 5. These five categories do allow for discreet differences in progress and tell a much more meaningful story than only three categories.

The categories themselves are not the focal point of the reporting however; how this data is used tells the story and effectiveness of programming.

February 2010Office of Early Learning & School ReadinessPSE SupervisorsKey Concepts Related to Progress CategoriesProgress categories require two data points for each child Based on growth trajectoriesCompare child to himself or herself over timeCompares each child to age expectationsEarly Childhood Outcomes Center

What is most important to know is that the story is based upon two points in time for OSEP. In Ohio, programs report results twice annually through EMIS and can use the outcome summary form more often. This is to assist local program s in identifying needs of children and programs each year instead of across a longer period of time.

For OSEP, ODE uses the first score and the last score reported to calculate which of the five categories applies. As shown in the next few slides, the ratings on the outcome summary forms are based upon developmental trajectories relative to closing the achievement gap. This is not about children with disabilities being cured and no longer disabled but allows us the opportunity to examine the childs personal progress over time (from one point to another) and also compare that progress to age-expected behaviors.February 2010Office of Early Learning & School ReadinessPSE Supervisors

FUNCTIONING

16Developmental trajectories are a good way to think about how the numbers we report to OSEP come from the COSF data. A childs developmental trajectory can be plotted from entry to exit using a points on the 7-point scale. On the graph displayed, the vertical axis shows the childs level of functioning, corresponding to the 7 points on the rating scale. The dotted line separates overall age appropriate (6-7) from less than age appropriate (1-5). The horizontal axis corresponds to the childs age in months.

Definitions for Outcome Ratings:For Use with the Child Outcomes Summary Form (COSF)

Completely means: 7 Child shows functioning expected for his or her age in all or almost all everyday situations that are part of the childs life. Functioning is considered appropriate for his or her age. No one has any concerns about the childs functioning in this outcome area.

Overall Age-Appropriate 6 Childs functioning generally is considered appropriate for his or her age but there are some significant concerns about the childs functioning in this outcome area. These concerns are substantial enough to suggest monitoring or possible additional support. Although age-appropriate, the childs functioning may border on not keeping pace with age expectations.

Somewhat means: 5 Child shows functioning expected for his or her age some of the time and/or in some settings and situations. Childs functioning is a mix of age-appropriate and not age-appropriate behaviors and skills. Childs functioning might be described as like that of a slightly younger child*.

4 Child shows occasional age-appropriate functioning across settings and situations. More functioning is not age-appropriate than age appropriate.

Nearly means: 3 Child does not yet show functioning expected of a child of his or her age in any situation. Child uses immediate foundational skills, most or all of the time, across settings and situations. Immediate foundational skills are the skills upon which to build age-appropriate functioning. Functioning might be described as like that of a younger child*.

2 Child occasionally uses immediate foundational skills across settings and situations. More functioning reflects skills that are not immediate foundational than are immediate foundational.

Overall Not Age-AppropriateNot yet means: 1 Child does not yet show functioning expected of a child his or her age in any situation. Childs functioning does not yet include immediate foundational skills upon which to build age-appropriate functioning. Child functioning reflects skills that developmentally come before immediate foundational skills. Childs functioning might be described as like that of a much younger child*.* The characterization of functioning like a younger child only will apply to some children receiving special services, such as children with developmental delays.February 2010Office of Early Learning & School ReadinessPSE SupervisorsThe a Category a.Percent of Part C/preschool children who did not improve functioningChildren who acquired no new skills or regressed during their time in the programDidnt gain or use even one new skillChildren with degenerative conditions/ significant disabilitiesEarly Childhood Outcomes Center

17Read slide

The second time and every time thereafter, the outcome summary form asks a question regarding the childs progress. Did the child make progress? Yes or no.

{Progress is a broad descriptor and means ANY new skill or behavior is observable since the previous summary was completed.

It is this question that basically indicates whether or not the child is in category a or not.

This is a very small percentage of the population as it represents the children who have severe disabilities. February 2010Office of Early Learning & School ReadinessPSE Supervisors

EntryExit

18IN this case, a child entered the program with a rating of 3 and over time, the child was functioning at a level of 2.Nearlymeans: 3 Child does not yet show functioning expected of a child of his or herage in any situation. Child uses immediate foundational skills, most or all of the time,across settings and situations. Immediate foundational skills are theskills upon which to build age-appropriate functioning. Functioning might be described as like that of a younger child*.2 Child occasionally uses immediate foundational skills acrosssettings and situations. More functioning reflects skills that are notimmediate foundational than are immediate foundational.February 2010Office of Early Learning & School ReadinessPSE Supervisors

EntryExit

19Not yetmeans: 1 Child does not yet show functioning expected of a child his or herage in any situation. Childs functioning does not yet include immediate foundationalskills upon which to build age-appropriate functioning. Child functioning reflects skills that developmentally come beforeimmediate foundational skills. Childs functioning might be described as like that of a muchyounger child*.* The characterization of functioning like a younger child only will apply to some children receiving special services,such as children with developmental delays.February 2010Office of Early Learning & School ReadinessPSE SupervisorsThe b Categoryb.Percent of Part C/preschool children who improved functioning but not sufficient to move nearer to functioning comparable to same-aged peersChildren who acquired new skills but continued to grow at same rate throughout their time in program Gained and used new skills but did not increase their rate of growth or change their growth trajectories while in servicesEarly Childhood Outcomes Center

20February 2010Office of Early Learning & School ReadinessPSE Supervisors

EntryExit

21Note the slope - the line is not flat; the child is showing improved functioning

Any child who maintains the same rating must have shown progress. Any developmental gains made over time are an indication of growth.

Somewhatmeans: 5 Child shows functioning expected for his or her age some of thetime and/or in some settings and situations. Childs functioningis a mix of age-appropriate and not age-appropriate behaviors andskills. Childs functioning might be described as like that of a slightlyyounger child*.

* The characterization of functioning like a younger child only will apply to some children receiving special services,such as children with developmental delays.February 2010Office of Early Learning & School ReadinessPSE Supervisors

EntryExit

22Note the slope - the line is not flat; the child is showing improved functioning

Somewhatmeans: 5 Child shows functioning expected for his or her age some of thetime and/or in some settings and situations. Childs functioningis a mix of age-appropriate and not age-appropriate behaviors andskills. Childs functioning might be described as like that of a slightlyyounger child*.4 Child shows occasional age-appropriate functioning across settingsand situations. More functioning is not age-appropriate than ageappropriate.February 2010Office of Early Learning & School ReadinessPSE Supervisors

EntryExit

23Note the slope - this child is showing improved functioning

Not yetmeans: 1 Child does not yet show functioning expected of a child his or herage in any situation. Childs functioning does not yet include immediate foundationalskills upon which to build age-appropriate functioning. Child functioning reflects skills that developmentally come beforeimmediate foundational skills. Childs functioning might be described as like that of a muchyounger child*.* The characterization of functioning like a younger child only will apply to some children receiving special services,such as children with developmental delays

The answer to the progress question is yesFebruary 2010Office of Early Learning & School ReadinessPSE SupervisorsThe c Categoryc.Percent of Part C/preschool children who improved functioning to level nearer to same-aged peers but did not reach itChildren who acquired new skills but accelerated their rate of growth during their time in programMade progress toward catching up with same aged peers but were still functioning below age expectations when they left programChanged growth trajectories narrowed the gapEarly Childhood Outcomes Center

24February 2010Office of Early Learning & School ReadinessPSE Supervisors

EntryExit

25Child showed sufficient improvement in functioning to move to a higher developmental trajectory (i.e., higher point on scale) but had not yet achieved age appropriate functioning.

Notice the slope4 Child shows occasional age-appropriate functioning across settingsand situations. More functioning is not age-appropriate than age appropriate.Nearlymeans: 3 Child does not yet show functioning expected of a child of his or herage in any situation. Child uses immediate foundational skills, most or all of the time,across settings and situations. Immediate foundational skills are theskills upon which to build age-appropriate functioning. Functioning might be described as like that of a younger child*.February 2010Office of Early Learning & School ReadinessPSE SupervisorsThe d Categoryd.Percent of preschool children who improved functioning to reach level comparable to same-aged peers

Children who were functioning below age expectations when they entered program but were functioning at age expectations when they leftStarted out below age expectations, but caught up while in servicesEarly Childhood Outcomes Center

26February 2010Office of Early Learning & School ReadinessPSE Supervisors

EntryExit

27An exit rating of 6 or 7 indicates the child is now showing age expected functioning. Note the exit rating is above the dotted line.

Completelymeans: 7 Child shows functioning expected for his or her age in all or almostall everyday situations that are part of the childs life. Functioningis considered appropriate for his or her age. No one has any concerns about the childs functioning in thisoutcome area.Overall Age-Appropriate6 Childs functioning generally is considered appropriate for his or herage but there are some significant concerns about the childsfunctioning in this outcome area. These concerns are substantialenough to suggest monitoring or possible additional support. Although age-appropriate, the childs functioning may border on notkeeping pace with age expectationsFebruary 2010Office of Early Learning & School ReadinessPSE SupervisorsThe e Categorye.Percent of Part C/preschool children who maintained functioning at level comparable to same-aged peers Children who were functioning at age expectations when they entered program and were functioning at age expectations when they leftEntered program at age expectations and were still up with age expectations at exitEarly Childhood Outcomes Center

28February 2010Office of Early Learning & School ReadinessPSE Supervisors

EntryExit

29Both 6 and 7 indicate age appropriate functioning. In this category, both the entry and exit scores are above the dotted line.

Completelymeans: 7 Child shows functioning expected for his or her age in all or almostall everyday situations that are part of the childs life. Functioningis considered appropriate for his or her age. No one has any concerns about the childs functioning in thisoutcome area.Overall Age-Appropriate6 Childs functioning generally is considered appropriate for his or herage but there are some significant concerns about the childsfunctioning in this outcome area. These concerns are substantialenough to suggest monitoring or possible additional support. Although age-appropriate, the childs functioning may border on notkeeping pace with age expectationsFebruary 2010Office of Early Learning & School ReadinessPSE Supervisors

EntryExit

30Once again both entry and exit scores are above the dotted line that denotes age expected behaviors.February 2010Office of Early Learning & School ReadinessPSE SupervisorsSummary Statement DataRequired Summary Statement 1: Of those children who entered or exited program below age expectations in each Outcome, percent who substantially increased their rate of growth by time they exited program.Required Summary Statement 2: Percent of children who were functioning within age expectations in each Outcome by time they exited program.Early Childhood Outcomes Center

The categories alone do not tell much of a story but when summarized into two specific statements, the effectiveness of the program can be shared. There is one other statement that can also be used and that will be evident when the aggregate data is examined.

These summary statements focus on the number and percent of the children exiting the program comparable to same aged peers and those that made substantial gains in growth. Even though a child may have a disability, the child can still demonstrate age appropriate behaviors. Having a disability does not mean a child cannot demonstrate some age appropriate behaviors unless the disability is so severe that the childs progress is limitedEach of these statements is based upon a mathematical calculation using the five categories. Note that these two summary statements will not add up to 100% as category d is used in each calculation and the denominator does not use the same number of categories.February 2010Office of Early Learning & School ReadinessPSE SupervisorsSummary Statement 1359 (a, b, c and d) or 46 percent of children entered or exited program functioning below age expectations425 (e) or 54 percent of children entered and exited functioning at age expectationsProg cat#%a172b10614c679d16922e42554

Lets look at program data showing the number of children in each category and the percent of the total number of children reported. Remember that this data is reflecting the number of children who exited the program and the comparison is between the entry score and the exit score, which can indicate anywhere from 6 months of service up to three years of preschool services.

Knowing the five categories, those in a through d are children who entered or exited the program functioning below age expectations below the dotted line on the developmental trajectories). Those in category e are those that entered and exited above the dotted line on the developmental trajectory.

A total of 784 children are represented. 359 of them are below age expectations 46% and 425 of them are functioning at age expectations -54%

PSE SupervisorsFebruary 2010Office of Early Learning & School Readiness32Percent of Children Who Made Greater than Expected ProgressProg cat#%a172b10614c679d16922e42554236 (c and d) of 359 (a, b, c, and d) changed growth trajectories (made greater than expected progress)236359= 66%

Summary Statement 1: Of those children who entered or exited program below age expectations in the Outcome, percent who substantially increased their rate of growth by time they exited program. This is reported for each of the three outcome areas.

To calculate this summary statement,. We look at the number of children in in categories c and d in relationship to all the children who enter or exit below age expectations. This eliminates those who entered functioning at age expectations and then exited functioning at age expectations. Remember that children in a did not improve functioning and were below age expectations; children in b are children who made progress but did move nearer to functioning comparable to same-aged peers; children in c are children who improved and are actually nearer to same aged peers (ie the dotted line) but did not cross the dotted line; and children in d are children who improved sufficiently to cross the dotted line to reach functioning comparable to same aged peers.

So we look at categories c and d in relationship to a, b, c, and d. but not e. There are 236 children in categories c and d and a total of 359 in a, b, c, and d for 66% of the children made greater than expected progress or substantially increased their rate of growth, thereby changing their growth trajectory.PSE SupervisorsFebruary 2010Office of Early Learning & School Readiness33Percent Who Exited at Age Expectations169 + 425 784= 76%22 percent of children reached age expectations by exit and 54 percent of children entered and exited at age expectationsProg cat#%a172b10614c679d16922e42554total784

Summary Statement 2: Percent of children who were functioning within age expectations in the Outcome by time they exited pro

To calculate this summary statement,. We look at the number of children in in categories d and e in relationship to all the children who enter or exit at age expectations. This includes those who [a] entered functioning at age expectations and then exited functioning at age expectations and [b] those that entered functioning below age expectations and then exited functioning at age expectations . Remember that children in d are children who improved sufficiently to cross the dotted line to reach functioning comparable to same aged peers and those in e are those that maintained skills and behaviors comparable to same aged peers.

So we look at categories d and e in relationship to all 5 categories or all children. There are 594 children in categories d and e and a total of 784 in a through e so that there are 76% of the children who exited at age expectations. This is reported for each of the three outcome area.PSE SupervisorsFebruary 2010Office of Early Learning & School Readiness34Data, Evidence, Results

Usually they are what they areNot debatable

The data reported in EMIS is the basis for our calculations in our annual report to OSEP and will be the basis of the public reporting ODE is required to do for each district. Whatever is in EMIS is not debatable, data are what they are and districts will not be modifying that data for local reporting. The results reported in EMIS are dependent upon fidelity of the process completing the early childhood outcomes summary form and therefore dependent upon the discussions of the core team at a minimum. PSE SupervisorsFebruary 2010Office of Early Learning & School Readiness35Inference, InterpretationBut what do data mean?Good news? Bad news? Debatable even reasonable people can reach different conclusionsMay be question of data qualityTo reach an inference, sometimes analyze data in other ways (ask for more evidence)

36Once the data has been analyzed, the summary statements tell a public story regarding the effectiveness of the program. But there is more to it than that to ensure accuracy and validity.

First we look at the summary statements to see if the data is reasonalbe and realistic.Is this a good story or not?The data should be looked at a program level for effectiveness and then again at the classroom level.Questions that can be asked includeAre there any patterns in the data?Is the aggregate data comparable to the state aggregate data?If not, why not?If yes, what contributes to strengths and/or opportunities for growth? Are any of the groups- either by class, or age, or ethnicity, or poverty, etc that have skewed results?Are there any teams that consistently score children at a 1 or 2 at entry and/or score above the dotted line at a 6 or 7 upon exit? If this is what is happening, how accurate is the data? How well are teams adhering to the protocol? Do they understand the summary form and the decision tree?February 2010Office of Early Learning & School ReadinessPSE SupervisorsActionGiven interpretation of evidence, what should be done?Recommendations or action stepsAction can be debatable and often isAgain, sometimes action is to improve quality of data

37After an analysis of the data (very similar to the OIP process), what is next?Remember the four basic questions:What did we do?What happened because of it?What could we have done differently?What might have happened then?

Then decide on a course of action, plan, implement and evaluate the results of course corrections. Repeat the cycle again- this is continuous improvement.February 2010Office of Early Learning & School ReadinessPSE Supervisors

As a review, this graph clearly indicates the 5 OSEP categories in relationship to closing the achievement gap.

This is not about making children with disabilities the same as nondisabled peers or about discounting the disability.

It is about recognizing that children with disabilities can demonstrate age appropriate skills.

It is about comparing a child's progress from a start point to a mid point to an end point or exit from preschool

It is about comparing a childs skills comparable to same aged nondisabled peers to keep perspective and understand the gap between age expectations and current levels of performance.

On this graph, the pink line indicates children who are functioning at a 6 or 7, across the dotted line, comparable to same aged peers or age expectations.

The aqua line is the line that in the past described the progress of most children with an IEP. These are the children in category a with severe disabilities or degenerative conditions.

Categories b and c in the yellow and red lines clearly demonstrate the childs progress over time but there is still a gap between their performance and same aged peers. Children in c on the red line are those that narrowed the gap by changing growth trajectories.

The orange line represents children in category d who entered the program below age expectations but exited comparable to same aged peers. These children are the gap closers

February 2010Office of Early Learning & School ReadinessPSE SupervisorsIndicator 7: Early Childhood OutcomesSAPEC November 2009 Baseline and Targets Re-AnalysisRe-analysis of data changes baseline and proposed new targets. Revised information for SAPEC consideration and approval during January 2010 meeting. This data in February 2010 APR.Summary Statements (% of children)BaselineTarget FFY 2009Target FFY 2010Positive Social-Emotional SkillsOf those children who entered program below age expectations in Outcome 1, percent who substantially increased their rate of growth by time they turned 6 years of age or exited program

64.7%65%66%Percent of children functioning within age expectations in Outcome 1 by time they turned 6 years of age or exited program

47.4%48%49%Acquisition and Use of knowledge and skillsOf those children who entered program below age expectations in Outcome 2, percent who substantially increased their rate of growth by time they turned 6 years of age or exited program 65.9%67%68%Percent of children who were functioning within age expectations in Outcome 2 by time they turned 6 years of age or exited program45.7%45%47%Use of appropriate behaviors to meet needs.Of those children who entered the program below age expectations in Outcome 3, percent who substantially increased their rate of growth by time they turned 6 years of age or exited program 66.94%65%

67%Percent of children who were functioning within age expectations in Outcome 3 by time they turned 6 years of age or exited program59.24%58%60%

In February of this year, our annual report to OSEP included the baseline and targets for this performance measure on child outcomes. This is not a compliance measure where OSEP sets the target. Baseline and targets are set according to the summary statements. Two summary statements for each of the 3 outcomes: positive social emotional skills; acquisition and use of knowledge and skills; and use of appropriate behaviors to meet needs. Therefore, there are a total 6 baselines and 12 targets.

The baseline is based upon the data from the 2008-2009 school year which is the second year Ohio used the ECO summary form. In the first year of using the form (2007-2008), there were approximately 1500 children with valid scores for entry and exit, who exited preschool. This subgroup of children received anywhere from 6 months to one year of services. The baseline data on this slide represents a subgroup of approximately 3000 children who exited preschool with anywhere from 6 months to 2 years of services. Data from the current year, 2009-2010 , and every year hereafter will represent a subgroup of children with anywhere from 6 months to 3 years of services.

For the annual report, Ohio was required to set targets for the next two years. Note that the targets are conservative given the small number of children represented in the subgroup- 3,000 out of a total of 23, 246 chidlren. In two years, Ohio will be developing another 6 year state performance plan and at that time, trend data over four years will be analyzed to set new targets.

This information was presented to the State Advisory Panel for Exceptional Children in November and again in January and this group approved the targets.

Note that the baseline and targets are very similar across summary statement 1 but there are a higher number of children meeting age expectations on appropriate behaviors to meet needs than in the other two outcome areas. One of the analyses, the ECO center will be doing for Ohio is comparing the ASQSE scores to the ECO summary results to indicate if this is an accurate results rating.PSE SupervisorsFebruary 2010Office of Early Learning & School Readiness39Outcome 1Outcome 2Outcome 3

This slide is an overview of the national data from 2007-2008 to our first year of data in 2007-2008 to Ohios baseline for 2008-2009. Note the patterns across all three outcomes. Category a is the top of each bar and category e is at the bottom. Nationally, approximately 4% of the children exiting preschool are in category a, those that did not make progress or demonstrate any new skills or behaviors. In Ohio, there were only 2% of the children in that category which based upon national discussions, 2% is believed to be more accurate than 4%.

Another summary statement that can be made, based upon the fact that 2% of the children did not acquire new skills is to state that 98% of the children exiting preschool special education made progress.

The next few slides break down the comparison by outcome areas.PSE SupervisorsFebruary 2010Office of Early Learning & School Readiness40

One can see that the pattern for Ohio in the 2 bars to the right are similar to the national data on the far left for positive social emotional skills.PSE SupervisorsFebruary 2010Office of Early Learning & School Readiness41

Acquisition and use of knowledge and skills is also very similar pattern for 2 years of Ohio data and one year of national summary data.PSE SupervisorsFebruary 2010Office of Early Learning & School Readiness42

the similar pattern follows for outcome area on appropriate behaviors to meet need except that Ohio has more children in category d and fewer in category b than the national pattern.

As additional information becomes available, Ohio data will be compared to the national aggregate just as local districts compare to the state aggregate.PSE SupervisorsFebruary 2010Office of Early Learning & School Readiness43