national call on public reporting of local child outcomes data

National Call on Public Reportingof Local Child Outcomes Data

NECTAC/ECOJune 11, 2010

The National Early ChildhoodTechnical Assistance Center

Objectives for the call

• Information sharing about the requirement to report local child outcomes data

• Discussion of questions and issues

2

Public Reporting Requirement in IDEA 2004 (Report C3 and B7 the same way and at the same time you do the local reporting on other indicators)

`(C) PUBLIC REPORTING AND PRIVACY- `(i) IN GENERAL- The State shall use the targets established in the plan

and priority areas described in subsection (a)(3) to analyze the performance of each local educational agency in the State in implementing this part.

`(ii) REPORT- `(I) PUBLIC REPORT- The State shall report annually to the public on

the performance of each local educational agency located in the State on the targets in the State's performance plan. The State shall make the State's performance plan available through public means, including by posting on the website of the State educational agency, distribution to the media, and distribution through public agencies.

`(II) STATE PERFORMANCE REPORT- The State shall report annually to the Secretary on the performance of the State under the State's performance plan.

3

Local Reporting on C3 and B7- Is anything different?

• The state C3 and B7 SPP data included:– Percentages of children in the progress

categories (a through e) for each of the 3 outcomes

– The 2 Summary Statement percentages for each outcome

– Baseline and target data on the 2 Summary Statements for each outcome

4

The Summary Statements

1. Of those children who entered or exited the program below age expectations in each Outcome, the percent who substantially increased their rate of growth by the time they exited the program.

2. The percent of children who were functioning within age expectations in each Outcome by the time they exited the program.

5

What to include in local reporting

• For each local program:– The percentages of children reported for each

Summary Statement for each outcome , i.e.The percent who changed growth trajectories and the percent who reached age expectations for each outcome

Compared to• The state targets for FFY2009

6

Methods for reporting - Tabular data

Local Area n Summary Statement 1 (State Target = 72%)

Summary Statement 2(State Target = 68%)

A local area 5553.8% 61.7%

B local area 11073.2 % 69.1%

C local area 2257.3% 29.1%

D local area 1029.1% 45.7%

E local area 100094.4% 90.3%

Outcome 3: Taking Appropriate Action to Meet Needs

7

Questions and issues:Reflecting local program

How can the data reflect our local program effectiveness when not all the children who exit from our program received all of their services here?

8

Questions and issues: Comparisons

How large does a difference have to be to be a meaningful difference?

• Local to state target, local to local• Big local programs are likely to track the state

Summary Statements closely• Cautions related to comparisons

– Quality of the data – Number of children in the calculations (i.e., small n)

• What factors explain a difference?9

Questions and issues: Messaging

Putting the message on the data – why do we do it? how do we do it?:– Understanding the potential for misinterpretation

of data– Needing to have different messages for different

audiences (levels for detail)– Making sure that persons explaining the data

have good understanding of the data– Other issues you have?

10

Questions and Issues: Getting local programs ready

What are programs doing to get ready for public reporting of local data?

• Make sure local programs understand and can explain the summary statements

• Make sure local programs understand and can explain the a-e progress categories

• Note: will need a-e to understand summary statements

11

0 6 12 18 24 30 36 42 48 54 60 660

10

20

30

40

50

60

70

Developmental Trajectories

Functioning like same aged peersImproved functioning to that of same aged peersMoved closer to function-ing like same aged peersImproved functioning, no change in trajectoryDid not improve function-ing

Age in Months

Gro

wth

in O

utco

me

12

The Summary Statements

1. Of those children who entered the program below age expectations in each Outcome, the percent who substantially increased their rate of growth by the time they turned 6 years of age or exited the program.

2. The percent of children who were functioning within age expectations in each Outcome by the time they turned 6 years of age or exited the program.

13

Other Ways to Think about Summary Statement 1

• How many children changed growth trajectories during their time in the program?

• Percent of the children who entered the program below age expectations made greater than expected gains, made substantial increases in their rates of growth, i.e. changed their growth trajectories

14

Formula for SS 1

(c+d)/(a+b+c+d)

15

Summary Statement 2

2. The percent of children who were functioning within age expectations in each Outcome by the time they turned 6 years of age or exited the program.

16

Other Ways to Think about Summary Statement 2

• How many children were functioning like same aged peers when they left the program?

• Percent of the children who were functioning at age expectations in this outcome area when they exited the program, including those who:

• started out behind and caught up and• entered and exited at age level

17

Formula for SS 2

(d+e)/(a+b+c+d+e)

18

Questions and Issues:Local data use

How are programs thinking about using the local child outcomes data?

• Local determinations? (optional)• Program improvement? (make sure you trust

the data!)• Focusing TA to local programs?

– For data quality– For program improvement

19

Questions and Issues

Others?

20

Resources

• There are several helpful documents on the NECTAC website– http://www.nectac.org/topics/quality/gensup.

asp#reportdata• Forthcoming report from Technical Assistance

Center on IDEA Accountability Data (DAC) with recommendations for small n’s

• Outcomes meeting – July 30-31 2010!

21