buffalo summer program analysis_041812-2

7/31/2019 Buffalo Summer Program Analysis_041812-2

1/12

Buffalo Public Schools

SUMMER - ELOP ANALYSIS OF

MCLASS:DIBELS PERFORMANCE


2/12

Summary Thoughts /Observations

2

Although open to all students, the ELOP disproportionately enrolls

Buffalos most challenged students (i.e., students in the intensive

intervention/high risk category)

Preliminary analysis indicates that ELOP students essentially perform

similarly to non-participating students from one year to the next

Research studies commonly find that these types of students are mostlikely to suffer summer learning losses. Given these expectations,

ELOPs ability to keep pace with non-participating students might be

viewed as a mitigant of summer learning loss. In other words,

participating students might be viewed as performing better than they

would have had they not attended the program.

Further analysis could explore whether students are making greater

gains at the DIBELS skill level (e.g., on ORF scores) rather than

requiring change at the broader instructional recommendation level.


3/12

Profile of ELOP Participants

3

The chart below provides a comparison of the starting skill levels of

ELOP participants, in comparison to those Buffalo students who did not

attend the summer session.


4/12

4

It is clear that the ELOP students draw disproportionately from thedistricts lowest performing students.

Note how for each grade there are as many, and sometimes more,

students performing at the red/intensive level who subsequently

attended the ELOP compared to those who did not.

Obviously, there are significantly more students who do notparticipate in the summer program than those who do, and the

majority of these students are not surprisingly already performing at

grade-level benchmarks (compare the size of the green/benchmark

bars in the graph).



5/12


5

The following chart makes this even clearer, by presenting the same

comparison using a percentage distribution rather than raw counts of

students. The bars skew toward red for ELOP participants, and much moretoward green for non-participants.


6/12

6

To compare the performance of ELOP students to those who did not

attend, we compared DIBELS instructional recommendations for EOY

from the prior grade (i.e., before the summer program) to those for

BOY from the current year (i.e., after the summer program). Ideally,

participation in ELOP would result in gains for students when they

return to school at BOY, at least when compared to those who did not

participate. A summary view does not show much difference between ELOP and

non-participating students. See the two charts that follow. The first

shows the change in DIBELS instructional recommendations from

EOY to BOY for ELOP participants. As can be observed, there is not

much change from EOY to BOY as students move from one grade to

the next, and in several grades, it actually appears to worsen

somewhat (see the increase in percentage of red/intensive students).

Comparative DIBELS Performance of ELOP

Participants


7/12


8/12

8

By contrast, the following chart provides a comparison of DIBELSinstructional recommendations for all students not participating in the

summer program. While the percentage of students performing at each

level differs notably from the ELOP students, their performance mirrors

the ELOP students in its general lack of change from end of the prior year

to beginning of the current year.


Participants


9/12

9


Participants


10/12

10

The table that follows shows this comparison at a more granular levelcomparing the degree to which students have moved from one

instructional level to another (similar to an effectiveness report in our

Reporting and Analysis Suite).

The table places the performance of non-ELOP and ELOP students side-

by-side for each grade-level transition, based on their prior years

performance level. For example, the first row shows Kindergartenstudents who performed at the intensive level at EOY, with each column

showing the percentage of these students who performed at each level

the following BOY (in 1st grade).

We see that for non-participating students, 46% of students who ended

Kindergarten in the intensive category remained in this category whenthey returned to school as first graders, while only 41% of ELOP students

with the same starting category remained there at BOY.


Participants


11/12

11

ELOP vs. Non-ELOP ComparisonPerformance at BOY 2011-12

Not ELOP ELOP

Intensive Strategic Benchmark Intensive Strategic Benchmark

PerformanceatEOY2010-11

K-to-1st

Intensive 46% 39% 15% 41% 40% 19%

Strategic 14% 48% 37% 18% 46% 36%

Benchmark 2% 14% 84% 3% 16% 81%

1st-to-2nd

Intensive 75% 12% 13% 74% 9% 17%

Strategic 33% 58% 9% 40% 47% 13%

Benchmark 2% 23% 75% 3% 31% 66%

2nd-to-3rd

Intensive 81% 17% 3% 75% 20% 4%

Strategic 15% 69% 15% 19% 65% 16%

Benchmark 1% 25% 75% 1% 26% 72%

3rd-to-4th

Intensive 90% 9% 1% 90% 8% 1%

Strategic 32% 48% 20% 37% 48% 14%

Benchmark 1% 19% 80% 3% 24% 73%

4th-to-5th

Intensive 86% 14% 1% 81% 18% 1%

Strategic 12% 45% 43% 14% 48% 38%

Benchmark 1% 8% 91% 1% 13% 86%

5th-to-6th

Intensive 57% 37% 7% 64% 28% 8%

Strategic 3% 43% 53% 4% 50% 47%

Benchmark 0% 5% 94% 2% 15% 84%


12/12

12

We have highlighted in green the specific instances in which ELOPstudents appear to outperform non-participating students, which is

primarily for students starting the summer at the intensive/high risk level.

The table shows that for most grades and risk levels, participating in

ELOP does not seem to produce significantly different outcomes by the

beginning of the following year.

ELOP vs. Non-ELOP Comparison

buffalo summer program analysis_041812-2

Documents