oregon reading first: three-year report preliminary impact evidence
DESCRIPTION
Oregon Reading First: Three-Year Report Preliminary Impact Evidence. Oregon Reading First Center LLSSC Meeting, November 29, 2006. Background. Oregon Reading First Participant Districts and Schools Cohort A: 14 districts and 33 schools Three full years of implementation - PowerPoint PPT PresentationTRANSCRIPT
1
Oregon Reading First:Three-Year Report
Preliminary Impact Evidence
Oregon Reading First Center
LLSSC Meeting, November 29, 2006
2
Background
Oregon Reading First Participant Districts and Schools
Cohort A: 14 districts and 33 schools Three full years of implementation
Cohort B: 8 districts and 16 schools One full year of implementation
3
Impact Analyses
Cohort A: Improvement over time Year 1 to Year 2 to Year 3
Cohort A and Cohort B 3 years of implementation compared to 1 year
of implementation
4
Cohort A Improvement Over Time
Percent of children at grade level or meeting benchmark performance goals Evidence of impact would be higher rates for
year 3 vs. year 2 vs. year 1 Percent of children at high risk for reading
difficulties Evidence of impact would be lower rates for
year 3 vs. year 2 vs. year 1
5
Comparability in Kindergarten at Beginning of Year
7.58.0
7.4
0123456789
10
Year 1 Year 2 Year 3
Lett
er
Nam
ing
Flu
en
cy
6
Percent Reaching Benchmark Goals on DIBELS
56
4540 38
61
5046
42
78
5955
50
30
40
50
60
70
80
90
Grade K NWF Grade 1 ORF Grade 2 ORF Grade 3 ORF
Year 1 Year 2 Year 3
7
Percent Reaching Grade Level on High Stakes Measures
58
47 47 47
60
5350 51
68
5855
53
40
45
50
55
60
65
70
G K SAT10 G 1 SAT10 G 2 SAT10 G 3 OSA
Year 1 Year 2 Year 3
8
Percent at High Risk on DIBELS
2328
43
29
2023
35
28
9
17
27
21
0
10
20
30
40
50
Grade K NWF Grade 1 ORF Grade 2 ORF Grade 3
Year 1 Year 2 Year 3
9
Percent At High Risk on High Stakes Measures
21
2831
28
19
25 2723
14
21 2318
0
10
20
30
40
50
G K SAT-10 G 1 SAT-10 G 2 SAT-10 G 3 OSA
Year 1 Year 2 Year 3
10
Effect Sizes for Large Scale Reading Interventions (Borman et al., 2003; Borman & D’Agostino (1996, 2001)
0.150.17
0.14 0.15
0
0.1
0.2
0.3
0.4
0.5
0.6
Title I 1 Year 2 Year 3 Years
Eff
ect
Siz
es
Large Scale CSRD Interventions
11
Cohort A Effect Sizes (Year 3 – Year 1)
0.52
0.300.33
0.15 0.150.18 0.18
0.20
0.250.25
0
0.1
0.2
0.3
0.4
0.5
0.6
Title I CSRD Grade K Grade 1 Grade 2 Grade 3
Eff
ec
t S
ize
s
Oregon Reading First
DIBELS
High Stakes Measure
12
Cohort A and Cohort B
Percent of children at grade level or meeting benchmark performance goals Evidence of impact would be higher rates for
greater years of implementation (Cohort A) Percent of children at high risk for reading
difficulties Evidence of impact would be lower rates for
greater years of implementation (Cohort A)
13
Comparability in Kindergarten at Beginning of Implementation Year 1
42.5 41.5
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
Cohort B Cohort A
Perc
en
t A
t H
igh
Ris
k
14
Performance on DIBELS After Year 1 of Implementation
55
47
42
35
56
4540 38
2025303540455055606570
Grade K NWF Grade 1 ORF Grade 2 ORF Grade 3 ORF
Per
cen
t at
Ben
chm
ark
Cohort B Cohort A
15
Performance Y3 (Cohort A) and Y1 (Cohort B) on DIBELS
5547
4235
78
5955
50
0
20
40
60
80
Grade K NWF Grade 1 ORF Grade 2 ORF Grade 3 ORF
Per
cen
t at
Ben
chm
ark
Cohort B Year 1 Cohort A Year 3
16
Percent at High Risk Y3 (Cohort A) and Y1 (Cohort B) on DIBELS
26 26
41
31
9
17
27
21
0
10
20
30
40
50
Grade K NWF Grade 1 ORF Grade 2 ORF Grade 3 ORF
Per
cen
t at
Hig
h R
isk
Cohort B Year 1 Cohort A Year 3
17
Performance on High Stakes Measure After Y1 of Implementation
54 5146
41
5847 47 47
0102030405060708090
100
G K SAT-10 G 1 SAT-10 G 2 SAT-10 G 3 OSA
Per
cen
t at
Gra
de
Lev
el
Cohort B Year 1 Cohort A Year 1
18
Performance Y3 (Cohort A) and Y1 (Cohort B) on High Stakes Measure
5451
46
41
68
5855
53
30
40
50
60
70
G K SAT10 G 1 SAT10 G 2 SAT10 G 3 OSA
Per
cen
t at
Gra
de
Lev
el
Cohort B Year 1 Cohort A Year 3
19
Percent at High Risk Y3 (Cohort A) and Y1 (Cohort B) on High Stakes Measures
2426
3329
14
21 23
18
0
10
20
30
40
G K SAT10 G 1 SAT10 G 2 SAT10 G 3 OSA
Per
cen
t at
Hig
h R
isk
Cohort B Year 1 Cohort A Year 3
20
Effect Sizes (Cohort A Year 3 – Cohort B Year 1)
0.58
0.270.31 0.30
0.15 0.15
0.30
0.23
0.17
0.33
0
0.1
0.2
0.3
0.4
0.5
0.6
Title I CSRD Grade K Grade 1 Grade 2 Grade 3
Eff
ec
t S
ize
s
Oregon Reading First
DIBELSHigh Stakes Measure
21
Summary of Preliminary Evidence
Evidence of increased achievement over years (Cohort A)
Evidence of improvement for longer implementation duration (three years vs. one year)
Improvement in performance on multiple reading achievement measures
Improvement in increasing the percent of children reaching grade level and benchmark goals
Improvement in reducing the percent of children at high risk for reading difficulties
22
Effect Sizes for Large Scale Longitudinal Interventions
(Borman et al., 2003)
0.170.14 0.15 0.13
0.25 0.23
0.39
0.50
0.00
0.10
0.20
0.30
0.40
0.50
0.60
1 Year 2 Year 3 Years 4 Years 5 Years 6 Years 7 Years 8-14Years
Years of Implementation
Ad
just
ed E
ffec
t S
izes
23
Policy Considerations
1. What role should the Oregon Reading First model play in beginning reading instruction throughout the state?
2. What role should ODE – or the LLSSC – play in directing decisions of districts and schools to build effective, strong beginning reading programs?
24
Policy Considerations
3. Should the focus on beginning reading programs be on K-3?
4. Should the major goal of beginning reading programs be grade level reading achievement at the end of grade 3?
5. Should outcomes determine school effectiveness in building a strong beginning reading program?
25
Policy Considerations
6. Should high quality data be used to determine how well students are reading in each grade of K-2?
7. What implications does an Oregon Reading First model have for inservice PD for teachers, coaches, specialists, and school, district, and state leaders?
26
Policy Considerations
8. What implications does an Oregon Reading First model have for preservice PD for teachers, coaches, specialists, and school, district, and state leaders?
9. What role and responsibilities do institutions of higher education have in helping build the capacity to implement a model of beginning reading instruction based on Oregon Reading First?