Academic Achievement Among English
Learners (ELs) in Wisconsin
An Analysis of ELs Based on 5th Grade
Reclassification Status and English Language
Proficiency Test Scores
Prepared for Carl Frederick,
Wisconsin Department of Public Instruction (DPI)
By
Richelle Andrae
Derek Field
Moira Lenox
Max Pardo
Workshop in Public Affairs
Spring 2017
©2017 Board of Regents of the University of Wisconsin System
All rights reserved.
For an online copy, see
http://www.lafollette.wisc.edu/outreach-public-service/workshops-in-public-affairs
The Robert M. La Follette School of Public Affairs is a teaching and research department
of the University of Wisconsin–Madison. The school takes no stand on policy issues;
opinions expressed in these pages reflect the views of the authors.
The University of Wisconsin–Madison is an equal opportunity and affirmative-action educator and employer.
We promote excellence through diversity in all programs.
Table of Contents
List of Tables ...................................................................................................................................v
List of Figures ................................................................................................................................ vi
Foreword ....................................................................................................................................... vii
Acknowledgments........................................................................................................................ viii
List of Abbreviations ..................................................................................................................... ix
Executive Summary .........................................................................................................................x
Discussion of English Language Learners in Wisconsin .................................................................1
Prior Research ..............................................................................................................................2
Research Questions ......................................................................................................................3
Data ..................................................................................................................................................3
Methods........................................................................................................................................6
Limitations and Assumptions ..........................................................................................................7
Findings............................................................................................................................................8
Research Question 1: Differences Between Student Groups .......................................................8
Research Question 2: Impact of Student and School Characteristics ..........................................9
Language ................................................................................................................................10
Free and Reduced Lunch Status .............................................................................................11
Gender ....................................................................................................................................11
Disability Status .....................................................................................................................11
Chronic Absenteeism .............................................................................................................11
Retention in Programming after Scoring a 5-5 in 4th Grade .................................................11
Low Number of Years in EL Programming ..........................................................................11
School Locale.........................................................................................................................12
Groups and Variable Interactions without Significant Findings............................................12
Discussion ......................................................................................................................................12
Recommendations ..........................................................................................................................13
Recommendations Related to Further Data Collection and Use ...............................................13
Recommendations Specific to Findings of Our Analysis ..........................................................14
Opportunities for Further Research ...............................................................................................15
Conclusion .....................................................................................................................................15
Appendix A: Relevant Literature ...................................................................................................17
Appendix B: 5th Grade as Inflection Point ....................................................................................19
Appendix C: Interview Protocol ....................................................................................................21
Appendix D: Student Characteristic Breakdowns by Reclassification Group...............................22
Appendix E: Limitations and Assumptions ...................................................................................25
Appendix F: Results by Student Characteristic Interactions .........................................................27
Appendix G: Data Preparation .......................................................................................................28
Appendix H: Regression Output ....................................................................................................29
References ......................................................................................................................................45
v
List of Tables
Table 1: Comparing Demographics of Students Exiting ................................................................................................. 4 Table 2: Categorization of 5th Grade Students .............................................................................................................. 5 Table 3: Literature Review on Reclassification Research ............................................................................................. 17 Table 4: Number of Students Exiting EL Programming by Grade, 2007-2016 ............................................................. 19 Table 5: Significant Results for Students by Group, Interacted with Student and School Characteristics ................... 27 Table 6: Change in 8th Grade Math and Reading Score Percentiles by Student Group, Controlling for 5th Grade and 4th Grade Baseline Score Percentiles........................................................................................................................... 29 Table 7: Change in 8th Grade Math Score Percentile by Student Group, Comparison of Three Models ..................... 29 Table 8: Change in 8th Grade Reading Score Percentile by Student Group, Comparison of Three Models ................. 30 Table 9: Change in 8th Grade Math Score Percentile by Student Group and Language ............................................. 31 Table 10: Change in 8th Grade Reading Score Percentile by Student Group and Language ....................................... 32 Table 11: Change in 8th Grade Math and Reading Score Percentiles by Student Group and FRL Eligibility ............... 34 Table 12: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Gender ......................... 35 Table 13: Change in 8th Grade Math Score Percentile by Student Group and School Locale ...................................... 37 Table 14: Change in 8th Grade Reading Score Percentile by Student Group and School Locale ................................. 38 Table 15: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Low EL Student Population.................................................................................................................................................................... 40 Table 16: Change in 8th Grade Math and Reading Score Percentiles by Student Group and 4th Grade In-Over Status ..................................................................................................................................................................................... 41 Table 17: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Chronic Absenteeism in or Before 5th Grade ..................................................................................................................................................... 42 Table 18: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Presence of a Learning Disability ...................................................................................................................................................................... 43 Table 19: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Participation in EL Programming of Fewer Than Three Years ................................................................................................................... 44
vi
List of Figures
Figure 1: Difference in Average 8th Grade and 4th Grade WSAS Score Percentiles by Group ......................................... 6 Figure 2: Difference in 8th Grade Math and Reading WSAS Percentile Scores by Group .............................................. 9 Figure 3: Results for ELs Overall by Characteristic ....................................................................................................... 10 Figure 4: Reclassified Students per Grade ................................................................................................................... 19 Figure 5: Breakdown of Gender in 5th Grade by Group ................................................................................................ 22 Figure 6: Breakdown of Home Language in 5th Grade by Group ................................................................................. 22 Figure 7: Breakdown of School Locale Code in 5th Grade by Group ............................................................................. 23 Figure 8: Breakdown of FRL Eligibility in 5th Grade by Group ...................................................................................... 23 Figure 9: Average WSAS Reading Score Percentiles in 4th and 8th Grade by Group ..................................................... 24 Figure 10: Average WSAS Math Score Percentiles in 4th and 8th Grade by Group ....................................................... 24 Figure 11: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Language .................... 33 Figure 12: Change in 8th Grade Math and Reading WSAS Score Percentiles by Group and FRL Eligibility ................. 35 Figure 13: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Gender ........................ 36 Figure 14: Change in 8th Grade Math and Reading Score Percentiles by Student Group and School Locale .............. 39 Figure 15: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Low EL Student Population.................................................................................................................................................................... 40 Figure 16: Change in 8th Grade Math and Reading Score Percentiles by Student Group and 4th Grade In-Over Status ..................................................................................................................................................................................... 41 Figure 17: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Chronic Absenteeism .. 42 Figure 18: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Learning Disability ...... 43 Figure 19: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Participation in EL Programming for Fewer Than Three Years .................................................................................................................. 44
vii
Foreword
This report is the result of collaboration between the Robert M. La Follette School of Public
Affairs at the University of Wisconsin–Madison and the Wisconsin Department of Public
Instruction (DPI), a state agency. The objective of this project is to provide graduate students at
the La Follette School the opportunity to improve their policy analysis skills while contributing
to the capacity of partner organizations.
The La Follette School provides students with a rigorous two-year graduate program leading to a
master’s degree in public affairs. Students study policy analysis and public management as well
as concentrated study in at least one policy area. The authors of this report all are in their final
semester of their degree program and are enrolled in the Public Affairs 869 Workshop in Public
Affairs at UW–Madison. Although studying policy analysis is important, there is no substitute
for engaging actively in applied policy analysis as a means of developing policy analysis skills.
The Public Affairs 869 Workshop gives graduate students that opportunity.
The DPI works to advance public education in Wisconsin and administers the systems that serve
students across school districts and programs. Wisconsin is viewed as a national leader in
English Language Learner programs, and the DPI has an ongoing focus on quality improvement,
including through research partnerships with UW–Madison. The DPI also has a strong history of
working with La Follette School students to perform rigorous research. This report includes an
analysis of administrative data, as well as interviews and other research, with the goal of helping
the DPI to improve services for students statewide.
I am grateful to the DPI for partnering with the La Follette School on this project. DPI staff have
been exceptionally generous with their time to support this project, including collaborating on
data analysis. The students have collectively contributed hundreds of hours to this project, and in
the process developed critical insights about state policies and programs. The La Follette School
is grateful for their efforts and hope that this report proves valuable for the DPI and the state of
Wisconsin to improve the outcomes of English Language Learners.
J. Michael Collins
Professor of Public Affairs
May 2017
Madison, Wisconsin
viii
Acknowledgments
We would like to express our gratitude to Carl Frederick, Audrey Lesondak, Justin Meyer, and
Jesse Roberts of DPI for their consistent and valuable feedback, support in developing nuanced
approaches to research, and passion for serving Wisconsin students. We would also like to thank
district administrators for providing on-the-ground insights to EL programming. Finally, we
extend our gratitude to our project advisor, Professor J. Michael Collins, for his ongoing
guidance, as well as Lisa Hildebrand for her editorial assistance.
ix
List of Abbreviations
ACCESS Assessing Comprehension and Communication in English State-to-State for
English Learners
AMAO Annual Measurable Achievement Objectives
BLBC Bilingual Bicultural
DPI Department of Public Instruction
EL English Learner
ELD English Language Development
ELP English Language Proficiency
ESSA Every Student Succeeds Act
ESEA Elementary and Secondary Education Act
FE Fixed Effects
FRL Free and Reduced Lunch
IDEA Individuals with Disabilities Education Act
LEP Limited English Proficient
NCLB No Child Left Behind
SE Standard Error
WIDA Originally: Wisconsin, Delaware, and Arkansas; Adjusted: World-class
Instructional Design; currently no acronym defined
WKCE Wisconsin Knowledge and Concepts Examination
WSAS Wisconsin Student Assessment System
x
Executive Summary
Many Wisconsin schools provide English learner (EL) students with targeted English language
(EL) programing to support their academic growth and positive educational outcomes. These
schools assess the English proficiency of EL students while they are still in EL programming to
help educators determine when the students are prepared to “exit” EL programming. The
Department of Public Instruction (DPI) would benefit from increased insight on how Wisconsin’s
EL programs can best support this historically underserved student population.
The purpose of this report is to better understand how “exiting” students out of EL programs affects
their future academic achievement. DPI provided nine years of student-level data, which includes
EL students’ demographics, school and district enrollment, Free and Reduced Lunch (FRL)
eligibility, disability status, standardized test scores, and English proficiency scores. We base our
analysis on two main questions: How do academic outcomes differ between students who exit EL
programming with various levels of English proficiency? What student or school characteristics
predict future academic performance?
To address the research questions, we compare the future performance on statewide standardized
tests of a sample of 5th grade EL students. We divide this sample of students into four groups,
determined by whether the students were exited out of or remained in EL programming after 5th
grade, and whether the students scored above or below the DPI-recommended guideline score on
the English proficiency assessment. We then use statistical modeling to determine whether English
proficiency level, exit status in 5th grade, or individual student-, school- or district-level
characteristics influence future academic performance.
We find a statistically significant relationship between future academic performance and the four
student groups. Our findings suggest that if a student has reached the recommended score on their
English proficiency assessment, it may be beneficial to exit a student rather than keep the student
in EL programming. We also find that student-level characteristics do matter, noting a non-uniform
relationship between future academic performance and the four student groups depending on
which student- and school-level characteristic is analyzed.
Based on these findings, we recommend that teachers and administrators continue to exercise
discretion when exiting students, but exit students at the recommended score whenever possible.
We also urge DPI to collect student-level data on EL programming to better understand the effects
of various intervention strategies.
1
Discussion of English Language Learners in Wisconsin
The U.S. Constitution requires that all children are provided with equal educational opportunity,
regardless of race, ethnicity, wealth, background, or citizenship status (ACLU). Current federal
and state policies seek to support equitable access to education for historically underserved
students such as racial minorities, low-income children, and English Learners (ELs). ELs are a
federally protected class under Title VI regulation, which outlaws any discrimination based on a
person’s limited English proficiency (U.S. Department of Education 2016). Achievement gaps
for the EL student population—as well as other student subgroups—are gaining attention from
both communities and policymakers, highlighting disparities in academic performance between
different learners (See Appendix I for more information on achievement gaps) (The Equity and
Excellence Committee 2013). The Department of Education encourages high standards for
educational equity, but the politics of educational access, standardized testing, and resource
allocation complicate its realization.
This report adds value to the discussion of programs and policies for the EL student population.
Approximately one in 10 U.S. students qualifies as an EL, and this subset of learners is growing
over time nationally, though currently leveling off in some states (NCES 2016). Our analysis
will focus on ELs in Wisconsin. Appendix I provides details regarding Wisconsin’s EL
population, including demographics and academic performance. Appendix I also gives an
overview—or snapshot—of EL policies and programs across the state. The overview details EL
experiences through the following processes:
Identification
Programming
Assessment and Accountability
Reclassification
Monitoring
This report also includes additional data about the funding of EL programming along with
relevant state and federal regulations. Two relevant areas of interest, assessment and
reclassification, addressed in this report are discussed in brief below.
Assessment Overview
EL students in Wisconsin generally take two kinds of performance assessments, each discussed
in detail throughout Appendix I. The first is the statewide standardized test in both reading and
math. The second is an EL-specific assessment that measures English Language Proficiency
(ELP). This exam is called the ACCESS test, which is administered to students each year that
they are identified as an EL. The ACCESS test is used to determine whether a student should be
exited from EL programming.
Reclassification Overview
DPI provides a guideline regarding when a student should be transitioned out of active EL status.
This guideline requires an ACCESS score of 5-5, meaning a 5 composite score and a 5 literacy
score, both on a 6-point scale. At this point, a student may be exited from programming, thus
2
considered a “reclassified” or “exited” student or a former EL; former ELs are still tracked and
monitored but no longer remain in EL programming. Regardless of DPI guidelines, teachers and
administrators may still exercise considerable discretion when deciding if and when to exit a
student from EL programming.
Our research questions add to the current body of evidence by comparing 5th grade students who
exit out of or remain in EL programming with different levels of English proficiency and
evaluating each component’s relationship with 8th grade academic achievement on standardized
assessments. The report is designed with Wisconsin’s state education agency, the Department of
Public Instruction (DPI), as the primary beneficiary, although the analysis may also be relevant
for administrators at the district level and authorities from other states.
Prior Research
A 2015 analysis completed by graduate students at the La Follette School of Public Affairs for
DPI investigated performance on EL assessments based on student characteristics (Babal et al.
2015). The report concluded that as the starting age of a student in EL programming increased,
the length of time in programming also increased. Student performance also varied by language
groups, with Spanish- and Hmong-speaking students—the two largest EL groups in
Wisconsin—demonstrating slower English language acquisition rates compared with other
language groups. Additionally, lower-income and disabled students fared worse on English
proficiency assessments compared with other students. Finally, that analysis showed that when
students started with a higher level of English proficiency, they reached full English proficiency
more rapidly. Key recommendations for policy included providing assessment subgroup
performance data to school districts and supporting analysis of EL subgroups.
Building on the research of the previous report, this analysis aims to analyze the academic
performance of these students after exiting EL programming. A comprehensive review of other
relevant literature on the topic of English language acquisition, along with reclassification, is
included in Appendix A.
Prior literature has addressed this topic in two primary directions, with mixed findings. Carlson
& Knowles (2016) and Kim (2011) both found that students who spend more time as English
learners have lower eventual testing and graduation outcomes than their peers who spend less
time in EL programs. However, Robinson-Cimpian & Thompson (1989) and Fernandez (2013)
both find conflicting results. Their research suggests that when states lower the testing standards
for removing proficient students from EL programming, students in districts with lower
reclassification thresholds who score between the old and new cutoffs have worse outcomes than
their peers in districts that didn’t lower reclassification standards. We suspect that the
discrepancy between these groups of research may be due to these studies’ assumptions that
changing the reclassification thresholds or the time spent in EL programing causally affects
future outcomes. This may not be equally true of more- and less-proficient students, or findings
from these groups of studies may not be generalizable outside of their individual contexts, so we
therefore seek to investigate this issue and landscape of EL student testing in Wisconsin.
3
Research Questions
Based on input from stakeholders, particularly the priorities expressed by DPI, we focus on two
key research questions aiming to evaluate former EL students after exiting EL programming.
These are:
1) How do EL students’ standardized test scores in 8th grade differ across
students based on based their English proficiency level and EL status in 5th
grade?
2) Based on English proficiency level and EL status in 5th grade, are there any
student- or school-level characteristics that can predict relative differences in
8th grade standardized test scores?
The goal of the first question is to determine whether teachers and administrators appropriately
reclassify students in 5th grade. While educators may reclassify EL students at a time that best
supports their future academic success, it is possible that they may exit students too late or too
early. The goal of the second research question is to identify any student- or school-level
characteristics that may impact students’ educational outcomes. In order to study educational
outcomes, we analyze differences in 8th grade standardized test scores based on whether a
student exited or remained in programming in 5th grade (see the Methods section for further
explanation of our decision to study the relationship between 5th grade characteristics and 8th
grade outcomes). We expand our analysis to include the outcomes of various student subgroups
across characteristics such as home language, gender, eligibility for participating in the federal
Free and Reduced Lunch program, and school type.
To address the research questions, we adopted a mixed-methods approach by reviewing prior
studies, conducting interviews with district administrators involved in EL programming, and
performing a quantitative analysis on student data. This approach provides a holistic perspective
on EL experiences by combining narrative with data. Information learned from interviews
informs how state policies and federal requirements impact on-the-ground administrators as well
as students.
Data
We analyzed four DPI data sets that contained English proficiency scores, statewide standardized
testing scores, disciplinary actions, and school-level data covering the time period 2007 through
2016. We combined all four data sets and removed any students who were observed for less than
four years. In order to analyze student outcomes in 8th grade and compare them against 5th
grade, we need to observe in the 5th and 8th grade. We therefore removed approximately 80,000
students from our sample to facilitate our analysis. The implications of excluding students who
were not consistently in the data are discussed later in the report.
We then converted the panel data to a cross section of 5th grade students. The analysis focuses
on 24,416 5th grade EL students who remained in the data by 8th grade and could be matched
over time.
4
Overall, the demographic characteristics of students exiting EL programming are similar
between grades and especially between 4th and 5th grade, when the largest number of EL
students exit programming. Third grade appears slightly anomalous, though not necessarily in
ways that would be unexpected—we would expect students who exit EL programming early to
be high academic achievers. The higher proportion of non-FRL students among students who
exited EL programming in 3rd grade makes sense given the correlation between socioeconomic
status, for which Free and Reduced Lunch (FRL) is a proxy, and academic achievement.
Table 1: Comparing Demographics of Students Exiting
EL Programming by Grade, 2007-2016
3rd 4th 5th 6th 7th
Hispanic 53% 61% 59% 59% 55%
Asian 34% 28% 32% 28% 35%
White 12% 9% 8% 12% 8%
Other Race 1% 2% 2% 2% 2%
Male 48% 47% 50% 48% 48%
Female 52% 53% 50% 52% 52%
FRL 62% 75% 76% 70% 75%
Non-FRL 38% 25% 24% 30% 25%
Spanish 53% 61% 59% 58% 54%
Hmong 17% 19% 23% 18% 26%
Other Language 30% 20% 18% 25% 20%
Source: Authors’ Analysis, DPI Data 2007-2016.
We then categorize 5th grade students into one of four groups, based on two questions:
1) Did the student remain in or exit out of programming? (In or out?)
2) Did the student reach the DPI recommended guideline score of 5-5 on the English
proficiency assessment? (Under or over?)
Consequently, four groups are created: those who exit programming with scores above the
guideline (Out-Over group), those who exit with scores below the guideline (Out-Under group),
those who remain in programming with scores above the guideline (In-Over group), and those
who remain with scores below the guideline (In-Under group). The following table depicts each
group:
5
Table 2: Categorization of 5th Grade Students
Did the student exit EL programming?
Yes → “Out” of programming No → “In” programming
Did the
student reach
the DPI-
recommended
guideline
score of 5-5 on
the English
proficiency
assessment?
Yes →
“Over”
score
Out-Over:
Exited as expected, having
met the DPI-recommended
guideline for exit.
2,343 students
16%
In-Over:
Remain in programming
despite having reached the
guideline score; an exception
to keep in longer.
1,371 students
9%
No →
“Under”
score
Out-Under:
Exited programming early;
an exception to exit students
earlier than guidelines.
146 students
1%
In-Under:
Have not reached the DPI-
recommended score and
remain in programming.
11,329 students
74%
Source: Authors’ Analysis, DPI Data 2007-2016 15,099 students total in 5th grade.
The In-Under group is by far the largest, with 11,329 students. This is unsurprising, as this group
represents the bulk of EL students who are still progressing toward proficiency. Next largest is
the Out-Over group with 2,343 students, followed by the In-Over group with 1,371 students. The
smallest group is the Out-Under group at 146 students. These are the students for whom teachers
or administrators decided to make exceptions and exit from programming before they had
reached the recommended guideline score of 5-5. All but one (145 of 146) received a five or
above for their overall ACCESS score, but only a four for their literacy subscore. Finally,
students who had already exited EL programming by 5th grade (former EL students) were
identified so they could be controlled for in analysis. There are 4,603 former EL students in our
5th grade sample.
Overall, the different groups look demographically similar, with some exceptions for the Out-
Under group. Students in the Out-Under group are more likely to be male, attend school in a
town, and are less likely to be FRL-eligible than students in the other groups are. However, it is
important to note that this group is the smallest and is more prone to variations due to random
error. It is also noteworthy that a higher proportion of the In-Over group is female, although it is
unclear why this may be. (see Appendix D for a more complete breakdown of characteristics
group).
We use 4th grade standardized test score percentiles on both mathematics and reading/English
language arts tests for each group of students in our model as a control for baseline academic
6
performance. Because standardized tests are administered later in the school year than ACCESS
tests, 4th grade standardized scores are the most appropriate pre-5th grade ACCESS-test
baseline. Depending on when the decision to reclassify students is made, these might be the most
recent available scores on which teachers and administrators might base their decisions.
Average score percentiles by group in both 4th and 8th grade reflect the expected relationship
between exit status, English proficiency score relative to the 5-5 guideline, and outcomes on
standardized assessments, with those in the Out-Over group performing best and those in the In-
Under group performing worst. However, we see a different pattern emerge for the change in
percentile scores between 4th and 8th grade. Figure 1 shows the difference in average
standardized test score percentiles between 4th and 8th grade. All groups show improvement
between 4th and 8th grade, even the In-Under group. However, while the Out-Over group shows
the most improvement in reading and math, at best the In-Over shows equivalent improvement
to the other two groups in reading and at worst shows the smallest improvement in math. This is
an early indication that students who exit in 5th grade benefit from doing so, while those who
remain may be kept in programming for too long. We further explore the relationship between
exit status and 8th grade test scores in our analysis below.
Figure 1: Difference in Average 8th Grade and 4th Grade WSAS Score Percentiles by Group
Source: Authors’ Analysis, DPI data 2007-2016
Methods
To answer our first research question, we examine the impact of reclassification and ACCESS
score in 5th grade on 8th grade standardized test score outcomes. Fifth grade appears to be an
0%
2%
4%
6%
8%
10%
Out-Over In-Over Out-Under In-Under
Reading Math
7
inflection point for students exiting out of EL programming, as shown in Appendix B. It is the
peak year in which students exit (2,842 exiters), closely followed by 4th grade (2,582 exiters).
The number of students reclassifying outside of 4th or 5th grade drops off with approximately
1,200 students exiting in 6th or 7th grade and only 18 students in 8th grade. Focusing on 5th grade,
therefore, provides us with the largest analytical sample of reclassified EL students. We use 8th
grade standardized test scores as outcomes because 8th grade is the last consecutive year of
testing for students. The next test that students take is the ACT® in 10th grade, which would
further attenuate our analytical sample, as we observe significantly fewer students through to
10th grade. Outcomes are measured as percentile scores for the math and reading subsections of
Wisconsin state standardized tests. To account for year-to-year variations in test formats, we use
score percentiles, rather than raw or scaled score.
We first use Ordinary Least Squares (OLS) regression to estimate differences in test scores for
the four groups listed above, controlling for 4th grade standardized test scores, to determine
changes in performance for each group. We then build upon this specification by introducing
controls for student characteristics such as gender, student language, and FRL eligibility. To
make our estimates more accurate, we finally include district fixed effects and account for
unobserved school-specific variation. Our full model, therefore, accounts for most of the
potentially important factors at the student, school, and district levels.
To address our second research question, we interact the student group variables with student
language, gender, time in EL programming, school EL student concentration, and others
characteristics. The interaction of these variables with the student groups allows their effect to
vary by student group, shedding light on how these characteristics might impact the score
estimates differently. For tables displaying regression output, see Appendix H.
Limitations and Assumptions
While we find our analysis to be rigorous, the report is limited by the availability of data, leading
to several constraints, along with necessary assumptions for the analysis to hold. State law does
not require private schools to report their students’ performance so we do not consider those
students here. Lack of consistent data is also an issue for highly mobile students. Our data was
also limited regarding students’ primary languages, as we cannot consider language-based
differences between the 3,277 students in the “other” language category.
We did not have access to data that could account for diversity of the many EL programs in
Wisconsin schools. Outcomes may vary between these schools based on structural differences
and delivery of programming.
Our qualitative information from practitioner interviews is inherently limited by scale. The
sample of people we interviewed is a small portion of program staff from Wisconsin’s 53 districts
with Bilingual Bicultural (BLBC) programs.
The stated limitations, reviewed in Appendix E, led to several necessary assumptions for
completing this report. Comprehensive assumptions are discussed in Appendix E, but several are
included here. We assume student performance across standardized assessments is comparable
8
over each year of test iterations. This report also assumes that reclassification processes and
implications are generalizable outside of 5th grade, the only year of analysis included. However,
we may conclude that this report provides implications for middle school students alone.
Additionally, we assume no changes in individual student characteristics across time such as
gender and FRL eligibility. We also included former EL students in the performance of the EL
population overall, likely skewing performance of that group higher than if the former EL group
had been excluded.
Findings
Research Question 1: Differences Between Student Groups
To answer our first research question, whether English proficiency level and timing of exit from
EL programming in 5th grade affects 8th grade academic outcomes, we look to our full models
controlling for student characteristics and school and district effects. Though we focus on the
results of those final models, our findings were consistent and both statistically and substantively
significant for each student group across all iterations of our models.
Effects for each group can be interpreted as percentage-point increases above a baseline
percentile for observationally equivalent students who score below the 5-5 guideline and remain
in EL programming (In-Under). In other words, the baseline state for a student is in the In-Under
group, and the effects stated below are the percentile increases that come from membership in
one of the three other groups (Out-Over, In-Over, or Out-Under). As expected, students who
score above the 5-5 guideline in 5th grade and are exited out of EL programming (Out-Over)
score the highest on average in 8th grade. More specifically, Out-Over students score nearly nine
percentile points higher on 8th grade math exams and nearly 10 percentile points higher on 8th
grade reading exams than In-Under students, when controlling for student characteristics and
school and district effects. Students who score above the 5-5 guideline but remain in EL
programming (In-Over) and students who score below the 5-5 guideline but are moved out of
EL programming (Out-Under) perform roughly the same, scoring five to six percentile points
higher on 8th grade math and reading exams than In-Under students. Figure 2 shows these
differences in percentile for each group of students relative to the In-Under group, with diamonds
representing coefficient point estimates and shaded bars showing 95 percent confidence
intervals.
9
Figure 2: Difference in 8th Grade Math and Reading WSAS Percentile Scores by Group
Source: Authors’ Analysis, DPI Data 2007-2016
Research Question 2: Impact of Student and School Characteristics
To answer our second research question, whether certain characteristics impact 8th grade
academic performance differently by our four groups, we turn to our interacted models. These
models explore how and if average 8th grade standardized test score percentiles for each student
group vary based on characteristics such as student language, FRL eligibility, gender, 4th grade
ACCESS score and exit status, absenteeism, learning disability, time in EL programming, school
locale, and school EL student concentration.
Figure 3 shows all statistically significant results for the un-interacted controls from our
interaction models, with diamonds representing coefficient point estimates and shaded bars
showing 95 percent confidence intervals. These are the results for characteristics that remain
significant even after we allow their relationship to 8th grade scores to vary by student group.
What is left, then, can be interpreted as applying to students overall, regardless of membership
in one of the four student groups. For example, in our model interacting Hmong speakers with
group membership, none of the interacted results was significant, but we did see significant
negative results for all Hmong speakers in both math and reading. These are the effects reported
in the first two bars of Figure 3. Estimates below the horizontal axis show that students associated
with those characteristics perform worse than their observationally equivalent peers who do not
share that characteristic, while estimates above the horizontal axis show that such students
perform better.
10
We see that Hmong-speaking students, those with learning disabilities, and those who are
chronically absent perform worse across the board when compared to their observationally
equivalent peers, while students who speak languages other than Hmong and Spanish, those
who scored above 5-5 in 4th grade but remained in EL programming, and those who have been
in EL programming for fewer than three years all perform better overall. For two categories,
gender and FRL eligibility, results were significant only for reading scores, with female
students performing better than their male peers overall and FRL eligible students performing
worse overall. We discuss these results, as well as the interacted results, in more detail below.
Additionally, Appendix H contains a table with all significant results from our interaction
models.
Figure 3: Results for ELs Overall by Characteristic
Source: Authors’ Analysis, DPI Data 2007-2016
Language
Hmong-speaking EL students perform 3.1 percentage points worse on math and 3.9 percentage
points worse on reading than their non-Hmong speaking peers. Students who speak minority
languages fare better than their Hmong- and Spanish-speaking peers by 1.7 percentage points in
math and 2.4 percentage points in reading.
Subgroup Analysis: Spanish-speakers in the Out-Over category performed 2
percentage points worse on math in 8th grade than those in the Out-Over group who
speak Hmong or other languages. Spanish speakers in the In-Over category also
performed worse on math than those in other categories, by 2.5 points. Those in the
“other” language category, who remain in programming despite having scored a 5-5
(In-Over group) also outperform their peers by 4.2 percentage points in math.
11
Free and Reduced Lunch Status
EL students eligible for FRL (a majority of ELs) fare worse than non-eligible EL students in
reading by 1.2 percentage points.
Gender
Female EL students outperform male EL students in reading by 3.4 percentage points.
Subgroup Analysis: Females in the Out-Over category underperform on math by 1.8
percentage points compared to males in the same category.
Disability Status
EL students with intellectual disabilities fare poorer than students without a disability, by 5.5
percentage points in math and 5.7 percentage points in reading.
Subgroup Analysis: While the number of students with disability status yields low
sample sizes, findings for subgroups were statistically significant. Out-Under ELs with
an intellectual disability performed 16.5 percentage points better in math than their
non-disabled Out-Under peers. In-Over disabled students performed 7.4 percentage
points better in math and 6.6 percentage points better in reading than In-Over non-
disabled students.
Chronic Absenteeism
Chronically absent EL students underperform their EL peers with consistent attendance by 3.2
percentage points in math and 3.8 percentage points in reading.
Retention in Programming after Scoring a 5-5 in 4th Grade
Some students scored a 5-5 on their 4th grade English proficiency exam but were kept in EL
programming through at least 5th grade. These students outperform students who were not
retained in EL programming for at least an additional year by 2.9 percentage points in math and
2.3 percentage points in reading.
Subgroup Analysis: EL students in the Out-Over group perform worse in math than
other Out-Over students who did not score a 5-5 in 4th grade by 2.7 percentage
points, and In-Over students who scored at least a 5-5 in 4th grade but remained in
programming fare worse than those who were exited by 3.3 percentage points in
reading.
Low Number of Years in EL Programming
EL students who, by 5th grade, have been in programming for fewer than three years fare better
than students in EL programming for longer periods of time. They outperform their peers by 2.2
percentage points in math and 2.5 percentage points in reading.
12
Subgroup Analysis: In-Over ELs outperform those in programming for longer
periods of time by 2.5 percentage points in math and 2.8 percentage points in
reading.
School Locale
Students in towns outperform ELs in other types of schools (cities, suburbs, and rural schools)
by 3.3 percentage points.
Subgroup Analysis: Students in the In-Over category who attend city schools also
perform worse in math by 3 percentage points than their peers. In-Over ELs in suburbs
perform 2.4 percentage points better in reading than In-Over students in other types of
schools.
Groups and Variable Interactions without Significant Findings
We did not find significant results for the majority of subgroups (see Appendix H for full report
results). For example, there were no significant results for students in schools with a low
concentration of EL students. These findings may be due to any number of factors, including
that such student or school characteristics truly have little effect on student outcomes, or that
variation is lost due to standard errors clustered by school. Regardless, it is important to note that
not all subgroups showed significant variation in outcomes.
Discussion
From these findings, we can conclude that variation in performance is non-uniform across
subgroups. We may have expected, for example, that In-Over students, having been selected to
remain in programming for at least a year past 5th grade, would fare better on later standardized
tests given the continuing supportive services. This was not the case. This group consistently
performed worse overall when compared to their peers who were exited out of programming in
5th grade with similar scores relative to the 5-5 guideline. It appears as though students who
have reached the 5-5 score benefit from exiting out of programming. Still, we did see
heterogeneity in the relationships between certain student characteristics and 8th grade test
scores for different student groups. For example, In-Over Spanish-speakers performed worse
than their peers in math, while In-Over “other language” speakers performed better. Student
characteristics seem to matter, but the noisiness of the results highlights the continued need for
discretion in decisions about reclassification.
The only consistent negative finding across subgroups based on exit score in 5th grade were the
Out-Over students, who performed worse as Spanish-speakers in math, females in math, and as
students retained in programming who had scored at least a 5-5 the previous year in both reading
and math. This suggests that some students do indeed struggle more upon exiting EL
programming than others. However, in no cases were these negative effects large enough to erase
the score difference between the Out-Over and In-Over groups. This means that though these
13
students may struggle more than their peers may after they are out of programming, they still
appear to do better than those who remain in programming.
Such inconsistent variation points to the challenge in creating comprehensive policies and
systems for exiting students. Additional analysis of student and school characteristics such as
program type, community supports, educational attainment of families, or student engagement
in extracurricular programs all may support evaluation of student performance. Discretion is
important in considering the unique needs of each child and the available resources to serve the
student, but decision-makers may benefit from considering how various subgroups with
significant results perform when compared to other students.
Recommendations
Based on our findings and discussion above, we will aim to provide DPI and other stakeholders
with key policy and programmatic recommendations. Recommendations are grouped based on
general suggestions related to further data collect and use, and suggestions directly related to
findings of our analysis.
As discussed in the data section of this report, we removed a significant number of students from
the analysis. The exact implications of dropping students who were not observed for at least four
years are unknown, but we can speculate that the students removed may be uniquely
differentiated from those who are observed consistently. Removed students may be more mobile,
transferring in and out of school systems and even the state. This may also be a group with lower
overall socioeconomic status and reduced educational opportunities. Therefore, excluding these
students from analysis may limit the generalizability of implications and recommendations for
all students. It is important to consider that these students also require EL resources and may
even face more barriers to success than their peers may. The following recommendations should
be considered with these caveats in mind.
Recommendations Related to Further Data Collection and Use
1. In this analysis, we were unable to control for any type of EL programming
characteristics. The interventions in a Dual Language Immersion program may
vastly differ from that in a Sheltered Instruction setting, for example. Thus,
outcomes may range broadly depending on the type of program. It is imperative
that DPI continues its work in cataloguing and classifying types of programming.
Ensuring accurate collection of this program data at the student level, not just
school or district, is necessary in advancing the analysis of both student and
program performance.
2. Similar to lack of programming data, we did not assess funding for EL services at
either the student-, school- or district-level. We recommend that DPI track funding
allocations and expenditures by different institutional levels to determine if
allocation of resources effectively supports student outcomes.
3. Likewise, current data on test accommodations reflects only whether students did
or did not receive accommodations. Information on what type of accommodations
14
are provided to students is not recorded, but it would be helpful to understand
student experiences and control for different testing circumstances.
4. We encourage districts and schools to track and assess their reclassification
decisions using data. At what ELP level are most students exited? Does this vary
between schools? How much discretion is used by primary teachers, ESL teachers,
and district administrators? What criteria do they use to make reclassification
decisions? Are patterns identifiable? Analyzing such school- and district-level data
would allow administrators to develop policy priorities and to note changes in
decision criteria used for reclassifying students.
5. Districts should consider analyzing data on students who re-enter EL programming
after exiting. Any patterns in students’ re-entry into EL programming may uncover
flaws in how and when administrators decide to exit students.
Recommendations Specific to Findings of Our Analysis
1. Our findings suggest that, all else equal, if a student has scored 5-5 by 5th grade,
he or she is better off exiting programing than remaining in. Teacher discretion in
keeping students in programming after that point should be exercised judiciously,
erring on the side of exiting a student. This recommendation relates to the primary
research question, comparing students of various ELP levels in 5th grade.
2. Variation is seen across students in different language groups and exit subgroups.
While findings are mixed, targeting resources specifically at students based on
language group could be a valuable investment in moving Spanish-speaking
students to English proficiency more quickly.
3. While the “other” language speakers fare better compared to their Spanish and
Hmong-speaking peers, we recommend that DPI continue to think critically about
this heterogeneous group of students. They speak a wide range of languages and
require student-specific interventions based on the needs of each individual child.
4. With such variety in outcomes for subgroups, we would not recommend that DPI
tighten recommendations on exiting, but rather encourage schools to evaluate their
own individual practices and focus on student outcomes.
5. ESL teachers and administrators, when deciding whether to exit a student from EL
programming, should use student- and school-level data that is available to them in
order to contextualize the decision. It is unclear to what degree this practice is
uniform across districts in the state, as varying degrees of discretion are used,
according to our limited interviews. Perhaps, districts also could learn more from
one another by communicating their practices and discussing patterns in outcomes.
Regardless, making broad generalizations is unwise because there are varying
effects of different characteristics for both students and schools. Rather, decisions
should be customized to the particular circumstances and context of each student,
with the 5-5 exit guideline used where possible.
15
Opportunities for Further Research
Data collected and reported on EL student progress and performance needs improvement. Most
of these improvements require policy changes rooted in state law along with a commitment by
the state legislature to refrain from annually changing the way that Wisconsin students are
assessed. Data structure and availability issues that we encountered in conducting this analysis
include:
1. The fact that EL academic performance and English language proficiency is tested
only once per year poses challenges to anyone trying to study populations of
students who are highly mobile. This is likely true for a portion of EL students, with
bilingual students often being raised in migrant or refugee families that may
relocate one or more times while a student is enrolled in EL programming.
2. Our analysis also excludes any students who attended private schools between 4th
and 8th grade in Wisconsin. We lack any data on the performance or proficiency of
EL students in these schools. The inevitable differences in English language
acquisition programming for these students is worth study and evaluation.
3. Our analysis of reclassifying students’ outcomes does not consider information
about students who reclassify before 5th grade. If these students are meaningfully
different from those who reclassify in 5th grade, perhaps by their access to
supplemental English tutoring, exposure to English in their homes, or academic
aptitude and speed of language uptake, any differences in outcomes among pre-5th-
grade reclassified EL students should be studied.
4. We were unable to consider any of the inevitably numerous differences in the
characteristics of individual EL programs because that data does not exist.
Characteristics of EL programs that differ school-to-school may have an effect on
EL students’ language uptake or outcomes, such as program structure, funding,
staff ratios, and staff qualifications, are worth study.
5. Charter schools are perhaps more likely than traditional public schools to be
heterogeneous in their EL program characteristics. Given recent policy-driven
efforts to expand access to school choice options, especially in populous parts of
the state with many bilingual students, this topic deserves attention by education
policymakers.
Conclusion
This report shines new light on the performance of a key group of historically disadvantaged
students. We find that where teachers exercise discretion in exiting ELs from programming after
5th grade, they should only do so using caution if a student has already achieved the
recommended 5-5 score. Various subgroups, including Spanish- and Hmong-speakers and
chronically absent students, have poorer outcomes on standardized tests after exiting EL
programming, depending on test subject. We find that categorizing students by their English
proficiency score level at time of exit is a helpful tool in determining the appropriate use of
teacher discretion in exiting behavior.
16
To accurately measure the effects of widely varying programs, we recommend that DPI require
and analyze student-level data on type of services received across the state. Doing so will provide
a more complete picture of programming impacts on student outcomes, along with the
opportunity to target resources at successful interventions.
Using both a quantitative and qualitative approach to the exploration of EL services will support
districts and administrators in creating policy that comprehensively addresses the needs of
students. The snapshot of EL experiences paired with provided recommendations can help DPI
better allocate resources for those Wisconsin students with the greatest need.
17
Appendix A: Relevant Literature
Several scholarly studies on reclassification focused on the effects of changes in the score cutoffs
used to determine reclassification. Robinson-Cimpian & Thompson (2016) and Will et al. (2014)
found that for states or districts that adopt higher reclassification cutoff scores, the students who
scored between the two and who no longer reclassify with that score have better outcomes than
similar students who did reclassify with that score before the policy change. However, Carlson
& Knowles (2016) and Kim (2011) found that students who spent more time classified as EL
have worse test-score achievement and grade progression than students who spend less time
classified as EL.
These studies appear to contradict each other. On the one hand, Robinson-Cimpian & Thompson
(2016) and Will et al. (2014) suggest that reclassifying students at lower ELP levels may remove
them from programming that could offer them support to improve their achievement outcomes.
On the other hand, Carlson & Knowles (2016) and Kim (2011) found that students who spend
more time in EL programming have worse achievement outcomes. Other studies find different
rates of ELP progress or post-reclassification outcomes based on students’ backgrounds.
Table 3: Literature Review on Reclassification Research
Source Research Question Sample and Method Findings
Robinson-Cimpian & Thompson 2016 “The Effects of Changing Test-Based Policies for Reclassifying English Learners”
This paper examines the effect of raising the threshold for reclassification on Latino/a EL students in the Los Angeles Unified School District.
Difference-in-regression-discontinuities method with student-level California student performance and outcomes data
Students who were reclassified before the changes but whose scores would not have met the higher thresholds had lower outcomes trends than students with these scores who weren’t reclassified under the higher threshold.
Carlson & Knowles 2016 “The Effect of English Learner Reclassification on Student ACT Scores, High School Graduation, and Postsecondary Enrollment: Regression Discontinuity Evidence from Wisconsin”
This paper estimates the causal effect of being reclassified at the end of 10th grade on several outcomes related to postsecondary educational attainment: ACT scores, high school graduation, and postsecondary enrollment.
Regression discontinuity that exploits the reclassification cutoff at or above a certain ACCESS score to identify the effect of reclassification of Wisconsin EL students between 2006 and 2013
Reclassification in 10th grade has a positive effect on ACT scores, the probability of high school graduation, and the probability of postsecondary enrollment relative to those who don’t reclassify in the 10th grade.
18
Kim 2011 “Relationships Among and Between EL Status, Demographic Characteristics, Enrollment History, and School Persistence”
This paper examines enrollment history, achievement gaps, and persistence in school for EL students compared to non EL students.
Multilevel logistic regression to show large achievement and socioeconomic gaps between EL and non-EL students in California
EL students who reclassified later or who remain in EL status in high school show larger gaps compared to EL students who reclassified earlier. The longer a student is designated as EL, the more likely they are to drop out before graduating high school.
Slama 2014 “Investigating Whether and When English Learners are Reclassified Into Mainstream Classrooms in the United States: A Discrete-Time Survival Analysis”
This study examines EL students’ tenure in language-learning programs and their academic performance following reclassification.
Discrete-time survival analysis of EL student data in the US to estimate the average time to and grade of reclassification with and without controlling for home language and socioeconomic status
The average EL student exits three years after school entry or in second grade, and the odds that a non-Spanish-speaking EL student was reclassified were nearly twice that of their Spanish-speaking EL counterparts, after controlling for income.
Umansky & Reardon 2014 “Reclassification Patterns Among Latino English Learner Students in Bilingual, Dual Immersion, and English Immersion Classrooms”
This study examines timing of reclassification among Latino ELs in four distinct linguistic instructional environments: English immersion, transitional bilingual, maintenance bilingual, and dual immersion.
The authors use hazard analysis and 12 years of data from a large school district to investigate whether reclassification timing, patterns, or barriers differ by linguistic program.
Latino EL students enrolled in two-language programs are reclassified at a slower pace in elementary school but have higher overall reclassification, English proficiency, and academic threshold passage by the end of high school.
Hill, Weston, & Hayes 2014 “Reclassification of English Learner Students in California”
This paper investigates whether California school districts with more rigorous reclassification standards have systematically lower EL reclassification rates and/or better student outcomes than districts with lower standards.
Cross-sectional cohort comparison using six years of California state-level student data: grade progression data and standardized test performance data
Reclassified EL students outperform non-reclassified students and perform as well as native English speakers on standardized tests and grade progression. Districts using more stringent reclassification criteria have lower reclassification rates and better outcomes among reclassified students.
19
Appendix B: 5th Grade as Inflection Point
In our discussions with DPI about how to focus our analysis, they suggested that we might want
to look at ELs in 5th grade, which they identified as an inflection point for reclassification of EL
programming. According to our data, 5th grade is the peak exit year for EL students, followed
closely by 4th grade. Therefore, our data supports using this cross-section of 5th grade students
for analysis.
Figure 4: Reclassified Students per Grade
Source: Authors’ Analysis, DPI data 2007-2016
Table 4: Number of Students Exiting EL Programming by Grade, 2007-2016
Grade Number of Students
in EL Programming
that Grade
Number of Students
Exiting EL
Programming that
Grade
Percentage of
Students who Exit
Programming that
Grade
1st 7,556 8 0.1
2nd 11,712 317 2.7
3rd 15,404 1193 7.7
4th 18,185 2582 14.2
5th 19,494 2842 14.6
6th 16,453 1018 6.2
7th 15,513 1332 8.6
8th 12,145 18 0.1
Source: Authors’ Analysis of DPI data, 2007-2016
20
Curriculum-based best practices and empirical evidence suggest that 5th grade, or around 11
years of age, is a critical inflection point in the rate of students’ learning progress toward English
proficiency. A 1989 study by Johnson and Newport gave an English grammar test to a sample
of adult Chinese and Korean immigrants who entered the United States and divided participants’
performance by age-of-entry categories. Mean scores for immigrants who arrived in the 3-7 age
group scored 269.3 (compared to the native-speaker mean score of 268.8). The mean score of
those who entered at ages 8-10 was 256.0, a 13.3-point drop; those who entered at ages 11-15
scored 235.9, a 20.1-point drop; and those who entered at ages 17-39 scored 210.3, a 25.6-point
drop. Declines in grammar test scores accelerate as the age of entry increases. The authors
suggest that the second-language acquisition of those with pre-puberty ages of entry and post-
puberty ages of entry are significantly different. They go on to identify puberty as the critical
period for second-language acquisition (Johnson 2009). Most entering 5th graders are 11 years
old, suggesting that the rate of English language uptake changes around 5th grade.
21
Appendix C: Interview Protocol
To provide a qualitative perspective on the EL experience in Wisconsin, we conducted
interviews with administrators at four mid-sized school districts. These districts have between
715 and 2,281 current EL students, with total enrollments of 5,418 to 22,160 students (DPI
2017j). We believe that through discussions with on-the-ground educators, we can more
comprehensively understand the experience for EL students in Wisconsin. We do not assume
that these stakeholders express experiences that are representative of all administrators; rather
their perspectives are unique and valued in their individual contexts. Interviewed staff manage
EL programming, supervise ESL teachers, oversee data collection and reporting, and provide
leadership in policy direction, among other activities. We used the following interview questions
to construct an informative conversation with stakeholders:
1) Brief description of report, research questions, target understanding of EL lifecycle.
2) What is your role in the district or school, specifically as related to EL students,
programming, and/or administration?
3) Are you able to describe the EL lifecycle, or any pieces? Address identification,
programming, interventions or supports, reclassification, monitoring, re-entry into EL
programming, if applicable.
4) Does your region follow a protocol regarding manual or automatic ELP classification?
If so, please describe. What is the preferred method for your region, and why?
5) Can you generalize any characteristics or trends that differ between those students who
exit at ELP 5 and ELP 6?
6) What is different about students who take longer to reclassify? Is there a tradeoff
between keeping students in EL programming for a longer period of time, versus
reclassifying a student as a former EL?
7) How cohesive is programming across schools and districts? Are there broad ranges of
policies and/or practices?
8) How prescriptive is DPI when providing guidelines for EL education?
9) How integrated are ELs in the general school population?
10) Can you comment on any differences between EL education in public non-charter
compared to public charter schools?
11) Can you comment on your experience with ELs and private school education?
12) How does student transience impact EL education?
13) What systems or policies would you like to see changed that relate to EL education?
22
Appendix D: Student Characteristic Breakdowns by Reclassification Group
Each of the following figures provides a snapshot of student outcomes by categorized group.
Figure 5: Breakdown of Gender in 5th Grade by Group
Source: Authors’ Analysis, DPI data 2007-2016
Figure 6: Breakdown of Home Language in 5th Grade by Group
Source: Authors’ Analysis, DPI data 2007-2016
49.9
39
55.1
45.1
50.1
61
44.9
54.9
0% 20% 40% 60% 80% 100%
Out-Over
Out-Under
In-Over
In-Under
Female Male
58.9
68.5
62.3
62.9
23.9
13.7
21.2
20.6
17.2
17.8
16.5
16.5
0% 20% 40% 60% 80% 100%
Out-Over
Out-Under
In-Over
In-Under
Spanish Hmong Others
23
Figure 7: Breakdown of School Locale Code in 5th Grade by Group
Source: Authors’ Analysis, DPI data 2007-2016
Figure 8: Breakdown of FRL Eligibility in 5th Grade by Group
Source: Authors’ Analysis, DPI data 2007-2016
58.2
41.1
48.2
55.3
24.1
24.7
30.1
24.2
11.2
28.1
12.6
12.6
6.4
6.2
8.9
7.5
0% 20% 40% 60% 80% 100%
Out-Over
Out-Under
In-Over
In-Under
City Suburb Town Rural
78.4
67.1
77.8
81.1
23.1
32.5
22.8
14.5
0% 20% 40% 60% 80% 100%
Out-Over
Out-
Under
In-Over
In-Under
FRL-Eligible Not FRL-Eligible
24
Figure 9: Average WSAS Reading Score Percentiles in 4th and 8th Grade by Group
Source: Authors’ Analysis, DPI data 2007-2016 Figure 10: Average WSAS Math Score Percentiles in 4th and 8th Grade by Group
Source: Authors’ Analysis, DPI data 2007-2016
0%
10%
20%
30%
40%
50%
60%
Out-Over In-Over Out-Under In-Under
Perc
entile
4th Grade 8th grade
0%
10%
20%
30%
40%
50%
60%
Out-Over In-Over Out-Under In-Under
Perc
entile
4th Grade 8th grade
25
Appendix E: Limitations and Assumptions
Assumptions Our analysis of the differences between the four groups of 5th grade EL students is confined by
the available data. State law does not require private schools to report their students’ performance
on standardized testing or to offer EL programming to the bilingual student population, so we
cannot consider how students in these schools are served in programs that support their English
language acquisition. This is also an issue for mobile students; there are gaps in the data for
students who were at some point classified as an EL, transferred to a private school or out-of-
state, and transferred back to a public school still classified as an EL.
Our data was also limited in its information about students’ primary languages. The data
collected identifies students in three language-based categories: Spanish-speaking, Hmong-
speaking, and “other.” This is limiting because it prevents us from considering language-based
differences between the 3,277 students in the “other” language category.
We did not have access to data that could account for diversity of the many EL programs in
Wisconsin schools. We did use data on the type of school each EL student attended for each year
he/she was tested, which included breakdowns for school type. These breakdowns included
designations for traditional public schools and two types of public charter schools: district and
non-district, which we use as a category “charter schools.” In our sample, 1,065 students attend
charter schools. Because the purpose of Wisconsin’s charter school policies is to allow for
schools that serve as alternatives to traditional public schools, these schools can operate
differently from traditional schools in many ways. This creates enormous potential for
heterogeneity in the structure, methods and curriculum in schools. Outcomes may vary between
these schools based on some of these major differences, although we were unable to consider
this in our analysis because we didn’t have data about the nature of these charter schools’
practices or operations.
Our data set was missing student ACCESS scores for about 300 students, which we are assuming
reflect random mechanical or reporting errors and not a systematic trend about those students. In
our analysis, we dropped observations for these students.
Finally, our qualitative information from practitioner interviews is inherently limited by scale.
The sample of people we interviewed is a small portion of program staff from Wisconsin’s 53
districts with BLBC programs.
Limitations Because of some of the limitations in our available data, we made a few key assumptions in the
design of our analysis. Some are basic structural assumptions that we make about the validity of
particular indicators of student characteristics, while others are specific assumptions about our
data and model.
1) The recent changes in standardized assessments administered to students in
Wisconsin make performance scores not directly comparable. To account for
26
this, we assumed that students’ performance percentiles relative to their peers
on a given exam are a valid way to compare achievement on different exams
with different scoring systems.
2) We analyzed outcome data among only students who reclassify between grades
5 through 8. Therefore, we make the assumption that outcomes for students
exiting in these grades are generalizable to those who exit in other grades. For
example, we assume that student and school demographics have the same effect
on students who exit in 5th grade as those who exit in 9th grade.
3) We assumed that there were no over-time changes in EL students’ eligibility
for the Free and Reduced Lunch program, which we use as a proxy for
socioeconomic status. We also acknowledge that using FRL status as a proxy
for poverty is flawed due to the over-inclusion of many students in federal
“community expansion” programs and other changing FRL-status criteria.
Regardless, many reporting entities and policymakers continue to use this
imperfect substitute (Chingos 2016).
4) We assumed that there were no changes in self-identified gender among EL
students in our data.
5) Because of limitations in our data, we treated all EL students from non-Spanish
and non-Hmong households in the “other” language category, which implies
that they are a homogenous group.
6) As mentioned in the limitations, we did not have data about differences in EL
program structure, methods, etc., and therefore needed to assume that all EL
programs in Wisconsin schools had the same effect on students’ future test
scores.
7) We assumed that charter schools are homogeneous in their EL student outcomes
for the sake of our comparison to traditional public schools.
8) We dropped approximately 350 students (~219 in 5th, ~139 in 4th) who lacked
an ACCESS score in our data, making the assumption that there is not
something systematically different about these students and that lack of a score
is random.
9) Finally, we include former EL students, who exited EL programming before
5th grade, in our discussion of overall ELs. Various DPI metrics include or
exclude former ELs, depending on the measure. Including former ELs skews
the performance of the overall EL population, as this subgroup outperforms
current EL students. At a given point, former ELs comprise approximately 15
percent of the total EL population in Wisconsin (NCELA 2016).
27
Appendix F: Results by Student Characteristic Interactions
The following table provides data regarding statistically significant interactions for student
subgroups and individual or school level characteristics.
Table 5: Significant Results for Students by Group, Interacted with Student and School Characteristics
Overall Out-
Over
Out-
Under
In-
Over
Language Groups
Math
Spanish - ↓ 2.0 - ↓ 2.4
Hmong ↓ 3.9 - - -
Other ↑ 1.7 - - ↑ 4.2
Reading
Spanish - - - -
Hmong ↓ 3.1 - - -
Other ↑ 2.4 - - -
Free/Reduced Lunch Eligibility Math - - - -
Reading ↓ 1.2 - - -
Gender (Female) Math - ↓ 1.8 - -
Reading ↑ 3.4 - - -
Learning Disability Math ↓5.5 - ↑ 16.7† ↑ 7.4†
Reading ↓ 5.7 - - ↑ 6.7†
Chronically Absent Math ↓ 3.2 - - -
Reading ↓ 3.8 - - -
In-Over in 4th Grade Math ↑ 2.8 ↓ 2.7 ↓ 8.6† -
Reading ↑ 2.3 ↓ 3.3 - -
In EL Programming Under 3
Years
Math ↑ 2.2 - - ↑ 2.6
Reading ↑ 2.5 - - ↑ 2.8
Census Locale Code
Math
City - - - ↓ 3.0
Suburb - - - -
Town - - - -
Rural - - - -
Reading
City - - - -
Suburb - - - ↑ 2.4
Town ↑ 3.3 - - -
Rural - - - -
Source: Authors’ Analysis, DPI data 2007-2016
† Sample under n=100
28
Appendix G: Data Preparation
This appendix details the steps taken to convert the raw data sets provided to us by DPI into our
analytical sample, the data set used to perform all of our statistical analyses.
We started with the data set containing demographic variables and ACCESS test scores for every
EL student in Wisconsin between 2007 and 2016. We merged the other data sets provided by
DPI—those including disciplinary, standardized test score, and school data—in with this base
ACCESS score data set, dropping the small number of observations (nine total) for which there
were duplicate combinations of student ID and school year. Unmatched data was also dropped
after each subsequent merge.
Merging the standardized test score data to the ACCESS test score data allowed us to observe
students beyond their time in EL programming because students may take standardized tests
every year between 3rd and 8th grade but take ACCESS tests only while in EL programming.
However, because demographic data was stored with ACCESS data in the files provided to us,
demographic information was missing in our data for students after they exited programming.
To address this, we carried forward and backward what information we did have on students
while they were in EL programming, essentially copying what we did observe about them when
they were in EL programming into the years when they were out of EL programming. We felt
this was a small assumption for some characteristics, such as student language or race/ethnicity,
which should be constant over time. Likewise, though a student’s gender may change, we
consider the likelihood that it did small enough as to not pose a significant challenge to the
validity of our assumption that it did not. More concerning was the necessity to carry forward
more likely time-variant characteristics, such as FRL eligibility and school code. This means that
we assume that students who were FRL-eligible when observed in EL programming remain so
throughout their academic career and that students do not change schools after they exit out of
programming. Again, we do not expect violations of these assumptions to be large enough to
significantly affect our results, but they do prevent us from being able to explore certain factors
related to student academic success, such as mobility.
After merging all of the data sets and filling in missing data, we dropped students that we didn’t
observe from 5th grade to 8th grade, reducing our analytical sample to 24,658 students from the
original 105,964. We also dropped the small populations of students who still appeared to be in
programming but were missing ACCESS scores in 4th and 5th grade (139 and 219 students,
respectively). We then moved relevant 4th and 8th grade test data into the 5th grade row for each
observation and dropped all observations that weren’t in 5th grade or that didn’t have a
standardized test score in 4th grade (4,435 students), converting our student panel data into a
cross section of 5th grade.
Finally, we dropped a few final duplicate students and unmatched results from our data merges,
bringing our final analytical sample to 19,792 students.
29
Appendix H: Regression Output
Table 6: Change in 8th Grade Math and Reading Score Percentiles by Student Group, Controlling for 5th Grade and 4th Grade Baseline Score Percentiles
5th Grade
Math
4th Grade
Math
5th Grade
Reading
4th Grade
Reading
Out-Over 0.0677*** 0.0959*** 0.0784*** 0.106***
(0.00468) (0.00479) (0.00493) (0.00496)
Out-Under 0.0397** 0.0602*** 0.0465** 0.0570***
(0.0140) (0.0161) (0.0146) (0.0157)
In-Over 0.0354*** 0.0611*** 0.0519*** 0.0681***
(0.00537) (0.00572) (0.00546) (0.00553)
Former EL 0.102*** 0.116*** 0.0856*** 0.0993***
(0.00414) (0.00435) (0.00447) (0.00467)
5th Grade Math 0.651***
(0.00618)
4th Grade Math 0.589***
(0.00643)
5th Grade Reading 0.667***
(0.00702)
4th Grade Reading 0.634***
(0.00753)
Constant 0.123*** 0.139*** 0.121*** 0.138***
(0.00332) (0.00351) (0.00306) (0.00327)
Observations 19311 19415 19136 19420
R2 0.557 0.508 0.561 0.525 Standard errors in parentheses
Year fixed effects included in model but omitted from table for clarity. * p < 0.05, ** p < 0.01, *** p < 0.001
Table 7: Change in 8th Grade Math Score Percentile by Student Group, Comparison of Three Models
4th Grade
Math
Student
Characteristics
Full Model
Out-Over 0.0959*** 0.0860*** 0.0885***
(0.00479) (0.00466) (0.00511)
Out-Under 0.0602*** 0.0564*** 0.0539**
(0.0161) (0.0162) (0.0164)
In-Over 0.0611*** 0.0536*** 0.0572***
(0.00572) (0.00553) (0.00551)
Student Controls No Yes Yes
School Clustered
SE
No No Yes
District FE No No Yes
Observations 19415 19415 19415
R2 0.508 0.544 0.576 Standard errors in parentheses * p < 0.05, ** p < 0.01, *** p < 0.001
30
Table 8: Change in 8th Grade Reading Score Percentile by Student Group, Comparison of Three Models
4th Grade
Reading
Student
Characteristics
Full Model
Out-Over 0.106*** 0.0972*** 0.0960***
(0.00496) (0.00485) (0.00511)
Out-Under 0.0570*** 0.0558*** 0.0548***
(0.0157) (0.0157) (0.0156)
In-Over 0.0681*** 0.0584*** 0.0635***
(0.00553) (0.00543) (0.00557)
Student Controls No Yes Yes
School Clustered
SE
No No Yes
District FE No No Yes
Observations 19420 19420 19420
R2 0.525 0.549 0.573 Standard errors in parentheses * p < 0.05, ** p < 0.01, *** p < 0.001
31
Table 9: Change in 8th Grade Math Score Percentile by Student Group and Language
Spanish Hmong Other
Language
Out-Over 0.103*** 0.0869*** 0.0867***
(0.00770) (0.00598) (0.00551)
Out-Under 0.0415 0.0603*** 0.0559**
(0.0298) (0.0166) (0.0179)
In-Over 0.0740*** 0.0578*** 0.0522***
(0.00901) (0.00617) (0.00584)
Spanish -0.00354
(0.00824)
Out-Over # Spanish -0.0203*
(0.00983)
Out-Under # Spanish 0.0208
(0.0333)
In-Over # Spanish -0.0242*
(0.0109)
Hmong -0.0393***
(0.00796)
Out-Over # Hmong 0.0113
(0.0116)
Out-Under # Hmong -0.0404
(0.0491)
In-Over # Hmong 0.00119
(0.0128)
Other Language 0.0172*
(0.00863)
Out-Over #
Other Language
0.0225
(0.0121)
Out-Under #
Other Language
0.00294
(0.0407)
In-Over #
Other Language
0.0418**
(0.0148)
Student Controls Yes Yes Yes
School Clustered SE Yes Yes Yes
District FE Yes Yes Yes
Observations 19415 19415 19415
R2 0.576 0.575 0.575 Standard errors in parentheses * p < 0.05, ** p < 0.01, *** p < 0.001
32
Table 10: Change in 8th Grade Reading Score Percentile by Student Group and Language
Spanish Hmong Other
Language
Out-Over 0.106*** 0.0934*** 0.0968***
(0.00768) (0.00606) (0.00542)
Out-Under 0.0613* 0.0598*** 0.0511**
(0.0273) (0.0170) (0.0172)
In-Over 0.0698*** 0.0665*** 0.0607***
(0.00855) (0.00630) (0.00604)
Spanish -0.0131
(0.00852)
Out-Over # Spanish -0.0137
(0.0101)
Out-Under # Spanish -0.00699
(0.0335)
In-Over # Spanish -0.00782
(0.0106)
Hmong -0.0313***
(0.00846)
Out-Over # Hmong 0.0158
(0.0113)
Out-Under # Hmong -0.0268
(0.0381)
In-Over # Hmong -0.00907
(0.0122)
Other Language 0.0237**
(0.00899)
Out-Over # Other
Language
0.00332
(0.0127)
Out-Under # Other
Language
0.0311
(0.0422)
In-Over # Other
Language
0.0248
(0.0147)
Student Controls Yes Yes Yes
School Clustered SE Yes Yes Yes
District FE Yes Yes Yes
Observations 19420 19420 19420
R2 0.571 0.571 0.571 Standard errors in parentheses * p < 0.05, ** p < 0.01, *** p < 0.001
33
Figure 11: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Language
Source: Authors’ Analysis, DPI Data 2007-2016
34
Table 11: Change in 8th Grade Math and Reading Score Percentiles by Student Group and FRL Eligibility
FRL Math FRL Reading
Out-Over 0.0975*** 0.102***
(0.00945) (0.00992)
Out-Under 0.0386 0.0670*
(0.0214) (0.0278)
In-Over 0.0806*** 0.0801***
(0.0127) (0.0121)
FRL -0.00411 -0.0115*
(0.00599) (0.00542)
Out-Over # FRL -0.00778 -0.00542
(0.0105) (0.0106)
Out-Under # FRL 0.0292 -0.0139
(0.0290) (0.0342)
In-Over # FRL -0.0266 -0.0190
(0.0141) (0.0137)
Student Controls Yes Yes
School Clustered
SE
Yes Yes
District FE Yes Yes
Observations 19415 19420
R2 0.576 0.571 Standard errors in parentheses * p < 0.05, ** p < 0.01, *** p < 0.001
35
Figure 12: Change in 8th Grade Math and Reading WSAS Score Percentiles by Group and FRL Eligibility
Source: Authors’ Analysis, DPI Data 2007-2016
Table 12: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Gender
Female Math Female Reading
Out-Over 0.0987*** 0.0935***
(0.00709) (0.00673)
Out-Under 0.0559** 0.0463*
(0.0215) (0.0191)
In-Over 0.0686*** 0.0582***
(0.00767) (0.00831)
Female 0.00397 0.0336***
(0.00326) (0.00300)
Out-Over # Female -0.0183* 0.00810
(0.00826) (0.00946)
Out-Under # Female -0.000371 0.0240
(0.0316) (0.0277)
In-Over # Female -0.0192 0.0133
(0.0101) (0.0105)
Student Controls Yes Yes
School Clustered SE Yes Yes
District FE Yes Yes
Observations 19415 19420
R2 0.575 0.572 Standard errors in parentheses * p < 0.05, ** p < 0.01, *** p < 0.001
36
Figure 13: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Gender
Source: Authors’ Analysis, DPI Data 2007-2016
37
Table 13: Change in 8th Grade Math Score Percentile by Student Group and School Locale
City School Suburban School Town School Rural School
Out-Over 0.0971*** 0.0852*** 0.0899*** 0.0898***
(0.00805) (0.00555) (0.00537) (0.00530)
Out-Under 0.0559** 0.0574** 0.0610** 0.0506**
(0.0196) (0.0201) (0.0196) (0.0165)
In-Over 0.0737*** 0.0515*** 0.0552*** 0.0586***
(0.00809) (0.00644) (0.00578) (0.00580)
City -0.00721
(0.00874)
Out-Over # City -0.0126
(0.00984)
Out-Under # City 0.00317
(0.0343)
In-Over # City -0.0300**
(0.0110)
Suburb 0.00899
(0.00935)
Out-Over # Suburb 0.0188
(0.0116)
Out-Under # Suburb -0.00789
(0.0320)
In-Over # Suburb 0.0242
(0.0126)
Town 0.0232
(0.0134)
Out-Over # Town -0.00311
(0.0159)
Out-Under # Town -0.0169
(0.0348)
In-Over # Town 0.0240
(0.0173)
Rural -0.0137
(0.00909)
Out-Over # Rural 0.000429
(0.0164)
Out-Under # Rural 0.0831
(0.0889)
In-Over # Rural -0.00438
(0.0184)
Student Controls Yes Yes Yes Yes
School Clustered SE Yes Yes Yes Yes
District FE Yes Yes Yes Yes
Observations 19415 19415 19415 19415 Standard errors in parentheses * p < 0.05, ** p < 0.01, *** p < 0.001
38
Table 14: Change in 8th Grade Reading Score Percentile by Student Group and School Locale
City School Suburban School Town School Rural School
Out-Over 0.0944*** 0.0984*** 0.0972*** 0.0975***
(0.00742) (0.00556) (0.00549) (0.00526)
Out-Under 0.0567* 0.0595*** 0.0514** 0.0576***
(0.0223) (0.0165) (0.0179) (0.0162)
In-Over 0.0708*** 0.0578*** 0.0671*** 0.0639***
(0.00749) (0.00678) (0.00587) (0.00582)
City -0.0122
(0.00874)
Out-Over # City 0.00447
(0.00928)
Out-Under # City -0.00171
(0.0296)
In-Over # City -0.0135
(0.0106)
Suburb 0.00222
(0.00916)
Out-Over # Suburb -0.00410
(0.0113)
Out-Under # Suburb -0.0112
(0.0407)
In-Over # Suburb 0.0235*
(0.0109)
Town 0.0331*
(0.0156)
Out-Over # Town 0.000633
(0.0130)
Out-Under # Town 0.0136
(0.0353)
In-Over # Town -0.0187
(0.0173)
Rural 0.0104
(0.00978)
Out-Over # Rural -0.000708
(0.0152)
Out-Under # Rural -0.0197
(0.0558)
In-Over # Rural 0.00935
(0.0196)
Student Controls Yes Yes Yes Yes
School Clustered SE Yes Yes Yes Yes
District FE Yes Yes Yes Yes
Observations 19420 19420 19420 19420 Standard errors in parentheses * p < 0.05, ** p < 0.01, *** p < 0.001
39
Figure 14: Change in 8th Grade Math and Reading Score Percentiles by Student Group and School Locale
Source: Authors’ Analysis, DPI Data 2007-2016
40
Table 15: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Low EL Student Population
Low EL Math Low EL
Reading
Out-Over 0.0882*** 0.0962***
(0.00547) (0.00540)
Out-Under 0.0421** 0.0591***
(0.0160) (0.0173)
In-Over 0.0573*** 0.0650***
(0.00588) (0.00591)
Low EL
Concentration
-0.00962 -0.00389
(0.00748) (0.00690)
Out-Over # Low EL
Concentration
0.0134 0.00845
(0.0132) (0.0129)
Out-Under # Low EL
Concentration
0.0770 -0.0162
(0.0510) (0.0382)
In-Over # Low EL
Concentration
0.00714 -0.00371
(0.0178) (0.0166)
Student Controls Yes Yes
School Clustered SE Yes Yes
District FE Yes Yes
Observations 19415 19420
R2 0.575 0.571 Standard errors in parentheses * p < 0.05, ** p < 0.01, *** p < 0.001
Figure 15: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Low EL Student Population
Source: Authors’ Analysis, DPI Data 2007-2016
41
Table 16: Change in 8th Grade Math and Reading Score Percentiles by Student Group and 4th Grade In-Over Status
4th Grade In-
Over Math
4th Grade In-
Over Reading
Out-Over 0.0907*** 0.102***
(0.00589) (0.00604)
Out-Under 0.0642*** 0.0585***
(0.0182) (0.0166)
In-Over 0.0585*** 0.0632***
(0.00639) (0.00661)
In-Over 4th 0.0280** 0.0231*
(0.00891) (0.00930)
Out-Over # In-Over
4th
-0.0267* -0.0326**
(0.0120) (0.0121)
Out-Under # In-Over
4th
-0.0861* -0.0282
(0.0438) (0.0446)
In-Over # In-Over 4th -0.0236 -0.0138
(0.0134) (0.0134)
Student Controls Yes Yes
School Clustered SE Yes Yes
District FE Yes Yes
Observations 19415 19420
R2 0.575 0.571 Standard errors in parentheses * p < 0.05, ** p < 0.01, *** p < 0.001
Figure 16: Change in 8th Grade Math and Reading Score Percentiles by Student Group and 4th Grade In-Over Status
Source: Authors’ Analysis, DPI Data 2007-2016
42
Table 17: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Chronic Absenteeism in or Before 5th Grade
Absentee Math Absentee
Reading
Out-Over 0.0903*** 0.0969***
(0.00519) (0.00517)
Out-Under 0.0551*** 0.0568***
(0.0166) (0.0158)
In-Over 0.0590*** 0.0647***
(0.00553) (0.00562)
Absentee -0.0324*** -0.0377***
(0.00866) (0.00888)
Out-Over # Absentee -0.0198 0.00995
(0.0237) (0.0266)
Out-Under #
Absentee
0.0349 -0.0207
(0.0211) (0.0198)
In-Over # Absentee -0.0358 -0.00744
(0.0411) (0.0384)
Student Controls Yes Yes
School Clustered SE Yes Yes
District FE Yes Yes
Observations 19415 19420
R2 0.575 0.571 Standard errors in parentheses * p < 0.05, ** p < 0.01, *** p < 0.001
Figure 17: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Chronic Absenteeism
Source: Authors’ Analysis, DPI Data 2007-2016
43
Table 18: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Presence of a Learning Disability
LD Math LD Reading
Out-Over 0.0898*** 0.0976***
(0.00518) (0.00526)
Out-Under 0.0514** 0.0521***
(0.0158) (0.0154)
In-Over 0.0565*** 0.0627***
(0.00558) (0.00574)
Learning Disability -0.0554*** -0.0565***
(0.00418) (0.00414)
Out-Over # Learning
Disability
0.0247 -0.0112
(0.0270) (0.0244)
Out-Under #
Learning Disability
0.165* 0.160
(0.0780) (0.0835)
In-Over # Learning
Disability
0.0738* 0.0666*
(0.0332) (0.0290)
Student Controls Yes Yes
School Clustered SE Yes Yes
District FE Yes Yes
Observations 19415 19420
R2 0.575 0.571 Standard errors in parentheses * p < 0.05, ** p < 0.01, *** p < 0.001
Figure 18: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Learning Disability
Source: Authors’ Analysis, DPI Data 2007-2016
44
Table 19: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Participation in EL Programming of Fewer Than Three Years
EL Under 3
Math
EL Under 3
Reading
Out-Over 0.0868*** 0.0954***
(0.00534) (0.00514)
Out-Under 0.0635** 0.0602**
(0.0202) (0.0185)
In-Over 0.0519*** 0.0575***
(0.00634) (0.00662)
EL Under 3 Years 0.0216** 0.0246***
(0.00689) (0.00714)
Out-Over # EL
Under 3 Years
0.0219 0.0141
(0.0114) (0.0128)
Out-Under # EL
Under 3 Years
-0.0205 -0.00616
(0.0413) (0.0321)
In-Over # EL Under
3 Years
0.0259* 0.0278*
(0.0128) (0.0129)
Student Controls Yes Yes
School Clustered SE Yes Yes
District FE Yes Yes
Observations 19415 19420
R2 0.575 0.571 Standard errors in parentheses * p < 0.05, ** p < 0.01, *** p < 0.001
Figure 19: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Participation in EL Programming for Fewer Than Three Years
Source: Authors’ Analysis, DPI Data 2007-2016
45
References
American Civil Liberties Union. Your Right to Equality in Education. Accessed April 1, 2017.
https://www.aclu.org/other/your-right-equality-education.
American Institutes for Research. 2012. National Evaluation of Title III Implementation -
Report on State and Local Implementation. Accessed April 1, 2017.
https://www2.ed.gov/rschstat/eval/title-iii/state-local-implementation-report.pdf.
Babal, Chris, Shiyao Cao, Angela Filer, Isaac Hedtke, and Keo Lo. 2015. "Analysis of English
Language Achievement among Wisconsin English Learners." May. Accessed April 1,
2017. http://www.lafollette.wisc.edu/research/publications/analysis-of-english-language-
achievement-among-wisconsin-english-language-learners.
Burgos, Laurie, interview by Richelle Andrae. 2017. April 3.
Chingos, Matthew. 2016. “No more free lunch for education policymakers and researchers.”
Brookings Institute, June 30.
Duncan, Greg J., and Richard J. Murnane. 2011. Whither Opportunity? Rising Inequality,
Schools, and Children's Life Chances. New York City: Russell Sage Foundation.
Harris, Elizabeth A. 2015. "Dual-Language Programs Are on the Rise, Even for Native English
Speakers." The New York Times, October 8.
Hosford, Sonya D., Christina Mokhtar, and Carrie Sampson. 2013. "Nevada's English Learner
Population: A Review of Enrollment, Outcomes, and Opportunities." Lincy Institute,
University of Nevada, Las Vegas.
Johnson, Jacqueline S. 1989. "Critical period effects in second language learning: The
influence of maturational state on the acquisition of English as a second language."
Cognitive Psychology 21(1): 60-99.
Kuncel, Nathan R., and Sarah A. Hezlett. 2007. "Standardized Tests Predict Graduate Students'
Success." Science 351(5815): 1080-1081.
Ladson-Billings, Gloria. 2016. "From the Achievement Gap to the Education Debt:
Understanding Achievement in U.S. Schools." Educational Researcher 35(7): 3-12.
Madison Metropolitan School District. 2014. Bilingual Program Model Criteria to be Met in
Order to Qualify for Bilingual-Bicultural Aid. Accessed April 1, 2017.
https://multilingual.madison.k12.wi.us/files/esl/dpi-guidance-on-bilingual-bicultural-
programs.pdf.
Madison Metropolitan. 2017. "Dual Language Immersion." Accessed April 3, 2017.
https://multilingual.madison.k12.wi.us/dual-language-immersion.
46
McKeon, Denise. 2005. Research Talking Points on English Learners. June. Accessed April 1,
2017. http://www.nea.org/home/13598.htm.
National Center for Education Statistics. 2015. "School Composition and the Black-White
Achievement Gap." Accessed April 1, 2017.
https://nces.ed.gov/nationsreportcard/subject/studies/pdf/school_composition_and_the_bw
_achievement_gap_2015.pdf.
National Center for Education Statistics. 2016. "English Learners in Public Schools." May.
Accessed April 1, 2017. https://nces.ed.gov/programs/coe/indicator_cgf.asp.
National Clearinghouse for English Language Acquisition. "Title III State Profiles -
Wisconsin." Accessed April 2, 2017. http://ncela.ed.gov/t3sis/Wisconsin.php.
Ovando, Carlos J. 2003. "Bilingual Education in the United States: Historical Development
and Current Issues." The Journal of the National Association for Bilingual Education
27(1): 1-24.
Shanebrook-Smith, Sarah, interview by Richelle Andrae. 2017. March 16.
U.S. Department of Education. 2016. Non-Regulatory Guidance: English Learners and Title
III of the Elementary and Secondary Education Act (ESEA), as amended by the Every
Student Succeeds Act (ESSA). Accessed April 1, 2017.
https://www2.ed.gov/policy/elsec/leg/essa/essatitleiiiguidenglishlearners92016.pdf.
U.S. Department of Education. 2016. "Race and National Origin Discrimination" Accessed
April 1, 2017. https://www2.ed.gov/about/offices/list/ocr/frontpage/faq/race-origin.html.
U.S. Department of Education. 2010. "Title III Accountability: Behind the Numbers." May.
Accessed April 1, 2017. https://www2.ed.gov/rschstat/eval/title-iii/behind-numbers.pdf.
WIDA Consortium. 2011. ACCESS for ELs, Interpretive Guide for Score Reports. Accessed
April 1, 2017.
https://www.wida.us/assessment/ACCESS/ScoreReports/ACCESS_Interpretive_Guide11.p
df
WIDA Consortium. 2014. Mission & WIDA Story. Accessed April 1, 2017.
https://www.wida.us/aboutus/mission.aspx.
WIDA Consortium. 2016. Changes are Coming to ACCESS for ELLS 2.0 Scores. Accessed
April 30, 2017.
https://www.wida.us/Assessment/ACCESS%202.0/documents/WIDA_flyer_standards_sett
ing.pdf.
47
Winkleby, Marilyn A., Darius E. Jatulis, Erica Frank, and Stephen P Fortmann. 1992.
"Socioeconomic status and health: how education, income, and occupation contribute to
risk factors for cardiovascular disease." American Journal of Public Health 816-820.
Wisconsin Department of Public Instruction. 2009. "ESEA Information Update - Initial
Identification and Placement of English Learners." September 25. Accessed April 1, 2017.
https://dpi.wi.gov/sites/default/files/imce/esea/pdf/bul_0701.pdf.
Wisconsin Department of Public Instruction. 2011. "ESEA Information Update." December
15. Accessed April 1, 2017.
https://dpi.wi.gov/sites/default/files/imce/esea/pdf/bul_0702.pdf.
Wisconsin Department of Public Instruction. 2015. Report on the Status of Bilingual-Bicultural
Education Programs in Wisconsin, Chapter 115, Subchapter VII, s. 115.996, Wis. Stats.
Content and Learning Team. December. Accessed April 1, 2017.
https://dpi.wi.gov/sites/default/files/imce/english-learners/pdf/2013-14-legislator-
report.pdf.
Wisconsin Department of Public Instruction. 2016. "Wisconsin Public Schools at a Glance."
May. Accessed April 1, 2017.
https://dpi.wi.gov/sites/default/files/imce/eis/pdf/schools_at_a_glance.pdf.
Wisconsin Department of Public Instruction. 2017a. "High School Completion Results."
Wisconsin Information System Data Dashboard. Accessed April 1, 2017.
http://wisedash.dpi.wi.gov/Dashboard/portalHome.jsp.
Wisconsin Department of Public Instruction. 2017b. "Report on the Status of Bilingual-
Bicultural Education Programs in Wisconsin, Chapter 115, Subchapter VII, s. 115.996,
Wis. Stats." Accessed April 2, 2017. https://dpi.wi.gov/sites/default/files/imce/english-
learners/pdf/2013-14-legislator-report.pdf.
Wisconsin Department of Public Instruction. 2017c. Accountability for English Learners (ELs).
Accessed April 1, 2017. https://dpi.wi.gov/accountability/ELamao.
Wisconsin Department of Public Instruction. 2017d. Badger Exam 3-8: Wisconsin's Smarter
Balanced Assessment. Accessed April 1, 2017.
https://dpi.wi.gov/assessment/historical/smarter.
Wisconsin Department of Public Instruction. 2017e. Bilingual/ESL Program. Accessed April
2, 2017. https://dpi.wi.gov/english-learners.
Wisconsin Department of Public Instruction. 2017f. Formula Allocation Spreadsheets.
Accessed May 1. https://dpi.wi.gov/esea/historical/funding/application#Formula Allocation
Spreadsheets
48
Wisconsin Department of Public Instruction. 2017g. Reclassification of English Learners.
Accessed April 25, 2017. https://dpi.wi.gov/english-learners/title-iii/reclassification
Wisconsin Department of Public Instruction. 2017h. Wisconsin Bilingual-Bicultural Programs.
Accessed April 3, 2017. https://dpi.wi.gov/english-learners/bilingual-bicultural
Wisconsin Department of Public Instruction. 2017i. Wisconsin Forward Exam. Accessed April
1, 2017. https://dpi.wi.gov/assessment/forward.
Wisconsin Department of Public Instruction. 2017j. Wisconsin Information System for
Education Data Dashboard. Accessed April 2, 2017. http://wisedash.dpi.wi.gov/.
Wisconsin Department of Public Instruction. 2017k. WKCE. Accessed April 1, 2017.
https://dpi.wi.gov/assessment/historical/wkce