academic achievement among english learners … achievement among english learners (els) in...

Academic Achievement Among English

Learners (ELs) in Wisconsin

An Analysis of ELs Based on 5th Grade

Reclassification Status and English Language

Proficiency Test Scores

Prepared for Carl Frederick,

Wisconsin Department of Public Instruction (DPI)

By

Richelle Andrae

Derek Field

Moira Lenox

Max Pardo

Workshop in Public Affairs

Spring 2017

©2017 Board of Regents of the University of Wisconsin System

All rights reserved.

For an online copy, see

http://www.lafollette.wisc.edu/outreach-public-service/workshops-in-public-affairs

[email protected]

The Robert M. La Follette School of Public Affairs is a teaching and research department

of the University of Wisconsin–Madison. The school takes no stand on policy issues;

opinions expressed in these pages reflect the views of the authors.

The University of Wisconsin–Madison is an equal opportunity and affirmative-action educator and employer.

We promote excellence through diversity in all programs.

Table of Contents

List of Tables ...................................................................................................................................v

List of Figures ................................................................................................................................ vi

Foreword ....................................................................................................................................... vii

Acknowledgments........................................................................................................................ viii

List of Abbreviations ..................................................................................................................... ix

Executive Summary .........................................................................................................................x

Discussion of English Language Learners in Wisconsin .................................................................1

Prior Research ..............................................................................................................................2

Research Questions ......................................................................................................................3

Data ..................................................................................................................................................3

Methods........................................................................................................................................6

Limitations and Assumptions ..........................................................................................................7

Findings............................................................................................................................................8

Research Question 1: Differences Between Student Groups .......................................................8

Research Question 2: Impact of Student and School Characteristics ..........................................9

Language ................................................................................................................................10

Free and Reduced Lunch Status .............................................................................................11

Gender ....................................................................................................................................11

Disability Status .....................................................................................................................11

Chronic Absenteeism .............................................................................................................11

Retention in Programming after Scoring a 5-5 in 4th Grade .................................................11

Low Number of Years in EL Programming ..........................................................................11

School Locale.........................................................................................................................12

Groups and Variable Interactions without Significant Findings............................................12

Discussion ......................................................................................................................................12

Recommendations ..........................................................................................................................13

Recommendations Related to Further Data Collection and Use ...............................................13

Recommendations Specific to Findings of Our Analysis ..........................................................14

Opportunities for Further Research ...............................................................................................15

Conclusion .....................................................................................................................................15

Appendix A: Relevant Literature ...................................................................................................17

Appendix B: 5th Grade as Inflection Point ....................................................................................19

Appendix C: Interview Protocol ....................................................................................................21

Appendix D: Student Characteristic Breakdowns by Reclassification Group...............................22

Appendix E: Limitations and Assumptions ...................................................................................25

Appendix F: Results by Student Characteristic Interactions .........................................................27

Appendix G: Data Preparation .......................................................................................................28

Appendix H: Regression Output ....................................................................................................29

References ......................................................................................................................................45

v

List of Tables

Table 1: Comparing Demographics of Students Exiting ................................................................................................. 4 Table 2: Categorization of 5th Grade Students .............................................................................................................. 5 Table 3: Literature Review on Reclassification Research ............................................................................................. 17 Table 4: Number of Students Exiting EL Programming by Grade, 2007-2016 ............................................................. 19 Table 5: Significant Results for Students by Group, Interacted with Student and School Characteristics ................... 27 Table 6: Change in 8th Grade Math and Reading Score Percentiles by Student Group, Controlling for 5th Grade and 4th Grade Baseline Score Percentiles........................................................................................................................... 29 Table 7: Change in 8th Grade Math Score Percentile by Student Group, Comparison of Three Models ..................... 29 Table 8: Change in 8th Grade Reading Score Percentile by Student Group, Comparison of Three Models ................. 30 Table 9: Change in 8th Grade Math Score Percentile by Student Group and Language ............................................. 31 Table 10: Change in 8th Grade Reading Score Percentile by Student Group and Language ....................................... 32 Table 11: Change in 8th Grade Math and Reading Score Percentiles by Student Group and FRL Eligibility ............... 34 Table 12: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Gender ......................... 35 Table 13: Change in 8th Grade Math Score Percentile by Student Group and School Locale ...................................... 37 Table 14: Change in 8th Grade Reading Score Percentile by Student Group and School Locale ................................. 38 Table 15: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Low EL Student Population.................................................................................................................................................................... 40 Table 16: Change in 8th Grade Math and Reading Score Percentiles by Student Group and 4th Grade In-Over Status ..................................................................................................................................................................................... 41 Table 17: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Chronic Absenteeism in or Before 5th Grade ..................................................................................................................................................... 42 Table 18: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Presence of a Learning Disability ...................................................................................................................................................................... 43 Table 19: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Participation in EL Programming of Fewer Than Three Years ................................................................................................................... 44

vi

List of Figures

Figure 1: Difference in Average 8th Grade and 4th Grade WSAS Score Percentiles by Group ......................................... 6 Figure 2: Difference in 8th Grade Math and Reading WSAS Percentile Scores by Group .............................................. 9 Figure 3: Results for ELs Overall by Characteristic ....................................................................................................... 10 Figure 4: Reclassified Students per Grade ................................................................................................................... 19 Figure 5: Breakdown of Gender in 5th Grade by Group ................................................................................................ 22 Figure 6: Breakdown of Home Language in 5th Grade by Group ................................................................................. 22 Figure 7: Breakdown of School Locale Code in 5th Grade by Group ............................................................................. 23 Figure 8: Breakdown of FRL Eligibility in 5th Grade by Group ...................................................................................... 23 Figure 9: Average WSAS Reading Score Percentiles in 4th and 8th Grade by Group ..................................................... 24 Figure 10: Average WSAS Math Score Percentiles in 4th and 8th Grade by Group ....................................................... 24 Figure 11: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Language .................... 33 Figure 12: Change in 8th Grade Math and Reading WSAS Score Percentiles by Group and FRL Eligibility ................. 35 Figure 13: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Gender ........................ 36 Figure 14: Change in 8th Grade Math and Reading Score Percentiles by Student Group and School Locale .............. 39 Figure 15: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Low EL Student Population.................................................................................................................................................................... 40 Figure 16: Change in 8th Grade Math and Reading Score Percentiles by Student Group and 4th Grade In-Over Status ..................................................................................................................................................................................... 41 Figure 17: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Chronic Absenteeism .. 42 Figure 18: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Learning Disability ...... 43 Figure 19: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Participation in EL Programming for Fewer Than Three Years .................................................................................................................. 44

vii

Foreword

This report is the result of collaboration between the Robert M. La Follette School of Public

Affairs at the University of Wisconsin–Madison and the Wisconsin Department of Public

Instruction (DPI), a state agency. The objective of this project is to provide graduate students at

the La Follette School the opportunity to improve their policy analysis skills while contributing

to the capacity of partner organizations.

The La Follette School provides students with a rigorous two-year graduate program leading to a

master’s degree in public affairs. Students study policy analysis and public management as well

as concentrated study in at least one policy area. The authors of this report all are in their final

semester of their degree program and are enrolled in the Public Affairs 869 Workshop in Public

Affairs at UW–Madison. Although studying policy analysis is important, there is no substitute

for engaging actively in applied policy analysis as a means of developing policy analysis skills.

The Public Affairs 869 Workshop gives graduate students that opportunity.

The DPI works to advance public education in Wisconsin and administers the systems that serve

students across school districts and programs. Wisconsin is viewed as a national leader in

English Language Learner programs, and the DPI has an ongoing focus on quality improvement,

including through research partnerships with UW–Madison. The DPI also has a strong history of

working with La Follette School students to perform rigorous research. This report includes an

analysis of administrative data, as well as interviews and other research, with the goal of helping

the DPI to improve services for students statewide.

I am grateful to the DPI for partnering with the La Follette School on this project. DPI staff have

been exceptionally generous with their time to support this project, including collaborating on

data analysis. The students have collectively contributed hundreds of hours to this project, and in

the process developed critical insights about state policies and programs. The La Follette School

is grateful for their efforts and hope that this report proves valuable for the DPI and the state of

Wisconsin to improve the outcomes of English Language Learners.

J. Michael Collins

Professor of Public Affairs

May 2017

Madison, Wisconsin

viii

Acknowledgments

We would like to express our gratitude to Carl Frederick, Audrey Lesondak, Justin Meyer, and

Jesse Roberts of DPI for their consistent and valuable feedback, support in developing nuanced

approaches to research, and passion for serving Wisconsin students. We would also like to thank

district administrators for providing on-the-ground insights to EL programming. Finally, we

extend our gratitude to our project advisor, Professor J. Michael Collins, for his ongoing

guidance, as well as Lisa Hildebrand for her editorial assistance.

ix

List of Abbreviations

ACCESS Assessing Comprehension and Communication in English State-to-State for

English Learners

AMAO Annual Measurable Achievement Objectives

BLBC Bilingual Bicultural

DPI Department of Public Instruction

EL English Learner

ELD English Language Development

ELP English Language Proficiency

ESSA Every Student Succeeds Act

ESEA Elementary and Secondary Education Act

FE Fixed Effects

FRL Free and Reduced Lunch

IDEA Individuals with Disabilities Education Act

LEP Limited English Proficient

NCLB No Child Left Behind

SE Standard Error

WIDA Originally: Wisconsin, Delaware, and Arkansas; Adjusted: World-class

Instructional Design; currently no acronym defined

WKCE Wisconsin Knowledge and Concepts Examination

WSAS Wisconsin Student Assessment System

x

Executive Summary

Many Wisconsin schools provide English learner (EL) students with targeted English language

(EL) programing to support their academic growth and positive educational outcomes. These

schools assess the English proficiency of EL students while they are still in EL programming to

help educators determine when the students are prepared to “exit” EL programming. The

Department of Public Instruction (DPI) would benefit from increased insight on how Wisconsin’s

EL programs can best support this historically underserved student population.

The purpose of this report is to better understand how “exiting” students out of EL programs affects

their future academic achievement. DPI provided nine years of student-level data, which includes

EL students’ demographics, school and district enrollment, Free and Reduced Lunch (FRL)

eligibility, disability status, standardized test scores, and English proficiency scores. We base our

analysis on two main questions: How do academic outcomes differ between students who exit EL

programming with various levels of English proficiency? What student or school characteristics

predict future academic performance?

To address the research questions, we compare the future performance on statewide standardized

tests of a sample of 5th grade EL students. We divide this sample of students into four groups,

determined by whether the students were exited out of or remained in EL programming after 5th

grade, and whether the students scored above or below the DPI-recommended guideline score on

the English proficiency assessment. We then use statistical modeling to determine whether English

proficiency level, exit status in 5th grade, or individual student-, school- or district-level

characteristics influence future academic performance.

We find a statistically significant relationship between future academic performance and the four

student groups. Our findings suggest that if a student has reached the recommended score on their

English proficiency assessment, it may be beneficial to exit a student rather than keep the student

in EL programming. We also find that student-level characteristics do matter, noting a non-uniform

relationship between future academic performance and the four student groups depending on

which student- and school-level characteristic is analyzed.

Based on these findings, we recommend that teachers and administrators continue to exercise

discretion when exiting students, but exit students at the recommended score whenever possible.

We also urge DPI to collect student-level data on EL programming to better understand the effects

of various intervention strategies.

1

Discussion of English Language Learners in Wisconsin

The U.S. Constitution requires that all children are provided with equal educational opportunity,

regardless of race, ethnicity, wealth, background, or citizenship status (ACLU). Current federal

and state policies seek to support equitable access to education for historically underserved

students such as racial minorities, low-income children, and English Learners (ELs). ELs are a

federally protected class under Title VI regulation, which outlaws any discrimination based on a

person’s limited English proficiency (U.S. Department of Education 2016). Achievement gaps

for the EL student population—as well as other student subgroups—are gaining attention from

both communities and policymakers, highlighting disparities in academic performance between

different learners (See Appendix I for more information on achievement gaps) (The Equity and

Excellence Committee 2013). The Department of Education encourages high standards for

educational equity, but the politics of educational access, standardized testing, and resource

allocation complicate its realization.

This report adds value to the discussion of programs and policies for the EL student population.

Approximately one in 10 U.S. students qualifies as an EL, and this subset of learners is growing

over time nationally, though currently leveling off in some states (NCES 2016). Our analysis

will focus on ELs in Wisconsin. Appendix I provides details regarding Wisconsin’s EL

population, including demographics and academic performance. Appendix I also gives an

overview—or snapshot—of EL policies and programs across the state. The overview details EL

experiences through the following processes:

Identification

Programming

Assessment and Accountability

Reclassification

Monitoring

This report also includes additional data about the funding of EL programming along with

relevant state and federal regulations. Two relevant areas of interest, assessment and

reclassification, addressed in this report are discussed in brief below.

Assessment Overview

EL students in Wisconsin generally take two kinds of performance assessments, each discussed

in detail throughout Appendix I. The first is the statewide standardized test in both reading and

math. The second is an EL-specific assessment that measures English Language Proficiency

(ELP). This exam is called the ACCESS test, which is administered to students each year that

they are identified as an EL. The ACCESS test is used to determine whether a student should be

exited from EL programming.

Reclassification Overview

DPI provides a guideline regarding when a student should be transitioned out of active EL status.

This guideline requires an ACCESS score of 5-5, meaning a 5 composite score and a 5 literacy

score, both on a 6-point scale. At this point, a student may be exited from programming, thus

2

considered a “reclassified” or “exited” student or a former EL; former ELs are still tracked and

monitored but no longer remain in EL programming. Regardless of DPI guidelines, teachers and

administrators may still exercise considerable discretion when deciding if and when to exit a

student from EL programming.

Our research questions add to the current body of evidence by comparing 5th grade students who

exit out of or remain in EL programming with different levels of English proficiency and

evaluating each component’s relationship with 8th grade academic achievement on standardized

assessments. The report is designed with Wisconsin’s state education agency, the Department of

Public Instruction (DPI), as the primary beneficiary, although the analysis may also be relevant

for administrators at the district level and authorities from other states.

Prior Research

A 2015 analysis completed by graduate students at the La Follette School of Public Affairs for

DPI investigated performance on EL assessments based on student characteristics (Babal et al.

2015). The report concluded that as the starting age of a student in EL programming increased,

the length of time in programming also increased. Student performance also varied by language

groups, with Spanish- and Hmong-speaking students—the two largest EL groups in

Wisconsin—demonstrating slower English language acquisition rates compared with other

language groups. Additionally, lower-income and disabled students fared worse on English

proficiency assessments compared with other students. Finally, that analysis showed that when

students started with a higher level of English proficiency, they reached full English proficiency

more rapidly. Key recommendations for policy included providing assessment subgroup

performance data to school districts and supporting analysis of EL subgroups.

Building on the research of the previous report, this analysis aims to analyze the academic

performance of these students after exiting EL programming. A comprehensive review of other

relevant literature on the topic of English language acquisition, along with reclassification, is

included in Appendix A.

Prior literature has addressed this topic in two primary directions, with mixed findings. Carlson

& Knowles (2016) and Kim (2011) both found that students who spend more time as English

learners have lower eventual testing and graduation outcomes than their peers who spend less

time in EL programs. However, Robinson-Cimpian & Thompson (1989) and Fernandez (2013)

both find conflicting results. Their research suggests that when states lower the testing standards

for removing proficient students from EL programming, students in districts with lower

reclassification thresholds who score between the old and new cutoffs have worse outcomes than

their peers in districts that didn’t lower reclassification standards. We suspect that the

discrepancy between these groups of research may be due to these studies’ assumptions that

changing the reclassification thresholds or the time spent in EL programing causally affects

future outcomes. This may not be equally true of more- and less-proficient students, or findings

from these groups of studies may not be generalizable outside of their individual contexts, so we

therefore seek to investigate this issue and landscape of EL student testing in Wisconsin.

3

Research Questions

Based on input from stakeholders, particularly the priorities expressed by DPI, we focus on two

key research questions aiming to evaluate former EL students after exiting EL programming.

These are:

1) How do EL students’ standardized test scores in 8th grade differ across

students based on based their English proficiency level and EL status in 5th

grade?

2) Based on English proficiency level and EL status in 5th grade, are there any

student- or school-level characteristics that can predict relative differences in

8th grade standardized test scores?

The goal of the first question is to determine whether teachers and administrators appropriately

reclassify students in 5th grade. While educators may reclassify EL students at a time that best

supports their future academic success, it is possible that they may exit students too late or too

early. The goal of the second research question is to identify any student- or school-level

characteristics that may impact students’ educational outcomes. In order to study educational

outcomes, we analyze differences in 8th grade standardized test scores based on whether a

student exited or remained in programming in 5th grade (see the Methods section for further

explanation of our decision to study the relationship between 5th grade characteristics and 8th

grade outcomes). We expand our analysis to include the outcomes of various student subgroups

across characteristics such as home language, gender, eligibility for participating in the federal

Free and Reduced Lunch program, and school type.

To address the research questions, we adopted a mixed-methods approach by reviewing prior

studies, conducting interviews with district administrators involved in EL programming, and

performing a quantitative analysis on student data. This approach provides a holistic perspective

on EL experiences by combining narrative with data. Information learned from interviews

informs how state policies and federal requirements impact on-the-ground administrators as well

as students.

Data

We analyzed four DPI data sets that contained English proficiency scores, statewide standardized

testing scores, disciplinary actions, and school-level data covering the time period 2007 through

2016. We combined all four data sets and removed any students who were observed for less than

four years. In order to analyze student outcomes in 8th grade and compare them against 5th

grade, we need to observe in the 5th and 8th grade. We therefore removed approximately 80,000

students from our sample to facilitate our analysis. The implications of excluding students who

were not consistently in the data are discussed later in the report.

We then converted the panel data to a cross section of 5th grade students. The analysis focuses

on 24,416 5th grade EL students who remained in the data by 8th grade and could be matched

over time.

4

Overall, the demographic characteristics of students exiting EL programming are similar

between grades and especially between 4th and 5th grade, when the largest number of EL

students exit programming. Third grade appears slightly anomalous, though not necessarily in

ways that would be unexpected—we would expect students who exit EL programming early to

be high academic achievers. The higher proportion of non-FRL students among students who

exited EL programming in 3rd grade makes sense given the correlation between socioeconomic

status, for which Free and Reduced Lunch (FRL) is a proxy, and academic achievement.

Table 1: Comparing Demographics of Students Exiting

EL Programming by Grade, 2007-2016

3rd 4th 5th 6th 7th

Hispanic 53% 61% 59% 59% 55%

Asian 34% 28% 32% 28% 35%

White 12% 9% 8% 12% 8%

Other Race 1% 2% 2% 2% 2%

Male 48% 47% 50% 48% 48%

Female 52% 53% 50% 52% 52%

FRL 62% 75% 76% 70% 75%

Non-FRL 38% 25% 24% 30% 25%

Spanish 53% 61% 59% 58% 54%

Hmong 17% 19% 23% 18% 26%

Other Language 30% 20% 18% 25% 20%

Source: Authors’ Analysis, DPI Data 2007-2016.

We then categorize 5th grade students into one of four groups, based on two questions:

1) Did the student remain in or exit out of programming? (In or out?)

2) Did the student reach the DPI recommended guideline score of 5-5 on the English

proficiency assessment? (Under or over?)

Consequently, four groups are created: those who exit programming with scores above the

guideline (Out-Over group), those who exit with scores below the guideline (Out-Under group),

those who remain in programming with scores above the guideline (In-Over group), and those

who remain with scores below the guideline (In-Under group). The following table depicts each

group:

5

Table 2: Categorization of 5th Grade Students

Did the student exit EL programming?

Yes → “Out” of programming No → “In” programming

Did the

student reach

the DPI-

recommended

guideline

score of 5-5 on

the English

proficiency

assessment?

Yes →

“Over”

score

Out-Over:

Exited as expected, having

met the DPI-recommended

guideline for exit.

2,343 students

16%

In-Over:

Remain in programming

despite having reached the

guideline score; an exception

to keep in longer.

1,371 students

9%

No →

“Under”

score

Out-Under:

Exited programming early;

an exception to exit students

earlier than guidelines.

146 students

1%

In-Under:

Have not reached the DPI-

recommended score and

remain in programming.

11,329 students

74%

Source: Authors’ Analysis, DPI Data 2007-2016 15,099 students total in 5th grade.

The In-Under group is by far the largest, with 11,329 students. This is unsurprising, as this group

represents the bulk of EL students who are still progressing toward proficiency. Next largest is

the Out-Over group with 2,343 students, followed by the In-Over group with 1,371 students. The

smallest group is the Out-Under group at 146 students. These are the students for whom teachers

or administrators decided to make exceptions and exit from programming before they had

reached the recommended guideline score of 5-5. All but one (145 of 146) received a five or

above for their overall ACCESS score, but only a four for their literacy subscore. Finally,

students who had already exited EL programming by 5th grade (former EL students) were

identified so they could be controlled for in analysis. There are 4,603 former EL students in our

5th grade sample.

Overall, the different groups look demographically similar, with some exceptions for the Out-

Under group. Students in the Out-Under group are more likely to be male, attend school in a

town, and are less likely to be FRL-eligible than students in the other groups are. However, it is

important to note that this group is the smallest and is more prone to variations due to random

error. It is also noteworthy that a higher proportion of the In-Over group is female, although it is

unclear why this may be. (see Appendix D for a more complete breakdown of characteristics

group).

We use 4th grade standardized test score percentiles on both mathematics and reading/English

language arts tests for each group of students in our model as a control for baseline academic

6

performance. Because standardized tests are administered later in the school year than ACCESS

tests, 4th grade standardized scores are the most appropriate pre-5th grade ACCESS-test

baseline. Depending on when the decision to reclassify students is made, these might be the most

recent available scores on which teachers and administrators might base their decisions.

Average score percentiles by group in both 4th and 8th grade reflect the expected relationship

between exit status, English proficiency score relative to the 5-5 guideline, and outcomes on

standardized assessments, with those in the Out-Over group performing best and those in the In-

Under group performing worst. However, we see a different pattern emerge for the change in

percentile scores between 4th and 8th grade. Figure 1 shows the difference in average

standardized test score percentiles between 4th and 8th grade. All groups show improvement

between 4th and 8th grade, even the In-Under group. However, while the Out-Over group shows

the most improvement in reading and math, at best the In-Over shows equivalent improvement

to the other two groups in reading and at worst shows the smallest improvement in math. This is

an early indication that students who exit in 5th grade benefit from doing so, while those who

remain may be kept in programming for too long. We further explore the relationship between

exit status and 8th grade test scores in our analysis below.

Figure 1: Difference in Average 8th Grade and 4th Grade WSAS Score Percentiles by Group

Source: Authors’ Analysis, DPI data 2007-2016

Methods

To answer our first research question, we examine the impact of reclassification and ACCESS

score in 5th grade on 8th grade standardized test score outcomes. Fifth grade appears to be an

0%

2%

4%

6%

8%

10%

Out-Over In-Over Out-Under In-Under

Reading Math

7

inflection point for students exiting out of EL programming, as shown in Appendix B. It is the

peak year in which students exit (2,842 exiters), closely followed by 4th grade (2,582 exiters).

The number of students reclassifying outside of 4th or 5th grade drops off with approximately

1,200 students exiting in 6th or 7th grade and only 18 students in 8th grade. Focusing on 5th grade,

therefore, provides us with the largest analytical sample of reclassified EL students. We use 8th

grade standardized test scores as outcomes because 8th grade is the last consecutive year of

testing for students. The next test that students take is the ACT® in 10th grade, which would

further attenuate our analytical sample, as we observe significantly fewer students through to

10th grade. Outcomes are measured as percentile scores for the math and reading subsections of

Wisconsin state standardized tests. To account for year-to-year variations in test formats, we use

score percentiles, rather than raw or scaled score.

We first use Ordinary Least Squares (OLS) regression to estimate differences in test scores for

the four groups listed above, controlling for 4th grade standardized test scores, to determine

changes in performance for each group. We then build upon this specification by introducing

controls for student characteristics such as gender, student language, and FRL eligibility. To

make our estimates more accurate, we finally include district fixed effects and account for

unobserved school-specific variation. Our full model, therefore, accounts for most of the

potentially important factors at the student, school, and district levels.

To address our second research question, we interact the student group variables with student

language, gender, time in EL programming, school EL student concentration, and others

characteristics. The interaction of these variables with the student groups allows their effect to

vary by student group, shedding light on how these characteristics might impact the score

estimates differently. For tables displaying regression output, see Appendix H.

Limitations and Assumptions

While we find our analysis to be rigorous, the report is limited by the availability of data, leading

to several constraints, along with necessary assumptions for the analysis to hold. State law does

not require private schools to report their students’ performance so we do not consider those

students here. Lack of consistent data is also an issue for highly mobile students. Our data was

also limited regarding students’ primary languages, as we cannot consider language-based

differences between the 3,277 students in the “other” language category.

We did not have access to data that could account for diversity of the many EL programs in

Wisconsin schools. Outcomes may vary between these schools based on structural differences

and delivery of programming.

Our qualitative information from practitioner interviews is inherently limited by scale. The

sample of people we interviewed is a small portion of program staff from Wisconsin’s 53 districts

with Bilingual Bicultural (BLBC) programs.

The stated limitations, reviewed in Appendix E, led to several necessary assumptions for

completing this report. Comprehensive assumptions are discussed in Appendix E, but several are

included here. We assume student performance across standardized assessments is comparable

8

over each year of test iterations. This report also assumes that reclassification processes and

implications are generalizable outside of 5th grade, the only year of analysis included. However,

we may conclude that this report provides implications for middle school students alone.

Additionally, we assume no changes in individual student characteristics across time such as

gender and FRL eligibility. We also included former EL students in the performance of the EL

population overall, likely skewing performance of that group higher than if the former EL group

had been excluded.

Findings

Research Question 1: Differences Between Student Groups

To answer our first research question, whether English proficiency level and timing of exit from

EL programming in 5th grade affects 8th grade academic outcomes, we look to our full models

controlling for student characteristics and school and district effects. Though we focus on the

results of those final models, our findings were consistent and both statistically and substantively

significant for each student group across all iterations of our models.

Effects for each group can be interpreted as percentage-point increases above a baseline

percentile for observationally equivalent students who score below the 5-5 guideline and remain

in EL programming (In-Under). In other words, the baseline state for a student is in the In-Under

group, and the effects stated below are the percentile increases that come from membership in

one of the three other groups (Out-Over, In-Over, or Out-Under). As expected, students who

score above the 5-5 guideline in 5th grade and are exited out of EL programming (Out-Over)

score the highest on average in 8th grade. More specifically, Out-Over students score nearly nine

percentile points higher on 8th grade math exams and nearly 10 percentile points higher on 8th

grade reading exams than In-Under students, when controlling for student characteristics and

school and district effects. Students who score above the 5-5 guideline but remain in EL

programming (In-Over) and students who score below the 5-5 guideline but are moved out of

EL programming (Out-Under) perform roughly the same, scoring five to six percentile points

higher on 8th grade math and reading exams than In-Under students. Figure 2 shows these

differences in percentile for each group of students relative to the In-Under group, with diamonds

representing coefficient point estimates and shaded bars showing 95 percent confidence

intervals.

9

Figure 2: Difference in 8th Grade Math and Reading WSAS Percentile Scores by Group

Source: Authors’ Analysis, DPI Data 2007-2016

Research Question 2: Impact of Student and School Characteristics

To answer our second research question, whether certain characteristics impact 8th grade

academic performance differently by our four groups, we turn to our interacted models. These

models explore how and if average 8th grade standardized test score percentiles for each student

group vary based on characteristics such as student language, FRL eligibility, gender, 4th grade

ACCESS score and exit status, absenteeism, learning disability, time in EL programming, school

locale, and school EL student concentration.

Figure 3 shows all statistically significant results for the un-interacted controls from our

interaction models, with diamonds representing coefficient point estimates and shaded bars

showing 95 percent confidence intervals. These are the results for characteristics that remain

significant even after we allow their relationship to 8th grade scores to vary by student group.

What is left, then, can be interpreted as applying to students overall, regardless of membership

in one of the four student groups. For example, in our model interacting Hmong speakers with

group membership, none of the interacted results was significant, but we did see significant

negative results for all Hmong speakers in both math and reading. These are the effects reported

in the first two bars of Figure 3. Estimates below the horizontal axis show that students associated

with those characteristics perform worse than their observationally equivalent peers who do not

share that characteristic, while estimates above the horizontal axis show that such students

perform better.

10

We see that Hmong-speaking students, those with learning disabilities, and those who are

chronically absent perform worse across the board when compared to their observationally

equivalent peers, while students who speak languages other than Hmong and Spanish, those

who scored above 5-5 in 4th grade but remained in EL programming, and those who have been

in EL programming for fewer than three years all perform better overall. For two categories,

gender and FRL eligibility, results were significant only for reading scores, with female

students performing better than their male peers overall and FRL eligible students performing

worse overall. We discuss these results, as well as the interacted results, in more detail below.

Additionally, Appendix H contains a table with all significant results from our interaction

models.

Figure 3: Results for ELs Overall by Characteristic


Language

Hmong-speaking EL students perform 3.1 percentage points worse on math and 3.9 percentage

points worse on reading than their non-Hmong speaking peers. Students who speak minority

languages fare better than their Hmong- and Spanish-speaking peers by 1.7 percentage points in

math and 2.4 percentage points in reading.

Subgroup Analysis: Spanish-speakers in the Out-Over category performed 2

percentage points worse on math in 8th grade than those in the Out-Over group who

speak Hmong or other languages. Spanish speakers in the In-Over category also

performed worse on math than those in other categories, by 2.5 points. Those in the

“other” language category, who remain in programming despite having scored a 5-5

(In-Over group) also outperform their peers by 4.2 percentage points in math.

11

Free and Reduced Lunch Status

EL students eligible for FRL (a majority of ELs) fare worse than non-eligible EL students in

reading by 1.2 percentage points.

Gender

Female EL students outperform male EL students in reading by 3.4 percentage points.

Subgroup Analysis: Females in the Out-Over category underperform on math by 1.8

percentage points compared to males in the same category.

Disability Status

EL students with intellectual disabilities fare poorer than students without a disability, by 5.5

percentage points in math and 5.7 percentage points in reading.

Subgroup Analysis: While the number of students with disability status yields low

sample sizes, findings for subgroups were statistically significant. Out-Under ELs with

an intellectual disability performed 16.5 percentage points better in math than their

non-disabled Out-Under peers. In-Over disabled students performed 7.4 percentage

points better in math and 6.6 percentage points better in reading than In-Over non-

disabled students.

Chronic Absenteeism

Chronically absent EL students underperform their EL peers with consistent attendance by 3.2


Retention in Programming after Scoring a 5-5 in 4th Grade

Some students scored a 5-5 on their 4th grade English proficiency exam but were kept in EL

programming through at least 5th grade. These students outperform students who were not

retained in EL programming for at least an additional year by 2.9 percentage points in math and

2.3 percentage points in reading.

Subgroup Analysis: EL students in the Out-Over group perform worse in math than

other Out-Over students who did not score a 5-5 in 4th grade by 2.7 percentage

points, and In-Over students who scored at least a 5-5 in 4th grade but remained in

programming fare worse than those who were exited by 3.3 percentage points in

reading.

Low Number of Years in EL Programming

EL students who, by 5th grade, have been in programming for fewer than three years fare better

than students in EL programming for longer periods of time. They outperform their peers by 2.2


12

Subgroup Analysis: In-Over ELs outperform those in programming for longer

periods of time by 2.5 percentage points in math and 2.8 percentage points in

reading.

School Locale

Students in towns outperform ELs in other types of schools (cities, suburbs, and rural schools)

by 3.3 percentage points.

Subgroup Analysis: Students in the In-Over category who attend city schools also

perform worse in math by 3 percentage points than their peers. In-Over ELs in suburbs

perform 2.4 percentage points better in reading than In-Over students in other types of

schools.

Groups and Variable Interactions without Significant Findings

We did not find significant results for the majority of subgroups (see Appendix H for full report

results). For example, there were no significant results for students in schools with a low

concentration of EL students. These findings may be due to any number of factors, including

that such student or school characteristics truly have little effect on student outcomes, or that

variation is lost due to standard errors clustered by school. Regardless, it is important to note that

not all subgroups showed significant variation in outcomes.

Discussion

From these findings, we can conclude that variation in performance is non-uniform across

subgroups. We may have expected, for example, that In-Over students, having been selected to

remain in programming for at least a year past 5th grade, would fare better on later standardized

tests given the continuing supportive services. This was not the case. This group consistently

performed worse overall when compared to their peers who were exited out of programming in

5th grade with similar scores relative to the 5-5 guideline. It appears as though students who

have reached the 5-5 score benefit from exiting out of programming. Still, we did see

heterogeneity in the relationships between certain student characteristics and 8th grade test

scores for different student groups. For example, In-Over Spanish-speakers performed worse

than their peers in math, while In-Over “other language” speakers performed better. Student

characteristics seem to matter, but the noisiness of the results highlights the continued need for

discretion in decisions about reclassification.

The only consistent negative finding across subgroups based on exit score in 5th grade were the

Out-Over students, who performed worse as Spanish-speakers in math, females in math, and as

students retained in programming who had scored at least a 5-5 the previous year in both reading

and math. This suggests that some students do indeed struggle more upon exiting EL

programming than others. However, in no cases were these negative effects large enough to erase

the score difference between the Out-Over and In-Over groups. This means that though these

13

students may struggle more than their peers may after they are out of programming, they still

appear to do better than those who remain in programming.

Such inconsistent variation points to the challenge in creating comprehensive policies and

systems for exiting students. Additional analysis of student and school characteristics such as

program type, community supports, educational attainment of families, or student engagement

in extracurricular programs all may support evaluation of student performance. Discretion is

important in considering the unique needs of each child and the available resources to serve the

student, but decision-makers may benefit from considering how various subgroups with

significant results perform when compared to other students.

Recommendations

Based on our findings and discussion above, we will aim to provide DPI and other stakeholders

with key policy and programmatic recommendations. Recommendations are grouped based on

general suggestions related to further data collect and use, and suggestions directly related to

findings of our analysis.

As discussed in the data section of this report, we removed a significant number of students from

the analysis. The exact implications of dropping students who were not observed for at least four

years are unknown, but we can speculate that the students removed may be uniquely

differentiated from those who are observed consistently. Removed students may be more mobile,

transferring in and out of school systems and even the state. This may also be a group with lower

overall socioeconomic status and reduced educational opportunities. Therefore, excluding these

students from analysis may limit the generalizability of implications and recommendations for

all students. It is important to consider that these students also require EL resources and may

even face more barriers to success than their peers may. The following recommendations should

be considered with these caveats in mind.

Recommendations Related to Further Data Collection and Use

1. In this analysis, we were unable to control for any type of EL programming

characteristics. The interventions in a Dual Language Immersion program may

vastly differ from that in a Sheltered Instruction setting, for example. Thus,

outcomes may range broadly depending on the type of program. It is imperative

that DPI continues its work in cataloguing and classifying types of programming.

Ensuring accurate collection of this program data at the student level, not just

school or district, is necessary in advancing the analysis of both student and

program performance.

2. Similar to lack of programming data, we did not assess funding for EL services at

either the student-, school- or district-level. We recommend that DPI track funding

allocations and expenditures by different institutional levels to determine if

allocation of resources effectively supports student outcomes.

3. Likewise, current data on test accommodations reflects only whether students did

or did not receive accommodations. Information on what type of accommodations

14

are provided to students is not recorded, but it would be helpful to understand

student experiences and control for different testing circumstances.

4. We encourage districts and schools to track and assess their reclassification

decisions using data. At what ELP level are most students exited? Does this vary

between schools? How much discretion is used by primary teachers, ESL teachers,

and district administrators? What criteria do they use to make reclassification

decisions? Are patterns identifiable? Analyzing such school- and district-level data

would allow administrators to develop policy priorities and to note changes in

decision criteria used for reclassifying students.

5. Districts should consider analyzing data on students who re-enter EL programming

after exiting. Any patterns in students’ re-entry into EL programming may uncover

flaws in how and when administrators decide to exit students.

Recommendations Specific to Findings of Our Analysis

1. Our findings suggest that, all else equal, if a student has scored 5-5 by 5th grade,

he or she is better off exiting programing than remaining in. Teacher discretion in

keeping students in programming after that point should be exercised judiciously,

erring on the side of exiting a student. This recommendation relates to the primary

research question, comparing students of various ELP levels in 5th grade.

2. Variation is seen across students in different language groups and exit subgroups.

While findings are mixed, targeting resources specifically at students based on

language group could be a valuable investment in moving Spanish-speaking

students to English proficiency more quickly.

3. While the “other” language speakers fare better compared to their Spanish and

Hmong-speaking peers, we recommend that DPI continue to think critically about

this heterogeneous group of students. They speak a wide range of languages and

require student-specific interventions based on the needs of each individual child.

4. With such variety in outcomes for subgroups, we would not recommend that DPI

tighten recommendations on exiting, but rather encourage schools to evaluate their

own individual practices and focus on student outcomes.

5. ESL teachers and administrators, when deciding whether to exit a student from EL

programming, should use student- and school-level data that is available to them in

order to contextualize the decision. It is unclear to what degree this practice is

uniform across districts in the state, as varying degrees of discretion are used,

according to our limited interviews. Perhaps, districts also could learn more from

one another by communicating their practices and discussing patterns in outcomes.

Regardless, making broad generalizations is unwise because there are varying

effects of different characteristics for both students and schools. Rather, decisions

should be customized to the particular circumstances and context of each student,

with the 5-5 exit guideline used where possible.

15

Opportunities for Further Research

Data collected and reported on EL student progress and performance needs improvement. Most

of these improvements require policy changes rooted in state law along with a commitment by

the state legislature to refrain from annually changing the way that Wisconsin students are

assessed. Data structure and availability issues that we encountered in conducting this analysis

include:

1. The fact that EL academic performance and English language proficiency is tested

only once per year poses challenges to anyone trying to study populations of

students who are highly mobile. This is likely true for a portion of EL students, with

bilingual students often being raised in migrant or refugee families that may

relocate one or more times while a student is enrolled in EL programming.

2. Our analysis also excludes any students who attended private schools between 4th

and 8th grade in Wisconsin. We lack any data on the performance or proficiency of

EL students in these schools. The inevitable differences in English language

acquisition programming for these students is worth study and evaluation.

3. Our analysis of reclassifying students’ outcomes does not consider information

about students who reclassify before 5th grade. If these students are meaningfully

different from those who reclassify in 5th grade, perhaps by their access to

supplemental English tutoring, exposure to English in their homes, or academic

aptitude and speed of language uptake, any differences in outcomes among pre-5th-

grade reclassified EL students should be studied.

4. We were unable to consider any of the inevitably numerous differences in the

characteristics of individual EL programs because that data does not exist.

Characteristics of EL programs that differ school-to-school may have an effect on

EL students’ language uptake or outcomes, such as program structure, funding,

staff ratios, and staff qualifications, are worth study.

5. Charter schools are perhaps more likely than traditional public schools to be

heterogeneous in their EL program characteristics. Given recent policy-driven

efforts to expand access to school choice options, especially in populous parts of

the state with many bilingual students, this topic deserves attention by education

policymakers.

Conclusion

This report shines new light on the performance of a key group of historically disadvantaged

students. We find that where teachers exercise discretion in exiting ELs from programming after

5th grade, they should only do so using caution if a student has already achieved the

recommended 5-5 score. Various subgroups, including Spanish- and Hmong-speakers and

chronically absent students, have poorer outcomes on standardized tests after exiting EL

programming, depending on test subject. We find that categorizing students by their English

proficiency score level at time of exit is a helpful tool in determining the appropriate use of

teacher discretion in exiting behavior.

16

To accurately measure the effects of widely varying programs, we recommend that DPI require

and analyze student-level data on type of services received across the state. Doing so will provide

a more complete picture of programming impacts on student outcomes, along with the

opportunity to target resources at successful interventions.

Using both a quantitative and qualitative approach to the exploration of EL services will support

districts and administrators in creating policy that comprehensively addresses the needs of

students. The snapshot of EL experiences paired with provided recommendations can help DPI

better allocate resources for those Wisconsin students with the greatest need.

17

Appendix A: Relevant Literature

Several scholarly studies on reclassification focused on the effects of changes in the score cutoffs

used to determine reclassification. Robinson-Cimpian & Thompson (2016) and Will et al. (2014)

found that for states or districts that adopt higher reclassification cutoff scores, the students who

scored between the two and who no longer reclassify with that score have better outcomes than

similar students who did reclassify with that score before the policy change. However, Carlson

& Knowles (2016) and Kim (2011) found that students who spent more time classified as EL

have worse test-score achievement and grade progression than students who spend less time

classified as EL.

These studies appear to contradict each other. On the one hand, Robinson-Cimpian & Thompson

(2016) and Will et al. (2014) suggest that reclassifying students at lower ELP levels may remove

them from programming that could offer them support to improve their achievement outcomes.

On the other hand, Carlson & Knowles (2016) and Kim (2011) found that students who spend

more time in EL programming have worse achievement outcomes. Other studies find different

rates of ELP progress or post-reclassification outcomes based on students’ backgrounds.

Table 3: Literature Review on Reclassification Research

Source Research Question Sample and Method Findings

Robinson-Cimpian & Thompson 2016 “The Effects of Changing Test-Based Policies for Reclassifying English Learners”

This paper examines the effect of raising the threshold for reclassification on Latino/a EL students in the Los Angeles Unified School District.

Difference-in-regression-discontinuities method with student-level California student performance and outcomes data

Students who were reclassified before the changes but whose scores would not have met the higher thresholds had lower outcomes trends than students with these scores who weren’t reclassified under the higher threshold.

Carlson & Knowles 2016 “The Effect of English Learner Reclassification on Student ACT Scores, High School Graduation, and Postsecondary Enrollment: Regression Discontinuity Evidence from Wisconsin”

This paper estimates the causal effect of being reclassified at the end of 10th grade on several outcomes related to postsecondary educational attainment: ACT scores, high school graduation, and postsecondary enrollment.

Regression discontinuity that exploits the reclassification cutoff at or above a certain ACCESS score to identify the effect of reclassification of Wisconsin EL students between 2006 and 2013

Reclassification in 10th grade has a positive effect on ACT scores, the probability of high school graduation, and the probability of postsecondary enrollment relative to those who don’t reclassify in the 10th grade.

18

Kim 2011 “Relationships Among and Between EL Status, Demographic Characteristics, Enrollment History, and School Persistence”

This paper examines enrollment history, achievement gaps, and persistence in school for EL students compared to non EL students.

Multilevel logistic regression to show large achievement and socioeconomic gaps between EL and non-EL students in California

EL students who reclassified later or who remain in EL status in high school show larger gaps compared to EL students who reclassified earlier. The longer a student is designated as EL, the more likely they are to drop out before graduating high school.

Slama 2014 “Investigating Whether and When English Learners are Reclassified Into Mainstream Classrooms in the United States: A Discrete-Time Survival Analysis”

This study examines EL students’ tenure in language-learning programs and their academic performance following reclassification.

Discrete-time survival analysis of EL student data in the US to estimate the average time to and grade of reclassification with and without controlling for home language and socioeconomic status

The average EL student exits three years after school entry or in second grade, and the odds that a non-Spanish-speaking EL student was reclassified were nearly twice that of their Spanish-speaking EL counterparts, after controlling for income.

Umansky & Reardon 2014 “Reclassification Patterns Among Latino English Learner Students in Bilingual, Dual Immersion, and English Immersion Classrooms”

This study examines timing of reclassification among Latino ELs in four distinct linguistic instructional environments: English immersion, transitional bilingual, maintenance bilingual, and dual immersion.

The authors use hazard analysis and 12 years of data from a large school district to investigate whether reclassification timing, patterns, or barriers differ by linguistic program.

Latino EL students enrolled in two-language programs are reclassified at a slower pace in elementary school but have higher overall reclassification, English proficiency, and academic threshold passage by the end of high school.

Hill, Weston, & Hayes 2014 “Reclassification of English Learner Students in California”

This paper investigates whether California school districts with more rigorous reclassification standards have systematically lower EL reclassification rates and/or better student outcomes than districts with lower standards.

Cross-sectional cohort comparison using six years of California state-level student data: grade progression data and standardized test performance data

Reclassified EL students outperform non-reclassified students and perform as well as native English speakers on standardized tests and grade progression. Districts using more stringent reclassification criteria have lower reclassification rates and better outcomes among reclassified students.

19

Appendix B: 5th Grade as Inflection Point

In our discussions with DPI about how to focus our analysis, they suggested that we might want

to look at ELs in 5th grade, which they identified as an inflection point for reclassification of EL

programming. According to our data, 5th grade is the peak exit year for EL students, followed

closely by 4th grade. Therefore, our data supports using this cross-section of 5th grade students

for analysis.

Figure 4: Reclassified Students per Grade


Table 4: Number of Students Exiting EL Programming by Grade, 2007-2016

Grade Number of Students

in EL Programming

that Grade

Number of Students

Exiting EL

Programming that

Grade

Percentage of

Students who Exit

Programming that

Grade

1st 7,556 8 0.1

2nd 11,712 317 2.7

3rd 15,404 1193 7.7

4th 18,185 2582 14.2

5th 19,494 2842 14.6

6th 16,453 1018 6.2

7th 15,513 1332 8.6

8th 12,145 18 0.1

Source: Authors’ Analysis of DPI data, 2007-2016

20

Curriculum-based best practices and empirical evidence suggest that 5th grade, or around 11

years of age, is a critical inflection point in the rate of students’ learning progress toward English

proficiency. A 1989 study by Johnson and Newport gave an English grammar test to a sample

of adult Chinese and Korean immigrants who entered the United States and divided participants’

performance by age-of-entry categories. Mean scores for immigrants who arrived in the 3-7 age

group scored 269.3 (compared to the native-speaker mean score of 268.8). The mean score of

those who entered at ages 8-10 was 256.0, a 13.3-point drop; those who entered at ages 11-15

scored 235.9, a 20.1-point drop; and those who entered at ages 17-39 scored 210.3, a 25.6-point

drop. Declines in grammar test scores accelerate as the age of entry increases. The authors

suggest that the second-language acquisition of those with pre-puberty ages of entry and post-

puberty ages of entry are significantly different. They go on to identify puberty as the critical

period for second-language acquisition (Johnson 2009). Most entering 5th graders are 11 years

old, suggesting that the rate of English language uptake changes around 5th grade.

21

Appendix C: Interview Protocol

To provide a qualitative perspective on the EL experience in Wisconsin, we conducted

interviews with administrators at four mid-sized school districts. These districts have between

715 and 2,281 current EL students, with total enrollments of 5,418 to 22,160 students (DPI

2017j). We believe that through discussions with on-the-ground educators, we can more

comprehensively understand the experience for EL students in Wisconsin. We do not assume

that these stakeholders express experiences that are representative of all administrators; rather

their perspectives are unique and valued in their individual contexts. Interviewed staff manage

EL programming, supervise ESL teachers, oversee data collection and reporting, and provide

leadership in policy direction, among other activities. We used the following interview questions

to construct an informative conversation with stakeholders:

1) Brief description of report, research questions, target understanding of EL lifecycle.

2) What is your role in the district or school, specifically as related to EL students,

programming, and/or administration?

3) Are you able to describe the EL lifecycle, or any pieces? Address identification,

programming, interventions or supports, reclassification, monitoring, re-entry into EL

programming, if applicable.

4) Does your region follow a protocol regarding manual or automatic ELP classification?

If so, please describe. What is the preferred method for your region, and why?

5) Can you generalize any characteristics or trends that differ between those students who

exit at ELP 5 and ELP 6?

6) What is different about students who take longer to reclassify? Is there a tradeoff

between keeping students in EL programming for a longer period of time, versus

reclassifying a student as a former EL?

7) How cohesive is programming across schools and districts? Are there broad ranges of

policies and/or practices?

8) How prescriptive is DPI when providing guidelines for EL education?

9) How integrated are ELs in the general school population?

10) Can you comment on any differences between EL education in public non-charter

compared to public charter schools?

11) Can you comment on your experience with ELs and private school education?

12) How does student transience impact EL education?

13) What systems or policies would you like to see changed that relate to EL education?

22

Appendix D: Student Characteristic Breakdowns by Reclassification Group

Each of the following figures provides a snapshot of student outcomes by categorized group.

Figure 5: Breakdown of Gender in 5th Grade by Group


Figure 6: Breakdown of Home Language in 5th Grade by Group


49.9

39

55.1

45.1

50.1

61

44.9

54.9

0% 20% 40% 60% 80% 100%

Out-Over

Out-Under

In-Over

In-Under

Female Male

58.9

68.5

62.3

62.9

23.9

13.7

21.2

20.6

17.2

17.8

16.5

16.5

0% 20% 40% 60% 80% 100%

Out-Over

Out-Under

In-Over

In-Under

Spanish Hmong Others

23

Figure 7: Breakdown of School Locale Code in 5th Grade by Group


Figure 8: Breakdown of FRL Eligibility in 5th Grade by Group


58.2

41.1

48.2

55.3

24.1

24.7

30.1

24.2

11.2

28.1

12.6

12.6

6.4

6.2

8.9

7.5

0% 20% 40% 60% 80% 100%

Out-Over

Out-Under

In-Over

In-Under

City Suburb Town Rural

78.4

67.1

77.8

81.1

23.1

32.5

22.8

14.5

0% 20% 40% 60% 80% 100%

Out-Over

Out-

Under

In-Over

In-Under

FRL-Eligible Not FRL-Eligible

24

Figure 9: Average WSAS Reading Score Percentiles in 4th and 8th Grade by Group

Source: Authors’ Analysis, DPI data 2007-2016 Figure 10: Average WSAS Math Score Percentiles in 4th and 8th Grade by Group


0%

10%

20%

30%

40%

50%

60%


Perc

entile

4th Grade 8th grade

0%

10%

20%

30%

40%

50%

60%


Perc

entile

4th Grade 8th grade

25

Appendix E: Limitations and Assumptions

Assumptions Our analysis of the differences between the four groups of 5th grade EL students is confined by

the available data. State law does not require private schools to report their students’ performance

on standardized testing or to offer EL programming to the bilingual student population, so we

cannot consider how students in these schools are served in programs that support their English

language acquisition. This is also an issue for mobile students; there are gaps in the data for

students who were at some point classified as an EL, transferred to a private school or out-of-

state, and transferred back to a public school still classified as an EL.

Our data was also limited in its information about students’ primary languages. The data

collected identifies students in three language-based categories: Spanish-speaking, Hmong-

speaking, and “other.” This is limiting because it prevents us from considering language-based

differences between the 3,277 students in the “other” language category.

We did not have access to data that could account for diversity of the many EL programs in

Wisconsin schools. We did use data on the type of school each EL student attended for each year

he/she was tested, which included breakdowns for school type. These breakdowns included

designations for traditional public schools and two types of public charter schools: district and

non-district, which we use as a category “charter schools.” In our sample, 1,065 students attend

charter schools. Because the purpose of Wisconsin’s charter school policies is to allow for

schools that serve as alternatives to traditional public schools, these schools can operate

differently from traditional schools in many ways. This creates enormous potential for

heterogeneity in the structure, methods and curriculum in schools. Outcomes may vary between

these schools based on some of these major differences, although we were unable to consider

this in our analysis because we didn’t have data about the nature of these charter schools’

practices or operations.

Our data set was missing student ACCESS scores for about 300 students, which we are assuming

reflect random mechanical or reporting errors and not a systematic trend about those students. In

our analysis, we dropped observations for these students.

Finally, our qualitative information from practitioner interviews is inherently limited by scale.

The sample of people we interviewed is a small portion of program staff from Wisconsin’s 53

districts with BLBC programs.

Limitations Because of some of the limitations in our available data, we made a few key assumptions in the

design of our analysis. Some are basic structural assumptions that we make about the validity of

particular indicators of student characteristics, while others are specific assumptions about our

data and model.

1) The recent changes in standardized assessments administered to students in

Wisconsin make performance scores not directly comparable. To account for

26

this, we assumed that students’ performance percentiles relative to their peers

on a given exam are a valid way to compare achievement on different exams

with different scoring systems.

2) We analyzed outcome data among only students who reclassify between grades

5 through 8. Therefore, we make the assumption that outcomes for students

exiting in these grades are generalizable to those who exit in other grades. For

example, we assume that student and school demographics have the same effect

on students who exit in 5th grade as those who exit in 9th grade.

3) We assumed that there were no over-time changes in EL students’ eligibility

for the Free and Reduced Lunch program, which we use as a proxy for

socioeconomic status. We also acknowledge that using FRL status as a proxy

for poverty is flawed due to the over-inclusion of many students in federal

“community expansion” programs and other changing FRL-status criteria.

Regardless, many reporting entities and policymakers continue to use this

imperfect substitute (Chingos 2016).

4) We assumed that there were no changes in self-identified gender among EL

students in our data.

5) Because of limitations in our data, we treated all EL students from non-Spanish

and non-Hmong households in the “other” language category, which implies

that they are a homogenous group.

6) As mentioned in the limitations, we did not have data about differences in EL

program structure, methods, etc., and therefore needed to assume that all EL

programs in Wisconsin schools had the same effect on students’ future test

scores.

7) We assumed that charter schools are homogeneous in their EL student outcomes

for the sake of our comparison to traditional public schools.

8) We dropped approximately 350 students (~219 in 5th, ~139 in 4th) who lacked

an ACCESS score in our data, making the assumption that there is not

something systematically different about these students and that lack of a score

is random.

9) Finally, we include former EL students, who exited EL programming before

5th grade, in our discussion of overall ELs. Various DPI metrics include or

exclude former ELs, depending on the measure. Including former ELs skews

the performance of the overall EL population, as this subgroup outperforms

current EL students. At a given point, former ELs comprise approximately 15

percent of the total EL population in Wisconsin (NCELA 2016).

27

Appendix F: Results by Student Characteristic Interactions

The following table provides data regarding statistically significant interactions for student

subgroups and individual or school level characteristics.

Table 5: Significant Results for Students by Group, Interacted with Student and School Characteristics

Overall Out-

Over

Out-

Under

In-

Over

Language Groups

Math

Spanish - ↓ 2.0 - ↓ 2.4

Hmong ↓ 3.9 - - -

Other ↑ 1.7 - - ↑ 4.2

Reading

Spanish - - - -

Hmong ↓ 3.1 - - -

Other ↑ 2.4 - - -

Free/Reduced Lunch Eligibility Math - - - -

Reading ↓ 1.2 - - -

Gender (Female) Math - ↓ 1.8 - -

Reading ↑ 3.4 - - -

Learning Disability Math ↓5.5 - ↑ 16.7† ↑ 7.4†

Reading ↓ 5.7 - - ↑ 6.7†

Chronically Absent Math ↓ 3.2 - - -

Reading ↓ 3.8 - - -

In-Over in 4th Grade Math ↑ 2.8 ↓ 2.7 ↓ 8.6† -

Reading ↑ 2.3 ↓ 3.3 - -

In EL Programming Under 3

Years

Math ↑ 2.2 - - ↑ 2.6

Reading ↑ 2.5 - - ↑ 2.8

Census Locale Code

Math

City - - - ↓ 3.0

Suburb - - - -

Town - - - -

Rural - - - -

Reading

City - - - -

Suburb - - - ↑ 2.4

Town ↑ 3.3 - - -

Rural - - - -


† Sample under n=100

28

Appendix G: Data Preparation

This appendix details the steps taken to convert the raw data sets provided to us by DPI into our

analytical sample, the data set used to perform all of our statistical analyses.

We started with the data set containing demographic variables and ACCESS test scores for every

EL student in Wisconsin between 2007 and 2016. We merged the other data sets provided by

DPI—those including disciplinary, standardized test score, and school data—in with this base

ACCESS score data set, dropping the small number of observations (nine total) for which there

were duplicate combinations of student ID and school year. Unmatched data was also dropped

after each subsequent merge.

Merging the standardized test score data to the ACCESS test score data allowed us to observe

students beyond their time in EL programming because students may take standardized tests

every year between 3rd and 8th grade but take ACCESS tests only while in EL programming.

However, because demographic data was stored with ACCESS data in the files provided to us,

demographic information was missing in our data for students after they exited programming.

To address this, we carried forward and backward what information we did have on students

while they were in EL programming, essentially copying what we did observe about them when

they were in EL programming into the years when they were out of EL programming. We felt

this was a small assumption for some characteristics, such as student language or race/ethnicity,

which should be constant over time. Likewise, though a student’s gender may change, we

consider the likelihood that it did small enough as to not pose a significant challenge to the

validity of our assumption that it did not. More concerning was the necessity to carry forward

more likely time-variant characteristics, such as FRL eligibility and school code. This means that

we assume that students who were FRL-eligible when observed in EL programming remain so

throughout their academic career and that students do not change schools after they exit out of

programming. Again, we do not expect violations of these assumptions to be large enough to

significantly affect our results, but they do prevent us from being able to explore certain factors

related to student academic success, such as mobility.

After merging all of the data sets and filling in missing data, we dropped students that we didn’t

observe from 5th grade to 8th grade, reducing our analytical sample to 24,658 students from the

original 105,964. We also dropped the small populations of students who still appeared to be in

programming but were missing ACCESS scores in 4th and 5th grade (139 and 219 students,

respectively). We then moved relevant 4th and 8th grade test data into the 5th grade row for each

observation and dropped all observations that weren’t in 5th grade or that didn’t have a

standardized test score in 4th grade (4,435 students), converting our student panel data into a

cross section of 5th grade.

Finally, we dropped a few final duplicate students and unmatched results from our data merges,

bringing our final analytical sample to 19,792 students.

29

Appendix H: Regression Output

Table 6: Change in 8th Grade Math and Reading Score Percentiles by Student Group, Controlling for 5th Grade and 4th Grade Baseline Score Percentiles

5th Grade

Math

4th Grade

Math

5th Grade

Reading

4th Grade

Reading

Out-Over 0.0677*** 0.0959*** 0.0784*** 0.106***

(0.00468) (0.00479) (0.00493) (0.00496)

Out-Under 0.0397** 0.0602*** 0.0465** 0.0570***

(0.0140) (0.0161) (0.0146) (0.0157)

In-Over 0.0354*** 0.0611*** 0.0519*** 0.0681***

(0.00537) (0.00572) (0.00546) (0.00553)

Former EL 0.102*** 0.116*** 0.0856*** 0.0993***

(0.00414) (0.00435) (0.00447) (0.00467)

5th Grade Math 0.651***

(0.00618)

4th Grade Math 0.589***

(0.00643)

5th Grade Reading 0.667***

(0.00702)

4th Grade Reading 0.634***

(0.00753)

Constant 0.123*** 0.139*** 0.121*** 0.138***

(0.00332) (0.00351) (0.00306) (0.00327)

Observations 19311 19415 19136 19420

R2 0.557 0.508 0.561 0.525 Standard errors in parentheses

Year fixed effects included in model but omitted from table for clarity. * p < 0.05, ** p < 0.01, *** p < 0.001

Table 7: Change in 8th Grade Math Score Percentile by Student Group, Comparison of Three Models

4th Grade

Math

Student

Characteristics

Full Model

Out-Over 0.0959*** 0.0860*** 0.0885***

(0.00479) (0.00466) (0.00511)

Out-Under 0.0602*** 0.0564*** 0.0539**

(0.0161) (0.0162) (0.0164)

In-Over 0.0611*** 0.0536*** 0.0572***

(0.00572) (0.00553) (0.00551)

Student Controls No Yes Yes

School Clustered

SE

No No Yes

District FE No No Yes

Observations 19415 19415 19415

R2 0.508 0.544 0.576 Standard errors in parentheses * p < 0.05, ** p < 0.01, *** p < 0.001

30

Table 8: Change in 8th Grade Reading Score Percentile by Student Group, Comparison of Three Models

4th Grade

Reading

Student

Characteristics

Full Model

Out-Over 0.106*** 0.0972*** 0.0960***

(0.00496) (0.00485) (0.00511)

Out-Under 0.0570*** 0.0558*** 0.0548***

(0.0157) (0.0157) (0.0156)

In-Over 0.0681*** 0.0584*** 0.0635***

(0.00553) (0.00543) (0.00557)

Student Controls No Yes Yes

School Clustered

SE

No No Yes

District FE No No Yes



31

Table 9: Change in 8th Grade Math Score Percentile by Student Group and Language

Spanish Hmong Other

Language

Out-Over 0.103*** 0.0869*** 0.0867***

(0.00770) (0.00598) (0.00551)

Out-Under 0.0415 0.0603*** 0.0559**

(0.0298) (0.0166) (0.0179)

In-Over 0.0740*** 0.0578*** 0.0522***

(0.00901) (0.00617) (0.00584)

Spanish -0.00354

(0.00824)

Out-Over # Spanish -0.0203*

(0.00983)

Out-Under # Spanish 0.0208

(0.0333)

In-Over # Spanish -0.0242*

(0.0109)

Hmong -0.0393***

(0.00796)

Out-Over # Hmong 0.0113

(0.0116)

Out-Under # Hmong -0.0404

(0.0491)

In-Over # Hmong 0.00119

(0.0128)

Other Language 0.0172*

(0.00863)

Out-Over #

Other Language

0.0225

(0.0121)

Out-Under #

Other Language

0.00294

(0.0407)

In-Over #

Other Language

0.0418**

(0.0148)

Student Controls Yes Yes Yes

School Clustered SE Yes Yes Yes

District FE Yes Yes Yes



32

Table 10: Change in 8th Grade Reading Score Percentile by Student Group and Language

Spanish Hmong Other

Language

Out-Over 0.106*** 0.0934*** 0.0968***

(0.00768) (0.00606) (0.00542)

Out-Under 0.0613* 0.0598*** 0.0511**

(0.0273) (0.0170) (0.0172)

In-Over 0.0698*** 0.0665*** 0.0607***

(0.00855) (0.00630) (0.00604)

Spanish -0.0131

(0.00852)

Out-Over # Spanish -0.0137

(0.0101)

Out-Under # Spanish -0.00699

(0.0335)

In-Over # Spanish -0.00782

(0.0106)

Hmong -0.0313***

(0.00846)

Out-Over # Hmong 0.0158

(0.0113)

Out-Under # Hmong -0.0268

(0.0381)

In-Over # Hmong -0.00907

(0.0122)

Other Language 0.0237**

(0.00899)

Out-Over # Other

Language

0.00332

(0.0127)

Out-Under # Other

Language

0.0311

(0.0422)

In-Over # Other

Language

0.0248

(0.0147)

Student Controls Yes Yes Yes

School Clustered SE Yes Yes Yes

District FE Yes Yes Yes



33

Figure 11: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Language


34

Table 11: Change in 8th Grade Math and Reading Score Percentiles by Student Group and FRL Eligibility

FRL Math FRL Reading

Out-Over 0.0975*** 0.102***

(0.00945) (0.00992)

Out-Under 0.0386 0.0670*

(0.0214) (0.0278)

In-Over 0.0806*** 0.0801***

(0.0127) (0.0121)

FRL -0.00411 -0.0115*

(0.00599) (0.00542)

Out-Over # FRL -0.00778 -0.00542

(0.0105) (0.0106)

Out-Under # FRL 0.0292 -0.0139

(0.0290) (0.0342)

In-Over # FRL -0.0266 -0.0190

(0.0141) (0.0137)

Student Controls Yes Yes

School Clustered

SE

Yes Yes

District FE Yes Yes

Observations 19415 19420

R2 0.576 0.571 Standard errors in parentheses * p < 0.05, ** p < 0.01, *** p < 0.001

35

Figure 12: Change in 8th Grade Math and Reading WSAS Score Percentiles by Group and FRL Eligibility


Table 12: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Gender

Female Math Female Reading

Out-Over 0.0987*** 0.0935***

(0.00709) (0.00673)

Out-Under 0.0559** 0.0463*

(0.0215) (0.0191)

In-Over 0.0686*** 0.0582***

(0.00767) (0.00831)

Female 0.00397 0.0336***

(0.00326) (0.00300)

Out-Over # Female -0.0183* 0.00810

(0.00826) (0.00946)

Out-Under # Female -0.000371 0.0240

(0.0316) (0.0277)

In-Over # Female -0.0192 0.0133

(0.0101) (0.0105)


School Clustered SE Yes Yes

District FE Yes Yes



36

Figure 13: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Gender


37

Table 13: Change in 8th Grade Math Score Percentile by Student Group and School Locale

City School Suburban School Town School Rural School

Out-Over 0.0971*** 0.0852*** 0.0899*** 0.0898***

(0.00805) (0.00555) (0.00537) (0.00530)

Out-Under 0.0559** 0.0574** 0.0610** 0.0506**

(0.0196) (0.0201) (0.0196) (0.0165)

In-Over 0.0737*** 0.0515*** 0.0552*** 0.0586***

(0.00809) (0.00644) (0.00578) (0.00580)

City -0.00721

(0.00874)

Out-Over # City -0.0126

(0.00984)

Out-Under # City 0.00317

(0.0343)

In-Over # City -0.0300**

(0.0110)

Suburb 0.00899

(0.00935)

Out-Over # Suburb 0.0188

(0.0116)

Out-Under # Suburb -0.00789

(0.0320)

In-Over # Suburb 0.0242

(0.0126)

Town 0.0232

(0.0134)

Out-Over # Town -0.00311

(0.0159)

Out-Under # Town -0.0169

(0.0348)

In-Over # Town 0.0240

(0.0173)

Rural -0.0137

(0.00909)

Out-Over # Rural 0.000429

(0.0164)

Out-Under # Rural 0.0831

(0.0889)

In-Over # Rural -0.00438

(0.0184)

Student Controls Yes Yes Yes Yes

School Clustered SE Yes Yes Yes Yes

District FE Yes Yes Yes Yes

Observations 19415 19415 19415 19415 Standard errors in parentheses * p < 0.05, ** p < 0.01, *** p < 0.001

38

Table 14: Change in 8th Grade Reading Score Percentile by Student Group and School Locale

City School Suburban School Town School Rural School

Out-Over 0.0944*** 0.0984*** 0.0972*** 0.0975***

(0.00742) (0.00556) (0.00549) (0.00526)

Out-Under 0.0567* 0.0595*** 0.0514** 0.0576***

(0.0223) (0.0165) (0.0179) (0.0162)

In-Over 0.0708*** 0.0578*** 0.0671*** 0.0639***

(0.00749) (0.00678) (0.00587) (0.00582)

City -0.0122

(0.00874)

Out-Over # City 0.00447

(0.00928)

Out-Under # City -0.00171

(0.0296)

In-Over # City -0.0135

(0.0106)

Suburb 0.00222

(0.00916)

Out-Over # Suburb -0.00410

(0.0113)

Out-Under # Suburb -0.0112

(0.0407)

In-Over # Suburb 0.0235*

(0.0109)

Town 0.0331*

(0.0156)

Out-Over # Town 0.000633

(0.0130)

Out-Under # Town 0.0136

(0.0353)

In-Over # Town -0.0187

(0.0173)

Rural 0.0104

(0.00978)

Out-Over # Rural -0.000708

(0.0152)

Out-Under # Rural -0.0197

(0.0558)

In-Over # Rural 0.00935

(0.0196)

Student Controls Yes Yes Yes Yes

School Clustered SE Yes Yes Yes Yes

District FE Yes Yes Yes Yes

Observations 19420 19420 19420 19420 Standard errors in parentheses * p < 0.05, ** p < 0.01, *** p < 0.001

39

Figure 14: Change in 8th Grade Math and Reading Score Percentiles by Student Group and School Locale


40

Table 15: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Low EL Student Population

Low EL Math Low EL

Reading

Out-Over 0.0882*** 0.0962***

(0.00547) (0.00540)

Out-Under 0.0421** 0.0591***

(0.0160) (0.0173)

In-Over 0.0573*** 0.0650***

(0.00588) (0.00591)

Low EL

Concentration

-0.00962 -0.00389

(0.00748) (0.00690)

Out-Over # Low EL

Concentration

0.0134 0.00845

(0.0132) (0.0129)

Out-Under # Low EL

Concentration

0.0770 -0.0162

(0.0510) (0.0382)

In-Over # Low EL

Concentration

0.00714 -0.00371

(0.0178) (0.0166)



District FE Yes Yes



Figure 15: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Low EL Student Population


41

Table 16: Change in 8th Grade Math and Reading Score Percentiles by Student Group and 4th Grade In-Over Status

4th Grade In-

Over Math

4th Grade In-

Over Reading

Out-Over 0.0907*** 0.102***

(0.00589) (0.00604)

Out-Under 0.0642*** 0.0585***

(0.0182) (0.0166)

In-Over 0.0585*** 0.0632***

(0.00639) (0.00661)

In-Over 4th 0.0280** 0.0231*

(0.00891) (0.00930)

Out-Over # In-Over

4th

-0.0267* -0.0326**

(0.0120) (0.0121)

Out-Under # In-Over

4th

-0.0861* -0.0282

(0.0438) (0.0446)

In-Over # In-Over 4th -0.0236 -0.0138

(0.0134) (0.0134)



District FE Yes Yes



Figure 16: Change in 8th Grade Math and Reading Score Percentiles by Student Group and 4th Grade In-Over Status


42

Table 17: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Chronic Absenteeism in or Before 5th Grade

Absentee Math Absentee

Reading

Out-Over 0.0903*** 0.0969***

(0.00519) (0.00517)

Out-Under 0.0551*** 0.0568***

(0.0166) (0.0158)

In-Over 0.0590*** 0.0647***

(0.00553) (0.00562)

Absentee -0.0324*** -0.0377***

(0.00866) (0.00888)

Out-Over # Absentee -0.0198 0.00995

(0.0237) (0.0266)

Out-Under #

Absentee

0.0349 -0.0207

(0.0211) (0.0198)

In-Over # Absentee -0.0358 -0.00744

(0.0411) (0.0384)



District FE Yes Yes



Figure 17: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Chronic Absenteeism


43

Table 18: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Presence of a Learning Disability

LD Math LD Reading

Out-Over 0.0898*** 0.0976***

(0.00518) (0.00526)

Out-Under 0.0514** 0.0521***

(0.0158) (0.0154)

In-Over 0.0565*** 0.0627***

(0.00558) (0.00574)

Learning Disability -0.0554*** -0.0565***

(0.00418) (0.00414)

Out-Over # Learning

Disability

0.0247 -0.0112

(0.0270) (0.0244)

Out-Under #

Learning Disability

0.165* 0.160

(0.0780) (0.0835)

In-Over # Learning

Disability

0.0738* 0.0666*

(0.0332) (0.0290)



District FE Yes Yes



Figure 18: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Learning Disability


44

Table 19: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Participation in EL Programming of Fewer Than Three Years

EL Under 3

Math

EL Under 3

Reading

Out-Over 0.0868*** 0.0954***

(0.00534) (0.00514)

Out-Under 0.0635** 0.0602**

(0.0202) (0.0185)

In-Over 0.0519*** 0.0575***

(0.00634) (0.00662)

EL Under 3 Years 0.0216** 0.0246***

(0.00689) (0.00714)

Out-Over # EL

Under 3 Years

0.0219 0.0141

(0.0114) (0.0128)

Out-Under # EL

Under 3 Years

-0.0205 -0.00616

(0.0413) (0.0321)

In-Over # EL Under

3 Years

0.0259* 0.0278*

(0.0128) (0.0129)



District FE Yes Yes



Figure 19: Change in 8th Grade Math and Reading Score Percentiles by Student Group and Participation in EL Programming for Fewer Than Three Years


45

References

American Civil Liberties Union. Your Right to Equality in Education. Accessed April 1, 2017.

https://www.aclu.org/other/your-right-equality-education.

American Institutes for Research. 2012. National Evaluation of Title III Implementation -

Report on State and Local Implementation. Accessed April 1, 2017.

https://www2.ed.gov/rschstat/eval/title-iii/state-local-implementation-report.pdf.

Babal, Chris, Shiyao Cao, Angela Filer, Isaac Hedtke, and Keo Lo. 2015. "Analysis of English

Language Achievement among Wisconsin English Learners." May. Accessed April 1,

2017. http://www.lafollette.wisc.edu/research/publications/analysis-of-english-language-

achievement-among-wisconsin-english-language-learners.

Burgos, Laurie, interview by Richelle Andrae. 2017. April 3.

Chingos, Matthew. 2016. “No more free lunch for education policymakers and researchers.”

Brookings Institute, June 30.

Duncan, Greg J., and Richard J. Murnane. 2011. Whither Opportunity? Rising Inequality,

Schools, and Children's Life Chances. New York City: Russell Sage Foundation.

Harris, Elizabeth A. 2015. "Dual-Language Programs Are on the Rise, Even for Native English

Speakers." The New York Times, October 8.

Hosford, Sonya D., Christina Mokhtar, and Carrie Sampson. 2013. "Nevada's English Learner

Population: A Review of Enrollment, Outcomes, and Opportunities." Lincy Institute,

University of Nevada, Las Vegas.

Johnson, Jacqueline S. 1989. "Critical period effects in second language learning: The

influence of maturational state on the acquisition of English as a second language."

Cognitive Psychology 21(1): 60-99.

Kuncel, Nathan R., and Sarah A. Hezlett. 2007. "Standardized Tests Predict Graduate Students'

Success." Science 351(5815): 1080-1081.

Ladson-Billings, Gloria. 2016. "From the Achievement Gap to the Education Debt:

Understanding Achievement in U.S. Schools." Educational Researcher 35(7): 3-12.

Madison Metropolitan School District. 2014. Bilingual Program Model Criteria to be Met in

Order to Qualify for Bilingual-Bicultural Aid. Accessed April 1, 2017.

https://multilingual.madison.k12.wi.us/files/esl/dpi-guidance-on-bilingual-bicultural-

programs.pdf.

Madison Metropolitan. 2017. "Dual Language Immersion." Accessed April 3, 2017.

https://multilingual.madison.k12.wi.us/dual-language-immersion.

https://www.aclu.org/other/your-right-equality-education

https://www2.ed.gov/rschstat/eval/title-iii/state-local-implementation-report.pdf

http://www.lafollette.wisc.edu/research/publications/analysis-of-english-language-achievement-among-wisconsin-english-language-learners

http://www.lafollette.wisc.edu/research/publications/analysis-of-english-language-achievement-among-wisconsin-english-language-learners

https://multilingual.madison.k12.wi.us/files/esl/dpi-guidance-on-bilingual-bicultural-programs.pdf

https://multilingual.madison.k12.wi.us/files/esl/dpi-guidance-on-bilingual-bicultural-programs.pdf

https://multilingual.madison.k12.wi.us/dual-language-immersion

46

McKeon, Denise. 2005. Research Talking Points on English Learners. June. Accessed April 1,

2017. http://www.nea.org/home/13598.htm.

National Center for Education Statistics. 2015. "School Composition and the Black-White

Achievement Gap." Accessed April 1, 2017.

https://nces.ed.gov/nationsreportcard/subject/studies/pdf/school_composition_and_the_bw

_achievement_gap_2015.pdf.

National Center for Education Statistics. 2016. "English Learners in Public Schools." May.

Accessed April 1, 2017. https://nces.ed.gov/programs/coe/indicator_cgf.asp.

National Clearinghouse for English Language Acquisition. "Title III State Profiles -

Wisconsin." Accessed April 2, 2017. http://ncela.ed.gov/t3sis/Wisconsin.php.

Ovando, Carlos J. 2003. "Bilingual Education in the United States: Historical Development

and Current Issues." The Journal of the National Association for Bilingual Education

27(1): 1-24.

Shanebrook-Smith, Sarah, interview by Richelle Andrae. 2017. March 16.

U.S. Department of Education. 2016. Non-Regulatory Guidance: English Learners and Title

III of the Elementary and Secondary Education Act (ESEA), as amended by the Every

Student Succeeds Act (ESSA). Accessed April 1, 2017.

https://www2.ed.gov/policy/elsec/leg/essa/essatitleiiiguidenglishlearners92016.pdf.

U.S. Department of Education. 2016. "Race and National Origin Discrimination" Accessed

April 1, 2017. https://www2.ed.gov/about/offices/list/ocr/frontpage/faq/race-origin.html.

U.S. Department of Education. 2010. "Title III Accountability: Behind the Numbers." May.

Accessed April 1, 2017. https://www2.ed.gov/rschstat/eval/title-iii/behind-numbers.pdf.

WIDA Consortium. 2011. ACCESS for ELs, Interpretive Guide for Score Reports. Accessed

April 1, 2017.

https://www.wida.us/assessment/ACCESS/ScoreReports/ACCESS_Interpretive_Guide11.p

df

WIDA Consortium. 2014. Mission & WIDA Story. Accessed April 1, 2017.

https://www.wida.us/aboutus/mission.aspx.

WIDA Consortium. 2016. Changes are Coming to ACCESS for ELLS 2.0 Scores. Accessed

April 30, 2017.

https://www.wida.us/Assessment/ACCESS%202.0/documents/WIDA_flyer_standards_sett

ing.pdf.

http://www.nea.org/home/13598.htm

https://nces.ed.gov/nationsreportcard/subject/studies/pdf/school_composition_and_the_bw_achievement_gap_2015.pdf

https://nces.ed.gov/nationsreportcard/subject/studies/pdf/school_composition_and_the_bw_achievement_gap_2015.pdf

https://nces.ed.gov/programs/coe/indicator_cgf.asp

http://ncela.ed.gov/t3sis/Wisconsin.php

https://www2.ed.gov/policy/elsec/leg/essa/essatitleiiiguidenglishlearners92016.pdf

https://www2.ed.gov/about/offices/list/ocr/frontpage/faq/race-origin.html

https://www2.ed.gov/rschstat/eval/title-iii/behind-numbers.pdf

https://www.wida.us/assessment/ACCESS/ScoreReports/ACCESS_Interpretive_Guide11.pdf

https://www.wida.us/assessment/ACCESS/ScoreReports/ACCESS_Interpretive_Guide11.pdf

https://www.wida.us/aboutus/mission.aspx

https://www.wida.us/Assessment/ACCESS%202.0/documents/WIDA_flyer_standards_setting.pdf

https://www.wida.us/Assessment/ACCESS%202.0/documents/WIDA_flyer_standards_setting.pdf

47

Winkleby, Marilyn A., Darius E. Jatulis, Erica Frank, and Stephen P Fortmann. 1992.

"Socioeconomic status and health: how education, income, and occupation contribute to

risk factors for cardiovascular disease." American Journal of Public Health 816-820.

Wisconsin Department of Public Instruction. 2009. "ESEA Information Update - Initial

Identification and Placement of English Learners." September 25. Accessed April 1, 2017.

https://dpi.wi.gov/sites/default/files/imce/esea/pdf/bul_0701.pdf.

Wisconsin Department of Public Instruction. 2011. "ESEA Information Update." December

15. Accessed April 1, 2017.

https://dpi.wi.gov/sites/default/files/imce/esea/pdf/bul_0702.pdf.

Wisconsin Department of Public Instruction. 2015. Report on the Status of Bilingual-Bicultural

Education Programs in Wisconsin, Chapter 115, Subchapter VII, s. 115.996, Wis. Stats.

Content and Learning Team. December. Accessed April 1, 2017.

https://dpi.wi.gov/sites/default/files/imce/english-learners/pdf/2013-14-legislator-

report.pdf.

Wisconsin Department of Public Instruction. 2016. "Wisconsin Public Schools at a Glance."

May. Accessed April 1, 2017.

https://dpi.wi.gov/sites/default/files/imce/eis/pdf/schools_at_a_glance.pdf.

Wisconsin Department of Public Instruction. 2017a. "High School Completion Results."

Wisconsin Information System Data Dashboard. Accessed April 1, 2017.

http://wisedash.dpi.wi.gov/Dashboard/portalHome.jsp.

Wisconsin Department of Public Instruction. 2017b. "Report on the Status of Bilingual-

Bicultural Education Programs in Wisconsin, Chapter 115, Subchapter VII, s. 115.996,

Wis. Stats." Accessed April 2, 2017. https://dpi.wi.gov/sites/default/files/imce/english-

learners/pdf/2013-14-legislator-report.pdf.

Wisconsin Department of Public Instruction. 2017c. Accountability for English Learners (ELs).

Accessed April 1, 2017. https://dpi.wi.gov/accountability/ELamao.

Wisconsin Department of Public Instruction. 2017d. Badger Exam 3-8: Wisconsin's Smarter

Balanced Assessment. Accessed April 1, 2017.

https://dpi.wi.gov/assessment/historical/smarter.

Wisconsin Department of Public Instruction. 2017e. Bilingual/ESL Program. Accessed April

2, 2017. https://dpi.wi.gov/english-learners.

Wisconsin Department of Public Instruction. 2017f. Formula Allocation Spreadsheets.

Accessed May 1. https://dpi.wi.gov/esea/historical/funding/application#Formula Allocation

Spreadsheets

https://dpi.wi.gov/sites/default/files/imce/esea/pdf/bul_0701.pdf

https://dpi.wi.gov/sites/default/files/imce/esea/pdf/bul_0702.pdf

https://dpi.wi.gov/sites/default/files/imce/english-learners/pdf/2013-14-legislator-report.pdf


https://dpi.wi.gov/sites/default/files/imce/eis/pdf/schools_at_a_glance.pdf

http://wisedash.dpi.wi.gov/Dashboard/portalHome.jsp






https://dpi.wi.gov/accountability/ellamao

https://dpi.wi.gov/accountability/ellamao

https://dpi.wi.gov/assessment/historical/smarter



https://dpi.wi.gov/english-learners

https://dpi.wi.gov/english-learners

48

Wisconsin Department of Public Instruction. 2017g. Reclassification of English Learners.

Accessed April 25, 2017. https://dpi.wi.gov/english-learners/title-iii/reclassification

Wisconsin Department of Public Instruction. 2017h. Wisconsin Bilingual-Bicultural Programs.

Accessed April 3, 2017. https://dpi.wi.gov/english-learners/bilingual-bicultural

Wisconsin Department of Public Instruction. 2017i. Wisconsin Forward Exam. Accessed April

1, 2017. https://dpi.wi.gov/assessment/forward.

Wisconsin Department of Public Instruction. 2017j. Wisconsin Information System for

Education Data Dashboard. Accessed April 2, 2017. http://wisedash.dpi.wi.gov/.

Wisconsin Department of Public Instruction. 2017k. WKCE. Accessed April 1, 2017.

https://dpi.wi.gov/assessment/historical/wkce

https://dpi.wi.gov/english-learners/title-iii/reclassification

https://dpi.wi.gov/english-learners/bilingual-bicultural

https://dpi.wi.gov/english-learners/bilingual-bicultural

https://dpi.wi.gov/assessment/forward

https://dpi.wi.gov/assessment/forward

http://wisedash.dpi.wi.gov/

http://wisedash.dpi.wi.gov/




academic achievement among english learners … achievement among english learners (els) in...

Documents