tatiana melguizo & bo kim (usc) hans bos (american institutes for research)
DESCRIPTION
Using a Regression Discontinuity Design to Estimate the Impact of Placement Decisions in the Los Angeles Community College District. Tatiana Melguizo & Bo Kim (USC) Hans Bos (American Institutes for Research) George Prather (LACCD, Retired) - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Tatiana Melguizo & Bo Kim (USC) Hans Bos (American Institutes for Research)](https://reader035.vdocuments.site/reader035/viewer/2022081520/56815c46550346895dca45d1/html5/thumbnails/1.jpg)
Using a Regression Discontinuity Design to Estimate the Impact of
Placement Decisions in the Los Angeles Community College
District
Tatiana Melguizo & Bo Kim (USC)Hans Bos (American Institutes for Research)
George Prather (LACCD, Retired)
Presented at the Research and Planning Group conference, April 2, 2013
This research is funded by a grant from the U.S. Department of Education’s Institute of Education Sciences (IES).
![Page 2: Tatiana Melguizo & Bo Kim (USC) Hans Bos (American Institutes for Research)](https://reader035.vdocuments.site/reader035/viewer/2022081520/56815c46550346895dca45d1/html5/thumbnails/2.jpg)
The Problem
• Community college students have widely varying initial skill levels and the majority arrive with substantial remedial needs in math
• Colleges have to offer classes to meet the needs of their diverse students and have to keep heterogeneity in the classrooms manageable
• Placing students incorrectly can reduce the likelihood that students succeed
![Page 3: Tatiana Melguizo & Bo Kim (USC) Hans Bos (American Institutes for Research)](https://reader035.vdocuments.site/reader035/viewer/2022081520/56815c46550346895dca45d1/html5/thumbnails/3.jpg)
Math Placement in the Los Angeles Community College
District
About sixty percent of the entering
students need at least one remedial
class before taking a math class that
counts towards a degree
![Page 4: Tatiana Melguizo & Bo Kim (USC) Hans Bos (American Institutes for Research)](https://reader035.vdocuments.site/reader035/viewer/2022081520/56815c46550346895dca45d1/html5/thumbnails/4.jpg)
Some Controversy about Impact of Remedial Education
• Some research finds that remedial education provides the preparation necessary for students to succeed in college (Boylan, Bliss, & Bonham, 1994; 1997; Lazarik, 1997)
• Critics contend that remedial education may hold students back and be ineffective (Calcagno & Long, 2008; Martorell & McFarland, 2011)
• We argue that it depends on how students are placed and how cut points between levels are set
![Page 5: Tatiana Melguizo & Bo Kim (USC) Hans Bos (American Institutes for Research)](https://reader035.vdocuments.site/reader035/viewer/2022081520/56815c46550346895dca45d1/html5/thumbnails/5.jpg)
Focus on Cut Points
• Math faculty set the cut points between the different levels based on who applies and how their course offerings are distributed
• If the cut points are too high, too many students languish in remedial courses
• If the cut points are too low, too many students fail higher-level courses and present a challenge to the instructors
• Getting the cut points just right is important
![Page 6: Tatiana Melguizo & Bo Kim (USC) Hans Bos (American Institutes for Research)](https://reader035.vdocuments.site/reader035/viewer/2022081520/56815c46550346895dca45d1/html5/thumbnails/6.jpg)
Changing Cut Scores
![Page 7: Tatiana Melguizo & Bo Kim (USC) Hans Bos (American Institutes for Research)](https://reader035.vdocuments.site/reader035/viewer/2022081520/56815c46550346895dca45d1/html5/thumbnails/7.jpg)
Different Pathways to Success(Arithmetic vs. Pre Algebra)
Test
Placed in Pre-
Algebra
Placed in Arithmeti
c
Enroll in Pre-
Algebra
Enroll in Arithmetic
No enrollme
nt
Success
Failure
Success
Failure
Next course
![Page 8: Tatiana Melguizo & Bo Kim (USC) Hans Bos (American Institutes for Research)](https://reader035.vdocuments.site/reader035/viewer/2022081520/56815c46550346895dca45d1/html5/thumbnails/8.jpg)
Success Above Cut Point
Test
Placed in Pre-
Algebra
Placed in Arithmeti
c
Enroll in Pre-
Algebra
Enroll in Arithmetic
No enrollme
nt
Success
Failure
Success
Failure
Next course
Direct but academically challenging
![Page 9: Tatiana Melguizo & Bo Kim (USC) Hans Bos (American Institutes for Research)](https://reader035.vdocuments.site/reader035/viewer/2022081520/56815c46550346895dca45d1/html5/thumbnails/9.jpg)
Success Below Cut Point
Test
Placed in Pre-
Algebra
Placed in Arithmeti
c
Enroll in Pre-
Algebra
Enroll in Arithmetic
No enrollme
nt
Success
Failure
Success
Failure
Next course
A less academically challenging but more time-consuming trajectory
![Page 10: Tatiana Melguizo & Bo Kim (USC) Hans Bos (American Institutes for Research)](https://reader035.vdocuments.site/reader035/viewer/2022081520/56815c46550346895dca45d1/html5/thumbnails/10.jpg)
The Impact Question
• For a student at the margin, does the longer road to success produce better results?
• Is greater likelihood of success worth the extra time and effort?
The Policy Question• Are cut points between math placement
options set correctly?
![Page 11: Tatiana Melguizo & Bo Kim (USC) Hans Bos (American Institutes for Research)](https://reader035.vdocuments.site/reader035/viewer/2022081520/56815c46550346895dca45d1/html5/thumbnails/11.jpg)
Using Regression Discontinuity to Evaluate Placements
• Regression discontinuity analysis is the strongest non-experimental method to estimate causal effects
• It depends on a continuous forcing variable and an exogenously established cut point
• Those two conditions are present in this situation
![Page 12: Tatiana Melguizo & Bo Kim (USC) Hans Bos (American Institutes for Research)](https://reader035.vdocuments.site/reader035/viewer/2022081520/56815c46550346895dca45d1/html5/thumbnails/12.jpg)
Out
com
e
Placement Test Score
Regression Line
Arithmetic
Pre-Algebra
Impa
ct
![Page 13: Tatiana Melguizo & Bo Kim (USC) Hans Bos (American Institutes for Research)](https://reader035.vdocuments.site/reader035/viewer/2022081520/56815c46550346895dca45d1/html5/thumbnails/13.jpg)
Analytical Details
• Not everyone follows the placement test’s recommendation
• Some students enroll below or above their placement level
• Compliance was high
• Impacts are estimated using discrete time survival analysis implemented within a regression discontinuity framework
• The density of the forcing variable around the cut point was normal (confirmed with McCrary test)
![Page 14: Tatiana Melguizo & Bo Kim (USC) Hans Bos (American Institutes for Research)](https://reader035.vdocuments.site/reader035/viewer/2022081520/56815c46550346895dca45d1/html5/thumbnails/14.jpg)
The Impact of Placement Decisions Needs to be Observed
Over Time
![Page 15: Tatiana Melguizo & Bo Kim (USC) Hans Bos (American Institutes for Research)](https://reader035.vdocuments.site/reader035/viewer/2022081520/56815c46550346895dca45d1/html5/thumbnails/15.jpg)
The Impact of Placement Decisions Needs to be Observed
by College
![Page 16: Tatiana Melguizo & Bo Kim (USC) Hans Bos (American Institutes for Research)](https://reader035.vdocuments.site/reader035/viewer/2022081520/56815c46550346895dca45d1/html5/thumbnails/16.jpg)
The Impact of Placement Decisions Needs to be Observed
by Level
![Page 17: Tatiana Melguizo & Bo Kim (USC) Hans Bos (American Institutes for Research)](https://reader035.vdocuments.site/reader035/viewer/2022081520/56815c46550346895dca45d1/html5/thumbnails/17.jpg)
Students are not Being Placed Effectively in the Higher Levels
(Elementary versus Intermediate Algebra)
![Page 18: Tatiana Melguizo & Bo Kim (USC) Hans Bos (American Institutes for Research)](https://reader035.vdocuments.site/reader035/viewer/2022081520/56815c46550346895dca45d1/html5/thumbnails/18.jpg)
Students Placed in Lowest Levels Accumulate More Degree
Applicable Credits
![Page 19: Tatiana Melguizo & Bo Kim (USC) Hans Bos (American Institutes for Research)](https://reader035.vdocuments.site/reader035/viewer/2022081520/56815c46550346895dca45d1/html5/thumbnails/19.jpg)
Placement in Less Effective in Terms of Transfer Degree Credit
Accumulation
![Page 20: Tatiana Melguizo & Bo Kim (USC) Hans Bos (American Institutes for Research)](https://reader035.vdocuments.site/reader035/viewer/2022081520/56815c46550346895dca45d1/html5/thumbnails/20.jpg)
Conclusions• Cut points between elementary algebra
and intermediary algebra may be set too high as marginal students do worse in elementary algebra
• At lower levels the cut points appear to be in the right place
• For students placed in lower levels of math sequence, there is no penalty in terms of accumulating 30 degree applicable credits
• For students placed in higher levels there is a penalty in terms of completing 30 transfer credits
![Page 21: Tatiana Melguizo & Bo Kim (USC) Hans Bos (American Institutes for Research)](https://reader035.vdocuments.site/reader035/viewer/2022081520/56815c46550346895dca45d1/html5/thumbnails/21.jpg)
THANK YOU!Questions
Tatiana [email protected]
http://www.usc.edu/dept/education/rossier_faculty/tmelguizo/