what randomized trials have taught us about what works and doesn’t work in education jon baron

What randomized trials have taught us about what works and doesn’t work in

education

Jon BaronCoalition for Evidence-Based Policy

December 9, 2003

Age 0-4

Abecedarian project: High-quality, educational child care & preschool for low-income children. Randomized trial of 111 children.

At age 15, reduces special education placements and grade retentions by nearly 50%;

At age 21, more than doubles the proportion attending four-year college and reduces the percentage of teenage parents by 44%. Raises reading 1.8 grade levels, math 1.3 grade levels.

School-age treatment alone had much smaller effect.

Age 0-4

Perry Preschool Study: High-quality preschool for low-income children. Randomized trial of 123 children. At age 27 follow-up: increases percentage with high school

diploma by 31% reduces percentage on welfare by 26% Reduces percentage of hard-core criminals

by 80%.

Age 0-4

Infant Health & Development Program: Intensive child development for kids age 0-3 born prematurely, low-birthweight. At age 8, effect is only on the children heavier at

birth -- 4 point increase in IQ, 38% decrease in special education, improved math/vocabulary

Early Head Start: Child development & parenting services for low-income families with infants. At age 3, 10-15% decrease in kids scoring “at

risk” in cognitive development & receptive vocabulary, 15% decrease in mothers having subsequent births.

Age 0-4

Even Start family literacy program for low-income families. Focus is on coordinating family access to existing literacy services (e.g., Head Start).

Randomized trial of 200 families. At 18-month follow-up, no effect on literacy

outcomes of children or adults.

Grades K-6

1-on-1 tutoring of at-risk readers by trained tutors (avg tutored student reads more proficiently than ~ 75% of controls).

Instruction for early readers in phonemic awareness and phonics (the avg student in these interventions reads more proficiently than ~ 70% of controls).

Reducing class size in grades K-3 (the avg student in small classes scores higher on the Stanford Achievement Test in reading/math than 60% of controls).

Grades K-6

21st Century Community Learning Centers -- provides after-school academic and recreational activities in mostly high-poverty schools. Randomized trial of 1000 elementary school

students. At 1-yr follow-up, no effect on grades or test scores,

student effort (e.g., homework completion), or behavioral problems.

Vouchers for disadvantaged youth (K-4) for private school. Trial of >2000 children. At year 3: No overall impact in math/reading scores; Possible impact for African American students. Parents report much higher satisfaction w/ school.

Middle and High School

Between-class ability grouping in middle and high schools (students in a particular grade are grouped into separate classes by ability level and taught variations on the same curriculum). 10 randomized trials: No overall effect on

achievement.

Joplin plan (which groups students across grades by ability level and uses curricula that are fitted to each group’s ability). 2 small randomized trials: Avg student in the intervention scores higher

than ~60% of the students in the control group.

Middle and High School

Big Brothers Big Sisters (matches adult mentors with disadvantaged youths age 11-13). Randomized trial of 1100 youths. At 18-month follow-up – Reduced initiation of drug use by 46%; Reduced initiation of alcohol use by 27%; Reduced days skipped school by 52%.

Job Corps (academic & vocational training for disadvantaged youths age 16-24). Randomized trial of >10,000 youths. At 4-year follow-up: Increased earnings by 8% Decreased welfare/food stamps by 11% Reduce number arrested by 12% No effect on substance abuse or childbearing.

Middle and High SchoolUpward Bound (provides instruction, tutoring, counseling starting 9-10 grade). Randomized trial of 2800 students. At 2-3 year follow-up: No effect on high school graduation rates, or % attending

college But some modest effects on lower-income & poorer-

performing students (~ 2 high school credits).

U.S. ED’s dropout prevention programs (varying interventions for students at-risk of dropping out). Randomized trial of >10,000 students. At 2-3 yr followup: Middle schools providing supplemental services (tutorg,

classes) had no effect on dropout rates or achievement; Middle schools providing alternative schools or schools

w/in school reduced dropout rate 18 to 9%; Alternative high schools had no effect on dropout rate.

Middle and High SchoolCareer Academies (provide academic/technical courses in small communities, career theme, partnership with local employers). Randomized trial of 1700 8th/9th graders. No effect on high school graduation rate or enrollment

in post-secondary education at 4-year follow-up. Jim Kemple will discuss 8-year follow-up.

Summer Training & Employment Program (provides summer jobs & academic classes to disadvged 14-15 yr olds). Randomized trial of 2600 youths: Only short-term academic impact at end of summer. At 1-year, no effects on academic scores, dropout rate,

college attendance, teen pregnancy, employment.

Substance-Abuse Prevention Programs

Life-skills training (reduces smoking by 20% and serious levels of substance abuse by 30% by end of high school).

DARE is ineffective in reducing substance use, according to randomized trials (now being redesigned).