Moving to Opportunity TOWN HALL

Abt Associates, June 26, 2003

Background and Challenges: Judie Feins Findings: Robin Jacob Implications: Larry Orr

Abt Associates Inc.

2

WHY MTO?

Program Objective: Provide poor families living in high-poverty public or assisted housing with the opportunity to move to low-poverty neighborhoods with a Section 8 rent subsidy and counseling.

Research Objective: Measure the impact of moving to lower poverty neighborhoods on the outcomes of adults and children.

3

Precursors of MTO

1) Long history of research showing harmful effects of bad neighborhoods

2) Chicago’s Gautreaux Program: court-ordered mobility of poor, African-American families seemed to show positive effects of suburban life—especially on the children

3) Trend of increasingly concentrated poverty in American cities 1980-1990 suggested growing harm, unless a remedy could be found to reverse these neighborhood effects.

4

The Demonstration Program

The MTO program ran from 1994 to1998 in five cities: Baltimore, Boston, Chicago, L.A. and N.Y.

Families with children were recruited from public and assisted housing in concentrated-poverty neighborhoods. Many of the housing developments were very distressed and dangerous places.

Any resident who wanted the chance to get a mobile housing subsidy and move out could sign up, as long as the family met Section 8 eligibility rules.

Some 5,300 families volunteered; 4,608 were found eligible.

5

The Research Design

For research purposes, families were randomly assigned into one of three groups:

The experimental group received special Section 8 vouchers that could be used only in census tracts with poverty rates below 10 percent. Nonprofit counseling agencies in each city helped the experimental group families to locate and lease suitable housing in low-poverty areas.

The Section 8 group received regular Section 8 vouchers, which could be used anywhere they found a suitable unit with a willing landlord and rent below the program cap. These families did not receive any mobility counseling.

The control group received no vouchers but continued to be eligible for project-based assistance.

CBS News Correspondent John Roberts. From Ghetto To White Picket Fence

BALTIMORE6/5/2000

 

11/17/2000

The Los Angeles Times

Housing, Not School, Vouchers Are Best Remedy for Failing Schools - Jan 31, 1999, Larry Cuban

A Social Experiment in Pulling Up Stakes; Aid: Does neighborhood affect economic and school success? Five cities relocate poor families to find out.- Sep 23, 1997; pg. 1, Larry Gordon

A Fresh Start Housing: The Moving to Opportunity program will take families out of the projects to see if a new environment helps them succeed. Nov 8, 1994; pg. 1, Larry Gordon

Washington Post

" In Baltimore, Getting a Lease on Middle-Class Life. " May 10, 2000; pg. 1 Amy Goldstein.

New York Times

" Better Than a Voucher, a Ticket to Suburbia. " October 18, 2000. Richard Rothstein

Chicago Tribune• "Foes kill housing plan funds" - Dec 15, 1994; Laurie Abraham• "Where Should Poor Families Live?" - Jul 23, 1994; Lori Montgomery• "Hostility Toward Relocating the Poor is a Matter of Race" - Apr 27, 1994; Clarence Page

7

A Long and Challenging Effort

8

THE EARLY CHALLENGES

Abt Associates began working on the MTO design with HUD in September 1993. Our initial job:

Convincing HUD to shift to a 3-group design

Developing uniform (but flexible) procedures

Managing implementation (started July 1994)

Surviving a political fire-storm

Quietly growing the demonstration

Finishing intake and lease-up (March 1999).

9

AND THEN…

Database construction: We designed and built sophisticated relational database, which continues to evolve;

Sample tracking: We kept up with 4,608 families (19,000+ persons) by active and passive means, despite high mobility rates and the disruption of public housing revitalization and demolition.

10

CHALLENGES OF THE INTERIM EVALUATION

Research team: Abt partnered with a group of academics, the Urban Institute, and two data subcontractors to win the interim evaluation contract in July 2000.

Broad scope: HUD’s interest extended well beyond the usual housing and neighborhood issues to education, health, delinquency, employment and earnings, income and self-sufficiency.

Complex data collection: The study required that several kinds of data be gathered from almost 11,000 sample members in a compressed period.

11

Evaluation Team

ANALYSTS FROM:

Abt Associates (design, implementation, data collection, impact analysis)

Urban Institute (qualitative analysis)

National Bureau of Economic Research (impact analysis)

Georgetown University (collection and analysis of crime data)

12

Interim Evaluation Funders

• U.S. Department of Housing & Urban Development

• Grants via National Bureau of Economic Research National Institute for Child and Health Development (NICHD) National Science Foundation (NSF) National Institute of Mental Health (NIMH) Robert Wood Johnson Foundation Smith Richardson Foundation Russell Sage Foundation W.T. Grant Foundation Spencer Foundation MacArthur Foundation

• Grants via Georgetown University National Consortium on Violence Research (NSF) Brookings Institution

13

Interim Evaluation Data Sources

Pre-Survey Qualitative Data In-depth semi-structured interviews with adult heads of

household and children informed survey design.

Quantitative Data Structured surveys with adults, teens, and children

Achievement tests of teens and children

Measurement of adult blood pressure, child height and weight

Administrative data from state and local agencies (earnings, TANF, food stamps, arrest records)

14

Interim Evaluation Surveys

Full Sample (adults, youth, children) Fielded from January through June 2002.

Achieved survey response rates of 80% of adults, 77% of children, 76% of youth.

3-in-10 Sub-sample Random draw late in June from non-complete cases.

Focused field resources on a sub-set of hard-to-find cases; included travel to remote sites.

Increased response rates considerably and reduced the risk of non-response bias.

15

Interim Evaluation Response Rates

High Effective Response Rates Attained a weighted adult response rate of 90 percent.

Attained youth/child surveys and achievement test response rates of 86 to 90 percent.

Group Differences Difference in response rates between random assignment

groups was less than 1 percent

Very low risk of non-response bias between groups

16

Interim Evaluation Findings

What did we learn?

17

Estimation Methods

All estimates regression-adjusted with standard set of covariates, including (where available) pre-RA value of outcome

ITT = “intention to treat” – impact on entire treatment group, including those who did not lease up

TOT = “treatment on treated” – impact on those who leased up only

Tests of significance at .05 level

18

Mobility Outcomes

Poverty rates of current locations are substantially reduced (entire experimental and Section 8 groups)

Fraction minority population in experimental group locations is reduced, but more than half moved to areas 80%+ minority

Almost half of experimental lease-ups were in tracts with increasing poverty from 1990 to

2000

More than half of experimental group lease-ups moved again to somewhat higher

poverty tracts

19

Impacts on Neighborhood Outcomes

Significant positive impacts for both experimental and Section 8 groups on:

Feeling safe in the neighborhood (day and night)

Police coming when called

All measures of neighborhood quality

Significant reductions for both experimental and Section 8 groups in:

Witnessing drug activity in the neighborhood

See public drinking, groups hanging out

Crime victimization over last six months

20

Impacts on Housing Outcomes

Significant increases for both experimental and Section 8 groups in:

Most measures of housing quality

Utility payment problems

Prevalence of housing assistance receipt

No significant impacts on current total housing cost

21

A Focus on Educational Outcomes

The following factors were hypothesized to be mediators of educational outcomes:

Community-Level

Quality of Schools

Community Norm and Values

Social and Physical Environment

Student and Family-Level

Parent Attitudes and Behaviors

Student Attitudes and Behaviors

22

School Characteristics

Modest improvements for both groups

70% of Experimental lease-ups remained in the same large urban school district

C Mean E-C S8-C

Percent free lunch .720 -.133 -.058

Percent white

Percent limited English proficient

.090

.180

.100

-.063

.046

ns

Pupil-teacher ratio 14.6 ns ns

Percentile rank on State Assessments

.170 .091 .026

23

School Climate

No significant effects on school climate

C Mean E-C S8-C

There is a lot of cheating in tests & assignments

.416 ns ns

Discipline in school is fair .724 ns ns

Disruptions from other students inhibit learning

.640 ns ns

Child feels safe in school .775 ns ns

Teachers are interested in students .810 ns ns

24

Other mediators of educational outcomes

Community-Level Mediators Significant impacts on social and physical environment

Some evidence of impact on community norms and values but not on peer role models

Some evidence of positive impact on economic opportunities but not

on earnings or employment of sample adults.

Student- and Family-Level Mediators No significant effects on parental monitoring

No significant effects on parental involvement in school

No significant effects on student school-related behaviors

25

Impacts on Education Outcomes

No significant impact on student achievement

No significant impacts on grades, coursework, special ed placement, graduate rates or college attendance.

26

Other Outcomes: Health

Adult Physical and Mental Health

Physical health & substance use: largely insignificant impacts

Mental Health: significant E-C impacts

Obesity: significant E-C impacts

Youth Physical and Mental Health

Physical health: no significant impacts

Mental health: improvement for girls

27

Other Outcomes: Youth Behavior

Youth Delinquency & Risky Behavior

Behavior Problems Index: significant increase in self-reported behavior problems among boys

Delinquency: no significant impact

Arrests & Risky Behavior: substantial gender differences

Employment and School Attendance

No significant effects for boys, but an increase in full-time school attendance for girls in the experimental group and a reduction in full-time employment for girls in Section 8

28

Other outcomes: Income & Earnings

Receipt of Public Assistance

No significant impacts on current receipt of public assistance, either for full sample or subgroups by ethnicity and barriers to employment

Household Income & Poverty Status

No significant impacts on income, poverty, food security and self-sufficiency

29

Implications for Policy

30

Does this mean there are no impacts on these outcomes?

On many outcomes, only fairly large impacts can be detected with confidence—e.g., to be 80% sure of detecting impacts as significant:

Adult earnings would have to be increased by about 40% in the experimental group, 30% in the Section 8 group

TANF benefits would have to be reduced by 50%

Youth asthma attacks would have to be reduced by 67%

The fact that an impact estimate is not statistically significant does not mean there was no impact—it

means we don’t know if there was an impact.

31

Policy Implications - Context

The worst concentrations of urban poverty are usually HUD-subsidized.

In US, high-poverty neighborhood almost always means high-crime neighborhood.

There are legitimate concerns about “relocating the ghetto”. These might lead to restricted mobility programs.

32

Will mobility programs work?

Can voluntary restricted mobility programs move extremely low-income public housing tenants to middle-class neighborhoods?

Yes—at least, for 48%.

However, geographic restrictions come at some cost to lease-up (60% of Section 8 comparison group moved)

Those who move are likely to follow the “path of least resistance.” In MTO lease-ups were predominantly in minority neighborhoods, where poverty rates trending up.

33

Who benefits, and for how long?

Those who move enjoy substantially better housing, safer neighborhoods, less stress, better mental health, and lower obesity. There is some indication that girls behavior and motivation to succeed improves.

These benefits are still significant 4-7 years after the initial move.

Benefits to the rest of society are less clear. There is some indication of lower rates of criminal behavior among girls, but higher rates of crime and delinquency among boys. There are no reductions in welfare costs.

34

What do these results imply about choice of policies for improving the lives of low-income families?

Are the problems of low-income families environmental, or due to attributes of the families themselves?

These results imply that a change in environment can improve the family’s sense of well-being directly, and have some salutary effects on the behavior of youth.

But physical health, educational performance and attainment, employment, earnings, and welfare dependence do not appear to be sensitive to residential environment, at least within 4-7 years.

To improve these outcomes within that time frame requires policies designed to deal directly with these specific problems—such as educational improvements, employment and training, or welfare-to-work programs.

35

Dr. Larry Orr, Abt Associates

Dr. Judith Feins, Abt Associates

Dr. Jeffrey Kling, Princeton University

Dr. Jens Ludwig, Georgetown University

Dr. Robin Tepper Jacob, Abt Associates

Dr. Barbara Goodson, Abt Associates

Dr. Lisa Sanbonmatsu, NBERDr. Lawrence Katz, Harvard University

Dr. Jeffrey Liebman, Harvard Univsersity

Dr. Erik Beecroft, Abt Associates

Dr. Rhiannon Patterson, Abt Associates

Dr. Alvaro Cortes, Abt Associates

Ms. Carissa Climaco, Abt Associates

Ms. Debi McInnis, Abt Associates

Mr. Robert Teitel, Abt Associates

Dr. Susan Popkin, Urban Institute

Analysts: