Evaluation of theMoving to Work Program

of the Housing Authority of Champaign

County

Department of Agricultural and Consumer EconomicsUniversity of Illinois

at Urbana-Champaign

Dr. Paul E. McNamara, PI

1

Department of Agricultural and Consumer Economics

Research Team

• Paul E. McNamara, MPP, Ph.D., Associate Professor, and principal investigator, supervising the research study.

• Cathy Strick, MSW, Visiting Research Coordinator with a Master of Social Work degree from the University of Illinois.

• Han Lee, BA, an advanced Ph.D. student in the Department of Agricultural and Consumer Economics.

2

• Operational Efficiency through Innovation

• Streamline business processes and implement advanced technological solutions. . . resulting in operational cost efficiencies and enabling resource reallocation.

• Self-Sufficiency

• Provide alternate incentives . . . to motivate families to actively seek financial independence and transition from dependency on housing subsidy. Carefully measure . . . each incentive to identify and replicate the greatest motivators.

• Expand Housing Opportunities through Repositioned Assets

• Maximize HACC’s economic viability and sustainability through repositioning the current real estate portfolio and development of new affordable housing opportunities.

3

Housing Authority of Champaign CountyMoving to Work Program Goals

Evaluation Overview

• The purpose of this evaluation is to measure the impact on households participating in the Housing Authority of Champaign County Moving to Work program, in terms of family self-sufficiency measured by income, employment, attainment of educational and job skills goals, and other variables related to family self-sufficiency.

• The evaluation is designed to: • quantitatively measure potential impacts and identify

factors that might be related to non-attainment of the program’s self-sufficiency goals;

• provide a long-term evaluation that measures program impacts over time.

4

Sources of Evaluation Information

We utilize four sources of information:

1. In-depth qualitative interviews with key informants from HACC and Decatur Housing Authority (DHA) program participants and HACC waitlist participants, which provide participants’ perspectives on self-sufficiency, work, stress, housing and neighborhood issues, education and job skills, family issues and overall quality of life.

2. The quantitative social survey we have fielded with MTW participants, and waitlist participants, and with DHA participants, and includes question domains such as education, housing choices, neighborhood issues, employment history, current work situation, family structure and social supports.

3. Administrative data available from HACC and HUD.

4. Treatment Plan data from HACC which outlines participants’ goals about steps they will take toward employment or education.

5

Evaluation Analytic Approach

6

• Track qualitative interview responses over time, identifying emerging themes that differ between Champaign and Decatur, and for Champaign that focus on program goals of self-sufficiency – work, education gains.

• Analyze social survey data, testing for statistically significant differences that can be attributed to program participation on, for example, employment and education levels.

• Utilize administrative data for statistical analyses of changes over time in income, employment, and education levels. Administrative data allows for matched controls at the individual level with public housing participants from other similar downstate cities (for example, Decatur IL).

• Describe Treatment Plan data and analyze the nature of Treatment Plans as baseline from which to make comparisons with regard to adherence and successful completion of Plans.

Quantitative Approach:Double Difference (DD),

Pre & Post with Comparison Group

• Using pre-MTW data and data from implementation period.

• Using data on HACC MTW participants and non-MTW public housing program participants from other housing authorities (Decatur, planned Bloomington-Normal).

• Assess the program impact through descriptive statistics and multivariate statistical analyses.• Double difference – pre- and post, with treated and non-

treated group. Can match Champaign participants with Decatur non-participants.

7

Evaluation Analytic Approach

Mean Sample Characteristics

8

 

Year

Champaign (n=436)

  Decatur (n=509)

2012 2013 2012 2013

Hsld head age 35 36   32 33Hsld head Race .85 .85   .82 .82Hsld size 3.49 3.46   3.00 3.04# of young children (below age of 6)

.51 .54   .77 .72

9

Data Collection Summary

• Baseline, Year 1

• 308 HACC and 162 DHA program participants completed a quantitative Housing & Self-Sufficiency Social Survey.

• 25 HACC and 24 DHA program participants completed a qualitative Key Informant Interview.

• In Year 2

• 213 HACC and 109 DHA of the same program participants completed a quantitative Housing & Self-Sufficiency Social Survey.

• Baseline, Year 2

• 145 HACC program and waitlist participants and 13 DHA participants completed a quantitative Housing & Self-Sufficiency Social Survey.

• 16 HACC waitlist participants completed a qualitative Key Informant Interview.

10

Data Challenges

• Quantitative Survey Completion Response• HACC participants completed 308 Surveys during

baseline, Wave 1 but only 213 of these same participants completed a Survey during Year 2, reflecting a completion rate of about 74%, adjusting for Surveys that were returned undeliverable.

• Administrative Data• Data quality and the issue of disaggregating income.

• Qualitative Data• The volume of qualitative key informant interviews.

HACC Participant Turnover

2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 20120

200

400

600

800

1000

1200

1400

1600

1800

All 2002MTW 2002All 2012MTW 2012

Source: HACC administrative data from Tenmast 50058 ‘Family Report’.

Housing & Self-SufficiencySocial Survey

Comparison of Food Security between Champaign and Decatur

12

  HACC DHA

Number of observations  301 % 162 %

Food Security*Food Secure(Raw score 0-1)Food insecure without hunger(Raw score 2-4)Food insecure with hunger(Raw score 5-6)

 118

 64 

76

 39.20

 21.26

 25.25

 68 

27 

43

 41.98

 16.67

 26.54

Percentage may not add to zero due to missing observations.*USDA 6 Item Food Security ScaleSource: 2012 Social Survey Data

Housing & Self-SufficiencySocial Survey

Comparison of Depression between Champaign and Decatur

13

  HACC DHA

Number of observations  301 % 162 %

CES-D*No symptoms

Mild to moderate

Possible major depression

 147

34

65

 48.84

11.30

21.59

 78

18

39

 48.15

11.11

24.07

Percentage may not add to zero due to missing observations.*CES-D - Center for Epidemiologic Studies Depression ScaleSource: 2012 Social Survey Data

MTW Effect on Employment Status

Source: HACC and DHA administrative data.

MTW Effect on Employment Status

Source: HACC and DHA administrative data.

byHanbum Lee

Paul E. McNamara

A paper selected to present at the Midwest Economics Association (MEA)

• Our paper investigates the relationship of mental health and unemployment, and analyzes possible initiatives to improve the mental health of subsidized housing recipients who have lost jobs or who experience various kinds of underemployment.

• We utilize the Generalized Method of Moments (GMM) in our econometric approach, using proximity variables that capture spatial variation in the demand for labor of a low-skill service job, and child care service as instrumental variables to correct for the endogenous regressor (employment status).

Effects of Unemployment on Mental Health: Evidence from Subsidized Housing Recipients

Descriptive Statistics

Preliminary Results

• A strong direct link between unemployment and higher levels of depression.

• Public housing recipients with lower numbers of family and friends (smaller social support network), older people, as well as people in more crowded housing units, have a greater level of depression.

• These results may provide a framework and motivation for targeting people who are at greater risk of depression in the public housing programs, which might affect their ability to adhere to educational and work requirements being implemented in the Moving to Work Program.

18

Conclusion

• We are using a mixed methods approach to evaluate the overall well-being and self-sufficiency of housing program and waitlist participants.

• We are faced with data challenges including data quality and volume of qualitative data.

• Evaluating this MTW initiative provides opportunity for not only program evaluation of the HACC but also academic research.

19