Twenty years of werfave studies provide a legacy of reliublejdings. New reforms will require new research strutegies and will risk unintended consequences.

Learning About Welfare Reform: Lessons from State-Based Evaluations Judith M. Guerun

Over the past twenty years, rigorous evaluations of work-focused welfare reform strategies have-produced a large body of reliable information that has played an important role in shaping social policy and program practice. From the start, this research has been unusually real-world in two senses: it was designed to answer questions posed by policymakers and program operators, and it resulted from field tests conducted in actual operating welfare offices throughout the country. Today we face a new world of welfare, where there is likely to be much more variation in state programs, creating at the same time greater opportunity for progress and a heightened risk of unintended conse- quences. As a result, the need is more acute than ever for hard information on the effects of different approaches to reweaving the safety net.

The purpose of this chapter is to summarize the origin, context, and main themes of many of the major studies conducted during this period and also to look ahead at some of the likely challenges and opportunities for further eval- uations. The author was asked to tell the Manpower Demonstration Research Corporation (MDRC) story. Thus, the lens in this chapter is focused on our work, without meaning to imply that this provides a complete picture of what was learned. Beyond that, the reader is cautioned that although she sought to be objective, the author was a direct actor in this work.

Step One: Recognizing the Need In 1974, the Ford Foundation and six federal agencies established MDRC with the goal of improving public policy for low-income people by identifying and strengthening effective programs. The intent was to provide policymakers and

NEW DIRECTIONS FOR EVALUATION, no. 76, Winter 1997 0 Jossey-Bass Publishers 79

92 PROGRESS AND FUTURE DIRECTIONS IN EVALUATION

that is the envy of other fields. In the early 1970s, it was not known whether social experiments could be used to test real-world operating programs. We now know that they can, that the results have been positive and convincing, and that the studies have mattered. But this is no reason for complacency The combination of block grants and the end of the Section 11 15 waiver process puts this approach at risk. States will surely innovate, but having a thousand flowers bloom does not mean that you will learn whether they bloom well. In the new climate, there is less money for research, states can decide not to eval- uate important innovations, and policymaking is likely to be much more polir- ical. The stakes are high for states because under the new law they will bear the full financial risk of welfare changes. This reality creates pressure to get reli- able and objective data (that is, to learn early about any unintended conse- quences and costs of state reforms), but the politicalization of the welfare debate pushes in the opposite direction. Not-in-my-backyard arguments may be transferred to the world of social policy research: studies are a good thing, but for your neighbors, not for you.

Beyond the political challenges, the magnitude of change and the likeli- hood that it will affect the full caseload make the random assignment paradigm less feasible and appropriate. Fortunately, states have chosen to complete many of the major random assignment studies launched under the 1990s waivers. This commitment to learning is impressive and important, because these stud- ies will provide critical, early information on alternative approaches to time limits, work incentives, welfare-to-work strategies, broad antipoverty efforts, programs to increase the employment and child support payments of noncus- todial fathers of children on AFDC, learnfare, family caps, and other building blocks of state T m F policies. In addition, a number of major studies will mon- itor what states do under the new legislation and assess the effect of the new policies on families and communities. There are also demonstrations targeted at particularly disadvantaged populations and communities. Finally, although it may prove impossible to use random assignment to assess the total system that replaces AFDC, it will remain possible to use this approach to compare key policy alternatives that will be central to state design choices and program costs. The impressive record of the past-and the early evidence of state inter- est in continuing to learn-sets a challenge for those of us working in this field to develop research strategies that provide convincing lessons and respond to the new policy framework.

References Aaron, H. Politics and the Professors. Washington, D.C.: Brookings Institute, 1978. Auspos, P., Cave, G., and Long, D. Maine: Final Report on the Training Opportunities in the

Bane, M. J , , and Ellwood, D. T. The Dynamics of Dependence: Tke Routes to Self-Sufficiency.

Bangser, M., Healy, J., and Ivry, R. Welfare Grant Diversion: Lessons and Prospects. New York

Private Sector Program. New York: MDRC, 1988.

Cambridge, Mass.: Urban Systems Research and Engineering, Inc., 1983.

MDRC, 1986.

LESSONS FROM STATE-BASED EVALUATIONS 93

Bardach, E. Improving the Productivity ofJOBS Program. Papers for Practitioners. New York MDRC, 1993.

Baum, E. 3. “When the Witch Doctors Agree: The Family Support Act and Social Science Research.”Journd @Policy Analysis and Management, 1991,lO (4), 603-615.

Bloom, D. After AEDC: Weyare-to-Work Choices and Challengesfor States. ReWORKing Wel- fare: Technical Assistance for States and Localities. New York: MDRC, 1997.

Bloom, D., and Butler, D. Implementing Time-Limited Welfare: Early Experiences in Three States. New York MDRC, ,1995.

Bloom, D., Kemple, J. J., and Rogers-Dillon, R, The Family Transition Pragram: Zmpkmentation and Early I m p t s offlordds Initial Time-Umited Welfare Program. New York MDRC, 1997.

Blum, B. B. “Bringing Administrators into the Process.” Public Welfare, 1990,3,4-12. Brock, T., Butler, D., and Long, D. Unpaid Work Experiencefor Welfare Recipients: Findings

and Lessonsfrom MDRC Research. MDRC Working Paper. New York MDRC, 1993. Brown, A. Work First How to Implement an Employment-Focused Apprmh to We@re Reform.

ReWORKing Welfare: Technical Assistance for States and Localities New York MDRC, 1997. Card, D., and Robins, P. Do Financial Incentives Encourage Welfare Recipients to Wurk? Initial

18-Month Findingsfrom the Self-Sufficiency Project. Ottawa, Ontario: Social Research and Demonstration Corporation, 1996.

Freedman, S., 3ryant, J., and Cave, G. NewJersey: Final Report on the Grunt Diversion Pro- ject. New York MDRC, 1988.

Freedman, S., and Friedlander, D. TheJOBS Evaluation: Early Findings on Program lmpacts in Three Sites. Washington, D.C.: U.S. Department of Health and Human Services, Office of the Assistant Secretary ‘for Planning and Evaluation, 1995.

Freedman, S., Friedlander, D., Lin, W., and Schweder, A. f i e GAIN Evaluatiox Five-Year Impacts on Employment, Earnings, and AFDC Receipt. GAIN Evaluation Working Paper 96.1. New York MDRC, 1996.

Friedlander, D., and Burtless, G. Five Years After: The Long-Term Effects ofwelfare-to-Work Programs. New York Russell Sage Foundation, 1995.

Goldman, B. lmpacts of the Immediate Job Search Assistance Experiment: Louisville WIN Research Project. New York MDRC, 1981.

Greenberg, D. H., and Mandell, M. B. “Research Utilization in Policymaking: A Tale of Two Series (of Social E!xperiments).”Journal of Policy Analysis and Management, 1991, 10 (4),

Greenberg, D., and Wiseman, M. “What Did the OBRA Demonstrations DO?^ In C. F. Man- ski and 1. Garfinkel (eds.), Evaluating Employment and Training Program. Cambridge, Mass.: Harvard University Press, 1992.

Gueron, J. M, “The Demonstration of Sta tNork Welfare Initiatives.” In R. F. Boruch and W. Wothke (eds.), Randomization and Field Experimentation. New Directions for Program Evaluation, no. 28. San Francisco: Jossey-Bass, 1985.

Gueron, J. M. .-A Research Context for Welfare Reform.”Joumal of Policy Analysis and Mart- agement, 1996,15,547-561.

Gueron, J. M., and Nathan, R. P. “The MDRC WorWelfare Project: Objectives, Status, Sig- nificance.” Policy Studies Review, 1985,4,417-432.

Gueron, J. M., and Pauly, E. From Welfare to Work. New York: Russell Sage Foundation, 1991.

Haskins, R “Congress Writes a Law: Research and Welfare Reform.”Journal ofpolicy Analy- sis and Management, 1991,lO (4), 616-632.

Miller, C., Knox, V., Auspos, P., Hunter-Manns, J., and Orenstein, A. Making Welfare Work and Work Pay: Implementation and 18-Month Impacts of the Minnesota Family Investment Program. New York MDRC, 1997.

633-656.

Murray, C. ‘What to Do About Welfare.” Commentary, 1994,98 (6), 26-34. Orr, L. L., Bloom, H. S., Bell, S. H., Doolittle, F., Lin, W., and Cave, G. Dws Training for the

Disadvantaged Work? Evidencefrom the National JTPA Study. Washington, D.C.: Urban Institute Press, 1996.

94 PROGRESS AND FUTURE DIRECTIONS IN EVALUATION

Riccio, J,, Friedlander, D., and Freedman, S. GAIN: Benefits, Costs, and Three-Year Impacts of. Welfare-b-WOrlz Program. New York MDRC, 1994.

Riccio, J., and Orenstein, A. “Understanding Best Practices for Operating Welfare-to-Work Programs.” Evaluation Review, 1996,20 (1),3-28.

Social Research and Demonstration Corporation. When Work Fuys Better Than Welfare: A Summary ofthe Se?f-Suflciency Project’s Implementation, Fom Group, and Initla1 18-Month Impact Reports. Ottawa, Ontario: Social Research and Demonstration Corporation, 1996.

Szanton, P. 1. “The Remarkable ‘Quango’: Knowledge, Politics, and Welfare Reform.”Jour- nal of Policy Analysis and Management, 1991,lO (4),590-602.

Wallace, J., and Long, D. GAIN: Planning and Early Implementation. New York: MDRC, 1987. Wiseman, M. “Research and Policy: An Afterword for the Symposium on the Family Sup-

port Act of 1988.”Journal ofpolicy Analysis and Management, 1991, I0 (41,457-666. Wolfhagen, C. Job Search Strategies: Lessons from the Louisville WIN Laboratory Project. New

York MDRC, 1983. Zambrowski, A,, and Gordon, A. Evaluation of the Minorify Female Single Parent Demonstru-

tion: F$&h-Year Impacts at CET. New York Rockefeller Foundation, 1993.

JUDITH M. GUERON is president of the Manpower Dmonstratiun Research Corpo- . ration, New Yorh City.