Translating Performance Data to Practice: Developing Curricula to Train Research-Minded MSW Students

Bryn King, MSWWendy Wiegmann, MSW

Emily Putnam-Hornstein, MSW, PhDCenter for Social Services Research University of California at Berkeley

Presented at the Council on Social Work Education, 58th Annual Program MeetingOctober 29, 2011

THE “PROBLEM”• IV-E students rank research courses as their least favorite

classes in the program (anecdotal…)

– coursework feels disconnected from practice • (Green Bretzin, Leininger, & Stauffer, 2001; Harder, 2010)

– most students do not enter with strong math or stats backgrounds & experience high anxiety • (Green, et al., 2001; Hardcastle & Bisman, 2003; Royse & Rompf, 1992)

– timeline allows only a superficial coverage of analytic methods– general lack of “statistical literacy”

• knowledge which enables people to think for themselves, judge independently, and discriminate between good and bad information (Dewey, 1930)

DATA IN THE “REAL WORLD”• While social work students may have positive attitudes

about research while they are students, they do not usually extend into practice (McCrystal & Wilson, 2010). – barriers may include beliefs that research will not be relevant to

practice (Green et al., 2001; Harder, 2010)– feelings that there will be too many demands when they enter

the field to devote time and energy to research (Wilson and Douglas, 2007; Shlonsky and Stern, 2007).

THE MOTIVATION

• experienced users of the state’s child welfare data, familiar with local and state performance goals, critical thinkers, confident

• practical research analysis skills – creating charts and tables in Excel, examining trends over time, understanding different data samples

• technical assistance to counties and state agencies, but no direct training of IV-E students at Berkeley/other universities

• “data are your friend” promote data informed practice and policies

• support counties and state in continuous improvement goals

• provide IV-E students with classroom experiences that can directly inform post-graduate work

• develop a research curriculum for use in other IV-E schools of social work throughout the state

• support counties in their System Improvement Plans

COMPETING COURSE MODELS

difficult (if not impossible) to do well

even successful training is lost as not used in post-grad work

technology = data are everywhere!

field is increasingly oriented around continuous improvement, outcomes

statistical literacy & critical thinking necessary for EBP and EIP

burdens agencies tasked with helping students access data

PEDAGOGICAL APPROACH• service learning model

– not uncommon for social work research courses• emphasized by asking students to align their

research project with agency performance goals or mandate

• students asked to write reports to be read by someone other than instructor

– help IV-E students connect classroom instruction to their experiences in field placements by working with administrative data relevant to their agency and clients

OPENING CLASSES• highlight that data really are used by social workers –

and one does not need to be a statistician– county child welfare director and analyst as guests– panel of former IV-E/MSW students– guest speaker on systems/practice improvement for

behavioral health services

• help students generate research ideas early– set-up one-on-one meetings during first two weeks– review of system improvement plans– reduce anxiety, increases relevance of exercises

STUDENT ANXIETY• Class was oversubscribed:

– lots of student interest/demand, looking for a departure from the anxiety-provoking status quo

– 28 students total (turned away 6 others – the cap was 24) and 13 of those were IV-E students (of 17 in the cohort)

• In a survey completed by CalSWEC (in the third week of class), students rated their anxiety level on a scale of 1-10:– 6.5 on a scale of 1 to 10 prior to starting the course

– 5.1 by the third class

– 4.2 by the end of the year-long course (differences were not statistically significant).

BRIEF OVERVIEW OF CSSR DATA• Organized around federal and state performance

measures• Dynamic site, ad hoc tabulations by users– county-level data– trends over time (1998-2010/11)

– categorical variables (race, gender, age, placement type, etc.)

– filters that allow user-created variables (e.g., age 0-4)

– Excel downloads

• PPT slide template

OTHER SOURCES OF SECONDARY DATA• University-based data archives, such as the Inter-University

Consortium for Political and Social Research at the University of Michigan

• Health and safety data from the CDC and the FBI (Uniform Crime Statistics)

• Population data from the census• National and local educational data• Mental health statistics• Local and state-level data regarding children’s well-being from a

variety of sources on kidsdata.org

DECIDING ON A PROJECT

SKILL DEVELOPMENT

ASSIGNMENTS FOR THE YEAR• From Topics to Questions • Background/Problem

Statement • Chart and Table • Research Question(s) • Description of Data

Source(s)• Review of Research • Structured Abstract

• Research Article Assignment

• Report Peer Review• Results Section Outline • Research Presentation • Final Report – included

introduction, abstract, lit review, description of data source, methods, results, discussion, graphs

THE PRODUCT: IV-E STUDENTS• Child Maltreatment and Permanency Planning in Contra Costa

County• Children Re-Reported for Maltreatment: An Analysis of

Alameda County• The Characteristics of Foster Children Authorized for

Psychotropic Medication in California and Contra Costa County

• Latino Children Involved in the Child Welfare System: An Overview of California

• Adoption in California’s Public Child Welfare System

THE PRODUCT: OTHER STUDENTS• The Achievement Gap, the Discipline Gap & Positive Behavior

Support: An Analysis of Progress at Willard Middle School• Severe Mental Illness and Poverty In California’s Fifty-Eight

Counties• Violent Crime Trends and Psychosocial Needs of Crime

Victims: A National, State, and Local Analysis• Cyberbullying: A Descriptive Analysis of Characteristics

Associated with Internet Victimization• Health Services Quality: Elderly Immigrants at Risk for

Miscommunication

EXAMPLES

QUALITATIVE RESULTS• Informal feedback was utilized to reevaluate course planning,

assignments, and course content• Overall, students reported that they had a positive experience of the

course (definitely exceeded their low expectations)• Appreciated the consistent feedback and support in their work and the

division of the work across two semesters that helped them develop their projects incrementally

• Found that learning to use basic statistical methods was much easier than anticipated

• Improved Excel and research skills• Projects were excellent!

QUANTIFIABLE RESULTS• Both the instructor and the teaching assistant were

rated very high in their evaluations• A student from this class won the annual MSW

research award• Four students (three IV-E) were hired for research

projects as their first post MSW position• Students reported less anxiety at post-test and all

students believed that research improves practice

FUTURE MODIFICATIONS• Orientation of classes around case studies– engage students in real examples of agency work with

data, provide a conceptual link between research and practice

– bring in guest speakers even earlier• Early instruction in Excel basics• Greater consultation with counties• More encouragement and emphasis on using

secondary data