ncwwi national webinar series session #10 beyond child...

A Service of the Children’s Bureau, a Member of the T/TA Network

NCWWI National Webinar Series Session #10

Beyond Child Welfare Education & Training: Pennsylvania’s Dynamic

University-Agency Partnership for Transfer of Learning, Quality Improvement &

Organizational Effectiveness

Wednesday, September 11, 2013

Panelists Terry Clark, MPA, Director of Operations, Pennsylvania Department

of Public Welfare, Office of Children, Youth & Families Helen Cahalane, PhD, ACSW, LCSW, Principal Investigator,

Child Welfare Education & Research Programs & Clinical Associate Professor/Chancellor's Distinguished Teacher, University of Pittsburgh School of Social Work

Maryrose McCarthy, MS, Director, Pennsylvania Child Welfare Resource Center

Michael Byers, MSW, Assistant Director, Pennsylvania Child Welfare Resource Center

Wendy Unger, MSW, Organizational Effectiveness/Regional Teams Department Manager, Pennsylvania Child Welfare Resource Center

Sharon England, J.D., MSW, LSW Curriculum & Trainer Development Department Manager, Pennsylvania Child Welfare Resource Center

www.ncwwi.org A Service of the Children’s Bureau, a Member of the T/TA Network 2

http://www.ncwwi.org/

Session Agenda

1. Why Focus on Education & Training? 2. Pennsylvania Presentation

– Background – The Public Agency Perspective – Core Activities – Major Impacts & Outcomes – Accelerators & Decelerators – Lessons Learned/Tips for Other Agencies

3. Q & A via web platform 4. Continuing the Conversation & Closing



Importance of Dynamic Education &Training

Partnerships 1. Workforce Preparation 2. Workforce Retention 3. Training & Transfer of Learning 4. Organizational Effectiveness 5. Organizational Climate/Culture 6. Research & Evaluation 7. Public Perception 8. Community-building 9. Leveraging Funds 10.Child, Youth & Family Outcomes



Leadership for Education &Training Partnerships

Accountability Capacity-building Service Orientation Decisiveness Entrepreneurship Financial Management Planning & Organizing Problem Solving Technical Credibility

Creativity & Innovation External Awareness Flexibility Strategic Thinking Vision

Conflict Management Developing Others Team Building Cultural

Responsiveness Leveraging Diversity

Partnering Political Savvy Influencing Negotiating

Leading People

Leading for

Results

Leading Change

Leading in

Context



BACKGROUND & CONTEX



Background (1) Pennsylvania At a Glance: CW = State-Supervised, County-Administered 67 counties Total population: More than 12 million

– Diversity: Philadelphia (population = 1.5 million; foster care = 4,570) to Forest County (population = 7,000; foster care = 3)

– Approximately 4,500 CW staff statewide – Approximately 14,000 children in placement

(as of March 31, 2013) www.ncwwi.org A Service of the Children’s Bureau, a Member of the T/TA Network 7


Background (2) Development of a statewide training program (1992)

–Driven by County Children & Youth Administrators and state’s goal to increase consistency and quality of the development and delivery of training

–County collaboration and partnership www.ncwwi.org A Service of the Children’s Bureau, a Member of the T/TA Network 8


Background (3) University Partnership

– 1992 Intergovernmental Agreement – Focus on core competencies and training delivery

American Humane Association (AHA) Evaluation – Evaluated the extent to which all of the functions of

the training program were operating – Findings:

1. Lack of support and structure by the university which held the contract with regard to resources allocated to the program

2. Need for more expertise to conduct training evaluation 3. Need to strengthen transfer of learning components



Background (4) Development of a New University Partnership – Multiple Universities approached – Requirements:

• School of Social Work • Willingness to match a % of funds/indirect costs • Connect a continuum of child welfare programs

– University of Pittsburgh selected in 2001 Structural Changes – Transitioned to centralized program with regional training

centers – Center for Excellence concept: Creating & building own

expertise by drawing on resources of faculty research to provide training & professional development/support for CW professionals



Background (5)

CFSR Round 1 results: Suggested the need for additional efforts to impact practice at the local level

2002: Expanded to include transfer of learning and skills-based training

2004: Launched the Center for Excellence, a technical assistance center designed to support practice improvement and organizational effectiveness, which later became the Organizational Effectiveness/Regional Teams (OE/RT) Department



The Pennsylvania Child Welfare Resource Center

[PACWRC] (1) Present Day: Our Vision Every child, youth, and family

experiences a life rich with positive opportunities, nurturing relationships, and supportive communities. The CW workforce is supported and

engaged in professional development and system improvement initiatives.



PACWRC (2) Present Day: Our Mission The Pennsylvania Child Welfare Resource

Center is a national leader in advocating for an enhanced quality of life for Pennsylvania’s children, youth and families. In partnership with families, communities, public, and private agencies, we prepare and support exceptional child welfare professionals and systems through education, research, and a commitment to best practice. www.ncwwi.org A Service of the Children’s Bureau, a Member of the T/TA Network 13


PACWRC (3)



PACWRC (4) Current Structure 6 Departments

– Administrative – Fiscal/HR – Technology Development – Curriculum and Trainer

Development – Organizational

Effectiveness – Statewide Quality

Improvement

Staffing 80 + staff In-house &

remotely located Regional

training sites



PACWRC (5) Funding Sources That Support Our Work: Title IV-E Title IV-B Child Abuse Prevention & Treatment Act (CAPTA) Children’s Justice Act Chafee Foster Care Independence Funds Caseworker Visitation Funds State of Pennsylvania Casey Family Programs



THE PUBLIC AGENCY

PERSPECTIVE



Public Agency Perspective (1)

County Benefits 1. Efficiency 2. Consistency (in expectations and support) 3. Development of the workforce 4. Individualized support when necessary 5. Practice improvement

State Benefits 1. Efficiency 2. Consistency 3. Helping to ensure alignment with state vision 4. Increased ability to address statewide themes 5. Practice improvement



Public Agency Perspective (2)

County/State Relationship Opportunities 1. The Pennsylvania Child Welfare Resource

Center Steering Committee 2. Regular meetings with the state 3. Support/Partner with the Pennsylvania

Children & Youth Administrators (PCYA) 4. Network of 16 university partners (Schools

of Social Work) throughout the state



CORE ACTIVITIES



Core Activities 1. Organizational Change, including consultation and

technical assistance 2. Training Development & Delivery, including

curriculum development and delivery methods 3. Transfer of Learning (TOL), including guides,

needs assessments, coaching and mentoring 4. Statewide Quality Improvement Projects,

including Continuous Quality Improvement (CQI) through Quality Service Reviews (QSRs)

5. Education, Evaluation & Research, including undergraduate and graduate education and multiple research/evaluation projects



Organizational Change (1)

Structure to Support Statewide and Agency Vision Collaboration and Partnerships Focus on Continuous Quality

Improvement (CQI), Organizational Change and Positive Outcomes Enhanced Organizational Effectiveness

(OE) Model




Implement Implement County Improvement Plan

Plan Develop County Improvement Plan

Define Desired Future State – What should our practice and system look and feel like?

QSR indicators set the mark

CQI / DAPIMTM Overlay

Monitor Monitor County Improvement Plan for completion & impact

Assess Review data, including QSR findings, identify strengths/gaps, uncover root causes

DAPIMTM © 2011 American Public Human Services Association. All rights reserved. www.ncwwi.org A Service of the Children’s Bureau, a Member of the T/TA Network 23











Projects 1. Statewide 2. Regional 3. County 4. Organizations



Organizational Change (4) Implementation Process

1. Continuous Quality Improvement (CQI)

2. Organizational Standards (fidelity to the OE model)

3. Building, Maintaining and Growing Partnerships

4. Staff Professional Development www.ncwwi.org A Service of the Children’s Bureau, a Member of the T/TA Network 25


Training Development & Delivery (1)

Purpose 1. Promote the Long-Term Career Development

Among Child Welfare Professionals 2. Elevate Child Welfare Practice in Pennsylvania 3. Support Implementation of:

– Child & Family Services Review/5 Year Plan – Continuous Quality Improvement (CQI)

• Quality Service Reviews • Practice Improvement Initiatives • Practice Model




Identification of Curriculum to be Developed In Partnership with Stakeholders:

– Office of Children, Youth & Families – PA Children & Youth Administrators – Courts

Prioritization Considerations: – Statutory requirements – Availability of existing resources – Extensiveness/widespread nature of stakeholder

need – Impact of not developing or delaying the

development of the curriculum




Curriculum Development Competency-Based and aligned with Practice Model Design

1. Assessment Design Development Implementation Evaluation (ADDIE) Model

2. Instructional Design Team 3. Youth and Parent Consumers 4. Quality Assurance Committees 5. Content Experts 6. Diversity Task Force

Transfer of Learning www.ncwwi.org A Service of the Children’s Bureau, a Member of the T/TA Network 28



Training Delivery Method Decision Guide (see handout)

1. Classroom 2. Online

– Learning Management System & Design Software

– Knowledge & Skill Building 3. Hybrid (Blended)

– Charting the Course




Trainer/Consultant Selection Process – Application & Panel Interview – Experienced Child Welfare Professionals

Trainer/Consultant Development Process

– Development of Trainers and Consultants (DOT/C) – Training on Content (TOC) and/or Mentoring – Observations

• First Time & New Content • Periodic or Biennial




Professional Development Ongoing Updates and Training Events Consultant/Trainer Advisory Group (CTAG) Trainer/Consultant Palette Trainer and Consultant Handbook Continuous Quality Improvement (CQI) Level 1 Evaluations Technical Assistance Trainer Feedback Trainer Incident Trainer Utilization Process



Transfer of Learning [TOL] (1)

Transfer of learning occurs when the knowledge, skills and attitudes we have learned are used on-the-job and benefit performance.

PACWRC definition: TOL is a structured, deliberate set of activities or resources intended to help participants make the connection from theoretical concept and associated skill to integrating that concept into practice. It is comprised of a planned series of steps or activities that continue outside of a learning event.



Transfer of Learning (2)

The Learning Partnership: – Consists of a triad formed with staff from the

PACWRC, agency representatives (i.e. supervisors, administrators, managers) and trainees

– Creates an effective learning environment – Facilitates effective application or transfer of

learning through shared responsibilities at each phase of the learning cycle



Transfer of Learning (3) Learning/training events alone have a modest

impact on performance. Learning/training events with transfer activities

can have a meaningful impact on performance. Transfer of learning can be supported by what

is accomplished by the triad members at three key phases: – BEFORE the learning/training event – DURING the learning/training event – AFTER the learning/training event



Transfer of Learning (4) BEFORE PACWRC includes stakeholders in curriculum

development & provides pre-TOL online activities & recommendations for supervisor pre-TOL activities

Participants identify the purpose of each module, evaluate their knowledge & skill level & participate in pre-TOL activities

Supervisors encourage participants to complete pre-work activities & tasks, identify & discuss purpose of attending the learning/training event prepared & ready to learn

Supervisors provide opportunities for shadowing colleagues, practicing skills, and/or consultation www.ncwwi.org A Service of the Children’s Bureau, a Member of the T/TA Network 35


Transfer of Learning (5) DURING Trainer/facilitator is familiar with the content & TOL Curriculum presents information repetitively and

gradually to build on previous knowledge Curriculum provides practice opportunities through role

plays and activities Participants attend learning/training events prepared

and ready to learn Supervisors support the participants’ ability to fully

participate in learning/training events by protecting them from interruptions & delegating calls/assignments



Transfer of Learning (6) AFTER Post-TOL online activities. TOL support sessions, technical assistance, and/or

facilitated discussions. Participants evaluate their knowledge and skill level

and participate in activities. Supervisors, recognizing the impact of universal

experience of results dip, provide coaching and debriefing sessions.

Supervisors also connect the learning/training event content to the agency’s overall objectives.



Statewide Quality Improvement Projects (1)

History Relatively new to the PACWRC Designed to support the development,

implementation and monitoring of statewide efforts/projects Current focus areas: Citizen Review Panels,

Youth and Family Engagement, Safety, CFSR, Children’s Justice Act, Practice Model, and Continuous Quality Improvement (CQI) www.ncwwi.org A Service of the Children’s Bureau, a Member of the T/TA Network 38



A Closer look at CQI Origins

– CFSR Round 2 had many findings similar to Round 1

– The CQI framework was created in response to developing a structure to better support and drive sustainable change at the local level.

– Shift from compliance to quality A Working Definition – CQI is the ongoing process by which an agency

makes decisions and evaluates its progress (“A framework for implementation” from Casey Family Programs & NRC-Organizational Improvement)




Implement Implement County Improvement Plan

Plan Develop County Improvement Plan

Define Desired Future State – What should our practice and system look and feel like?

QSR indicators set the mark

CQI / DAPIMTM Overlay

Monitor Monitor County Improvement Plan for completion & impact

Assess Review data, including QSR findings, identify strengths/gaps, uncover root causes

DAPIMTM © 2011 American Public Human Services Association. All rights reserved. www.ncwwi.org A Service of the Children’s Bureau, a Member of the T/TA Network 40











Quality Service Review (QSR): An in-depth case-specific review using a combination of record reviews and interviews with those involved with the case, including the family. Purpose: 1. To measure child, youth, and family outcomes and system

performance. 2. To assure that the actions of a team of child welfare

professionals are guided by clear and comprehensive policies and are rooted in Pennsylvania’s Practice Model/Standards.

3. To support teaching and learning processes that clarify expectations, provide useful feedback, and affirm good work.

4. To stimulate actions taken to improve practice and results at all levels of the organization.




CQI Statewide Implementation Phased in rollout – active in 18 counties State-supported QSRs occur every 1-3

years in each county Existing staff at all levels manage the

work: CQI is a cornerstone of our work and therefore can not be “assigned” to a specialized unit. www.ncwwi.org A Service of the Children’s Bureau, a Member of the T/TA Network 42



How are we doing so far? 1. Increased community understanding and support of

child welfare work – “I am certainly more appreciative of the work of the CYS” – QSR

reviewer 2. Positive feedback from caseworkers, supervisors, and

families – “I appreciated that the interviewer seemed genuine and truly invested

in ultimately helping these families -- not simply gathering statistics or information.” - Child Welfare Professional

– “I felt heard with everything that was said and notes were taken.” - Family Member

3. Largest county conducts their own QSRs throughout the year to evaluate progress

4. QSR is seen as a tool that measures what matters most



Education, Evaluation & Research



Child Welfare Education & Research Programs

Child Welfare Resource Center (CWRC)

Degree Education Programs

Curriculum Development,

Training, Technical Assistance

Organizational Effectiveness, Quality

Improvement, Research & Evaluation

Child Welfare Education for Baccalaureates

(CWEB)

Child Welfare Education for Leadership (CWEL)



Temple University - Harrisburg

Marywood University – Center Valley

Widener University - Harrisburg

Millersville University

Marywood University – Reading

Kutztown University

Shippensburg University

University of Pittsburgh

Slippery Rock University

California University

Edinboro University

Mansfield University Marywood

University – Main Campus

University of Pittsburgh - Bradford

Marywood University – Central PA

Bloomsburg University

Lock Haven University

Bryn Mawr University

University of Pennsylvania

West Chester University

Widener University

Temple University – Main Campus

CWEL Only CWEB Only CWEB and CWEL

Participating Universities



CWEB/CWEL Profile: 2013-14

CWEB

• Established 2001 • 46 active students • 877 individuals have

entered into the county child welfare system since CWEB began

CWEL

• Established 1995 • 141 active students • 1,099 grads to date • In current academic

year, 13% of CWEL students are also CWEB grads

Statewide; 97% of counties participate



County With IV-E Graduate As Current Administrator or Assistant Administrator

Leadership By IV-E Grads



Developmental & Social-Emotional Screening in

CW Services Three phase, statewide study examining the

screening of young children receiving child welfare services using standardized instruments (ASQ™ & ASQ:SE™) County policies & procedures analyzed Screening results of 4,669 children from 60

counties analyzed (48.7% showed concerns) In-depth interviews conducted with a random

sample of caregivers (n=337) www.ncwwi.org A Service of the Children’s Bureau, a Member of the T/TA Network 49


Safety & Risk Assessment

PA is taking a closer look at the psychometric properties of its safety and risk assessment tools. Conducting a study that is examining

reliability and validity of both tools Also looking to our own workforce and to

other states to learn more about what works and how we may be able to streamline the process without losing valuable information.



Training Evaluation Rigorous evaluation of core training series for

new caseworkers and new supervisors – Consulting with Cynthia Parry, Ph.D.

7 levels of evaluation 1. Tracking attendance at training 2. Course evaluation 3. Satisfaction/Opinion 4. Knowledge 5. Skills 6. Transfer 7. Agency/Client Outcomes



Transfer of Learning Evaluation (1)

TOL is part of all of our training & learning activities. Conducted study to examine training outcomes

among CW workers in relation to receipt of TOL package (initial training + post-work sessions + booster shot training). Findings included:

– Perceptions of peer & supervisor support significantly improved b/w baseline & 6-mo. follow up for TOL participants

– Caseworkers in control group reported feeling significantly less knowledgeable about training content 6-mo. post-training

– Although not statistically significant, caseworkers in TOL group showed positive trends in items related to content knowledge, ability to use training content, and opportunities to use training content six months post training.



Transfer of Learning Evaluation (2)

• We are now developing a TOL How-To Guide that can be utilized with any training offered by the PACWRC.

• We are also looking at standard ways of collecting qualitative and quantitative data after workers participate in TOL – Structured After Action Reviews – Brief surveys assessing TOL post-training



Evaluation of OE Interventions

1. Multi-state evaluation of the APHSA Organizational Effectiveness (OE) model

2. Mixed methods design with 2 components – Survey and key informant interviews with

current and past OE client agencies focusing on the extent to which organizational outcomes are achieved following OE work, the role of organizational readiness for change, use of specific OE practices, and client perceptions

3. PA-specific sample of counties exploring more detailed information on the process and achievement of OE outcomes.



Evaluation of PA’s Title IV-E Demonstration

Project (1) Three focus areas:

1. Family engagement 2. Comprehensive assessment of child &

family strengths and needs 3. Utilization of evidence-based practices



Evaluation of PA’s Title IV-E Demonstration

Project (2) Process evaluation – Readiness to implement – Fidelity of implementation

Outcome evaluation – To what extent do counties’ proposed activities relate to improved

placement decisions and improved child & family well-being? – How does level of fidelity to proposed activities relate to improved

placement decisions and improved child & family well-being? – To what extent do placement decisions and child & family well-being

improve as a result of financial flexibility afforded by the Waiver? Cost analysis

– Do expenditure patterns for out-of-home care change over the 5 years of the Project, and if so, were the changes related to unit costs, care days, or both?

– Does the ratio of out-of-home care spending to spending for prevention and family preservation decrease over the 5 years of the Project?



MAJOR IMPACTS & OUTCOMES



Impacts & Outcomes (1) PA’s Child Welfare System (as a whole): Safely reduced the # of children in

placement by over 30% in the last 5 years (21,000 to 14,000) Changing culture to collaborative

and quality-focused



Impact & Outcomes (2)

CQI Impact The QSR data provides a common framework, which allows us to:

– Have a real-time view of practice

– Identify strengths and needs

– Garner commitment for change

– Allocate resources to address areas of need

QSR Data 95 88 84

39 59 59

0102030405060708090

100% Strength



ACCELERATORS & DECELERATORS



Accelerators 1. Leadership 2. Active involvement of key

stakeholders 3. Shared vision (practice model) 4. Open and honest communication at

all levels 5. Flexibility 6. Commitment to funding “the vision”



Decelerators

1. Pace of change

2. Varied abilities to implement change

3. Competing demands

4. Funding challenges



LESSONS LEARNED/TIPS



Lessons Learned/Tips (1) 1. Modeling our Practice 2. Commitment to internal CQI 3. Use of CW professionals as trainers 4. Focus on building internal capacity 5. Organizational approach 6. Slow down to speed up – change

takes time



Lessons Learned/Tips (2)

7. Commitment to structured, comprehensive implementation

8. Build evaluation into projects from the beginning

9. Promote the use of data-driven decision making



Next Steps

1. Learner-focused instructional design

2. Online development 3. Project management 4. Next level of training evaluation



QUESTIONS? COMMENTS?

IDEAS?



Continuing the Conversation

1. Visit http://www.ncwwi.org/events/archive.html for the webinar RECORDING & PRESENTER HANDOUTS

2. Participate in this webinar session’s follow-up LEARNING LAB teleconference on Wednesday, September 18 at 3 pm for more discussion with your peers & the presenters (register here: http://events.constantcontact.com/register/event?llr=pfnuq9dab&oeidk=a07e7uje1toae04d360)

3. Provide feedback to strengthen future offerings by completing a quick EVALUATION SURVEY (via email from Robin Leake, Butler Institute for Families at the University of Denver)


http://www.ncwwi.org/events/archive.html

http://events.constantcontact.com/register/event?llr=pfnuq9dab&oeidk=a07e7uje1toae04d360

http://events.constantcontact.com/register/event?llr=pfnuq9dab&oeidk=a07e7uje1toae04d360


1. PACWRC Website: http://www.pacwrc.pitt.edu/ 2. NCWWI 1-pager: Specialized Education &

Training #12 3. NCWWI's Child Welfare Traineeships: 12

Program Summaries 4. NCWWI Traineeship Syllabi Collection 5. NCWWI Traineeship Teaching Resources 6. NCWWI Traineeship Program Promising

Approaches & Strategies 7. NCWWI Traineeship Student Profiles 8. Forthcoming (October 2013): NCWWI Traineeship

Legacy Documents www.ncwwi.org A Service of the Children’s Bureau, a Member of the T/TA Network 69

Additional Resources

http://www.pacwrc.pitt.edu/

http://www.ncwwi.org/docs/Specialized_Training_Education_1pager_12.pdf

http://www.ncwwi.org/docs/Specialized_Training_Education_1pager_12.pdf

http://www.ncwwi.org/docs/NCWWI_Traineeships_Summary_Packet.pdf

http://www.ncwwi.org/docs/NCWWI_Traineeships_Summary_Packet.pdf

http://www.ncwwi.org/docs/NCWWI_Syllabi_Collection.pdf

http://www.ncwwi.org/docs/Collection_Traineeship_Teaching_Resources_from_Syllabi.pdf

http://www.ncwwi.org/docs/NCWWI_Traineeship_Promising_Approaches_Strategies.pdf

http://www.ncwwi.org/docs/NCWWI_Traineeship_Promising_Approaches_Strategies.pdf

http://www.ncwwi.org/docs/Traineeship_Student_Profiles.pdf


Pennsylvania Department of Public Welfare, Office of Children, Youth & Families - contact Terry Clark at

[email protected]

University of Pittsburgh School of Social Work, PA Child Welfare Resource Center - contact

Helen Cahalane at [email protected] Maryrose McCarthy at [email protected]

Michael Byers at [email protected] Wendy Unger at [email protected]

Sharon England at [email protected]

Learning Lab – contact Sharon Kollar at [email protected]

NCWWI National Webinar Series & other products – contact Sara Munson at [email protected]


To learn more about….

mailto:[email protected]







