Out of School Time Part II: Quality Improvement Systems: Panel on City and State Models.

Gina Gallo-Asheville, NC,Candy Markman - Nashville, TNErica Mullen - Seattle, WA Facilitator-Nikki HaleApril 17, 2012

#readyby21

Agenda

• Welcome• Weikart Center Reach• An emerging framework of best

practice-from the “Wallace Guide for Quality Improvement System” (a work in progress)

• Panel • Questions• Closing thoughts

Weikart Center Reach in 2011

Poin

t of

Serv

ice

Setti

ngO

rgan

izati

on

Setti

ngPo

licy

Setti

ng

>17,030 StaffEstimate based on mean of 6.5

staff per site in YPQI Study Sample

70 Networks/Systems

>222,700 Child & YouthEstimate based on mean daily attendance of 85

youth per day in YPQI Study Sample

>2620 Sites

l

National Organizations:

Light Green- full-state implementation

Dark Green- place-based implementation

Gold- full-state + place-based

YPQI Quality Improvement Systems Across the Country

Performance Metrics: PQA, Lead Indicators

Team-based Planning with Data

Performance Feedback: Instr Coaching

Training for Front Line

Site Level

System Level

T&TA Model•Youth Work Mgmt - Quality Assessment (6hr) - Plan w/ Data (6hr) - Instruction Coach (6hr)•Youth Work Methods -12 Live modules (2hr/) -12 Online supp. (.5hr/)•TA/Coaching - Site Mgr Check-in (6hr) - Grantee Intensive (36hr)

•Time Per Site to Implement - YPQI Study: 72 hours (1 manager and 2 staff)

•Cost of T&TA Per Site to Fully Implement in Pilot Year - YPQI Study:

$333/staff member$3,028/site

- State 21st CCLC: ~$3,000/site (<25 sites ) Out years are less

QIS Supports: YPQI

Information System(s)

Guidelines and

Incentives for

participation

Shared Definition of Quality

Continuous Improvement

process

Lead Organization

Engaged Stakeholders

Common Component And Characteristics of an Effective QIS

Engage and Plan Design and Build Adjust and Sustain

3 Broad Stages for Developing an Effective QIS

system

Gina Gallo

• Youth Success Manager at United Way of Asheville and Buncombe County

• Utilizing the Developmental Assets Profile within your QIS/YPQI

• ggallo@unitedwayabc.org/828-239-1155

Laying the Foundation

• Fall 2010 – 9 sites at 7 different agencies

• Mayoral support with program recruitment and co-hosting events

• Building relationships with program level and agency administrative staff

• Long term commitment to continuous program improvement from United Way

Building Up

• Utilize programs that are highly committed to the process to recruit new organizations

• Creation of a professional development system for youth development workers

• A growing community-wide interest in developmental assets spearheaded by multiple agencies

Sustainability

• YPQA shows improvements in program quality, DAP shows improvements in assets reported = deeper commitment

• Implementing MOU/MOAs in 2012-2013 to ensure everyone plays their part

• A willingness to tailor our quality improvement work to program requests and needs

Challenges

• Turnover of program staff requires additional trainings

• Small number of programs serving middle school students in Buncombe County

• Need for additional assessors to prevent burnout

Candy Markman

• Planning Director, Afterschool Initiatives at the Mayor’s Office and NAZA

• Nashville, TN

• Body Text

Quality Standards• Based on national

research• Based on national and

state examples• Informed by Nashville

experience

Quality Indicators• Reflects best practice

and Principles of PYD • Aligned with other

funders• With community input

Assessment Strategies• Reflects Nashville's standards and

indicators • Informed by High/Scope and the

Search Institute • External and self assessment

elements• Informed by student input and

feedbackTracking Tool

• Participation and retention data from schools and programs

• Tied to School's key indicators: school attendance, behavior, academic achievement

Capacity Building• Training tied to standards and

indicators• Staff, board, and volunteer

training opportunities tied to assessment results

• All NAZA partners participate in QIC and show improvement

Improvement Efforts• Offer QIC elements to

Nashville Youth Coalition and wider community

• Create links between school day and out-of-school time Program Quality

Improvement Cycle:

Nashville’s Plan

15

Quality Standards• Non-profit & public providers convened (early 2009)• 25 adopted set of principles (4/09)• NAZA standards adopted & embedded into RFP (11/09)

Quality Indicators• sub-set of indicators must be met

before provider can apply to NAZA (mid 2010)

• Other Metro departments embed standards (2010)

• Metro Arts Commission creates new afterschool arts category with new principles and standards (2011)

Assessment Strategies• Adopted 2 surveys as assessment tools

(12/09) • Independent external evaluations

conducted by Weikart annually• YPQA surveys & feedback are ongoing

(Programming, Relationships, Environment, Staffing, Administration)

• Assessment results influence program improvement AND NAZA level improvements

Tracking Tool• Schools & Metro partner to integrate NAZA data with

Schools student data.• Plans in place to refine data reports on NAZA participants

to track & analyze.

Capacity Building• NAZA Quality Improvement Coordinator manages

training and assessments (8/10)• Full YPQI participation required by provider contract• Trainings adapted for volunteers and boards• 25 certified coach/assesors conduct external

assessments and support improvement• Youthworker Certifications available through NYC (7/12)

Improvement Efforts• NAZA workshops open at no cost: 72 agencies have

sent over 1100 staff & board members since 2010• Collaboration with local YMCA on marketing and use

of training space• Collaboration with Nashville public schools to provide

NAZA partners with additional math and reading academic support training (spring 2011)

Nashville’s Action Steps & Where We Are Today

Erica Mullen

• Associate Executive Director• YMCA of Greater Seattle• Program Quality Initiative- Taking it

to scale• emullen@seattleymca.org

PQA in the YMCA of Greater Seattle

• YPQA pilot in 2009 – 1 site• YPQA in Teen programs 2010 – 24

sites and programs• SAPQA in 2011 – 42 school age sites• CPQA in 2011 – 43 day camps and 2

overnight camps• Next stop…Swim, Sports and Play –

20,000 + participants

Seattle YMCA Engage and Plan

• Identify a small group of stakeholders - get them on board and understanding the process

• Identify early adopters and who might be slower to embrace the process

• Gain support from multiple levels of leadership

• Think long term in the initial planning

Seattle YMCA Design and Build

• Do a dry run – practice the process• Think about issues of flexibility and

fidelity when developing the process.• Be clear and specific with

expectations• Dedicate enough time to the process• Think about staff time and resources

Seattle YMCA Adjust and Sustain

• Ask and listen to multiple levels of staff, get their feedback, leaders must participate in the process

• Identify people who can be future trainers/leaders

• Internal resources/expertise to aid sustainability

• Integrate into other systems• Identify the group that will continue to

drive and refine the process• Executive Directors becoming “Data

Coaches”

Seattle YMCA Challenges and Adjustments

• Needed to address – Why did we pick PQA? Is it just the “new thing”? How does this connect with Assets and other frameworks we have adopted in the past?

• Quick growth• Staff changes/turnover – continuous training

process• Timing throughout the year• Integrating into multiple systems, program

survey data, professional development systems, staff training

Thank you!

For more information about building your QIS, contact Joe Bertoletti, joe@cypq.org or 734-714-2522