BUILDING STRONG PROGRAMS: LESSONS FROM WORKING IN THE SCHOOLS (WITS)

January 31st, 2014

DISCUSSION TOPICS

• Introductions

• Overview of WITS

• Training, Expectations, & Strategies

• Evaluation and Assessments

• Small Group Discussions

WITS VIDEO

WITS believes that when mentors focus their energy and time on the success of

our city’s students, their volunteer service directly impacts the lives of young people and helps support a

better public school system.

BRIDGING THE GAP

WITS PORTFOLIO OF PROGRAMS

Programs:• Mid-Day Mentoring

• Workplace Mentoring

• Early Childhood

• Classroom Reading Tutors

• Saturday Tutoring

• Early Childhood Summer Program

• WITSummer in the Parks

STAKEHOLDERS

*

Volunteers (community,

corporate groups,

universities, churches)

StudentsSchools

POSITIONING VOLUNTEERS FOR SUCCESS

Frequency and Quality of Support

SettingAppropriate Expectation

s

Modeling,Coaching,Reflection

STRATEGIES FOR WORKING WITH YOUTH

OVERCOMING OBSTACLESModel Problem Solving• Name the problem• Talk through possible solutions• Model appropriate problem solving

Encourage the Student • Praise their process• They aren’t alone and—neither are

you!

The Goal: Students to see themselves as capable problem solvers and valuable individuals

BUILDING STUDENT FLUENCY

Model Fluency• Reading expressively• Use character voices• Modulating speed and

volume

Encourage the Student• Help student choose books• Re-read to build confidence

The Goal: students see reading as an opportunity to explore their interests—not a task to complete.

THE FIVE C’S OF POSITIVE YOUTH DEVELOPMENT (PYD)

PYD: focus on the strengths of youth and the positive qualities and outcomes we wish youth to develop

Competence Confidence Connection Character Caring

*

USING EVALUATION TO STRENGTHEN PROGRAMS

Types:• Formative• Summative

Methods:• experimental• quasi-experimental• observational

Considerations:• What is the purpose of the evaluation?• Who will use the evaluation results?• How will they use the evaluation results?• What do stakeholders need from the

evaluation?

*

THE POWER OF OBSERVATION (FORMATIVE EVALUATION)

Questions:• Are we meeting our program goals?

• Is there consistency across programs?

• What immediate improvements can we make?

• What changes can/should we make in the future?

• Use observation forms/standardize

WORKPLACE MENTORING LOGIC MODEL

Priorities

Inputs Outputs Outcomes

-need for consistent and caring mentors

-need for literacy enrichment programming in the afterschool space

-scaffold student self-efficacy

-scaffold student attitude

Assumptions

-all students are capable of becoming better readers

-students are motivated to participate in WITS programs

-volunteers are competent tutors/mentors

-students and volunteers form a positive relationship over time

-human resources: staff; corporate + university volunteer time investment

-partners: teacher/principal time investment; experts

-WITS Board; Associates Board

-financial resources

-build knowledge base

-other: transportation; bus chaperones; snacks; maintain program materials

identify external factors

Situation

Program Goals

EVALUATION

-effective 1:1 literacy-focused mentoring programs

-300 students from 13 schools participate

-19 corporate and university groups provide 600 volunteers

-curriculum implemented

-volunteers coached to provide student with effective support

Student:

-improved attitude toward reading

-increased self-efficacy

-improved reading fluency

Volunteer:

- sustained school/community partnership

-high volunteer satisfaction

-high volunteer retention

WITS EVALUATION GOALS

• mixed method approach

• multiple data sources

• triangulate data

• balance formative/summative

• make decisions on multiple findings

BEST PRACTICES• align data collection with program goals

• think before you collect

• weigh pros and cons of types of questions

• balance formative and summative

• pilot a new assessment

• ask the hard/unpopular questions

• consult peers

• know your research domain

EXAMPLE: LEVERAGE EXTERNAL DATA

BENEFITS OF WORKPLACE VOLUNTEERISM

2011 Deloitte Volunteer IMPACT Survey

EXAMPLE: LEVERAGE INTERNAL DATA

STUDENT-REPORTED SATISFACTION

Youth Emotional Engagement Benchmark Survey adapted from: Herrera, C. (2004). School-Based Mentoring. A Closer Look. Public/Private Ventures, 42.

EXAMPLE: LEVERAGE INTERNAL DATALENGTH OF PARTICIPATION IN WORKPLACE MENTORING PROGRAM

Enthusiasm Toward Read-

ing

Positive Attitude - Silent Reading

Positive Attitude - Oral Reading

Positive Attitude - School in General

0%

20%

40%

60%

80%

100%

Teacher Ratings of Student Motivation/Attitude Toward School and Reading

0 years minimum of 2 years

COMMON EVALUATION ERRORS

Avoid:• failing to leverage the data you already have• collecting more data than you really need/can

handle• using the same data points when a project has

changed• poor survey/question design

o wrong question typeo vague wording/jargono incorrect assumptionso leading questionso double-barrelingo missing response options o incorrect use of scales

SMALL GROUP DISCUSSION TOPICS

1. Discuss some challenges you have had when working with youth. What changes did you make to your program structure, training or curriculum due to those challenges?

2. How do you gain feedback from your volunteers/staff? Provide an example of feedback your organization has received and programmatic changes you have made due to the feedback?

3. Discuss manageable formative evaluation activities that you and your organization can add to improve program quality.

4. Discuss realistic summative evaluation activities that you and your organization can add to improve program quality.

5. How should an organization respond to negative evaluation results? How should an organization respond to positive results?

KEY IDEAS

• Set clear and appropriate expectations!

• Constantly observe and troubleshoot

• Think critically (but constructively)

• Small organizations can easily take big steps toward evaluating program quality