asq - national institute on out-of-school time · welcome to asq —after‐school quality: the...

After‐School Quality:The Process of Program Improvement

Learn how to:

• Pair ASQ with program quality assessment tools

• Involve multiple stakeholders

• Select and train an ASQ Team

• Gather and organize data

• Develop an action plan to reach improvement goals

ASQA guide for facilitating a 5‐step, team‐based approach to improving program quality.

Welcome to ASQ — After‐School Quality: The Process of Program Improvement.

With today’s increasing interest in program quality, there are a growing number of tools that programs

can use to assess and improve quality. What’s missing is a process for involving multiple stakeholders in

the effort to document program quality. That’s where ASQ comes in.

ASQ is designed to assist your program to become the best it can be. ASQ is a five‐step self‐assessment

process that helps you examine the quality of your program and then make changes that will lead to

better practices and more positive youth outcomes.

The cornerstone of ASQ is a team approach. ASQ relies on the ideas and energy of all the people who

care about your afterschool program— staff, families, children, youth, program directors, boards of

directors, and hosts. All involved are invested, have a voice, and can help shape the solutions because

ASQ includes everyone in program improvement.

The ASQ process is about change. ASQ is based on the belief that the most successful program

improvements happen when key stakeholders are involved in a slow, gradual change.

ASQ supports the use of any youth program quality assessment tool. Open‐ended and flexible, ASQ

helps you make the most of any program observation tools and questionnaires/surveys. The ASQ process

focuses on how one uses an instrument, who is involved in using it, and, most importantly, what is

learned from the information gathered. This leads to developing strategies for program improvement.

ASQ: Supporting Program Assessment through the Years

ASQ was first developed in 1991 and has changed over time to meet the needs of programs like yours. ASQ

originally stood for “Advancing School‐Age Childcare Quality” and was used as the basis for the National

Afterschool Association program improvement and accreditation system.

Today ASQ (now known as “After‐School Quality: The Process of Program Improvement”) has been

reformatted to support anybody who is interested in improving the quality of their program—for self‐

assessment purposes, state level quality rating system advancement, and as a way to become accredited

from the Council on Accreditation. In each case, ASQ is a flexible, effective tool for ensuring that the

process of improving program quality comes from a team of people who are invested in and care about

your program.

‐ i ‐

Welcome

In gratitude to the original authors of ASQ, the many individuals that have used ASQover the decades in support of youth, and those who contributed to this version.Please turn to page 32 where we acknowledge our authors and supporters.

National Institute on Out‐of‐School TimeA program of the Wellesley Centers for Women at Wellesley College

May 2012

Copyright © 2011 National Institute on Out‐of‐School Time. All rights reserved. Programs that have purchased this material for internal self‐assessment and program improvement are allowed to copy this guide for use by their staff and ASQ Team. For all other organizations and purposes, no part of this work may be reproduced without prior written permission.

Introduction

p. iii Overview & Goals

p. iv Introduction to the ASQ Process

p. vii The ASQ Approach

Step 1: Preparing the Program Community for Improvement

p. 2 Overview & Goals

p. 3 Prepare the Staff

p. 5 Prepare the Program Stakeholders

Step 2: Preparing the ASQ Team


p. 9 Select the ASQ Team and the Tools

p. 12 Train the ASQ Team

Step 3: Gathering the Information


p. 16 Observe the Program

p. 19 Complete the Questionnaires

p. 21 Summarize the Information

Step 4: Creating the Action Plan


p. 26 Make Sense of the Information

p. 28 Share the Information

p. 30 Create the Action Plan

Step 5: Taking Action


p. 34 Coordinate the Tasks of the Action Plan

p. 35 Check on the Process of Program Improvement

p. 36 Solve Problems

Acknowledgements

p. 37 Our Authors and Contributors

Table of Contents

‐ ii ‐

After School Quality: The Process of Program Improvement (ASQ) is a five‐step process

that helps you and a team of people committed to program improvement examine the

quality of your program and then make changes that will lead to better practices. You

will involve the program community in an open dialogue about your program.

Together, you will set a program vision and assess how well your program is meeting

the needs of the children, youth, and families it serves. Your ASQ Team will develop an

action plan to reach improvement goals. The ASQ process takes into account that

"you"—administration, staff, families, children, youth, and community members—are

the best people to make the changes that will improve your program.

ASQ is the process you will need to make the most of the program observation tools

and questionnaires/surveys you are using. This guide will help you understand the ASQ

five‐step process and how to use it. Each step will be discussed in depth with an

overview and goals, and tips for success. In addition, the ASQ Web Portal at

www.niost.org contains downloadable resources to accompany each step. This guide is

intended to provide a starting point for implementing a process that will support your

program quality assessment efforts. ASQ Team leaders should feel free to adapt the

materials in the way that makes sense for their programs.

OVERVIEW

• Become familiar with the ASQ process

• Understand the ASQ approach

• Prepare to launch the five steps of ASQ

GOALS

Introduction

Visit the ASQ Portal for More ResourcesAt www.niost.org under Our Work/Training, the portal link is http://www.niost.org/asq_login

As a companion to this guide, the ASQ Portal contains materials to support each step of the ASQ process. Resources include customizable forms and letters, step‐by‐step instructions for running meetings, and more.

‐ iii ‐

INTRODUCTION TO THE ASQ PROCESS

What Are the Five Steps in the ASQ Process?

Step 1: Preparing the Program Community

ASQ involves people in defining the problem and discovering the solutions

Introduce staff and the broader program stakeholders to ASQ and program improvement,

and clarify what they want children and youth to gain from the program.


Select the ASQ team, determine which tools, questionnaires and surveys

to use, train and prepare team for observation and survey collection.


Learn how your program is doing through observation and questionnaires surveys.

Review the “big picture” by summarizing your data.


Identify areas of strength and areas that need to be improved, set priorities, and identify

strategies for improvement.


Coordinate the tasks set forth in the Action Plan and check on the process of

making program improvements.

Who Uses ASQ?

Any organization that wants to improve the quality of their afterschool program. In the past,

programs found they were most successful implementing ASQ when they were financially stable and

not undergoing significant changes. Ideally programs will have committed leadership that has been in

place for several years, several staff members who have been with the program for some time,

agencies that fully support their improvement efforts, and actively involved families.

Who Makes the ASQ Process Happen?

The ASQ process is set in motion by an ASQ Team leader. The team leader is usually

the program director, but it can be anyone with the time, authority, and enthusiasm

to make changes in the program. The Team works best when it includes

representatives from all the stakeholders in the community, including staff, families,

older children or youth, program administration, and host sites (school, community

center, etc.). If your program is not ready to involve the larger community of

stakeholders, it is fine to begin the ASQ process with internal staff as a first step

towards involving a broader team.

Do People in the Program Who Are Not on the Team Have a Say?

Absolutely. ASQ encourages two meetings that are held for all who are interested in the success of

the program—one at the very beginning of the process, and one toward the end. Families, older

children or youth, staff, administration, the host, and other interested members of the program

community are encouraged to attend so they can learn about ASQ, program quality, and progress

made by the ASQ Team.

‐ iv ‐

ASQ involves peoplein defining the problem and discovering the

solutions.

ASQ is designed to be flexible. Ideally the ASQ Team represents all

program stakeholders. If you are not ready for a

team of staff and stakeholders, you can begin using ASQ with

just staff.


How Does ASQ Support Your Work with Quality Assessment Tools?

You may already be working with a specific program observation tool or questionnaires/surveys. If

you have been using these tools with your staff only, ASQ recommends expanding involvement to a

new, broader team by holding a community meeting and inviting other stakeholders to get involved.

If you have not yet chosen questionnaires/surveys or an assessment tool or are not required to use

specific tools, look to Step 2 to learn how to select the tools that make sense for your program.

How Long Does the ASQ Process Take?

The time it takes to assess your program and develop an action plan depends on your program, your

program’s needs, and who is involved. It may take several months to a year. Once you identify areas

that need improvement, it is best to choose a few areas to begin working on and then create an

action plan. The time it will take to accomplish the goals in the action plan depends on how ambitious

the action plan is and what resources will be needed to implement it.

This process is a continuous cycle (see Figure 1 on pg. vi). You will continue to reflect on your practice,

identify areas of improvement and establish better practices.

Can You Use ASQ When You Are Pursuing COA Accreditation?

Yes! The ASQ process allows for effective self‐assessment, making it a valuable tool for any program

that is in the process of obtaining accreditation from the Council of Accreditation.

How Will You and Your Program Benefit from the ASQ Process?

• You will have a way of communicating what is working well to those directly connected to your

program, as well as to the larger community.

• You will demonstrate to the families who rely on you for out‐of‐school time programming that

you are truly interested in meeting their needs.

• You will gain increased parental involvement. This may be the first time that families have a say in

the decision‐making process through questionnaires/surveys, speaking up at program‐wide

meetings, or being on the ASQ Team.

• You will provide a way to involve children and youth in planning the program. Have them

complete questionnaires/surveys and involve them in the ASQ Team.

• You will ensure that families and stakeholders gain a deeper understanding of what it takes to run

a quality program.

• You will gain both increased support and valuable feedback by opening up the process to the

whole program community.

• You will build relationships among staff, families, the host and the program community. ASQ is a

team‐building experience.

‐ v ‐

Questionnaires Available at the ASQ Portal• Visit the ASQ Portal to download ready‐to‐use questionnaires for Staff, Family, Child and

Youth, Host, and Director. The ASQ Observation tool that measures quality items linked to the NAA national standards is available.

• In addition, go to www.niost.org for information on useful tools such as the APT and SAYO tools which make up the Afterschool Program Assessment System (APAS) system.


How Will You Benefit from the ASQ Process?

• You will be energized by the valuable input of people who care about the program.

• You will learn a process that you will be able to use as an ongoing tool for improvement. You will

make time to reflect on the way you do your work.

• You will have the opportunity to compare the way you work now to quality standards.*

• You will become part of a national movement to work on program improvement through self‐

study.

• You will learn better ways to communicate to people within and outside the program what your

program does well.

• You will be able to clarify your goals for the future.

• You will improve your observation skills.

• You will share leadership, which will reduce the burden on the director by sharing the power and

sharing the work.

‐ vi ‐

Set Goals& Select Outcomes

Document& Track. Collect Data

Reflect on Findings & Make

Action Plan

Make Program Changes

Re‐Assess.Collect Data

Figure 1: The Continuous Improvement Cycle

*You will be using observation tools and questionnaires to gather information about your program. In general, these tools are research‐based instruments which focus on standards of quality for out‐of‐school time programs in the areas of relationships, activities, program planning, staff development, administration, etc.

For information on national standards visit the National Afterschool Association at http://www.naaweb.org/default.asp?contentID=612. The NAA Standards for Quality School‐Age Care, are divided into 36 “keys” of quality. NAA offers these standards to programs via two publications.

THE ASQ APPROACH

There is always room for improvement.

Every program can become better by building on its strengths. Children and youth grow, learn, and

change every day. We must be ready to change and improve to meet their needs. Everyone with a

connection to the program can suggest ways to help meet the program’s goals.

Change which involves everyone is the most successful.

ASQ is based on the belief that the most successful program improvements happen when the key

stakeholders are involved in slow, gradual change. This is the same belief found in the Japanese

model for organizational learning known as the Kaizen model. It is human nature to want things to

remain the same. But sometimes resistance comes from not being clear about what needs to change

and why. When you are part of the process, you can see and understand the need for changes. It is

then easier to let go of the reluctance to change.1

Slow, gradual change will last.

Another principle from the Kaizen model is that it takes time to change. Start with a few changes and

work on them over time. See what happens before you take on other changes. Work at a pace that is

comfortable for your program. Small but constant improvement that involves everyone is apt to have

lasting effects. Broad, sweeping change is hard to maintain.

People working together are the best resource.

When people openly share ideas, creative solutions can result. When you respect each other's ideas,

you will find that you can handle problems you thought were impossible to solve. Using the ASQ

process, programs have produced amazing results without spending much money.

Sharing the leadership role will increase participation.

An important concept for the ASQ Team is shared leadership. What do we mean by that? The success

of ASQ really depends on the strength of the ASQ Team. This team is made up of a leader and

members. The contribution of every member of the team is necessary for the process to work. Shared

leadership does not mean no one is in charge. It means that all group members participate by sharing

leadership behaviors. See the “Who Does What?” chart (page viii) for a description of the

responsibilities of the team leader and team members.

‐ vii ‐

1Masaaki Imai, KAIZEN: The Key To Japan's Competitive Success. New York: Random House, 1986, pp. 2‐7.

THE ASQ APPROACH

‐ viii ‐

2Peter Senge, The Fifth Discipline: The Art and Practice of the Learning Organization. New York: Doubleday, 1990, p. 6.

Who Does What?

ASQ Team Leader ASQ Team Members Ideally Will

• Organizes meetings • Introduces agendas • Makes sure the process is moving along • Encourages leadership behaviors • Gives the group an opportunity to talk about the process

• Coordinates the work of the action plan

• View each other as valued players • Listen to each other • Contribute to meeting agendas • Summarize what is being said • Identify problems • Suggest solutions • Point out conflicts • Encourage each other • Work toward general agreement • Implement the action plan • Solve problems which arise

General agreement comes with compromise.

Any group of people is bound to have a range of opinions. Sometimes the result is conflict. Opening

up the process to many points of view can present new possibilities. Negotiation and compromise are

the way to reach realistic goals. Work toward general agreement. Search for strategies that everyone

can live with.

One change will lead to another.

Change has a ripple effect. For example, you may decide to create new interest areas in your

program. This simple act will probably spark changes in children’s or youth’s activity choices. You may

notice changes in their behavior. Observing how a change in one area of the program affects other

areas is an important part of the ASQ process.

Make time for reflection.

Build in time to reflect on your findings as well as the process of self‐assessment. Reflection involves

pausing to examine and inquire about beliefs, goals, and practices. Looking at what you do, and how

and why you do it can lead to deeper understanding.

ASQ helps you create a "learning organization"

In his book The Fifth Discipline, Peter Senge defines “learning organizations” as organizations where

people continually expand their capacity to create the results they truly desire, where new and

expansive patterns of thinking are nurtured, where collective aspiration is set free, and where people

are continually learning how to learn together. Learning organizations "continually enhance their

capacity to realize their highest aspirations.”2 Senge has identified five core disciplines that influence

the ability of organizations to "learn": Personal Mastery, Mental Models, Shared Vision, Team

Learning, and Systems Thinking. Many ideas from these disciplines have informed the philosophy and

practice of the five steps of the ASQ.

THE ASQ APPROACH

‐ ix ‐

3Senge, p. 7, 8, 9, 10, 13, 208‐9, 239.

How ASQ Aligns with Senge’s Learning Organization Disciplines3

Learning Organization Discipline What ASQ Does

Personal Mastery• Clarifying and deepening our personal vision

• Focusing our energies • Seeing reality objectively • Developing needed skills

The ASQ process encourages staff, community, and ASQ Team members to state their goals and priorities. During the ASQ process, Team members learn to observe and rate what they see. They also compare what they see to what others see. The questionnaires/surveys help Team members to understand others' points of view. Team members are asked what training they need to master needed skills.

Mental Models• Deeply ingrained assumptionsthat influence how we understand the world and how we take action

• Be open to hearing new ways of looking at some old problems

As ASQ Team members listen to people discuss the program, they may discover that it has been stuck in a pattern that is not best serving the needs of children and youth. For example, the daily routine may be to serve a snack as soon as the kids come in. But other models are possible: a snack can be offered at different times, or it can be served buffet‐style and left out because kids are hungry at different times.

Shared Vision• A pull toward a goal which people truly want to achieve

• Shared “pictures of the future” foster genuine commitment rather than compliance

During Step 1, the program community meets to establish a shared vision. The ASQ Team will revisit this vision as it sets goals and strategies for an action plan. ASQ Team meetings provide time for a continual dialogue about individual and shared visions for the organization.

Team Learning• Starts with “dialogue” so people can enter into a genuine “thinking together”

• Collectively we can be more insightful than we can possibly be individually

The focus of the ASQ process is teamwork. The ASQ Team assesses program quality and discusses what has been learned. The action plan will be developed with analysis from the group as well as input from the whole community. The Team works together to solve new problems that arise as the plan is put into place.

Systems Thinking• You understand the system by contemplating the whole

• Continually discovering how people create their reality and how they can change it

The observations and questionnaires/surveys typically cover the elements of program quality. These elements are interconnected. For example, you might consider offering youth more activity choices. The attempt to solve one problem may inadvertently solve another. You may find that by changing the schedule for outdoor physical activity and homework, that the children may be more focused when they return from outside.

STEPPreparing the Program Communityfor Improvement

Prepare the Staff

Prepare the Stakeholders( Staff | Family | Children & Youth | Host )

Step 1 Step 2 Step 3 Step 4 Step 5

1

STEP 1 Preparing the Program Community for Improvement

From the moment you decide to improve your program, participation is the key to

success. The process begins by introducing staff to the ASQ process. They may want

some time to become familiar with the ASQ approach and the five steps of the process,

as well as local or national quality standards. When you are ready to involve other

program stakeholders (families, children, youth, the host, and others who play a role in

the program), start with a program‐wide meeting to clarify what benefits the

stakeholders want children and youth to gain from their program experiences. In

addition to a community‐wide meeting, reach out to those who cannot attend a

meeting through phone interviews, surveys, and conversations at pick‐up time with

parents or guardians. Once general agreement is reached on your vision and goals,

allow plenty of time for people to play a part in the decisions and changes.

OVERVIEW

• Introduce staff and the broader program stakeholders to ASQ and program improvement

• Clarify what staff and program stakeholders want children and youth to gain from the program

• Involve staff and program stakeholders in deciding to work on program improvement

GOALS

Step 1: Preparing the Program Community for Improvement ‐ 2 ‐

PREPARE THE STAFF

Success depends on community involvement, and the foundation of that success is the "buy‐in" of staff,

the people who work to create great experiences for the children and youth. They are on the front lines

and have important contributions to program improvement. Many of the changes will need to be

implemented by the staff. This means they have to believe in the goals of the program and the strategies

used to reach those goals. Sometimes staff feel overworked and underpaid. They may feel nervous

about program improvements when they feel they are being overworked already, being observed or

worrying that their needs are not understood by the community they serve. It is wise to invest the time

needed to ensure staff are comfortable and invested with what is happening.

Start with yourself.

If you will be leading the drive for improvement, think about what this role will mean. How much

time will it take? What supports will you need?

Introduce the idea of ASQ to your staff.

Take a few minutes during a staff meeting to introduce ASQ and suggest that you schedule time at

the next staff meeting to consider working more formally on program improvement. Remind staff

that meetings often provide time for reflection on program practice. The ASQ process takes reflection

a step further. It takes a comprehensive look at the program through gathering information using

surveys, questionnaires, or observation tools. It involves other members of the program community.

Tell staff that this meeting will be an opportunity for everyone to step back and take a look at the “big

picture”. See page 3 for a sample meeting agenda.

Find ways to compensate staff for time spent on work issues.

Pay staff for the meeting time, allow comp time, offer free training opportunities, or provide some

other special recognition.

Step 1: Preparing the Program Community for Improvement

“ We have always spent time in staff meetings to talk about how things are going, but we have never done a formal assessment, nor invited parents and others in the community to share their opinions about the direction of the program. We are excited and nervous at the same time.” — Program Director

‐ 3 ‐


SAMPLE AGENDA

ASQ Resources available on NIOST Portal• Tips for Preparing Staff • Sample Meeting Agenda – Staff Meeting • Leader’s Guide to the Staff Meeting • What Is Strong and What Could Improve?

Note: The times listed in all sample meeting agendas are suggestions. Feel free to modify the time allotments based on the size of your group

A staff meeting is a great way to introduce the ASQ process and establish buy‐in from staff.

Sample Agenda: Introducing Program Improvement to Staff

Goals• Involve staff in deciding to work on program improvement• Introduce ASQ process

5 minutes WelcomeThank staff for their time and review the agenda.

10 minutes Establish ground rulesSet guidelines to help people feel comfortable participating.

15 minutes Warm‐up exercise: “Looking at what we do”Ask staff to brainstorm a list of rewards and challenges in their work. Look for patterns within each category and notice similarities/differences between categories.

35 minutes What is strong/What could improveConduct an exercise to help identify program strengths and areas for improvement.

15 minutes What is ASQIntroduce ASQ and the five steps. Invite staff to ask questions and offer ideas on implementing ASQ.

5 minutes Next stepsSet next steps for planning a program‐wide meeting to introduce other stakeholders to ASQ.

PREPARE THE PROGRAM STAKEHOLDERS

Once you and your staff are ready to involve more stakeholders in the ASQ self‐assessment process, you

may wonder if it is necessary to convene a program‐wide meeting. Why not simply ask stakeholders to

be part of the ASQ Team? Let staff know you want the broader program community to "buy into" the

idea of improvement. Allowing staff, families, and other program stakeholders to talk about the

commitment that ASQ and program improvement takes will ensure there will be more support for

changes. It is also important because you begin to create a dialogue between staff, family members, and

stakeholders about goals and visions for the program. Programs that have taken the time to hold a

program‐wide meeting have found it improved communication, brought new energy to the program,

and provided important insights about the goals and value of their work.

Use the program‐wide meeting to build widespread support.

This meeting is your first and best opportunity to energize stakeholders and generate enthusiasm for

the ASQ process. It gives stakeholders the opportunity to “buy into” the ASQ process. See Page 5 for a

sample meeting agenda.

Think carefully about who should be “at the table.”

Invite everyone who cares about your program's success. This could include the custodian of your

school space, the swimming instructor who teaches the youth every week, or the woman across the

street who is patient about the children’s outdoor noise. Be sure to invite the people who control the

money, the space, or the rules about how your program runs, as well as youth attending the program.

Choose a convenient time, date, and place for the meeting for a successful turnout.

Ask your staff, several parents, members of the host community, and even children and youth to tell

you what strategies they think will get the broadest attendance at the meeting.

Publicize to ensure the best possible attendance.

Announce the upcoming meeting through your newsletter, “kid mail,” and other venues.

Prepare the meeting space ahead of time.

Gather all necessary materials in advance, arrange the furniture so people can talk to each other, and

assign someone to take notes so you have a good record of the meeting.

Continued communication is important.

Once the meeting is over, be sure to communicate important information about program

improvement to the stakeholders.

Keep in MindASQ is designed to be flexible. If you are not ready for a team of staff and stakeholders, begin using ASQ with internal staff only as a first step toward involving the broader community.

“ ASQ gave me a concrete way to work together with our school. By having the principal as part of the ASQ Team, she was able to understand some of the challenges we face. It really helped to demystify what happens in afterschool.” — Program Director



SAMPLE AGENDA

ASQ Resources available on NIOST Portal• Sample Announcement for Program Community Meeting • Sample Meeting Agenda – Program Community Meeting • Leader’s Guide to the Program Community Meeting• Memory Exercise Form• What Do We Want for Our Children – Exercise

A program‐wide meeting allows stakeholders to become involved in decision making and clarifies their

vision for the program. Invite everyone involved in the program to this meeting.

Sample Agenda: Program Community Meeting

Goals• Clarify what stakeholders want children and youth to get out of the program• Introduce the ASQ process• Involve stakeholders in deciding to work on program improvement

10 minutes Welcome and Introductions• Introductions• Goals of the meeting• Review agenda• Setting ground rules

20 minutes Memory ExerciseHave participants think about how they spent their out‐of‐school time and compare how things are different for kids today.

45 minutes What do we want for our children and youthConduct an exercise to explore what participants want children and youth to accomplish during program hours. Identify top priorities.

5 minutes Break

20 minutes What is ASQIntroduce the ASQ process. Invite participants to ask questions and offer ideas on implementing ASQ.

5 minutes Next stepsBuild the ASQ team. Broad‐based support

and participation will be necessary to help you achieve the best results from your efforts. To help you accomplish this, make sure you reach out to those who cannot attend a community meeting to include them in the process and keep them informed.

STEP

• Orient the ASQ Team• Train the ASQ Team• Complete a Work Plan


2

Select the ASQ Team( Staff | Families | Children & Youth | Host )

Preparing the ASQ Team

STEP 2 Preparing the ASQ Team

The work of ASQ is done by an ASQ Team that represents the key "players" in an out‐

of‐school time program. The team should include staff, family members, youth and

older children in your program, representatives of organizations that have a say in your

space or budget, and interested community members. ASQ has been designed so

everyone in the program can participate. People do not need to be experts about out‐

of‐school‐time programs to be on the team. In fact, a successful ASQ Team contains

people of many backgrounds who are enthusiastic about being involved and can make

the commitment the ASQ process requires.

During Step 2, you will select and train team members so that they are well‐prepared

for the work ahead. For many members, it may be the first time they have gathered

together to talk about the program. People who have volunteered to be on ASQ Teams

around the country have found it a rich learning experience. Carefully choosing and

training the ASQ Team sets a solid foundation for the work ahead.

OVERVIEW

• Select the ASQ Team that represents the key players in your program.

• Select appropriate tools, surveys, or questionnaires.

• Orient the ASQ Team.

• Train the ASQ Team and complete a work plan for the months ahead.

GOALS

Step 2: Preparing the ASQ Team ‐ 8 ‐

SELECT THE ASQ TEAM AND THE TOOLS

Determining the ASQ team

The best ASQ Team will include people who share the many different perspectives of various groups

of people in the program. Team members might include the program director (most likely as the ASQ

Team leader), staff, people who do the daily work that makes the program run, family members, and

even children and youth. Consider including a board member, a member of the host agency, and key

community people (child‐care advocates, school principals, government officials).

Include the Host Site on the ASQ team

The host is any place where an afterschool program is located such as a school, church, or

community‐based organization. A program has a lot to gain from a good working partnership with the

host. The host may make crucial decisions about program resources. But the "host" is rarely a single

person. If it makes sense, include more than one member of the host community on your team.

ASQ team size

The size of the team depends of the size of the program and the various groups that want to be

included. A medium‐sized program with about 60 children might have a team of eight: two family

members, three staff (including the director), two older children or youth, and a host representative.

A larger program may want to include more people, but it is always important to remember it is hard

to get groups of more than 10 people to meet and make decisions.

The use of a Coach

For many afterschool programs, successful program improvement happens with the support of

coaches—individuals with the technical expertise to guide programs through the process. Who are

coaches and what do they offer? Coaches are typically local experts in afterschool, quality

improvement, and mentoring. They provide ongoing training and technical assistance as the program

assesses quality and develops an action plan. Coaches can:

• answer questions

• keep programs on task by periodically checking in

• help in tool selection, set up databases, and input data

• assist in explaining data and setting goals for program improvement based on the reports.

Coaches can provide key assistance in helping you summarize and make sense of your data. In

addition, coaches help staff establish a climate of dialogue and build capacity for reflective practice.

Coaches build positive, trusting relationships and use open‐ended questions to facilitate dialogue and

inquiry. NIOST is happy to provide coaching or facilitation services.

‐ 9 ‐

“ Participating in this process, as a staff person, brought me to a deeper level of practice. Even though I was really nervous about participating, it was one of the best things I’ve done.” — Assistant Group Leader



For multi‐site agencies

Each site is unique, even when they are all run by the same agency. Consider having a separate ASQ

Team for each site. You may also choose to coordinate the work by having an ASQ Team with

representatives from each site. In this case, each site would have a smaller "action team," which

conducts its own program observation, collects its own questionnaires/surveys, and develops its own

action plan. The coordinating ASQ Team develops the action plan for administrative issues that apply

to all sites.

Qualities of an ASQ team member

• Look for people who like to work with others toward a common goal

• Look for individuals who are sensitive and respectful toward other people

• Look for energy and enthusiasm

Tool selection

Tool selection should be based on the outcomes desired for your program. It could be helpful to have

an internal data specialist or coach to assist with tool selection. You can review the Forum for Youth

Investment’s report “Measuring Youth Program Quality: A Guide to Assessment Tools, Second

Edition,” available for free at: http://www.forumforyouthinvestment.org/node/297

This guide compares 10 youth program quality assessment tools and offers guidance on issues to

consider when selecting and using these tools. Information on the Assessment of Afterschool

Program System (APAS) tools are accessible at www.niost.org.

Sample ASQ Team:

Program director, site coordinator, line staff, board/advisory council member(s), leadership from

a large organizations (i.e. where fiscal and policy decisions are made), host agency (school principal,

school liason, community center representative, etc.), familes, older youth.

How much of a time commitment do ASQ members need to make?

Once your team is in place, the members will need to set aside approximately 15 hours for their core

functions. These include:

• Orientation

• Training

• Program observation

• Making sense of the information

• Developing an action plan

• Checking on progress

• Attending the second community‐wide meeting

‐ 10 ‐

“ I feel that I now really understand how much skill it takes to provide my child and large groups of children with consistently good experiences.” — Parent



What if some of the most valuable team members cannot spare 15 hours?

Sometimes a school principal or a family member who has much to contribute to the team cannot

attend every meeting that has been listed. Programs that have used ASQ say that there are three

important times to have the involvement of key people with busy schedules. They feel everyone

should be involved in:

• the program‐wide meeting, where everyone shares their opinions about the vision for the

program;

• program observation/observation‐meeting, which allows team members to rate the program, or

hear the results of observations

• developing an action plan

‐ 11 ‐

“ The program has been operating in my school for several years. Now I see what an important part of the day this is for the children and what an essential service it provides to families.” — School Principal

“ ASQ is the foundation that openedthe minds of my staff and broughtus together as a team.” — Site Leader


“ This was the first time that staff and family members had talked about the program. It was great to have time to hear another point of view. One night, we got into a long discussion about homework. There had been some tensions about it, but we had never talked it over in a group. In addition to coming to a compromise, I think that each of us learned something important about communication that night.” — ASQ Group Leader

TRAIN THE ASQ TEAM

Now that you’ve selected members of the ASQ team, it’s time to prepare them by running an orientation

and a training meeting. The sample agendas (see Pages 10 and 11) outline suggested content for each of

these meetings.

An orientation meeting helps team members become comfortable working together.

‐ 12 ‐

ASQ Resources available on NIOST Portal• Leader’s Guide to the ASQ Team Orientation• Sample Agenda ASQ Team Orientation Meeting


Sample Agenda: ASQ Team Orientation

Goals• To set a foundation for smooth teamwork• To set up a schedule of team meetings

5 minutes Statement of GoalsUpdate the group on what has happened to introduce the ASQ process up to this point to staff or to the program‐wide community. Review the goals from the list of “What do we want to accomplish for our children” generated at the program‐wide meeting and the “What is strong/what could improve” list identified in the staff introduction meeting from Step 1.

10 minutes Introductions IcebreakerConduct an icebreaker activity to help team members get to know each other.

15 minutes Establishing Ground RulesSet guidelines to help people feel comfortable participating.

25 minutes What’s in a TeamConduct an exercise to come to agreement on ways the group can best work together.

35 minutes What Do We Do When We DisagreeConduct an exercise to learn how to work together and to handle conflict.

10 minutes Next stepsSchedule the next meeting. Review what team members should prepare (e.g., become familiar with the observation tool and questionnaires/surveys, review local or national standards, etc.). Part of the next meeting will be devoted to training ASQ Team members to use the assessment tools.

TRAIN THE ASQ TEAM

‐ 13 ‐

ASQ Resources available on NIOST Portal• Sample Agenda ASQ Team Training Meeting• Leaders Guide to the ASQ Team Training Meeting• Developing a Work Plan handout • ASQ Team Work Plan Chart • Keeping Meetings on Track• Checking In on Group Process• Exercise: Understanding the Developmental Needs of Children & Youth• Handout: The Developmental Needs of Children & Youth

Note: The content of your agenda will depend on the tools being used.


The training ensures all team members share a basic understanding of the standards and tools they will

be using.

Sample Agenda: ASQ Team Training

Goals• Become familiar with local or national quality standards• Learn how to use the observation tool and questionnaires/surveys

5 minutes Welcome

30 minutes Standards of QualityConduct an activity to become familiar with the observation tool and questionnaires/surveys. Examine how these tools relate to local or national quality standards.

60 minutes Observing the ProgramConduct an activity to practice using the observation tool.

25 minutes Developing a Work PlanMake a plan about how work will be done, when, and by whom.

10 minutes ASQ Team Meeting Reporting FormIntroduce a process to keep the entire staff informed about what is happening in the ASQ process (e.g., updates through email, staff meetings, or forms placed in staff boxes). Assign someone to be the staff representative. It is best to have staff on the ASQ Team take responsibility for communication between the ASQ Team and the whole staff. The staff can communicate issues directly with the staff representative to the Team.

5 minutes Next stepsLet all team members know where they can get copies of tools. You will not be meeting again until all the observations have been completed and the questionnaire/survey results have been gathered.

STEP


3

ASQ Team Observes:

• Before school• After school• Kindergarten• Summer• Full day

Program participants fill out questionnaires / surveys:

• Family• Child/youth• Staff• Host• Director

Program observationsare tallied

Questionnaires / Surveysare tallied

Self‐Assessment Summary

Gathering the Information

STEP 3 Gathering the Information

You have arrived at the heart of the self‐assessment process. This is an exciting time of

learning. Depending on the assessment tools you are using, team members will observe

the program and everyone involved in the program will be encouraged to complete a

questionnaire/survey. The program observation process and questionnaires/surveys

are rich sources of information. People experience the program very differently. You

will learn how many people share the same feelings and how strong those feelings are.

As you analyze the information, you will begin to see patterns to guide your work in

developing an action plan.

OVERVIEW

• Learn how the program is doing through the observation process.

• Learn how the program community feels about the program throughfeedback from questionnaires/surveys.

• Review the "big picture" by summarizing the information.

GOALS

Step 3: Gathering the Information ‐ 15 ‐

OBSERVE THE PROGRAM

ASQ Team members often report that observing the program is the most valuable experience in the self‐

assessment process. Staff enjoy the chance to step back from everyday tasks to reflect on how the

program works. Hosts often find new ideas about how to support the program. Families are usually

amazed at how much goes into each program day. The team frequently finds that this observation

experience can bring a deeper commitment to the program improvement work that lies ahead. Here are

some tips to make the observation a positive and productive experience:

Each team member brings their own special perspective to the observation.

Remember, there are no right or wrong answers when it comes to what you see in the observations.

The strength of the team process is that everyone will contribute something. You may see things that

other people will not; others may see things you do not.

Come prepared.

Review the observation tool that you have chosen to use the night before. Bring it on a clipboard,

along with a pencil for easy rating. Dress comfortably and be prepared to go outside when the

children and youth do. Arrive early enough to get a short tour from the director before the children

and youth arrive and confirm that you are observing a typical day.

See the the full range of program activities.

All members of the team should visit the afterschool program as an “observer” rather than as the

role they usually play (i.e. parent, director, etc.). It is recommended to see the program in its entirety,

from arrivals and departures, transition times, and indoor and outdoor activities. Three hours should

be enough time to observe. At least two members of the team should visit other program

components: before‐school, kindergarten, full‐day, and summer. You will find that each part of the

program may have different strengths and challenges.

‐ 16 ‐

Consider this

• Watching staff and children struggle with a space that is too small and has few supplies means much more to an observer than hearing about it at a meeting.

• Feeling either the stuffiness or the chill in a program's space motivates people to search immediately for solutions.

• Seeing how hard the staff work to care for and challenge the youth inspires the team to increase its support for the staff.

“ By watching the program in action, I realized how difficult it is to have so many program activities in the gym. I want to explore other space possibilities, so the gym can just be used for sports and games.”— Host

“ Having the afternoon to observe the program was one of the best things I've done all year. I have a renewed sense of what the day feels like for the kids, and I have new ideas for the space and for our schedule.” — Staff Member


OBSERVE THE PROGRAM

Be respectful of program staff.

Even the most experienced staff can feel "on the spot" when being observed. Remember that they

are working hard every day to create a good program for the children and youth. Before you begin

observing, make sure you are introduced to each staff member and refer to them by name. While you

observe, be aware of your facial expressions and body language. Maintain a neutral expression—you

do not want staff to worry that you look displeased or uninterested. Speak to other observers only

when necessary. Most of all, maintain confidentiality about what you observe. For example, if you

hear a staff person mishandle a situation with a child, do not discuss this with your family and friends.

When you describe it to the ASQ Team, describe the situation without naming the staff member. Only

the director needs to be told the name of the person in order to address the situation during

supervision. Also, do not discuss the observation with other team members in public places (e.g.

grocery store, restaurant, public area of the program).

Make careful ratings

Move through the whole program so you can see as much as possible. Write down what you see,

then decide on your ratings while the visit is fresh in your memory (some tools may require that you

rate all before you leave the observation, some may say rate and revisit). Rate fairly. Do not rate high

because you feel sympathy for problems you see or low because something annoyed you. If you are

unable to decide on a rating because you do not have enough information, ask the director questions

to learn what you need to know. Ask open‐ended questions that start with "What do you do about

...?" or "Tell me about ..." You will get better information than if you ask "Do you ...?" which can be

answered with a yes or no. If it was raining when you visited the program, you will need to ask: "What

do the children and youth do on a typical day for outdoor play?“

‐ 17 ‐

ASQ Resources available on NIOST Portal• Checklist for Program Observation • Sample Program Observation Confirmation Letter

“ As a parent, I had never really taken the time to watch the program in action. I was always in a hurry to pick up my child. It was a wonderful experience to spend the afternoon at the program, using the program observation. It made me realize how much goes into making a program that works well for my 5-year-old AND for my 10-year-old. I am so impressed with the staff.” — Parent


OBSERVE THE PROGRAM

‐ 18 ‐Step 3: Gathering the Information

Questions & Answers about the Observation

How can we make the time for staff to observe?

One of the first hurdles the ASQ Team leader faces is how to free up the staff for an observation. You

may feel tempted to skip this step in the self‐assessment. But staff members often feel this is the most

valuable step in the process. Here are some ideas:

• Most programs start by spreading the observations over a period of days so only one staff person

is missing at a time.

• Some programs hire a substitute or ask a volunteer or agency staff member to step in, making

sure they meet regulations in filling the empty spot. Substitutes should not be observed.

• Other programs have the director, site coordinator, or specialist fill in.

Plan to leave money in the budget to cover this cost, or add it to a request for funds to support self‐

assessment from a local foundation or civic organization.

Should the observers visit every room of the program?

Visit every room and as many activities as possible. Staying for longer periods per activity is preferable.

Observers can comfortably view four to five rooms and outdoor areas in the course of an afternoon

observation. If your program is spread over more than four areas and serves more than 80 children,

observers will find it difficult to see everything in one afternoon. In this case, you can divide up the

program so each group of observers works in only one‐half of the program. You will then analyze the

observations of each area separately.

How does the observer make the best use of his or her time in the program?

Timing is often a problem both for observers and programs. We suggest seeing an activity from start

to completion during your observation. Several visits may be needed to see the full range of a

program.

Do we need to observe the summer program?

Some summer programs are quite similar to the school‐year program, but more often, summer

programs are quite different. At the least, the pace of the day and the activities offered are usually

different. It is important to observe summer programs as an important component of your offerings,

but address it separately with its own action plan.

How do we rate a standard when staff and their actions differ so much?

You will be observing a number of staff members at once and observing hundreds of interactions they

have with the children. Usually, the patterns you observe make choosing a rating easy. But what if

there is a staff member or situation that breaks the pattern?

Imagine you are observing how staff interacts with youth. There are 3 staff members, two are

completely involved with the youth but one staff person is disengaged. Not only is this person distant,

but is so busy straightening the cupboards that he or she refuses to help a child who is crying. How do

you average this into the rating? Ask yourself how much this one staff person and his or her behavior

affects the individual child and the tone of the group. If the individual child quickly gets help from

another member of the staff and the children tend to avoid the problem staff person, you may only

bring the rating down slightly. But if it has a major impact on the child and the group, you’d drop your

rating more. Be sure to describe the situation in your comments and discuss it with the ASQ Team

leader. This is the type of situation that needs attention through staff supervision.

COMPLETE QUESTIONNAIRES/SURVEYS

Questionnaires/surveys are a wonderful way to involve everyone in the self‐assessment process and give

them a say in how the program will change. The team may learn that families love the program, but find

the 5:30 closing time difficult. Children and youth may wish for a nicer outdoor play space and larger

snacks. The staff may want more training. This information will help you create an action plan that takes

the range of experiences into account.

Get lots of volunteers to help.

The ASQ Team leader should help identify parents, staff, and even children and youth who can help

distribute questionnaires/surveys.

Offer convenient times and methods for people to fill out the questionnaires/surveys.

You get the best results when people have time to give thoughtful answers. Staff may prefer to fill out

the questionnaires/surveys at home or at staff meetings. Children may want to have a chance to eat

and play in the gym first. Parents and guardians usually do best if they have a quiet place to fill out

questionnaires/surveys when dropping off/picking up or at home. You can provide

questionnaires/surveys in hardcopy and electronically through email. You might post a PDF of the

questionnaire/survey on your web site for respondents to print. Also consider using a web‐based

survey tool, like Survey Monkey. Note that programs that have mailed family questionnaires home or

sent them by “kid mail” found they had low return rates.

Provide support for family or staff members with limited English proficiency.

For people who speak other languages, translate the questionnaires/surveys or have a translator read

the questions and then write the answers in English so the team can include responses in the

summary. For some people, the telephone interview may be a good option.

Ensure Confidentiality of Responses.

Everyone who fills out a questionnaire wants to feel certain that their responses will be confidential

and carefully reported to the ASQ Team.

Aim for a high return.

The more questionnaires/surveys returned, the more accurate your information will be. Offer

incentives for questionnaire/survey returns. Keep a record of who has filled out

questionnaires/surveys so reminders can be given to those who still need to complete them.


Ideas for Incentives

To encourage participants to complete questionnaires/surveys, try:

• Offering free pizza at pick up time for youth and parents. Stay, enjoy a bite to eat, and complete the survey.

• Giving families a coupon for a small credit of their fee.

• Having the children make or bake something that can be given to their family member who completes the survey.

• Asking local businesses to donate stickers or pens and other small gifts for youth. Try tickets to the movies or bowling for families.

COMPLETE QUESTIONNAIRES/SURVEYS


ASQ Resources available on NIOST Portal• Checklist for Program Observation• Questionnaires for Staff, Family, Child and Youth, Host, and Director• Checklists for the Family, Child/Youth, Staff, and Host Questionnaires • Sample Letters for Family, Staff, and Host Questionnaires • Family Focus Groups • Tips for Running Successful Focus Groups • Tips for Talking with Family Members

Questions & Answers about the Questionnaires/Surveys

If our families do not come in for pick‐up, when can they complete the surveys?

Interview family members by telephone. You will often get fuller responses when you talk with family

members. They may only spend five to 10 minutes filling out the survey, but they will probably speak

with you on the telephone for 10 to 20 minutes. Telephone calls to parents can also be a great learning

experience for the caller.

Provide a brief training for ASQ Team members. Role‐play responses to various issues which might

come up in the conversation, for example, a parent talking too long; a complaining parent. (See the

ASQ Portal for Tips for Talking with Family Members.)

Conduct a focus group meeting with family members.Many parents enjoy the opportunity to meet

with each other to discuss the program. Some family members will feel more comfortable answering

questions in a group than answering by themselves on paper. Some programs use the focus group to

get the surveys completed by every parent. Other programs select just a sample to attend the focus

group.

Here are some tips on conducting a focus group:

• Select a date for the focus group.

• Limit the number of participants to eight to 10 people. This size makes it possible for everyone to

speak.

• Hand out surveys to each person attending if they have not already filled one out.

• Analyze the responses by the date agreed in the work plan.

• See the ASQ Portal for the following resources: Family Focus Groups, Tips for Running Successful

Focus Groups, and Keeping Meetings on Track.

Families enjoy special events. The youth in one program created a play called "Future Program

Improvements." Children expressed through a play their hopes for new and improved snacks, fabulous

field trips, and a new play area. Your program could do the same. Family members could come to

watch this or some other type of performance. Potluck dinners or family pizza nights could include 20

minutes for children and families to complete surveys.

SUMMARIZE THE INFORMATION

You have collected a lot of information from multiple tools (observation tool and questionnaire/surveys)

and sources (families, children and youth, staff, the host, and other stakeholders). Now is the time to

summarize the feedback. This is the first step in examining the big picture without overlooking important

details and feelings.

Summarize your data by following the directions provided by your assessment tool. Here are some

additional tips. It is recommended that at a minimum, an electronic database such as Microsoft Excel be

used for organizing, exploring and summarizing data.

To summarize the information, we suggest that you organize data by program standards or quality

elements. For example, you may look at “Staff relate to all children and youth in positive ways” or

“Children and youth can choose from a wide variety of activities.” For each standard or element, pull

together the data collected from each source (observation tool, child & youth survey, family survey,

director survey, staff survey, and host survey). This will give you a collective impression about how things

are going.

1. Summarize the Observations

Create a summary sheet, or use one provided by your assessment tool.

This sheet should list each person who observed the program, the group they observed, and the

date of the observation. List the ratings for each observer. You may want to use a separate tally

sheet for each separate group observed so you can clearly see the ratings for each group.

Determine an average rating for each standard observed.

When there is substantial disagreement among observers, you can still do a simple average, but

CIRCLE that standard so you can note the disagreement in the self‐assessment summary. You can

note the average and the range. This will alert the ASQ Team to discuss their disagreement when

they are reviewing the summary.

You may also want to circle any standards that receive low ratings and high ratings so they can

also be discussed during Step 4.

Categorize the comments.

Categorize the frequency of comments and summarize (see next page).


Consider working with a partner or coach to analyze responses

It can be useful to work with a coach who can assist with setting up the database. It is also often easier to work in a team especially when you have more than 25 questionnaires/surveys.


2. Summarize the Questionnaires/Surveys

Create a summary sheet or use one provided by your assessment tool.

This sheet should include individual responses from each person surveyed, the number of

questionnaires/surveys completed, total number of possible respondents in the programs, and

the date tallied.

Determine an average rating for each questionnaire/survey item or categorize individual

responses.

Summarize the comments for open‐ended questions.

Summarize comments and write number of times they were mentioned. You do not need to

write out each whole comment. Use short phrases and capture the general flavor. You will not be

able to write down every comment, but be sure to summarize them when:

- the comment explains why ratings are particularly high

- a number of people have a similar comment

- the comment explains why ratings might have been particularly low

For example:

“Five parents want more communication about their child.”

“One parent suggests a sliding fee scale.”

“Three staff members want training on behavior management.”

It is common for there to be disagreements in the comments section. For example, some observers

may feel there are enough materials for the program while others don’t. Bring the information to

the ASQ Team when you meet again in Step 4. Remember that involving many people in the process

means dealing with different opinions. The team will discuss these differences and work toward

general agreement.

Important: Questionnaires and surveys should be anonymous. Do not use any names in your

summary. Delete names or change them to pronouns in the quotes you use.

Address negative information.

You are bound to find some criticism of the program. Criticism not directed at specific individuals

can be discussed by the ASQ Team. The team will need to decide if it is an isolated complaint or

shared by a number of people. If many people mention the issue, you may want to have an

immediate response.

If the negative comments are about specific staff members, it is best to share them with the

director. The issue can then be handled through ongoing supervision. If the issue is about the

director, the team should let the director know and, if necessary, pass the information on to the

Board of Directors or the Advisory Board.



3. Create a Self‐Assessment Summary

Merge all the information to create a rich portrait of your program. You have collected data from

multiple sources informing how things are going at your program. By pulling together all the

information, you will see the "big picture" without losing sight of details.

Each summary should include:

• The average ratings from the program observations

• Comments from anything the observers want to bring to the group’s attention

• The tallied responses from all questionnaires/surveys

• Summarized comments from the open‐ended questions in the questionnaires/surveys

Summarize the most frequently cited comments from the program observation process and

questionnaires/surveys. Note questionnaire/survey items and program observations that received

very low ratings. These are areas that will most likely have priority in the Action Plan. In addition,

include items that have very high ratings and share this information.


ASQ Resources available on NIOST Portal• Tally Sheets for Staff, Family, Child/Youth,

and Host Questionnaires/Surveys• Directions for Summarizing Comments from Staff,

Family, Child/Youth, and Host Questionnaires/Surveys

The tools are a rich source of information. People experience the program very differently which will be reflected in the data but you will begin to see patterns, frequencies, and themes. This will guide your work in developing an action plan.


4

Make sense of the information

• Identify areas of strength • Identify areas that need to improve• Identify priorities

Create an action plan

Share informationwith the program

community

STEP Creating the Action Plan

Self‐Assessment Summary

STEP 4 Creating the Action Plan

Now is the time to put together all the pieces of information to create a portrait of your

program. This will give you a clear idea of the program's strengths and areas that need

improvement. You will have an opportunity to share your work with the community

again and hear their ideas about how to approach your priority areas. Most important,

you will develop an Action Plan that will guide your work for the months to come.

OVERVIEW

• Review the information from the observations and questionnaires/surveys.

• Identify areas of strength and areas that need to be improved.

• Develop priorities for program improvement.

• Share this information with the program community and solicit feedback.

• Create an Action Plan.

GOALS

Step 4: Creating the Action Plan ‐ 25 ‐

MAKE SENSE OF THE INFORMATION

By the time you meet again as a team, the hardest part of organizing the information is complete. Now is

the time to gather the team together to make sense of the information (see page 23 for sample meeting

agenda) and reflect on your findings.

When you look at the information, some information will jump out at you. It is natural to start thinking of

strategies as soon as you identify a problem. Some members of your team will be eager to roll up their

sleeves and get to work. It will be very tempting to let them. However, if you take action at this stage,

you will be doing it without broad agreement from the community. What you think is a high priority now

may turn out to be less important later when you see the whole picture. Finally, you may be using

precious resources that you will later realize should have been used in a different way.

The assessment tools you are using may already provide guidance on how to make sense of the data

collected. Here are some additional tips:

Start by sorting out clear areas of strength.

Depending on the instruments you are using, these may include observation items that receive high

ratings, questionnaire/survey items with positive responses, and specific notes and comments that

are also positive.

Then sort out areas that clearly need improvement.

An area that needs improvement may include observation items with low ratings and

questionnaire/survey responses at the low end of the scale.

Note areas where more information is needed.

Sometimes the information is in conflict. The program observation may indicate that there are a

variety of activities, for example, but over one‐third of the children say they do not have enough

activity choices. Or, perhaps a comment was made that raises questions for the team. These items

will need further discussion and or further information gathered.

Set priorities for working on improvements.

Once you have identified areas of strength and areas that needs improvement, you are ready to

decide whether an improvement item will receive high priority attention or low priority attention.

“ Creating the action plan was a wonderful time of creativity within our program. Everyone shared their best ideas and immediately began bringing in resources to make things happen.” — ASQ Team Member

‐ 26 ‐Step 4: Creating the Action Plan

MAKE SENSE OF THE INFORMATION

‐ 27 ‐

ASQ Resources available on NIOST Portal• Sample Meeting Agendas‐Make Sense of the Information Meetings• Leader’s Guide to Make Sense of the Information Meetings


Working together to make sense of the information will help set the stage for creating the action plan.

Note: This sample agenda below combines two meetings into one to show the steps that will take place

at this stage. Most programs will find that these steps take time and therefore are best conducted over

two meetings. On our portal, there is an agenda and leaders guide for these steps as two meetings.

Choose how you would like to present the information.

Sample Agenda: Make Sense of the Information

Goals• Create a list of strengths and areas for improvement• Prioritize areas for improvement• Build the foundation for an action plan

5 minutes WelcomeThank everyone for their efforts so far and congratulate yourselves on having come this far.

10 minutes Warm‐up ExerciseConduct an activity to help team members share their experience of working on ASQ so far.

90 minutes Strengths and areas for improvement• Small group work • Reporting backAsk small groups to use the summary sheets created in Step 3 to develop a list of strengths and areas for improvement. Have each group share their lists. Then review the results of the entire team. If you discovered an issue that needed further discussion, explore that issue before you go further into the agenda.

15 minutes Checking in on group progressConduct an exercise to check that the team is working well.

90 minutes Prioritize areas for improvementCome to an agreement on the level of priority for each area for improvement.

5 minutes Next stepsPlan who will present your results at the program‐wide meeting.

SHARE THE INFORMATION

During Step 1, you held a program‐wide meeting at which the ASQ process was discussed and approved.

Now is your opportunity to share your findings with the community and be sure the community is

comfortable with them. Remember, improvements will be easier to make if everyone is on board with

you.

The best way to share the information is to hold a second program‐wide meeting.

When people work together and build on each other’s ideas, often the soundest strategies will

emerge. Programs that chose this route reported that the meeting was the kick‐off to a huge

volunteer effort for putting the chosen strategies into place. See page 25 for a sample meeting

agenda.

If you think a meeting will have low attendance, find other ways to involve the community.

Create a brief written report and share it with the Board of Directors, at a staff meeting, and in the

program newsletter. You might also call families directly, create a video presentation that families can

watch at home, or tap expertise outside of the program for new ideas. Go over the list of areas that

are strengths and those that need to be improved. Ask for feedback and ideas for solutions.


SHARE THE INFORMATION

‐ 29 ‐

ASQ Resources available on NIOST Portal• Sample Meeting Announcement • Sample Meeting Agenda Program Community Meeting• Leader’s Guide to the Program Community Meeting


By sharing findings with the program community, you can establish support and invite the community to

identify strategies to address areas for improvement. Note: the purpose of this meeting is to pre

Sample Agenda: Program Community Meeting

Goals• To present the findings of the ASQ Team which identify program strengths and areas for improvement.

• To celebrate program strengths. • To brainstorm specific strategies which can help address the areas for improvement.

10 minutes Welcome and Introduction• Goals of the meeting• Review of the agenda• Setting of the toneWelcome everyone, introduce the ASQ Team members to the group, and review the agenda.

30 minutes The work of the ASQ Team• Presentation• DiscussionBriefly summarize the steps in the process that the ASQ Team completed and present the findings. Discuss the areas for improvement and prioritize the ones most important to children and families.

40 minutes Generating strategies for improvement• Small‐group brainstorming• Reporting back• Whole‐group feedbackAssign a priority to each small group and have them brainstorm strategies to address the priority. Have small groups share their ideas and gather additional ideas from the whole group.

10 minutes Introduce the ASQ Team Action PlanExplain that the ASQ Team will be using ideas generated at the meeting to develop an Action Plan. Let people know the plan will be made available to all.

5 minutes Next stepsAsk for a show of hands from people who would be willing to work on specific improvements.

As you share the information, be sure to highlight program strengths and acknowledge people who work hard to keep the program running.

CREATE THE ACTION PLAN

You have learned a great deal about your program. You are about to create a plan which will improve

the program for the children and families who rely on it. Take a moment, as a team, to feel good about

the work you have done. Here is a 10‐step guide for developing the Action Plan. This guide will steer you

through the brainstorming process, evaluating strategies most likely to succeed, and allocating

resources. Remember that small but constant improvements that involve everyone will have lasting

effects on your program. Start by making a few small changes and build off these successes as your team

continues to work on program improvement.

ASQ Resources available on NIOST Portal• Action Plan worksheet • Problem Solving Model

Remember that participation is the key to successful change. Assume that people want to help. Spread the work around. If everyone is involved in making the improvements, there will be less resistance to change. There will also be a greater sense of community spirit.

Constant, gradual change is change that will last. Start with the goals you can realistically handle.

“ There were moments when I felt our plan was too ambitious, but as everyone pitched in, I knew we would reach our goals. And it wss a relief to have everyone involved, and to feel it wasn’t my job alone to tackle these issues.” — ASQ Team Leader

Step 4: Creating the Action Plan ‐ 30 ‐

CREATE THE ACTION PLAN

Guidelines for Developing an Action Plan

1. Review your list of areas of strength and areas that need improvement. Then review the

priorities and strategies that have been generated from the ASQ Team and the program

community.

2. Combine the issues that can be solved by addressing one goal.

As you look at the information you have, you will see that several ideas and areas of concern

really fit into one category. Be careful not to make a goal too broad.

3. Write clear goals that specifically name the actions you will take.

This will help when you have to come up with strategies for each goal.

4. Generate possible strategies.

Start by reviewing the strategies generated by the ASQ Team and the community. Then add

other possible strategies.

5. Choose the strategies most likely to succeed.

For each strategy, consider:

• How much will it cost?

• Who would undertake the effort (families, staff, children and youth, other stakeholders)?

• How much work is involved?

• What are the advantages and disadvantages?

• Can it be done? Is it realistic?

6. Assign Tasks. Assign tasks to the people who know the most or care the most about a particular goal.

7. Decide what resources will be needed to meet the goal. Decide how much the plan will cost and how many people will have to be involved. You may need to accomplish your goals in stages. Work on one to three of the top priority goals in the first year. Save the low‐priority goals for the second year or later.

8. Create a timetable. Be realistic about when you can complete the task. Check to see how the timetable might be affected by other goals and tasks.

9. Review the completed plan.

Look over your work and ask these questions:

• Is it realistic?

• Do we have enough resources to complete it?

• Have we picked the best strategies?

• Is it clear who is working on each task?

• Will people in the program help to put the plan into action?

10. Send an outline of the plan to everyone in the program.

This Action Plan is a map for their future and you will need their help to implement it.



STEP Taking Action5

The ASQ Team and Program Community

Checking in on Tasks

Checking in on Progress

SolvingProblems

CelebratingSuccess

STEP 5 Taking Action

During Step 5 you will finally begin to see some of the results of your work. You will feel

the excitement that comes with making changes to improve your program. However,

things don't always go as you may expect. You will be listening to all members of the

community and watching for signals that the plan may need to be adjusted. Remember,

when you are guiding change you have two jobs that are equally important. The first is

to keep track of the progress toward the goals you have established. The second is to

keep track of the process and the feelings people have about making changes.

OVERVIEW

• Coordinate the tasks set forth in the Action Plan.

• Check on the process of making program improvements—and celebrate the successes.

• Solve problems in areas which are not progressing as well as expected.

GOALS

Step 5: Taking Action ‐ 33 ‐

COORDINATE THE TASKS OF THE ACTION PLAN

Program improvement means getting involved in the process of change. Some changes will require

simple, decisive action. Other changes are complex and require more thought and effort. To successfully

accomplish your goals, your team must have effective leadership and work well together. It is important

to check on progress every four to six weeks to get the most from your efforts. Here are some tips to

keep you moving in the right direction to ensure all tasks laid out in the Action Plan are accomplished.

Give each person involved in the Action Plan a full copy of the plan.

It is important that each person involved in implementing the Action Plan sees how his/her task fits

into the larger picture.

Make it clear how each person reports back on progress or problems.

Check on progress regularly so you can catch problems before time is lost or people become

discouraged.

Continue to build "buy‐in" and commitment to the Action Plan.

It is essential to let people in the program community know about your progress and problems.

Decide how you will update the full staff, families, the host, and key stakeholders.

Check back to make sure goals are met.

Spot checks with staff, families, children, and youth will help you learn if the strategy developed by

the ASQ team is working the way you planned. When all goals in the Action Plan have been met, the

ASQ Team can conduct a program observation to check on overall progress.

‐ 34 ‐

Questions & Answers about Tasks

What if the ASQ Team is not able to carry out the Action Plan?

Many programs hope the ASQ Team will be able to continue the self‐assessment work until the Action

Plan is complete, but sometimes this is not possible. There may have been a lot of turnover on the

team, or team members might no longer have the necessary time. Programs that have experienced

this have shifted the work over to others. The staff team or a committee from the Board of Directors

might take over the work. No matter who takes on the responsibility, it is important that

representatives from each group of stakeholders still have a chance to comment on the progress of

the Action Plan.

What do we do if a change leads to unanticipated results?

Change can have a ripple effect—one change often leads to another. Some of these changes may be

unanticipated, but positive. Other changes may not be so fortunate. When you check in to see how a

change is working, be sure to ask if it has had any unanticipated results. Take these results into

account as you fine‐tune your Action Plan.


CHECK ON THE PROCESS OF PROGRAM IMPROVEMENT

Sometimes we welcome change, but more often, change can feel unsettling. Some people may want to

race ahead with changes. Other people may feel overwhelmed or resist changes. To be successful in

program improvement, it is important that you pay as much attention to the process of change as to

tasks themselves. Here are some tips on ways to check on how people are feeling.

Take time to celebrate your efforts and successes.

Find ways to let people know about progress being made and publicly recognize the people who are

making things happen.

Check in frequently with the people who are doing the work.

Even those who are 100% behind the Action Plan may feel discouraged or overwhelmed at times. Talk

individually with people to see how things are going. Include time during ASQ Team meetings to

check in formally.

Check in frequently with the people affected by the changes.

Staff, families, children, and youth often give informal feedback about changes being made. Find ways

to check in with larger groups so you avoid assuming that one small group speaks for everyone.

Take the time you need to make successful, lasting changes.

Focus on a few goals at a time. If you take on too much, people may become overwhelmed or the

results might not be as successful as possible.

Note when things are not going well.

As with any developmental process, growth will be uneven. Programs often report it is easy to make

concrete changes. You might sail through improvements in materials, equipment, and indoor space. It

is more difficult to make changes that affect long‐standing habits or behavior. Problems can appear in

how people react to the goals and strategies of the Action Plan, when work is not getting done, or

when there is low energy and much resistance.

‐ 35 ‐Step 5: Taking Action

SOLVE PROBLEMS

It is human nature to resist change. Think of the last time you wanted to change something in your life.

Perhaps you decided to sign up for an exercise class. Even when you feel determined to succeed, things

can get in the way. You may find that money, time, unexpected crises, or your psychological readiness to

change throw roadblocks in your path. Making changes in a program requires the cooperation of many

people. Expect that there will be some problems. Success will depend on your ability to solve some of

the typical problems that arise.

An unexpected crisis occurs.

When a crisis will require the energy of only a small group of people, you can assign the problem to a

small committee. Seeing new successes in the midst of a crisis is often especially important for

morale. For situations where funding is unexpectedly cut, space is terminated, a director leaves, or a

staff member or family experiences a crisis, it might be wise to stop the action on ASQ. The staff

leadership, board and ASQ Team should decide jointly what to do. It may help to ask yourselves how

much effort it will take to resolve the crisis:

• Does the program’s survival depend on resolving the crisis?

• Is the crisis affecting the morale of staff, children and youth, or families?

• Will the resolution take the full energy and attention of the leadership core?

• Is it impossible to offer a quality experience to children and families until the crisis is solved?

If you answer “yes” to any of these questions, it might be best to temporarily set aside the Action Plan.

Maintaining quality becomes the most important issue. A temporary slow down in the process will not

alter your long range goals.

The NIOST portal has several resources to help guide the director and the team through the change

process and to help find ways to work through issues that may arise.

Congratulations! You have embarked on the journey of program improvement, to make your program

the best it can be for the youth and families you serve as well as those who work at the program and are

affiliated with your organization.

‐ 36 ‐Step 5: Taking Action

ASQ Resources available on NIOST Portal• Questions and Answers about How to Solve Typical Problems • How to Keep Turnover from Derailing the ASQ Process • The Role of the ASQ Team Leader During Stage 5• Progress Report Form

We hope that you find the ASQ process useful. We welcome your comments, suggestions, and feedback. Please contact us at [email protected] and let us know how the ASQ process is being used at your program and what new ideas you have to share.

ACKNOWLEDGEMENTS

NIOST Director Ellen Gannett and all those associated with NIOST who assisted with this project

including Karen Lachance, Jenny Lisle, Rebecca Krumholz, Kristel Dupaya, Melissa Magner, Julie

Dennehy, Wendy Surr, Georgia Hall, Genevieve Solomon , Donna Tambascio, Rebecca Mongeon, Sue

Sours, Gwen Holbrow and Kathy Schleyer, project manager.

Original Authors

Susan O'Connor, Ellen Gannett, Cathy Heenan and Kathryn A. Wheeler.

NAA formerly NSACA

ASQ, the program improvement process, was gifted by NIOST to the National School Age Care

Association, now NAA, almost twenty years ago. At that time, ASQ was the program improvement

process that NSACA required for afterschool programs seeking national accreditation. Embedded

within the ASQ process were the NAA National Quality Standards that had also been developed in

collaboration with NIOST and with support from other leaders and organizations in the afterschool

field. Those standards are reflected in the ASQ Observation Tool and the surveys and questionnaires

that made up the original ASQ. Two years ago, ASQ came back to NIOST. We are grateful to NAA for

returning ASQ to NIOST.

Proponents of the original ASQ that encouraged and supported the development of this ASQ Guide

Susan O’Connor, Leslie Roesler, Susan Brasel, Sara Espinosa, Judy Nee, Lynn Siebert, Betsy Saatman,

Carmen Gatti, Thomasine Washington, Pat Hess, Virginia Brown, and others who helped this project.

And our deepest gratitude to Liz Joye and Juanita Svendsen for their

commitment to ASQ over the years as ASQ instructors, coaches and proponents.

1Masaaki Imai, KAIZEN: The Key To Japan's Competitive Success. New York: Random House, 1986, pp. 2‐7.

‐ 37 ‐

Why ASQ?

ASQ was originally developed as the afterschool program improvement process to guide programs seeking national accreditation. This process was tied to the ASQ Observation Tool and NAA’s national afterschool quality standards. Today, a wide range of afterschool programs use a variety of observation and survey tools.

What hasn’t changed is the need for the process to enact the benefits that comes from using measurement tools. This is why ASQ is still vital. Today’s ASQ, is available on the internet via the NIOST website portal. We now offer the ingredients of the process including all the agendas, forms and other support materials electronically so programs and sites can download them and adapt them to their own needs. And it works with any assessment tools or surveys that a program chooses to use. Join those who are actively engaged in continuous program improvement.

asq - national institute on out-of-school time · welcome to asq —after‐school quality: the...

Documents