Beginning With the End in Mind: Evidence-Based Interventions to Improve Current and Future Outcomes for Youth with Disabilities in Urban Schools Urban Collaborative Fall Meeting October 24, 2009

Got Data! Now What?

Charlotte Y. Alverson April 14, 20111

2

State and Local Educators and Administrators have data3

State and Local Educators and Administrators have datalots and lots of data.attendancegraduationEngagement dropoutGenderRACE/ETHNICITYDisabilityAgeSo 4

State and Local Educators and Administrators have datalots and lots of data.attendancegraduationEngagement dropoutGenderRACE/ETHNICITYDisabilityAssessment BEHAVIORLRE5

In and of itself, having data isnt a challenge for educatorsTeachers, schools, districts, and states collect vast amounts of information. In fact, there is so much data about students, schools, and districts, that the term Data Rich, Information Poor has been used to describe the condition. The challenge lies in having accessible data, The amount of data collected by schools, about schools, students, 6

DataRICHInformationPoorIn and of itself, having data isnt a challenge for educatorsTeachers, schools, districts, and states collect vast amounts of information. In fact, there is so much data about students, schools, and districts, that the phrase Data Rich, Information Poor has been used to describe the condition. We have the numbers, the facts, we dont have is the information, meaning, relevance. The challenge lies in having accessible data, The amount of data collected by schools, about schools, students, 7

Challenges includeHaving access to these data; Having time to review data; Having the right data to drive the right interventions for the right students at the right time, and in the right dosage. DataRICHInformationPoorThe challenge lies in having accessible data, time to review the data to make meaning from them, and having data that inform the work we do. As educators were getting better. 8Weve Got Data! Now What?

9Knowing the dataUnderstanding the dataUsing the dataKnowing what data we haveWhat data are available to usHow to access the dataKnowing the data details. Who are these data about what leavers, what school year, what cohort of students, when were these data collected, the rates and percentages, Knowing what the numbers are

UnderstandingWhat the data mean What the data represent and what the data dont represent How to design graphs and how to read the graph, graphic literacy displaying graphs is a way that accurately representing the data without misleading the audience Making sense of the numbers,

Using What changes do we need to make, what changes do we not need to make whats working, what isnt? For whom is it working, for whom is it not? When is it working and when is it not. 10Knowing the Data11Do you know your state and local data?

Graduation rate?Dropout rate?Employment rate?Enrollment rate?How many females are employed full time? How many students in the 18-22 year old program are working 1 year out of high school? How many students who enrolled in post-secondary completed a term? Knowing - 12Data Use Toolkit Revised

Was Designed to help local educators have a conversation about their post-school outcome data

Structured to guide the audience from Knowing what the data are to understanding the data to using the data

13Examining Local PSO: District Facilitators Guide & PPTOutlines a Process for using PSO dataAssembling the DataOrganizing the DataModifying the PowerPointExamining the DataQuestions for a Guided Discussion Description of each slide in the PowerPoint AppendicesBlank Data TablesMaster Handouts

14Outline of Data Use Toolkit- Revised 15Understanding the dataPurpose for Federal Collection & Reporting Requirements (Why)Indicator 14: Post-School Outcomes (What)Method used to Collect PSO Data (How)Common languageState Response Rates Whos represented in the dataOutline of DUT walks the audience through the information they need to understand the dataWhyWhat data How

Outline of Data Use Toolkit- Revised Knowing the dataState DataMethod of Exit (Graduation and Dropout Rates)Engagement RateDistrict DataMethod of Exit, Engagement RateEngagement rate by DemographicsOutline of DUT walks the audience through the information they need to understand the dataWhyWhat data How

Engagement Rate% enrolled in higher education% competitively employed% enrolled in postsecondary education or training% employed some other

States Engagement Rates Of the [TOTAL NUMBER] youth who responded to the interview/survey across the state

Data Source: Sample PSO SY 200x-0x

Graphs and Charts were designed following the principles of data displayReasons for displaying data visually:To see patterns in the dataTo make comparisons

Shading & colors in the chartsCompare segments of the pie 18

Data Source: Sample (SY 200x-0x) State and District Engagement Rates Facilitator's Notes: (Purpose #3)

Take a few minutes to make observations about the comparison between the State and districts results. Record the observations on chart paper for further discussion.If the state or district percentage in the engagement categories do not total 100%, explain why. In most cases, the number of youth who graduate and dropout will not total 100% of the youth who left in a school year for several reasons: Differences in definitions Missing data- questions not answered Some youth die, or move out of State, and cannot be identified as either

Numbers in the chart are place holders.

To change the numbers in the graph:Right click on a bar in the graphSelect edit data

In the Excel spreadsheet:Change the numbers you want to changePush the return key on the keyboardClose the spreadsheet

Engagement Rate19Percent of Males & Females Engaged Data Source: Sample (SY 200x-0x) Facilitator's Notes: (Purpose #3)

Take a couple of minutes to make observations about whats displayed.

Simple charts to facilitate comparisons;

19Knowing PSO DataKnowing your state and local

The Toolkit is designed in such a way that the audiences moves from hearing about why these data are collected how, when PSO data are collected, broad rates and targets: the engagement rate, response rates to understanding each

20Understanding PSO DataDemographics: Gender, Method of Exit, DisabilityIn school Programs: Participation in specific programs, transition experiencesComparisons: Graduates to Dropouts; Males to Females; Disability CategoriesPredicting Success & Directing Resources Understanding your state and local

The Toolkit is designed in such a way that the audiences moves from hearing about why these data are collected how, when PSO data are collected, broad rates and targets: the engagement rate, response rates to understanding each

21Drilling into the Engagement Rate Looking at the percent of youth engaged in the post-school activities by the following categories: NPSO Data Use Toolkit v.222Females v. MalesGraduates v. Dropouts Various Races/Ethnicities Disability CategoriesWhat other groups do you want to look at?

70% engagement

22Brainstorm questions to answer about the engagement rates of males and females. 23Are Males & Females engaged at the same or similar rate?

Facilitator's Notes: (Purpose #3)

Ask the stakeholder group to brainstorm the questions they would like to answer about the engagement rates of males and females. You may ask people to write questions on sticky notes and read them aloud or display them on chart paper. Place the groups questions on chart paper. Using the data, answer the questions.

Examples of questions: Are males and females competitively employed at approximately the same rate? If not, which group is employed at a higher rate? What do you think contributes to the higher employment rate for some groups? What could the district do to promote competitive employment for groups with a lower employment rate?

23Look for Trends & PatternsIdentify trends in dataWhich targets were made/missed last year?Are things getting better or worse?For whom are things getting better or worse?See what patterns the data show in the districtLook for patterns in classes and students at the school levelYou will use any trends and patterns you see to focus TA efforts24At the district level identify schools that need assistance as well as the specific areas in which they need it. Look for patterns of similar/related indicators showing that there is a particular type of problem.Look for trends across timeare things improving, static, or getting worse?

At the school level, look too for patterns and trends indicating need or developing need. Use these to help determine the type of technical assistance that might help.What do these data tell you?Guiding Questions to Understand the Data General Transition Questions Graduation and Dropout QuestionsCompetitive Employment QuestionsPostsecondary Education/Training Questions

When a student turns 16, how does the district provide explicit transition planning services specifically designed to help students move from high school into work and or postsecondary education and training opportunities?

25# engaged# not engagedParticipated in specific program Did not participate in specific program Drilling into to Program Specific Data: In-School Transition Experiences & Participation in Specific Programs to Predict Success Youth who participated in specific programs were X times more likely to be engaged in work or school 1-year out of high school than those who did not participate in these programs. 129/2498/27 = 5.375/3.629 = OR of 1.48

n= 278we started included this information in 06-07 when school districts included participation in any or all of these activities in the contact/demographic information that they provided

26

Guiding Questions help to Identify Objectives & Define Questions that lead to Using the Data to Improve Programming for Youth with DisabilitiesWhat problem(s) do you want to address?What do you need to learn/accomplish?Do you need to look at additional data or ask additional questions? 27What is the purpose of the investigation? For example, does it seem like dropout is higher in one particular school than in others in the district? OR Do you think that particular groups of students might be dropping out at a higher rate than the rest of the students?

What are your goals for this process? (Learn the truth Address the problem.)

Hope to learn the cause(s) of the problem and to remedy it! Will involve analyzing the data, forming an intervention plan to address the needs, putting the plan in place, following through with it, and assessing progress as you go.

Thinking about Transition Related Program ImprovementUsing DataIn what areas are youth with disabilities doing well? What areas need improvement? What is the district doing well? What does the district need to improve? 28

Facilitator's Notes: (Purpose #4)

Place each question on individual sheets of chart paper. Ask the group to answer each question. Depending on group size, you may want them answer on sticky notes, to allow individual think time, or go one by one around the group.

The goal with this slide is to start to look critically at the district or school program and identify how the program contributes to positive post-school outcomes and how it does not.

If you have several areas that need improvement, you may need to prioritize the needs. There are several criteria that can be used to prioritize needs. Examples of criteria include: Resource investment (people, money, or equipment/materials): no additional resources, few resources, or significant resources time investment: changes can be addressed immediately, in the short-term (defined as .), or in the long-term (defined as ) Degree of change anticipated, Number of youth/families affected28What Does the Research Say?Academic/General Education Career AwarenessCommunity Experiences Exit Exam Requirements/ High School Diploma Status Interagency Collaboration Occupational Courses Paid Work Experience Parental Involvement School Integration Self-Advocacy/Determination Self-Care/ Independent Living Skills Social Skills Student Support Transition Program Vocational Education Work Study Predictors of Post-School Success29NSTTAC, 2008Facilitator's Notes: (Purpose #4)

Compare the predictors to the strengths and area identified for improvement for the district. Are some or most of the predictors evident in your programs for youth with disabilities? If not, what would it take to incorporate the predictors into your program? What do you need to make changes to the district or school programs? Use one Action Planning Worksheet to capture the changes the group wants to make. 29Programmatic StrengthsWhy does [DISTRICT] have positive outcomes for some leavers and not others? What attributes can be associated with the outcomes, positive and negative? What changes need to be made? Think about the 16 predictors of post-school success and our high school transition and academic programs: 30Facilitator's Notes: (Purpose #4)

Encourage the group to take a minute to think about the overall transition program in the context of the predictors of post-school success.

Brainstorm why some leavers experience positive outcomes and other leavers experience negative (less positive) outcomes; capture on chart paper.

The goal with this slide is to start to look critically at the district or school program and identify how the program contributes to positive post-school outcomes and how it does not.

If you have several areas that need improvement, you may need to prioritize the needs. There are several criteria that can be used to prioritize needs. Examples of criteria include: -Resource investment (people, money, or equipment/materials): no additional resources, few resources, or significant resources-Time investment: changes can be addressed immediately, in the short-term (defined as .), or in the long-term (defined as ) -Degree of change anticipated, -Number of youth/families affected

30Using Data for Action Planning What are our next steps? What do we need to do to improve the positive outcomes for our youth with disabilities? 31

Facilitator's Notes: (Purpose #5)

Use one of the Action Planning Worksheets to identify next steps and or specific changes that can be made to improve the positive post-school outcomes for youth with disabilities. 31Data Use Toolkit Revised

Knowing, Understanding & Using Data to Improve Programs for Youth with Disabilities Was Designed to help local educators have a conversation about their post-school outcome data

Structured to guide the audience from Knowing what the data are to understanding the data to using the data

32Pie Chart0.690.120.080.040.07

STATE

Sheet1WorkingIn schoolBothOtherNeithertotalSTATE69%12%8%4%7%100%

Pie Chart20.180.250.320.150.1

Sheet1Higher EducationCompetitive EmploymentPostsecondary Education or TrainingOther EmploymentNot EngagedSTATE18%25%32%15%10%100%

Chart

Sheet1Higher EducationCompetitively EmployedPostsecondary Education or TrainingOther EmploymentNeitherState (N = 1925)8%45%8%6%33%100%District (N = 475)15%35%18%19%13%100%

Chart10.460.30.380.20.040.220.060.150.060.129

Males (n = )Females (n = )

Sheet1Females (n = )Males (n = )Higher Education30%46%Competitive Employment20%38%Postsecondary Education & Training22%4%Other Employment15%6%Not Engaged13%6%100%100%To resize chart data range, drag lower right corner of range.