a ccountability 2013

accountability 2013

Interpreting Your 2013 Accountability ReportIt’s Like Learning To Read All Over Again

Ervin Knezekervin@lead4ward.com

John Fessendenjohn@lead4ward.com

TEPSA Webinar | September 5, 2013

Welcome!

goals2013 accountability

Construction

Clarity Of calculation . . . and application

Connection

Of indexes What measures/data are included?

To instructional processes and systems

TEPSA Webinars (May 10 and 17)

Today

Ongoing

the vibe

YourQuestion

s

5

accountability

2013the BIG picture

p. 21

p. 22

Accountability | Overall Design

• Four Performance Indexes

• Designed to meet 4 statutory goals1. Improving student achievement at all levels in core subjects

2. Ensuring progress of all students toward Advanced Academic Performance (STAAR Level III)

3. Closing Performance Gaps among groups in Advanced Academic Performance (STAAR Level III)

4. Closing gaps among groups in % of students graduating under Recommended or Advanced high school programs

Index 1

Index 2

Index 3

Index 4

p. 22

2013 AccountabilityAchievement

All Students Only Single percentage

combined over ALL Subject Areas

≥ Level II (Phase 1) STAAR

≥ Met Std Grade 11 TAKS

Progress• 10 student groups

– All Students– 7 Race/Ethnicity– SpEd– ELL

• Minimum size– 10 for All Students– 25 for other groups

• By Subject Area– Rdg & Math in 2013*

• Based on PROGRESS– 1 point for % Met– 2 points for %

Exceeded

Closing Gaps• 3 student groups

– Eco Dis and – 2 lowest

Race/Ethnicity groups from prior year

• By Subject Area (all subjects)

• 1 point for % Level II (Phase 1)

• Level III to be added (2014)

Postsecondary• 4-year or 5-year

Graduation Rate (or Annual Dropout Rate)– 10 student groups

• % RHSP/DAP– 8 student groups

(not SpEd or ELL)

• STAAR Postsecondary Readiness – Level II Final (2014)

Index 1Student

Achievement

Index 2Student Progress

Index 3Closing

Performance Gaps

Index 4Postsecondary

Readiness

Target = 50 Target = ≈ 5th %ile

Target = 55Target = 75

Target =30 29 17 21

ES MS HS Dist

p. 21

accountability

2013Interpreting our ReportsIt’s like learning to read all over again!

2013 accountability . . .

Part Section # Pages1 Accountability Summary 12 Index Calculations and Data Tables 123 Distinction Designations (Campuses Only) 4 (8/29/13)

4 System Safeguards 3 (8/29/13)

Total Pages 20 (8/29/13)

. . . a report in 4 parts

p. 1p. 2-13 p. 14-

17p. 18-

20

Index 1

Index 1Student

Achievement

Index 1Student

Achievement

p. 2

p. 3-4

Index 1: Student AchievementIndex 1Student

Achievement

p. 22

10 Additional Student Groups reported 7 Race/

Ethnicity SpEd Eco Dis ELL

Used for System Safeguards

p. 2

• Calculate Overall Pass Rate For All Students1. Add # Passed (≥ Phase-in 1 Level II) each subject area (R + M + W + S + SS)

• 277 + 284 + 65 + 60 + 34 = 720

2. Add # Tests taken each subject area (R + M + W + S + SS)• 485 + 473 + 174 + 127 + 127 = 1,386

3. Divide # Passed by # Tests to determine % Passed• 720 ÷ 1,386 = 51.9%, which ROUNDS to 52%

4. Assign 1 point for each % Passed = Index 1 Score• 52% = 52 POINTS

• Overall pass rate IS NOT average pass rate– Average pass rate would equal: (57% + 60% + 37% + 47% + 27%) ÷ 5 = 47%

Index 1 Score57% 60% 37% 47% 27%

p. 3-4

p. 2

• Where did these numbers come from?• Pages 3-4!

% at Final Level II (13%)% at Level III (3%)

[Instructional Planning]

Disaggregated Data There are a LOT of numbers on pages 3 and 4!

“I thought Index 1 ONLY evaluated the All Students group!”

True!

BUT . . . the data are included for System Safeguards

p. 3

• System Safeguard– Low Performance in a single cell for a particular student group does NOT

automatically cause the district/campus to miss Index 1– However, ANY student group meeting MSC with a pass rate below 50% MUST

be addressed in the district/campus improvement plan• MSC = 0 for All Students | 25 for all other Student Groups

• So how do we “find” System Safeguards?1. In each subject area, determine which student groups meet MSC

• MSC = 0 for ALL Students | 25 for all other student groups

2. Then determine if % Passing < 50% (at Phase-in Level II or above) for any group meeting MSC

3. If % Passing < 50% AND Student Group meets MSC, then Student Group’s Performance MUST be addressed in improvement plan

p. 3

Others?

# GroupsMissing Safeguard

8

# GroupsMeeting Safeguard

9

p. 3

# GroupsMeeting MSC

Reading, Math, Writing 17

p. 4

Science?Social Studies?

# GroupsMeeting MSC

Reading, Math, Writing 17

# GroupsMissing Safeguard

8

# GroupsMeeting Safeguard

9Science, Social Studies 8 8 0

TOTAL 25 16 9

p. 3

v

p. 4

System Safeguards• 25 student groups met

MSC (across 5 subject areas)

• ONLY 9 Met Safeguard target of 50%

• 16 Missed Safeguard target (including EVERY group in Writing, Science, Social Studies)

• BUT campus MET Index 1 Target!

All 16 Student Groups that Missed Safeguard target of 50% must be

addressed in CIP

System Safeguards• Are summarized with Y/N indicators in

Part 4 of the Campus’ 2013 Accountability Report (p. 18) p.

18

p. 3

p. 19

System Safeguards• Performance Rate data from p. 3-4

are duplicated on p. 19 (slightly reorganized)

p. 19

System Safeguards• Performance Rate data from p. 3-4

are duplicated on p. 19 (slightly reorganized)

• Participation Rate data are also provided

Index 1is summarized in

the campus’ 2013 Accountability

Summary(p. 1)

2013

Acc

ount

abilit

y Su

mm

ary

p. 1

27

Your Question

s

Index 1Student Groups?

MSC?Safeguards?

Index 2

Index 2Student Progress

Index 2Student Progress

p. 5

p. 6-7

Index 2: Student Progress Index 2Student Progress

p. 22

Index 2: Student Progress

Based on actual student progress (2012 to 2013)

Index 2Student Progress

STAAR Level IIIfocuses on progress toward . . .

Each student is assigned a progress category based on his/her change in scale score in relation to progress expectations

–Did Not Meet

–Met

– Exceeded

Index 2: Student ProgressIndex 2Student Progress

Index 2: Student Progress

• Subject areas evaluated for 2013– Reading• Grade 4 through English II

– Math • Grade 4 through Algebra I

– Writing • English I to English II

Index 2Student Progress

Performance Standard Met Progress Exceeded Progress

Index 2 is based on a fraction where Numerator = # of points based on % of students who Met or Exceeded Progress

1 point for % Met 2 points for % Exceeded

Denominator = Maximum Points Possible based on number of student groups meeting minimum size criteria (MSC) Maximum Points Possible = 200 points for each student group meeting MSC

Index 2: Student ProgressIndex 2Student Progress

What is “progress”?

Index 2: Student ProgressIndex 2Student Progress

• With exceptions for high-performing (“almost perfect”) and low-performing (“lower than guessing”) students, the general rule is

MetProgress = Level II Final

(Higher Grade) – Level II Final(Lower Grade)

ExceededProgress = Level III

(Higher Grade) – Level II Final(Lower Grade)

p. 23

Subjectand

GradeLevel/Course

ItemsPoints

Level II Standard

(FinalRecom-

mended)

Level III Standard

Grade 4 (Eng. & Span.) 48 1599 1677

Grade 5 (Eng. & Span.) 50 1627 1710

Met Progress?Level II 5th Grade – Level II 4th Grade1627 – 1599 = 28

p. 23

Exceeded Progress?

Level III 5th Grade – Level II 4th Grade1710 – 1599 = 111

p. 23

Subjectand

GradeLevel/Course

ItemsPoints

Level II Standard

(FinalRecom-

mended)

Level III Standard

Grade 4 (Eng. & Span.) 48 1599 1677

Grade 5 (Eng. & Span.) 50 1627 1710

p. 23

Index 2Simple

Example

All Students

# Tests 500

# Met 300

# Exceeded 100

% Met 60%

% Exceeded 20%

1 point for % Met OR Exceeded 80

1 point for % Exceeded 20

Weighted Progress Points 100

Maximum Possible Points 200

Index 2 Score(Weighted Progress Points ÷ Maximum Possible Points)

50

1 point for % Met 60

2 points for % Exceeded 40

Weighted Progress Points 100

Index 2 Data Reports

Build “back to front”

Start with page 7

Then page 6

Then page 5

p. 7

For each student group meeting MSC in each subject, determine: % Met or Exceeded Progress % Exceeded Progress

From Index 1 (p. 2)

REMINDERS (Why Index 2 numbers will not match Index 1): Only students with 2 years of “matched” data are included Progress starts at 4th grade (no 3rd graders receive a Progress Measure) ELL students are “mostly excluded” from Index 2 in 2013 STAAR Modified and STAAR Alternate are not included in Index 2 in 2013 Writing is ONLY evaluated for English I to English II

MSC = 10 tests for All Students | 25 tests for other Student Groups

p. 6

Data from page 7 are reproduced on page 6 In each subject area, for each Student Group meeting MSC

Assign 1 point for each % Met OR Exceeded Assign another 1 point for each % Exceeded

Total = Weighted Progress Rate for each Student Group meeting MSC Effectively 1 point for each % Met and 2 points for each % Exceeded

Add Weighted Progress Rate in each subject for each student group meeting MSC Maximum Points = # Student Groups meeting MSC x 200

+ +

+ +

p. 5

Data from page 6 are summarized on page 5 (remember: read back to front!) Weighted Progress Rate for each Student Group meeting MSC in each subject

area is reported and summed Reading: 75 + 64 + 77 = 216 points Math: 70 + 67 + 72 = 209 points

Sum the Total Points in Reading and the Total Points in Math 216 + 209 = 425

Sum the Maximum Points in Reading and the Maximum Points in Math 600 + 600 = 1200

425 ÷ 1200 = 35.4, which rounds to 35 . . . SO Index 2 Score = 35 points

Index 2 Score

+ ++ + + +

Index 2is summarized in

the campus’ 2013 Accountability

Summary(p. 1)

2013

Acc

ount

abilit

y Su

mm

ary

p. 1

47

Your Question

s

Index 2Progress Categories

PointsData

Index 3

Index 3Closing

PerformanceGaps

p. 8

p. 9-11

Index 3: Closing Performance Gaps

Index 3Closing

Performance Gaps

Impacts every student group of which the ELL student is a member

p. 24

Index 3: Closing Performance Gaps

Index 3 emphasizes Level III (Advanced Academic Performance) for “Performance Gap Groups”

BUT . . . STAAR Level III cannot be included in accountability until 2014, so for 2013 . . .

Measure = % of students in Performance Gap Groups “passing” state assessment Evaluated in EACH subject area (Reading, Math, Writing, Science, Social

Studies) Then cumulated to a single score

Index 3Closing

Performance Gaps

Index 3: Closing Performance Gaps

Index 3: Closing Performance Gaps?

Index 3Closing

Performance Gaps

All

Eco Dis

Eco Dis

% Achieving Level III

Index 3: Closing Performance Gaps

Student Groups Identified for Evaluation in 2013

Economically Disadvantaged (no MSC)

2 lowest performing Race/Ethnicity groups meeting MSC (≥ 25 tests) across all subjects from Index 1 in 2012 The 2 Race/Ethnicity Groups are determined based on combined

performance across all subjects (Index 1) So the Race/Ethnicity Groups analyzed in Index 3 are the same in all subjects For Index 3 evaluation, MSC for each Race/Ethnicity Group in each subject

area = 25 tests taken by students in that Race/Ethnicity Group in that subject area

If a Race/Ethnicity Group does not meet MSC in a subject area, then it is NOT evaluated in that subject area

Index 3Closing

Performance Gaps

p. 8

• Summary Data – page 8• How were the 2 Race/Ethnicity groups

selected?• From 2012 performance data

p. 10

• Summary Chart of 2012 Performance for the 7 Race/Ethnicity Groups• Which Race/Ethnicity groups met MSC in 2012 (≥ 25 tests taken by students in that

Student Group across all subject areas)?– African American, Hispanic, White

• If ≥ 3 Race/Ethnicity groups meet MSC, then 2 lowest performing are used for Index 3– African American (44%) and Hispanic (51%)

• If only 2 Race/Ethnicity groups meet MSC, then only 1 Race/Ethnicity group is used for Index 3 (the lower performing group)

• If only 1 (or 0) Race/Ethnicity groups meet MSC, then Index 3 is based solely on Economically Disadvantaged

Index 3 Data Reports

Build “back to front”

Start with pages 10 and 11

Then pages 8 and 9

Then top of page 8

p. 10

For each subject and each student group, build from the bottom row up Total Tests # at Level II Phase 1 or above % at Level II Phase 1 or above

p. 11

Repeat for Science and Social Studies Total Tests # at Level II Phase 1 or above % at Level II Phase 1 or above

Notice that there are fewer than 25 tests taken by African American students in Science and Social Studies

A Student Group is NOT EVALUATED in a subject area UNLESS it meets MSC IN THE SUBJECT AREA (≥ 25 tests taken by students in that student group in that subject area)

p. 8

These data come from pages 10-11

p. 10

p. 8

In each subject area, for each Student Group meeting MSC: Assign 1 point for each % at Phase-in 1 Level II or above Then total Weighted Points Maximum Points = 100 x Number of Groups meeting MSC

+ +

+ +

p. 9

Determine Weighted Performance Rate for Writing, Science, Social Studies <25 tests taken by Af. Am. students in Sci and SS Af. Am. student group is NOT evaluated for Index 3 (Max. Pts. = 200)

+ +

+ +

+ +

p. 8

• Index 3 Summarized Data – top of page 8• Total Weighted Performance for each Student Group meeting MSC in each

subject area (from bottom of page 8 and page 9)• Total Points per subject and Maximum Points per subject• Summed across all subjects• Total points (750) ÷ Maximum Points (1300) = 57.6, which rounds to 58• Index 3 Score = 58 points

Index 3 Score

p. 2

What will be the Race/Ethnicity groups in Index 3 for this campus in 2014? Look at Index 1 data table for 2013 3 Race/Ethnicity groups meet MSC (≥ 25 tests taken by students in that group) 2 lowest performing are: African American and Hispanic In 2014, Index 3 will be weighted:

1 Point for each % at Level II 2 points for each % at Level III

Index 3is summarized in

the campus’ 2013 Accountability

Summary(p. 1)

2013

Acc

ount

abilit

y Su

mm

ary

p. 1

65

Your Question

s

Index 3Closing Gaps

Future – % at Level III

Index 4

Index 4Postsecondary

ReadinessIndex 4

Postsecondary Readiness

p. 12

p. 13

Index 4Postsecondary

Readiness

Index 4: Postsecondary Readiness

For 2013 Accountability, Index 4 is based on Graduation Rate (Cohort Measure)

4-year rate for Class of 2012 Cohort (entered 9th grade in 2008-09), OR 5-year rate for Class of 2011 Cohort (entered 9th grade in 2007-08)

RHSP/DAP Diploma Rate (Annual Measure) %age of students who were awarded diplomas in 2011-12 school year

REGARDLESS of when they entered 9th grade

For high schools without a Graduation Rate, the Grade 9-12 Annual Dropout Rate is used

Coming Soon to Index 4 . . .

Postsecondary Readiness in 2014 . . .

Scheduled to include for all campuses % of students passing one or more STAAR

assessments at Level II – Final

Index 4is summarized in

the campus’ 2013 Accountability

Summary(p. 1)

2013

Acc

ount

abilit

y Su

mm

ary

p. 1

71

Your Question

s

Index 4Graduation RateRHSP/DAP Rate

2013 AccountabilityAchievement

52points

Progress

35points

Closing Gaps

58points

Postsecondary

N/A

Index 1Student

Achievement

Index 2Student Progress

Index 3Closing

Performance Gaps

Index 4Postsecondary

Readiness

Target = 50

Target = 55Target = 75

Our Sample Campus

Target =30 29 17 21

ES MS HS DistMet Standard

Campus Distinction

Designations

Distinction Designations Campuses that earn Met Standard are eligible

for 3 campus distinction designations:

p. 14-17

1. Top 25% Student Progress

2. Academic Achievement Distinction Designation (AADD) in Reading/ELA

3. AADD in Math

Campus Distinction Designation

p. 16

Campus must rank in top quartile (top 10 scores) of Campus Comparison Group

Top 25% Student Progress

Determining Academic Achievement Distinction Designations (AADDs)

Academic Achievement Distinction Designations (AADDs)

AADDs recognize outstanding campus achievement in ELA and Math

Each campus is compared to its campus comparison group on a number of indicators in ELA and in Math that are specific to its campus type

To earn AADD in a subject area, a campus must be in the top quartile of its campus comparison group on the following percentages of the indicators applicable to the campus group type in that subject area: Elementary and Middle Schools: ≥ 50% of the indicators High Schools and K 12 campuses: ≥ 33% of the indicators‐

AADDsp. 24

Reading/ELA Indicators1. Attendance Rate2. Exceeded Progress

Reading3. Grade 7 Writing (Level

III)4. Grade 8 Reading (Level

III)

Math Indicators1. Attendance Rate2. Exceeded Progress Math3. Grade 5 Math (Level III)4. Algebra I by Grade 8

Participation5. Algebra I by Grade 8

Performance (Level III)

p. 14

p. 15

Distinction Designations

But how far off are we?

p. 14

Score NeededTo Get

IntoQ1

97.1%17%

2%16%

p. 15

Score NeededTo Get

IntoQ1

97.1%11%

13%50%

Distinction Designations

How do we know this?

p. 19

NEW!Released 8/29/2013

My Score

Where I WANT To Be

Distinction Designationsare summarized in

the campus’ 2013 Accountability

Summary(p. 1)

2013

Acc

ount

abilit

y Su

mm

ary

p. 1

AADDs(for Elementary

Campuses)p. 37

Reading/ELA Indicators1. Attendance Rate2. Exceeded Progress

Reading3. Grade 3 Reading

(Level III)4. Grade 4 Writing

(Level III)

Math Indicators1. Attendance Rate2. Exceeded Progress

Math3. Grade 5 Math (Level

III)

Connecting to

Instruction

2013 AccountabilityAchievement

How did we do?

Progress

Are our interventions

working?

Closing Gaps

Are we meeting the

needs of under-

performing groups?

Postsecondary

Are students ready for college or

career?

Index 1Student

Achievement

Index 2Student Progress

Index 3Closing

Performance Gaps

Index 4Postsecondary

Readiness

Are we meeting the needs of under-performing groups?

Are our interventions working?

Are students ready for college or career?

How did we do?

Appealing Ratings• Deadline: September 9, 2013

• Appeals are only accepted for a change in rating

• Appeals are not accepted for:– System Safeguard Measures– Distinction Designations

• Appeals are not an opportunity to correct data submissions

accountability connect

Year-long support for district leaders Webinars, tools,

resources, templates Telephone/email support

District and campus-level support through district contact Interpretation of data Appeals Planning for 2014

Thank You!John Fessenden

john@lead4ward.com