joint meeting of the commissioner’s and ayp task force october 14, 2010 nh doe 1joint task force...

Joint Meeting of the Commissioner’s and AYP Task Force

October 14, 2010NH DOE

1Joint Task Force Meeting: October 14, 2010

Overview of Key Policy Decisions• Subgroups—SWD, Low SES, “other”, & Whole School • Minimum n—5 • ELL performance

– Reading—progress towards English language proficiency as determined by ACCESS scores (AMAO 1)

– Math—use NECAP scores• Participation rate—rated as “met” or “not met” for every

subgroup (above) and every test. “Met” = 95% • K, K-1,K-2 Schools—Must participate in Level 2• High school indicators• Content areas for inclusion in the performance system—

reading, math, writing, science• Proposed cutscores for growth, achievement, and total

system


Participation• The distributions we looked at two weeks ago indicate

that essentially all schools/subgroups meet the 95% threshold

• However, both groups felt strongly that it still be included as both a signal and reward

• Therefore, we suggest awarding each school points for each subgroup that meets the participation target and not giving any points for subgroups not meeting the target.

• Proposal: • For each subgroup meeting 95% criterion = 1 point• For each subgroup not meeting 95%=0 points

Joint Task Force Meeting: October 14, 2010 3

Review of Subgroup Performance

• Switch to PDF slides/ bubble charts


Individual Targets• As we discussed in May, individual targets

should (must) be created, evaluated, and reported– The group decided to establish individual student

targets for students currently below proficient, to reach proficient in 3 years or less or by 8th grade (whichever is first), while proficient/advanced students stay above proficient

– The target is based on a defined and meaningful criterion (proficient) and can be used in the aggregate to establish school and subgroup targets


Aggregate Criterion Targets• Similar to aggregating the observed student

growth percentiles, we can aggregate the targets for all of the students in the school/subgroup and find the median– We can then compare the median of all of the

observed growth percentiles with the median of the targets


Norm-referenced growth still counts

• Schools with a lot of high achieving students will have relatively low aggregate targets so that low observed median growth percentiles could still allow schools to meet targets

• Colorado required schools, in order to be classified in one of the higher rubric categories, to still have a relatively modest median growth percentile


A rubric-based approach• As seen on the following slide, a rubric is used to

“score” growth• We would also establish rubrics for the other

indicators, such as status, attendance, graduation, etc.– We would also do these rubric ratings for subgroups

• We could then aggregate these rubric scores into the major classifications of inclusion, status, “gaps”, and “readiness”

• We could, but not sure if we would want to, aggregate across all rubric scores into a single composite– Or we could make adequacy decisions without creating a

single composite


Growth Rubric with Cut Scores for Median SGPs (based on CO, but slightly different)

4 (rubric score)

3

2

1

Yes No

55-69

Did median SGP exceed target SGP?

45-55

56-9970-99

40-54

30-44

1-391-29


Checking the values

• But, how do we know the growth percentiles demarking each rubric category are the right ones?

• Well, there is no “right”, but it needs to make sense and appear fair….


Percentages of schools in each rubric category: Math


Whole School Low SES SWD All Other

Rubric Score

Did Not Meet

Met Target

Did Not Meet

Met Target

Did Not Meet

Met Target

Did Not Meet

Met Target

4 2.94 32.81 0.00 34.16 0.79 51.28 0.00 39.76

3 0.00 39.38 2.78 33.66 12.70 35.90 0.00 38.28

2 32.35 26.56 33.33 29.70 38.89 12.82 0.00 21.36

1 64.71 1.25 63.89 2.48 47.62 0.00 100.00 0.59

Percentages of schools in each rubric category: Reading


Whole School Low SES SWD All Other

Rubric Score

Did Not Meet

Met Target

Did Not Meet

Met Target

Did Not Meet

Met Target

Did Not Meet

Met Target

4 0.00 27.17 0.00 27.49 0.00 50.00 0.00 39.48

3 0.00 45.09 0.00 28.29 6.36 36.11 0.00 37.46

2 9.09 26.88 5.00 38.65 39.09 13.89 0.00 21.04

1 90.91 0.87 95.00 5.58 54.55 0.00 100.00 2.02

How about Science & Writing?

• We’ll look at some data and proposed cutscores for science and writing for elementary-middle school


Elementary-Middle School Science


Percent Proficient & Advanced in Science

Whole School ELL SWD Low SES All Other

N Valid 369 146 355 331 348

Missing 16 239 30 54 37

Mean 40.49 18.53 20.45 34.15 51.11

Median 41.00 0.00 14.30 33.30 52.55

Std. Deviation 20.93 32.90 21.68 24.76 21.38

Minimum 0.00 0.00 0.00 0.00 0.00

Maximum 100.00 100.00 100.00 100.00 100.00

Percentiles 25 24.45 0.00 2.50 14.30 33.30

50 41.00 0.00 14.30 33.30 52.55

75 55.80 29.78 33.30 50.00 67.45

Elementary-Middle School Science• What’s the lowest percent

proficient/advanced we can tolerate for “level 3”? 40%, 50%?

• Suggest as a starting point:– Level 1- 0-20% – Level 2 - 21-40%– Level 3 - 41-65%– Level 4 - 66-100%


Elementary-Middle School Science Rubric Score Distributions


Rubric Score

Whole School ELL Low SES SWD All Other

4 12.5 4.6 10.0 4.1 27.1

3 39.2 3.3 22.2 9.8 37.4

2 29.8 3.0 27.4 24.7 22.7

1 18.5 89.2 40.4 61.4 12.7

Elementary-Middle School Writing


Percent Proficient & Advanced in Writing

Whole School ELL SWD Low SES All Other

N Valid 350 121 337 283 305

Missing 35 264 48 102 80

Mean 50.68 26.94 16.89 53.02 66.46

Median 53.45 0.00 11.40 50.00 68.40

Std. Deviation 21.42 37.32 19.33 24.51 16.42

Minimum 0.00 0.00 0.00 0.00 14.30

Maximum 100.00 100.00 100.00 100.00 100.00

Percentile 25 39.03 0.00 0.00 39.20 57.00

50 53.45 0.00 11.40 50.00 68.40

75 65.53 50.00 25.00 66.70 77.80

Elementary-Middle School Writing• Suggest as a starting point:

– Level 1- 0-30% – Level 2 - 31-50%– Level 3 - 51-65%– Level 4 - 66-100%


Elementary-Middle School Writing Rubric Score Distributions


Rubric Score

Whole School ELL Low SES SWD All Other

4 14.4 5.7 16.9 2.3 39.2

3 41.2 1.4 21.8 1.7 33.4

2 28.2 4.6 29.8 12.4 11.8

1 16.2 88.3 31.5 83.6 15.6

Let’s look at some actual school results to evaluate the rubric cutscores

(see Word doc—an excerpt below)


School NameBakersville School

Bicentennial Elementary School

Campton Elementary School

Dr. Norman W. Crisp School

Math_Whole Sch MEDIAN_SGP 59.5 61.0 61.5 28.0Math_Whole Sch Target MED_SGP 51.0 16.0 37.0 41.5Math_”Met Target?”WS 1 1 1 0Math_Growth_Level_WS 4 4 4 1Math Whole School %prof& adv 53.5 90.3 63.0 51.2Read_Whole Sch MEDIAN_SGP 50.0 58.0 55.0 26.0Read_Whole Sch Target MED_SGP 43.0 16.0 23.0 32.0Read_”Met Target?”WS 1 1 1 0Read_Growth_Level_WS 3 4 3 1Read Whole Sch %prof& adv 58.3 90.8 77.5 54.8

Points• Reading “growth”

– Whole school = 4– Econ Disadvantaged = 4– SWD = 4– All Others = 4

• Math “growth”– Whole school = 4– Econ Disadvantaged = 4– SWD = 4– All Others = 4

• Science “status”– Whole school = 4– Econ Disadvantaged = 4– SWD = 4– All Others = 4

• Writing “status”– Whole school = 4– Econ Disadvantaged = 4– SWD = 4– All Others = 4

• Participation rate• Attendance


Weights• Reading and math indicators include both growth

and status– Should they then count twice as much as science &

writing?• Reading and math are tested 3x more than

science and writing– Should they count 3x more than science & writing?

• So should reading and math count 6x more than science and writing?– Should growth count more than status?

• What about attendance and participation?


joint meeting of the commissioner’s and ayp task force october 14, 2010 nh doe 1joint task force...

Documents

joint meeting

pointsjoint task force

subgroup meeting

ayp task forceoctober

observed growth percentiles

low aggregate targets

individual student targets

school points