AnchorageGiving Grades in a Formative

Assessment System

March 19,2010

Feedback results in achievement gains Positive vs. negative Information vs. non-content (praise, punishment) Type of feedback – lowest gains – right vs. wrong; highest

gains based on criteria (scale) The most common form of feedback is assessment Increased frequency of assessment results in increased

achievement gain

Summary: to improve achievement1. Provide positive feedback2. Provide feedback based on what the students knows or is able

to do3. Provide feedback based on a set of criteria or scales4. Provide lots of feedback! Assess frequently

Research on Formative Assessment

The reliability of a typical classroom assessment varies from .45 to .75

SD = 12 points

Reliability Score Minimum Score Maximum Score

.45 70 52 88

.55 70 54 86

.65 70 56 84

.75 70 58 82

You can never rely on a single assessment!

Teachers use scales to assess students on a number of Measurement Topics per unit

Data is collected from a variety of sources and recorded for each topic

Student Progress is tracked over time Students can track their own

progress

Gathering Formative Data

Topic: Describes the causes of the Civil WarPretest Foldable

summaryQuiz – short constructed response

Debate Probing discussion

Summative

Debbie

1.0 3.0 3.0 2.0 3.0

Mark 1.0 2.0 3.0 3.0Lindsay

1.5 2.0 2.0 3.0 2.5

Chayla 2.0 2.5 3.5 4.0

Traditional grading systems use a measure of central tendency, typically the mean, to determine a score.

The Learning Trend or Power Law use a student’s scores on various assessments over time to determine a student’s current level of understanding of a given topic

Averaging vs. the Learning Trend or Power Law of Student Learning

Averaging assumes no learning has taken place between or among the assessments

Assumes the content on respective assessments is completely different

Tends to include lots of data points that don’t measure student knowledge

Averaging tends to hide what the student does really well, and what the student still needs to work on

Why not average formative data?

The power law can be applied to come up with a more accurate estimation of a student’s true score

Power law estimations are typically far closer to a student’s observed score than averaged scores

The power law is a mathematical function that takes into account the number of assessments, the score on each assessment and the time between assessments to calculate an estimated ‘true’ score y=atb where y is a score on a particular assessment, t

is the time at which the assessment was administered and a and b are constants

The Power law

Power Law

3

2.5

2

1.5

1

.5

0Pre-Test Score 2 Score 3 Score 4 Score 5 Score 6 Post-Test

Average Score = 1.64

Learning Trend = 2.21

.71

1.24

1.551.78

1.942.08

2.21

Mode = 1.5

1 1 1.5 1.5 1.52 3ObservedScore

Imagine that a student has received the following scores on a measurement topic: 1.0, 1.0, 1.5, 1.5, 2.0, 1.5, 3.0

What summative score would they receive? In your professional judgment –

Do they deserve a 3.0? Why or why not? Do they deserve a 2.0? Why or why not? Considerations: Look at the trend in the data – is it

going up? Have they demonstrated consistent success at any level? Do you believe they can accomplish a specific level? Do you need more data?

What would it be averaged?

The Learning Trend

0

0.5

1

1.5

2

2.5

3

3.5

4

Student AStudent BStudent C

In your professional judgment – What summative score has each student learned?

Considerations: Look at the trend in the data – is it going up? Down?

Have they demonstrated consistent success at any level?

Do you believe they can accomplish a specific level?

Do you need more data?

The rubrics are designed so that a teacher can use fewer data points based on a set criteria to estimate the “true score”

The short answer is that you need as many as it takes to get a good picture - using professional judgment, assessment data, and your knowledge of the student - of what a student knows at any given period of time

4- 5 are ideal. The less certain you are about a student’s “true score”, the more data you need

How many data points do I need?

Zeros given on an assignment or assessment because the student did not do it skew the calculation of a true score

If you are trying to measure what a student knows and is able to do, use other means to measure and report work completion, behavior etc.

A separate set of rubrics, or a separate grade can be used

Implications

DETERMINING AN OVERALL GRADE

Report summative scores for each of the topics studied in the reporting period

Reporting where a student started and finished gives one more piece of information – the “growth”

Summative scores from each topic can be combined to give an overall grade for the course/subject

Reporting student progress

Up until now, averaging (the mean) has been a bad word!

However, when scores are aggregated across topics or learning goals – to come up with a “grade” – the mean is a viable option

Anything done to summarize the topic specific data across topics is arbitrary, and there are no right or wrong answers!

The mean is a “compensatory” approach – one good score compensates for one bad score

Averaging Across Topics

Topic Formative Score Summative Grade

1 2.0Average = 2.5

Teacher uses their best judgment to

translate the computed average into the half point

scale score that is the most probable

representation of the students true score

2 2.5

3 3.0

4 1.5

5 3.5

Translating Formative scores into final grades

Unweighted average – all learning goals or topics are treated equally

Weighted Average – some topics or learning goals receive more “weight” than others When weights are used, multiply the

scale score times the weight… Add all of them together… And divide by the total number of

weights

Unweighted vs. Weighted

Once the mean is determined the numeric score can be translated into: A letter grade

Mean Scale Score Across Topics

Possible Letter grade

3.75-4.0 A+3.26-3.74 A3.00-3.25 A-2.84-2.99 B+2.67-2.83 B2.50-2.66 B-2.34-2.49 C+2.17-2.33 C2.00-2.16 C-1.76-1.99 D+1.26-1.75 D1.00-1.25 D-Below 1.00 F

Once the mean is determined the numeric score can be translated into: Words

Mean Score Score Descriptor

3.5-4.0 Advanced

2.5-3.0 Proficient

1.5-2.0 Basic/Partially Proficient

0.0-1.0 Below Basic/Unsatisfactory

Once the mean is determined the numeric score can be translated into: Percentages

Scale Score Percentage Score

4.0 1003.5 953.0 902.5 802.0 701.5 651.0 60Below 1.0 50

The conjunctive approach employs goal or target scores determined by the teacher for each topic

Minimum scores are established for each grade on each topic

This approach is useful when the teacher has not addressed all levels of the topic.

Conjunctive

A= Goal 1: 2.0 or above

Goal 2: 2.0 or above

Goal 3: 3.0 or above

B= Goal 1: 1.5 or above

Goal 2: 1.5 or above

Goal 3: 2.5 or above

Conjunctive

  Name: Aida Haystead Subject Areas:  

  Address: 123 Some Street Language Arts B  

  City: Mathematics B  

  Grade Level: 5 Science D  

  Homeroom: Ms. Becker Social Studies A  

  Art B  

                               

  Language Arts  

  :  

  Word Recognition and Vocabulary 3.5                

   

  for Main Idea 2.5            

   

  Literary Analysis 3.0              

   

  Writing:  

  Language Conventions 4.0                  

   

  Organization and Focus 2.0          

   

  Research and Technology 1.5        

   

  Evaluation and Revision 2.5            

   

  Writing Applications 1.0      

   

  Listening and Speaking:  

  Comprehension 3.0              

   

  Organization and Delivery 3.5                

   

 

Analysis and Evaluation of Oral Media 2.0          

   

  Speaking Applications 2.0          

   

  Life Skills:  

  Participation 4.0                  

   

  Work Completion 3.0              

   

  Behavior 4.0                  

   

  Working in Groups 2.5            

                               

                               

Report Card with Overall Grades