Assessment: It’s what’s for dinner!

Laura Henriques

California State University, Long Beach

www.csulb.edu/~lhenriqu/CSTA2004.ppt

Assessment

Some AppetizersTrue or False?

1. When writing multiple choice items you should always use “none of the above”.

2. Multiple choice items allow you to ask more questions in a given amount of time.

3. Multiple choice tests are less subjective than essay tests.

Multiple Choice4. When writing multiple choice items the stem, the part

of the questions which provides background information and the prompt,A) should include more information than students

need.B) should contain multiple parts, enabling you to ask

more than one question at a time, thereby resulting in more data about students’ understanding of the content.

C) should minimize reading demands.D) should include negatives so that you can catch

students who don’t read carefully.

Short Answer

5. _____________ assessment is the type of assessment given at the end of a unit.

6. You should have more ____________ items on a true/false test than ______ items.

Multiple Choice

7. When grading student work it is best to . . .A) grade the best student’s paper first.B) grade the worst student’s paper first.C) grade the same item for every student at the

same time before moving on to the next question.

D) all of the above.

Overview of Today’s Talk Understand what makes for “good” test items

& how to write them (This is important to understand this because the people who make high stakes tests follow these rules.)

Use assessment results to inform instruction Understand the relationship between your

objectives/goals/standards, instruction, and assessment

How were you assessed? How do you assess your

students? Multiple choice True/false Matching Fill in the blank Short/long essay

questions Papers

Labs Journals Projects Reports Presentations Posters Observations (check

lists)

For what purpose do you assess?

(or were you assessed?) Grades Rank order To move on to the next level

To inform instruction & better help students learn!

As we begin to think about assessment, consider the following two forms: Formative Assessment- assessments

occurring throughout the unit - sometimes referred to as Checking for Understanding

Summative Assessment- the assessment done at the end of a unit or year

Multiple Choice Items

Multiple choice items have a stem or proposition followed by several potential answers. Students must select the most appropriate/best/ideal response.

We call the answer choices foils or distracters.

Writing Multiple Choice Test Items

Formulate useful propositions Translate the propositions into questions. Develop a concise answer. Develop plausible distracters. Introduce novelty. When all else fails, consider a true/false

alternative.

What does a correctly answered multiple choice question tell you about what students know? Not necessarily a whole lot!

Types of Multiple Choice(Wiggins & McTighe)

• TRADITIONAL- used for testing content standards worth being familiar with and/or important to know and do.• Stem with 4 or 5 choices

• ENHANCED- also used for testing content standards worth being familiar with and/or important to know and do. • Stem supported by graph, chart, table or diagram with 4 or

5 choices

Enhanced Multiple ChoiceTaken from CA Dept. of Education (1994) A Sample of Science Assessment (p.100)

Leon and Lupe helped to plant a garden for their fifth grade science project on the grounds of their school.

Leon decided to sprout his bean seeds before planting them in the school garden. The picture show several bean seeds starting to grow. The direction of the growth of the root is a response of the root to . . .

a) light b) heat c) oxygen d) gravity

Types of Multiple Choice(Wiggins & McTighe)

• TRADITIONAL- used for testing content standards worth being familiar with and/or important to know and do.• Stem with 4 or 5 choices

• ENHANCED- also used for testing content standards worth being familiar with and/or important to know and do. • Stem supported by graph, chart, table or diagram with 4 or

5 choices• JUSTIFIED- used for a deeper understanding of

content knowledge.• A traditional or enhanced question that requires a written

justification of the student’s choice.

Justified Multiple ChoiceTaken from CA Dept. of Education (1994) A Sample of Science Assessment (p.104)

Directions: Please circle the best answer. Explain why you chose your answer.

Judy wanted to find out whether certain materials were good conductors of electricity. She replaced one of the wires in her circuit with a piece of string.

What will happen to the bulb now?A. It will blink.B. It will not light.C. It will shine more brightly.D. It will shine less brightly.

Developing the stem: single idea or question use of negatives (underlined or capitalized) minimize reading (if you must have lots of reading

consider having several questions linked to the reading)

don't use lots of window dressing/excess baggage that is not needed to answer the question

Developing the options/foils/distracters: one correct answer is ideal, a "best“ answer is okay

exclusive 3-5 choices is the best -- 2 is not enough, >6 too many (takes too long to read/sort)

use of none of above is okay but it shouldn't be in each item. You don't want it to be seen as filler so be sure that it is the right answer some of the time.

all of the above should be avoided because it doesn’t distinguish between students who know it really is ALL of the above from students who know it’s 2 of the above

Distracters … continued use sensibly common misconceptions/misunderstandings make

attractive wrong answers avoid paired opposites (mutually exclusive answers) common phrases and textbook language are attractive consistent units and language; watch that grammar

'matches‘ be sure answers are approximately the same length be careful about convergence Use distracters that provide you with

meaningful information

Paired Opposites

In a matriarch which group rules?A) KingsB) MenC) PriestsD) Women

This item was a $8,000 question on the TV show Who Wants to Be a Millionaire?

Consistent Units And Language

A cart with a mass of 10 kg is initially at rest. It accelerates at 4 m/s2. How far does it travel if 5 seconds?A) 100 mB) 20 m/sC) 10 m/sD) 5 s

d = ½ a t2 + vo t

The only answer that has distance units

Meaningful Distracters A cart of mass 10 kg is traveling at a speed of

4 m/s. How much kinetic energy does it have?

A) 160 joules B) 80 joules C) 40 joules D) 20 joules

Forgot to multiply by 1/2

Forgot to square velocity

Used formula for momentum instead of KE

Correct answer KE = ½ m v2

Let’s Review…..

1. When writing multiple choice items you should always use “none of the above”.

4. When writing multiple choice items the stem, the part of the questions which provides background information and the prompt,

A) should include more information than students need.

B) should contain multiple parts, enabling you to ask more than one question at a time, thereby resulting in more data about students’ understanding of the content.

C) should minimize reading demands.D) should include negatives so that you can

catch students who don’t read carefully.

True / False Items

A true/false item is a sentence about which students must decide the veracity.

Often times students are instructed to correct (or make true) a false statement.

Developing True/False Items Write T-F items in pairs. Be sure the content is

parallel with one T and one F statement. Use specific determiners wisely and sparingly

* absolutes: never, only, always, every, all * qualifiers: generally, sometimes, usually, some, most, few, often

Internal comparisons help reduce ambiguity. Use more False than True. You can ask more T/F items in a given amount of

time than you can multiple choice (less reading involved).

Let’s Review…..

2. Multiple choice items allow you to ask more questions in a given amount of time.

Matching Items

Matching items consist of a group of questions, propositions, definitions or problems in one column and a set of answers in another column. Students must match the question with its appropriate answer.

Creating Matching Items Be sure to group the items around a common

theme or topic Have the longer text be in the left column with

shorter texted ‘answers’ on the right column (reduces the reading demands)

Be sure to have the answer column in some logical order (alphabetical, numerical, etc.)

Have more answers than questions Give instructions that answers can be used more

than once

Short Answer (Fill in the Blank)

Short answer items are statements, definitions or propositions which are missing a word or phrase. Students insert the missing word(s) to complete the statement.

Creating Short Answer Items Put the blank(s) towards the end of the

statement. Be sure that there’s only one answer that

works. Use as many blanks as there are words in

the answer. Don’t give clues by length of blank, make all

blanks the same length. Don’t give grammatical clues i.e., a ___ or an

____ make it a(n) _____

Let’s Review

5. _____________ assessment is the type of assessment given at the end of a unit.

6. You should have more ____________ items on a true/false test than ______ items.

Grading Issues(so far)

Quick and easy! Poor handwriting is less of a concern.

Students view these types of items as being less subjective than essay or short answer tests.

Grading multiple choice, true/false, matching and short answer items is fast and easy. Creating good items, however, is time consuming and difficult.

Essay Questions & Problems

Essay questions are prompts or questions to which students must respond in written format.

Given information, students are required to solve scientific problems.

Both essay questions and problems require students to show their work and often result in the awarding of partial credit.

Essay Questions & Problems Try to use a larger number of essays that require

brief responses as opposed to fewer long essays (able to sample from a larger content area --> more reliable results)

Do not let students select which essay or problems to answer.

It is important to write ideal response before using the item (you will need to do it anyway, it's better to do it as part of item development!)

Specify number of points or the amount of time you expect students to spend on essay.

Essay Questions & Problems (cont.) Make physical separation between parts of

the essay question - prompt/remind students about them (bullets, listed by #, space, etc.)

Pay attention to the amount of space allocated for the answers.

Word choice is fairly important - don't want value questions or philosophical questions, "discuss" is often too vague - focus students on what is wanted.

An Example Surprised to see a number of students

suddenly leave the room only halfway through an exam, the professor followed them into the corridor. He discovered the students completing the balance of the questions while leaning against the walls. When he asked for an explanation, one student pointed to the third question, which asked "Describe Hemingway's The Old Man and the Sea. You may supply a drawing but please leave room to answer."

http://www.educationworld.com/a_lesson/lesson275.shtml

Scoring Issues for Essays & Problems

Grade questions in a group (all students’ #1, all students’ #2, etc.) Consistent use of partial credit Halo effect

Try not to look at student names when grading

Try not to let sloppy/neat papers influence your assessment of their content

Scoring Rubrics or Grading Criteria

Determine the criteria that will be evaluated Develop levels of performance/quality for

each criteria Decide if certain areas are to be weighted

more heavily than others

Visit http://rubistar.4teachers.org/index.php for help in creating rubrics

Science Fair Experiment : Science Fair 2004

CATEGORY 4 3 2 1

Scientific Question Independently identified a question which was interesting to the student and which could be investigated.

Identified, with adult help, a question which was interesting to the student and which could be investigated.

Identified, with adult help, a question which could be investigated.

Identified a question that could not be tested/investigated or one that did not merit investigation.

Description of Procedure

Procedures were outlined in a step-by-step fashion that could be followed by anyone without additional explanations. No adult help was needed to accomplish this.

Procedures were outlined in a step-by-step fashion that could be followed by anyone without additional explanations. Some adult help was needed to accomplish this.

Procedures were outlined in a step-by-step fashion, but had 1 or 2 gaps that require explanation even after adult feedback had been given.

Procedures that were outlined were seriously incomplete or not sequential, even after adult feedback had been given.

Variables Independently identified and clearly defined which variables were going to be changed (independent variables) and which were going to be measured (dependent variables).

Independently identified which variables were going to be changed (independent variables) and which were going to be measured (dependent variables). Some feedback was needed to clearly define the variables.

With adult help, identified and clearly defined which variables were going to be changed (independent variables) and which were going to be measured (dependent variables).

Adult help needed to identify and define almost all the variables.

Teacher Name: Dr. Henriques

Student Name:     ________________________________________

Other Points to Consider The allocation of points should mirror

instructional time (content) Questions ought to span Bloom’s Taxonomy

and be of varying difficulty levels Types of items/tasks should mirror

instructional strategies (process) Notice the amount of reading - you want to

test science knowledge, not reading skills Develop scoring criteria BEFORE giving test

This should never happen!

Let’s Review…..

3. Multiple choice tests are less subjective than essay tests.

Last one!

7. When grading student work it is best to . . .A) grade the best student’s paper first.B) grade the worst student’s paper first.C) grade the same item for every student at the

same time before moving on to the next question.D) all of the above.

THANK YOU!

This presentation is available @www.csulb.edu/~lhenriqu/CSTA2004.ppt

Laura HenriquesCalifornia State University, Long Beach