Extended Assessment Administration Training

Oregon Department of Education

Behavioral Research and Teaching

January 2007

Welcome!

Dr. Gerald Tindal University of Oregon

Steve Jonas Lane ESD

Dianna Carrizales Oregon Department of Education

Agenda• DAY 1• Wednesday, January 24th

– 7:30 Check in/registration (arrangement will vary by site)– 8:00 Training begins: – 8:00 -- 9:30 Introduction, background, and pre-survey– 9:30 – 11:30 Mathematics– 11:30 -12:30 Lunch (local host sites have different arrangements)– 12:30 – 1:00 Mathematics (continued)– 1:00 – 3:30 Reading– 3:30 – 4:00 Wrap up

• DAY 2• Thursday, January 25th

– 7:30 Check in (arrangement will vary by site)– 8:00 Training begins:– 8:00 – 8:30 Follow-up/review – 8:30 –11:00 Writing– 11:00—11:30 Science– 11:30 – 12:30 Lunch (local host sites have different arrangements)– 12:30 – 2:30 Science (continued)– 2:30 – 4:00 Data Entry, next steps, post-survey and wrap up

Objectives

• To become familiar with the new format of the Extended Assessments

• To gain understanding of the general scoring process associated with the new format

• To gain understanding of the administration process associated with the new format in general and for each of the subject areas

• To gain an understanding of the different administrations of the Extended Assessments

• To become familiar with the structure of the Extended Assessment system

Regarding Technical Difficulties

• Local issues

• Issues in the studio

• Act of Nature

• 1 ½ hour delay: access DVD for that area

• DVD availability: approximate mid-February

Facilitator Role

• Facilitate pre- and post-survey• Consolidate and field questions for ODE• Maintain a list of participant names,

phone numbers and email addresses for qualification

• Orient participants toward appropriate materials

• Facilitate break-out exercises

Facilitators Introduce Yourselves

Format of Training

• 1200 trainees statewide• 50 sites• Frequent breaks• Opportunities for facilitation and

organization– Write down questions about the materials and

administration– Consider the differences in administration you

will anticipate from student to student – Examine the student materials

On Becoming a Qualified Assessor 2007 (QA07)

• Qualified Assessors must:– Attend and complete an entire state-approved training

(Cadre I, II or DVD)– Submit name to ODE to be maintained in the state assessor

database– Conduct one level (elementary or middle/high) of the

assessment with a peer or student– Submit materials* for all four subject areas– *Two scaffold administration tasks and two standard

administration tasks– *One task each (both pre-requisite tasks, and content

prompts)– Send administered assessments to ODE– Be prepared for ODE audit follow-up questions

Pre-Survey

Assessment Options Then and Now: What’s left?

• 2001-2005 • 2006-2007

10 CIM

8

7

6

5

4

3

10 CIM

8

7

6

5

4

3

Extended CLRAS

Exte

nd

ed

A

ssessm

en

t

Sta

nd

ard

Ad

min

istr

ati

on

Exte

nd

ed

Assessm

en

tS

caff

old

A

dm

inis

trati

on

Accom

mod

ate

d A

ssessm

en

t

Gen

era

l A

ssessm

en

t

Targ

ete

d A

ssessm

en

t

Juri

ed

Assessm

en

t

Accom

mod

ate

d A

ssessm

en

t

Mod

ified

Assessm

en

t

Extended K - 2

What is Scaffold Administration?

• Same items

• More prompting and support

• Possible different student materials

IEP Team Decision Making

Student

Elementary Middle High

Reading ReadingWritingMathematics Science WritingMathematics Science

ST

ST

ST

ST

ST

ST

ST

STST

IEP Team Decision Making

• The IEP team may choose to administer a scaffold administration of one subject area and a standard administration of another subject area

Consider General Assessment with or without accommodations if:

• Student:• Performs at or around grade level• Has academic difficulties that primarily surround reading but falls within the

normal range in other subject areas• Has academic difficulties in areas other than reading that are “mild to

moderate” and can typically be addressed by using simplified language• Is reading within two to three grades of his or her enrolled level

• Instruction:• Is primarily general curriculum instruction (but may also use a specialized

curriculum in some areas)

• Some Judgment variables: • What assessment did he take last year? • How is his attention? • What types of behaviors should be considered?

Consider Extended Assessment Standard Administration if:

• Student:• Performs well below grade level• Is significantly below grade level in reading• Has academic difficulties that are generalized (to all subject areas) and

are significant• Benefits from specialized individual supports

• Instruction:• Is primarily from a specialized curriculum and/or• From general curriculum must be significantly reduced in breadth,

depth, and complexity

• Some Judgment variables:• What assessment did he take last year?• How is his attention? • What types of behaviors should be considered?• Previous relevant experiences

Consider Scaffolded Administration of Extended Assessment if:

• Student:• Performance is significantly impacted due to the nature

of/characteristics of the disability • Does not read • Has academic, mobility, receptive and expressive language difficulties

that are generalized and significant• Relies on individual and significant supports and adaptations to access

reduced content materials

• Instruction:• Is from a specialized curriculum and has functional components and/or• Includes academic goals that are significantly reduced in depth,

breadth, and complexity from grade level content

• Some Judgment variables:• Is the student able to interact with instructional material in a way that

provides meaningful feedback?

Accommodations and Modifications

• Accommodations: Do not change the intended content standards but provide access to the assessment without altering the construct

• Modifications: Change the intended content standards of the measure and impact the validity of the item/measure

Accommodations

Accommodations of – Setting– Response– Presentation– Timing/Scheduling

• Examples– Extended time – Frequent breaks – Change in test directions– Change in font or size of text – Assistive technology – Test setting – Manipulatives

• When used? – In the classroom vs. Testing only?

Accommodations, Modifications, & the Extended Assessment

• Accommodations Table usage• Use of accommodations

– All students

– As used in the classroom

– Submission of recommendations to panel

• Modifications table and usage• Use of modifications

– IEP team judgment (consideration of meaningful access)

– As needed based on team judgment

– No recommendations are currently being submitted or considered

Testing Schedule

• Extended Assessment Window• March through April• Students follow same testing grades as TESA

• http://www.ode.state.or.us/teachlearn/testing/schedules/asmttestschd0607.pdf

• See table

Extended Assessment• Grade Levels

– Elementary level assessment grades 3 - 5(Scaffold or Standard administration)

– Middle/High level assessment grades 6 – 8 & 10 (Scaffold or Standard administration)

• Content– Mathematics, Reading, Writing, Science

• Skill levels– Grade level content standards reduced in depth, breadth,

and complexity – 5 Prerequisite skills per task– 5 Content prompts per task

• Scoring– Levels of Independence – Accuracy

Materials(Elementary)

Document Reading Writing Math Science

Elementary – Extended Standard Administration Scoring Protocol

Elementary – Extended Standard Administration Student Materials

Elementary – Scaffold Administration Scoring Protocol

Elementary – Scaffold Administration Student Materials

Power Point presentation materials

Materials (Middle High)

Document Reading Writing Math Science

Middle/High – Extended Standard Administration Scoring Protocol

Middle/High – Extended Standard Administration Student Materials

Middle/High– Scaffold Administration Scoring Protocol

Middle/High – Scaffold Administration Student Materials

Power Point presentation materials

Format of the New Assessment

New Scoring Protocol

Prerequisite Skills

• Questions that all students can participate in

• Administer to students who are primarily working on life skills

• Appear on each task for all extended assessments

• Five questions per task

Prerequisite Skills

• Each of the pre-requisites for each task have a similar sequence of difficulty

• All can be answered by pointing or communicating when the assessor is touching the correct item

Prerequisite Skills

• Each question measures the level of support the student needs in order to be successful in answering the question correctly

• What you are measuring is the support you give to the student

• Scoring is not for accuracy

Prerequisite Skills

• Scoring is the measure of the level of independence for success

• Always allow the student to have an opportunity to Independently answer the question

Prerequisite Skills

• Levels of Independence:– 4 = Independent– 3= With Verbal, Gesture, or Visual Prompt– 2= With Partial Physical Assistance– 1= With Full Physical Assistance

Prerequisite Skills

• I= Question is Inappropriate based on the nature of the student’s disability

• R= Does not Complete with Any Form of Assistance (Refusal)

Level of Independence selection process for proceeding to content prompts

• Prerequisite total is the sum of the five prerequisite item scores

• Level of Independence (level of support provided) for content area is based on the lowest value selected during administration of the prerequisite skills (i.e. the most support provided to the student during prerequisite administration)

Examples

• Example1: Lily receives prerequisite scores of: 4, 3, 2, 2, 1 on any given task, then the Level of Independence = 1

• Example 2: Brad receives prerequisite scores of: 1, 1, 1, 2, 1; Level of Independence =1

• Example 3: Roy receives 4, 1, 1,R*, R*; Level of Independence = 1

Partial Scoring for Participation

• Student must complete a minimum of 5 Prerequisite Skills tasks with the following contingencies

• If a student receives a score of 5 or less per prerequisite task the student is eligible to stop testing and can still count for participation

• If a student receives a score greater than 5 on the Prerequisite Skills associated with a task, the student must proceed to the Content Prompt for the associated task even if the teacher must score an "I" or "D" (as appropriate).

• Completion of 5 prerequisites is the minimum participation expectation, however, all accommodations and appropriate provisions should be considered thoroughly prior to discontinuing an Extended Assessment administration

Proceeding to Content Prompts

• If tasks are too difficult for student, administrator may score “D” for each item.

• If tasks are completed accurately via full physical support (i.e. Independence was scored as 1), then Accuracy Score can be either 1 or 2

• If a student taking the Standard Administration is deemed proficient at all prerequisite skills then Level of Independence is scored as 4, and Content Prompts are administered with no supports.

Scoring Summary

• Prerequisites are administered with whatever level of support is required and appropriate for the student to be successful on the item.

• Prerequisite total: is the sum of the prerequisite items. • Level of independence for moving into the content prompts is

the lowest value from the prerequisites: this demonstrates the level of permission for the additional supports that can be provided (beyond standard administration) to allow the student to access the materials in the content prompts.

• Partial participation: A minimum of 5 prerequisite tasks are necessary to count toward participation.

• A score of 5 or less on prerequisites means that the student is not obligated to proceed into the content prompts.

Elementary Mathematics

General Overview

Elementary MathematicsTask Names

• Task 1: Numbers• Task 2: Computation• Task 3: Measurement of Time/Temperature• Task 4: Geometry/Shapes• Task 5: Measurement• Task 6: Interpret Data and Graphs• Task 7: Applications• Task 8: Fractions and Conversions• Task 9: Probabilities and Predictions• Task 10: Algebra

Administration – Big Picture

• Task Content

• Pre-requisite Skills

• Content Prompts

• Materials Preparation

• Scoring

• Anticipations and Possibilities

• Role play

Elementary Mathematics: Task Content • Numbers: Recognition, ordering, digits placement, count-bys

• Computation: Simple addition, simple subtraction, multiplication, division

• Time/Temperature: Telling time, calculating time, temperature, season

• Geometry/Shapes: Shape recognition, angles, perimeter calculation

• Measurement: Measuring in units, measurement calculations, use of weights and measures in day to day life

• Interpret data and graphs: Interpreting a simple graph

• Application: Money recognition, counting money, adding money, cost and making change

• Fractions & Conversions: Recognizing fractions, decimals, and percentages

• Probabilities & Predictions: Coin tosses and probability, odds, likelihood

• Algebra: Missing values, missing symbols, word problem

Elementary Mathematics Pre-requisite Skills

• Measure of a student’s level of independence in response to :– Requests for attention– Requests to interact– Specific queries regarding: numbers, shapes, clocks, or coins– Specific queries regarding: symbols, shapes, time, graph, tool, bills, coins

• Numbers: put the cards into two groups• Computation: put two number cards together• Time/Temperature: show me a picture of a thermometer and snow• Geometry/Shapes: show me two objects that are similar• Measurement: tell me show me something you can measure with a ruler• Interpret data and graphs: show me the tallest bar on the graph• Application: put two coins together• Fractions & Conversions: show me a fraction• Probabilities & Predictions: show me which coin shows heads• Algebra: show me a number sentence

Elementary Mathematics Materials PreparationTask Cutting Other materials

(Have scratch paper available for all tasks)

1 Item 2 Student materials only (with cover page)

2 Optional Student materials only (with cover page)

3 Optional Student materials only (with cover page)

4 Optional Student materials only (with cover page)

5 Optional Ruler and student materials (with cover page)

6 No Student materials only (with cover page)

7 Optional Coins, dollar bills and student materials (with cover page)

8 Optional Student materials only (with cover page)

9 Optional Student materials. Optional: coins, 3-point spinner, jelly beans or manipulative (with cover page)

10 Optional Student materials only (with cover page)

Elementary Mathematics Scoring

• Correct and specific answer = full credit• Incorrect answer or refusal inaccurate

attempt = no credit• General correct answer = partial credit• On the right track to a correct conclusion =

partial credit• Correct working incorrect final calculation

= partial credit

Scoring (Continued)

• Additional querying/prompting may be used if the student’s response suggests that he or she misunderstood/misheard the item

• Preamble consistency

Elementary Mathematics Anticipations and Possibilities

• Difficulties: – Some answers have only a right or wrong answer – Some of the graphics are focal points only– Tasks are not in order of difficulty

• Opportunities: – Some of the questions lend themselves to the use of classroom

materials that are familiar to the students– This format provides instructional ideas for presenting concepts

that students have not yet encountered • Next steps:

– Streamlined graphics– More details regarding partial scoring

Middle/High Mathematics

General Overview

Middle/High MathematicsTask Names

• Task 1: Computation & Calculation (C & C) — Numbers, Computation, Estimation

• Task 2: C & C — Numbers, Problem Solving, Rational Numbers, and Equivalence

• Task 3: C&C — Fractions and Numbers• Task 4: Statistical Measures and Probabilities• Task 5: Probability (Representation) and Algebra (Patterns)• Task 6: Algebra — Variables, Equations, and Relationships• Task 7: Algebra — Graphing and Change• Task 8: Measurement — Determining Measurement• Task 9: Geometry — Properties, Transformation, and Symmetry• Task 10: Geometry — Coordinate Graphing & Modeling

Administration – Big Picture

• Task Content

• Pre-requisite Skills

• Content Prompts

• Materials Preparation

• Scoring

• Anticipations and Possibilities

• Role play

Middle/High Mathematics: Task Content • Task 1: Recognition of equality, equality in fractions, reduction of fractions• Task 2: prime numbers, creating fractions, recognizing relative size in

fractions/decimals• Task 3: Recognizing fractions in shapes, fractions in day to day life• Task 4: calculating averages, medians, combinations and patterns, likelihood

with coin tosses• Task 5: Representing data, likelihood, patterns of numbers in sequence• Task 6: Missing variables, number patterns• Task 7: identifying growth on a graph, points on a graph, value of y given x• Task 8: measuring, perimeter of a triangle, speed (miles per hour), selecting

correct units of measurement• Task 9: Recognizing shapes (equilateral triangles, parallel lines, radius of a

circle, lines of symmetry), applying the Pythagoras theorum.• Task 10: Properties of shapes (rectangles), measurement of a shape

Middle/High Mathematics Pre-requisite Skills

• Measure of a student’s level of independence in response to :Requests for attentionRequests to interactSpecific queries regarding:

numbers, graphs, shapesSpecific queries regarding:

percents, decimals, math problems, general knowledge, various objects, graphs and axes, specific shapes, and properties of shapes

• Task 1: show me a fraction• Task 2: show me a fraction• Task 3: show me a sequence of numbers• Task 4: show me which coin shows heads• Task 5: show me a pattern of numbers• Task 6: show me a table on the paper• Task 7: show me a point on the graph• Task 8: show me a length• Task 9: show me an angle• Task 10: show me the long side of a

rectangle

Middle/High Mathematics Materials PreparationTask Cutting Other materials

(Have scratch paper available for all tasks)

1 Optional Student materials only (with cover page)

2 Optional (but recommended)

Student materials only (with cover page

3 Optional Student materials only (with cover page)

4 Optional Coin to flip (optional) Student materials only (with cover page)

5 Item 1 Photo-copy table for item 1 and provide student materials (with cover page) for remainder

6 Optional Student materials only (with cover page)

7 Optional Student materials only (with cover page)

8 Optional Student materials (with cover page), ruler (optional)

9 Optional Student materials only

10 Optional Ruler, Student materials, cover page

Middle/High Mathematics Scoring

• Correct and specific answer = full credit• Incorrect answer or refusal inaccurate

attempt = no credit• General correct answer = partial credit• On the right track to a correct conclusion =

partial credit• Correct working incorrect final calculation

= partial credit

Middle/High Mathematics Anticipations and Possibilities

• Difficulties: – Some answers have only a right or wrong answer – Some of the graphics are focal points only– Tasks are not in order of difficulty

• Opportunities: – Some of the questions lend themselves to the use of classroom materials

that are familiar to the students– This format provides instructional ideas for presenting concepts that

students have not yet encountered– If students are more comfortable with familiar manipulatives they may be

substituted as appropriate • Next steps:

– Streamlined graphics– More details regarding partial scoring

Standard and Scaffold Administration

• Materials

• Exercises

Querying and Prompting in Content Prompts

• Once in content prompt, administration is standardized

• Support is provided to Access to material and not to ensure success on an item

• Queries and prompting is provided in the script, however,

• Administrators may query further (or probe) if a student’s response suggests that the student has misunderstood the question due to its presentation