Exploring SBAC

August 2, 2013MCLA

A Balanced Assessment System

Common Core State Standards specify

K-12 expectatio

ns for college and

career readiness

All students

leave high

school college

and career ready

Teachers and schools have information and tools

they need to improve

teaching and learning

Summative: College and career

readiness assessments for

accountability

Interim: Flexible and open

assessments, used for actionable

feedback

Formative resources:Supporting

classroom-based assessments to

improve instruction

How SBAC is different…

Transparent test development using evidence-centered design

Uses technology to enable students to demonstrate in new ways

Common standards and common achievement levels across multiple states

Flexibility in administration for individual students

Specific knowledge and skills to be measured are identified

Available options seem to support student-centered curriculum and instruction

Summative Assessments Replace all current Maine summative assessments in grades 3-8 & 11 in

ELA and math beginning in the spring of 2014 – last 12 weeks of school. There will be an 18 month gap between next fall’s NECAP and SBAC.

Individual students can do the test in segments, if necessary, and can repeat it during the 12 week window.

Testing time is 8 hours total, which includes 1.5 – 2 hour performance task.

Accommodations are built into test items and keyed by individual student needs.

Items are adaptive within each grade level, but not across grade levels.

Scoring will account for adaptive items, but exactly how this is reported hasn’t been determined yet.

Descriptors at each score point have been identified, but cut scores won’t be determined until after 2014 administration.

Scores available 2 weeks after administration for summative assessment.

Determination of “college-ready” at grade 11 & agreements with higher ed.

College Content-Readiness Definition English Language Arts/Literacy3

“Students who perform at the College Content-Ready level in English language arts/literacy demonstrate reading, writing, listening, and research skills necessary for introductory courses in a variety of disciplines. They also demonstrate subject-area knowledge and skills associated with readiness for entry-level, transferable, credit-bearing English and composition courses.”

Mathematics

“Students who perform at the College Content-Ready level in mathematics demonstrate foundational mathematical knowledge and quantitative reasoning skills necessary for introductory courses in a variety of disciplines. They also demonstrate subject-area knowledge and skills associated with readiness for entry-level, transferable, credit-bearing mathematics and statistics courses.”

Policy Framework Grade 11 College Content-Ready Assessment Results

• Not Yet Content-Ready - Substantial Support Needed

• K-12 & higher education may offer interventionsLevel 1

• Not Yet Content-Ready – Support Needed• Transition courses or other supports for Grade

12, retesting optionLevel 2

• Conditionally Content-Ready/Exempt from Developmental

• In each state, K-12 and higher ed jointly develop Grade 12 requirements to earn exemption

Level 3

• Content-Ready/Exempt from Developmental• In each state, K-12 and higher education may

jointly set Grade 12 requirements to retain exemption

Level 4

Assessment Types

Selected Response Items

Constructed Response and Extended Response Items

Technology- Enhanced Items

Performance Tasks

Selected Response Items

Selected Response Items (SR) contain a series of options from which to choose correct responses. Items:

Reflect important knowledge and skills consistent with the expectations of the CCSS across the Depths of Knowledge (i.e., Recall/Literal Comprehension, Interpretation/Application, and Analysis/Evaluation).

Allow students to demonstrate their use of complex thinking skills, such as formulating comparisons or contrasts; identifying cause and effects; identifying patterns or conflicting points of view; categorizing, summarizing, or interpreting information.

Constructed Response and Extended Response Items

Constructed Response (CR) is a general term for items requiring the student to generate a response as opposed to selecting a response.

Short constructed response items may require test-takers to enter a single word, phrase, sentence, number, or set of numbers, whereas extended constructed response items will require more elaborated answers and explanations of reasoning.

comparisons or contrasts; proposing cause and effects; identifying patterns or conflicting points of view; categorizing, summarizing, or interpreting information; and developing generalizations, explanations, justifications, or evidence-based conclusions

Technology- Enhanced Items

Technology-Enhanced Items employ technology to: Elicit a response from the student (e.g., selecting one or more points on

a graphic, dragging and dropping a graphic from one location to another, manipulating a graph)…, and/or

Employ technology to assess content, cognitive complexity, and Depth of Knowledge not assessable otherwise.

Items include:

drag and drop

hot spot

drawing

graphing

gridded-response items

simulation technologies

use of online tools

Performance Tasks

Measure the student’s ability to integrate knowledge and skills across multiple standards and to better measure capacities such as depth of understanding, research skills, and complex analysis, which cannot be adequately assessed with selected response or constructed response items. Performance tasks:

Measure capacities such as depth of understanding, research skills, and/or complex analysis with relevant evidence;

Require student-initiated planning, management of information and ideas, and/or interaction with other materials;

Require production of more extended responses (e.g., oral presentations, exhibitions, product development), in addition to more extended written responses that might be revised and edited;

Reflect a real-world task and/or scenario-based problem;

Lend itself to multiple approaches;

Represent content that is relevant and meaningful to students;

Allow for demonstration of important knowledge and skills, including those that address 21st century skills such as critically analyzing and, synthesizing media texts;

Focus on big ideas over facts;

Allow for multiple points of view and interpretations.

Performance Tasks

Integrate knowledge and skills Measure understanding, research skills,

analysis, and the ability to provide relevant evidence

Require student to plan, write, revise, and edit Reflect a real-world task Demonstrate knowledge and skills Allow for multiple points of view Feasible for classroom environment

General Specifications forPerformance Tasks

Allow teacher and peer interactions and group work

Organization of complex task directions

Simulated Internet access

Rubrics

Options

Interim assessments

• 3-8 and 9, 10, & 12

• Can be used for any number of students

• Reported on the same scale as the summative assessment,

• Schools will have the flexibility to assess small elements of content or the full breadth of the Common Core State Standards at locally-determined times throughout the year.

Digital Library

• Resources, including formative assessments, based on an accepted formative assessment cycle.

Formative Process

Evidence-Centered DesignModern Approach to Designing Items and Tasks

Traditional Item Development

versus Evidence-Centered Design

Sampling: Traditional Approach to Item Development1.2.1 Apply reference skills to determine word meanings.

1.3.1 Understand and apply new vocabulary.

1.4.1 Know common sight words appropriate to grade-level.1.4.2 Apply fluency to enhance comprehension.1.4.3 Apply different reading rates to match text.2.1.1 Understand how to use questioning when reading.2.1.2 Understand how to create mental imagery.

2.1.4 Apply comprehension monitoring strategies before, during, and after reading: use prior knowledge/schema.

2.1.6 Apply comprehension monitoring strategies before, during, and after reading: monitor for meaning, create mental images.

1.2.2 Apply vocabulary strategies in grade level text.

1.3.2 Understand and apply content/academic vocabulary.

2.1.3 Understand and determine important or main ideas and important details in text.

2.1.5 Apply comprehension monitoring strategies before, during, and after reading: predict and infer.

2.1.7 Apply comprehension monitoring strategies during and after reading: summarize informational/expository text and literary/narrative text.

1.2.2

1.3.2

2.1.3

2.1.5

Why evidence-centered design?

Inconsistency between what performance level descriptors said students were able to do and what they were actually able to do based on further investigation.

So, while the content of an item might be aligned with a content standard, the information elicited by the item may not say anything meaningful about whether or not the student has achieved the standard.

Traditional Approach to Item Development

Item:

Beth says that 2 + 4 = 6. Explain why Beth is correct.

Traditional Approach to Item Development

Item:

Beth says that 2 + 4 = 6. Explain why Beth is correct.

Content Standard 2.2.3: Perform addition accurately for single and two digit numbers.

Do we know if the student can actually add single and two digit numbers?

Applying Evidence-Centered Design toItem and Task Development

Beth says that 2 + 4 = 6. Explain why Beth is

correct.2 + 4 = ____

Content Standard 2.2.4: Perform mathematical

operations and justify solutions.

Content Standard 2.2.3: Perform addition accurately for single and

two digit numbers.

Using Evidence-Centered Designto Guide Item Design

1. What evidence is required given the assessment target

I am measuring?

2. What are the key features that must be included in

the item?

3. Will this item allow for the production of the evidence

I am seeking?

4. Is there anything about this item that may make it

more difficult to collect evidence from some students?

6 Key Components of Evidence-Centered Design

6. Develop Items or Performance Tasks

1. Define the domain Common Core Standards Math/ELA

2. Define claims to be made 4 ELA & 4 Math ClaimsContent Specifications

3. Define assessment targets Knowledge, Skills, & Abilities

4. Define evidence required Evidence to be Elicited from Student

5. Develop Task Models Methods for Eliciting Evidence

Discussion Questions

1. What are the local implications of the SBAC summative assessments?

2. How will SBAC summative assessments mesh with other CCSS-aligned assessments we currently use?

3. What are the potential local impacts of SBAC options:

Interim 3-8?

HS 9,10, & 12?

Digital library?