1

Understanding the SMARTER Balanced Assessment Design: ELA Specifications and

Claims

2

Learning Targets for Assessment DesignI can explain the basic design principles of the

SMARTER Balanced Assessment framework. • Evidence-Based Design• Depths of Knowledge• Learning Goals organized by Claims

• I can learn about how SMARTER Balanced proposes to assess student learning for each of the five claims through a jigsaw activity.

3

Curriculum-Instruction-Assessment Connections

Norming (5 min.) Students

Curriculum

Instruction Assessment

What should students know and to be ale to do?What should students learn?What should students be taught?

What are student being taught?How are students being taught?

What have students learned?What haven’t students learned?

4

Evidence Based Design Framework

“Assessment Triangle”

Observation Interpretation

Cognition

A set of specifications forassessment tasks that willelicit illuminating responsesfrom students

The methods and analytic tools used to make sense of and reason from the assessmentobservations/evidence

Beliefs about how humans represent information and develop competence in a particular academic domain

5

Think-Pair-Share

How do you think this model is different from how the CST measures student learning?

6

Models of Cognition

Knowledge

Knowledge

Comprehension

Application

Analysis

Synthesis

Evaluation

7

Shortcomings of Generic Taxonomies

• Capture a single point in time; do not look at a developmental process or learning progressions.

• A one-size-fits-all model for all learning domains that doesn’t take into consideration what varies by content.

• A focus on the verb and not the context of learning: the content or the task.

8

Webb’s Depth of Knowledge

• A different way of looking at the cognition point of the “Assessment Triangle”

• Adapted from a model developed by Norman Webb from the University of Wisconsin to align assessments to standards

9

Depths of Knowledge (DOK)

Webb’s Depth of Knowledge levels:

Recall and Reproduction: Level 1Basic Application of

Skills & Concepts: Level 2Strategic Thinking: Level 3Extended Thinking: Level 4

10

10

BLOOM’S REVISED TAXONOMY WEBB’S DEPTH OF KNOWLEDGE

REMEMBER Retrieving relevant knowledge from long-term memory (e.g., recognizing, recalling)

Recall – Recall of a fact, information, or procedure (e.g., What are the Red Cross Emergency Action steps [check, call, care]?)UNDERSTAND

Determining the meaning of instructional messages, including oral, written, and graphic communication (e.g., interpreting, exemplifying, classifying, summarizing, inferring, comparing, explaining)

APPLYINGCarrying out or using a procedure in a given situation (e.g., executing, implementing)

Basic Application of Skill/Concept – Use of information, conceptual knowledge, procedures, two or more steps, etc.

(e.g., Given a emergency scenario, students determine the care needed for a victim, and explain the reason for their actions).

ANALYZINGBreaking material into its constituent parts and detecting how the parts relate to one another and to an overall structure on purpose (e.g., differentiating, organizing, attributing)

Strategic Thinking – Requires reasoning, developing a plan or sequence of steps; has some complexity; more than one

possible answer; generally takes less than 10 minutes to do (e.g., Module 363 –ER – Stressed due to parents’ divorce;

Crunched for time; Signs of stress – ways to relieve stress – why managing stress is important to health.)EVALUATE

Making judgments based on criteria and standards (e.g., checking, critiquing)

CREATINGPutting elements together to form a novel, coherent whole or make an original product (e.g., generating, planning, producing)

Extended Thinking – Requires an investigation; time to think and process multiple conditions of the problem or task; and more than 10 minutes to do non-routine manipulations (e.g.,

Task 608 – Welcome to Health High – Create fact sheet/brochure from research activity)

Cognitive Rigor Matrix

Depth + thinking

Level 1Recall & Reproduction

Level 2Skills & Concepts

Level 3Strategic Thinking/ Reasoning

Level 4Extended Thinking

Remember - Recall, locate basic facts, details, events

Understand - Select appropriate words to use when intended meaning is clearly evident

- Specify, explain relationships- summarize– identify main ideas

- Explain, generalize, or connect ideas using supporting evidence (quote, example…)

- Explain how concepts or ideas specifically relate to other content domains or concepts

Apply - Use language structure (pre/suffix) or word relationships (synonym/antonym) to determine meaning

– Use context to identify meaning of word- Obtain and interpret information using text features

- Use concepts to solve non-routine problems

- Devise an approach among many alternatives to research a novel problem

Analyze - Identify whether information is contained in a graph, table, etc.

– Compare literary elements, terms, facts, events– analyze format, organization, & text structures

- Analyze or interpret author’s craft (literary devices, viewpoint, or potential bias) to critique a text

– Analyze multiple sources- Analyze complex/abstract themes

Evaluate – Cite evidence and develop a logical argument for conjectures

- Evaluate relevancy, accuracy, & completeness of information

Create - Brainstorm ideas about a topic

- Generate conjectures based on observations or prior knowledge

- Synthesize information within one source or text

- Synthesize information across multiple sources or texts

©2009 Karin K. Hess

12

Depths of Knowledge (DOK)• Describes the assessment task, not the student

• Not the same as difficulty, focuses on the level of cognition (thinking) required by a task

• The Depth of Knowledge is NOT determined by the verb, but the context in which the verb is used and the depth of thinking required

13

Depths of Knowledge (DOK)• DOK 3- Describe a model that you might

use to represent the relationships that exist within the rock cycle. (requires deep understanding of rock cycle and a determination of how best to represent it)

• DOK 2- Describe the difference between metamorphic and igneous rocks. (requires cognitive processing to determine the differences in the two rock types)

• DOK 1- Describe three characteristics of metamorphic rocks. (simple recall)

Same verb, three levels of DOK

14

Checking for Understanding• At your table, think of a skill or concept that you might teach to students.

• What would you ask students to do to reach each of the four levels of Depths of Knowledge?

15

Depths of Knowledge (DOK)

SMARTER Balanced assesses at a variety of DOK levels so we have opportunities to understand what students do & don’t know, and where they are in the novice- expert spectrum

Performance assessments can offer varying levels of DOK embedded in a larger, more complex task

Planned formative assessments intentionally focus on lower DOK levels so we can see where the student is in relation to the standard

16

Evidence Based Design Framework

“Assessment Triangle”

Observation Interpretation

Cognition

A set of specifications forassessment tasks that willelicit illuminating responsesfrom students

The methods and analytic tools used to make sense of and reason from the assessmentobservations/evidence

Beliefs about how humans represent information and develop competence in a particular academic domain

17

Claims

• SBAC organizes the standards into 5 “claims”. These claims…

• Frame a manageable number of learning goals around which instruction can be organized.

• Guide the specification of appropriate evidence.

• Provides a basis for meaningful reporting to different interested audiences.

18

Summary of Claims & ELA and Literacy CCSS

Claim #1 – Students can read closely & critically to comprehend a range of increasingly complex literary and informational texts.•Literature: RL 1-3, 5-7, & 9•Informational text: RI 1-3, 5-9•Read/ Literacy: 1-3, 5-9 (gr 6–11)

Claim #2 – Students can produce effective writing for a range of purposes and audience.•Narrative (W1), Informational (W2); Opinion / Argument (W3)•Writing Literacy 1-2 (gr. 6-11)•Plan / Organize / Edit (W4); Language – Edit (L1, L2)•Writing – Gather evidence (W8) and Draw evidence (W0) (gr 4-11)

19

Summary of Claims & ELA and Literacy CCSS

Claim #3 – Students can employ effective speaking and listening skills for a range of purposes and audiences.•Listening (SL – 1d, 2,3)•Speaking (SL – 4,5,6)

Claim #4 – Students can engage appropriately in collaborative and independent inquiry to investigate / research topics, pose questions, and gather and present information.•Writing (W7, W8) & draw evidence: W9 (gr 4-11)•Reading Literacy: 1-3, 5-9 (gr 6-11)•Writing Literacy: 1-2 (gr 6-11)

20

Summary of Claims & ELA and Literacy CCSS

Claim #5 – Students can use oral and written language skillfully across range of literacy tasks.•Read Literature (RL4); Info (RI4); Read /Literacy: 4 (6-11)•Language Use: 3,4,5•Speaking / Listening: LS1d,2,3

This claim is “bundled with the other claims” in the assessment design, i.e. it is not assessed in isolation but in the context of reading, writing, listening and speaking.

21

Content Specifications: Assessment Target Anatomy

P. 23 of content specification

22

Sources

Smarterbalanced.org

Norman Webb’s Depth of Knowledge, www.education.ky.gov/users/jwyatt/.../DOK%2012-151.PPT

Increasing Rigor by The Network of Dynamic Learning Communities, www.cfn107.org/.../cfn_cognitive_rigor_and_tweaking_tasks_2_5_1...

23

Jigsaw

Your task: In small groups, you will learn about how one claim is mapped out in terms of how it is linked to the CCSS and probable methods of assessment. You will also look at sample tasks that serve as examples of how student learning will be assessed. Note: These tasks are existing and public assessments that serve as examples, and are not designed by SMARTER Balanced.

24

Jigsaw• Four groups (one for claims #1-4)• 30 minutes to look at and discuss

• Summative Assessment Targets (look at one grade level)

• Sample Assessment Tasks • Chart main points and report back on:

• A summary of what is assessed for each claim

• How they are assessed• Examples of sample assessment tasks at one or two grade levels

• 1-3 Implications for our work• 5 minutes for each group to report back + 2 minutes questions

25

Jigsaw: Expert groupsClaim #1: Suki, Erin, Sasha, Leah

Claim #2: Mike, Sarah, Stan, Ann

Claim #3: Brett, Jody, Nancy, Shelly

Claim #4: Alexis, Elizabeth, Nicole, Nabila

26

LUNCH!

Please return at 1:15

27

Jigsaw part II

Each group has 7 minutes to report back

28

Independent Reading Time

Choose from the list of reading resources posted on the wiki or from your own list of CCSS readings.

You have ONE hour to find a comfortable space to read.

Be prepared to share some of the highlights and key passages.

29

Closing