reviewing sbac summative assessment results to guide instructional practice shannon wells ekaterina...

Reviewing SBAC Summative Assessment Results to Guide Instructional Practice

Shannon Wells

Ekaterina Forrester

Printed by the Riverside County Office of Education

From Scores to Instruction

Overall Scores:

ELA

Math

Claim Scores:ELA: Reading, Writing, Listening/Speaking, Research/Inquiry

Math: Concepts and Procedures, Communicating Reasoning, Data

Guiding Teaching and Learning:

Use of guiding questions to consider implications for practice

SBAC Results Reporting: What is Available?

• Public reporting site: research files, aggregate reports (by subgroup) http://caaspp.cde.ca.gov

• Electronic data file (final individual student results in TOMS)

• Student score reports

http://caaspp.cde.ca.gov/

http://caaspp.cde.ca.gov/

Site Review of SBAC Results• How will your district or school review the SBAC results? What are you going

to be looking as a team?o Site-wide data teams, o Grade-level/Department teams, o Individual teachers

• What format will be available for review?o Data management systems (Illuminate, OARS, EADMS)

• What will be available?o Electronic files, graphs/charts, reports?

• Will data be disaggregated by significant subgroups

• Will individual student names and needs (e.g., SED, SWD, Foster, EL) be available along with their scores?• Disaggregated data can include: Parent Ed level; gender; gender by (data not

publicly available); LTELs; Native Indian? (not from America); Migrant, etc

5

Site- and Grade-Level Teams

Guiding Questions for Site- and Grade-Level Teams• What do we notice about % of students in each level? What is surprising?

• How does overall % in each AL for our site compares to district, county, state?• Grade level % in each AL: any particular grade stands out?• Performance of subgroups: high-performing vs low-performing• Performance by parent education level

• Do the overall scores suggest one content area is stronger than the other?

• What areas of need (weaknesses) do claim scores within each subject and grade level suggest?• Look at % of level 1 students• What areas should particular grade levels focus on this year?

• What do we notice about average scale scores for each grade level? (please clarify – What g/l are they performing at?)• How do the average grade level scores compare with the “Meet the Standard” threshold?• How are average grade level scores different by subgroups?

7

3 r d G r a d e 4 t h G r a d e 5 t h G r a d e 6 t h G r a d e 7 t h G r a d e 8 t h G r a d e 1 1 t h G r a d e C o u n t y A l l S t a t e A l l0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

36% 33%

45%39% 42%

46%50%

41%

31%

29% 38%

30%

33%31%

29%26%

31%

25%

26%21% 15%

17% 17% 15%17%

19%

28%

9% 8% 10% 10% 9% 10% 6% 9%

16%

Sample SITE: Achievement Levels by Grade

Standard Exceeded Standard Met Standard Nearly Met Standard Not Met

% in

Ach

ieve

men

t Lev

el

What do you notice about the % in each achievement level? What surprises you?

Which grades are doing better? What do you think contributed to this?

8

African American Hispanic White African American Hispanic WhiteELA Math

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

41%38%

20%

54%48%

27%

27%29%

24%

29%31%

32%

23% 25%

35%

13% 15%

26%

8% 8%

21%

5% 5%

15%

Sample Site Percent of Students in Achievement Levels by Ethnicity

Standard Exceeded Standard Met Standard Nearly Met Standard Not Met

What do you notice about the % in each achievement level

for different subgroups?

9

HS Grad or Less Some College College Grad 0%

10%

20%

30%

40%

50%

60%

70%

28%

44%

60%

17%

28%

46%

Percent of Students Meeting or Exceeding Standard by Parent Education Levels

ELA Math

Parent Education Level

% M

eetin

g or

Exc

eedi

ng

What do you notice about the % meeting

or exceeding standard based on parent education

level?

10

Standard Met/Exceeded (3-4) Standard Not Met (1) Standard Met/Exceeded (3-4) Standard Not Met (1)ELA Math

0

10

20

30

40

50

60

70

80

46

2832

37

10

65

9

68

44

28 30

37

9

75

9

75

Sample Site Percent of Students Meeting/Exceeding versus Not Meeting Standard by Language Fluency and Disability

EO EL Non-SWD SWD

What do you notice about the % of EL and SWD students meeting or exceeding standard?

What do you notice about the % of EL and SWD students not meeting the standard?

11

3rd Grade 4th Grade 5th Grade 6th Grade 7th Grade 8th Grade 11th Grade2300

2350

2400

2450

2500

2550

2600

2650

Average SBAC ELA Scale Score by Grade and Ethnicity

African AmericanHispanicWhiteMeets Standard

Aver

age

Scal

e Sc

ore

College Ready 2682

How do the average grade level scores of different subgroups compare with the “Meet the Standard” threshold?

12

3rd Grade 4th Grade 5th Grade 6th Grade 7th Grade 8th Grade 11th Grade2300

2350

2400

2450

2500

2550

2600

2650

Average SBAC Math Scale Score by Grade and Ethnicity

African AmericanHispanicWhiteTarget

Aver

age

Scal

e Sc

ore

College Ready 2718

How do the average grade level scores of different subgroups compare with the “Meet the Standard” threshold?

13

Individual Teacher Rosters and Reports

FEED BACK: How Did We Do?Questions for 2014-15 teachers of record looking at their 2014-15 students’ scores:

• How did my students do compared to other students in the same grade level?• Looking at my 2014-15 students, who met or exceeded standards, what contributed to

their success?• Looking at my 2014-15 students, who were below standard, what instructional

strategies or supplemental resources would I add? (implementation of strategies, practices or resources)

• How can instructional time be adjusted to increase the number of students who are meeting or exceeding standard? (usage of time)

• Which claims within the content area are higher performing?• Which claims within the content area are lower performing? (What are you going to do

to go deeper in the claims?) • How do these results match the areas where I provided targeted instruction?• How do these results point to gaps in instruction? • What might the implications be for instructional practice? For student learning?

FEED FORWARD: Where Are we Going?

Questions to ask about students in your class in 2015-16:

• What are the strengths and needs of my class(es) based on their claim scores in ELA and math?

• Which claims within the content area are higher performing? Lower performing? • (What are you going to do to go deeper in the claims?)

• If there are patterns of need (e.g., more than half of my class scored below standard on “Concepts and Procedures”), what changes can I make to this year’s instructional plan to address this need?

• What do you feel worked well or assisted students based on the results?• What is the implication for instruction?

16

LastName FirstName SSID Gender School Grade ELStatus SED Disability Ethnicity ParentEd

SS_ELA AL_ELA Reading Writing Listening Research SS_Math AL_Math Concept & Procedures

Problem Solving/Data Analysis

Communicating Reasoning

Chanel Coco 1111111111 FemaleABC

middle 08 EO Yes 500 10 2766 4 3 3 3 3 2774 4 3 3 3

Lacoste Rene 2222222222 MaleABC

middle 07 EL Yes SLD 500 14 2398 1 1 1 1 2 2392 1 1 1 1

Vuitton Louis 3333333333 MaleABC

middle 07 EO Yes 700 12 2557 3 1 3 1 3 2485 2 1 2 2

Lauren Ralph 4444444444 MaleABC

middle 08 RFEP Yes SLD 400 11 2464 1 1 2 1 2 2512 2 1 2 2

Kors Michael 5555555555 MaleABC

middle 07 EO No 600 15 2521 2 2 1 3 2 2439 1 1 1 2

Average 2541 2 1.6 2 1.8 2.4 2520 2 1.4 1.8 2

Sample Roster

17

Reading Writing Listening Research/Inquiry



Grade 6 Grade 7 Grade 8

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

40%35%

20% 20%

40%

33%26% 28%

34%30%

24% 25%

47%

48%

69%

60%

45%

48% 64%54%

47%52% 67%

57%

13%16%

11%

19%14%

19%

10%

18% 18% 17%

10%

18%

Sample Middle School: ELA Performance by Claim

Above Standard

At or Near Standard

Below Standard

Which ELA claim is highest performing for 8th grade students?

Lowest performing?

18

Conc

epts

& P

roce

dure

s

Prob

lem

Sol

ving

& M

odel

ing/

Data

Ana

lysi

s

Com

mun

icati

ng R

easo

ning

Conc

epts

& P

roce

dure

s

Prob

lem

Sol

ving

& M

odel

ing/

Data

Ana

lysi

s

Com

mun

icati

ng R

easo

ning

Conc

epts

& P

roce

dure

s

Prob

lem

Sol

ving

& M

odel

ing/

Data

Ana

lysi

s

Com

mun

icati

ng R

easo

ning

Grade 6 Grade 7 Grade 8

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

52%42%

38%

52%

40%

21%

53%

35% 38%

33%47%

50%

33%49%

67%

33%

53% 52%

15%10% 12% 15% 11% 11% 14% 11% 11%

Sample Middle School: Math Performance by Claim

Above StandardAt or Near StandardBelow Standard

Which Math claim is highest performing for 6th grade students?

Lowest performing?

19

Connecting SBAC Claim Scores to Targets and Standards

Guiding Questions for Informing Instruction• Do teachers know how to access resources to help them understand targets and standards

that make up each claim?

• Do teachers have sufficient materials and resources aligned to each claim?

• Are learning success criteria articulated for each claim and its targets?

• How can teachers use/adapt local formative assessments to gauge overall learning and improvement in the areas of need (e.g., low performing claims)?

• Do teachers have access to sample assessment items aligned to the targets and standards that make up each claim?

21

Resources for Understanding Targets and Standards for Each Claim

KDS’ SBAC Claims, Targets, Standards Alignment

22

http://www.rcoe.us/educational-services/files/2014/09/SBAC-Claims-Targets-Standard-Alignment-Grade-8-ELA.pdf



KDS’ SBAC Claims, Targets, Standards Alignment

23

http://www.rcoe.us/educational-services/files/2014/09/SBAC-Claims-Targets-Standard-Alignment-Grade-7-Math.pdf



Claim 1: Reading• Literacy instruction that centers on careful examination of texts

• Reading closely and drawing evidence from the text to support inferences and judgments made

• Variety of text types (literary and informational)

• Students should be able to• Cite the textual evidence that most strongly supports an analysis of what the text says

explicitly, as well as inferences drawn from the text.

• Determine a theme or central idea of the text and analyze its development over the course of the text. Provide an objective summary of the plot.

• Determine the meaning of words and phrases in a text, including figurative and connotative meanings. Analyze the meaning of certain word choices on meaning and tone.

• Analyze how differences in the points of view of the characters and the audience create such effects as suspense or humor.

• Compare and contrast the structure of two texts and analyze how their differing structures contribute to meaning and style.

• Distinguish among fact, opinion and reasoned judgement in a text based on research findings and speculation in the text.

• Describe how a text presents information (e.g., comparatively, sequentially, causally).

25

Threshold Achievement Level Descriptors –Grade 8 ELA

http://www.smarterbalanced.org/wordpress/wp-content/uploads/2012/11/Smarter-Balanced-ELA-Literacy-ALDs.pdf



26

Sample Items Aligned to Targets

27

Sample Item: MC – Grade 6 ELA

“Summer on Wheels” by Gary Soto

Claim 1: Students can read closely and analytically to comprehend a range of increasingly complex literary and informational texts.

Question 1: Bentley feels hurt and upset after falling off his bike. Which of the following sentences from the passage best supports this statement?

A. Bentley sat at the kitchen table running an ice cube back and forth across the knot on his forehead. B. The ice cube glided across smooth skin before it jumped up and over the knot. C. Bentley whimpered like the puppy he was. D. And the air left his lungs when he belly flopped

Target 1: KEY DETAILS: Given an inference or conclusion, use explicit details and Implicit information from the text to support the inference or conclusion provided.

28



Target 3: WORD MEANINGS: Determine intended or precise meanings of words, including academic/tier 2 words,domain‐specific (tier 3) words, and words with multiple meanings, based on context, word relationships (e.g., synonyms), word structure (e.g., common Greek or Latin roots, affixes), or use of resources (e.g., dictionary, glossary), with primary focus on the academic vocabulary common to complex texts in all disciplines

29

Sample Item: MS – Grade 8 ELA


Target 7: LANGUAGE USE: Interpret figurative language,literary devices, or connotativemeanings of words and phrases used in context andthe impact of those wordchoices on meaning or tone

30

Sample Item: HT – Grade 8 ELA


Target 3: WORD MEANINGS: Determine intended or precise meanings of words, including academic/tier 2 words,domain‐specific (tier 3) words, and words with multiple meanings, based on context, word relationships (e.g., synonyms), word structure (e.g., common Greek or Latin roots, affixes), or use of resources (e.g., dictionary, glossary), with primary focus on the academic vocabulary common to complex texts in all disciplines

31

Sample Item: EBSR – Grade 8 ELA“Ansel Adams, Painting with Light” by Melanie G. Snyder


Target 11. REASONING & EVALUATION: Apply reasoning and a range of textual evidence to justify analyses of author’s presentation of information (author’s line of reasoning, point of view/purpose; relevance of evidence or elaboration to support claims; development or connections among complex concepts, ideas).

Sample Item: ST/CR – Grade 6 ELA

32

“Planes on the Brain”Read the article “Planes on the Brain” by Elisabeth Deffner, from Faces Magazine.


Question 1: How does the author emphasize the point that the TAM program was a positive influence on the sisters’ lives? Use details from the text to support your answer.

Target 1: KEY DETAILS: Given an inference or conclusion, use explicit details andImplicit information from the text to support the inference or conclusion provided.

Target 13: TEXT STRUCTURES & FEATURES: Relate knowledge of text structures or genre-specific features to analyze or integrate information

http://sampleitems.smarterbalanced.org/itempreview/sbac/ELA.htm



Sample Item: ST/CR – Grade 6 ELA

33

“Planes on the Brain”Read the article “Planes on the Brain” by Elisabeth Deffner, from Faces Magazine.


Question 3: What does the author mean by “the sky is no longer the limit”? How does the meaning apply to the Anyadike sisters? Use details from the text to support your response.

Target 11. REASONING & EVALUATION: Apply reasoning and a range of textual evidence to justify analyses of author’s presentation of information (author’s line of reasoning, point of view/purpose; relevance of evidence or elaboration to support claims; development or connections among complex concepts, ideas).




34

Sample Item: WR/CR – Grade 7 ELAClaim 2: Students can produce effective and well-grounded writing for a range of purposes and audiences

Target 6. WRITE/REVISE BRIEF TEXTS: Apply a variety of strategies when writing or revising one or more paragraphs of text that express arguments about topics or sources: establishing and supporting a claim, organizing and citing supporting evidence using credible sources, providing appropriate transitional strategies for coherence, appropriate vocabulary, or providing a conclusion appropriate to purpose and audience.

Cell Phones in School—Yes or No? Some of the reasons to support cell phones in school are as follows:• Students can take pictures of class projects to e-mail or show to parents.• Students can text-message missed assignments to friends that are absent.• Many cell phones have calculators or Internet access that could be used for assignments.• If students are slow to copy notes from the board, they can take pictures of the missed notes and

view them later.• During study halls, students can listen to music through cell phones.• Parents can get in touch with their children and know where they are at all times.• Students can contact parents in case of emergencies. Some of the reasons to forbid cell phones in school are as follows:• Students might send test answers to friends or use the Internet to cheat during an exam.• Students might record teachers or other students without their knowledge. No one wants to be

recorded without giving consent.• Cell phones can interrupt classroom activities.• Cell phones can be used to text during class as a way of passing notes and wasting time. Based on what you read in the text, do you think cell phones should be allowed in schools? Using the lists provided in the text, write a paragraph arguing why your position is more reasonable than the opposing position.

35

Sample Item: EBSR – Grade 11 ELA

Claim 3: Students can employ effective speaking and listening skills for a rangeof purposes and audiences

Target 4: LISTEN/INTERPRET:Analyze, interpret, and useInformation delivered orally

36


Claim 4: Students can engagein research/inquiry to investigate topics, and toanalyze, integrate, and presentinformation

Target 3: EVALUATE INFORMATION/SOURCES: Use reasoning, evaluation,and evidence to assess credibility and accuracy of each source in order to gather and select information to support analysis, reflection, and research.

37

Sample Item: MA – Grade 11 ELA

Claim 4: Students can engage in research/inquiry to investigate topics, and toanalyze, integrate, and present information.

Target 4: USE EVIDENCE:Cite evidence to support analyses, arguments, or critiques. Support claims with logical reasoning and relevant evidence.

38

Sample Item: DD - Grade 7 Math

Claim 1: CONCEPTS & PROCEDURES: Students canexplain and apply mathematical concepts and carry outmathematical procedures withprecision and fluency.

Target 1B: Apply and extend previous understandings of operations with fractions to add, subtract, multiply, and divide rational numbers..

39

Sample Item: Grade 8 Math (Drag and Drop)

Claim 2: Problem Solving: Students can solve a range of well‐posed problems in pure and applied mathematics, makingproductive use of knowledge andproblem‐solving strategies

Target 2A: Apply mathematicsto solve well‐posed problemsin pure mathematics and arisingIn everyday life, society, and theworkplace.

Target 2B: Select and use appropriate tools strategically.

40

Sample Item: ST Grade 11 MathClaim 3: Communicating Reasoning: Studentsclearly and precisely construct viable argumentsto support their own reasoning and to critiquethe reasoning of others.

Target 3A: Test propositions or conjectureswith specific examples.

Target 3B: Construct, autonomously, chains of reasoning that will justify or refute propositionsor conjectures.

41

Sample Item: ST– Grade 11 Math

Claim 4: Modeling and Data Analysis: Students can analyze complex, real‐world scenarios and can construct and use mathematical models to interpret and solve problems.

Target 4A: Apply mathematics to solve problems arising in everyday life, society, and the workplace. Target 4B: Construct, autonomously, chains of reasoning to justify mathematical models used, Interpretations made, and solutions proposed for a complex problem. Target 4D: Interpret results in the context of a situation. Target 4F: Identify important quantities in a practical situation and map their relationships (e.g., using diagrams, two‐way tables, graphs, flowcharts, or formulas).

42

[email protected]

[email protected]

mailto:[email protected]

mailto:[email protected]

reviewing sbac summative assessment results to guide instructional practice shannon wells ekaterina...

Documents