equip rubric & quality review mathematics training module: grades k-5 1

1

EQuIP Rubric & Quality Review Mathematics Training Module:

Grades K-5

2

Use the EQuIP quality review process to determine the quality and alignment of lessons and units to the Common Core State Standards (CCSS) in mathematics.

During this session, reviewers will:1. Develop a common understanding of the EQuIP quality review

process 2. Develop a common understanding of the EQuIP Rubric including

its criteria and rating scale 3. Practice using the EQuIP quality review process and rubric to

evaluate and provide feedback on CCSS-aligned instructional materials

Session Goals

1. Common Core: Before beginning a review, all members of the panel are familiar with the Common Core Standards (CCSS) for their grade band and discipline(s).

2. Inquiry: Review processes emphasize inquiry rather than advocacy and are organized in steps around a set of guiding questions.

3. Respect & Commitment: Each member of the panel is respected as a valued colleague and contributor who makes a commitment to the EQuIP process.

4. Criteria & Evidence: All observations, judgments, discussions, and recommendations are criterion- and evidence-based.

5. Constructive: Lessons/units to be reviewed are seen as “works in progress.” Panel members are respectful of contributors’ work and make constructive observations and suggestions based on evidence from the work.

6. Individual to Collective: Each member of the panel independently records his/her observations prior to discussion. Discussions focus on understanding all panel members’ interpretations of the criteria and the evidence they have found.

7. Understanding & Agreement: The goal of the process is to compare and calibrate our judgments so that we reach agreement about alignment and quality with respect to the CCSS.

EQuIP Quality Review ProcessPrinciples & Agreements

4

EQuIP Quality Review ProcessThe EQuIP quality review process is a collegial process that centers on the use of criteria-based rubrics for English language arts (ELA)/literacy and mathematics. The criteria are organized into four dimensions:

EQuIP Quality Review ProcessRubric Dimensions

Alignment to the depth of the CCSS

Key shifts in the CCSS

Instructional supports

Assessment

As educators examine instructional materials against the criteria in each dimension, they are able to use common standards for quality and generate evidence-based commentary and ratings on the quality and alignment of materials.

5

Two Version of the Quality Review RubricThe Electronic VersionFor each dimension:

• Select the checkbox for each criterion for which clear and substantial evidence is found.

• Make observations and suggestions related to criteria and evidence.

• Determine a rating for each dimension based on checked criteria and observations.

For Dimension I:• Use alignment rating to

determine whether to proceed with review.

6

Two Versions of the Quality Review Rubric The One-Page Version

7

EQuIP Quality Review ProcessThe Five Steps

Step 1. Review Materials

Step 2. Apply Criteria in Dimension I

Step 3. Apply Criteria in Dimensions II – IV

Step 4. Apply Overall Rating and Provide Summary Comments

Step 5. Compare Overall Ratings and Determine Next Steps

8

EQuIP Quality Review ProcessThe Five StepsStep 1. Review Materials • Record the grade and title of the lesson/unit on the Quality Review Rubric PDF• Scan to see what the lesson/unit contains and how it is organized • Read key materials related to instruction, assessment and teacher guidance• Study and work the tasks that serve as the centerpiece for the lesson/unit,

analyzing the content and mathematical practices the tasks require Step 2. Apply Criteria in Dimension I: Alignment to the Depth of the CCSS• Identify the grade-level CCSS that the lesson/unit targets• Closely examine the materials through the “lens” of each criterion• Indicate each criterion for which clear and substantial evidence is found • Record input on specific improvements needed to meet criteria or strengthen

alignment• Enter a rating of 0–3 for Dimension I

Note: Dimension I is non-negotiable. For the review to continue, a rating of 2 or 3 is required. If the review is discontinued, consider giving general feedback that might help developers/teachers make decisions regarding next steps.

9

EQuIP Quality Review ProcessThe Five StepsStep 3. Apply Criteria in Dimensions II–IV • Examine the lesson/unit through the “lens” of each criterion • Indicate each criterion met, record observations and feedback, and then rate 0–3

Step 4. Apply an Overall Rating and Provide Summary Comments • Individually review ratings for Dimensions I–IV, adding/clarifying comments as

needed• Total dimension ratings and record an overall rating (E, E/I, R, N) based on total

score• Individually write summary comments for the overall rating on the Quality Review

Rubric PDF

Step 5. Compare Overall Ratings and Determine Next Steps • Note the evidence cited to arrive at final ratings, summary comments, and

similarities and differences among raters. Recommend next steps for the lesson/unit and provide recommendations for improvement to developers/teachers.

10

EQuIP Quality Review ProcessThe Flowchart

Discussion and collaboration must occur after Dimension I and then again either for all dimensions after Dimension IV or …

… separately after each dimension and …

… always during the overall rating process and summary comments.

11

First Grade – Exploring Two-Digit NumbersRead key materials related to instruction, assessment, and teacher guidance:

EQuIP Quality Review ProcessStep 1: Review Example Materials

Review materials on your own to make sure you know what they contain and how they are organized. Work key tasks and study activities with the grade-level student strategies in mind.

Lesson Structure:• Goals• Content

Standards• Practices• Prior Knowledge• Vocabulary• Materials

Task Components – • Engage• Explore• Explain• Elaborate• Evaluation• Differentiation

12

EQuIP Quality Review ProcessStep 2: Apply the Criteria of Dimension I

The lesson/unit aligns with the letter and spirit of the CCSS:

Targets a set of grade-level CCSS mathematics standard(s) to the full depth of the standards for teaching and learning.

Standards for Mathematical Practice that are central to the lesson are identified, handled in a grade-appropriate way and well connected to the content being addressed.

Presents a balance of mathematical procedures and deeper conceptual understanding inherent in the CCSS.

Determine rationales for checks or no-checks individually for this dimension and then discuss and collaborate with your review team.

13

The lesson/unit aligns with the letter and spirit of the CCSS:

Targets a set of grade-level CCSS mathematics standard(s) to the full depth of the standards for teaching and learning.

Standards for Mathematical Practice that are central to the lesson are identified, handled in a grade-appropriate way and well connected to the content being addressed.

Presents a balance of mathematical procedures and deeper conceptual understanding inherent in the CCSS.

Step 2: Apply Criteria of Dimension IExample Materials – Checked Criteria

14

1. Targets a set of grade-level CCSS mathematics standard(s) to the full depth of the standards for teaching and learning.

Observations and suggestions:

Step 2: Apply Criteria of Dimension I, Criterion 1Example Materials – Rationale for Checks

The focus of the lessons in this unit is on 2-digit numbers and none of the lessons call for students to extend their understandings to 120 (1.NBT.1). While there are extensions that ask advanced students to represent and explain place value concepts for numbers higher than 100 (Lesson 2.1), and to work with the numbers 101-200 on hundred boards (Lesson 2.8), there is no expectation for all students to be able to represent numbers to 120. In addition, while 1.OA.6 is cited as a target on page 5, no lessons within this unit address addition and subtraction within 20 or fluency for addition and subtraction within 10. Since the focus of this unit is on two-digit numbers, perhaps 1.NBT.1 should be excluded from the target standards listed in the introduction on page 5; and since addition and subtraction within 20 is not clearly addressed, consideration should be given to either developing a lesson that addresses this standard (1.OA.6) or eliminating it from the list on page 5. In most lessons the specific targeted standards listed in the Overview and Background Information section are not clearly addressed in the activities that follow. For example in Lesson 1.2, 1.NBT.3 is listed in the Overview. However in the activities of the lesson there is no comparison of numbers required.

15

2. Standards for Mathematical Practice that are central to the lesson are identified, handled in a grade-appropriate way and well connected to the content being addressed.



This criterion is partially addressed in the unit. All eight Standards for Mathematical Practice are cited in the introduction as being addressed within this 10-lesson unit. Then each lesson cites a somewhat shorter list of specific Practices targeted, with from three to six targeted in each lesson, with no lessons citing MP.1 and MP.8. This more specific targeting is a good idea but most teachers would find it challenging to focus on six Practices within one lesson. For example in Lesson 2.1 MP.2, 3, 4, 5, 6, and 7 are all cited as “emphasized” in the approximately one-hour lesson. It is likely that some of these are less central to the goals of the lesson than others. Developers should consider whether the lesson-level lists of Practices could be honed even further and whether listing all eight of the Practices in the introduction is at all helpful for teachers. Focusing only on those Practices that are CENTRAL to the set of lessons is ideal. To improve this unit, and the lessons within it, clear references and connections to the Standards for Mathematical Practices should be noted within activities. Such explicit references will help teachers understand when and how to emphasize various Practices.

16

3. Presents a balance of mathematical procedures and deeper conceptual understanding inherent in the CCSS.



The ten lessons comprising this unit do a good job of building the procedural skills and the conceptual understanding of students with respect to 2-digit numbers and place value. Conceptual understanding is built using a variety of tools, including ten-frame mats, counters, snap cubes, and hundred boards. Students are expected to be able to represent numbers in a variety of ways and to compare numbers based on their understanding of the tens and ones digits. The lessons include questions throughout that will guide teachers in gauging both the procedural and conceptual understandings of students.

17

Rating Scale for Dimensions I–IV: 3: Meets most to all of the criteria in the dimension

2: Meets many of the criteria in the dimension

1: Meets some of the criteria in the dimension

0: Does not meet the criteria in the dimension

EQuIP Quality Review ProcessUsing Dimension Ratings and Rating Scales

18





Descriptors for Dimensions I–IV: 3: Exemplifies CCSS Quality — meets the standard described by criteria in the dimension, as explained in criterion-based observations

2: Approaching CCSS Quality — meets many criteria but will benefit from revision in others, as suggested in criterion-based observations

1: Developing toward CCSS Quality — needs significant revision, as suggested in criterion-based observations

0: Not representing CCSS Quality — does not address the criteria in the dimension


19





Descriptors for Dimensions I–IV: 3: Exemplifies CCSS Quality — meets the standard described by criteria in the dimension, as explained in criterion-based observations

2: Approaching CCSS Quality — meets many criteria but will benefit from revision in others, as suggested in criterion-based observations

1: Developing toward CCSS Quality — needs significant revision, as suggested in criterion-based observations

0: Not representing CCSS Quality — does not address the criteria in the dimension


20

Compare Criterion-Based Checks, Feedback, and Rating• What is the pattern within our team in terms of the criteria we

have checked?

• Do our observations and feedback reference the criteria and evidence (or lack of evidence) in the instructional materials?

• Do our ratings correspond to the rating and descriptors in the rubric?

Step 2: Apply the Criteria of Dimension IExample Materials – Discuss and Collaborate

Determine and discuss your individual dimension ratings with your review team.

21

Rating: 2 Approaching CCSS Quality: Meets many criteria but will benefit from revision in others, as suggested in criterion-based observations.

Rationale: Even though the criteria are not fully addressed for this dimension, there is sufficient evidence in the unit of both content and Practice standards being targeted. In both areas it is a case of too many standards listed rather than too few. The partial alignment to the criteria for both of the first two criteria and the overall promise of this unit warrants a 2-rating and continuing with the review.

Step 2: Apply the Criteria of Dimension I Example Materials – Rating

EQuIP Quality Review ProcessConstructive Feedback

Effective feedback is vital to the EQuIP Quality Review Process. • Criterion-Based: Written comments are based on the criteria

used for review in each dimension. No extraneous comments are included.

• Evidence Cited: Written comments indicate that the reviewer looked for evidence of each criterion of a given dimension. Examples cite where and how the criteria are met or not met.

• Improvement Suggested: Improvements are specifically identified to meet criteria or strengthen the lesson or unit.

• Clear Communication: Written comment are constructed in a manner keeping with basic grammar, spelling, sentence structure and conventions.

Use these points to determine and discuss the observations and suggestions appropriate for this dimension with your review team.

23

Step 2: Apply the Criteria of Dimension IExample Materials – Providing Feedback

Observations/Feedback The focus of the lessons in this unit is on 2-digit numbers and none of the lessons call for students to extend their understandings to 120 (1.NBT.1). While there are extensions that ask advanced students to represent and explain place value concepts for numbers higher than 100 (Lesson 2.1), and to work with the numbers 101-200 on hundred boards (Lesson 2.8), there is no expectation for all students to be able to represent numbers to 120. In addition, while 1.OA.6 is cited as a target on page 5, no lessons within this unit address addition and subtraction within 20 or fluency for addition and subtraction within 10. Since the focus of this unit is on two-digit numbers, perhaps 1.NBT.1 should be excluded from the target standards listed in the introduction on page 5; and since addition and subtraction within 20 is not clearly addressed, consideration should be given to either developing a lesson that addresses this standard or eliminating it from the list on page 5. In most lessons the specific targeted standards listed in the Overview and Background Information section are not clearly addressed in the activities that follow. For example in Lesson 1.2, 1.NBT.3 is listed in the Overview. However in the activities of the lesson there is no comparison of numbers required. All eight Standards for Mathematical Practice are cited in the introduction as being addressed within this 10-lesson unit. Then each lesson cites a somewhat shorter list of specific Mathematical Practices targeted, with from three to six Practices targeted in each lesson. This more specific targeting is a good idea but most teachers would find it challenging to focus on six Practices within one lesson. For example in Lesson 2.1 MP.2, 3, 4, 5, 6, and 7 are all cited as “emphasized” in the approximately one-hour lesson. It is likely that some of these are less central to the goals of the lesson than others. Developers should consider whether the lesson-level lists of Practices could be honed even further and whether listing all eight of the Practices in the introduction is at all helpful for teachers. Focusing only on those Practices that are CENTRAL to the set of lessons is ideal. To improve this unit, and the lessons within it, clear references and connections to the Mathematical Practices should be noted within activities. Such explicit references will help teachers understand when and how to emphasize various Practices. The ten lessons comprising this unit do a good job of building the procedural skills and the conceptual understanding of students with respect to 2-digit numbers and place value. Conceptual understanding is built using a variety of tools, including ten-frame mats, counters, snap cubes, and hundred boards. Students are expected to be able to represent numbers in a variety of ways and to compare numbers based on their understanding of the tens and ones digits. The lessons include questions throughout that can guide teachers in gauging both the procedural and conceptual understandings of students.

24

Step 2: Apply the Criteria of Dimension IExample Materials – Providing Feedback

Observations/Feedback The focus of the lessons in this unit is on 2-digit numbers and none of the lessons call for students to extend their understandings to 120 (1.NBT.1). While there are extensions that ask advanced students to represent and explain place value concepts for numbers higher than 100 (Lesson 2.1), and to work with the numbers 101-200 on hundred boards (Lesson 2.8), there is no expectation for all students to be able to represent numbers to 120. In addition, while 1.OA.6 is cited as a target on page 5, no lessons within this unit address addition and subtraction within 20 or fluency for addition and subtraction within 10. Since the focus of this unit is on two-digit numbers, perhaps 1.NBT.1 should be excluded from the target standards listed in the introduction on page 5; and since addition and subtraction within 20 is not clearly addressed, consideration should be given to either developing a lesson that addresses this standard or eliminating it from the list on page 5. In most lessons the specific targeted standards listed in the Overview and Background Information section are not clearly addressed in the activities that follow. For example in Lesson 1.2, 1.NBT.3 is listed in the Overview. However in the activities of the lesson there is no comparison of numbers required. All eight Standards for Mathematical Practice are cited in the introduction as being addressed within this 10-lesson unit. Then each lesson cites a somewhat shorter list of specific Mathematical Practices targeted, with from three to six Practices targeted in each lesson. This more specific targeting is a good idea but most teachers would find it challenging to focus on six Practices within one lesson. For example in Lesson 2.1 MP.2, 3, 4, 5, 6, and 7 are all cited as “emphasized” in the approximately one-hour lesson. It is likely that some of these are less central to the goals of the lesson than others. Developers should consider whether the lesson-level lists of Practices could be honed even further and whether listing all eight of the Practices in the introduction is at all helpful for teachers. Focusing only on those Practices that are CENTRAL to the set of lessons is ideal. To improve this unit, and the lessons within it, clear references and connections to the Mathematical Practices should be noted within activities. Such explicit references will help teachers understand when and how to emphasize various Practices. The ten lessons comprising this unit do a good job of building the procedural skills and the conceptual understanding of students with respect to 2-digit numbers and place value. Conceptual understanding is built using a variety of tools, including ten-frame mats, counters, snap cubes, and hundred boards. Students are expected to be able to represent numbers in a variety of ways and to compare numbers based on their understanding of the tens and ones digits. The lessons include questions throughout that can guide teachers in gauging both the procedural and conceptual understandings of students.

25

Consider the rating for Dimension I:

• Is the overall rating for alignment a 3 or 2?

• Does the quality of the alignment to the CCSS warrant continuing with the review?

If yes, continue with Step 3 for Dimensions II – IV.

EQuIP Quality Review ProcessStep 2: Dimension I Reflection

If we agree that the materials warrant a 2- or 3-rating for Dimension I, we continue the review, applying the criteria of Dimensions II – IV.

26

EQuIP Quality Review ProcessStep 3: Apply the Criteria of Dimension II

The lesson/unit addresses key shifts in the CCSS:

Focus: Lessons and units targeting the major work of the grade provide an especially in-depth treatment, with especially high expectations. Lessons and units targeting supporting clusters have visible connection to the major work of the grade and are sufficiently brief. Lessons and units do not hold students responsible for material from later grades.

Coherence: The content develops through reasoning about the new concepts on the basis of previous understandings and provides opportunities for students to transfer knowledge and skills within and across domains and learning progressions.

27

Rigor: Requires students to engage with and demonstrate challenging mathematics with appropriate balance among the following:

• Application: Provides opportunities for students to independently apply mathematical concepts in real-world situations and problem solve with persistence, choosing and applying an appropriate model or strategy to new situations.

• Conceptual Understanding: Develops students’ conceptual understanding through tasks, brief problems, questions, multiple representations and opportunities for students to write and speak about their understanding.

• Procedural Skill and Fluency: Expects, supports and provides guidelines for procedural skill and fluency with core calculations and mathematical procedures (when called for in the standards for the grade) to be performed quickly and accurately.

EQuIP Quality Review ProcessStep 3: Apply the Criteria of Dimension II


28

Step 3: Apply Criteria of Dimension II Example Materials – Checked CriteriaThe lesson/unit addresses reflects evidence of key shifts that are reflected in the CCSS: Focus: Lessons and units targeting the major work of the grade provide an especially in-depth treatment, with

especially high expectations. Lessons and units targeting supporting clusters have visible connection to the major work of the grade and are sufficiently brief. Lessons and units do not hold students responsible for material from later grades.

Coherence: The content develops through reasoning about the new concepts on the basis of previous understandings and provides opportunities for students to transfer knowledge and skills within and across domains and learning progressions.

Rigor: Requires students to engage with and demonstrate challenging mathematics with appropriate balance among the following:o Application: Provides opportunities for students to independently apply mathematical concepts in real-

world situations and problem solve with persistence, choosing and applying an appropriate model or strategy to new situations.

o Conceptual Understanding: Develops students’ conceptual understanding through tasks, brief problems, questions, multiple representations and opportunities for students to write and speak about their understanding.

o Procedural Skill and Fluency: Expects, supports and provides guidelines for procedural skill and fluency with core calculations and mathematical procedures (when called for in the standards for the grade) to be performed quickly and accurately.

29

The lesson/unit addresses reflects evidence of key shifts that are reflected in the CCSS: Focus: One of the four critical areas cited in the CCSS for first grade students is

developing understanding of whole number relationships and place value, including grouping in tens and ones, to 100. This 10-lesson unit focuses on this major work of the grade, with respect to 2-digit numbers. At the unit level, reconsidering the inclusion of 1.NBT.1, which extends counting, reading and writing numerals, and number representation to 120, and 1.OA.6, which addresses addition and subtraction within 20 and fluency within 10, could sharpen the focus.

Step 3: Apply Criteria of Dimension II Example Materials – Rationale for Checks

1. Criterion-based? 2. Evidence Cited? 3. Improvement Suggested? 4. Clear Communication?

30

The lesson/unit addresses reflects evidence of key shifts that are reflected in the CCSS: Coherence: The lesson Overview and Background Information for each lesson indicate

the prior knowledge that students will be expected to draw upon. For example, Lesson 1.2 cites prior knowledge needed as knowing number names and the count sequence and counting up to 19 objects---which are defined as expectations in the CCSS for Kindergarten. Each lesson within this unit also begins with an activity to engage students, building upon their prior knowledge. As Lesson 1.2 builds to work with larger numbers, it reinforces the students’ understanding that the teen numbers are composed of one ten and some leftovers—a CCSS for Kindergarten but one that is important as a strong foundation for first grade. Coherence within first grade is also evident since the first half of the year focuses on numbers 0-50, with the second half on 0-100, thereby providing students with an opportunity to build on previous understandings.



31

Rigor: o Applicationo Conceptual Understandingo Procedural Skill and Fluency


The intent of this 10-lesson unit is to build conceptual understanding of place value with two digit numbers and a level of procedural skill so that students can represent 2-digit numbers in a variety of ways and also compare 2-digit numbers based on their understanding of place value. It is not the intent of this unit to engage students in real-world problem solving so application is not an element of rigor that should heavily weighted in this instance. Students are given opportunities within the lessons to write or speak about their understandings. There are numerous references to student use of journals, and suggested questions probe students to do such things as identify and explain their strategies and explain how they know which number is larger/smaller. Lessons include both procedural and conceptual understanding. Ten frames and snapping cubes are used to build conceptual understanding, while partner activities build procedural understanding and fluency. The unit could be improved if other manipulatives and visual representations were used rather than using ten frames so frequently.


32

Compare Criterion-Based Checks, Observations/Feedback and Rating

• What is the pattern within our team in terms of the criteria we have checked?



Step 3: Apply Criteria of Dimension II Example Materials – Discuss and Collaborate

Determine and discuss your individual dimension ratings with your review team…

33

Rating: 3 Exemplifies CCSS Quality – meets the standard described by criteria in the dimension, as explained in criterion-based observations.

Rationale: Attention to Focus, Coherence, and Rigor, the key shifts in the CCSS, is a strength of this unit.

Step 3: Apply Criteria of Dimension II Example Materials – Rating

34

EQuIP Quality Review ProcessStep 3: Apply the Criteria of Dimension III

The lesson/unit is responsive to varied student learning needs:

Includes clear and sufficient guidance to support teaching and learning of the targeted standards, including, when appropriate, the use of technology and media.

Uses and encourages precise and accurate mathematics, academic language, terminology, and concrete or abstract representations (e.g., pictures, symbols, expressions, equations, graphics, models) in the discipline.

Engages students in productive struggle through relevant, thought-provoking questions, problems and tasks that stimulate interest and elicit mathematical thinking.

Addresses instructional expectations and is easy to understand and use.

35

EQuIP Quality Review ProcessStep 3: Apply the Criteria of Dimension III Provides appropriate level and type of scaffolding, differentiation, intervention

and support for a broad range of learners. *

Supports diverse cultural and linguistic backgrounds, interests and styles.

Provides extra supports for students working below grade level.

Provides extensions for students with high interest or working above grade level.

* Note: All three of these components are required in a high quality lesson or unit.

36


A unit or longer lesson should: longer lessons:

Recommends and facilitates a mix of instructional approaches for a variety of learners such as using multiple representations (including models), using a range of questions, checking for understanding, flexible grouping, pair-share, etc.

Gradually removes supports, requiring students to demonstrate their mathematical understanding independently.

Demonstrates an effective sequence and a progression of learning where the concepts or skills advance and deepen over time.

Expects, supports and provides guidelines for procedural skill and fluency with core calculations and mathematical procedures (when called for in the standards for the grade) to be performed quickly and accurately.

37

The lesson/unit is responsive to varied student learning needs: Includes clear and sufficient guidance to support teaching and learning of the targeted standards, including,

when appropriate, the use of technology and media. Uses and encourages precise and accurate mathematics, academic language, terminology, and concrete or

abstract representations (e.g., pictures, symbols, expressions, equations, graphics, models) in the discipline. Engages students in productive struggle through relevant, thought-provoking questions, problems and tasks

that stimulate interest and elicit mathematical thinking. Addresses instructional expectations and is easy to understand and use. Provides appropriate level and type of scaffolding, differentiation, intervention and support for a broad

range of learners. Supports diverse cultural and linguistic backgrounds, interests and styles. Provides extra supports for students working below grade level. Provides extensions for students with high interest or working above grade level.

A longer unit or lesson should: Recommends and facilitates a mix of instructional approaches for a variety of learners such as using

multiple representations (including models), using a range of questions, checking for understanding, flexible grouping, pair-share, etc.





38

Step 3: Apply Criteria of Dimension III Example Materials – Sharing FeedbackINDIVIDUALLY:• Closely examine the materials through the “lens” of each criterion• Check each criterion for which clear and substantial evidence is found • Record evidence for each check or where you looked and were unable to

find evidence • Write feedback using the four qualities for ONE of the criterion that you

checked/did not checkedCOLLECTIVELY:• Compare and discuss checks and evidence• What is the pattern within our team in terms of the criteria we have

checked?• Do our observations and feedback reference the criteria and evidence (or

lack of evidence) in the instructional materials?• Choose ONE piece of feedback for the group to share with entire roomDetermine rationales for checks or no-checks individually and then discuss and collaborate with your review team. For this dimension we will pause to share some of our findings with the larger group…

39

The lesson/unit is responsive to varied student learning needs: Includes clear and sufficient guidance to support teaching and learning of the targeted standards, including,

when appropriate, the use of technology and media. Uses and encourages precise and accurate mathematics, academic language, terminology, and concrete or

abstract representations (e.g., pictures, symbols, expressions, equations, graphics, models) in the discipline. Engages students in productive struggle through relevant, thought-provoking questions, problems and tasks

that stimulate interest and elicit mathematical thinking. Addresses instructional expectations and is easy to understand and use. Provides appropriate level and type of scaffolding, differentiation, intervention and support for a broad

range of learners. Supports diverse cultural and linguistic backgrounds, interests and styles. Provides extra supports for students working below grade level. Provides extensions for students with high interest or working above grade level.

A longer unit or lesson should: Recommends and facilitates a mix of instructional approaches for a variety of learners such as using

multiple representations (including models), using a range of questions, checking for understanding, flexible grouping, pair-share, etc.




Step 3: Apply Criteria of Dimension III Example Materials – Checked Criteria

40

The lesson/unit is responsive to varied student learning needs: Includes clear and sufficient guidance to support teaching and learning of the

targeted standards, including, when appropriate, the use of technology and media.

The lessons within this unit provide some guidance to support teaching and learning, including some suggested links to online resources. However, more explicit guidance is needed in the unit to support teachers and learners. The lessons offer suggestions for questions teachers might use to probe student thinking but do not always include sample student responses. These would be helpful in some cases to guide teachers and their instruction. For example, the Explain section of Lesson 2.6 tells teachers—after the Get 100! game has been played, discuss the game and various strategies with students, asking them what transpired during the game and what strategies they used. Sample student responses would be helpful, particularly for novice teachers. In some cases, there are references to materials (e.g., the “Greater, Less, Equal Cover Up” game board in Lesson 2.5), and those materials do not appear as attachments to the lesson. On page 31, there were no directions for putting together "I Have/Who Has" game cards and clear and sufficient guidance was not provided for the arrow cards activity. Complete sets of materials, or information about where to get the required materials, would be useful supports for teachers.

Step 3: Apply Criteria of Dimension III Example Materials – Rationale for Checks

41

Step 3: Apply Criteria of Dimension III Example Materials – Rationale for Checks Uses and encourages precise and accurate mathematics, academic language,

terminology, and concrete or abstract representations (e.g., pictures, symbols, expressions, equations, graphics, models) in the discipline.

The lessons within this unit encourage precise and accurate language, terminology, and representations for both teachers and students. Each lesson overview notes specific vocabulary terms that are important in the lesson. Representations are key elements in many of the lessons, with students using ten-frame maps, hundred boards, snap cubes, and number sentences to represent 2-digit numbers.

Engages students in productive struggle through relevant, thought-provoking questions, problems and tasks that stimulate interest and elicit mathematical thinking.

The questions offered to stimulate discussion, and many of the activities, appear to be ones that could be thought provoking and elicit mathematical thinking, if handled appropriately in the classroom. However more guidance is needed for teachers to know how to guide discussion and to engage all students in the productive struggle the the questions may elicit. While there are no real-world application problems in the unit, there are numerous activities and games that should prove to be interesting for students. However with no clear call for students to work independently, there is way for teachers to judge the individual struggle of students in the class discussions, when playing the games, or when working with a small group.

42

Step 3: Apply Criteria of Dimension III Example Materials – Rationale for Checks Addresses instructional expectations and is easy to understand and use.The format, flow, and most directions for instruction in the unit are clear and understandable but there are elements that could be improved upon to make the lessons easier to understand and use. Materials lists are sometimes incomplete or unclear. For example, the activities for Lesson 2.2 call for ten-frame cards and counters, while the Materials list only asks for number cards and 10-sided dice; and Lesson 2.7 depends upon “Arrow Cards,” which are not provided or adequately described. In some cases, activity sheets are referenced but not provided, and in one case (Lesson 1.1) the spinner referenced in the lesson (with words number, 1 more, and 1 less) does not match up with the one attached to the lesson (with words 1 more, 2 more, and 1 less).

Provides appropriate level and type of scaffolding, differentiation, intervention and support for a broad range of learners.

Each lesson plan contains a section called Plans for Individual Differences. This attempted attention to diverse learners is commendable but the level of detail provided to teachers often is not sufficient. No suggestions are given for working with students from diverse cultural or linguistic backgrounds. The interventions seem to generally call for using smaller numbers or smaller student groupings with struggling students. Suggestions for more adept students usually include using bigger numbers, which often means just moving them ahead to the next lesson. More detail and specific suggestions for addressing individual differences would be helpful to all teachers, but especially to inexperienced teachers.

43

A unit or longer lesson should: Recommends and facilitates a mix of instructional approaches for a variety of learners

such as using multiple representations (including models), using a range of questions, checking for understanding, flexible grouping, pair-share, etc.

Collectively, the lessons provide a mix of instructional approaches; some of these involve various student-grouping strategies including small groups, student pairs, and large group. Other approaches involve using various types of instructional tools and manipulatives—such as ten frame cards, hundred boards, snap cubes---and encouraging students to progress so they can respond with less reliance on manipulatives and tools.


Lessons often transition students from doing larger group to small group or pair work, or vice versa. Sometimes it is not clear if students are being transitioned into individual work, where they demonstrate their mathematical understanding. For example, in the Elaborate section of Lesson 2.7 teachers give students a 2-digit number and ask them to say the number that is ten more. It is stated that this might be used as a summative assessment, but there is no clear indication whether this is intended as an individual activity.


44

A unit or longer lesson should: (cont.) Demonstrates an effective sequence and a progression of learning where the concepts

or skills advance and deepen over time.The sequence of lessons generally demonstrates a progression of learning (e.g., from using ten-frame mats and counters to using hundred boards). The mathematical goals on page 3 explain that while the first half of the year (and this unit) primarily deals with numbers from 0 to 50, students will use numbers from 0 to 100 in the second half of the year.


While procedural skill is expected in the lessons, the need for fluency (i.e. speed of procedural skills) does not appear to be applicable. Accuracy is expected, as in Lesson 2.5 where students are asked to accurately mark an appropriate number. Other procedural skills are addressed and supported in the lessons, such as recognizing and representing numbers using place value, comparing numbers, and quickly counting on or counting back.


45





Step 3: Apply Criteria of Dimension III Example Materials – Discuss and Collaborate

Determine and discuss your individual dimension ratings with your review team…

46

Rating: 1 Developing Toward CCSS Quality: — needs significant revision, as suggested in criterion-based observations

Rationale: While there are several criteria checked, this unit needs significant revision in this dimension to ensure that the content is consistently and effectively used by teachers, and especially for new teachers. More guidance is needed in the form of sample responses to guiding questions; more detailed suggestions are needed for differentiation, intervention, and work with English language learners; more complete attachments and descriptions of resources for the games and activities; and more guidance and attention needs to be paid to individual student work.

Step 3: Apply Criteria of Dimension III Example Materials – Rating

47

EQuIP Quality Review ProcessStep 3: Apply the Criteria of Dimension IV

The lesson/unit regularly assesses whether students are mastering standards-based content and skills: Is designed to elicit direct, observable evidence of the degree to which a student

can independently demonstrate the targeted CCSS. Assesses student proficiency using methods that are accessible and unbiased,

including the use of grade-level language in student prompts. Includes aligned rubrics, answer keys and scoring guidelines that provide

sufficient guidance for interpreting student performance.

In addition, for units and longer lessons: and longer lessons: Uses varied modes of curriculum-embedded assessments that may include pre-,

formative, summative and self-assessment measures.


48

The lesson/unit regularly assesses whether students are mastering standards-based content and skills: Is designed to elicit direct, observable evidence of the degree to which a student

can independently demonstrate the targeted CCSS. Assesses student proficiency using methods that are accessible and unbiased,

including the use of grade-level language in student prompts. Includes aligned rubrics, answer keys and scoring guidelines that provide

sufficient guidance for interpreting student performance.

In addition, for units and longer lessons: Uses varied modes of curriculum-embedded assessments that may include pre-,

formative, summative and self-assessment measures.

Step 3: Apply Criteria of Dimension IV Example Materials – Checked Criteria

49

The lesson/unit regularly assesses whether students are mastering standards-based content and skills: Is designed to elicit direct, observable evidence of the degree to which a student can

independently demonstrate the targeted CCSS.

It is not clear that the assessments described in the lessons are designed to elicit direct, observable evidence of the degree to which a student can independently demonstrate understanding of the concepts required in the targeted CCSS. Questions to give formative feedback while observing students are prevalent in the Evaluation of Student Performance components of the lessons; sometimes it is noted that an activity sheet can be used for summative performance (as in Lesson 1.2) but, even then, it is not clear that this summative activity will provide insight as to whether a student can independently demonstrate understanding. It is also noted on occasion that student math journal work can be collected as a summative assessment (e.g., for Lesson 2.2). Assesses student proficiency using methods that are accessible and unbiased, including

the use of grade-level language in student prompts.With no formal assessment instruments included with the lessons or unit, it is difficult to comment on accessibility or bias. However guiding discussion questions and scripted instructions provided in the lessons are grade-level and appear to be appropriate for first grade students.

Step 3: Apply Criteria of Dimension IV Example Materials – Rationale for Checks

50

Includes aligned rubrics, answer keys and scoring guidelines that provide sufficient guidance for interpreting student performance.

There are no rubrics, answer keys, or scoring guidelines provided to help teachers gauge and interpret student performance.

In addition, for units and longer lessons: Uses varied modes of curriculum-embedded assessments that may include

pre-, formative, summative and self-assessment measures.All the lessons in this unit contain a section on Evaluation of Students, and the Overview of Lessons shows that this section is to describe both formative assessment (i.e., how the teacher can assess student learning during the lesson) and summative assessment (i.e., how the teacher can assess students’ understanding after the lesson). However there is not enough guidance for teachers in this area and there is no mention of student self-assessment or pre-assessment.

Step 3: Apply Criteria of Dimension IV Example Materials – Rationale for Checks

51





Step 3: Apply Criteria of Dimension IV Example Materials– Discuss and Collaborate

Determine and discuss your individual dimension ratings with your review team.

52

Rating: 1 Developing Toward CCSS Quality— needs significant revision, as suggested in criterion-based observations

Rationale: With very little attention paid to assessment, this is the weakest dimension for the unit. Even though only one criterion was checked, reviewers believe there is an assessment kernel to build upon in the unit.

Step 3: Apply Criteria of Dimension IV Example Materials– Rating

53

EQuIP Quality Review ProcessStep 4: Overall Rating Descriptors

Overall Rating for the Lesson/Unit:

E: Exemplar — Aligned and meets most to all of the criteria in Dimensions II–IV (total 11–12)

E/I: Exemplar if Improved — Aligned and needs some improvement in one or more dimensions (total 8–10)

R: Revision Needed — Aligned partially and needs significant revision in one or more dimensions (total 3–7)

N: Not Ready to Review — Not aligned and does not meet criteria (total 0–2)

54







55







56

EQuIP Quality Review ProcessStep 4: Apply Overall Rating and Provide Summary• Go back through dimensions and add the individual

dimensional ratings to initially determine the overall rating category.

• Consider how your rating, based on the total points, matches your overall sense of the quality of the materials.

• Consider if your specific feedback statements are matched with the appropriate dimensions.

• Consider how your dimensional feedback supports your judgments.

• Consider if the lesson falls in the appropriate overall rating category.Determine and discuss your individual OVERALL ratings with your

review team.

57

EQuIP Quality Review ProcessStep 4: Developing Summary Comments

Summary Comments:

• Highlight the strongest aspects of the unit.

• Succinctly summarize key areas for improvement articulated in the dimensional comments.

Determine individually appropriate overall rating and summary comments and discuss with your review team.

585858

Step 4: Overall Rating and Summary CommentsExample Materials – Overall Rating and Summary

DIMENSIONAL RATINGS: 2311 R – [Overall rating]This unit, comprised of 10 lessons, is aligned partially to the CCSS it purports to measure. Alignment can be further enhanced if two of the CCSS that are not well addressed in this unit (1.NBT.1 and 1.OA.6) are eliminated from the list of targeted standards or lessons are added to address these two standards. The latter may be a poor choice, particularly with respect to 1.NBT.1, given the unit’s focus on 2-digit numbers. This unit (and its component lessons) receives its highest ratings in the dimensions related to Key Shifts in the CCSS and Alignment to the Depth of the CCSS. The greatest improvements can be made to this unit by expanding upon and clarifying instructional supports for teachers—including providing examples of student responses to suggested guiding questions, ensuring teacher access to and understanding of instructional materials, and clarifying and adding specific assessment tasks, rubrics and scoring guides to enhance the assessment capabilities for each lesson and the unit as a whole.

Normed ResponseExploring Two-Digit Numbers1

59

EQuIP Quality Review ProcessStep 5: Discuss Summary and Next Steps

Compare overall ratings and synthesize feedback:

• How do our overall ratings compare?

• Does this example serve as a model of CCSS instruction? What are its strengths? Areas for improvement?

• What communication and support will the developer receive?

• What are the next steps for this material?

60

EQuIP Quality Review ProcessReflection

• What additional practice is needed on the EQuIP Review Process and Rubric?

• What other ways can the EQuIP processes and materials influence and be incorporated into our practice?

• How will we plan for applying the EQuIP Quality Review Process? Who will be involved?

61

EQuIP Quality Review ProcessThe Review Team

When forming and/or working with a review team:

• Make sure all team members have training in the process and know the CCSS (at least for their grade level).

• Have a review plan that considers the experience and expertise of all team members.

• Team members may choose to compare individual ratings after each dimension or wait until each person has individually rated and recorded all input for Dimensions II–IV before beginning discussion.

• Individuals should record their overall rating prior to discussion.

• Adjustments to ratings and/or commentary should take place as a part of the group discussion.

62

Achievewww.achieve.org1400 16th Street, NW / Suite 510 Washington, DC 20036

equip rubric & quality review mathematics training module: grades k-5 1

Documents

quality review rubric

criteria evidence

review processes

review materials step

use of criteria

collegial process

checked criteria

common understanding