© 2014 Region One Education Service CenterDivision of Instructional Support
Curriculum, Instruction & Assessment
Power of 7 – Eduphoria STAARSeptember 16th, 2016
Office of CIA- Learning Resources Integration
Norms
Honor all start times
Use electronic devices to enhance your
learning experiences
Those that do, learn, so participate actively
Share best practices and Network
Incorporate what you have learned into your
own best practices
Agenda
• Triangle of Data Driven Instruction
– Percent Score Data
– Reporting Category Data
– Student Expectation Data
– Factor Analysis Data
• Degree of Mastery Patterns
• Implications for Curriculum and Instruction
• Trend Reports
Objectives
• Access Eduphoria Reports for instructional purposes
• Engage in the Degree of Mastery process for student expectations
• Analyze the implications for curriculum and instruction
• View trend reports as an ongoing process to monitor student progress
Ice Breaker
Data Analysis is like…….
Data Analysis
RootProblem
CorrectiveInstruction
Implementation
CalibrationTool
Systemic Analysis and Instructional Practices for Educators
Percent Score Data
Report Category Data
Student Expectation
Data
Root Cause
Percent Score DataReports: Percent Score View &
Percent Score by Students View
Reporting Category DataReports: Reporting Category View &
Reporting Category Student View
How do you define
?
Reflection activityThink-Write-Pair-Share
What is rigor?
Individually: write down some characteristics of rigor that will help to define it
Pair up with a person next to you to compare notes
Share out with large group.
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment16(TEA, 2010, HB3 Sec1 Chapter 2 p. I-28)
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment17
RIGOR
FOCUS
Content
Complexity
Learning Standards
TEKS
CLARITY DEPTH
Cognitive
Complexity
Assessments
DOK
Performance Standards
TEKS/STAAR
What is Rigor?
Performance Level Descriptions
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment18
What is Rigor?
Content Complexity
Cognitive Complexity
Specificity PLDs
Student Expectations
Content Complexity
Cognitive Complexity
Specificity DOK
Student Expectations
Instruction Assessment
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment19
Content Complexity Cognitive Complexity
Specificity Blooms Taxonomy
Student Expectations
Curriculum
What is Rigor?
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment20
STAAR Performance Labels
• Advanced Academic PerformanceLevel III
• Satisfactory Academic PerformanceLevel II
• Unsatisfactory Academic PerformanceLevel I
Cut Score
Cut Score
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment21
Purpose of the PLDs• Provide a snapshot of student’s academic
characteristics based on assessment performance
• Describe a student’s knowledge and skills by levels of performance:• Level III- Advanced• Level II - Satisfactory • Level I - Unsatisfactory
• Translate into grade or course specific descriptions of student achievement
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment22
Purpose of the PLDs
Level III
Level II
Level I
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment23
Performance Level Descriptors (PLD): Progressions of Learning
Performance Level
Descriptors
Curriculum
Instruction
Assessment
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment24
Reflection Question
• What are the implications of Level I, Level II, Level III for Instruction?
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment25
Thinking about data: Digging Deeper
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment26
Student Expectation Data
Reports: Student Expectation View &Student Expectation by Student View
Starting the Road Map
What is the rigor of the standards?
What do students need to know and be able do?
Content Complexity Cognitive Complexity
Specificity Blooms Taxonomy
Student Expectations
Curriculum
What is Rigor?
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment29
What is Rigor?
Content Complexity
Cognitive Complexity
Specificity PLDs
Student Expectations
Content Complexity
Cognitive Complexity
Specificity DOK
Student Expectations
Instruction Assessment
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment30
STAAR is assessing content and skills at a greater depth
and higher level of cognitive complexity.
(TEA, 2010, p. I-4)
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment31
Cognitive Complexity and
The Revised Bloom’s Taxonomy
A New Framework
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment32
Revised Bloom’s Taxonomy
Knowledge Dimension
knowledge that shares a consensus of acceptance within the
discipline
there are four types of knowledge: factual.
conceptual, procedural, and metacognitive
Cognitive Process Dimension
verbs which reflect the underlying thinking
processes in order of increasing complexity
there are six major cognitive process
categories
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment33
OLD
Evaluation
Synthesis
Analysis
Application
Comprehension
Knowledge
NEW
Create
Evaluate
Analyze
Apply
Understand
Remember
N
O
U
N
S
V
E
R
B
S
Adapted form Anderson & Krathwohl, 2001)
Revised Bloom’s Taxonomy
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment34
Let’s analyze the TEKS
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment35
Cognitive Expectation
• Cognitive
– The level at which students are
expected to perform in order to
adequately meet the standard.
– Determined by the verbs used in both
the knowledge and skills statements
and student expectation(s).
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment36
• TEKS Format – Essential Knowledge and Skills
• Grade 8
• (8) History. The student understands individuals,
issues, and events of the Civil War.
TEKS Breakdown: Cognitive
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment37
• TEKS Format – Student Expectation
• 8B - explain the causes of the Civil War, including sectionalism, states' rights, and slavery, and significant events of the Civil War, including the firing on Fort Sumter; the battles of Antietam, Gettysburg, and Vicksburg; the announcement of the Emancipation Proclamation; Lee's surrender at Appomattox Court House; and the assassination of Abraham Lincoln; and (R)
Cognitive Expectations
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment38
Content Expectations
• Content
– The content items for which
students must demonstrate
understanding at the appropriate
cognitive level in order to
adequately meet the standard.
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment39
• TEKS Format – Essential Knowledge and Skills
• Grade 8
• 8.8 History. The student understands individuals, issues,
and events of the Civil War.
TEKS Breakdown: Content
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment40
• TEKS Format – Student Expectations
• 8B - explain the causes of the Civil War, including sectionalism, states' rights, and slavery, and significant events of the Civil War, including the firing on Fort Sumter; the battles of Antietam, Gettysburg, and Vicksburg; the announcement of the Emancipation Proclamation; Lee's surrender at Appomattox Court House; and the assassination of Abraham Lincoln; and (R)
Content Expectations
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment41
Finding the Root Cause
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment42
Finding the Root Cause
Root CauseAnalysis
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment44
Let’s Find the Standard
Specificity of assessment
items
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment45
(4)(B) explain the roles played by significant individuals during the American Revolution, including Abigail Adams, John Adams, Wentworth Cheswell, Samuel Adams, Mercy Otis Warren, James Armistead, Benjamin Franklin, Bernardo de Gálvez, Crispus Attucks, King George III, Haym Salomon, Patrick Henry, Thomas Jefferson, the Marquis de Lafayette, Thomas Paine, and George Washington; (16 names)
Standard Specificity!
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment46
• SE: 1A - identify the major eras and events in U.S. history through 1877, including colonization, revolution, drafting of the Declaration of Independence, creation and ratification of the Constitution, religious revivals such as the Second Great Awakening, early republic, the Age of Jackson, westward expansion, reform movements, sectionalism, Civil War, and Reconstruction, and describe their causes and effects; (R)
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment47
SE: 19D - identify examples of responsible citizenship, including obeying rules and laws, staying informed on public issues, voting, and serving on juries; (S)DUAL: 29C - organize and interpret information from outlines, reports, databases, and visuals, including graphs, charts, timelines, and maps; (P)
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment48
Depth of Knowledge
What item levels are being assessed?
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment49
“A primary feature of STAAR’s test design is a focus on preparedness for success in subsequent grades or courses and, ultimately for college and career. This requires the test to emphasize depth rather than breadth in assessing student expectations.”-2010, House Bill 3: Transition to STAAR
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment50
Dr. Norman Webb & Depth of Knowledge
Norman Webb’s work has been applied to different content areas and used in test item development, as well as in alignment studies to determine the degree of alignment (match) between states’ standards and the tests used by states for accountability purposes.
2005, 2006 Cognitive complexity: Applying Webb DOK Levels of Bloom’s Taxonomy, Karin K. Hess, National Center for Assessment,
Dover, NH © Karin K. Hess permission to reproduce is given when authorship is fully cited [email protected]
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment51
What is Dr. Norman Webb’s Depth of Knowledge?
A common language for discussing rigor
A tool for alignment in educational assessment
A way to revise assessment practices
More informed dialogue for instruction
A part of reflective teaching
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment52
Content Complexity• What is content complexity in assessment?
Differentiates learning expectations and outcomes by considering the amount of prior knowledge, processing of concepts and skills, sophistication, number of parts, and application of content structure required to meet an expectation or to attain an outcome.
-Dr. Norman Webb-
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment53
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment54
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment55
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment56
Depth of Knowledge Tool
Root Cause
Analysis
Finding the Root Cause
Root Cause
Analysis
Root Cause Analysis View
Data-Driven Instruction..what does it mean?
Data-driven instruction is the philosophy that schools
constantly focus on one simple question:
Are our students learning?
Quick Share out
How do you know when your students have mastered your content and when adjustments may be needed in your instructional timeline?
Data AnalysisLevel 1:
Wide Angle View of Student Performance
• How well did the class do as a whole?
• What are the strengths and weaknesses in the standards: where do we need to work the most?
• How did the class do on previously taught versus currently taught standards? Are they forgetting or improving on previous material?
Data Analysis
Level 2:
Close-up View of Student Performance
• Did students all choose same wrong answer? Why or why not?
• Break down each standard: Did students do similarly on every question within the standard or were some questions harder? Why?
• Compare similar standards: Do results in one influence the other?
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment68
Sample Questions
20%
11%
38%
31%
62% Chose the wrong answer!
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment69
Sample Question
22%
10%
50%
18%
50% Chose the wrong answer
• 2016 STAAR Item Analysis- Demographic w/Responses by Reporting Category
• Did students all choose same wrong answer? Why or why not?
• Did students do similarly on every question within the standard or were some questions harder? Why?
• Compare similar standards: Do results in one influence the other?
Zooming in..
Degree of Mastery• The DOM Analysis is to support teachers in
monitoring the effectiveness of instruction with the end result of greater teacher effectiveness in the classroom.
• The DOM Analysis is to provide Leadership with data to internalize--the need for quality curriculum, evaluation of learning, effective intervention, and effective instructional approaches.
Error Response Patterns
Percent of student group selecting individual distractor options.
• Type I• No Distractor options selected at a rate of 10% or more respectively
• Type II• Only One Distractor option selected at a rate of 10% or more respectively
• Type III • Only two distractor options selected at a rate of 10% or more respectively
• Type IV• Three or more distractor options selected at a rate of 10% or more respectively
Type I
Percent of student group selecting individual distractor options.
• No distractor options selected at a rate of 10% or more respectively; no distractor options are plausible to the examinee group.
Instructional Condition Instructional Alignment Rationale
Highly Aligned InstructionInstruction is highly aligned to knowledge and skills being assessed and mastery learning is taking place.
Type II
Percent of student group selecting individual distractoroptions.
• Only one distractor option is highly plausible to the examinee group
Instructional Condition Instructional Alignment Rationale
Aligned InstructionInstruction is aligned to knowledge and skills being assessed.
Type III
Percent of student group selecting individual distractor options.
• Only two distractor options are plausible to the examinee group
Instructional Condition Instructional Alignment Rationale
Minimally Aligned InstructionInstruction is minimally aligned to Knowledge and skills being assessed.
Type IV
Percent of student group selecting individual distractoroptions.
• All three distractor options are plausible to the examinee group.
Instructional Condition Instructional Alignment Rationale
No Aligned InstructionInstruction is not aligned to knowledge and skills
being assessed or item content not taught.
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment77
DOM TaxonomyDistractor Pattern Functioning Taxonomy
Distractor
PatternDistractor Rationale Performance Standard Instructional Condition
Instructional Alignment
Rationale
IV
All three distractor options
are plausible to the
examinee group.
Knowledge and Skills
Assessed
Ineffective or
Lack of Instruction
Instruction is not aligned to
knowledge and skills being
assessed
or item content not taught.
III
Only two distractor options
are plausible to the
examinee group.
Knowledge and Skills
AssessedModerate Instruction
Instruction is minimally aligned to
Knowledge and skills being
assessed.
II
Only one distractor option
is highly plausible to the
examinee group.
Knowledge and Skills
AssessedEffective Instruction
Instruction is aligned to knowledge
and skills being assessed.
I
No distractors options are
plausible to the examinee
group.
Knowledge and Skills
AssessedHighly Effective Instruction
Instruction is highly aligned to
knowledge and skills being
assessed
and mastery learning is taking
place.
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment78
Item Analysis Summary ReportSocial Studies
PERCENT RESPONDING
ITEM (A,F) (B,G) (C,H) (D,J)
1. 2 1 1 96*
2. 62* 20 6 12
3. 4 81* 2 13
4. 14 18 56* 12
5. 93* 3 2 2
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
50. 22 13 15 38*
51. 37 53* 4 1
52. 3 3 86* 4
Type I
Type III
Type II
Type IV
Type I
Type IV
Type II
Type I
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment79
Your Turn!
• Find the Degree of Mastery for your assessment items.
• What did you discover in the Degree of Mastery Analysis?
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment80
• Based on the information what are the implications for curriculum, instruction and assessment?
Data Analysis
RootProblem
CorrectiveInstruction
Implementation
CalibrationTool
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment82
Percent Score Data
Report Category Data
Student Expectation
Data
Root Cause
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment83
Planning for High Quality Instruction
• What needs to be addressed all year long?
• Which content needs more time?
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment84
Effective Instructional Approaches
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment85
Data Analysis
RootProblem
CorrectiveInstruction
Implementation
CalibrationTool
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment86
Eduphoria Assessment
• TEKSbank – Online Testing
• Eduphoria – Online Student Response System
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment87
Systemic Analysis and Instructional Practices for Educators
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment88
Recommended Trend Reports
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment89
As the teachers continue to analysis their local data by systemically practicing the process of the root cause analysis, another key component will be progress monitoring. It’s imperative to monitor the progress of the district, campus, teacher, class, students and focus groups.
Progress monitoring will follow the same concept of the triangle which includes percent score, reporting category and expectation data. In this example, we are checking how the District is performing.
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment90
As the teachers continue to analysis their local data by
systemically practicing the process of the root cause analysis, another key component will be progress monitoring. It’s imperative to monitor the progress of the district, campus, teacher, class, students and focus groups.
Progress monitoring will follow same concept of the triangle which includes percent score, reporting category and
expectation data. In this example, we are checking how our campus is performing.
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment91
As the teachers continue to analysis their local data by
systemically practicing the process of the factor analysis, another key component will be progress monitoring. It’s imperative to monitor the progress of the district, campus, teacher, class, students and focus groups.
Progress monitoring will follow same concept of the triangle which includes percent score, reporting category and
expectation data. In this example, it will allow us to check how our teacher is performing.
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment92
As the teachers continue to analysis their local data by
systemically practicing the process of the root cause analysis, another key component will be progress monitoring. It’s imperative to monitor the progress of the district, campus, teacher, class, students and focus groups.
Progress monitoring will follow same concept of the triangle which includes percent score, reporting category and
expectation data. In this example, we are checking how our students are performing.
© 2014Division of Instructional Support
Curriculum, Instruction & Assessment93
Connect/Extend/Challenge
• How are the ideas/information presented connected to what you already knew?
• What ideas did you get that extended or broadened your thinking?
• What challenges remain from the ideas/information presented?
Sources
© 2014 Region One Education Service CenterDivision of Instructional Support
Curriculum, Instruction & Assessment
Greenstein, L. (2010). What teachers really need to know about formative assessment.Alexandria, VA: ASCD.
Ritchhart, R., Church, M., & Morrison, K. (2010). Making thinking visible. San Francisco, CA: Jossey-Bass.
TEA, Curriculum Division. (2007-2012). Texas essential knowledge and skills. Retrieved April 30, 2013, from Texas Education Agency: Online
TEA, Student Assessment Division. (2011-2012, June 8). STAAR assessed curriculum. Retrieved May 5, 2013, from Texas Education Agency: Online
TEA, Student Assessment Division. (2013, January 11). STAAR performance level descriptors. Retrieved January 12, 2013, from Texas Education Agency.
TEA, Student Assessment Division. (2013, January 11). STAAR Grade 8 Social Studies April 2013 Released. Retrieved August 5, 2013, from Texas Education Agency.
Walsh, J. A., & Sattes, B. D. (2011). Thinking through quality questioning. Thousand Oaks, CA: Corwin.