alignment powerful tool for focusing instruction, curricula, and assessment
TRANSCRIPT
-
Alignment Powerful Tool for Focusing Instruction, Curricula, and Assessment
-
Major QuestionsWhat is alignment?
How is alignment measured?
How are state standards and assessments aligned?
What are the implications of alignment results?
-
What Is Alignment?
-
AlignmentThe degree to which expectationsand assessments are in agreementand serve in conjunction with one another to guide the system toward students learning what is expected.
-
Degree of AlignmentStandardsAssess-mentAssessmentStandardsStandardsStandardsAssessmentAssessment Items
-
Alignment Is ImportantFederal statutes require alignmentEducation systems are easily fragmented
-
PurposePolicyProgramPracticeNational StandardsHigher Education RequirementsWork ExpectationsBest ThinkingPublic Opinion Standards Frameworks AssessmentsTeacher CertificationProfessional DevelopmentSchool OrganizationTextbooksClassroom InstructionStudent Outcomes
-
Alignment Is ImportantTeachers and students receive mixed messagesLearning expectations can be lowered for some students Federal statutes require alignment Education systems are easily fragmented
-
Curriculum TypesIntendedImplementedAssessedAchieved
-
Alignment has been an issue for as many as forty years,dating back to behavioral objectives and mastery-learning.
-
How is alignment measured?
-
Three Methods:Common Framework
Expert Consensus
Common Criteria
-
Content-by-Process Framework
Topics
Categories of Cognitive Demand
Measurement & Calculation in Science
Coverage
Memorize Facts/etc.
Understand Concepts
Perform Procedures
The International System
0 1 2 3
0 1 2 3
0 1 2 3
0 1 2 3
Mass & Weight
0 1 2 3
0 1 2 3
0 1 2 3
0 1 2 3
Length
0 1 2 3
0 1 2 3
0 1 2 3
0 1 2 3
Volume
0 1 2 3
0 1 2 3
0 1 2 3
0 1 2 3
Time
0 1 2 3
0 1 2 3
0 1 2 3
0 1 2 3
Temperature
0 1 2 3
0 1 2 3
0 1 2 3
0 1 2 3
-
Categories of Cognitive DemandMemorizeUnderstand ConceptsPerform ProceduresGenerate Questions/HypothesesCollect DataAnalyze & Interpret InformationUse Information to Make Connections
-
Achieve Matrix Grade 3 Mathematics Data Analysis and Probability
Obj. #
Text of Objective
A
B
Content Centrality
Type of Performance Centrality
Source of Challenge
Organize, describe and make predictions from existing data
10.A.
1a
Organize and display data using pictures, tallies, tables, charts, or bar graphs.
81
10A.
1b
Answer questions and make predictions based on given data.
5
20
41
53
66
69
-
Grade 8 Standards from Three StatesState A: The student will use proportions to solve scale-model problems with fractions and decimals.State B:Students compute with rational numbers expressed in a variety of forms; they solve problems involving ratios, proportions, and percentages. Use ratio and proportion to solve problems.State C:Apply proportional thinking in a variety of problem situations that include, but are not limited to: 1) ratios and proportions, and 2) percents, including those greater than 100 and less than one.
-
Achieve Alignment CriteriaItem-Standard Match
Content Centrality
Performance Centrality
Source of ChallengeInstrument-Standard MatchLevel of Challenge
Balance
Range
-
Five General Criteria1. Content Focus2. Articulation Across Grades and Ages3. Equity and Fairness4. Pedagogical Implications5. System Applicability
-
Specific CriteriaContent FocusA. Categorical ConcurrenceB. Depth-of-Knowledge ConsistencyC. Range-of-Knowledge CorrespondenceD. Structure-of-Knowledge ComparabilityE. Balance of RepresentationF. Dispositional Consonance
-
Indiana Mathematics StandardsGrade 8
Obj. #
Text of Objective
DOK
Item # 1
Item # 2
5A
Express any decimal in scientific notation
5B
Order a set of rational numbers
Fractions, Percents, Integers, & Irrationals
Reinforce an understanding of fractions, and develop an understanding of percent, integers, and irrationals
6A
Use models to compare two or more integers and/or irrational numbers
6B
Recognize the relationships between fraction, percent, ratio, and proportion
-
Depth of Knowledge
Level 1 Recall
Recall of a fact, information, or procedure.
Level 2 Skill/Concept
Use information or conceptual knowledge, two or more steps, etc.
Level 3 Strategic Thinking
Requires reasoning, developing plan or a sequence of steps, some complexity, more than one possible answer.
Level 4 Extended Thinking
Requires an investigation, time to think and process multiple conditions of the problem.
-
EXAMPLE OF STANDARDS AND DEPTH-OF-KNOWLEDGE LEVELS CONTENT AREA: GEOMETRY
Mathematics Standard
Depth-of- Knowledge Level
State D
Grade 8
VI. Geometric and Spatial Sense
VI.2
Explore transformations of geometric figures.
4
State B
Grade 8
II. Geometry
II.4
Graph on a coordinate plane similar figures, reflections, and translations.
2
State A
Grade 6
IV. Geometry and Spatial Sense
IV.D.
Investigate and predict the results of transformations of shapes, figures, and models including slides, flips, and turns.
IV.D.1
Identify and describe the results of translations (slides),
reflections (flips), rotations (turns), or glide reflections.
2
-
EXAMPLE OF STANDARDS AND DEPTH-OF-KNOWLEDGE LEVELSCONTENT AREA: PROBABILITY AND STATISTICS
Mathematics Standard
Depth-of-Knowledge Level
State D
Grade 8
VII. Data Analysis, Probability & Statistics
VII.3
Formulate, predict, and defend positions taken that are based on data collected.
4
State B
Grade 8
VI. Probability and Statistics
VI.1
Collect data involving 2 variables and display on a scatter plot; interpret results.
3
-
Which of the following numbers, when rounded to the nearest thousand, becomes 27,000?
(a) 26,099
(b) 26,490
(c) 27,381
(d) 27,550
(e) 27,640
-
121
1) 190
13
2) 200
32
3) 290
+ 34
4) N
-
A car odometer registered 41,256.9 miles when a highway sign warned of a detour 1,200 feet ahead. What will the odometer read when the car reaches the detour? (5,280 feet = 1 mile)
(a) 42,456.9
(b) 41,279.9
(c) 41,261.3
(d) 41,259.2
(e) 41,257.1
Did you use the calculator on this question?
Yes
No
This question refers to pieces N, P, and Q.
In Mr. Bells classes, the students voted for their favorite shape for a symbol. Here are the results.
Class 1
Class 2
Class 3
Shape N
9
14
11
Shape P
1
9
17
Shape Q
22
7
2
Using the information in the chart, Mr. Bell must select one of the shapes to be the symbol. Which one should he select and why?
The shape Mr. Bell should select: __________________
Explain:
-
This question refers to pieces N, P, and Q.
In Mr. Bells classes, the students voted for their favorite shape for a
symbol. Here are the results.
Class 1
Class 2
Class 3
Shape N
9
14
11
Shape P
1
9
17
Shape Q
22
7
2
Using the information in the chart, Mr. Bell must select one of the
shapes to be the symbol. Which one should he select and why?
The shape Mr. Bell should select: __________________
Explain:
-
New CubesYour school is planning a casino night to raise funds to construct a wall aquarium in your school. As a mathematics student, you are given the job of developing a dice game for this event.A regular pair of number dice consists of two cubes, each with its faces numbered 1 through 6. Often, dice games are played by rolling the two dice and then finding the sum of the two numbers turned upward.
1. Show that, with a regular pair of number dice, the probability of rolling a sum of 7 is greater than the probability of rolling any other sum.
-
You decide to call your casino game New Cubes. To make it interesting, you decide to construct new dice that have different numbers on their faces than regular dice. Here is how you will construct them:
Only the single digits 0 through 9 can be used.
Any digit can be used more than once.
When the dice are rolled every sum from 4 to 14 must be possible and
no other sums can occur.
The two dice do not have to be identical.
2. What numbers would you put on the 6 faces of each of the two dice so that the above
conditions are met?
Die 1: _____ _____ _____ _____ _____ _____
Die 2: _____ _____ _____ _____ _____ _____
3. Which sum(s) do you think would turn up most frequently if your New Cubes were
rolled 1000 times?
Explain why.
-
Categorical Concurrence State B Grade 8 Mathematics
-
Depth-of-Knowledge Consistency State B Grade 8 Mathematics
-
Range-of-Knowledge Correspondence State B Grade 8 Mathematics
-
Balance IndexState B Grade 8 Mathematics
-
Four State StudyPercent of Standards with Acceptable AlignmentCategory
Chart1
67Grade 3
67Grade 8
Grade 367
Grade 6100
Grade 4100
Grade 8100
Grade 10100
Grade 4100
Grade 883
38Grade 3
62Grade 7
62Grade 10
Grade 490
Grade 850
Science
Math
Sheet1
State AState BState CState D
Grade 3Grade 8Grade 3Grade 6Grade 4Grade 8Grade 10Grade 4Grade 8Grade 3Grade 7Grade 10Grade 4Grade 8
Science6767386262
Math67100100100100100839050
Sheet1
00
00
00
00
00
00
00
00
00
00
00
00
00
00
Science
Math
Sheet2
Sheet3
-
Four State StudyPercent of Standards with Acceptable AlignmentDepth of Knowledge
Chart1
83Grade 3
17Grade 8
Grade 350
Grade 6100
Grade 457
Grade 871
Grade 100
Grade 4100
Grade 883
25Grade 3
62Grade 7
12Grade 10
Grade 430
Grade 840
Science
Math
Sheet1
State AState BState CState D
Grade 3Grade 8Grade 3Grade 6Grade 4Grade 8Grade 10Grade 4Grade 8Grade 3Grade 7Grade 10Grade 4Grade 8
Science8317256212
Math5010057710100833040
Sheet1
00
00
00
00
00
00
00
00
00
00
00
00
00
00
Science
Math
Sheet2
Sheet3
-
Four State StudyPercent of Standards with Acceptable Alignment Range
Chart1
33Grade 3
33Grade 8
Grade 30
Grade 60
Grade 457
Grade 886
Grade 10100
Grade 433
Grade 80
0Grade 3
25Grade 7
12Grade 10
Grade 480
Grade 830
Science
Math
Sheet1
State AState BState CState D
Grade 3Grade 8Grade 3Grade 6Grade 4Grade 8Grade 10Grade 4Grade 8Grade 3Grade 7Grade 10Grade 4Grade 8
Science333302512
Math0057861003308030
Sheet1
00
00
00
00
00
00
00
00
00
00
00
00
00
00
Science
Math
Sheet2
Sheet3
-
Four State StudyPercent of Standards with Acceptable Alignment Balance
Chart1
100Grade 3
100Grade 8
Grade 3100
Grade 683
Grade 486
Grade 871
Grade 10100
Grade 483
Grade 883
88Grade 3
100Grade 7
100Grade 10
Grade 490
Grade 870
Science
Math
Sheet1
State AState BState CState D
Grade 3Grade 8Grade 3Grade 6Grade 4Grade 8Grade 10Grade 4Grade 8Grade 3Grade 7Grade 10Grade 4Grade 8
Science10010088100100
Math10083867110083839070
Sheet1
00
00
00
00
00
00
00
00
00
00
00
00
00
00
Science
Math
Sheet2
Sheet3
-
Average Measure Intraclass Correlation of Depth-of-Knowledge Levels Ratings for Mathematics
State/GradeNumber of ReviewersNumber of ItemsAlpha95% CILower-UpperA4461.47.11-.68 A7461.93.89-.95A7 (Repl)463.77.66-.85A9463.81.72-.88B5449.79.67-.87B8446.58.34-.75C4448.88.81-.93C4 (Repl)452.89.84-.93C84146.86.82-.89C11456.91.86-.94Average.79
-
Average Measure Intraclass Correlation of Depth-of-Knowledge Levels Ratings for Language Arts
State/GradeNumber of ReviewersNumber of ItemsAlpha95% CILower-UpperA4584.52.34-.67A7374.70.56-.80A10382.62.45-.74B5638.79.66-.88B8635.60.36-.78B11688.85.79-.89C4645.92.88-.95C8640.87.80-.92C116192.86.83-.89Average.75
-
Table 1Categorical Concurrence Between Standards and Assessment as Rated by Four Reviewers Grade 4 Mathematics (Number of Assessment Items65 Multiple Choice Items)1Includes one generic objective because coded items did not correspond to existing objectives.2Includes two generic objectives because coded items did not correspond to existing objectives.
StandardsLevel by ObjectiveHitsCategorical Concurr. AcceptableTitleGoals #Objs#Level# of objs by Level% w/in std by LevelMeanS.D.
I. Number Sense417.75112 117613926.75.43YESII. Algebra & Functions27.251252712920.501.12YESIII. Measurement & Geometry3172 1231511886615.25.43YESIV. Statistics, Data Analysis & Probability251233116020202.75.43NOTotal1146.75123341127224465.25.83
-
Table 2Depth-of-Knowledge Consistency Between Standards and Assessment as Rated by Four Reviewers Grade 4 Mathematics (Number of Assessment Items65 Multiple Choice Items)
StandardsLevel of Item w.r.t. StandardDepth-of-Knowledge Consistency Acceptable%Under%At%AboveTitleMeanS.D.Mean S.D.MeanS.D.I. Number Sense2644734428YESII. Algebra & Functions1029882938YESIII. Measurement & Geometry420952117YESIV. Statistics, Data Analysis & Probability5050505000WEAKTotal1838813817
-
Table 3ARange-of-Knowledge Correspondence and Balance of RepresentationBetween Standards and Assessment as Rated by Four Reviewers Grade 4 Mathematics (Number of Assessment Items65 Multiple Choice Items)
StandardsRange of ObjectivesRange of Knowledge Acceptable#Objs Hit% of TotalTitleMeanS.D.Mean S.D.I. Number Sense13.51.80769YESII. Algebra & Functions5.25.43722YESIII. Measurement & Geometry11.50.50683YESIV. Statistics, Data Analysis & Probability2.50.505010WEAKTotal8.194.596713
-
Table 3BBalance of Representation Between Standards and Assessmentas Rated by Four Reviewers Grade 4 Mathematics (Number of Assessment Items65 Multiple Choice Items)
StandardsBalance Index1 (perfect-0 no Balance)Balance of Representation Acceptable% Hits in Std/Ttl HitsIndexTitleMeanS.D.Mean S.D.I. Number Sense411.81.01YESII. Algebra & Functions311.67.05WEAKIII. Measurement & Geometry231.81.01YESIV. Statistics, Data Analysis & Probability41.96.07YESTotal2514.81.12
-
Table 4Summary of Attainment of Acceptable Alignment Level on Four Content Focus Criteria Grade 4 Mathematics (Number of Assessment Items65 Multiple Choice Items)
StandardsAlignment CriteriaCategorical ConcurrenceDepth-of-Knowledge ConsistencyRange of KnowledgeBalance of RepresentationI. Number SenseYESYESYESYESII. Algebra & FunctionsYESYESYESWEAKIII. Measurement & GeometryYESYESYESYESIV. Statistics, Data Analysis & ProbabilityNOWEAKWEAKYES
-
What are the implications of the results?
-
IMPLICATIONSConfirm quality of assessment items and standardsAttend to depth-of-knowledge levelsWrite standards at a moderate level of specificityUse multiple measuresIdentify acceptable levels for alignment criteria