new york state physical setting: physics a review of the june 2006 exam nys aapt fall meeting...
Post on 19-Dec-2015
215 views
TRANSCRIPT
New York State Physical Setting: Physics New York State Physical Setting: Physics
A Review of the June 2006 ExamNYS AAPT Fall Meeting
Binghamton, NY
J. ZawickiSUNY Buffalo State College
T. JohnsonErie 1 BOCES, Data Warehouse
Mike DuPréBiology Mentor Network
A Review of the June 2006 ExamNYS AAPT Fall Meeting
Binghamton, NY
J. ZawickiSUNY Buffalo State College
T. JohnsonErie 1 BOCES, Data Warehouse
Mike DuPréBiology Mentor Network
October 7, 2006 J. Zawicki, T. Johnson, M. DuPré 2
Assessment PurposesAssessment Purposes
Teachers Measure knowledge Measure gain in knowledge Sorting (Grading)
Students/Parents Measure preparation (predict success)
School District/State Education Department Degree requirements (benchmarks)
Others…
Teachers Measure knowledge Measure gain in knowledge Sorting (Grading)
Students/Parents Measure preparation (predict success)
School District/State Education Department Degree requirements (benchmarks)
Others…
October 7, 2006 J. Zawicki, T. Johnson, M. DuPré 3
•Frameworks
•Syllabi
•Guides
•Blueprints
•Benchmarks
•Objective tests
•Performance assessments
•Portfolios
•Teacher Observations
•Group Activities
•Program Evaluations
Curriculum Standards
Assessment/Evaluation System Instructional Program
alignment
validity
correlation
•Instructional styles
•Print materials
•Equipment
•Facilities
•Technology
•Community
October 7, 2006 J. Zawicki, T. Johnson, M. DuPré 4
Types of AnalysisTypes of Analysis
Traditional Difficulty Discrimination Response pattern
Rasch Analysis Item difficulty equated to student ability Standard setting benchmark’s essential
Traditional Difficulty Discrimination Response pattern
Rasch Analysis Item difficulty equated to student ability Standard setting benchmark’s essential
October 7, 2006 J. Zawicki, T. Johnson, M. DuPré 5
Types of Analysis (Continued)Types of Analysis (Continued)
Cognitive Level - Bloom’s taxonomy Knowing Using integrating
Alignment Curriculum and Assessment Andrew Porter
Item format
Cognitive Level - Bloom’s taxonomy Knowing Using integrating
Alignment Curriculum and Assessment Andrew Porter
Item format
CreatingEvaluatingAnalyzingApplying
UnderstandingRemembering
October 7, 2006 J. Zawicki, T. Johnson, M. DuPré 6
Types of Analysis (Continued)Types of Analysis (Continued)
Teacher Review (Biology Mentor Network) Difficulties analyzed in the context of:
Student issues Testing issues Instructional issues
Use of formative techniques to support conjectures
Teacher Review (Biology Mentor Network) Difficulties analyzed in the context of:
Student issues Testing issues Instructional issues
Use of formative techniques to support conjectures
October 7, 2006 J. Zawicki, T. Johnson, M. DuPré 7
ConceptsConcepts
Difficulty – Percentage or proportion that are successful on an item
Discrimination – How well does an item differentiate between students who understand the subject and those who do not?
Validity – Does an item measure student understanding of the intended concept?
Difficulty – Percentage or proportion that are successful on an item
Discrimination – How well does an item differentiate between students who understand the subject and those who do not?
Validity – Does an item measure student understanding of the intended concept?
October 7, 2006 J. Zawicki, T. Johnson, M. DuPré 8
Concepts (Continued)Concepts (Continued)
Reliability – can the results be replicated? Inter-rater Test/Re-test Internal Consistency
Criterion referenced tests Latency
Reliability – can the results be replicated? Inter-rater Test/Re-test Internal Consistency
Criterion referenced tests Latency
October 7, 2006 J. Zawicki, T. Johnson, M. DuPré 12
Test Data – Discussion and Analysis
Test Data – Discussion and Analysis
Collecting Data Analysis
Difficulty Response Pattern
Collecting Data Analysis
Difficulty Response Pattern
October 7, 2006 J. Zawicki, T. Johnson, M. DuPré 13
Interpreting DataInterpreting Data
October 7, 2006 J. Zawicki, T. Johnson, M. DuPré 14
Multiple Choice - EasierMultiple Choice - Easier
Item Difficulty 1 2 3 4 NR
27 0.95 61 19 22 1903 3
October 7, 2006 J. Zawicki, T. Johnson, M. DuPré 15
Multiple Choice - EasierMultiple Choice - Easier
Item Difficulty 1 2 3 4 NR
23 0.90 43 67 93 1802 3
October 7, 2006 J. Zawicki, T. Johnson, M. DuPré 16
Multiple Choice, EasierMultiple Choice, Easier
Item Difficulty 1 2 3 4 NR
46 0.89 43 1793 9 72 4
October 7, 2006 J. Zawicki, T. Johnson, M. DuPré 17
Multiple Choice, EasierMultiple Choice, Easier
Item Difficulty 1 2 3 4 NR
18 0.88 96 1764 78 63 7
October 7, 2006 J. Zawicki, T. Johnson, M. DuPré 18
Multiple Choice, EasierMultiple Choice, Easier
Item Difficulty 1 2 3 4 NR
16 0.86 62 132 1733 75 6
October 7, 2006 J. Zawicki, T. Johnson, M. DuPré 19
Multiple Choice, More DifficultMultiple Choice, More Difficult
Item Difficulty 1 2 3 4 NR
44 0.32 649 151 902 302 4
October 7, 2006 J. Zawicki, T. Johnson, M. DuPré 20
Multiple Choice, More DifficultMultiple Choice, More Difficult
Item Difficulty 1 2 3 4 NR
25 0.32 56 645 1045 258 4
October 7, 2006 J. Zawicki, T. Johnson, M. DuPré 21
Multiple Choice, More DifficultMultiple Choice, More Difficult
Item Difficulty 1 2 3 4 NR
50 0.39 775 492 220 512 9
October 7, 2006 J. Zawicki, T. Johnson, M. DuPré 22
Multiple Choice, More DifficultMultiple Choice, More Difficult
Item Difficulty 1 2 3 4 NR
51 0.42 486 267 394 851 10
October 7, 2006 J. Zawicki, T. Johnson, M. DuPré 23
Multiple Choice, More DifficultMultiple Choice, More Difficult
Item Difficulty 1 2 3 4 NR
40 0.47 940 722 164 178 4
October 7, 2006 J. Zawicki, T. Johnson, M. DuPré 24
Constructed Response, EasierConstructed Response, Easier
Item Difficulty 0 1 2
64 0.92 152 1854 0
October 7, 2006 J. Zawicki, T. Johnson, M. DuPré 25
Constructed Response, EasierConstructed Response, Easier
Item Difficulty 0 1 2
53 0.90 208 1798 0
October 7, 2006 J. Zawicki, T. Johnson, M. DuPré 26
Constructed Response, EasierConstructed Response, Easier
Item Difficulty 0 1 2
61 0.87 144 249 1613
October 7, 2006 J. Zawicki, T. Johnson, M. DuPré 27
Constructed Response, EasierConstructed Response, Easier
Item Difficulty 0 1 2
59 0.86 274 1732 0
October 7, 2006 J. Zawicki, T. Johnson, M. DuPré 28
Constructed Response, EasierConstructed Response, Easier
Item Difficulty 0 1 2
65 0.86 154 250 1602
October 7, 2006 J. Zawicki, T. Johnson, M. DuPré 29
Constructed Response, More DifficultConstructed Response, More Difficult
Item Difficulty 0 1 2
58 0.37 995 524 487
October 7, 2006 J. Zawicki, T. Johnson, M. DuPré 30
Constructed Response, More DifficultConstructed Response, More Difficult
Item Difficulty 0 1 2
72 0.42 964 388 654
October 7, 2006 J. Zawicki, T. Johnson, M. DuPré 31
Constructed Response, More DifficultConstructed Response, More Difficult
Item Difficulty 0 1 2
62 0.50 749 506 751
October 7, 2006 J. Zawicki, T. Johnson, M. DuPré 32
Constructed Response, More DifficultConstructed Response, More Difficult
Item Difficulty 0 1 2
63 0.51 980 1026 0
October 7, 2006 J. Zawicki, T. Johnson, M. DuPré 33
Constructed Response, More DifficultConstructed Response, More Difficult
Item Difficulty 0 1 2
68 0.65 410 576 1020
October 7, 2006 J. Zawicki, T. Johnson, M. DuPré 34
In ConclusionIn Conclusion
Summary of findings Conceptually challenging items “Inscription” Calculations, showing work…
Future directions Revisiting standard setting… How well are the current standards working?
Next steps… Considerations within our classrooms
Summary of findings Conceptually challenging items “Inscription” Calculations, showing work…
Future directions Revisiting standard setting… How well are the current standards working?
Next steps… Considerations within our classrooms
October 7, 2006 J. Zawicki, T. Johnson, M. DuPré 35
Resources from this presentation…Resources from this presentation…
http://physicsed.buffalostate.edu/pubs/AAPTmtgs/NYSS/Fall06
Email: [email protected]
Office Phone (716) 878-3800
http://physicsed.buffalostate.edu/pubs/AAPTmtgs/NYSS/Fall06
Email: [email protected]
Office Phone (716) 878-3800