Download - OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP
![Page 1: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/1.jpg)
OSPI Science Assessment UpdateWERA, December 2010
The State of Material on 2011 Science MSP
![Page 2: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/2.jpg)
Science Assessment OfficeElaine Wood—Secondary Specialist
Kara Monroe—Elementary [email protected]
Troi Williams—Administrative [email protected]
![Page 3: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/3.jpg)
Agenda2011 Measurements of Student
ProgressStandard SettingResources availableLessons learned from scoring
![Page 4: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/4.jpg)
2011 Measurements of Student Progress
•Grades 5 and 8•First assessment of the K-12 Science Learning Standards (adopted June 2009)
![Page 5: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/5.jpg)
Big Ideas in 2009 Science Standards
5
EALRs 1-3Crosscutting Concepts and
Abilities
EALR 4 Domains of Science
EALR 1 Systems Physical Science
…is a way of thinking that makes it possible to analyze and understand complex phenomena.
Force and MotionMatter: Properties and ChangeEnergy: Transfer, Transformation and Conservation
EALR 2 Inquiry Earth and Space Science
…is a process of asking and answering questions about the natural world that forms the bedrock of science.
Earth and SpaceEarth Systems, Structures and ProcessesEarth History
EALR 3 Application Life Science
…is about the interaction between science and technology, and how both can help solve real-world problems.
Structures & Functions of Living SystemsEcosystemsBiological Evolution
![Page 6: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/6.jpg)
Operational ScoringOperational ScoringOperational
Range FindingCommittee
Operational Range Finding
Committee
Items available for Operational TestingItems available for
Operational TestingData ReviewCommittee
Data ReviewCommittee Pilot ScoringPilot Scoring
Pilot Range Finding
Committee
Pilot Range Finding
Committee
Scenario & Item Pilot Testing
Scenario & Item Pilot Testing
Science Assessment Development Cycle
Science Assessment Development Cycle
Content Review & Bias/Sensitivity
Committees
Content Review & Bias/Sensitivity
Committees
Scenario & Items reviewed for content by
OSPI/contractor
Scenario & Items reviewed for content by
OSPI/contractor
Scenario & Item Writing Committee
Scenario & Item Writing Committee
Develop Test and Item Specifications
Develop Test and Item Specifications
![Page 7: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/7.jpg)
Scenarios and ItemsScenarios provide context for sets of
items.Systems, Inquiry, and Application Physical, Earth/Space, and Life Science
The items associated with a scenario cover a range of standards & range of cognitive complexity.
Items not connected to a scenario are called “Stand Alone” items, which is clearly marked for students.
![Page 8: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/8.jpg)
From WASL to Measurements of Student Progress
Grade 5 2009WASL
2010MSP
2011MSP
Multiple Choice 21 26 22-25
Completion NA pilot only 1-4
Short Answer 8 4 4
Extended Response 0 0 0
% Points from MC/CP 57% 76% 76%
Embedded pilot items makes the total number of items 35.
![Page 9: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/9.jpg)
From WASL to Measurements of Student Progress
Grade 8 2009WASL
2010MSP
2011MSP
Multiple Choice 28 30 26-29
Completion NA pilot only 1-4
Short Answer 7 5 5
Extended Response 2 0 0
% Points from MC/CP 56% 75% 75%
Embedded pilot items makes the total number of items 40.
![Page 10: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/10.jpg)
Test Specifications Grade 5 & 8The test map designates the following proportions: At least 20% Systems (with connected EALR 4
content)30% Inquiry 20% Application (which will apply EALR 4 content)At most 30% EALR 4 domain knowledge that is not
associated with a cross-cutting concepts or abilities*
*EALR 4 content may be assessed in questions that only assess EALR 4 or in questions that cut across both EALR 4 and a cross-cutting concept or ability (Systems, Inquiry, Application).
![Page 11: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/11.jpg)
More on Science MSP
Test LengthGrade 5: Single session 75-90 minutes*Grade 8: Single session 90-110 minutes**additional time is needed for passing out materials and reading directions
New Item Type Completion: worth 1 score point
Test Window May 2 to May 19 for paper/pencil testsMay 2 to June 3 for online tests
Online Testing Voluntary in grades 5 and 8 in 2011Goal: 25%
![Page 12: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/12.jpg)
Standard Setting: Summer 2011
Following the process used by Mathematics in grades 3-8 last year.
![Page 13: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/13.jpg)
Science Standard Setting Plan
Date EventJune 2009 New Science standards approvedMay 2010 Spring 2010 MSP; New items pilotedAug 2010 Test Map meeting
Sept 2010 Performance Level Descriptors (PLDs) developedOct 2010 Spring 2011 tests built
![Page 14: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/14.jpg)
Science Standard Setting Plan
Date EventJune 2009 New Science standards approvedMay 2010 Spring 2010 MSP; New items pilotedAug 2010 Test Map meeting
Sept 2010 Performance Level Descriptors (PLDs) developedOct 2010 Spring 2011 tests built
Feb/Mar 2011 Teachers from across state trained on how to use PLDs to participate in Contrasting Groups Study (CGS)
Mar/Apr 2011 NTAC and SBE formally approve standard setting planApril 2011 Teachers participate in CGS by providing ratings of studentsMay 2011 Spring 2011 MSP administered
![Page 15: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/15.jpg)
Science Standard Setting Plan
Date EventJune 2009 New Science standards approvedMay 2010 Spring 2010 MSP; New items pilotedAug 2010 Test Map meeting
Sept 2010 Performance Level Descriptors (PLDs) developedOct 2010 Spring 2011 tests built
Feb/Mar 2011 Teachers from across state trained on how to use PLDs to participate in Contrasting Groups Study (CGS)
Mar/Apr 2011 NTAC and SBE formally approve standard setting planApril 2011 Teachers participate in CGS by providing ratings of studentsMay 2011 Spring 2011 MSP administeredAug 2011 Standard Setting events:
--Grade level panels (use “Bookmark process”, start with teacher ratings)
--“Policy panel” recommendations--NTAC certifies the planned process was followed
Aug 2011 SBE Sets Achievement Standards (Reviews all recommendations)
![Page 16: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/16.jpg)
“Contrasting Groups” method affords broader involvement in standard settingEveryone in the state who teaches science in grades 5 and 8 is invited to help set the performance standards.Performance Level Descriptors (PLDs) are developed for “Basic” and “Proficient”Web-based training on PLDs is provided to participating teachersIn April, participating teachers rate each of their students as being “Basic or below” or “At or above Proficient” Two distributions of scores are produced: Scores obtained by students rated as “Basic”, and scores for “Proficient” Where the two distributions merge represents the separation point between “Basic” and “Proficient.”
![Page 17: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/17.jpg)
Students rated as “Basic or below” using criteria in PLD for Basic
Students judged to be at or below "Basic"
Points on MSP (simulated)0 2 10 6 84 12 14 16 18 20 22 23 24 26 30 3228 34 40 36 38
Raw score points
Number of studen
ts
![Page 18: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/18.jpg)
Students rated as “Proficient or above” using criteria in PLD for Proficient
Students judged to be "Proficient" or above
Points on MSP (simulated)0 2 10 6 84 12 14 16 18 20 22 23 24 26 30 3228 34 40 36 38
Raw score points
Number of studen
ts
![Page 19: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/19.jpg)
Students judged to be at or below "Basic"
Students judged to be "Proficient" or above
Points on MSP (simulated)0 2 10 6 84 12 14 16 18 20 22 23 24 26 30 3228 34 40 36 38
Intersection indicates a region for where “Basic” separates from “Proficient”
Raw score points
Number of studen
ts
![Page 20: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/20.jpg)
PLD trainingAvailable via Moodle, February 1, 2011Learn to use evidence to evaluate 5th or 8th grade
students’ proficiency (other grade levels can participate)
Using Moodle websiteShort videos of presentationsAll documents available for downloadCan accomplish all at once, or in multiple sittings
Free clock hours (3)Encourage groups of teachers to participate
togetherTo sign up, go to link and create an account
http://moodle.ospi.k12.wa.us/course/view.php?id=19
![Page 21: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/21.jpg)
Contrasting Groups StudyAvailable via EDS, April 11-29, 2011After completing the PLD Training, 5th or
8th grade teachers can predict their students’ performance on the Science MSP
5th and 8th grade science teachers can gain access to the EDS application by contacting their District Assessment Coordinator
![Page 22: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/22.jpg)
Resources Available
On the OSPI website:www.k12.wa.us/Science/
![Page 23: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/23.jpg)
OSPI Science Assessment Webpagewww.k12.wa.us
www.k12.wa.us/science
www.k12.wa.us/Science/Assessments.aspxSelect Science
Assessment from
Assessment Drop-
down
Select Assessm
ents
![Page 24: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/24.jpg)
Science Assessment ResourcesTest and Item Specifications—more
information about:Types of scenariosTest mapItem specificationsVocabulary listsCognitive complexity
![Page 25: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/25.jpg)
www.k12.wa.us/Science/Assessments.aspxTest and Item Specs
www.k12.wa.us/Science/TestItemSpec.aspx
Select Test and
Item Specificati
ons
Select a
Grade
![Page 26: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/26.jpg)
Test and Item Specswww.k12.wa.us/Science/TestItemSpec.aspx
Pages 5-14
Page 22
![Page 27: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/27.jpg)
Science Assessment ResourcesUpdates for 2011—each grade level
contains:General information on the 2011 testSample items for classroom useScoring information, including rubrics and
sample student responses
![Page 28: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/28.jpg)
Updates for 2011www.k12.wa.us/Science/Assessments.aspx
http://www.k12.wa.us/Science/EducatorResources.aspx
Select a
Grade
Select Educat
or Resourc
es
![Page 29: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/29.jpg)
Science Assessment ResourcesOnline Testing Website:
http://wa-online.caltesting.org/ Demo
Flash playerTutorial
CAL testing program must be installedPractice items, same as Update for 2011
documentsBoth tools help students become familiar
with testing platform, and tests the computer system
![Page 30: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/30.jpg)
CAL online testinghttp://wa-online.caltesting.org
Select Downloads or Demos
Stand-Alone item from 8th grade tutorial
![Page 31: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/31.jpg)
CAL online testingShort answer item from 5th grade tutorial
![Page 32: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/32.jpg)
CAL online testingShort answer item from 5th grade tutorial
Same item scrolled down to show symbols button and symbols tool pop-up.
![Page 33: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/33.jpg)
Science Assessment ResourcesTeacher Resource tool—tests based on
previous standards:Types of scenariosGeneric questionsItem specifications (previous standards)Cognitive complexityState-wide statistics
![Page 34: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/34.jpg)
Teacher Resource Toolwww.k12.wa.us/Science/Assessments.aspx
www.k12.wa.us/TeacherResourceTool2009-10/default.aspx
Select a Grade
and a Year
Select Scienc
e
Select Teacher
Resource Tool
![Page 35: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/35.jpg)
Teacher Resource Tool
Type of scenario
Generic question,
rolls over to item
specification
Percent correct or
mean score
![Page 36: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/36.jpg)
Science Assessment ResourcesPowerful Classroom Assessments (PCAs)
—currently linked to WASL released scenarios and items:Sample itemsScoring information, including rubrics
New PCAs will be developed from future released items
![Page 37: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/37.jpg)
Powerful Classroom Assessmentswww.k12.wa.us/Science/Assessments.aspx
http://www.k12.wa.us/Science/EducatorResources.aspx
Select PCA
Select Educat
or Resourc
es
Select grade level
http://www.k12.wa.us/Science/PCAs/default.aspx
![Page 38: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/38.jpg)
Lessons Learned from Scoring
![Page 39: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/39.jpg)
Transition to new standards
Grade 5 Grade 8Terms :
“system” and “subsystem”
“inputs” and “outputs”
Human needs leading to inventions
Weight as a measurement of the force of gravity
Terms “closed” and
“open” systems“balanced” and
“unbalanced” forcesRoles of engineers
and scientists in technological advances
![Page 40: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/40.jpg)
Common misconceptions (gr. 5)When the results don’t match the prediction,
students struggle with how to report honestlyStudents think the mass of an substance
changes when the substance breaks or meltsStudents think that plants receive food from
soilStudents have misconceptions about what
makes something a producer in an ecosystem.Students think genetic characteristics are
acquired from the environment
![Page 41: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/41.jpg)
Common misconceptions (gr. 8)Students have trouble comparing a model to
the real-world system
Students think the mass of an substance changes when the substance changes state or temperature
Students have trouble calculating speedStudents don’t understand photosynthesisStudents struggle to see the how natural
selection leads to changes in a population over time
![Page 42: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/42.jpg)
Going well (gr. 5)Variables questions
65% to 90% of the timeStudents are including data in
conclusion items28-37% get full points Between 30% and 40% give each supporting
data point Students are attempting energy transfer
and new procedure items3.1% to 5.0% blanks
![Page 43: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/43.jpg)
Going well (gr. 8)Variables questions
66% to 84% of the timeStudents are including data in
conclusion items54% get full points Between 55% and 58% give each supporting
data point Students are attempting new procedure
items7.9% - 10% blanks
![Page 44: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/44.jpg)
Constructed responses (both)Students confuse different forces (gravity, push,
pull, magnetic) with forms of energy (5th: light, heat, sound, motion, electricity; 8th: light, thermal (heat), chemical, sound, kinetic, electrical)
Students need more help writing their own procedures: Changing the manipulated variable indicated in the
questionMeasuring the responding variable indicated in the
questionStudents need to be more detailed when answering
application items; more specific than “it will work” or “it must be broken, so fix it.”
![Page 45: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/45.jpg)
Upcoming EventsPerformance Level Descriptors training
Via Moodle website (February-March 2011)
Scenario Writing Workshops5th and 8th grade (February 2011)High School (February 2011)
Contrasting Groups Study (April 2011)
![Page 46: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/46.jpg)
Upcoming EventsOperational Rangefinding Committees
High School (Olympia, April 2011)5th and 8th grade (Cincinnati, Ohio, May
2011)
Pilot Rangefinding Committees (July 2011)
Standard Setting (August 2011)
![Page 47: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/47.jpg)
Opportunities for InvolvementPEPPER email list
PreSALTers Enthusiastically Promoting Powerful Educational Resources
Get updates about science happeningsSign up today with your e-mail address, or e-
mail us and ask to be a PEPPERSALT list
Science Assessment Leadership TeamAnyone can apply Respond to emails to sit on Item Development
committeesCommittees are chosen to balance state
representation and content expertise
![Page 48: OSPI Science Assessment Update WERA, December 2010 The State of Material on 2011 Science MSP](https://reader030.vdocuments.site/reader030/viewer/2022013011/56649e0d5503460f94af6762/html5/thumbnails/48.jpg)
Science Assessment OfficeElaine Wood—Secondary Specialist
Kara Monroe—Elementary [email protected]
Troi Williams—Administrative [email protected]