florida association of science supervisors presentation november 3, 2005 orlando, florida

Florida Associationof Science

SupervisorsPresentation

November 3, 2005Orlando, Florida

22November 3, 2005November 3, 2005 Florida Department of Education Florida Department of Education

Presentation Topics• FCAT Update – Science• Setting FCAT Science Achievement

Levels• Grade 11 Student Motivation Discussion• Interpretive Products• Release of FCAT Tests• 2006 Testing Schedule • FCAT Update – Other Content Areas• Cognitive Complexity


FCAT Science Update• FCAT Science Standard Setting meeting

for educators representing Grades 5, 8, and 11 held in January and September 2005

• DOE plans to report student scores and achievement levels beginning with the 2006 test

• Science to be included in school grade calculations beginning with the 2007 test


Modifications to Appendix B

Grade 5 B.1.2.2 – now also assesses B.1.2.3,

B.1.2.5, andB.1.2.6

C.2.2.4 – now also assess C.2.2.3 D.2.2.1 – now a CS also assessing

B.2.2.2 and B.2.2.3 (was with G.2.2.3)



Grade 8Grade 8B.1.3.1 – now also assesses B.1.3.4B.1.3.1 – now also assesses B.1.3.4F.2.3.3.- now an Annually Assessed F.2.3.3.- now an Annually Assessed

also assessing F.2.3.4 and G.1.3.2also assessing F.2.3.4 and G.1.3.2G.2.3.2 – now an Annually Assessed G.2.3.2 – now an Annually Assessed

assessing D.2.3.2, G.2.3.3, and assessing D.2.3.2, G.2.3.3, and G.2.3.4 (previously the AA ‘parent’ G.2.3.4 (previously the AA ‘parent’ was G.2.3.4was G.2.3.4



Grade 11Grade 11A.1.4.3 – now Annually Assessed also A.1.4.3 – now Annually Assessed also

assessing B.1.4.3assessing B.1.4.3C.1.4.1 – now Annually Assessed also C.1.4.1 – now Annually Assessed also

assessing C.1.4.2 and C.2.4.6assessing C.1.4.2 and C.2.4.6H.1.4.2 – now also assesses H.1.4.5H.1.4.2 – now also assesses H.1.4.5


Upcoming FCAT Science Meetings

Item Review – 3 weeks, 5 different groupsOctober 31, November 7, and January 9(review of item development for field testing in 2007)

Rangefinder Review November 30 through December 9 (review of scoring criteria for 2006 operational performance task items)

Rangefinding Wave 1 April and May Wave 2 September(setting of scoring criteria for performance tasks field tested in 2006)


Process for Nominating Science Educators

• Nomination forms Nomination forms • Sent to SuperintendentsSent to Superintendents• Given to District Science Supervisors and Given to District Science Supervisors and

educators serving on committeeseducators serving on committees• More diversity needed in Science More diversity needed in Science

databasedatabase• GenderGender• EthnicityEthnicity• Science content areasScience content areas


Next Version of FCAT Science Item Specifications

• On hold pending the revision to the Sunshine On hold pending the revision to the Sunshine State StandardsState Standards

• 2002 Draft of Item Specifications on website 2002 Draft of Item Specifications on website • PDF Appendix B (dated October 2004)sent to PDF Appendix B (dated October 2004)sent to

District Assessment Coordinators also District Assessment Coordinators also available via email request to meavailable via email request to me

• Sample Test Materials have current version of Sample Test Materials have current version of the reference sheets and scoring rubricsthe reference sheets and scoring rubrics

Setting Achievement

Levels for FCAT Science

Summary of Process and

Timeline


What is Standard Setting?

• Process of defining the frame of reference for the interpretation of test scores in criterion-referenced assessments, or

• The matching of achievement levels to test scores

• Involves both quantitative and qualitative judgment of test items and expected student performance


Standard Setting Process

• Performance review meeting - January 2004• Grades 5 and 8 FCAT Science Standard

Setting meeting held with educators - January 2005

• Grade 11 FCAT Science Standard Setting meeting held with educators – September 2005

• Educator committees represented schools and districts from across the state


Standard Setting ProcessTable and Group Discussions focused on:

•Desired student content knowledge and skills of the SSS benchmarks

•How much content knowledge and skills are necessary for each of the achievement levels

Materials used during the meeting included:• Performance descriptions from previous

meeting• FCAT Achievement Level descriptions• Test items ordered by difficulty (based on

student performance)


Setting of Achievement Levels:

What Happens NextFCAT Science Achievement Level Timeline• Business Panel meeting – November 2• Staff review of input and recommendations• Draft of State Board of Education Rule• Regional workshops – November• Rule development workshop• Workshop with State Board of Education• Advertisement of Rule• State Board of Education Rule Adoption -

February• Reporting of student scores by Achievement

Level - May 2006


Student MotivationStudent MotivationPrevious discussions on Grade 11

student motivation included:• Requiring a specific score on FCAT Science

for graduation • Requiring a specific score on FCAT Science

OR four science courses for graduation• Science endorsement for students

receiving a specific score on FCAT Science


InterpretiveInterpretiveProductsProducts


FCAT Interpretive Products

FCAT Handbook— A Resource for

Educators• Program-level

information about the FCAT

• Posted to DOE website• Copies are now in

districts


FCAT ResourcesDepartment of Education FCAT

Web Addresses http://www.firn.edu/doe/sas/fcat.htm

or http://www.fldoe.org• FCAT Publications (General Public)• FCAT Publications (Educator)• FCAT Scores and Reports• Content Focus Reports for 2004 and

2005 tests(2006 Report to be posted in May)

http://www.firn.edu/doe/sas/fcat.htm


Some FCAT Publications Available

• Test Design Summary Details about the structure of the test,

including number of items, time limits, and session breakdowns

See handout provided or check the DOE website

• Content Focus Reports• For each content or reporting cluster• More specific description of benchmark

content assessed in each reporting cluster, each year


FCAT Interpretive FCAT Interpretive ProductsProducts

Spring DeliveriesSpring DeliveriesMid May 2006• Florida Inquires! (Grades 5, 8, 11)• Florida Reads! (Grades 4, 8, 10)• Florida Solves! (Grades 5, 8, 10)• Florida Writes! (Grades 4, 8, 10)• FCAT Results Folder for parents to keep track

of various reports• See 2005 – 2006 FCAT Interpretive Products

Shipment and Delivery Schedule handout for more information

Released FCAT Tests

What is available now? What are the plans for future

release?Where to find more information?


Tests Released in 2005Reading and Mathematics

Grade 4 and 82005 Operational Tests

Grade 10 ReadingMarch 2004 Retake version

Grade 10 Mathematics2004 Operational version


Future plans (if resources are available)

Tentative Schedule for FCAT ReleasesTentative Schedule for FCAT Releases

Fall 05Fall 05 Fall 06Fall 06 Fall 07Fall 07 Fall 08Fall 08

ReadingReading 4, 8,4, 8, and 10and 10 3, 7, and 93, 7, and 9 5 and 65 and 6 Release the Release the same tests same tests as in 2005as in 2005MathematicsMathematics 4, 8,4, 8, and 10and 10 3, 7, and 93, 7, and 9 55 and 6 and 6

ScienceScience NoNo NoNo 5 and 85 and 8 1111Note: For the fall 2005 release, the Grade 10 tests were used in 2003-04 Note: For the fall 2005 release, the Grade 10 tests were used in 2003-04 and the Grade 4 and 8 tests were used in 2004-05. and the Grade 4 and 8 tests were used in 2004-05.


Where to Find More Information

• Additional information about released tests is available at www.fldoe.org

• Release includes:• Fact Sheet – “How to use Released Tests”

and how not to use • Test questions formatted like the FCAT • Includes all test items that contribute to a

student’s score• Answer Key• Frequently Asked Questions (and answers)• And more

http://www.fldoe.org/


2005-06 Testing Schedules•FCAT Writing+: February 7-8•FCAT SSS (Reading, Mathematics,

Science): February 27-March 10

•FCAT NRT: same period as FCAT SSS

•Summer Reading and Mathematics Retakes:

June 19-23

FCAT UpdateFCAT UpdateWhat is new for

Writing+, Reading, and Mathematics?


FCAT WritingFCAT Writing++

February 2006February 2006Prompted Essay Test with Prompted Essay Test with Multiple-Choice (MC) Multiple-Choice (MC)

ItemsItems


FCAT WritingFCAT Writing++Writing and Item TypesWriting and Item Types

• Cloze Cloze SelectionSelection

o ConventionsConventionso 3-option MC3-option MC

• Stand-alone Stand-alone StructureStructure

o ConventionsConventionsº 3-option MC3-option MC

•Writing PlanWriting Plan ºFocus and OrganizationFocus and Organizationº4-option MC4-option MC•Writing Writing Sample Sample ºFocus, Organization, Focus, Organization, SupportSupport

º4-option MC4-option MC


FCAT WritingFCAT Writing++

•Standard Setting no earlier than Standard Setting no earlier than Fall 2006Fall 2006

•FCAT Writing+ will count for FCAT Writing+ will count for graduation requirement graduation requirement beginning with the graduating beginning with the graduating class of 2010class of 2010


Reading and Reading and MathematicsMathematics

Reading Test Item Reading Test Item SpecificationsSpecificationson hold until the SSS revisions complete

Mathematics Test Item SpecificationsReleased on DOE Website in October


Cognitive ComplexityPrior to 2004: based upon Bloom’s Taxonomy

Level I and Level II Considers instructional background and students’

approach to a problemMovement to “Cognitive Complexity” to classify test

items. Low, moderate, and high Based upon Webb’s depth of knowledge and NAEP

system Considers the cognitive demand of a test item


Based on recall and recognition of previously learned concepts and principles

Examples include:• Identifying common examples or recognizing a

concept• Recognizing a standard scientific representation of a

simple phenomenon• Retrieving information from a chart, table, diagram, or

graph• Calculating or completing a familiar single-step

procedure or equation using a reference sheet

Low Complexity Items


Require students to decide what to do using informal methods of reasoning and problem-solving strategies from various domains

Examples include:• Applying or inferring relationships among facts,

terms, properties, or variables• Describing examples and non-examples of

scientific processes or concepts• Predicting the next logical step

Moderate Complexity Items


High Complexity ItemsRequire students to solve non-routine or

unique problems, require abstract reasoning, analysis, judgment, or creative thought

Example include: • Generalizing or drawing conclusions• Analyzing an experiment to identify a flaw

and propose a method for correcting it


FAST Conference Presentation

Florida Inquires! and the Grade 5 FCAT Released Performance Task

Presented by Roberta McDonald, FCAT Science Development Team

Saturday, November 5 – 9:00 a.m.


QUESTIONS??QUESTIONS??Contact Information:Contact Information:Linda OravetzLinda OravetzFCAT Science CoordinatorFCAT Science CoordinatorTest Development CenterTest Development CenterFlorida Department of EducationFlorida Department of [email protected]@mail.leon.k12.fl.us

florida association of science supervisors presentation november 3, 2005 orlando, florida

Documents