January 2018 Agenda Item 07 Attachment 2 - Meeting Agendas (CA State Board of Education)

***DRAFT*** CAASPP Scope of Work B

for the 201819 and 201920 Administrations

CDE Agreement #CN150012

pptb-adad-jan18item07

Attachment 2

Page 312 of 312

This is Item 07 Attachment 02 from the California State Board of Education (SBE) Meeting Agenda for January 2018 posted at http://www.cde.ca.gov/be/ag/ag/yr18/agenda201801.asp.The document starts following this initial page.

DRAFTCAASPP Scope of Work Bfor the 201819 and 201920 Administrations

***DRAFT*** CAASPP Scope of Work B

for the 201819 and 201920 Administrations

CDE Agreement #CN150012

pptb-adad-jan18item07

Attachment 2

Page 1 of 312

Table of ContentsTASK 1: Comprehensive Plan & Schedule of Deliverables61.1. Schedule of Deliverables (formerly referred to as the Work Plan), Narrative Schedule, and Timeline71.1.A. Project Management Plan81.2. Orientation Meeting111.3. Management Meetings111.3.A. Weekly Meetings111.3.B. Annual Meetings121.3.C. State Board of Education (SBE) Meetings121.3.D. Technical Advisory Group (TAG) Meetings131.4. Coordination, Continuous Improvement, and Independent Evaluation131.4.A. Coordination with the Smarter Balanced Assessment Consortium and CDE/SBE Entities and Staff131.4.B. Development of Plan for Continuous Improvement141.4.C. Coordination with the Independent Evaluator151.4.D. Responding to Concerns161.5. Transition of Contracts161.6. Records and Minutes161.7. Monthly Accomplishments Reports and Weekly Management Status Reports171.8. Document Format and Style191.9. CDE Notification and Approval Schedule19TASK 2: Program Support Services232.1. Coordinators232.2. Administration Management System LEA Support242.3. Data Driven Improvement252.4. Technical Assistance Center262.5. Student Accessibility Tool302.6. Internet Resource Site302.7. Workshops and Webcasts312.8. Local Assessments: Smarter Balanced Interim Assessments and Digital Library322.8.A. Smarter Balanced Interim Assessments332.8.B. Digital Library of Formative Assessment Resources332.9 CAASPP Science Academy33TASK 3: Technology Services613.1. Special Requests for Smarter Balanced Paper Tests623.2. Assessment Delivery System623.2.A. Project Management Plan633.2.B. System Requirements63TASK 4: Test Security804.1. Test Security Plan804.2. Test Administration Monitoring854.3. Investigating Security Breaches87TASK 5: Accessibility and Accommodations895.1 Accessibility Plan for Computer-Based and Paper-Pencil Tests895.1.A. Computer-Based Tests895.1.B. Special Version Paper-Pencil Tests of Smarter Balanced Summative Assessments935.2. Unlisted Resources95TASK 6: Assessment Development976.1. Assessment Design976.2. Item and Task Development1056.2.A. Pilot Testing1116.2.B. Field Testing1116.2.C. Forms Construction1126.3. Standard Setting (excluding Smarter Balanced assessments)1136.4. Test Administration System Familiarization1206.5. Annotations to the Practice Test Items (Excluding Smarter Balanced Assessments)1236.6. Analysis of Test Results1246.7. Item Banks1266.7.A. California Item Bank1266.7.B. Item Bank for the CAASPP Assessments1266.8. Activities in Support of Future Assessment Development128TASK 7: Test Administration1347.1. CAASPP Test Administration Requirements1367.1.A. Manuals and Context-Sensitive Help1367.2. Paper-Pencil Administrations1417.3. Computer-Based Assessments1457.4. Contracting with LEAs for STS for Dual Immersion-Programs155TASK 8: Scoring and Analysis1568.1. Scoring1568.1.A. Methods of Scoring1588.1.B. Smarter Balanced Interim Assessment Scoring1708.1.C. Cumulative Scores1718.2. Analysis of Test Results1728.2.A. Item, Calibration, Scaling, and Test Analyses1748.2.B. Summary Analysis1768.2.C. Replication of Analyses178TASK 9: Reporting1809.1. Reporting to Local Educational Agencies1819.2. Reporting to the CDEPublic Reporting Web Site1909.3. Data Files1959.4. Secure File Transfer System1979.5. Technical Report1989.6. Other Analyses or Reports200Appendix ASample Program Schedule202Appendix BReporting Expectations for Special Studies and Research Projects268Appendix CMinimum System Requirements271Appendix DSummary of Embedded Universal Tools, Designated Supports, and Accommodations Supported by the CAASPP 201718 Test Delivery System288Appendix EGlossary of Terms303

List of Tables

Table 1. Planned CAASPP Training Workshops and Webcasts34

Table 2. High-level Test Development Timeline for CAST and CAA for Science98

Table 3. High-Level Test Development Timeline for CSA104

Table 4. Summary of Process for Item and Task Development for California Assessments105

Table 5. High-Level Timeline of Standard Setting Activities by Assessment113

Table 6. Sample Panel Configuration for CSA Content- and Grade-Specific ALD Workshops115

Table 7. Sample Panel Configuration for CAST and CAA for ScienceContent- and Grade-Specific ALD Workshops115

Table 8. Panel Configuration for CSA Standard Setting Workshops117

Table 9. Panel Configuration for CAST and CAA for Science Standard Setting Workshops117

Table 10. Planned Availability of the Practice and Training Tests, by Assessment122

Table 11. Distribution Plans for the CAASPP Assessments135

Table 12. Estimated CAASPP Test Takers135

Table 13. CAASPP Manuals138

Table 14. Secure Browser Support Policy for Operating Systems146

Table 15. Web Browser Support Policy147

Table 16. Summary of Assessment Delivery Components148

Table 17. 201718 Security and Test Administration Incident Reporting System(STAIRS) - DRAFT152

Table 18. Performance Task and Constructed-Response Scoring of Summative Assessments, by Content Area and Grade160

Table 19. High-level Timeline for Reporting Test Results, by Assessment180

Table 20. CAASPP Test Results Provided to LEAs Operational Summative Assessments Only182

Table 21. CAASPP Test Results Provided to the Public191

Table 22. Planned CAASPP Technical Report for Summative Assessments by Administration198

Table 23. Minimum System Requirements271

List of Figures

Figure 1. CAASPP Assessment Technology Platform61

Figure 2. CAASPP Assessment Delivery System65

Figure 3. Scalable Architecture of the CAASPP Assessment Delivery System66

Figure 4. Overall Schematic of the Test Delivery System149

Figure 5. Computer-based Test DeliveryScoring and Reporting Flow156

Figure 6. Paper-pencil Test DeliveryScoring and Reporting Flow157

TASK 1: Comprehensive Plan & Schedule of Deliverables

Task 1 describes the activities, assumptions, and requirements to manage and administer the California Assessment of Student Performance and Progress (CAASPP) System for the 201819 and 201920 school years and builds on the work completed during the first three years of the contract. Educational Testing Service (ETS) is the prime contractor and is responsible for the overall management and administration of the services provided to the state under this contract and will work closely with the California Department of Education (CDE) to ensure the success of the CAASPP administrations in the next three years.

The Amendment 4 referred to the new CAASPP assessments as the California Alternate Assessments, the California Next Generation Science Standards (CA NGSS and CA NGSS Alternate), and the primary language assessments. The amendment refers to the active CAASPP assessments and tools as the following:

Smarter Balanced Digital Library

Smarter Balanced Interim Assessments

Smarter Balanced Summative Assessments

California Alternate Assessments for English-Language Arts and Mathematics (CAA for ELA and mathematics)formerly known as CAA

California Science Tests (CAST)formerly known as CA NGSS

California Alternate Assessments for Science (CAA for Science)formerly known as CA NGSS Alternate

California Spanish Assessments (CSA)formerly known as the primary language assessments in Spanish

While ETS further details individual roles and responsibilities within the Scope of Work (SOW), the following text provides a high level summary of responsibilities for ETS and its partners:

ETS will manage the administration, scoring, and reporting activities and have overall responsibility for the constructed-response human scoring and artificial intelligence (AI) scoring. In addition, ETS will manage the logistics and coordination of all management meetings, along with the development of all relevant materials. ETS will also provide Help Desk services and psychometric support. ETS will provide item development for all state-specific assessments: CAST, CAA for ELA and mathematics, CAA for Science, and CSA. ETS will host and provide support for the Test Operations Management System (TOMS).

American Institutes for Research (AIR) will provide hosting and support for its test delivery system and online reporting system, a component of the overall CAASPP Assessment Delivery System. These are the same systems used in the successful spring 2014 Field Test and in the first three operational administrations of CAASPP.

Measurement Incorporated (MI) will assist ETS in the constructed-response scoring for various grades for all Smarter Balanced-related assessments, including human and AI scoring.

WestEd will provide training to local educational agencies (LEAs) and educators about CAASPP with the goal of supporting classroom improvements using the assessment information. WestEd activities will include but are not limited to: developing and facilitating training workshops, materials, and videos.

Red Dog Records (RDR) will serve as the programs multimedia