preparing for 2015 smarter balanced summative … · preparing for 2015 sbac 2 . pretest ... k–12...
TRANSCRIPT
Preparing for 2015 Smarter Balanced Summative Assessments
Rick Phelan
February 24, 2015
Sonoma County Office of Education www.scoe.org/sbac
Appreciations to Rocklin Unified School District and the California Department of Education for slide deck elements.
Goals Participants will:
o Understand roles & responsibilities for administering the 2015 Smarter Balanced Summative Assessments
o Obtain tools to support scheduling and logistical planning o Understand student access options o Gain materials to prepare test administrators/proctors o Learn about communication resources for students & parents o See an overview of the Test Operations Management System
2 Preparing for 2015 SBAC
Pre-‐‑Test
Preparing for 2015 SBAC 3
SBAC Test Design
Preparing for 2015 SBAC 4
Smarter Balanced Assessment System
Common Core State Standards
specify K–12
expectations for college and career readiness
All students leave
high school ready for college
and career
Teachers and schools have
information and tools they need to improve teaching
and learning Interim assessments
flexible, open, used for actionable feedback
Summative assessments benchmarked to college
and career readiness
Formative assessment tools and practices
for teachers to improve instruction
Preparing for 2015 SBAC
Smarter Balanced Computer Tests • Test Administrators Directions for Administration
(DFA) are located at: http://caaspp.org/administration/instructions/index.html
• Grades 3-8 and 11 in ELA and Math The only students exempt from participating in the test are:
• Students participating in the Alternate Assessment • English Language learners who have been attending schools in
the United States for less than 12 months at the time of the test
• Computer-adaptive test (CAT) in each content area • Performance Task (PT) in each content area
• Classroom activity to familiarize students with question content • Students respond to the PT through online platform
6 Preparing for 2015 SBAC
Smarter Balanced Item Response Types • Multiple choice
– one correct response – multiple correct responses – two-part
• Matching tables – yes/no – true/false
• Fill-in tables • Select or order text or graphics • Drag and drop • Graphing • Equation or numeric response • Short text • Long essay
SBAC Practice Tests
http://www.scoe.org/pub/htdocs/sbac-practice.html
8
• Allow students and administrators to become familiar with and experience • Grade-level assessments • A variety of grade-specific items and difficulty levels • Performance Tasks • Format and structure of the operational
assessments • Allow students to become familiar with accessing
the test (log-in process), testing software and navigational tools.
Preparing for 2015 SBAC
Interim Assessments
9
• The Smarter Balanced Interim Assessments are comprised of interim comprehensive assessments (ICAs) and interim assessment blocks (IABs)
• ICAs - Examples of the use include:
• Beginning of the year a student from another state is given the previous year’s ICA.
• Mid-year a teacher gives an ICA to gauge how students might perform on the summative assessment.
• Scoring Protocol for Interim Assessments will be available at the end of February.
Preparing for 2015 SBAC
Interim Assessments (cont.)
10
• The Smarter Balanced Interim Assessments are comprised of interim comprehensive assessments (ICAs) and interim assessment blocks (IABs)
• IABs - Assess fewer sets of skills, and: • Use the same targets, by grade level, as the
summative blueprints. • Consists of short, focused sets of items. • Provide information about a student’s strengths and
needs in relation to the assessment targets. • Offer varied blocks by grade level and subject area.
Preparing for 2015 SBAC
SBAC Roles & Responsibilities
Preparing for 2015 SBAC 11
Roles and Responsibilities
12
District Test Coordinator (DC) • Designate Site Coordinators (SCs), add them to the Test
Operations Management System (TOMS) • Promote understanding of SBAC with stakeholder groups • Coordinate with SC’s so that tests are administered in the
appropriate grade(s) and content area(s) • Set appropriate supports and accommodations in TOMS • Ensure that the SC’s and TA’s are appropriately trained
regarding test administration and security policies • Report test security incidents to CDE • Work closely with Computer Lab Tech/Site Tech Coordinators • Relay issues/questions to the CalTAC call center
Preparing for 2015 SBAC
Roles and Responsibilities School Test or Site Coordinators (SC)
• Collect security affidavits from Test Administrators and retain at test site • Train TA’s and verify their accounts on TOMS
• Promote understanding of SBAC with stakeholder groups
• Verify supports and accommodations in TOMS minimum of three days before your testing window
• Print student log-in strips/cards (confirmation code and SSID)
• Confirmation code is the student’s first name from SIS
• If you have a new students, notify your district CALPADS engineer
• Ensure that testing at the school sites is conducted in accordance to test security policies
• Create/approve test schedules and procedures for the school.
• Monitor testing and address problems, as needed
• Report testing security incidents to the DC
13 Preparing for 2015 SBAC
Roles and Responsibilities Test Administrator (TA)
• Completely read and sign security affidavit • Promote understanding of SBAC with stakeholder groups • Complete test administration training • Log in and verify access to the testing system using the CAASPP
account sent via email • View student information prior to testing, to ensure it is correct • Ensure that appropriate designated supports and
accommodations have been set prior to test • Set up and proctor testing • Report all potential test security to the SC and District CAASPP
Coordinator
14 Preparing for 2015 SBAC
Technical Support Staff • Read and sign security affidavit • Complete test administration training • Ensure all student computers can launch and run
secure browser application • Ensure bandwidth test is completed at each school
prior to testing windows • Ensure all student computers have functional
headphones • Assist the TA with set up and test proctoring • Increase comfort levels and reduce anxiety of
students & staff
15
Roles and Responsibilities
Preparing for 2015 SBAC
SBAC Technology
Preparing for 2015 SBAC 16
Technology Setup
17
• Insure devices are operational • Secure browsers properly installed a minimum
2 weeks before testing • Necessary hardware available and working
(headphones, external keyboards, etc.) • Content filters, firewalls, and proxy servers
configured • Stable (wired or wireless) internet connection
Preparing for 2015 SBAC
Secure Browser
18
The Secure Browser is: • Specifically designed for CA to provide
secure access • Prevents students from accessing other
hardware and software functions NOTE: Test Administrator computers will use a standard Web browser Preparing for 2015 SBAC
Secure Browser
19
• Students must use the browser to log on to take a summative test
• Prior to testing, TA’s should verify that:
• External user applications are closed (browser will not work if they’re on)
• Testing does not occur on computers with dual monitors
Preparing for 2015 SBAC
Preparing for 2015 SBAC 20
Technology Tools & Resources http://www.scoe.org/pub/htdocs/sbac-technical.html
SBAC Scheduling Issues
Preparing for 2015 SBAC 21
Smarter Balanced Computer Based Tests
Time Estimates
Preparing for 2015 SBAC 22
Smarter Balanced Test Schedule
• Overall testing will take approximately 7 to 8.5 hours • Recommended: Non-PT first, PT second (Performance
Task) • Classroom activity should be completed within 3 days
prior to starting PT • ELA PT’s have 2 parts and can be completed in 2 days • Math PT’s have 1 part and can be completed in 1 day • Recommended: PT should not be administered on the
same day as the non-PT
23 Preparing for 2015 SBAC
Smarter Balanced Test Schedule (continued)
• SC’s must ensure that TA’s know which students take which tests
• Schedule activities for students not taking exam or for those who finish early (eg. may read a hardcopy book
• A students CAT test remains active until the student completes and submits the test or 45 calendar days after the student has begun the CAT
• A student’s PT remains active for 10 calendar days after the student has begun.
24 Preparing for 2015 SBAC
2015 SBAC Scheduling Estimator
Preparing for 2015 SBAC 25
hFp://www.scoe.org/files/sbac-‐‑estimator-‐‑2015.xls
SBAC System Operations
Preparing for 2015 SBAC 26
Three Systems Connect to Deliver
Tests to Students
27
CALPADS • Student enrollment information • Student demographics
TOMS • Manage test users • Manage test windows • Manage student supports • Student rosters
Smarter Balanced • Verify student test settings • Select student tests • Test delivery portal
Preparing for 2015 SBAC
CALPADS
28
• Sole source for student demographics and program participation data for TOMS
• Districts cannot update student demographics or program
participation data in TOMS; must be updated in CALPADS first
• Contact your CALPADS administrator to correct student grade level or demographics
• Data from CALPADS is rolled to TOMS daily and the turnaround time is about 48 hours.
• If a new student enrolls during the District Testing window contact your CALPADS administrator to get them set up in TOMS Preparing for 2015 SBAC
Test Operations Management System (TOMS)
29
• TOMS is a password-protected Web site that provides administrators with tools to manage users and students participating in Smarter Balanced assessments
• Repository for student information, including initial test settings and
accommodations • Different levels of user access (DC, SC, TA) • District and school staff must set designated supports and
accommodations for students in TOMS prior to testing • In TOMS, DCs and SCs can
• Add and manage users • View student roster and update student test settings
Preparing for 2015 SBAC
30
• District and site coordinators authorize users for the system. • Authorized users receive an email with login instructions and must
activate their login before the temporary password expires. • Usernames and passwords used to access last year’s TIDES system
are no longer active. • A manual on how to use TOMS is available:
http://goo.gl/GCSgBv
Accessing TOMS Step 1. hFp://californiatac.org/ Step 2. Log In Step 3. Navigate the System
Preparing for 2015 SBAC
Operating an SBAC Test Session STEP 1: Preparation • Prepare computers
o Student computers (No testing on computers with dual monitors ) • A secure browser must be installed on all student computers/devices (updated
annually) o Test Administrator computers
• Use a standard Web browser. • Prior to testing, TAs should verify on student computers that external user
applications are closed to allow the lockdown browser to work • Secure browser available for download through the portal:
http://ca.browsers.airast.org/
• Prepare student login information o SSID, and confirmation code (CALPADS student first name) for login.
• This may be provided on a card or piece of paper. NOTE: Student personal information is secure and must be collected and securely destroyed at the end of a test session.
• Verify student test settings • Ensure that the correct student test settings have been uploaded to TOMS (see the
later section on accommodations and designate supports)
31 Preparing for 2015 SBAC
Smarter Balanced- Additional Features Test Settings and Accommodations • Before you approve students to test, you can
review student test settings. • Only DC and SC may change certain settings • If changes are made, select [Set] to confirm
the current test settings and return to the list of students awaiting approval. You will still have to approve the student for testing.
• Select [Set & Approve] to establish the updated settings and approve the student for testing.
• Select [Cancel] if you want to return to the Approvals and Student Test Settings screen.
• Note: Translations (stacked) (for math items) and turning off universal tools only occurs in the test delivery system.
32 Preparing for 2015 SBAC
Operating an SBAC Test Session
STEP 2: TAs Log into the system and generate a test session o Go to http://californiatac.org/ and select “Test Administrator Interface.” o Enter your user name and password. o Select the test(s) you will be administering during that session and click on Start
Session. o Write the test Session ID on the board or large paper where students can see it.
Keep a record of this test session ID in the event that the session suddenly shuts down.
o Do NOT navigate away from the TA interface page. The session will stop and all students will be logged out.
o Read the testing script from the Test Administrator Manual o Provide students with their SSID, and confirmation code (CALPADS student first
name) for login. o Make sure students know that the Test Session ID must be entered exactly as it is
written, without extra spaces or characters.
33 Preparing for 2015 SBAC
Operating an SBAC Test Session
o Step 3: Students log into the system
• To log in to the Smarter Balanced test, each student will need:
• Confirmation Code: Student’s legal first name as spelled in CALPADS
• State-SSID: State abbreviation (CA) followed by a hyphen and the student’s Statewide Student Identifier
• Session ID
• Students will select their test • Students must wait for TA approval for entry
into the test session • Students must then verify their test
34 Preparing for 2015 SBAC
Operating an SBAC Test Session
Step 4: Approving Students for Testing
• TAs can either select [Approve] for each individual student (recommended) or select [Approve All Students]. • Reminder: If any student’s test settings are incorrect, do
NOT approve that student. • To refresh the list of students awaiting approval at any time,
select the [Refresh] button at the top of the pop-up window. Do not use the browser refresh button.
35 Preparing for 2015 SBAC
Operating an SBAC Test Session Step 5: Students Begin their Test • Students will be taken through a sound check
verification (two if text to speech is activated) • Followed by a Test Instructions and Help screen
36 Preparing for 2015 SBAC
Operating an SBAC Test Session STEP 6. Monitoring Student Progress • The table displays:
• Students who have logged in and been approved for testing. • Test completion Status • Print requests • Paused tests
• Additional tests can be added to a session; you will receive a confirmation message box
37 Preparing for 2015 SBAC
Ending an SBAC Test Session
38
STEP 7: Ending the Test Students complete the test:
TA ends the test session: To stop the ENTIRE session (and pause tests for all students in the session):
– Select the [Stop Session] buFon in the upper-‐‑left corner of the screen. An “Important!” box will appear, requesting verification to end the session and log students out.
– Select [OK] to continue or [Cancel] to keep the test session open. Users should exit or log out of the Test Administrator Interface only after stopping the test session.
Preparing for 2015 SBAC
39
Pausing and Inactivity Rules • Through the TA Interface, TAs may:
• Stop an entire session; or • Pause individual student tests.
• Users should exit or log out of the Test Administrator Interface only after stopping the test session.
• If there is a technical issue (i.e. power outage or network failure), students will be logged out and the test will automatically be paused.
• Regardless of when or how users log out or navigate away from the Test
Administrator Interface, student data will NOT be lost.
• Note: If a test is paused for any reason or length of time, the student must log in again to resume testing. Highlighted text and item notes will not be preserved on the computer adaptive tests.
• There is no pause option for the PT (performance task). Note for ELA (2
parts) a student cannot go pack to part 1 after completing part 2.
Do NOT stop tests without forewarning students
Preparing for 2015 SBAC
SBAC Pausing and Inactivity Rules
• As a security measure, students are automatically logged out of the test after 20 minutes of inactivity. When this occurs, students are:
• Presented with the test page containing the test item he or she was last working on (if page contains at least 1 unanswered item) OR
• Presented with the next test page (if all items on the previous test page were answered); • NOT permitted to review or change any test items on previous test pages.
• Activity means: • Selecting an answer • Using a navigation option in the test (i.e. selecting [Next] or [Back], using the Past/Marked
Questions drop-down list) • NOTE: Moving the mouse or selecting an empty space on the screen is NOT considered
activity.
• Before the system logs out, a warning message will be displayed. • If a student does NOT select [Ok] within 30 seconds, the student will
be logged out. • Selecting [Ok] will restart the 20 minute inactivity timer.
40 Preparing for 2015 SBAC
General Rules & Conditions for
SBAC Student Responses • Students must answer all test items on a page before moving
on to the next page • Students may review already answered questions before
submitting test • Students may not return to a test segment once it has been
completed and submitted • Students must answer all test items before the test can be
submitted • Students may not return to a test once it has been completed
and submitted • There is a difference between a “segment” and a “session” • Expiration and pause rules vary between PT and non-PT
Preparing for 2015 SBAC 41
Administration of the Classroom Activity and
Performance Task
Classroom Activity Administration
Guidelines • Classroom Activity Purpose: To provide students
with important context, concepts, and key terms prior to taking the PT.
• For each content area, all students will have a classroom activity before they complete the PT.
• May be administered in a classroom or any other appropriate space.
• May take place on a separate day from the PT. • Recommended: No more than a 3-day lapse between
the classroom activity and the PT administration. • TA may need a chalkboard or dry-erase board.
43 Preparing for 2015 SBAC
Classroom Activity Administration
Guidelines (cont.) • Computers, projectors, and other technology
are allowed but not required for the classroom activity.
• Involves the participation of all students in an instructional task.
• PT will be completed by individual students in the computer-based test delivery system.
• Students may take notes during the classroom activity, but the notes must be collected before proceeding to the PT.
44 Preparing for 2015 SBAC
Classroom Activity Administration
Guidelines (cont.) • Designed to fit into a 30-minute window.
• Will vary due to complexity of topic and individual student needs.
• The SC should download the assigned classroom activity and provide to TAs at least 1–2 days prior.
• Download from http://californiatac.org/ • After receiving the activity from the SC, the TA or
classroom teacher should administer the classroom activity.
• Provide the appropriate accommodations for students who normally use them during instruction.
45 Preparing for 2015 SBAC
Classroom Activity and Performance
Task Administration Ø Step 1: Identify the classroom activities for your school.
Ø Step 2: Download the classroom activity.
Ø Step 3: Review the teacher directions for the classroom activity carefully.
Ø Step 4: Complete the classroom activity. Ø Step 5: Administer the PT to students online. The students must
work independently
46 Preparing for 2015 SBAC
Universal Supports, Designated Supports, &
Documented Accommodations
Accessibility and Accommodations
Three categories of support… • Universal Tools:
o Available to all; based on student preference and selection (e.g., breaks, strikethrough, highlighter).
• Designated Supports: o Need to be indicated by an educator or group of educators (e.g., color contrast,
read aloud, glossary, magnification).
• Accommodation: o Based on individualized education programs (IEPs) or Section 504 plans (e.g.,
American Sign Language, scribe).
48 Preparing for 2015 SBAC
Accessibility and Accommodations Two systems of delivery Ø Embedded supports: digitally delivered through the
technology platform(i.e., digital notepad, calculator) MUST be activated in TOMS prior to testing.
Ø Non-embedded supports: provided by the District(i.e.,
adapted keyboard, multiplication table) but are still noted in TOMS
49 Preparing for 2015 SBAC
Accessibility and Accommodations
50 Preparing for 2015 SBAC
Accessibility and Accommodations • Questions:
• How do I know what supports are available? • How do I match the IEP and 504 supports to the supports listed?
• Answers and Resources: • Matrix One: Universal tools, designated supports, and accommodations for the
CAASPP system http://www.cde.ca.gov/ta/tg/ai/documents/caasppmatrix1.pdf • Shows tables of all available items by exam and level of support
• Smarter Balanced Usability, Accessibility, and Accommodation Guidelines • Provides details about each of the supports and contexts in which they are
appropriate • Describes who is eligible for them and guides the decision making process • Detailed Q & A section in an appendix
• ISAAP Tool & SEIS • Helps staff select and organize appropriate support categories • Summarizes students’ supports that can be uploaded to TOMS
51 Preparing for 2015 SBAC
Accessibility and Accommodations • Not all tools, supports, and accommodations are available for all
exams or to all grade levels. For example: o ELA – a dictionary is not allowed on short paragraph responses but are allowed on the PT
long essay. o ELA - read-aloud of passages or text-to-speech is not allowed on passages for students
in grades 3-5. o Math – multiplication tables are not allowed in grade 3.
• When an IEP or 504 specifies that a “disallowed” tool is required for a student to participate in testing, special permission must be granted to use these “Individualized Aids.” o The District CAASPP Coordinator or the CAASPP Test Site Coordinator must submit the
CAASPP Individualized Aid Request Form. Approval to use these is granted by the California Department of Education.
o When an Individualized Aid changes the construct being measured, the scores for that student will not count towards the participation rates. This is similar to the “modifications” policy under the prior STAR program – the student gets a score report, but the score is not used for accountability.
52 Preparing for 2015 SBAC
Accessibility and Accommodations Planning Ahead
• Determination of which accommodations an individual student will have available for the assessment is necessary because these accommodations must be made available before the assessment, either by entering information into the TOMS for embedded accommodations, or by ensuring that the materials or setting are available for the assessment for non-embedded accommodations.
• Accommodations could increase cognitive load or create other challenges for students who do not need them or who have not had experience using them. It is wise for students to practice the tests using the selected tools.
• Parents/guardians should be made aware of the conditions under which their child participates in the assessment.
• Once the appropriate supports are identified, site and district coordinators will work together to ensure that they are uploaded to TOMS.
• TA’s are not authorized to change student test settings in TOMS • Prior to testing review the students test settings and verify that they are
correct.
53 Preparing for 2015 SBAC
Accessibility and Accommodations Online Resources • Smarter Balanced Usability, Accessibility, and Accommodations Guidelines:
http://www.smarterbalanced.org/wordpress/wp-content/uploads/2014/08/SmarterBalanced_Guidelines.pdf
• Smarter Balanced Usability, Accessibility, and Accommodations Implementation
Guide: http://www.smarterbalanced.org/wordpress/wp-content/uploads/2014/03/Usability-Accessibility-and-Accommodations-Implementation-Guide.pdf
• Guidelines: Frequently Asked Questions:
http://www.smarterbalanced.org/wordpress/wp-content/uploads/2013/12/SmarterBalanced_Guidelines_FAQ.pdf
• Accessibility and Accommodations for CAASPP 2014–15: An Overview archived
Webcast http://caaspp.org/rsc/videos/archived-webcast_102214.html • Matrix One: http://www.cde.ca.gov/ta/tg/ai/documents/caasppmatrix1.pdf • Using the CAASPP Individual Student Assessment Accessibility Profile (ISAAP)
Tool video tutorial: http://caaspp.org/rsc/videos/archived-webcast_ISAAP-overview-and-demo.html
• Embedded Universal Tools and Online Features training module: http://www.brainshark.com/ets/vu?pi=zI9zwnui5zJ0cZz0&intk=37891690
54 Preparing for 2015 SBAC
Test Security
55 Preparing for 2015 SBAC
56
Test Security There are 3 different types of test security incidents • Impropriety (Correctable and containable at local level)
o Unusual circumstance, low risk of affecting student performance, test security, or test validity
o Must be reported to District CAASPP Coordinator or Test Site Coordinator o Example: Students talking during testing
• Irregularity (Correctable and containable at local level) o Unusual circumstance, may affect student performance, test security, or test validity o Must be reported to District CAASPP Coordinator within 24 hours o Examples: Student cheating or providing answers to another; student accessing or using
electronic equipment (e.g., cellphone)
• Breach (External implications for the Consortium) o Event that threatens test validity (e.g., release of secure materials) o A breach incident must be reported to District CAASPP Coordinator within 24 hours o Examples:
• Administrator or coordinator modifying student responses or record at any time
• Administrator allowing students to take home test items, passage, prompts, or scratch paper
Preparing for 2015 SBAC
Resources
• Sonoma County Office of Education SBAC Support Pages http://www.scoe.org/sbac
• Manuals http://goo.gl/9MQhfl o Administration Manuals o Universal Tools, Designated Supports, and Accommodations o TOMS Users Guide o Administration of Paper and Pencil Tests
• Smarter Balanced Test login site http://californiatac.org/
57 Preparing for 2015 SBAC