assessment coordinator training tom corbett, governor ▪ ronald j. tomalis, secretary of education...

Assessment Coordinator Training

Tom Corbett, Governor ▪ Ronald J. Tomalis, Secretary of Education www.education.state.pa.us

2013 Spring PSSA Administration Training

1

2013 PSSA


Pennsylvania Department of Education

Data Recognition Corporation




2

• Introductions – PDE• General Overview – PDE• PSSA Overview – PDE• Test Security – PDE• PDE Security Training – PDE• Planning for the Assessment – PDE• Participation and Exceptions to Participation – PDE• Administering the Assessment – PDE• AYP – PDE• Review Key Dates – DRC• Roles and Responsibilities – DRC• Paper/Pencil Administration – DRC• Online Administration – DRC

Agenda




3

General OverviewAll policy decisions come from PDE

All PSSA logistics are the responsibility of DRC and include:

Delivery of Paper Materials eDIRECT System Insight System (Online test engine)




4

PSSA Overview

4

• Available to students either online or paper/pencil

• Multiple-Choice (MC) and Constructed-Response (CR) items




5

PSSA Overview• Students in grades 3-8 will be required to take the

PSSA. • PSSA results will be used for Adequate Yearly

Progress (AYP).• The 2013 PSSA will be based on the current AA/EC• Grades 3-5 will have Field Test items based on the

PACC• The entire PSSA will be based on the PACC in 2015




6

• PSSA and Keystone Exams

– Students in 8th grade or below who complete a course for which there is a Keystone Exam will take both the Keystone Exam and the PSSA.

PSSA Overview




7

PDE Policy – Test Security The assessments rely on the measurement of individual

achievement. Any deviation from assessment procedures is strictly prohibited and will be considered a violation of test security.

School administrators, teachers, and any other education personnel who are involved in the assessment program must maintain the security of all assessment materials at all times.

Because the assessment must remain secure, teachers/Test Administrators should not have assessment materials in their possession at any time other than during the actual assessment administration.




8

PDE Policy – Test SecurityTest Security Certifications• The Test Security Certifications must be signed after the

assessments are administered– Principal, District Assessment Coordinator, School Assessment

Coordinator, Test Administrator and Proctor (if applicable)

• Provided in Appendix D of Handbook for Assessment Coordinators

• All other individuals handling secure materials MUST have a signed School Assessment Coordinator Test Security Certification on file

• The Chief School Administrator must retain all Certifications for three years

• DO NOT return these forms to DRC




9

PDE Policy – Test SecurityTest Security Certifications• If an individual does not sign the appropriate Test

Security Certification, the Chief School Administrator must provide the Department of Education’s Bureau of Assessment and Accountability with a written explanation of the reason or reasons why the Certification was not executed. The explanation must provide a description of the action taken by the Chief School Administrator in response to the failure to execute the Test Security Certification.




10

PDE Policy – Test SecurityPAsecureID List Distribution• School assessment coordinators must generate a list(s)

of students by PAsecureID to show all students taking the PSSA.

• The list should include for each student the test administrator, school name, year, and subject being administered.

• All lists must be retained at the district/school for three years; and be available when requested by PDE.




11

PDE Policy – Test Security

Maintain test security:

Refer to PDE security guidelines located at

www.education.state.pa.us

Search for “Accommodations and Security”




12

Roles and Responsibilities• District Assessment Coordinator (DAC)

– Attend required PDE training

– Train school assessment coordinators

– Receive tests

– Inventory tests

– Distribute tests to school assessment coordinator(s)

– Collect tests from school assessment coordinator(s)

– Inventory, package, and return tests




13

Roles and Responsibilities

• School Assessment Coordinator (SAC)– Attend training offered by DAC

– Train Test Administrators (TAs) and Proctors

– Receive tests from DAC (ship to district sites) or DRC (ship to school sites)

– Inventory tests for school

– Put labels on test booklets (Do not open test booklets while doing this!)

– Create testing schedule




14

Roles and Responsibilities• School Assessment Coordinator (cont.)

– Distribute tests to TAs on the day and time scheduled for test administration.

– Monitor testing sessions– Collect tests from TAs– Make sure no prohibited materials are on the classroom or

hallway walls– Make sure tests are in a locked area with controlled and

limited access– Return all tests to DAC (ship to district sites) or DRC (ship

to school sites)




15


You are responsible for the security and proper administration of these

tests.




16


• Test Administrators– Be familiar with all testing protocols, including the

“Ethical Standards of Test Administration (formerly the “DO’s and DON’Ts”)

– Follow all testing protocols– Sign the Test Security Certification Statement– Make students familiar with testing procedures– Make students feel comfortable




17

Roles and Responsibilities• Test Administrators

– Answer any questions students may have about test directions

– Walk around the room during testing– Maintain a calm, quiet environment during testing– Collect and return test books to the SAC when

testing is complete– Report any suspected testing violations to the SAC

and to PDE




18

Chain of Custody

• Tests should be secure at all times!

• DACs should inventory the tests as soon as they arrive in the District and keep them in a locked storage area that has limited access.

• SACs should inventory the tests as soon as they arrive in the School and keep them in a locked storage area that has limited access.




19

Chain of Custody

• When distributing tests to TAs, the tests should be counted by the SAC in the presence of the TA

• When the tests are collected, they should be counted by the SAC in the presence of the TA

• Tests should be collected as soon as a testing session is completed unless another section is to be administered immediately




20

Chain of Custody

• Tests should be stored in a locked storage area. Only people who require access to the tests should have a key to the storage area.

• Samples of the test booklets are NOT to be furnished to the TAs as part of the training. TAs should NOT receive any test booklets until the test is to be administered.




21

Common ViolationsThe following are examples of testing violations. This is NOT an exhaustive list.•Misuse of accommodations (e.g., reading the Math test to the entire class; reading aloud the Reading test)•“Under” proctoring (e.g., inadequately monitoring the testing session)•“Over” proctoring (e.g., reminding a student without the appropriate accommodation to answer a specific question)




22

Common Violations

• Changing answers or encouraging students to change their answers.

• Discussing the test or test questions with individual students.

• Reading aloud a math question or answer choice when doing so cues the correct answer.




23

Common Violations• Not removing or covering instructional materials

in the testing room or the hallways.• Giving reminders to individual students or to the

entire class (e.g., “Remember, we covered this last week.”)

• Allowing students to possess cell phones during testing.

• Allowing devices in the testing room that take pictures.




24

Common Violations

• Copying or recording any part of the secure test materials

• Viewing secure test material by teachers, proctors, administrators, etc.

• Returning test materials to students to complete or enhance answers




25

Potential Consequences of Testing Violations

For Educators:

•Verbal reprimand

•Written reprimand

•Suspension with or without pay

•Termination

•Revocation of certification

•Criminal charges




26


For Students:

•Consequences for students violating test security (e.g., cheating, taking pictures of test questions) are up to the individual LEA.

•Students should be made aware of these consequences prior to testing.




27


For Students (cont.)

•If a student violates test security, it is the responsibility of the LEA to determine the consequences for the student. These consequences may include disciplinary action and/or the student retaking the test.

•ALL test security violations by students must be reported to PDE immediately.




28

Data Available to PDE- examples

• Student erasures

• How long it takes a student to answer an item when taking the test on a computer

• When answers are changed and how often they are changed on a computer

• Statistical analyses of student scores and performance




29

Cautions!

• Do actively proctor the test but don’t give any individual student help.

• Do encourage the class to check their answers but don’t encourage specific students to check their answers.

• Do report all prohibited actions.




30

Cautions!• Do not review student test or answer booklets (except as stated in

the Administration Manual and the Accommodations Guidelines).

• Do not assist, direct, or counsel students during the administration of the test in any way that would influence student answers.

• Do not provide answers to students.

• Do not alter, influence, or interfere with a test response in any way, fill in any unanswered item, or instruct the student to do so.

• Do not discuss or provide feedback regarding test items.

• Do not use the answer sheet as scratch paper or as a method to eliminate incorrect answer options.




31

To Report Suspected Violations, Contact The

Pennsylvania Department of Education at 717-787-4234




32

Additional Contact Information

• Jay Gift, Assessment Security Advisor– 717-783-1144– [email protected]

• Kerry Helm, Assessment Specialist– 717-783-6538– [email protected]

mailto:[email protected]





33

Planning for the AssessmentTesting Preparation DOs:• Plan for testing rooms and times• Post PA General Description of Scoring Guidelines• Math Formula Sheet(s)Provide test administrators with:• Manuals• Test Security Certifications (located in back of PSSA

Handbook)• A schedule and plan for secure distribution and collection of

materials after each session• Training to include correct labeling and coding




34

Planning for the AssessmentAccommodations

The Accommodations Guidelines for students with IEPs, ELL students, and all students are available on department’s web site – www.education.state.pa.us




35

Planning for the Assessment

Be prepared for parental requests to view the test. Parental review must be done with a paper assessment booklet.

Schedule small group setting(s). Schedule extended time setting(s). Assign test administrators for breaks and

movement to other setting(s).




36

PASA

Eligible grade 3-8 students can participate in the PASA for Math, Reading, and Science

If a student meets the criteria to participate in the PASA, he/she must also participate in a school or district provided alternate assessment for Writing




37

ELL students

ELL students are treated the same as every other student with the following exceptions:– Participation in the Math PSSA is mandatory but if

the student is in their first year of enrollment in a U.S. school their scores do not count for AYP purposes

– Participation in the Reading PSSA is optional for students in their first year of enrollment in a U.S. school




38

ELL students ELL students are treated the same as every

other student with the following exceptions:– Participation in the Science PSSA is mandatory for

students in their first year of enrollment in a U.S. school but scores do not count for AYP purposes




39

Administering the AssessmentDO:• maintain calm & quiet• remove or cover all

instructional materials• provide:

– blank scratch paper– sharpened pencils– highlighters

• encourage students• return all test materials to

secure location

DO NOT:• allow unsupervised breaks• display visual cues (e.g.,

picture icons)• provide pre-printed graphic

organizers• allow pens/colored pencils;

stray marks• prompt students• keep any test materials• use the answer sheet as

scratch paper




40

Make-ups

Make-up missed sessions as soon as possible

Make-ups for all subjects can be administered up to the end of the testing window




41

Important Information• The AYP Performance and Participation

Attribution Map is located on the PDE website:• Programs > Programs S-Z > State Assessment

System>Pennsylvania Accountability System (PAS) > 2013 AYP Score and Participation Attribution Map

• A district that has placed a student into an alternate educational setting inside or outside the district is responsible for ensuring the student’s participation in the assessment.




42

Important Information• The entity delivering the student’s education is

responsible for ordering and receiving materials and administering the test.

• Know your district’s procedures for Home Schooled students.

• Testing Cyber Charter School students.




43

Bubbling bookletsBubbling booklets may include:1. Filling in all information (for students who do not have a

Precode Label) • Demographic information provided on an answer

booklet will not override information on a precode label.– Please complete certain demographic information after the

test has been completed (e.g., economically disadvantaged)

And/or

2. Filling in fields that are applicable to the testing situation (i.e., Accommodations)




44

Assessment Coordinators:Ensure all information is bubbled correctly!

Demographic data– Assessment

Coordinator?• Race/ethnicity• Economic status

– Test administrator?– Other?

Testing situation data– Assessment

Coordinator?• Non-assessed codes

– Test administrator?• Absent without make

up• Accommodations

– Other?




45

Assessment Coordinators:• Training for those who will administer the

assessment is required.• Administrators must be a member of the

school/district staff. No volunteers.• This PowerPoint is available on PDE

website.• Utilize test administrator’s handbook.




46

Students who withdraw/transferStudent withdraws or transfers prior to the start of the testing

window• Paper pencil testers – Discard student label• Online testers – Remove student from test session

Student begins a test but is completely removed from the school’s enrollments before completing the entire assessment• Paper pencil testers – If the un-enrollment occurs before

the end of the testing window, Place a “Do Not Score” label on top of the student label on the booklet

• Online testers – Send an email with student information (PAsecureID and School Code) and an explanation to [email protected]




47

Encourage students to complete the test• Encouraging students to

meet only the minimum attempt logic may result in meeting the participation rate, but student’s score will be based on only those questions that were answered.

Encourage all students to do their best!




48

Participation• Does not negatively affect participation rate:

– Medical emergency• A recent physical or mental diagnosis that renders a student

incapable of participating in the assessment

• The emergency is occurring at the time of the testing window. Mark the booklet at the time of the testing window!

• If you’re unsure, CALL!!!

– ELL student in his/her first year of enrollment in a US school (Reading only)

– Student participated in the PASA




49

Participation

• Negatively affects participation rate– Extended absence during entire testing window– Parental request– Other – such as refusal to participate– Absent without make-up– Failure to meet attempt logic




50

Performance vs. Participation• Performance

– Only students enrolled for a full academic year (on or before October 1, 2012) count towards performance. ALL students must participate in the assessment.

• Participation– Every student enrolled as of the last day of the testing

window must take the assessment.




51

PDE-established Testing Windows

Make-ups can occur anytime during the PDE-established windows

Last day to return materials to DRC for all PSSA reporting is May 3, 2013

Writing Testing Window March 11-15, 2013

Writing Make-up Window March 18-22, 2013

Reading/Mathematics Testing Window April 8-19, 2013

Science Testing Window April 22-26, 2013

Reading/Mathematics/Science Make-up Window

April 29-May 3, 2013




52

Key DatesDescription Date Available

PA Online Assessment Software Currently Available

PA Online Assessment Test Tutorials Currently Available

Manuals and User Guides (electronically) Currently Available

Handbook and DFA Hardcopies Delivered By February 11, 2013

PA eDIRECT Test Setup System (online testing only) February 19, 2013

Secure Materials Delivered By February 25, 2013

PSSA Attributions/Demographic Corrections Window May 20-24, 2013

1% Redistribution Window July 8-10, 2013

District Student Performance Data Files and Parent Letters June 13, 2013

Preliminary Student Data Files and DRC AYP Reporting System July 22, 2013




53


District Assessment Coordinator (DAC)

Responsibilities Online Paper/Pencil

Attend DRC-led Assessment Coordinator training X X

Verify/set up eDIRECT accounts for District and School Users, and Test Administrators

X X

Review Handbook for Assessment Coordinators and DFAs X X

Review Test Setup User Guide X

Create and distribute a communication plan to all SACs prior to testing X X

Review tutorials and Online Tools Training X

Provide training and support to SACs X X




54

Roles and ResponsibilitiesSchool Assessment Coordinator (SAC)


Attend DAC-led Assessment Coordinator training X X

Verify/set up eDIRECT accounts for school-level users X X

Review DFAs and distribute DFAs to Test Administrators X X

Review Handbook for Assessment Coordinators X X



Provide training and support to Test Administrators X X

Set up Test Sessions via eDIRECT X

Collect secure materials after testing and return to DAC or DRC X X




55


Test Administrators (TA)


Attend SAC-led Test Administrators training X X


Review DFAs X X


Oversee and monitor Test Sessions X X

Collect secure materials after testing and return to SAC X X




56

Paper/Pencil Administration




57

• Notice of Secure Material receipt MUST be provided via eDIRECT

• Permissions have been granted to District Assessment Coordinators

• District Assessment Coordinators can provide permissions to School Assessment Coordinators

Materials Receipt Notice




58




59

Accommodated Materials

• Accommodated Assessments are available in Braille and Large Print versions

– All student responses from Accommodated Assessments must be transcribed into a scannable answer booklet.

• A Spanish-translation version of the Mathematics and Science assessments are available




60

Additional Materials• Inventory/account for all materials. Additional materials should be

requested AFTER material has been received and counts have been verified.

• All orders will be placed through eDIRECT, which will be available on February 19, 2013.

• Order additional materials as soon as possible.




61




62

Match Booklets by Form Number




63

Answer booklets returned to DRC for processing must have a label attached.

The only answer booklets that should NOT have labels attached are booklets that are unused and COMPLETELY blank.

Whether a Student Precode or a District/School Label is used, page three needs to be filled out for any student using an accommodation, as well as any not tested codes on page one.

Barcode Labels




64

STUDENT PRECODE LABELS Student Precode Labels contain student demographic

information as reported in eDirect

Barcode Labels




65

DISTRICT/SCHOOL LABELS District/School Labels contain district and school

specific information only. These labels must beused when Student Precode Labels are not used.

Barcode Labels




66

DISTRICT/SCHOOL LABELS (cont)When using District/School Labels, the student’s name,

birth date, and PAsecureID must be bubbled exactly as it appearsin your school’s student information system.

Barcode Labels




67

Barcode LabelsDO NOT SCORE LABELS All sites will receive Do Not Score Labels for answer

booklets that should not be scored.




68




69

Distributing and Tracking Secure Materials

• Use the Security Checklist• Update the Materials Accountability Form

Mr. Gillin

Martin Pinkerton

4/22

4/16

Destroyed due to illness

2012

4/17 4/18

70‹#›




71

Materials Accountability Form

• Available electronically via the eDIRECT website• Inventory all materials to be returned to DRC• Document discrepancies in the online form• Form can be accessed and updated throughout the

testing window• Updates should be made by May 3, 2013




72




73

Return Instructions – Answer Booklets




74

Return Instructions – Answer Booklets




75

Return Instructions – Test Booklets and Unopened Packages




76

Return Instructions – Test Booklets and Unopened Packages




77

Return Instructions

Material Pick Up

If you do not have a regularly scheduled UPS pick-up or delivery, call UPS at (866) 857-1501 to arrange a pick up at least 24 hours prior to your requested pick-up.




78

Special Note• Keep a copy of the UPS-RS tracking numbers

used.

• When DRC has questions about a district’s assessment materials, we make follow-up phone calls after the testing window has closed.

• It is important to return these calls as soon as possible in order ensure fast and accurate reporting and to keep your schools off the missing materials report sent to PDE.




79

Online Administration




80

Online Administration• System Overview

• Before Testing – Preparation

• During Testing – Monitoring

• After Testing – Closeout




81

System Overview

• Internet connectivity is required for testing

• Student responses are regularly saved to DRC servers throughout testing

• The PA Online Assessment software must be installed on all computers to be used for testing. This should be done as soon as possible.




82

Administer Tutorials

• Prior to online testing, ensure students view the video tutorials for the applicable subject(s)

• Allow a minimum of 20 minutes for viewing• Tutorials can be reviewed as often as needed




83

PA Online Assessment TutorialsAccess by double-clicking PA Online Tutorials icon on computer desktop




84

Administer Online Tools Training

• Students can practice using the online tools and become familiar with functionality of the software

• Schedule time for students to take the Online Tools Training (OTT) prior to testing

• One OTT for each subject• Allow 15 minutes to take a single OTT• DACs, SACs, and TAs should use OTT prior to

administering an assessment




85

Online Tools Training

Access by double-clicking PA Online Assessments icon on computer desktop




86

Verify Student Data

• Student data from a PIMS file cannot be updated in eDIRECT

• If a student is not listed in eDIRECT, the student may need to be added to the system




87

Add Students – Manual Entry

• Click Add Student

• Enter all required fields

• Click Save




88

Manage Students – Online Accommodations

Accommodations• Use the Accommodations tab

to enter student accommodations

• The Audio and Color Chooser Accommodation must be updated prior to testing




89

Create Test Sessions

• Select Test Sessions

• Select administration and school

• Click Show Sessions

• Click Add Sessions

• Enter required information

• Search for and assign students

• Click Save




90

Print Student Login Tickets

• Find Test Session

• Select Edit/Print icon

• Click Print All to print all the test tickets in the session

• Click Print Selected to print only selected test tickets in the session

• Day of Testing - Provide tickets to TAs/Teachers




91

During Testing

• Monitor student testing

• Test inactivity

• Pausing and exiting the test

• Loss of Internet connectivity




92

Special Situations

If any of the following situations occur, send an email with the student’s information and an explanation to [email protected]•Student goes home ill before completing online assessment

•Student begins testing with wrong test ticket

•Student begins testing with wrong accommodated version

•Student begins testing in the wrong subject




93

Collect Materials

• TAs must return all Student Login information and scratch paper to the SAC

• After testing is complete, SACs must destroy the documents to ensure that secure data is not compromised




94

Testing Codes

• Access the Edit Student page to update Testing Codes for a student




95

To contact someone from the Pennsylvania Department of Education

(717) 787- 4234Or

(717) 705-2343

The mission of the Pennsylvania Department of Education is to lead and serve the educational community, to enable each

individual to grow into an inspired, productive, fulfilled lifelong learner.




96

PA CUSTOMER SERVICE Data Recognition Corporation understands the

intricacies involved with large scale assessments.

Our customer service representatives are always ready and willing to help with any questions or problems.

Our number is 1-800-451-7849

Or email us at

[email protected]

PSSA 2013• Test Proctor Procedures

• Review of Materials Security Procedures– Embargo– Quarantine at Selected Schools

• Reminder - Submit Testing Schedule

• PDE Requirements

Test Proctoring ProceduresFollowing is not allowed• Teachers administering the test to their own

students.

E.g. A math teacher may administer the math test as long as he/she isn’t administering it to his/her own students

The same applies for Reading, Writing, and Science

Exception for accommodations• Only if the teacher provides unique accommodation• Only if there is a second proctor in the room

Scope of Monitoring• Monitors will circulate through classes

• Monitors will inspect test materials storage area using a new Materials Inspection form

• Monitors may observe small-group testing

• Select group of schools receive enhanced monitoring

Embargo - Materials Security• Schools must leave all testing materials in shrink-wrap

until directed by the Office of Accountability

– School can unbox packets to count materials

– OA delegate informs school when they may open materials

– Embargo will generally be lifted 1 week prior to testing

– If you have not been contacted by Monday, April 1st (March 4th

for writing and April 15th for science) , you may open materials

on Tuesday, April 2nd (March 5th for writing and

April 16th for science)

Firm Embargo - Materials Security

• Before testing begins, schools may be asked to leave all testing materials unopened until directed by the Office of Accountability

– OA delegate inspects and secures materials upon receipt

– OA delegate informs school when they may open materials

– OA delegate randomly inspects materials prior to testing

Quarantine - Materials Security• During testing, the storage and distribution of testing

materials may be supervised by delegates from the Office of Accountability– For selected schools, testing materials are stored by an OA

delegate in a secure manner between testing sessions

– For selected schools, OA delegate arrives to open secure testing materials the morning of testing and remains on-site to re-secure materials once testing concludes for the day.

!REMINDER! Submit Testing Schedule• Schools required to submit Testing Schedule to Office

of Accountability no later than March 8– Please use the online form located on OA website

http://www.philasd.org/offices/accountability/assessments/pssa


PDE Requirement ---NEWBY April 5th

• The Test Coordinators (called School Assessment Coordinators, SACs) MUST take a 45-minute online training and Pass with a score of 80% or better and

• All the test administrators (TAs) and proctors at the school MUST complete this training

• The training video MUST be included in the turn-around training on March 7

• https://pssatraining.com.‹#› 104

http://pssatraining.com/

PDE Requirement ---NEW• Schools MUST inform the parents and

students of PDE’s policy on the possession and/or use of electronic devices during testing

• Schools come up with consequences of non-compliance and inform parents and students

• Must be done by March 11 (sample letter in the folder)

PDE Requirement - Testing Rosters• PDE asking for student-level test administration data

– Verify and/or list the staff member who administered the Keystone for each student

– Indicate any student that required make-up testing– Excel file will be sent prior to testing and must be returned

when testing is completed; pre-loaded data where possible

PDE Requirement - Certification• Principal, School Assessment Coordinator,

Test Administrator, and Proctor must sign a Test Security Certification

– Must be signed after testing

– Security Certifications will be kept on file for 3 years at the

school

Contact Information• Office of Accountability website


• Contact Information– Office of Accountability/Assessment

[email protected] (215-400-4250)– DRC's Customer Service Representatives

(1-800-451-7849)




assessment coordinator training tom corbett, governor ▪ ronald j. tomalis, secretary of education...

Documents

assessment materials

assessment pde participation

assessment procedures

assessment program

pssa overview slide

pssa logistics

pssa results

entire pssa