cella test chairperson training session february 15 and 17, 2011 2011 comprehensive english language...

CELLA

Test Chairperson Training Session

February 15 and 17, 2011

2011 Comprehensive English

Language Learning Assessment

Assessment, Research, and Data AnalysisFelicia Mallory, Executive Director

Denetra Collins, Staff Specialist

Program Overview

Comprehensive English Language Learning Assessment (CELLA) is designed to meet the requirements of Title III of the NCLB.

A four-skill language proficiency assessment in• Speaking

• Listening• Reading • Writing

Administered to all English Language Learners (ELLs) in grades K-12.

TAM, p.5

Meeting Annual Measureable Achievement Objectives (AMAOs)

Standards for Measuring English Language Learners (ELLs)

AMAO 1 is based on progress in English language acquisition as measured in each of the three CELLA areas. Making progress is defined as either increasing a proficiency level or staying within the “Proficient” level in a specific area.

AMAO 2 is based on achieving English proficiency as measured byCELLA. Achieving proficiency is defined as scoring within the proficient level in all three domains: Listening/Speaking, Writing, and Reading. AMAO 2 is established separately for four grade clusters: K-2, 3-5, 6-8, and 9-12.

AMAO 3 is based on demonstrating proficiency in reading and mathematics on the FCAT. Demonstrating proficiency is defined as scoring at achievement level three or higher.

Administration Features

Kindergarten• Individual administration required in all four modalities

Grades 1-12 • Individually administered in Speaking(One-on-One)• Group administered in Listening, Reading, and Writing

TAM, p. 6

Training for Coordinators and Administrators of CELLA

• Speaking Administration Training (One-On-One)

General training conducted by the Division of Bilingual Education and World Languages and Charter Schools

Training for Special Education English Language Learners conducted by the Division of Special Education

• Group Administration Training

Conducted by Student Assessment and Educational Testing (SAET)

Activity DateOne-on-One Test Administration Training

(Regional Trainings by the Division of Bilingual Education and World Languages and Charter Schools)

February 22 – March 3, 2011

One-on-One Test Administration Training for Special Education

February 18, 2011

Delivery of Training Kits to Schools February 7 – 14, 2011

Delivery of the 2011 CELLA Test materials (including PreID labels) to schools

February 16 – 23, 2011

Individual CELLA Administration March 7 – April 8, 2011

Group CELLA Administration (including make-up) March 21 – April 8, 2011

Return all “To Be Scored” and “Not to Be Scored”(Schedule pick-up through K2 Logistics)

March 21 – April 12, 2011

District Coordinator EnvelopeSend to TDC via School Mail


Complete the 2011 CELLA Customer Satisfaction Survey online at http://www.fldoe.org/aala/CELLA.asp

Upon completion of testing (Test administrators and School Coordinators)

Score report delivery to schools June – July 2011

Schedule of Activities

Packet , pp. 2-3

Students To Be Tested

All ELLs in grades K through 12 in membership on March 7, 2011

All former ELLs who exited from the English for Speakers of Other Languages (ESOL) program on or after September 1, 2010

Pre ID labels provided for all eligible students in membership for Survey 2 and/or Survey 7

TAM, p. 7

CELLA Levels of Testing Overview Booklet levels for the 2011 CELLA are as follows:

Grades K – 2 Level A Grades 3 – 5 Level B Grades 6 – 8 Level C Grades 9 – 12 Level D

Test Sections – Listening, Speaking/One-on-One, Reading, and Writing

Test Format:

Listening: All Multiple-Choice (MC)

Speaking: All Constructed-Response (CR)

Reading : All MC

Writing: MC and CR

All students will test “on grade level” in all modalities.

TAM, p. 6

Test Materials: Level AGrades K - 2

TAM, pp. 8-9

The following materials are needed to administer the Level A test:

• Student Test Book• One-on-One Prompt Book• Levels A2 & B2 Listening CD• Directions for Administration (DFA)

The Reading section contains:

Core items (#1–15)• for all students in grades K–2• read aloud by the Test Administrator• approximate testing time: 15 minutes

Extension items (#16–25)• for grade 2 ONLY• completed by the student independently• approximate testing time: 20 minutes

Test Materials: Level AGrades K – 2 (Reading)

The Writing section contains:

Core items (#1–7)• for all students grades K–2• read aloud by the Test Administrator• approximate testing time: 15 minutes

Extension items (#8–16)• for grade 2 ONLY• completed by the students independently• approximate testing time: 15 minutes

Test Materials: Level AGrades K – 2 (Writing)

The One-on-One section is administered individually using the One-on-One Prompt Book.

The student’s responses are scored by the Test Administrator, who records the scores in the student’s Level A test book.

The Test Administrator must have completed the CELLA training (either the Questar/FDOE- developed or district-developed training).

Approximate testing time: 15 minutes per student.

Test Materials: Level AGrades K – 2 (One-on-One/Speaking)

Test Materials: Level AGrades K – 2 (One-on-One/Speaking)

Test Administrators score the student’s responses and record the scores in the One-on-One Scoring Section of the test book.

Page 37 of the Level A Test Book:

Marking Answer Responses-Level A(Listening)

Correct

Incorrect

ADMINISTERING LEVEL AThe Stopping Rule

The Stopping Rule is ONLY applicable to individually administered sections.

If a student cannot answer several questions in a row, you should administer at least the first question of each type.

1. If the student is able to respond to the question even minimally, continue to administer the questions of that type.

2. If the student cannot or does not respond to the first question of the type, skip to the next type. Then fill in the NR bubbles of the corresponding questions in the One-on-One scoring area of the student’s test book.

DFA, pp. 7, 14, 21, & 26

TEST MATERIALS: LEVELS B, C, and DGrades 3 - 12

Level B (Grades 3 – 5):Levels B2 Test BookLevels A2 & B2 Listening CD

Level C (Grades 6 – 8):Level C2 Test BookLevels C2 & D2 Listening CD

Level D(Grades 9 – 12):Level D2 Test BookLevels C2 & D2 Listening CD

Levels B, C, and D:Levels B, C, and D Answer SheetDirections for Administration

The Reading section is divided into two parts:

Part One: The student answers discrete vocabulary questions.Assesses knowledge of English vocabulary through use of synonyms, antonyms, idioms, roots, and affixes

Part Two: The student reads passages and answers 4–6 questions.Assesses reading comprehension

Approximate testing time: 45 minutes

All reading items are multiple-choice

ADMINISTERING LEVELS B, C, and D Reading

The Writing section is divided into four parts:

Parts One & Two: The student answers multiple-choice questions.

• Grammar, Structure, Written Expression: tests knowledge of grammar

• Paragraph Choices: tests elements of extended writing such as use of transitions, and topic and concluding sentences

• Recognizing Errors: tests editing skills

ADMINISTERING LEVELS B, C, and D Writing

ADMINISTERING LEVELS B, C, AND D WRITING (continued)

Parts Three & Four: The student writes sentences and paragraphs.• Writing Sentences: The student writes a sentence

based on a picture.• Writing Paragraphs: The student writes a paragraph

based on a prompt. Paragraph might be descriptive, persuasive, comparative, etc., depending on grade level.

Approximate testing time: 70 minutes

ADMINISTERING LEVELS B, C, AND D SPEAKING

The Speaking section is administered individually to all students.

The student’s responses are scored by the Test Administrator, who records the scores on the Level B, C, and D Answer Sheet.

The Test Administrator must have completed the CELLA training (either the Questar/FDOE-developed or district-developed training).

Approximate testing time: 10–15 minutes per student

Recording Speaking Responses for Levels B, C, and D

CELLA Document Configuration by Grade Level (Review)

Grade Level Test Book(s)

Kindergarten (K) and Grades 1 & 2 Level A Student Test BookLevel A One-on-One Prompt Book

Grades 3, 4, and 5 Level B Listening, Speaking, Reading and Writing Test Book

Level B, C & D Answer Sheet

Grades 6, 7, and 8 Level C Listening, Speaking, Reading and Writing Test Book


Grades 9, 10, 11, and 12 Level D Listening, Speaking, Reading and Writing Test Book


Materials shown in bold are answer documents.TAM, p. 8

CELLA Materials for Students’ UseLarge Print/Braille Materials

The Large Print version of CELLA has the same content as the standard version of the test.

The Braille version of CELLA differs from the standard version of the test (Level A and some questions from other levels have been omitted).

Note: The Test Administrator will also need a regular-print version of the test booklet that matches the Braille level being administered.

TAM, pp. 9-10

Preparing for Testing: Receiving Materials

Verify that all materials indicated on the packing list were received.

Notify the District Coordinator immediately if any of the materials on the packing list are missing.

Should you need to order additional materials, please call TDC at 305-995-3743.

Store materials in a secure access-restricted location. Retain ALL boxes in which materials were received for

return (except the ones which contained large print and Braille documents).

TAM, p. 35

School Packing List

TAM, p. 35

TAM, p. 35

Shipment Summary List

Box Content List

TAM, p. 35

Security Numbers

security number check number

TAM, pp. 20 - 21

TAM, p. 18

Retrieving the Pre-populated Administration Record/Security Checklist

https://fl-servicepoint.questarai.com/Login.aspx

Select “Click Here”

TAM, p. 21

https://fl-servicepoint.questarai.com/Login.aspx

Retrieving the Pre-populated Administration Record/Security Checklist

Choose “Miami- Dade”

from the dropdown

Choose your

school from the

dropdown

Click on Security Checklist under School Files

TAM, p. 21

2011 CELLA ADMINISTRATION RECORD/SECURITY CHECKLIST

Pre-populated version

TAM, p. 36

Preparing for Testing: Scheduling Test Sessions

Schedule grade K testing and the Speaking section in grades 1-12 for individual administration

• Individually administered sections may be

administered at anytime during the testing window, but are recommended to be tested first

It is recommended that CELLA group-administered sections are given in the following order: Listening, Reading, Writing

• Group administration window:


Administration Times for Grades K-2

Level A Approx. Time in Grade K

(individually)

Approx. Time in Grade 1

Approx. Time in Grade 2

Speaking (One-on-One)

15 min. 15 min. 15 min.

Listening 15 min. 15 min. 15 min.

Reading 15 min. 15 min. 35 min.

Writing 15 min. 15 min. 30 min.

Sections shown in bold require more time because they contain additional, “extension” items only for grade 2 students.

TAM, p. 6

Levels B, C, or D Approx. Time

Speaking 10-15 min.

Listening 25 min.

Reading 45 min.

Writing 70 min.

Administration Times for Grades 3-12

TAM, p. 7

Preparing for Testing: Training Test Administrators

Review the Test Administration ManualReview the relevant sections from the Directions for

Administration (DFA)Have test administrators familiarize themselves with the

Listening CDs and equipment for playing themRemind test administrators of the State’s and District’s

security policies and procedures (including those concerning electronic devices)

After the training, collect signed 2011 CELLA Administration and Security Agreement forms

TAM, pp. 24-26

Preparing for Testing: Accommodations for all ELLs

Review the list of accommodations in the DFA . The following accommodations ARE permissible for all ELLs:

Flexible timing: CELLA is not a timed test, sufficient time should be given to complete the test.

Assistance in heritage language: Directions may be translated into heritage language on an individual basis, if requested.

DFA , pp. 258-266

Preparing for Testing: Accommodations for all ELLs

The following accommodations ARE NOT permissible for any student:No English-to-heritage language translation dictionary or heritage language-to-English translation dictionary.No help in the ELLs’ heritage language on specific test questions NOR translate any part of the test other than the directions.Test Administrators may NOT translate directions to entire classes.

DFA, pp. 258-266

Preparing for Testing: Accommodations for SPED Students

Students with a current IEP or 504 plans are eligible for accommodations based on their individual needs.

An accommodation for a student must be explicitly permitted on the IEP or 504 plan.• Presentation of Test Materials• Responding to Questions• Test Scheduling• Test Setting• Assistive Devices

Refer to the Visually Impaired and Hearing Impaired section in the DFA for allowable accommodations and/or exemptions.

DFA, pp. 259-266

Preparation of Testing Materials

Pre-ID labels are provided.

Only the Test Chairperson or the person designated by the site administrator may open the packages and affix labels.

Pre-ID labels should be affixed only to answer documents; only one answer document per student:

Level A Student Test Book, or Level B, C, & D Answer Sheet

TAM, p. 17

Preparation of Testing Materials

Level A Student Test Book Levels B, C, D Answer Sheet

TAM, pp. 10 & 17

Preparation of Testing Materials (continued)

ALL of the following information MUST be correct for the student label to be used:

1. Student’s Last Name2. Student ID Number3. District Number4. School Number (code)5. Grade Level

TAM, pp. 17-18

Pre-ID Label Used: Information to be Completed

Box 1 Student Name Box 2 Test Administrator Name Box 3 School Name Box 4 School Number Box 5 District Name (MDCPS) and Number (13) Box 9 Test Date Box 11 Grade/Test Level (record on B, C, D Answer Sheet) Box 21 Do Not Score (DNS) or Undo (if applicable) Box 22 Accommodations (if applicable)

TAM, pp. 12-16, Packet , pp. 5-6

No Pre-ID Label Used: Information to be Completed

If a student has no pre-identified label, the following additional information MUST be gridded on all student answer documents:

Students or school staff Boxes 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, and 16

School staff only Boxes 2, 12, 17, 18, 19, 20, 21, 22, 23, and 24

Student demographics provided on the student list are indicated in bold.

TAM, pp. 11-16; Packet pp. 5-6

Student List for Gridding

Packet, p. 7

Gridding Demographic Information

Fill in the bubble for the following fields as indicated below:

Box 16 - Home Language

Box 17 – Primary Exceptionality

Box 18 - Length of Time Enrolled in ESOL Program

Box 19 - Title III YES/NO

TAM, pp. 11-16 , Packet, p. 6

Preparation of Testing Materials Non-Pre ID Label Used: Other Fields

Fill in the bubble for the following fields as indicated below:Box 23 Receives Special Services F/R Lunch if “Y” is indicated on the student listMigrant if “Y”Box 24 Program Participation ESOL if “Y” (do not mark if student is former ELL) Dual Language Program – determined by school

Packet, p. 6

Test Security Policies and Procedures

Florida Test Security Statute 1008.24 and Florida State Board of Education – Test Security Rule 6A – 10.042

M-DCPS: Standards, Guidelines, and Procedures for TestAdministration and Test Security http://oada.dadeschools.net/TestChairInfo/29testsecuritymanual.pdf

The security of all test materials must be maintained before, during, and after the test administration.

Under no circumstances are students permitted to handle secure materials before or after the test administration.

TAM, p. 19

http://oada.dadeschools.net/TestChairInfo/29testsecuritymanual.pdf

http://oada.dadeschools.net/TestChairInfo/29testsecuritymanual.pdf

Distribute and collect all One-on-One test materials daily.

Distribute all testing materials for group administration just prior to test sessions, collect immediately afterwards, and store in a secure location.

Distribute copies of the 2011 CELLA Administration Record/Security Checklist

Distribution and Tracking of Materials


Test Level/Grade Student Material Needed

Teacher Material Needed

Level A / Kindergarten(all four sections)

Level A Student Test Book

Directions for Administration (DFA)Test Administration Manual (TAM)One-on-One Prompt BookListening CD (Levels A & B)CD Player/ computer with speakers

Level A / Grades 1 – 2(One-on-One)


DFATAMOne-on-One Prompt Book

Levels B-D / Grades 3 – 12(Speaking)

Listening, Speaking, Reading & Writing Student Test Book (Appropriate level)Levels B,C, & D – Answer Sheet

DFATAM

Individually Administered SessionsAdministrators need:

TAM, pp. 8 & 27-31

Group Administration for Grades 1 – 12 (Listening) Administrators need:




Level A / Grades 1 – 2 Level A Student Test Book

Directions for Administration (DFA)Test Administration Manual (TAM)Listening CD (Levels A & B)CD Player/ Computer with speakers

Level B / Grades 3 - 5Level C/ Grades 6 - 8Level D/ Grades 9 - 12


DFA & TAMListening CD (Appropriate Level)CD Player/ Computer with speakers

TAM, p. 8

Group Administration for Grades 1 – 12 (Reading and Writing)Administrators need:




Level A / Grades 1 – 2


Directions for Administration (DFA)Test Administration Manual (TAM)

Level B / Grades 3 - 5Level C/ Grades 6 - 8Level D/ Grades 9 - 12


DFATAM

TAM, p. 8

Rooms used for testing must: be free from distractions have comfortable seating have good lighting have adequate ventilation have sufficient workspace

Remove or cover all visual aids in the testing rooms.

Make sure that: test administrators have a copy of the TAM and DFA required administration information is collected for each student test scripts are read verbatim there is sufficient space between students so that students cannot view other student’s answer sheets students and assessors record all answers using a #2 pencil

Administering the Test

TAM, pp. 24 & 25

The Listening portion must be administered by playing the Listening CD.

Test administrators are required to actively monitor students during a test session.

Test administrators are required to record all required administration information on the CELLA Administration Record/Security Checklist

Test administrators are required to maintain an accurate 2011 CELLA Security Log.

Test administrators and proctors must sign the 2011 CELLA Administration and Security Agreement

Administering the Test

TAM, pp. 8, 26 & 33

Security Log

TAM, pp. 26 & Appendix B

Test Invalidation Procedures

Box #21

DO NOT SCORE – Contains DNS and UNDO bubbles for each test section

From Level B,C, & D Answer Sheet

TAM, pp. 22 - 23


The appropriate DNS bubble MUST be gridded if:

1) Any of the following information on the PreID label is INCORRECT and affixed to a USED document.

- Last Name- Student ID Number- District Name- School Name- School Code- Grade Level

2) A student label has been placed over another label on a USED document.

3) A student has an electronic device during testing.4) A student is caught cheating or is disruptive during testing.

TAM, pp. 22 - 23


TAM, pp. 23, 31, & 38

The appropriate DNS bubble MAY be gridded if:

1) A student becomes ill and is unable to complete the session.

2) A student is not allowed sufficient time to finish.

Answer documents with gridded DNS bubbles should be submitted with TO BE SCORED materials.


TAM, p. 23

It is not necessary to invalidate an answer document if:

1) The preidentified document belongs to a student who has withdrawn before testing begins.

2) The preidentified document belongs to a student who has

been absent during the entire testing window.

3) The document is defective and UNUSED.

4) The preidentified information is incorrect and the document is UNUSED.


TAM, p. 24

Defective Materials

If a defective test book is found during testing, give the student a replacement test book.

If the defective book is a Level A , upon completion of the test, the student must (under supervision of the test administrator) transcribe the exact answers to the appropriate replacement document.

If the defective book is Level B, C, or D, the student will continue using the Levels B, C, & D answer sheet.

If a test book is defective or becomes soiled, notify Student Assessment and Educational Testing at 305-995-7520.

Preparing Materials for Return

RED return label – TO BE SCORED answer documents

YELLOW return label – NOT TO BE SCORED materials

All 2011 CELLA materials must be returned to Questar as follows:

TAM, pp. 41 – 43, Packet, pp. 10-13


Verify that all distributed secure materials have been returned by the test administrators.

TO BE SCORED (Red label) materials include:

Used Level A Test Books Used Levels B, C, and D Answer Sheets Documents with gridded DNS bubbles

Preparing TO BE SCORED Test Materials

TAM, pp. 38 - 41


Complete the Document Count Form

for TO BE SCORED materials (PURPLE)

NOTE: Complete one form per grade level and be sure to include documents with gridded DNS bubbles in your

counts.


TAM, pp. 38 - 41


Complete the School Return Summary for To

Be Scored materials (RED)

NOTE: ONE PER SCHOOL


TAM, p. 40


Place the School Return Summary on top of the materials in Box 1.

Affix the RED label to the top of the box

In the lower left corner of each RED label, write the number of the box and the total number of boxes of TO BE SCORED materials that you are returning. (Ex. 1 of X, 2 of X, 3 of X, etc.)

TO BE SCORED Test Materials

TAM, pp. 31, 41, & 45 & Packet, pp. 10-11


NOT TO BE SCORED materials include:

Unused Level A Test Books Unused Levels B, C, and D Answer Sheets Level A One-on-One Prompt Books Used and Unused Levels B, C, and D Test Books Training CD’s for Speaking - Levels A, B, C, and D Listening CD’s - Levels A2 & B2 and Levels C2 & D2 Directions for Administration Used and unused large print and braille test books

Preparing NOT TO BE SCORED Test Materials

TAM, pp. 32 & 41-42, Packet, p. 12


Place a copy of the CELLA Administration Record/Security Checklist on top of the materials in Box 1.

Affix the YELLOW label to the top of the box

In the lower left corner of each YELLOW label, write the number of the box and the total number of boxes of NOT TO BE SCORED materials that you are returning. (Ex. 1 of X, 2 of X, 3 of X, etc.)

NOT TO BE SCORED Test Materials

TAM, pp. 32, 42 & 45, Packet, p. 12


Test Administration Manuals* Unused Paper Bands Unused Document Count Forms and School Return Summaries Unused Questar Return Labels Unused Student PreID Labels

Destroy at School Site

TAM, p. 42, Packet, p. 13

*Keep test administration manuals until results have been provided, then destroy at the school site.


Items to include: Original CELLA Administration Record/Security Checklist Original 2011 CELLA Security Logs Copy of School Return Summary for the To Be Scored Materials Copy of Document Count Form for To Be Scored Materials by Grade Accounting for All 2011 CELLA Materials School Procedural Checklist Bill of Lading

On the front of the envelope write “DISTRICT COORDINATOR ONLY”

District Coordinator ONLY Envelope

TAM, p. 42 & Packet, p. 13

Signed Administration and Security Agreements should be kept on file at your school site.

Scheduling Pick-Up via K2 Logistics

Schedule your CELLA pick-up 48 hours in advance

Access the online booking site at www.k2assessments.com

Follow instructions on pages 51 and 52 of the Test Administration Manual

Check your email to ensure that a confirmation was issued by K2 Logistics

Ensure that you obtain the Bill of Lading provided by K2 Logistics upon pick-up

http://www.k2assessments.com/

Review of the Return of CELLA 2011Test Materials

Materials should be returned March 21, 2011 – April 15, 2011

K2 LogisticsTo Be Scored Materials

Not To Be Scored Materials

School Mail: 9038 – Test Distribution CenterDistrict Coordinator Envelope

TAM, p. 51 & Packet, pp. 10-13

Comment Form

2011 CELLA Customer Satisfaction Survey

Please access the following site after testing has been completed.

http://www.fldoe.org/aala/CELLA.asp

Score reports will be delivered to schools in August

Questar provides schools with only two copies of the CELLA Student Report

Reports should be distributed in the following manner:

• File one copy in the student’s LEP folder that goes in the cumulative records• Send one copy home with the student

Receiving 2011 CELLA Student Score Reports

Contact Information

STUDENT ASSESSMENT AND EDUCATIONAL TESTING

Felicia [email protected]

Denetra [email protected]

305-995-7520Fax – 305-995-7522

cella test chairperson training session february 15 and 17, 2011 2011 comprehensive english language...

Documents

english proficiency

group cella administration

english language acquisition

proficiency level

cella test materials

onegeneral training

cella areas

delivery of training