1

North CarolinaNational Assessment of Education

(NAEP)

2009

LEA Training --Session BDecember 2008

Iris L. Garner, Ph.D.igarner@dpi.state.nc.usNAEP State Coordinator

North Carolina Department of Public Instruction

Session B--Training Objectives:

1) To illustrate the NAEP sampling and data collection design

2) To describe the corresponding NAEP activities that take place before, during, and after the assessment

NAEP Sampling&

Data Collection Design

Training Objective #1

NAEP 2009Assessment Assessment Window:

1/26/09 – 3/6/09

Subjects to be Assessed: Reading, Math, Science

Some schools will have a Science Interactive Computer Task (SCIT) or Hands-on Task (HOT) for the science portion of the Field Test.

NAEP 2007 NC Performance

Math grade 8: NC average score was (284) in 2007 was higher than that of the nation’s public schools (280).

Reading grade 8: NC average score (259) in 2007 was not significantly different from that of the nation’s public schools (261).

NAEP 2009Assessment

Results will be reported at the state and national level

The assessment will take approximately 90 minutes of student time (two 25 min. subject matter blocks and survey questions).

Students take one subject

NAEP Design: Sampling and Data Collection

Multi-Stage Stratified Random Sampling selection of schools students within schools are selected

randomly

List of schools are obtained from the National Center of Education Statistics (NCES) Common Core of Data (CCD)

NAEP Design: Sampling and Data Collection

The Common Core of Data (CCD) U.S. Department of Education's National Center

for Education Statistics Program

Annually collects fiscal and non-fiscal data about all public schools, public school districts and state education agencies in the United States

Data are supplied by state education agency officials

NAEP Activities

Training Objective #2

NAEP Team Structure

NAEP Coordinator(Iris Garner)

NAEP Field Manager(Gene Biggerstaff)

Field Supervisor Field Supervisor

Assessment Admin.(AAs)

Assessment Coord. (ACs)

Schools

Before Assessment

NAEP Activities

Preliminary Activity The North Carolina Department of Public Instruction

notified the districts of the school that were selected to participate in NAEP.

NAEP Data Collection Forms have been completed and submitted on-line via the MyNAEP Web site at www.mynaep.com.

NAEP field staff started contacting the school test coordinators on 12/15/08 to set up the Pre-Assessment Visit (PAV).

NAEP Activities

Preassessment Visit (PAV) PAV will be scheduled between January 6 and

January 23.

PAV should take approximately two hours.

The visit could take longer if the school test coordinator has not prepared for the PAV.

Prior to the PAV you will receive a packet of materials. Please refer to the INSTRUCTIONS FOR THE SCHOOL COORDINATOR.

NAEP Activities

Before Assessment

Before the Preassessment Visit

1) Distribute questionnaires to staff most knowledgeable about the student

Refer to the Guide for Completing SD/ELL Questionnaires

Select the method for completing the questionnaire: online or hardcopy

Questionnaire GuidanceStudent with Disabilities & English Language Learners

Students with Disabilities (SD) and English Language Learners (ELL) Questionnaires

Questionnaires for each student in the sample that was identified as SD or ELL.

The questionnaires need to be filled out and returned to the coordinator prior to the PAV.

If you have more students in the sample that are identified as SD/504 or ELL, let the NAEP field staff if additional questionnaires are needed.

Student with Disabilities and English Language Learners

If the student participates in the regular EOGs or EOCs with or without accommodation, they should participate in NAEP.

If the student can access a “paper/pencil” assessment, they should participate in NAEP with accommodations.

In 2002, NC had the highest exclusion rate in the nation for grade 4 reading. The exclusion rate has reduced in 2003, 2005 and 2007.

Student with Disabilities and English Language Learners

The exclusion rates will be recorded after the pre-assessment visit and will be monitored for high rates.

NAEP field staff will administer all accommodated sessions. These sessions may have to occur after the standard administration.

If a special needs student feels more comfortable with a known test administrator, then the teacher can administer the assessment with the NAEP field staff in the room.

Questionnaire GuidanceStudent with Disabilities & English Language Learners

Questionnaire Guidance Includes:Folder-A Guide For Completing The Student With Disabilities and English Language Learner Questionnaires

Frequently Asked Questions

Step-by-step directions

Accommodations Chart

State-specific policies

Questionnaire GuidanceStudent with Disabilities & English Language Learners

Students assessed using the NCEXTEND1 most likely will not be able to be assessed on NAEP.

Students assessed using the NCEXTEND2 most likely will be assessed on NAEP since they can access a “paper-pencil” assessment.

Supply of Blank Paper: The NAEP assessment does not allow a supply of blank paper. All notes, explanations, etc. should be written directly in the test booklet.

Questionnaire GuidanceStudent with Disabilities & English Language Learners

All students that take the NAEP assessment are allowed to write in the test booklet. NAEP does have separate answer sheets. Writes in test booklet is not considered an accommodation for NAEP.

Questionnaire GuidanceStudent with Disabilities & English Language Learners

A Spanish version of the NAEP assessments in Mathematics and Science is available. North Carolina at this time does not have translated versions of the state tests. This accommodation is not allowed.

Except for the items listed above the North Carolina State Testing Program accommodations can be used with the NAEP assessments. If an accommodation is part of a student’s Individualized Education Program (IEP) or 504 plan and is used regularly in the classroom, and on the state assessments, it should be able to be used on the NAEP 2009.

NAEP Activities

Before Assessment

Before the Preassessment Visit

2) Distribute the parent letters

Parent Letter

Parent letters will need to be sent home to students selected to participate in the NAEP assessment prior to the PAV.

Use school letterhead

Include date

Parent Letter

Share the Sample Questions Booklet with the parents.

NAEP Activities

Before AssessmentBefore the Preassessment Visit

2) Review and update Student Information Report – List of all participating students and demographic information.

Note: The Student Information Report for the National Indian Education Study (NIES) are included in the Important NIES Material envelope.

NAEP Activities

Student Information Review Collect any missing demographic information.

Review student demographic data for accuracy.

Determine if there have been any changes in students identified as SD and/or ELL.

Identify which students, if any, have withdrawn or graduated, and who are neither enrolled nor attend the school in the grade to be assessed.

Make all corrections directly on the report.

NAEP Activities

Before Assessment—Before the Preassessment Visit

3) Print out a current list of the students in the selected grade

4) Complete logistics questionnaire.

NAEP Activities

Before Assessment—During the Preassessment Visit

At the beginning of the PAV the NAEP field staff will meet with you to collect the information you completed prior to the PAV:

Printed list of current students at the selected grade Corrected Student Information Report SD and ELL Questionnaires Copy of the parent letter Logistics Questionnaire

After the Pre-Assessment Visit

Notify the students of the assessment.

Notify the teachers of the assessment.

Place the NAEP Storage Envelope in a safe place.

Supporting NAEP in Your School

NAEP has produced a short movie that you can share with your teachers and students to help them understand the importance of NAEP.

You can access Introducing NAEP to Teachers at the NCDPI Web site: http://www.ncpublicschools.org/accountability/policies/naep/5

You can access Introducing NAEP to Students at the NCDPI Web site: http://www.ncpublicschools.org/accountability/policies/naep/5

Before Assessment

NAEP Assessment Day

Assessment Day The NAEP assessment team will arrive at the school

an hour and half prior to the scheduled assessment time.

The NAEP assessment team will need a staging area throughout the assessment day.

The NAEP team will administer all assessments, including the accommodated sessions.

NAEP Assessment Day

Assessment Day NAEP does not need proctors for the assessments.

Classroom teachers and/or other school personnel are welcome and encouraged to stay in the room during the NAEP assessment.

School personnel will have to sign a confidentiality agreement if they observe the NAEP assessment.

NAEP Activity

Assessment Day Your biggest role on assessment day is to make

sure all students in the sample arrive at the designated assessment locations.

You may need to work with you attendance staff to make sure missing students are really absent that day.

If 90% of the students in the sample are not there that day, then a make-up session will need to be scheduled.

The 90% does not include students who have withdrawn, students who’s parents have refused participation, or students who were excluded.

After Assessment

NAEP Activity

After AssessmentNAEP representative conducts a short interview with the school coordinator

A make up assessment day is scheduled if needed.

The school coordinator will securely destroy the contents of the storage envelope on the date listed on front of the envelope.

The school coordinator will mail the postcard to NAEP stating that the materials have been securely destroyed.

Thanks You

We appreciate all you do; you make the NAEP administrations possible.