1

2013 Spring DAC Meeting

KDE:OAA:DSR: 2/7/2013

2

HOT TESTING TOPICS 2013Important Things to Know about Testing

KDE:OAA:DSR: 1/28/2013

3

Hot, Hot, Hot

KDE:OAA:DSR: 1/28/2013

2. Time Chart revised to incorporate time buffer from 2012. Chart shows times and number of items per grade level and subjects.

1. Administration Code and Inclusion of Special Populations regulations are unchanged, 2009 versions still effective.

3. Student Honor Code – Added to focus on test security. TAM is scripted to direct student review and completion.

4

Hot, Hot, Hot

KDE:OAA:DSR: 1/28/2013

4. EOC constructed response (CR) items move to local administration (instead of state) in February 2013. • EOC test is the first time Kentucky has tried to blend a

state accountability model with a local school grading model. The results of the test must support both grading and accountability.

• The CR items did not provide feedback and instructional information. EOC CRs did not support local grading.

• KDE strongly encourages CR scores be included in the EOC final course grade.

·       

5

Hot, Hot, Hot

KDE:OAA:DSR: 1/28/2013

• CRs may be generated and scored locally. QualityCore® resources may be used.

• CRs may be administered at a different time than the required multiple-choice sessions.

• Local districts and schools will determine how to incorporate the CRs into a student’s final course grade.

• KDE will be monitoring inclusion of CRs in its annual EOC survey.

• The Kentucky Board of Education may decide in the future to develop regulatory language to require the inclusion of CRs.

 

6

Hot, Hot, Hot

KDE:OAA:DSR: 1/28/2013

6. Agreements with ACT, Inc. and NCS Pearson, Inc. to provide Kentucky assessments are through 2017 spring testing. • KDE is monitoring the development of new assessments;

staff participates with the PARCC consortium.• Kentucky schools may be asked to field test by new tests.• KDE, at this time, has NOT committed to use any specific

future assessment (i.e., PARCC, SBAC, ACT Aspire™, DLM, NCSC).

7

TEST SECURITY AND DATA QUALITY

Key assessment themes require the focus of schools, districts and the state

KDE:OAA:DSR: 1/28/2013

Key Assessment Themes

8

TEST SECURITY—checks and processes to monitor appropriate administration of state-required tests

DATA QUALITY—collaborative process between OAA and school/district staff to ensure accurate reporting of test results

KDE:OAA:DSR: 1/28/2013

Test Security• Continues to be a critical issue in Kentucky and

nationally. o Loss of certification for Kentucky educators

emphasizes the seriousness of the issue.• OAA implements recommendations from

Kentucky’s security audit.o A Student Honor Code statement has been added to

2013 Student Response Booklets. 9KDE:OAA:DSR: 1/28/2013

10

Student Honor Code

KDE:OAA:DSR: 1/28/2013

Test Security Required Activities• Appropriate training of regulations—Administration Code and

Inclusion of Special Populations (2009 versions)• Signatures on Appropriate Practices Form and Administration Code• Seating charts that reflect everyone in each testing room (adults

and students)• Bubbling of testing room location codes by content area tests on

student response booklets• DACs collect BAC testing certification and KDE collects DAC testing

certification form (provided by KDE, not vendor)

11KDE:OAA:DSR: 1/28/2013

Test Security Required ActivitiesCertification Form DACs 

and BACs• E-mail from KDE/OAA 

and copy in                DAC/BAC manual

•  Completed at end of year to reflect all 

testing12KDE:OAA:DSR: 1/28/2013

Test Security State Monitoring• Test window visits• Review of seating charts • Collection of DAC certification forms• Review/audit of randomly collected AAA Folders for

alternate assessment students• Analysis of outlier test results by Office of Education

Accountability• Analysis by vendors (i.e., erasure and others)

13KDE:OAA:DSR: 1/28/2013

Test Security State MonitoringAnalysis by vendor can produce multiple types of information/reports at room, school or district levels.• Large score changes for a student• Large mean score increases• Probability of matched response analysis• Previous year’s distribution around scores• Pattern analysis• Comparison of individual student scores by item type

14KDE:OAA:DSR: 1/28/2013

Test SecurityReporting of Allegations

Get the facts straight.• Record the date and time of the incident.• List all involved (i.e., staff, volunteers, students).• Identify content area and test parts impacted.• Identify specific content question(s), if applicable.• Record what you learn.• Report the incident.

15KDE:OAA:DSR: 1/28/2013

Test SecurityReporting of Allegations

Support Documents • Testing Allegation Reporting Form • Notarized Statements written and signed by each individual

involved (teacher, proctor, BAC, principal and/or others). One statement signed by multiple individuals is not acceptable.

• Testing Schedule• Seating Chart and Room Number

16KDE:OAA:DSR: 1/28/2013

Test SecurityReporting of Allegations

Support Documents • Verification of Administration Code Training• Verify/Proctor Signature Sheet• Nondisclosure Agreement (classified/volunteers)• Student Name & SSID• For an allegation involving special education:

– verification of student test accommodation(s).– verification of Inclusion of Special Populations training.– a signed copy of the Code of Ethics, record of testing room and test

administrator for allegations regarding alternate assessment. 17KDE:OAA:DSR: 1/28/2013

Data Quality• The authoritative source of student data in

Kentucky is the state student data system, Infinite Campus (IC).

• Data are downloaded from IC for vendors and to populate internal applications (i.e., SDRR).

• Accountability requests made to OAA are cross-checked with Infinite Campus.

KDE:OAA:DSR: 1/28/2013 18

Data QualityKeys to Improved Data Matching• Correct entry of student information in IC is

critical (name, date of birth and SSID).• Consistently use the student’s name as it

appears in IC on all testing records.• State-testing programs are provided SSIDs.

KDE:OAA:DSR: 1/28/2013 19

Data QualityKDE/OAA Student Data Review and Reporting (SDRR) Application, organized by test, has two critical data accuracy functions:

1. Rosters • Required by regulation• Testing information for some tests• Record of accountability for all grades 3-12

– (Grade 9 is needed for PLAN and grade 12 is needed for College/Career Readiness reporting.)

2. Data Review (cleanup before reporting)KDE:OAA:DSR: 1/28/2013 20

Accountability Rules

KDE:OAA:DSR: 1/28/2013 21

• Tested– Every student enrolled on the first day of the

testing window must be tested for participation rate purposes.

• Accountable– Accountability determined by full academic

year (any 100 instructional days of enrollment).

Accountability Rules

KDE:OAA:DSR: 1/28/2013 22

• Spring K-PREP summative window (last 14 instructional days of district calendar) is the point in time where the 100 days of enrollment (full academic year) is determined.

MarchACT

AcrossYear

K-PREP EOC

September EXPLORE

PLAN

January-FebruaryApril-May Alternate

Assessment

April-JuneK-PREP (Pearson)

Summative Test Window

Accountability Rules

KDE:OAA:DSR: 1/28/2013 23

Accountability Terms for Schools• “A1” means a school under administrative control

of a principal and eligible to establish a school-based decision-making council.

• “A1” school is not an alternative program operated by, or as a part of, another school.

• “Alternative” program means all other education programs not classified as A1.

KDE:OAA:DSR: 1/28/2013 24

START

Has this student been

enrolled in this school for 100 instructional days

this school year?

YES

NO

Student scores will count

in this school’s accountability.

Has this student not been

in any one school but has been enrolled in

a district for 100 instructional days

this school year?

Student scores will count in district’s

accountability.

YES

NOStudent scores

will count in state’s

accountability only.

Flowchart for Determining Accountability For Students in A1 Schools 2013 Kentucky Performance Rating for Educational Progress (K-PREP)

Has this student been

enrolled inanother school for 100

instructional days this school

year?

NO

Student scores will count

in other school’saccountability.

YES

KDE:OAA:DSR: 2/7/2013 25

Every student enrolled on the first day of the testing window is tested for participation rate purposes. Accountability is determined by full academic year (any 100 days of enrollment).

Flowchart for Determining Accountability For Students in Alternative Programs2013 Kentucky Performance Rating for Educational Progress (K-PREP)

Accountable to A1 school where the student would have attended if not enrolled in the alternativeprogram(s) or to district (if student would have attended multiple A1 schools).

Was student placed by agovernmentalagency or the court system?

Was the student enrolled

100 days in the A1(or multiple A1s in a district)

and alternative program(s) combined or

in the alternativeprogram(s) only?YES

Accountable to the state

YES

NOWas the student enrolled

in an A1 school in a district for 100 days this school year before placement in an alternative program?

Note: A student at Kentucky School for the Blind, Kentucky School for the Deaf or Gatton Academy tracks back to the A1 school that he or she would have attended if not in the alternative program.

NO

YESNO

KDE:OAA:DSR: 2/7/2013 26

Accountable to A1 school

START:

27

SDRRStudent Data Review and Reporting Application

KDE:OAA:DSR-1/24/13

Work in SDRR2013 Spring Rosters

• ACCESS – (EL)• The ACT – Grade 11• K-PREP – Grades (3-12)

– Pearson– QualityCore- End-of-Course

(EOC)

2013 Data Review• EXPLORE (Feb.)• PLAN (Feb.)• The ACT • Alternate Assessment• K-PREP End-of-Course (QualityCore)• K-PREP (3-8, 10,11)• College/Career Ready Data

– COMPASS– The ACT (includes National Test Days)– KYOTE– WorkKeys– KOSSA– ASVAB– Industry

CertificateKDE:OAA:DSR-1/24/13 28

Training opportunities for SDRR Rosters and Data Reviews will be

announced in a DAC Monday E-mail.

TEDS

29

Prioritizing Your Data ReviewSchool and districts may review all available student information. OAA encourages focus on these priorities.

• Step 1 Accountability (Priority One)• Confirm your 100 day students that will contribute to your district or

school accountability.• Indicate any non-100 day students that should be removed from

your accountability.• Step 2 Demographics (Priority Two)

• Check free-reduced lunch and students with disability status (IEP), since these have potential to change over time.

• Confirm ethnicity and race categories.KDE:OAA:DSR-1/24/13

30

Two Sides of Data Review

Student ListingUse the Student Listing to check

1. Accountability2. Demographics

Ticket ListingUse Ticket Listing to

– Update ticket information

– Close tickets opened in error

– Check ticket status

KDE:OAA:DSR-1/24/13

31

Accountability Rule – 100 DaysTest Name Grade(s) 100 Days as of …EXPLORE 8 Spring 2012 PLAN 10 Spring 2012

The ACT 11 Spring 2013 K-PREP Window K-PREP 3-8, 10,11 Spring 2013 K-PREP Window Alternate Assessment 3-12 Spring 2013 K-PREP WindowEND-OF-COURSE 9-12 Spring 2013 K-PREP Window

Algebra IIBiologyEnglish llU.S. History

KDE:OAA:DSR-1/24/13

32

Rosters

• Everything is organized by test this year in SDRR.• Student Data Tool is no longer available in SDRR

but is a part of CIITS.

KDE:OAA:DSR-1/24/13

33

Rosters in SDRR

KDE:OAA:DSR-1/24/13

Assessment Opens Closes Fields/CommentsThe ACT March 6 March

28Name, SSID, DOB, Testing School, Exemptions

K-PREP Pearson Mid-April

Mid-June

Accommodations used on test collected and fields listed below

K-PREP EOC (English ll, Algebra ll, U.S. History, Biology)

Name, SSID, Grade, Gender, DOB, School Name, Test Type, Exemptions

Alternate K-PREP Same fields as EOC

College and Career Readiness

Name, SSID, Grade, Gender, DOB, School Name

9th Grade (Fall 2013 Plan)

Name, SSID, Grade, Gender, DOB, School Name

K-PREP Rosters• Verify student enrollment in grades 3-12• Include all students enrolled as of the first day of the testing

window – Mark Alternate Assessment and Foreign Exchange, as necessary

• Tickets created for:– Foreign Exchange Students – Nonparticipation

for K-PREP and EOC– Exempted Students – Medical or Extraordinary Circumstances

• Save an electronic version of all rosters

34KDE:OAA:DSR-1/24/13

Some data is pre-populated from

Infinite Campus

35

CIITSContinuous Instructional Improvement Technology System

KDE:OAA:DSR-1/24/13

36

What is CIITS?• Kentucky academic standards and deconstructed

standards searchable by subject and grade• Student level information for each teacher• Administrative level reports• Classroom Interim Assessments created from Item

Bank or local teacher/school/district items• Summative and Formative Assessments

KDE:OAA:DSR-1/24/13

POC = Instructional Supervisor

37

Assessment Data

KDE:OAA:DSR-1/24/13

K-PREP (includes Lexile & Quantile)

Alternate K-PREP

EXPLORE PLAN ACT

Stanford 10Growth

NWEA MAP

Discovery Education

District Formative

School Formative

KOSSA

ASVAB

WorkKeys

Industry Certs.

KYOTE

ACCESS for ELLs

COMPASS

End-of-Course

Proposed Additions

In Progress

38

Student Data Tool• Student Data Tool reports will be created and

pushed down from state level.• Districts will be capable of:

– Modifying reports– Modifying student sets

• Training sessions will be recorded.

KDE:OAA:DSR-1/24/13

39

Data Details• MAP – New linking study between MAP and K-PREP

by spring 2013.• Discovery Education – No linking, yet, to K-PREP.• State Assessment Individual Student results available

in fall 2013.• Career Data – Some available by fall 2013.• New Science Standards available after approval by

KBE.KDE:OAA:DSR-1/24/13