overview of the 2012-2013 alternate proficiency assessment administrator/test coordinator training...

Overview of the 2012-2013Overview of the 2012-2013Alternate Proficiency Alternate Proficiency

AssessmentAssessmentAdministrator/Test Coordinator

Training

12012-2013 NJ APA Administrator Training

Administrator and Teacher Administrator and Teacher TrainingTraining

This session is only for APA administrators.

Districts are responsible for teacher training.◦ Teacher training modules are available on the Pearson website

http://pem.ncspearson.com/nj/apa◦ An administrator must ensure that all teachers participate in

the training sessions.◦ Teachers may be trained immediately, but every effort must

be taken to ensure all teachers are trained prior to October 1, 2012.

◦ If a student enrolls after October 1 and a teacher needs training, train the teacher as quickly as possible. The modules will be available all year, though it is recommended that you download these modules and retain them in your district.


AgendaAgenda APA Participation pages 4 – 22 Student Transfers, Medical Leave, and Excessive Absenteeism

pages 23 – 31 Administrator Responsibilities pages 32 – 34 Material Survey, Pre-ID data, record changes, etc., pages 36 –

45 APA Test Design pages 46 – 80 Common Errors and APA Revisions pages 81 – 84 Scoring Rubric Information pages 85– 131 Sample Entry from 2011-2012 pages 132 – 140 Professional and Ethical Responsibility When Compiling and

Submitting APA page 141 – 146 Score Reports and Accountability pages 147 – 156 Administrator APA Review and Other Tasks pages 157 – 167 Request for Teacher /District Assistance pages 168 - 169 Contact Information pages 170 – 174 Winter Training Materials page 175


What is the Purpose of the What is the Purpose of the APA?APA?To measure performance of students with

the most significant cognitive disabilities on the CCCS – the knowledge and skills that ALL New Jersey

students are learningTo ensure that ALL New Jersey students

are included in the accountability system that is required by the No Child Left Behind Act


What is the APA?What is the APA?

An alternate assessment that measures achievement of CCCS specifically designed for students with the most significant cognitive disabilities.

A portfolio of student work that demonstrates a student’s performance of knowledge and skills that are LINKED to grade level knowledge and skills.◦ This is a requirement of the USDOE.


Who are the Students who Who are the Students who Participate in the APA?Participate in the APA?

Students with the most significant cognitive disabilities

Decisions about who participates in the APA are not based on:◦ Student’s disability category◦ Student’s educational placement◦ On which assessment the student is most

likely to score highest


Participation DecisionsParticipation DecisionsStudents with the most

significant cognitive disabilities will participate in:

◦ The APA in one/some content area(s) and the general assessment with accommodations in the other content area(s) or

◦ The APA in all content areas (LAL, Mathematics, and Science).

Note a student may not take the general education assessment and the APA assessment in the same content area.


Determining Who Participates Determining Who Participates In The APAIn The APA

Decisions are made:◦ By the IEP team◦ Separately for each content area

assessed by the general statewide assessment

◦ Using information about the student and the assessments Test Specifications/Sample Test forms Present Levels of Academic Achievement and

Functional Performance/Goals and Objectives Student Work Samples


Participation GuidelinesParticipation GuidelinesInformation on APA participation guidelines

is provided in your training packet.This checklist must be completed in order

to determine which students are eligible for the APA.

An administrator must sign the form and include it in the APA binder.◦ It is usually the sending school administrator, but

if both sending and receiving school agree on the answers to the questions, then a receiving school administrator may sign the form. A teacher may not sign the form.

Access to the Directory of Test Specifications and Sample Test for the general assessment is necessary in order to complete the checklist correctly.


Participation Criteria Participation Criteria ChecklistChecklist

1. Is the nature of the student’s cognitive disability severe?

◦ The student’s general level of cognitive functioning is significantly below the grade level expectations for his or her same aged peers and the student requires substantial modifications to the grade level curriculum.

◦ The student requires more intensive direct instruction than his or her peers to acquire knowledge and skills in the grade level curriculum.

◦ The student requires more extensive systems of support in order to participate in the grade level curriculum.


Participation Criteria Participation Criteria ChecklistChecklist2. Is the student’s cognitive disability so severe

that the student is not receiving instruction in any of the knowledge and skills measured by the general statewide assessment?

◦ While all students are to be instructed in the CCCS for their grade level, students with more significant cognitive disabilities typically receive instruction in the knowledge and skills at a less complex level than that measured by the general statewide assessment.

◦ Upon reviewing the sample test forms, if the student is NOT receiving ANY instruction at the same skill level, then the answer to this question is YES.


Participation Criteria Participation Criteria ChecklistChecklist3. Is the student’s cognitive disability so severe

that the student cannot complete any of the types of questions on the general statewide assessment in the content area, even with accommodations and modifications?

◦ Upon reviewing the sample test forms and questions for each content area of the general assessment, along with the state approved testing accommodations and modifications, if it is determined that the student is NOT able to complete any types of questions even with accommodations and supports, then the answer to this question is YES.


Participation Criteria Participation Criteria ChecklistChecklist4. Is the student’s Individualized

Education Program (IEP) aligned with grade level CCCS through modified expectations?

◦ The student is receiving grade level instruction at a significantly modified and less complex level than their grade level peers.

◦ The student may need more extensive systems of support, as identified in the IEP, in order to access the curriculum.


The answer to all four questions must be YES in order to assess the APA for a content area.

A signed copy of the checklist must be placed in the inside front pocket of the APA binder.

Usually the Sending district administrator signs the form. However, if the Sending and Receiving district agree on the APA status, the Receiving district administrator may sign the form.

14

Participation Criteria Participation Criteria Checklist Checklist (continued)(continued)


What Must Be Written in Each What Must Be Written in Each Student’s IEP?Student’s IEP? If the student is taking the general assessment,

list the assessment and the accommodations/modifications (from approved list) that will be provided.

If the student is not taking the general assessment in one or more content areas, include a rationale for this decision.

If the student is not taking the general assessment for LAL, Math, and/or Science, list that he/she will be participating in the APA.

IEP model forms and information can be found on the DOE website http://nj.gov/education/specialed/iep_form_ann.pdf


Assessment and CCCS Assessment and CCCS MaterialsMaterials

Additional copies of all APA materials, including the teacher training materials, may be obtained on the Pearson website: http://pem.ncspearson.com/nj/apa

Copies of the Core Curriculum Content Standards, test specifications, and sample tests for the general assessments can be obtained on the DOE website: www.state.nj.us/education ◦ Copies can be ordered for a fee by calling the

DOE publications office at 609-984-0905.


http://pem.ncspearson.com/nj/apa

What Grades and Subjects Are What Grades and Subjects Are Assessed?Assessed?Students in grades 3 – 8 and 11 must be

assessed in language arts literacy and mathematics.

Students in grades 4 and 8 must also be assessed in science.

Students in high school must be assessed in science the year they receive instruction in biology (grade 9, 10, 11, or 12).◦ High school students may have an APA

submitted for science in a different year than the year the APA is submitted for language arts and math.


Grade 11 and 12 StudentsGrade 11 and 12 StudentsIf a senior is new to the state this year and has

not participated in either the APA or the HSPA, the IEP team must determine which assessment is appropriate and the student must participate in this assessment.

Students who were juniors last year and should have participated in the APA, but did not, must participate in the APA this year whether they are still in 11th grade or are now in 12th grade.

Students in high school do not need to retake the APA if the test result was Partially Proficient. Score report results on the high school APA do not affect the ability to graduate.


Retained StudentsRetained StudentsIf a student has been retained in a

grade due to poor academic performance, the student must be reassessed in that grade.◦ Students who remain in the same assigned

grade twice due to reasons not related to academic performance must be assessed only once in that assigned grade, except for the circumstances described on the next slide.

Example of reason not related to academic performance: student is kept in the middle school an extra year because it was determined that the student was not yet ready for the high school environment (socially, structure of new large building, etc.)


Retained StudentsRetained Students If a student received a score report with a

Void Code rather than a Proficiency Level in a content area, and the student is retained in that grade, the student must participate in the APA again in the same grade.

Also, if a student in 11th grade received a Void Code in language arts literacy and/or mathematics, the student must be assessed again in these content areas, either in 11th grade if they remain in that grade, or as a 12th grader.


Assigned Grade LevelAssigned Grade LevelThe grade level of the student must be marked

on the Demographic Scan Sheet that is included in the portfolio. This is the assigned grade level used during scoring of the APA.◦ If a record change for the student’s grade level is

submitted in April 2013, the APA assigned grade level becomes the grade level provided during record changes.

The assigned grade level will be matched against the grade level of the CPI Links selected for assessment during the scoring process.◦ If the grade level of the CPI Links does not match the

assigned grade level provided during data collection, the entry will receive a zero score. Please note that students in high school (grade 9, 10, 11, or

12) are tested on the grade 12 CPI Links.212012-2013 NJ APA Administrator Training

LEP LAL ExemptionLEP LAL Exemption

If a student has entered the United States after July 1, 2012, AND is currently enrolled in a language assistance program, this student is EXEMPT from the LAL assessment.◦ This exemption is not allowable for high

school students.The APA scan sheet delivered to

districts in February 2013 must indicate the LEP LAL exemption.


Student Transferred to NJ District Student Transferred to NJ District from Out of State after October 31, from Out of State after October 31, 20122012

◦Students who require an APA, and enter your school after October 31, 2012, transferring from out of state, have missed the cut-off date for the development of the APA assessment.

Submit a binder, a completed scan sheet, and a letter from an administrator that indicates from which state the student transferred, the date of the transfer, and noting this as the justification for missing APA evidence.

The score report will indicate a Void 4, no portfolio evidence, when the student is a late registrant.

◦ Please note that instruction related to the CCCS must still occur throughout the year, regardless of the enrollment date.


Student Transferred from Student Transferred from One NJ District to Another NJ One NJ District to Another NJ DistrictDistrict

The district that has the student during any part of a collection period must compile the portfolio. Once the student transfers within NJ, the district must send the portfolio to the new district. The new district must complete the portfolio and submit it for scoring.◦ If a student is enrolled in a school or district

after July 1, 2012, the administrator may bubble on the scan sheet Time In School less than one year. This will remove the student’s results from the Accountability report. Note this is different than last year and is based on

the accountability waiver calculation.242012-2013 NJ APA Administrator Training

Student Transferred from Student Transferred from One NJ District to Another NJ DistrictOne NJ District to Another NJ District

If the student moved to another district you must send the portfolio to the district where the student is newly enrolled.

As long as a student is in your district, even for a limited time, you must begin instruction of the CPI Links, and begin collecting APA evidence. Be sure to transfer the APA materials to the “new” district in a timely manner.

You, the “former” district, do not need to submit a binder or letter to the APA program since the “new” district will send in the portfolio.

.


Students Leaving the Students Leaving the StateStatePlease note that if a student

transfers out of New Jersey prior to the end of the second collection period (February 15, 2013), the portfolio does not need to be submitted for scoring.

It may be sent to the out-of-state school for use when reviewing the student’s educational program, or given to the parent


Medical EmergencyMedical Emergency• Medical emergency is the occurrence of a severe

medical or psychiatric condition or episode which requires medical attention or supervision during which time the student is not able to participate in state assessments.

• APA students who are not receiving instruction due to a medical emergency may be eligible for a Void Code due to Medical Emergency when there is not enough APA evidence.

• APA uses the term extensive sick leave/hospitalization interchangeably with the term medical emergency. Note that no other definition may be used when determining a void due to medical emergency.

• See the requirements for this void on the next slide.


• When a student is out of school for an extended amount of time and not receiving instruction due to extensive sick leave or hospitalization, the portfolio may be eligible to receive a Void 1 (medical emergency code). Eligibility is based only on the information below. If the student is receiving instruction for 10 days or less

during a collection period, and The student has an extended hospitalization or leave

due to illness and is not receiving instruction, and An official record documenting the student absences is

available, then you must: include a letter on school letterhead from an administrator

documenting the medical emergency. include a formal district/school record of the student’s

attendance and absences. If all of these criteria are met then the portfolio will be

voided due to extended illness during one or both collection period(s).

28

Medical EmergencyMedical Emergency


Record of AbsencesRecord of AbsencesThe record of absences must be a formal

attendance record produced from the district’s record system. It must indicate all days school is open and which days school is closed for holidays, etc. This record must be marked with the student’s attendance – present or absent.

You may not submit an informal record of attendance such as a memo indicating the student was absent during the year.


Medical Emergency Medical Emergency PaperworkPaperwork

If the paperwork submitted in the APA binder does not include all necessary components as defined on the previous slides, a Void for Medical Emergency/Illness will not be granted.

An administrator may contact the Office of Assessments with any questions on this topic.


AbsenteeismAbsenteeism – – Non Medical Non Medical EmergencyEmergency If a student is receiving instruction for 10 days or less in a

collection period (excessive absenteeism), and the reason for absence is unrelated to a medical emergency/extensive illness, then the school administrator should contact the Office of Assessments.

If it is determined that a Void for No Evidence is appropriate, two things must be submitted by an administrator in the APA binder:◦ a letter on letterhead indicating the reason for the excessive

absenteeism (parent took student out of country, etc.)◦ a formal record of the student’s attendance/absences

produced from the district’s record system.


Test Coordinator’s Main Test Coordinator’s Main ResponsibilitiesResponsibilitiesEnsure APA participation guidelines are used

correctly by IEP teams. ◦ Ensure IEP teams have all of the necessary information

about the general assessments and the APA participation guidelines clarifying questions to ensure the decision is made based on the requirements.

Ensure all APA teachers complete the teacher training modules and review all training materials including the Procedures Manual and Content Guide Five Item Samples document.◦ Have the teachers sign in to your district training and keep

a record of attendance.◦ Ensure all teachers view all modules.◦ Ensure all teachers utilize the critical additional training

materials listed above as the materials as a whole deliver the information necessary to successfully implement the APA.


Test Coordinator’s Main Test Coordinator’s Main ResponsibilitiesResponsibilities

Periodically review teacher’s APA process, use of test specifications and selected CPI Links, instructional lessons, etc.◦ This review must occur at the beginning of each

collection period, and at least once prior to the end of each collection period. Reviewing the materials before the cut off date allows time if necessary for production of additional/different APA evidence.

◦ Ensure that only the 2012-2013 CPI Link documents are used. The APA continues to use the older standards until the other assessments finalize the alignment to the Common Core Standards and the APA specifications are revised.


Ensure registration of APA students, review and update district contact info, and submission of Student Pre-Identification file prior to the deadline.

Complete Demographic Scan sheet providing accurate data

Complete final portfolio review prior to shipping

Ensure timely shipment of portfoliosReview APA record change rosters and

correct/revise any inaccurate data

34

Test Coordinator’s Main Test Coordinator’s Main ResponsibilitiesResponsibilities


Vendor TransitionVendor TransitionThe APA program is transitioning to a

new vendor, Questar Assessment.This transition will begin in

September.Details and deadlines related to

student registration, Pre-ID submission, the new Online Entry Cover Sheet, etc., will be forthcoming.

We appreciate your patience and will send details on these topics regularly as they become available.


Materials Survey – Registration Materials Survey – Registration of APA Students and District of APA Students and District InformationInformationRegister the number of APA students in your

district no later than (date to be determined).

◦ Failure to register your students will result in a failure of delivery of APA materials to your district.

◦ Survey should be completed online (see memo).◦ Do not register out-of-district students that are

assessed in New Jersey. The receiving district will register these children.

◦ Register students that are sent by your district to an out-of-state placement as if they are in their home school.

◦ If no APA students this year, then indicate 0 (zero) for the applicable grade level.

◦ Confirm/revise district address, emails, and contact information to ensure accurate deliver of information.


Online Registration – Material Online Registration – Material SurveySurvey

The online material survey is not yet available. A memo will be sent to districts when the website is ready.

https://nj-servicepoint.questarai.com/Login.aspx

is the Questar website address for use in the future.

Log on information including User IDs and Passwords will be provided to your Chief School Administrator (CSA)

If your CSA does not receive this information by October 30, call Questar customer service at 1-866-690-4733 or send an email to [email protected]


mailto:[email protected]

Students Sent to an Out-of-State Students Sent to an Out-of-State SchoolSchoolThe Sending District is responsible for communication to

the out-of-state placement. ◦ This includes providing training materials to the school and

ensuring that the CCCS is instructed and assessed. ◦ This includes ensuring that the APA is in process, evidence is

being collected, and that the APA will be completed and submitted to you prior to the return portfolio shipment deadline.

Portfolios must be sent from the out-of-state school to the sending district prior to February 15, 2013. The sending district includes these portfolios in the district’s shipment.

Registration of these students, completion of scan sheets, record changes, etc., is the responsibility of the sending district.◦ Include these students in the sending school APA enrollment

counts.


Electronic Pre-ID File Electronic Pre-ID File SubmissionSubmission

Pre-ID File◦ This is the first opportunity to submit

accurate student information/data. ◦ Administrators will need to provide such

info as student birth date, SE classification code, grade level, etc.

◦ Student demographic information must be submitted electronically on the Questar website. Please be patient while the website is developed. Pre-ID Instructions will be sent to your Chief

School Administrator. Pre-ID information will be printed on the

demographic scan sheet in lieu of gridding the information.


Electronic Pre-ID FileElectronic Pre-ID FileLast date to submit pre-ID files is:

Date to be determined. It is not advisable to wait until the last day

since you may find that the file is not accepted due to data errors.

Within the submission window you may submit a revised file when necessary.

Pre-ID information will be printed on scan sheets for those participating districts and sent with the Return Materials shipment in February 2013. Blank scan sheets will also be provided.

Errors/omissions on the Pre-ID label must be corrected by gridding the info on the scan sheet.


Scan SheetScan SheetStudent demographic scan sheets, along with

directions for the completion, are shipped to districts in February.

This is the second opportunity for districts to submit accurate student information/data.

A county-district-school code (CDS) must be provided for the school the student is attending. This is either a Home (neighborhood) school or a Receiving school.◦ If the student is in a receiving school, a CDS code for the

sending school must also be provided. The sending school must provide this code to the receiving school for inclusion on the scan sheet.

◦ The home or receiving school code is pre-gridded on the scan sheet. APA will begin to use the most up-to-date CDS codes for the private and

specialized, separate schools beginning this year. Please ensure you know your current code which is also being used for NJASK and HSPA. We will no longer be using CDS codes that start with “90”.


Scan Sheet Scan Sheet If a Pre-ID file was submitted, check the

preprinted information on the scan sheet for accuracy.◦ You may correct information submitted in the

Pre-ID file by gridding the correct information on the scan sheet. The gridded information will override the preprinted information. The only exception to this is changes to the student

name, which may only be corrected during the record change process. We must maintain the same exact student name on the file in

order to process the information until record changes. Only at that point can we make changes to the spelling of the student’s name.


Taking General Assessment Taking General Assessment FieldField

Detailed instructions are provided in the use of this field in the Pre-ID directions and the Scan Sheet directions.

Only code this field when the student is NOT taking the APA, but is taking the general assessment (NJASK, HSPA, NJBCT) in a content area.◦Remember, a student may only take the

APA OR the general assessment, not both, for a content area.

◦Marking the student as Taking the General Assessment will Void all results for the APA.


Record Change ProcessRecord Change ProcessA roster of APA student demographic

information is shipped to all districts in April 2013. (Will verify date with new vendor and confirm in Record Change Memo.)

For student’s who remain in their Home (neighborhood) school, the “Home” district must review and submit roster changes.

For student’s who have both a receiving and sending school, both districts must review the record change rosters for accuracy.

Beginning this administration, the Sending School, not the Receiving School will submit the record changes to the vendor.


Record Change ProcessRecord Change ProcessThe Sending School/Home School is responsible

for correcting all student information that was provided on the scan sheet.◦ The Receiving School must also review the roster, as errors

(e.g. , CDS code, demographic information) may impact the student score reports and accountability reports. Corrections and revisions to the data must be submitted to the

student’s Sending School at least several days prior to the record change deadline. The Sending School must submit these changes to Questar during the record change timeframe.

This is your final opportunity to correct/revise the data. This includes the field Taking the General Assessment. The final record change roster data will be used for the accountability report. No changes will be made after this time.


APAAPA

What will this portfolio look like?

What are the required components?


What is the Format of the What is the Format of the APA?APA?Portfolio: Paper based, submit in a 3-

ring binder◦ Binders are mailed to districts in October,

based on the APA registration counts the district administrator submitted.

Content Areas: Mathematics, Language Arts Literacy, and Science◦ Science is included only in grade 4, 8, and

high school.


What is the Format of the What is the Format of the APA?APA?

Entries relate to content standards, grade-level CPIs, and CPI Links. Each content area will have four entries, each one reflecting the assessment of one CPI Link.

Evidence documents educational instruction and student performance of skills. The instructional activity used at the beginning and at the end of the CPI Link instruction is submitted as the assessment evidence.


Test DesignTest Design

NOTE: NO entry should contain more than four pieces of evidence.

This graphic represents a student who is being assessed in all three subjects with the APA. Not all portfolios will include all three subjects.

49

•APA Portfolio

•Table of Contents

•Language Arts•4 Entries

•Mathematics•4 Entries

•Science•4 Entries


Test Design ExampleTest Design Example – – Language Arts Language Arts (Same Structure for Each Content Area)(Same Structure for Each Content Area)

NOTE: NO entry should contain more than four pieces of evidence. An entry containing more than four pieces of evidence will result in zero scores for all dimensions.

50

•APA Portfolio

•Table of Contents

•Language Arts•Entry 1




•Entry Cover Sheet reflecting 1st

Standard, Strand, CPI, and CPI Link

•Entry Cover Sheet reflecting 2nd


•Entry Cover Sheet reflecting 3rd


•Entry Cover Sheet reflecting 4th


•2 pieces of evidence •2 pieces of evidence •2 pieces of evidence •2 pieces of evidence


When Do I Collect When Do I Collect Evidence?Evidence?

Activity Collection Period

Initial Activity September 4 - November 16, 2012

Final Activity December 10, 2012 - February 15, 2013

Dates on the evidence must include month, day, and year.

Be sure to plan early in the year when you will instruct and assess each CPI Link and content area.All students will need to be assessed on an

initial activity within the correct time frame above, even if the majority of instruction will not occur until the winter.


Four Entries per Content Area

Based onStandard and Strand 1




Select CPI &CPI Link




Provide instruction & document activities and results

One Piece of EvidencefromSeptember 4 - November 16, 2012

One Piece ofEvidencefromDecember 10, 2012-February 15, 2013




One Piece of EvidencefromSeptember 4 - November 16, 2012


One Piece ofEvidencefromSeptember 4 - November 16, 2012


One Piece ofEvidencefromSeptember 4 - November 16, 2012


Entry One Entry Two Entry Three Entry Four


Page NumberingPage NumberingA table of contents that references page

numbers in the portfolio must be included.Page numbering for evidence that is longer

than one page should be numbered using a combination of page number and letters (i.e., 2, 2a, 2b, etc.)

For example, a student was given a reading comprehension test (matches the CPI Link) with 8 test items. The test was two pages long, but it is one activity/piece of evidence. The pages might be numbered page 3 and 3a.

Entry Cover sheets get their own page numbers.


Entry Cover SheetEntry Cover SheetThe vendor will build an electronic

entry cover sheet that will allow teachers to use drop-down boxes to populate most of the fields on the form. This form will then be printed from the web.◦Teachers must still type/write in their

activity descriptions on the form.◦This form will not be ready until later

in the fall.542012-2013 NJ APA Administrator Training

Entry Cover SheetEntry Cover SheetTeachers should use/complete the old

version of the cover sheet as a place holder until the new form is available.

The Procedures Manual and Pearson’s website has the old style Entry Cover Sheet for 2012-2013.

Once the new online form is available teachers should use the old form to assist in transferring the information to produce the new Entry Cover Sheet.


Contents of the EntryContents of the EntryEach entry requires documentation (evidence) of

student performance of the skill stated in the CPI Link.

Two pieces of evidence is the requirement for an entry.

Includes two different activities.◦ One piece of evidence from the initial activity, and one

piece of evidence from a different, “final” activity.Evidence may come from classroom work,

community-based settings, or other places where student is working on grade-level CPI Links.


Contents of the Entry Contents of the Entry (continued)(continued)

The first piece of evidence documents the initial, or very early, performance of the CPI Link. This evidence must capture what the student does/does not know related to the CPI Link. This evidence must reflect the entire CPI

Link in one activity, and be collected between September 4 and November 16, 2012.

In order for the CPI Link to be eligible for the student’s APA entry, the accuracy score on the activity can not be above 39%.Additional information on scoring the evidence

is included later in the training.


The second piece of evidence documents final performance of the CPI Link. This evidence should capture what the student has learned related to the CPI Link.This evidence should assess the same CPI

Link using a different instructional activity, and be collected between December 10, 2012, and February 15, 2013.

More information on what makes a different activity is found in the Teacher Training Modules.

Contents of the Entry Contents of the Entry (continued)(continued)


Evidence RequirementsEvidence RequirementsEvidence must document the entire CPI

Link noted on the Entry Cover Sheet.Evidence should relate only to the CPI

Link, not assess other skills as well.◦ Providing evidence that documents

multiple skills (skills not part of the selected CPI Link), or less than the entire CPI Link, will result in a low scoring entry.

Have at least 5 items/questions/task elements included in each activity


How are CPI Links Related How are CPI Links Related to the CCCS?to the CCCS?

Grade-level CPIs are the building block.Content Centrality establishes what

concept must be assessed.◦ Intent/Essence of the CPI

Performance Centrality directs the level of skill statement.◦ Complexity/difficulty

APA CPI Link can be less complex than the grade-level CPI and/or less difficult

Three types of links–Matched, Near, and Far–reflect the degree of complexity/difficulty when compared to the grade-level CPI.


CPI 3.1.5G13 Recognize figurative language in text (e.g., simile, metaphor, personification, alliteration)Essence of the CPI: Identify figurative language

Matched Link Near Link Far Link Find examples of three

different types of figurative language in a text*

Match at least two different types of figurative language to its type*

Identify examples of similes Identify examples of

personification Identify examples of

alliteration Identify examples of

metaphors

62

CPI LinkCPI Link

•Esse

nEsse

ncece

•Near LinksNear Links •Far LinksFar Links•Matched LinksMatched Links

• Lin

ksLi

nks

• Lin

ksLi

nks

• Lin

ksLi

nks


Using the CPI Links for Using the CPI Links for InstructionInstruction

63

CPI Links should not be used in isolation. Read the Standard and Strand for the

overall concept.Read the CPI and Essence.Make sure instruction is within the Essence

of the Standard, Strand, and CPI.◦ Evidence of instruction on CPI Link will be scored not only

on the discrete Link but also within the Essence of the standard, strand, and CPI

◦ For example, 3.1.3G3 is a reading standard with a Matched Link that states “Interpret information from graphs, charts, and/or diagrams found in text.” The evidence must assess the student’s interpretation of graphs/charts/diagrams within the context of reading text and not within the mathematical context of data analysis.


Assess ONLY one CPI Link◦ Remember, when reviewing the CPI Links in the APA

Procedures Manual, a CPI Link is only one bullet. ◦ Once the type of link to assess a student is

determined (Matched, Near, or Far Link), then select only one bullet in that column.

Assess the entire CPI Link providing the necessary systems of support

Include the student’s response to the test item◦ Must be completed in the student’s mode of

communication◦ Ensures that the student can review his own work

This is a good way to “test” if the evidence documentation includes the necessary components, but it is not a requirement that you document the student’s actual review of the work.

64

Evidence Evidence MustMust


Acceptable Format for the Acceptable Format for the EvidenceEvidence

• Evidence must show the test items and student responses. Acceptable evidence includes– Samples of graded student work– Series of captioned photographs

• Must evidence the student actually performing the skill

– Snapshot of completed student work that documents student performance of the skills contained within the CPI Link

• Must be a photo of a student work product that is too large or bulky to include in the portfolio (i.e., a poster, a written piece using objects or textured cues, etc.)


66

Acceptable EvidenceAcceptable EvidenceWhat else is needed to make evidence acceptable?All evidence must include: •Student’s Name•Date of Activity Completion using Month, Day, and Year (must be within the collection windows to be valid)•The accuracy of each student response

• Use + (plus) for correct and – (minus) for incorrect responses•The independence level of each response

• I=Independent performance, V=Verbal prompt, G=Gestural prompt, M=Model prompt, P=Physical prompt (presented from least to most intrusive)

• More information on use of prompting appears later in the presentation


What else is needed to make evidence acceptable?All evidence must include: A minimum of 5 items that assess the CPI link

◦ A piece of evidence must include at least 5 test items that assess the skills contained within the CPI Link. For example

Ordering decimals would require 5 sets of numbers with which to order—not just that 5 decimals were ordered.

More information is found in Modules IV, V, and VI of the online training, in the 2012-2013 Procedures Manual, and in the CPI Content Guide and Five Items Resource.

67

Acceptable EvidenceAcceptable Evidence


For CPI Links, an activity must include at least 5 test items. ◦ For example

A student will answer 5 comprehension questions about a completed job application.

A student will use multiplication to check the answers to 10 division problems.

NOTE via an example: If a student is given 7 pictures, and is asked to circle the plants, they must also be asked to cross out the pictures that are not plants. They may not have items that do not require a response and then score it as if the item was correct. Also, there must be enough plant pictures to test the skill, and a balance of right to wrong options.

A A MinimumMinimum of 5 Test of 5 Test ItemsItems


Rule 5 : Example Five ItemsRule 5 : Example Five ItemsWhen the Link has an “and” in it, 5 items

across the skills are required rather than 5 items for each part of the Link. ◦ For instance, 4.2.7E1 Near Link “Calculate

the area of a triangle, rectangle and square”


Acceptable EvidenceAcceptable EvidenceStudent work samples serve as the best

evidence to document all of the necessary components of the assessment activity.◦ This type of evidence includes the test

items/questions, response options (if applicable), and student responses.

◦ Teachers may need to scribe the student’s responses when necessary.

The use of videotape, audiotape, data charts, and most photographs are not acceptable evidence.


Student Work Samples Used Student Work Samples Used as Evidence May Beas Evidence May BeComputer screen printouts of student’s test

item and response after hitting a switch.◦ Summary printouts where each response is not

evidenced is unacceptable.An essay written using picture symbols and

scored using a rubric.A worksheet completed by the student.Drawings to extend patterns.Completed job application using name,

address, and other stamps.A graph made from ordered pairs.


Student Work Samples May BeStudent Work Samples May BeA paragraph written using sentence

stripsA printout of a sentence written using

a picture writer (i.e., Writing with Symbols, PicWriter, etc.)

A snapshot of work too large to fit into the portfolio (a poster, a model, etc.)


Acceptable Photographic Acceptable Photographic EvidenceEvidence

A series of captioned photographs that document the student performing the skills included in the CPI Link will be accepted only if:◦ Each item and the student’s response is

clearly present in each photograph Must include a description for each photo Must include running record of the items

performed Must include accuracy and independence

information for each item and include total percent scores for each dimension

Must meet the requirements for the Universal Scoring Rules


Acceptable Photographic Acceptable Photographic Evidence Evidence (continued)(continued)

A photograph of a large or bulky completed project will be accepted only if the picture captures the evidence requirements for the activity.

A description of the activity must be included along with the documentation of the scoring, but the actual work must be clearly shown in the picture.

The Universal Scoring Rules requirements must be met.


Scribing Work Scribing Work SamplesSamples

Acceptable Scribing Technique Unacceptable Scribing Technique

Uses the student’s communication mode to document the student’s answer

Does not use the student’s communication mode

Allows the student to review her/his work

Does not allow the student to review her/his work

Is used to record the student’s response on a work sample from eye gaze, a voice-output device (VOD), Augmentative Communication Device, reach and grasp, etc.

Does not show the student performance of the skill (observational data)


Unacceptable EvidenceUnacceptable EvidenceData chartsSingle photographs of student working on an activityPhotographs of student work that could have been

includedAny work that is not completed by the student

◦ Note, the student responses must be marked at the time of the activity, not at a later point in time, in order to be acceptable evidence.


Has fewer than 5 items/questions/task elements

Has no name, date, or incomplete dateIs scored using anything other than percentages◦e.g., ratios (4/5), percent ranges (90-100%), letter grades (A, B+)

Has percent scores that can not be replicated by scoring staff

77

Unacceptable EvidenceUnacceptable Evidence


• Universal Scoring Rules must be met on two pieces of evidence for the entry to be scored based on the rubric. Otherwise, the entry will receive a score of zero for all dimensions.

• If any entry does not have at least two pieces of evidence OR has more than four pieces of evidence per entry, it will receive a score of zero. Two pieces of evidence are all that is

required for APA.

78

EvidenceEvidence


79

Universal Scoring RulesUniversal Scoring Rules1. The evidence must include the student’s name.

2. The evidence must include the complete date (month/day/year). Verify that the dates fall within the appropriate collection period:

1.September 4, 2012 - November 16, 2012 for the first piece of evidence2.December 10, 2012 - February 15, 2013 for the second piece of evidence

3. The evidence must be presented in the appropriate format.1. The evidence must include a writing rubric when specified in a Writing CPI

Link.

The evidence must reflect the student’s mode of communication.

The evidence must include at least 5 items. The evidence must show student responses for at least 5 items.. The evidence must assess the entire link while connecting to

the essence of the CPI, standard, and strand. Both pieces of evidence must assess the same CPI Link and

skills. The evidence must not include more than the skills contained

within the CPI Link.

Note: The USRs pertain to both student work samples, as well as the writing rubric when required. If the entry does not adhere to all rules it will receive zero scores.2012-2013 NJ APA Administrator Training

Activities – Errors to AvoidActivities – Errors to Avoid The first piece of evidence should not include a more

difficult application/context of the skill than the final activity/evidence.

Examples of this type of error (a more difficult initial activity) are below:

• The initial activity has longer test items with more complex steps (e.g., open-ended essay questions vs. multiple-choice) than the final activity.

• The initial activity has multiple-choice test items where no correct answers are offered as options.

• Fewer additional supports (steps, formulas, pictures, application, calculators, etc.) are given in the initial activity than the final activity.

• Regular grade-level text is used in the initial activity, but summarized text with picture cues and key details highlighted are used in the final activity.

• No explanation of what to do is given for the initial activity, but directions are given for the final activity. Note: directions should ALWAYS be given during an activity.

• No prompts on the first piece of evidence with no answers written, or all answers are clearly off topic (answers are not at all related to concept).

• The initial activity is not in the student’s mode of communication, but the final activity is.


Common Mistakes found in Common Mistakes found in 2011 - 2012 Assessments2011 - 2012 Assessments

Assessing the skills contained in the CPI Link without connection to the critical essence, intent or big idea of the content and concepts of CPI, Strand, and/or Standard

Assessing the skills contained within the CPI Link without correct understanding of the Content or concepts of CPI, Strand, and/or Standard

Assessing more than one CPI Link in an entryAssessing different parts of a CPI Link in

each piece of evidenceAssessing only part of the CPI Link


Common Mistakes found in Common Mistakes found in 2011 - 2012 Assessments2011 - 2012 AssessmentsFailure to mark all test items with accuracy

and/or independence/prompt informationFailure to give supports on the first activityFailure to give prompts, instead marking

student response as “Don’t Know” or “No Response”◦ For assessment, an item may not be marked as

“don’t know” or “no response.” Teachers must utilize a prompt(s) in order to assist the student in generating a response.


Changes for 2012-2013Changes for 2012-2013Districts must use the 2012-2013

updated CPI Links when assessing a standard, strand, and CPI for the APA.◦ A few CPI Links were revised to make them more

clear.◦ The asterisk following a link is a reminder that there

is a sample test item available in the Five Items document.

Important CPI Link clarifications are included in the new Content Resource Guide and Five Items document. Be sure to read all of the relevant grade-level materials thoroughly.


Content Guide RevisionContent Guide RevisionSome curriculum issues have been identified

and must be avoided in the future. Some evidence contained errors in the instruction of the skills.◦ Example: Our Solar System only contains one star,

the Sun. Therefore, stars may not be used as a correct answer when asking what is in our Solar System.

◦ Example: Teachers are submitting math Symmetry work sheets that don’t correctly assess the CPI Link that was selected.

These topics were addressed in last year’s document.

These errors will result in zero scores for 2012-2013.


Developing Standards-Based Developing Standards-Based EntriesEntries


86

Step 1: Select a CPI and one related CPI Link to be assessed. Step 2: Plan instruction and assessment relating to the CPI and CPI

Link. Step 3: Design activities that will be used to assess the CPI Link. Step 4: Assess the student to get an initial piece of evidence for APA

purposes. Step 5: Implement instruction.

◦ Ensure instruction reflects the essence of the CPI, strand, and standard.

◦ Ensure the instruction is age- and grade-level appropriate.◦ Retain a working folder of instructional activities and classroom-based

assessments implemented between the activities that generate the initial and final pieces of APA assessment evidence.

Step 6: Determine when evidence can be collected to document the final instructional assessment of the CPI Link for APA purposes.

Step 7: Based on the student’s accuracy score and level of prompt information on the “final” activity, determine if additional instruction and collection of evidence needs to occur for the entry.

Step 8: Review APA evidence to ensure that all information related to test design requirements are included.

Developing an EntryDeveloping an Entry


How is the APA Scored?How is the APA Scored?Each portfolio entry is scored based on

specific scoring criteria described in the APA scoring rubric found in the Procedures Manual.

The criteria are grouped into three categories called dimensions.


APA Scoring RubricAPA Scoring RubricScoring Dimensions

◦ Complexity Link Match, near, or far link selection

◦ Performance Percentage of accuracy when performing skill

◦ Independence Percentage of time skill was performed

independently


Scoring DimensionsScoring Dimensions


Scoring Rubric and APA Scoring Rubric and APA FormsFormsThe APA Scoring Rubric found

on the previous slide is also found in the 2012- 2013 APA Procedures Manual. Don’t forget to review the Procedures Manual, which includes the APA Scoring Rubric, scoring clarifications, and other topics.


Complexity DimensionComplexity DimensionEvaluates the CPI Link assessed and how closely the complexity and difficulty (Matched, Near, Far) links to the Core Curriculum Content Standards (CCCS) and grade-level cumulative progress indicators (CPI).


92

0 1 2 3 4

Evidence provided is unscorable; all dimensions will receive a score of zero

CPI Link was assessed, but there are major flaws in the evidence

CPI Link is a Far Link to the grade-level indicator

CPI Link is a Near Link to the grade-level indicator

CPI Link is a Matched Link to the grade-level indicator

Complexity DimensionComplexity Dimension


Complexity Score of 2, 3, Complexity Score of 2, 3, or 4or 4A score of 2, 3, or 4 is obtained if

the CPI Link is assessed correctly and a different activity is used for the initial and final piece of evidence.◦A score of 2 is obtained when a Far Link

is assessed correctly.◦A score of 3 is obtained when a Near

Link is assessed correctly.◦A score of 4 is obtained when a

Matched Link is assessed correctly.93


Two activities are required when submitting evidence for an entry.◦ Both of these activities must assess the same

CPI Link.◦ The two activities must be different activities in

order to address the Universal Scoring Rules.An activity is considered to be different

than another when the context of the instruction of the CPI Link is different.

An activity is also considered to be different from another when the application of the skill in the CPI Link is different.

94

What Makes an Activity What Makes an Activity Different?Different?


While the skills assessed and the supports needed to perform the skills remain the same, an activity is considered different:◦With a different context in which the

framework or events within which the student is demonstrating the skill changes

◦With a different application in which the types of questions used, or the way in which the student demonstrates the skill changes

95



Examples of two activities with different context:

The student is working on following vertex-edge graphs1. after whole class introduction and demonstration of vertex-edge

graphs, the student completes a worksheet 2. after having a lesson on flight plans, the student uses a vertex

edge graph to follow flight paths The student is working on adding two matrices

1. by completing a worksheet2. by completing a different worksheet with different problems

than used in the initial activity The student is working on writing a cover letter

1. after researching a job of a particular interest, write a cover letter that would be used when applying for the job

2. after listening to speakers about different careers, write a cover letter for a job of interest

The student is working on identifying theme in a given work of fiction1. by reading The Outsiders and answering multiple-choice

questions2. by reading Flowers for Algernon and answering multiple-choice

questions 96



Examples of two activities with different applications:

The student is working on classifying types of planets1. by using a graphic organizer2. by matching

The student is working on addition and subtraction1. by completing a worksheet with problems on it2. by using a Smart Board to complete math problems

The student is working on writing an essay1. by using a computer or word processor2. by using paper and pencil

The student is working on identifying theme in a given work of fiction1. by reading a passage of The Outsiders and then answering

multiple-choice questions2. by reading a passage of The Outsiders and then answering

short answer questions97



Teachers Must Think about the Teachers Must Think about the Two Activities in ConjunctionTwo Activities in ConjunctionAs you plan for instruction, you should not think about the initial and final activity in isolation. Things to think about:What type of activities might be easier and harder for the student? Do not use the easier activity as the final activity.What type of prompt, if any, might the student need on the final activity? Do not use a more intrusive prompt level on the final activity than was used on the initial activity.How can I keep the student engaged in the various activities? How will the initial activity be different than the final activity and still engage the student?


99

Complexity Score of 1: Major Complexity Score of 1: Major FlawsFlaws An entry that demonstrates work in a CPI

Link but has a major flaw will score a 1 in Complexity.

• A major flaw includes 1. Assessing the same activity for both pieces of

evidence2. Assessing only part of the CPI Link– This may be found in the work sample, and both

pieces of evidence must be assessing the same part of the link.

– This may also be found in the writing rubric.


100

Complexity Major Flaw Example (Same Complexity Major Flaw Example (Same Activity)Activity)

•9/10/12

•2/11/13

The directions given to the student were “Match herbivores, carnivores, and omnivores to the type of food they eat.”


101

Complexity Major Flaw Example(Activity assesses only part of a link)

Complexity score would be 1 for only assessing part of the link.

3.1.8G1 Matched Link: “After reading newspapers, magazines, or electronic texts, identify the type of propaganda used and explain its purpose”

10/1/12

1/10/13


Complexity Score of ZeroComplexity Score of ZeroWhen the Universal Scoring Rules

are not adhered to, the entry will receive a score of zero for Complexity, Performance, and Independence.

Please review Teacher Training Module VII: Universal Scoring Rules for more information.


This dimension measures the student’s accuracy performing the skills represented in the CPI Links identified within the portfolio.

Performance DimensionPerformance Dimension



104

• Each item must be marked as correct (+) or incorrect (–).

• The score for accuracy should be expressed as a percentage.

• Any item for which the student received a physical prompt should be marked as incorrect (–P).

• The second piece of evidence should not include a more intrusive prompt than was used on the first piece of evidence.

• The student’s accuracy score on the first piece of evidence should not exceed 39%.

• The first piece of evidence should not be more difficult than the second piece of evidence.

• When assessing a 3.2 Writing Link that requires a writing rubric, there should be feedback on the student’s work explaining how/why the work was scored in relation to the writing rubric.


Each item must be marked as correct (+) or incorrect (–).◦ The score for accuracy should be expressed as a

percentage.Scoring the activity for accuracy requires a

consistent understanding of when to mark an answer right or wrong.◦ Certainly, if the student performed the skill

independently, the answer is either correct or incorrect.◦ But what about when the student receives a prompt? How

do you score the item correct or incorrect?

106

Scoring Student Evidence for Scoring Student Evidence for AccuracyAccuracy


• Understanding the difference between the following provisions will help ensure that the scoring information on the evidence is accurate:•providing task directions•providing supports•providing indirect prompts (verbal, model,

and gestural)•providing physical prompts•providing the answer (directly prompting

the student using a verbal, model, or gestural prompt to correctly answer the question)

107



• If the student performs the item independently, then•Score as correct (+) any item the student

performs correctly.•Score as incorrect (–) any item the student

performs incorrectly.•Any system other than +/– must include a key.

•If you used a different scoring system, such as ✗ equals incorrect and ✓ equals correct, you must explain that clearly on the evidence.

108



• If the student received an indirect verbal, gestural, or model prompt when performing the skill, then•Score as correct (+) any item the

student answered correctly.•Score as incorrect (–) any item the

student answered incorrectly.•Remember to mark next to each item

the prompt level used: V, G, or M (e.g., +V means that an indirect verbal prompt was given and that the student answered the item correctly.)

109



• Any item for which the student received a physical prompt should be marked as incorrect (–P).•A physical prompt is a direct prompt

whereby the teacher physically guides the student to the correct answer (e.g., hand over hand).

•However, if the student receives a physical prompt when performing the skill, then•Score the item as incorrect (–P).

110



• Total accuracy score must be in the form of a percent.•The number of items correct divided by

the total number of items multiplied by 100 equals the total accuracy score.

Accuracy

# items correct divided by # of total items times 100 equals % Correct

____/_____ x 100 = ______%

111



• The second piece of evidence should not include a more intrusive prompt than was used on the first piece of evidence.

Prompting can affect the reliability of the performance score if not implemented correctly.

• Therefore, it is important to adhere to the prompt level hierarchy described in your 2012-2013 Procedures Manual.

• If the second piece of evidence includes a more intrusive prompt level than the first piece of evidence, a score of 1 will be assigned for the Performance dimension.

PerformancePerformance


Scoring Performance - Scoring Performance - Non-Non-ExampleExample

10-11-122-1-13


The initial piece of evidence should not be more difficult than the final piece of evidence.

◦ If a support (e.g., a calculator or formula) is provided in the final piece of evidence, it must also be provided in the initial piece of evidence. Otherwise, the initial piece of evidence would be more difficult.

◦ Supports provided in the initial piece of evidence do not have to be present in the final piece; they can be faded if no longer needed by the student.

◦ The final piece of evidence can be of the same difficulty level as the initial piece of evidence, or it can be more difficult.

◦ The student should be given enough task directions, supports, and prompting (if needed) on the initial piece of evidence to elicit on-topic responses, even if incorrect.



Busy, Overly ComplicatedBusy, Overly Complicated

10-25-12Sandy

Problems: the coordinate grid isn’t labeled, the answer choices indicate the grid was intended to be marked by 2’s, the colored lines make the graph jumbled, and use of colored pencils makes the task overly complicated.

4.3.12B2: Near Link: “Given a graph of a linear function, identify the x- and y- intercept”


116

Make It ClearMake It Clear4.3.12B2: Near Link: “Given a graph of a linear function, identify the x- and y- intercept”

This activity clearly and simply assesses the link.

Sandy Date: 1-17-13100% Independent


• When assessing a 3.2 Writing Link that requires a writing rubric, there should be feedback on the student’s work explaining how the work was scored in relation to the writing rubric. ◦ If the student did not earn all of the points for a

particular dimension on the writing rubric, there should be feedback on the student’s work explaining why/how the work was insufficient or incorrect.

◦ Even when a student earns all of the points allotted on a writing rubric, there should still be feedback affirming the student’s work or explaining why it was particularly acceptable in accordance with the writing rubric.



Writing Feedback

Non-example: Evidence does not include any feedback that corresponds to writing rubric.

118


•Larissa 10/28/12 10/28/12

3.2.8B2 Near: Write a short story that includes a setting, a plot, and characters rubric

Amanda T.


Independence evaluates the extent to which the student completed items/tasks independently.

119

Independence Dimension


Not all direction is considered a prompt. Some direction is simply a support, which does not need to be indicated, tracked, or part of the score. Examples of supports:

•Redirecting the student to stay on task is not a prompt; it is a support. •Reading directions with the student is a support.•Restating the directions for the student is a support.•Scribing for the student is a support.•Providing accessible materials (large print, textured paper, manipulatives, etc.) is a support.•Supports are not included when calculating the independence score for the student’s work.

120

Supports vs. PromptsSupports vs. Prompts


Examples of Supports for Examples of Supports for MathematicsMathematicsUse of manipulatives or picturesUse of a calculator Task-analyzed skillsNumber lineStamps to write numbersWord problems read to studentEnlarged textChunked problems


Examples of Supports for Examples of Supports for ReadingReading

Student reads by Following along with the textFollowing along with picture symbolsFollowing along with objectsFollowing along with tactile cues

Adapted textShortened textChunked informationLarge fontHighlight to emphasize important

information


Examples of Supports for Examples of Supports for WritingWritingStampsGraphic organizersWrite using pictures or objectsWrite by completing cloze sentencesScribing (work must use the student’s

communication mode)Speech-to-text writing programsLarge writing utensils or pencil gripsRaised lined paper


Examples of General Examples of General SupportsSupports

Reading initial task directions to the student

Redirecting the student to stay on task◦ “you need to finish math now”

If a student requires supports, he/she must be given supports for both the first and second piece of evidence.


125

Independence DimensionIndependence Dimension


• A prompt level must be documented next to each item performed by the student.• I = independent• V= verbal• G= gestural• M= model• P= physical• If some other system is used there must be a key.

• Remember, if your prompt hierarchy is not the same as the one above, you must explain your prompt hierarchy clearly, giving the names of the prompts, hierarchical order, and letter key for the prompts.

• Even if the student performs with 100% independence, each item must be marked with an I.

• Independence scores must be summarized as a percent.

• Reviewers must be able to recreate the score. If the score is not replicable, the dimension can not be scored and will receive zero for Independence. 126

Teacher Scoring for Teacher Scoring for IndependenceIndependence


• An indirect verbal prompt (V) can • provide the student with a clue to try and spark the

student’s recollection of the activity or lesson so that he/she can respond to the question (e.g., “Who was the story about?” or “Remember, the main character has red hair and pigtails. Point to the main character.”)

• An indirect gestural prompt (G) can• Provide the student with a clue as to the general

location of an answer (e.g., when looking up a word in the dictionary, the teacher may tap the page on which the word can be found but not exactly where the word is on the page)

127

Prompting During Prompting During Assessment- Allowable Assessment- Allowable PromptsPrompts


• An indirect model prompt (M) can • provide the student with a clue through

demonstration of the skill (e.g., demonstrate how to carry in an addition problem then give the student a different problem)

• Provide the student with a clue through acting out a scenario (e.g., when presenting a choice of three pictures and asking the student which picture represents an unbalanced force, the teacher may make a sweeping or moving motion to represent an “unbalanced force”)

128

Prompting During Prompting During Assessment - Allowable Assessment - Allowable PromptsPrompts


• A direct verbal prompt• provides the student with the specific answer to a question or

item (e.g., “Remember, the main character was Pippi. Point to the picture of the main character.”)

• A direct gestural prompt• points out the specific answer to the student (e.g., when

presenting a choice of three pictures and asking the student which picture represents an unbalanced force, the teacher points to or taps the correct picture)

• A direct model prompt• models the exact problem and answer that the student must

perform (e.g., when sorting producers and consumers, the teacher says, “Remember, corn is a producer,” picks up the picture of corn, and places it in the producer column of a chart then asks, “Which one is a producer?”)

129

Direct Prompting During Direct Prompting During Assessment - Assessment - NOT NOT ALLOWEDALLOWED


• A physical prompt (P) • requires physical contact between the

student and teacher• may be a touch at the elbow prompting the

student to move his/her hand to a particular answer, or it may be more invasive, involving the teacher physically moving the student’s hand to the correct answer

• If the student must be given a physical prompt to correctly answer the test question, the answer must be marked wrong.• This applies to full and partial physical

prompts. 130

Prompting During Prompting During Assessment - Allowable Assessment - Allowable PromptsPrompts


Scoring Evidence for Scoring Evidence for IndependenceIndependence

• The total Independence score must be in the form of a percent.• Take the number of items completed independently

divided by the total number of items multiplied by 100 to calculate the total Independence score.

• When scoring for Independence, the information does not include accuracy.

131

Independence# items performed independently divided by # of total items times 100

equals % Correct

____/_____ x 100 = ______%


LANGUAGE ARTS LANGUAGE ARTS LITERACYLITERACY

Sample EntryFrom 2011-2012

Albert

132


APA 2011-2012

ENTRY COVER SHEET Student Name__Albert__________ Evidence of Initial Activity – Dated Between September 1, 2011 - November 11, 2011 Evidence of Final Activity – Dated Between December 12, 2011 - February 17, 2012 This entry is for: (Circle one) LAL MATH SCIENCE This is entry number: (circle one) 1 2 3 4 ___3.1___ _4______ __F______ _2_______ Standard Grade Strand CPI CPI Link: (record the specific bulleted CPI Link to be assessed) After reading a passage, identify the meaning of unknown words using context clues____________ CPI Link type: (Circle one) Matched Near Far Description of Initial Activity: Albert will read the passages by following along with the picture cues and text. He will then identify the meaning of the underlined words using context clues. _ Description of Final Activity: Albert will read the passages by following along with the picture cues and text. He will then use the word bank to identify the appropriate definition of the underlined word, using context cues to find the definition. _

In addition to the code above, you may want to include the written information: Standard: 3.1 Reading Strand: F Vocabulary and Concept Development CPI: 3.1.4F2 Infer specific word meanings in the context of reading passages

Language Arts Literacy Entry - ExampleLanguage Arts Literacy Entry - Example

133


134

9-15-11


2a

135


2b

136


137

12-15-11


3a

138


3b

139


Language Arts Literacy Evidence Index – Sample Entry for Albert 2011-2012

4th Grade LAL Entry

Entry 1Standard 3.1 Strand F CPI 2Matched Link : “After reading a passage, identify the meaning of unknown words using context cues”

Rubric Dimension

Score Evidence

Complexity 4 Pgs. 2-2b and 3-3b show the student working on the complete CPI Link from the “Matched” column

Pgs. 3-3b (evidence 2) are a different activity than pgs. 2-2b (evidence 1) Different context: different passages and

words assessed Different application: multiple choice vs. fill

in the blank

Performance 3 Pgs. 2-2b student scored 20% (which is below 39%) on initial activity Pgs. 3-3b student scored 80% Each item is clearly marked Pgs. 3-3b do not include a more intrusive prompt than pgs. 2-2b

Independence 3 Pgs. 3-3b student scored 80% Independence Each item is clearly marked

140


Professional and Ethical Professional and Ethical ResponsibilityResponsibility

141

The Alternate Proficiency Assessment is intended to assess the academic progress of students with significant cognitive disabilities toward the standards established by the state of New Jersey.

The format of the APA requires teachers administering this assessment to develop assessment activities appropriate for each participating student and to provide appropriate documentation of how the activities were used to assess students on what they gained from instruction.

Instruction is the key. Assessments should never be used as the instruction. Instruction should take place first, and then assessments should be designed to reflect what was taught. The first piece of APA evidence is based on the initial

introduction of the skill, which usually occurs prior to implementing extensive instruction.


142


Teachers are to follow the steps for the development of an entry described in the teacher training materials.

Teachers are to save not only copies of the assessments that are being included in the student’s portfolio but also copies of student work that, while not included in the portfolio, provide further proof that the student received instruction and was given opportunities to practice and show what he/she was taught prior to the final assessment activity.◦ These additional student work activities may be

reviewed during a site visit or required to be presented for a security breach visit.


143

Professional and Ethical Professional and Ethical ResponsibilityResponsibility Each assessment presented to the student should be

unique, meaning that the student should not have previously seen the assessment.

The following practices are not allowed:Teaching students to memorize answers for an assessment.Giving students the same assessment (activity) over and over

until they complete the assessment satisfactorily.Allowing students to see the assessment and its test questions

ahead of time.Allowing someone else to complete work for a student.

Remember, there are guidelines for scribing that must be followed if a student needs assistance documenting their answer.

Giving an assessment when no instruction has occurred.Giving the students a writing task response and having them

copy it and submitting it as their own work.Leaving the date blank on a completed activity and adding a

date later.Changing the date on a completed activity.Any other practice that results in misleading information

regarding what the student was taught and has learned.


144


Some students may not be able to complete an entire assessment activity all at once. It is acceptable to administer assessment items one or two at a time, as appropriate for the student, but when doing so, the student should only be presented with the item or items that he/she will be expected to complete at that time.It is acceptable to mask the remaining items on the

page until they are completed by the student. If a student becomes sick during an assessment or

cannot complete an assessment at the time it is given, that assessment should be used as practice and a new assessment should be given at a better time.

Remember that the APA has two rather long testing windows that should help address most special testing conditions that may arise for students participating in the APA.



It is the professional and ethical responsibility of all contributors to a student’s portfolio to ensure that any and all documentation reflect authentic, accurate, and truthful information.◦ Reminder, it is not appropriate to manipulate the

date the evidence was created, the prompt/independence levels, the student’s work, or the scoring percentages.

◦ Include only evidence that reflects the student’s actual performance, even if the student did not make as much progress as you expected.

◦ Follow all APA guidelines when administering the assessment.

145


Any student portfolio that is found to contain inauthentic documentation may result in professional consequences for school and district staff.◦ All contributors to a portfolio must sign and submit the

assessment security form verifying that the APA was conducted properly and in accordance with all directions and training materials. District administrators will provide this form. This is a new policy, and is in line with the general assessment procedures.

It is important that administrators review the teacher training modules to ensure you know what is acceptable, and not acceptable, regarding APA assessment. Remember that this is an individual assessment, not a group assessment.

146



Test Security AgreementTest Security AgreementGive the teachers the agreement

this fall so that they can see what they will sign and submit when they turn in the portfolio.◦Every teacher that produces APA

evidence must submit a signed Test Security Agreement to be placed in the binder.

◦If teachers worked on only one subject, they may write that on the agreement.

1472011-2012 NJ APA Administrators Training

Pearson Scoring Center staff scored the portfolios during April and May.

Score reports were shipped in June.Most portfolios have been returned.Portfolios designated as security

breaches (Void 5) are in process. Information regarding the breach consequences will be forthcoming.◦ Please contact me to arrange a meeting to

discuss the district’s security breach portfolios.

Status of 2011 - 2012 Portfolios Status of 2011 - 2012 Portfolios and Score Reportsand Score Reports


Contact InformationContact Information

For information on interpreting the 2011-2012 APA score reports, please review the Score Interpretation Manual on Pearson’s website http://pem.ncspearson.com/nj/apa

For missing score reports please contact:NJ Customer Service Team – 888-705-9416

For score report interpretation please contact:Jo-Lin Liang – 609-633-6884


What is Proficient?What is Proficient?New cut scores were set in June 2009 which

determine how the total score for each subject area is classified. Total scores may be classified as advanced proficient, proficient, or partially proficient.◦ The cut scores were determined based on the grade-

level linked skills and content being assessed; therefore, the same number of total points may be classified as advanced proficient in one subject area for a grade level, and classified as proficient for a different grade level, due to differences in the CPI Links and developmental demands of students in different grades.

◦ This mirrors the general assessments where each grade level and subject area cut scores are determined independently.


How is a Total Score Calculated?How is a Total Score Calculated?The scoring formula was determined by reflecting

on the purpose of the APA, which is to:◦ assess the district and school grade-level instructional

program provided to the student, ◦ assess a student’s attainment of knowledge and skills of

the CCCS, and ◦ encourage high but appropriate instructional standards

for all students, while providing necessary supports to access the curriculum and foster the highest level of independence possible for the student.

The scoring formula uses the three dimension scores that each entry receives:◦ Complexity◦ Independence◦ Performance


How is a Total Score How is a Total Score Calculated?Calculated?The total score for each content area

is calculated by adding the scores for all three dimensions for each of the four entries.

Each entry has a minimum possible score of zero and a maximum possible score of 16 points.

The total score for each subject area has a minimum score of 0 and a maximum score of 64 points.


How is a Total Score How is a Total Score Calculated?Calculated?

Each scorable entry will receive a score of 0 – 4 for each dimension.◦ Unscorable entries receive a zero for each of the three

dimensions.Each entry is scored by a minimum of two people

(readers).◦ First person is reader one (R1) and second person is reader

two (R2).The performance dimension score for any entry is

the sum of the two reader scores (R1+R2).◦ The addition of the two reader scores gives more weight to

this dimension in the overall total score than the other two dimensions.

The complexity dimension score for any entry is the average of R1 & R2.

The independence dimension score for any entry is the average of R1 & R2. 1532012-2013 NJ APA Administrator Training

AccountabilityAccountabilityAPA test results will be combined with

the results from the general assessment for accountability purposes for state and federal reports.

For accountability purposes, the APA is both a student assessment, and a school/district program assessment.

APA test results are included in the Final Accountability report.


Federal Results – 1% CapFederal Results – 1% CapThe NCLB 1% cap limits the number of APA

proficient and advanced proficient results that can be used in the accountability calculation. It does not limit the number of students who may participate in the APA.

Districts legitimately testing more than 1% of the total grade population with the APA should apply for a waiver from the Office of Special Education. The waiver, if approved, allows proficient and advanced proficient results greater than 1% to be used in the accountability calculations. ◦ This form is usually mailed by the department to

districts in the winter.


Accountability ReportsAccountability Reports

Student data is collected on the APA scan sheet. This data is submitted to districts for verification.

All requested revisions to student data must be made during the record change process. No changes may be submitted after this time frame.

Accountability reports are produced based on the student data on file as of the close of the record change process.

Reports for sending districts include students assessed out of district, out of state, and in district.


Information – Title I OfficeInformation – Title I OfficeUp-to-date information on the

calculation of Accountability Reports can be obtained on the Department of Education’s website www.state.nj.us/education◦ Review the information provided from

the Office of Title I. The Accountability Workbook provides specific details.

If you have questions after reviewing the online information, please call the Office of Title I at 609-943-4283.


Main ResponsibilitiesMain Responsibilities Complete scan sheet materials in February

2013.◦ Work directly with administrators from the child’s

sending or receiving school to ensure accurate information. Errors on the scan sheet may affect your Accountability Report calculations.

◦ Several fields on the scan sheet must be completed in February since they are time sensitive and can not be completed during the Pre-ID process.

◦ A few fields on the scan sheet may need to be completed by the student’s teacher.

Collect portfolios from teachers by end of day on February 15, 2013.◦ Administrators may choose to collect them earlier

in order to complete portfolio review process when high number of portfolios are produced.

Review final student data and submit record change rosters. This is the last opportunity to make data changes.


Final Administrator ReviewFinal Administrator Review Test Design Review

◦ Did the evidence address the required components? Prescribed standards and grade-level CPIs Correct amount of evidence Meets the Universal Scoring Rules

◦ Conducting this review periodically will allow you to find and correct errors during the collection period.

Security Breach Review◦ Is white out being used to change information?◦ Were prompt levels changed to independent

performance?◦ Were dates on the evidence changed?◦ Were wrong answers changed to correct answers?◦ Were wrong answers marked correct?◦ Are the percentage scores accurate?◦ Are there other indications of inauthentic information?


Final Administrator ReviewFinal Administrator ReviewReview portfolios for accuracy and

authenticity by February 22, 2013.◦ Review the portfolios in light of the student’s

assigned grade level.◦ Review the portfolios in light of the current CPI Links.◦ Review the portfolios in light of the test design

requirements.◦ Review the portfolio to ensure all evidence is

authentic.Reminder that portfolios should be reviewed

multiple times within each collection period.◦ Doing so helps you identify errors early, giving

enough time to produce new evidence without errors prior to the end of the collection period.


Shipment of PortfoliosShipment of Portfolios

The shipment process will be communicated to districts at a later time. Portfolios must be shipped between February 26 and March 11, 2013. ◦ Any portfolio shipment dated after March 11

will not be accepted for scoring.

◦ Portfolios must be submitted on time in order to be processed.

◦ Shipping materials and instructions will arrive in the Chief School Administrator’s office in February 2013.


Administrator/Coordinator Administrator/Coordinator ActivitiesActivities

General Responsibilities

When information is sent to you, pass it on to district and school administrators and teachers. Review the time lines and request APA information from your chief school administrator when it is scheduled to be delivered.

Review participation guidelines carefully.

Ensure that all students who need to be assessed are identified (both in-district and out-of-district).


General Responsibilities (continued)

Review school educational programs and ensure alignment with dimensions of the scoring rubric.

Ensure that the APA teachers participate in the online teacher training sessions.

Review the APA Procedures Manual.

Pass along questions and concerns to NJ DOE or contractor.

163

Administrator/Coordinator Administrator/Coordinator ActivitiesActivities


Administrator/Coordinator Administrator/Coordinator ChecklistChecklist

Before Assessment

Identify a School Coordinator for each school. Register number of students participating in

APA and update mailing addresses using Material Survey no later than Date to Be Determined.

Create a teacher training schedule for your school, identify teachers required to participate, and implement training.◦ Ensure all materials are downloaded prior to the

selected training day(s).◦ Have a sign in sheet for all teachers to verify

attendance and completion of the training.◦ Utilize all APA training materials, including the Content

Resource Guide and Five Item documents, in addition to the Teacher Training Modules and Procedures Manual.

Distribute materials to schools/teachers.


During Assessment

Distribute materials to schools/teachers. Binders and manuals will be shipped to district/receiving school.

Copies of the APA training materials may be downloaded as of September 7, 2012, from the Pearson’s website http://pem.ncspearson.com/nj/apa. Click on Documentation, then Procedures Manual.◦ Content Resource Guide and Five Item Sample

documents may not be available until September 19, 2012.

165



During Assessment (continued)

Gather information on students’ sending/receiving status and collect student demographic info.◦ Work with the receiving/sending school to

confirm student data.

Check periodically that teachers are working on the APA.

166



After Assessment

Notify teachers of timeframe for returning the completed APA portfolios.

Collect and review all APA portfolios for accuracy and authenticity of information and inclusion of all required components.◦ Be sure to review the portfolios for adherence

to the universal scoring rules. Review the scan sheet information to ensure

accuracy of all data.

167



After Assessment (continued)

Package the APA materials securely and label them according to instructions.

Contact the designated carrier for return of the portfolios to arrange pickup of the materials.

Ensure all portfolios are submitted by March 11, 2013 deadline.◦ Portfolios not received by shipping carrier by March

11, 2013 will not be accepted for scoring.

168



APA Teacher InvolvementAPA Teacher InvolvementTeachers are needed in March 2013 for a week

to score portfolios. Send an email to Elizabeth Celentano if someone is interested in assisting in the scoring process. Email address is [email protected]

Summer committees will be formed to assist in the test specification revision for alignment to the general assessments and Common Core Standards. ◦ Teachers and curriculum specialists will be needed to

complete this work.◦ More information will be forthcoming in Spring 2013.


Dynamic Learning MapsDynamic Learning MapsNew Jersey is one of 13 states collaborating

with the University of Kansas on the future version of an alternate assessment for students with the most significant cognitive disabilities.

Opportunities arise for districts to participate in the process via document reviews, meetings, and implementing pilot components. If you would like to become involved please email Elizabeth Celentano.

This revised assessment is expected to be implemented in some form (pilot?, operational?) in 2014-2015. As information becomes available it will be disseminated.


Where To Get More Where To Get More Information?Information?Educational Links, Information, and Resources on Assistive Technology and Universal Design for Learning, an excellent reference document for teachers and administrators can be downloaded from the DOE website http://www.state.nj.us/education/specialed/tech/


Where To Get More Where To Get More Information?Information?Other educational materials and resources can be accessed through the New Jersey Learning Resource Center Network website: http://www.nj.gov/education/lrc


Where To Get More Information?Where To Get More Information?Elizabeth Celentano (test design, training, test administration, etc.)Office of AssessmentsNew Jersey Department of [email protected]

Jo-Lin Liang (student data, Pre-ID, record changes, score interpretation, etc.)

Office of AssessmentsNew Jersey Department of [email protected]

Office of Special Education (general special education questions)New Jersey Department of Education609-292-0147http://www.nj.gov/education/specialed/

Department of Education website:www.state.nj.us/education


Where to Get More Where to Get More Information?Information?

Topics: Training materials, score reports 2011-2012, portfolio binder return

Pearson - Bruce [email protected]

Customer [email protected]

Phone:888-705-9416

Pearson website:http://pem.ncspearson.com/nj/apa

Topics: Explanation and Error Sheet issues, Evidence questions (through November 2012)

ILSSA – Lou Ann Land [email protected] Cole [email protected]


Where to Get More Where to Get More Information?Information?Topics: Student registration, Pre-ID, scan sheets,

portfolio shipments, record changes, online entry cover sheets, etc.

Questar Assessment, Inc.NJ APA Customer Support:[email protected] Broderick [email protected]: 866-690-4733 NJ APA Website address:https://nj-servicepoint.questarai.com/Login.aspx


Winter 2013 Training Winter 2013 Training MaterialsMaterials

In January 2013 a new Power Point presentation will be uploaded for your review. You will be notified when it is posted and on which website.

It will include reminders about key concepts provided today as well as information about necessary tasks related to returning the portfolios.

Please make sure you read these materials and contact the Office of Assessments if you have questions.


Thank YouThank You

Thank you for taking time to attend this session. If you have not had all of your

questions answered, please see one of the presenters for more information.


overview of the 2012-2013 alternate proficiency assessment administrator/test coordinator training...

Documents

apa assessment

administrator apa review

apa decisions

apa page

apa administrators

apa binder

apa revisions pages

apa participation guidelines