The Effect of Public Policy on Alternate

Assessments

Sue Rigney

Alternate Assessment ConferenceUniversity of Maryland

College Park, MD October 2007

Historical

1990-2000: Many SWD routinely excluded from state & national assessments

Exemption of a special education student requires:

a) “the student has been found eligible for special educations services through an IEP; and

b) receives Special Educations services prior to the first day of testing; and

c) receives 49% or less of his/her reading/English instruction per week through general education instruction.”

Source: MEAP Assessment Administration Manual, 1991

Alternate Assessment

Key Federal Statutes

• IASA 1994 Standards and assessments by 2000-01 All SWD to be included in assessments

• IDEA 1997 Access to general curriculum Alternate assessment in place July 2000

• NCLB 2001 SWD included in assessments & accountability

for all public schools

• IDEA 2007 Follows NCLB

Alternate Assessment

NCLB + Regulations

• 1% AA-AAS December 2003• Permits alternate achievement standard

for students with most significant cognitive disability

• 2% AA-MAS April 2007• Permits modified academic achievement

standard for students whose disability prevents them from meeting grade level standard in period covered by current IEP

Alternate Assessment

Examining Policy Effects

• Intent• Implementation• Impact on State practice

Alternate Assessment

Intent

• Is always good• Realized through implementation

Diverse actions, actors Slow, must be sustained

• Consequences may be unexpected Perception vs reality Perception is reality

Alternate Assessment

Intent - IDEA 04 & IASA

Paradigm ShiftsParadigm ShiftsIDEA 04• Access to general curriculum for SWD

IASA 97 for Title I Schools• All students included in State assessments • Scores of SW must be publicly reported for

school and district accountability• State must explain how scores from alternate

assessment are integrated into accountability system

Alternate Assessment

Intent - NCLB

“To ensure that all children have a fair, equal, and significant opportunity to obtain a high quality education…” All schools publicly accountable for

performance of SWD

Alternate achievement standard permitted only for students with most significant cognitive disability

1% cap as safeguard for students

Alternate Assessment

Implementation

• Statute clarified by guidance

• Occurs in the field - monitoring must examine evidence of compliance

• Compliance alone may not ensure that policy goals are reached

• Successful implementation requires State as well as federal action

Alternate Assessment

Federal Policy Implementation

• Statute, regulations & guidance drafted and disseminated

• Compliance monitoring carried out by multiple offices e.g.,OSEP, OESE, SASA

• Peer review of Title I State Plan required

• Technical assistance

• $$

Alternate Assessment

State Policy Implementation

• Inclusion policies and procedures

• Optional development & implementation of AA-AAS or AA-MAS consistent with statute

• Support for test administration and use

• Infrastructure for local implementation

Assessment training

Professional development to support effective instruction

Alternate Assessment

Implementation - IASA

• Compliance monitoring• Assessment system peer review

Focus on test administered in 2000-01 Continued under NCLB for States not

approved

Alternate Assessment

IASA Peer Review – AA Must

“When assessment procedures are altered, it is critical to ensure that scores, decisions, and judgments based on those assessments are fair, reliable, and valid. The criteria for technical quality outlined in…. “Professional Standards of Technical Quality,” apply to modified, accommodated, and alternate assessments.

IASA Peer Review Guidance, p. 15

Implementation - NCLB

• Accountability workbooks• Title I monitoring• OSEP monitoring• Peer review of State assessment

systems

Alternate Assessment

“…the NCLB standards and assessment peer review process increased the requirements for documenting the technical quality of all assessments, but the biggest shift was for AA-AAS. The type of technical documentation necessary to fulfill the peer review requirements has never been expected from AA-AAS developers previously.”

Marion & Pellegrino

NCLB Peer Review: AA-AAS Must

Yield results separately in reading and math Clear guidelines for student participation provided to all LEAs Designed and implemented in a manner that supports use of results

for AYP• Aligned with state content standards

• Assessment design - appropriate for school accountability measure (e.g., results comparable across schools and districts)

State provides evidence of technical quality,• Validity, reliability accessibility, objectivity, and consistency with nationally recognized professional and technical standards

• Description of the standard-setting process, the judges and their qualifications, and state adoption of alternate achievement standards

Reports results to teachers and parents in a manner consistent with the alternate achievement standards

Alternate Assessment

Impact-IASA

On January 19, 2001

Alternate Assessment

Decision # States

Full Approval 11 DE, IN, KS, LA, MD, MA, PA, RI, VT, VA, WY

Conditional Approval (Complete by Spring 2001)

6 KY, MO, NC, OR, TX, WA

Timeline waiver 14 CO, CT, GA, HI, ME, MS, NE, NV, NH, NY, ND, OH, SC, SD

Compliance Agreement 3 CA, WV, WI

Still under review 18 AL, AK, AZ, AR DC, FL, ID, IL, IA, MI, MN, MT, NJ, NM, OK, PR, TN, UT,

Impact-IASA

Issues Facing States on January 19, 2001 Requirement #

Inclusion of limited English proficient students

22

Inclusion of students with disabilities

14

Disaggregated Reporting 30

Finish Standards-based System 11

Impact NCLB

States Revising/Developing Alternate Assessment in 2005

Alternate Assessment

Area # States

Approach 8

Content 10

Standard-setting

13

Scoring Criteria 17Source: 2005 State Special Education Outcomes, NCEO

Current Status

As of 8/6/0731 States = Approved + Approval Expected

12-16 States working on AA-AASMajor concerns:

alignment with grade level content documenting technical quality

Completing the AA-AAS

2005-06 DEADLINE EXTENDED2005-06 DEADLINE EXTENDEDApproval Pending (does not meet all of the requirements)

If only significant issues with an alternate assessment based on alternate achievement standards or an assessment for limited English proficient students… Condition on its fiscal year 2007 Title I, Part A grant

award Mandatory Oversight, pursuant to 34 C.F.R. §80.12. Agreement with the Department

demonstrating a commitment and investment of resources to resolve all outstanding issues for the 2007–08 administration of its assessments.

a mutually acceptable timeline for how and when the remaining work toward having a fully approved standards and assessment system will be accomplished.

Review of AA-MAS

Standards and Assessment Peer Review Guidance: Information and examples for meeting requirements of the No Child Left Behind Act of 2001

•Revised to include AA-MAS requirements

•Distribution to States TBA

•Peer reviewer training Jan 2008

Impact on Assessment Practice

Virtually all State assessment participation policies changed since IASA

Participation of SWD in State assessments is substantially increased

22/50 states have changed participation policies/guidelines for AA-AAS since the Dec 9, 2003 regulation

Peer Review has prompted linkage to academic content for all states

Alternate Assessment

Impact on Assessment Practice

State examples of rigorous practice emerging, e.g. Alabama standard setting report

New methodology emerging: e.g. Links for Learning, NAAC Learner Characteristics Inventory

Articles in professional journals focus on AA-AAS

Questions about validity of AA-AAS challenges some assumptions about general test

Impact on Instruction

• Anecdotal and case studies• Most pre-date requirement for academic

content

• Inclusion in accountability makes a difference:“I think our expectations are higher.”

Impact on Student Outcomes

• Evidence of student outcomes limited

Reports do not separate general test results and alternate results

OSEP collects detailed data in biennial report but it’s hard to find

Alternate Assessment

MSA Snapshot (State)

With trend data

ALT-MSA Snapshot (State)

With trend data

http://www.mdreportcard.org/Assessments.aspx?WDATA=State&K=99AAAA#ALTsnapshot

Impact on Student Outcomes

• Evidence of student outcomes limited Reports do not separate general test results

and alternate results

OSEP collects detailed data in biennial report

• Evidence of student outcomes difficult to interpret Many state alternates redesigned in last 3

years, so trend data is not interpretable

Test results confounded with OTL

Alternate Assessment

Lessons Learned?

• Collaboration needed to develop alternate assessments: assessment, special ed, content experts

• Resources needed to build local support systems

• Consequences must be documented

• Interpretation of outcomes difficult because student results confounded with opportunity to learn

Alternate Assessment