national accessible reading assessment projects defining reading proficiency for accessible large...

National Accessible Reading Assessment Projects

Defining Reading Proficiency for Accessible Large Scale Assessments

Discussion of the Principles and Issues Paper

General Advisory Committee Meeting

October 17, 2005

Deborah Dillon


Purpose

Our task is to design accessible reading assessments. Precise definitions of the constructs being measured is key. Step one involves developing a definition of “reading proficiency.” To that end, this paper presents a synthesis of information and describes (a) a set of principles that will guide the research and development phases of our projects, and (b) unresolved issues that need to be addressed related to each principle.


Sources for the Development of the Four Principles and Related Issues

• A review of existing definitions of reading proficiency

• A panel of experts to provide input

• Focus groups


Overview of Principles

• Principle 1: Students with disabilities deserve the opportunity to show their proficiency as readers – to show what they know and are able to do as readers, and this should be reflected in the way reading proficiency is defined.

• Principle 2: Reading proficiency must be defined by grade level.


Principles—cont.• Principle 3: Definitions of reading proficiency

must include both comprehension and foundational skills.

• Principle 4: Reading proficiency must be defined in such a way that flexible expressions of reading are allowed while preserving the essential nature of reading. This is essential as we seek to make assessments accessible to students with a variety of disabilities.


Principle 1: Students with disabilities deserve the opportunity to show their proficiency as readers – to show what they know and are able to do as readers, and this should be reflected in the way reading proficiency is defined.

• Undergirds the other principles re: defining reading proficiency for accessible large-scale assessments; no issues follow this principle

• Promotes a definition of reading proficiency that requires assessments to reflect what students are able to do, not just what students are not able to do

• Suggests the need for large-scale assessments that do not rely solely on one means of entry to demonstrating proficiency


Principle 2: Reading proficiency must be defined by grade level.

• Access to, participation in, and progress in the general curriculum is a foundational requirement

• States address areas of reading proficiency and component skills through grade-level definitions of content standards

• The 2009 NAEP assessment, which is required to be used by all states, focuses on grade-level “cognitive targets” and defines these as “the mental processes or kinds of thinking that underlie reading comprehension”


Issues under Principle 2

• How do the important component reading skills vary as a function of grade level?

• How is a determination made about which measure is best to use at a specific grade level?

• At what point are grade-level expectations inappropriate for individual students or groups of students? How are changes made to achievement levels while retaining grade-level content?

• How are differences in reading achievement level (e.g., modified or alternate achievement levels) developed and defined? How are these varying achievement levels reflected in definitions of reading proficiency?


Principle 3: Definitions of reading proficiency must include both comprehension and foundational skills.

• The ultimate goal of reading for students with and without disabilities is comprehension; foundational skills can be measured indirectly for many students

• If students with disabilities are not proficient on comprehension measures, we need to assess foundational skills to understand what students can do and skills that may impact performance

• The overall goal is to develop flexible assessments to assess reading comprehension and/or foundational skills based on student performance (perhaps technology based ways)


Issues under Principle 3

• To what degree can component skills be measured independently?• Comprehension is the primary goal for readers. If students are proficient

in this area with accommodations, do we need to measure the foundational skills?

• If comprehension is our primary goal, should the comprehension score be weighted more heavily when foundational skills are also assessed?

• What feasible techniques are available for measuring foundational skills such as fluency or phonics knowledge in the context of large-scale assessment?

• For students who do not achieve grade-level proficiency, what processes can be developed or applied to aggregate their performance on component skills into an overall measure of reading proficiency?

• Some components appear to be problematic for certain disabilities (e.g. phonemes and deaf students). Do we develop alternate definitions based on a better understanding of reading for those populations?

• Are some skills less critical to measure than others?


Issues under Principle 3—cont.

• If foundational skills were only assessed after a student had performed below proficient on comprehension, what proportion of ALL students (with and without disabilities) would be assessed on each of the foundational skills? Is this proportion small enough to assess students in small groups, individually, or via computer?

• Can some foundational skills be assessed together (e.g., decoding and phonemes)?

• If fundamental skills are going to be measured only in students that are not proficient on as assessment of comprehension, can accommodations be allowed that invalidate the foundational skills (e.g., read aloud for decoding or extra time for fluency).

• If a student is not proficient on a measure of reading comprehension, should listening comprehension be assessed prior to measuring foundational skills?


Principle 4: Reading proficiency must be defined in such a way that flexible expressions of reading are allowed while preserving the essential nature of reading. This is essential as we seek to make assessments

accessible to students with a variety of disabilities.

• Proficient readers, including students with disabilities, may rely on any set of component proficiencies to read and may compensate for some skills they lack by drawing on others—depending on their disability (e.g., a student with congenital deafness may comprehend what he/she reads, but need to deploy alternate strategies to understand sound-symbol relationships)

• Public views of what constitutes reading are reflected in various ways; these are indicated in how states allow/do not allow different approaches to assessments in reading and if assessments are allowed without restrictions


Principle 4: Reading proficiency must be defined in such a way that flexible expressions of reading are allowed while preserving the essential nature of reading. This is essential as we seek to make assessments

accessible to students with a variety of disabilities.

• Federal statutes allow a range of options in types of assessments used and the achievement standards applied to students with disabilities. Some students can take assessments based on modified achievement standards or alternate assessments based on grade level standards or on alternate achievement standards. Alternate achievement standards must be aligned to grade-level content standards but may differ from grade level achievement standards in breadth, depth, or complexity.


Issues Under Principle 4

• How can we determine what constitutes “reading” that is still based on grade-level achievement standards?

• Does what constitutes “reading” differ by disability category or by needed accommodation?

• When do the concepts of modified and alternate achievement standards apply to reading?

• How can we determine the ways that students with disabilities compensate for weaknesses in specific proficiency components due to their disability or multiple disabilities?


Summary Comments and Reactions

• Suggestions for other Principles?

• Suggestions for Unresolved Issues?

• Other comments or suggestions?


Plans

Principles & Issues Paper will be posted on the NARAP Web site

national accessible reading assessment projects defining reading proficiency for accessible large...

Documents