accommodation: lang, elliot, bolt, and kratochwill (2008) stated that testing accommodations are...

What is an accommodation?

Accommodation: Lang, Elliot, Bolt, and Kratochwill (2008) stated that testing

accommodations are changes made to the administration of tests to

provide students with disabilities the opportunity to demonstrate their

knowledge and understanding of the constructs measured by the

tests without the interference of their disability.

Elbaum (2007) stated there is a general consensus that to be

considered a valid accommodation, a modification in test

administration should remove disability-related variance. For

example, allowing students with motor difficulties to dictate their

solutions to mathematics problems to a scribe addresses the

students’ specific disability without affecting their mathematics skills.

Most commonly allowed accommodations

Thurlow and Bolt (2001) found the following accommodation to be the most often allowed in state policy Individual administration Dictated response Small group administration Large print Braille Extended time Interpreter for instructions Read/reread/simplify/clarify directions Computer machine response Read aloud Writing in test booklets Testing with breaks

Accommodations allowed in Wisconsin

General considerations for the use of accommodations on state and district assessments:

Accommodations for a student with a disability must be documented on a current IEP or 504 plan.

Accommodations should be consistent with day-to-day instructional methods.

Accommodations should not be first introduced during testing; students should be comfortable using accommodations.

Accommodations should enhance access without changing the skill or construct measured.

Districts should monitor appropriate use of accommodations by comparing actual assessment accommodations received with those stated in the student’s IEP or 504 plan.

Found on Wisconsin Dept of Instruction website http://dpi.wi.gov/oea/pdf/accom09.pdf

http://dpi.wi.gov/oea/pdf/accom09.pdf


Test Directions Read directions aloud and reread

as needed Use an audio recording of

directions Use directions that have been

marked or highlighted by teacher or student

Simplify, explain, clarify, or translate language in directions

Have student reread and/or restate directions in his/her own words

Use sign language or oral interpreters for directions

Content Presentation Use visual magnification Use audio amplification Use a colored overlay Use page markers to maintain

place Allow students to mark in test

book Turn pages for student Provide Braille or large print

edition of test Provide extra test book for

student to view Sign questions and content to

student (not allowed on Reading test)

Student uses a text-talker converter (not allowed on Reading test)

Read questions and content to student (not allowed on Reading test)


Content Presentation Continued Read aloud the Reading test to

students with visual impairments Provide translator to translate

questions Provide bilingual word lists Simplify words not allowed on

Read and Language Arts test

Response Use of calculator or

multiplication table for students in grades 3 or 4 on Mathematics test

Use graph/lined/grid paper, template, or graphic organizer

For selected response items, student indicates responses to a scribe orally, by pointing, or by using a communication device.

For constructed response items, student indicates responses orally to a scribe

Student responds orally or in writing in his/her native language and a translator records/translates student responses into regular test book in English. (Not allowed on Writing test. )


Response Student uses sign language to

indicate responses to a scribe. (Not allowed on Writing test )

Student reads out loud to him/herself in an individual setting.

Student records responses using an audio or video device

Student uses computer or word processor for recording responses that are then transcribed into regular test book.

Provide spelling assistance or a spell-check device, where appropriate. (Not allowed on Language Arts or Writing tests. )

Setting Provide distraction-free space or

alternative location for student

Student takes test in an individualized and supervised setting

Student takes test with a small group or a different class

Homebound or hospitalized student takes test at home or in a care facility (e.g., hospital) with district supervision.

Student uses adaptive furniture. Student uses special lighting

and/or acoustics Allow student to move, stand, or

pace during individual administration


Time and Scheduling Breaks: allow student to take

breaks without exceeding total testing time

Extra time: provide extra time for any timed test, as long as a test session is completed within the same day the student started the session

Scheduling: allow student to test across multiple days, as long as a test session is completed within the same day the student started the session.

Other Any accommodation not on this

list must be submitted to DPI for approval, as it may represent a modification which changes the skill or construct being measured

Effectiveness of Accommodations

As of 2004, Bolt and Thurlow found that very

little research had been done to show whether

accommodations allow for appropriate

assessment

Cawthon, Ho, Patel, Potvin, and Trundt (2009)

noted that the research on the effects of

accommodated test scores continues to grow,

but offers few conclusive findings.

EFFECTS OF THE READ ALOUD

ACCOMMODATION ONSTUDENT SCORES AND PERCEPTIONS OF TEST

A Field Project ReportBy Steve Schoen

January 15th, 2011

Relevance

Reading is a prerequisite skill for demonstrating knowledge in many academic areas. Questions in science, social studies, math, and reading all require reading skills to correctly answer. (Thurlow and Bolt 2001).

Only after removing the obstacle of the reading can the students’ true abilities to answer the question be demonstrated and the possibility of error based solely on the reading aspect removed (Thurlow and Bolt 2001).

Previous Literature

Fuchs(2000), Rieck(2005) and Phillips(1994) noted that high-stakes testing is growing in use.

Ketterlin-Gellar (2007) found the read aloud accommodation does help low fluency readers on math assessments.

Elbaum (2007) found that the read aloud accommodation helped both student with and without disabilities. The latter twice as much.

Meloy(2002) found that the read aloud accommodation helped both students with and without disabilities resulting in an unfair advantage for those that receive it.

Data Collection

Population n =30 high school students ages 14-18 Subpopulations

Students with SLD n = 9 Students without SLD n =21

Random assignment to two test groups Two test forms

Math and reading comprehension Equal number of questions Similar difficulty

Procedure Students took an accommodated test and a non-

accommodated test. Students responded to a survey about testing and the

accommodation

Examination for form effect

The difference in mean scores was 0.30

A P-value of 0.301 demonstrates no significant difference in scores obtained on different forms.

A Cohen’s d of 0.05 shows virtually no effect.

Total Score Form A Total Score Form BMean 22.033 21.733Variance 44.516 33.995Observations 30 30Pearson Correlation 0.885

Hypothesized Mean Difference 0df 29t Stat 0.529

P(T<=t) one-tail 0.301t Critical one-tail 1.699P(T<=t) two-tail 0.601t Critical two-tail 2.045

Examination for form effect on reading subtest

The difference in mean score was 0.27

A P-value of 0.290 demonstrates no significant difference in scores obtained on the reading subtest by form.


Reading Score Form A Reading Score Form B

Mean 11.233 10.967

Variance 12.047 10.033

Observations 30 30

Pearson Correlation 0.694

Hypothesized Mean Difference 0

df 29

t Stat 0.559

P(T<=t) one-tail 0.290t Critical one-tail 1.699

P(T<=t) two-tail 0.580

t Critical two-tail 2.045

Examination for form effect on math subtest

A P-value of 0.464 demonstrates no significant difference in test scores on the math subtest by form.


The difference in mean scores 0.34

Math Score Form A Math Score Form B

Mean 10.733 10.767

Variance 14.754 12.530

Observations 30 30



df 29

t Stat -0.091




Comparison of total test scores between ability groups

STUDENTS WITH DISABILITIES STUDENTS WITHOUT DISABILITIES

Read Aloud Not Read Aloud

Mean 15.889 15.778

Variance 20.611 6.944

Observations 9 9



df 8

t Stat 0.135




Read Aloud Not Read Aloud

Mean 23.714 25.143

Variance 33.014 23.229

Observations 21 21



df 20

t Stat -2.118




• Students without SLD • P = 0.023 significant • Cohen’s d = 0.28 small effect

• Students with SLD• P = 0.448 not significant• Cohen’s d = 0.03 virtually no effect

Comparison of math test scores between ability groups


Math Read Aloud Math Not Read AloudMean 7.667 7.556

Variance 10.750 9.278

Observations 9 9



df 8

t Stat 0.141

P(T<=t) one-tail 0.446

t Critical one-tail 1.860



Math Read Aloud Math Not Read AloudMean 12.048 12.143Variance 10.548 8.329Observations 21 21Pearson Correlation 0.815Hypothesized Mean Difference 0df 20t Stat -0.230

P(T<=t) one-tail 0.410t Critical one-tail 1.725P(T<=t) two-tail 0.820


• Students without SLD• P = 0.410 not significant• Cohen’s d = 0.03 virtually no effect

• Students with SLD• P = 0.446 not significant• Cohen’s d = 0.04 virtually no effect

Comparison of reading test scores between ability groups


Reading Read Aloud Reading Not Read AloudMean 8.222 8.222Variance 7.444 2.944Observations 9 9Pearson Correlation 0.549Hypothesized Mean Difference 0df 8t Stat 0

P(T<=t) one-tail 0.5t Critical one-tail 1.860P(T<=t) two-tail 1


Reading Read Aloud Reading Not Read AloudMean 11.667 13Variance 9.833 6.5Observations 21 21Pearson Correlation 0.644Hypothesized Mean Difference 0df 20t Stat -2.488

P(T<=t) one-tail 0.011t Critical one-tail 1.725P(T<=t) two-tail 0.022t Critical two-tail 2.086

• Students without SLD• P = 0.011 significant• Cohen’s d = 0.48 approaching medium effect

• Students with SLD• P = 0.5 significant• Cohen’s d = 0.0 no effect

Differential Boost

Population/TotalNot Read Aloud

Mean ScoreRead Aloud Mean

ScoreDifference in Mean

Score Students w/o Disabilities

25.14 23.71 -1.43

Students with Disabilities

15.78 15.89 0.11

Population/Reading Test

Not Read Aloud Mean Score

Read Aloud Mean Score

Difference in Mean Score

Students w/o Disabilities

13.0 11.67 -1.33


8.20 8.22 0.02

Population/Math Test Not Read Aloud Mean Score

Read Aloud Mean Score

Difference in Mean Scores


12.14 12.05 -0.09


7.56 7.67 0.11

• There is a large difference in mean scores between ability groups on the over all test scores

• This large difference is not due to students with SLD substantially increasing their scores.

• There is no boost due to low increase by students with SLD and decrease of scores for students without SLD.

Differential Boost Continued

Population

Total Participants

Participants that showed increased score on Reading

Test

Participants that showed increased score Math Test


21 4 9


9 3 4

• An examination of individual students showed that few students increased their scores on either subtest.

• Students without SLD • Math: One student increased score by more than 2• Reading: Two students increased score by more than 1

• Students with SLD• Math: One student increased score by more than 2• Reading: Two students increased score by more than 2

• The final analysis is that there was no boost for either population even when considering individual students.

Differential Decay

Population

Total Participants

Participants that showed decreased score on

Reading Test

Participants that showed decreased score on Math Test


21 14 10


9 5 4

• The mean scores of the students without SLD decreased on both the reading and math tests when read aloud.• Reading -1.33• Math -0.09

• When comparing mean differences of the ability group there is a small difference in math and a large difference in reading.• Reading 1.35• Math 0.20

• Individual students without SLD displayed large negative score changes• Reading: Seven students decreased by score more than 1. Some by

as much as 3, 4 or 7.• Math: Five students decrease score by more than 1

• The final analysis shows a decay in reading scores for students with disabilities.

Results of Survey:How do you rate your test taking skills?

STUDENT WITHOUT DISABILITIES STUDENT WITH DISABILITIES

32%

58%

11%

1 2 3 4

67%

33%

1 2 3 4

1 = very poor, 2 = poor, 3 = strong, 4 = very strong

Were the tests you took during the study difficult?

STUDENTS WITHOUT DISABILITIES

STUDENTS WITH DISABILITIES

9%

77%

14%

1 2 3 4

22%

78%

1 2 3 4

1 = very difficult, 2 = difficult, 3 = easy, 4 = very easy

Do you think a test would be easier or more difficult if it was

read aloud?STUDENTS WITHOUT

DISABILITIESSTUDENTS WITH DISABILITIES

22%

61%

17%

1 2 3 4

11%

67%

22%

1 2 3 4

1 = much more difficult, 2 = more difficult, 3 = easier, 4 = much easier

How do you feel about a test being read aloud?



10%

10%

50%

30%

1 2 3 4

50%50%

1 2 3 4

1 = hated it, 2 = did not like it, 3 = liked it, 4 = loved it

Did having a test read aloud make you believe that you did better or worse on the test?



20%

60%

20%

1 2 3 4

88%

13%

1 2 3 4

1 = much worse, 2 = worse, 3 = better, 4 = much better

Limitations of Study

Small sample size 30 participants

Small number of participants with SLD, 9

Data was prone to be skewed by outlying scores

Not much demographic variation in the sample

Setting and sample size made generalization difficult

The design and data could not answer all 5 sub-

questions of the overall research question.

Strength of Study

The design of the study mitigated outside

effects

The test instruments used were measured to be

similar in difficult for both subtests

Same administrator for all tests

Research Question

How did the read aloud accommodation effect test scores in the content areas of mathematics and reading comprehension for students with and

without disabilities?

Results of data analysis show little effect in regards to positive score shifts.

By answering the 5 sub-questions a better understanding of the effect can be attained.

Sub-question #1

Does the accommodation work to provide students with disabilities the opportunity to

demonstrate their knowledge and understanding?

Very small difference in mean scores on subtests for student with SLD. Large P-values Reading 0.11

P = 0.5 Math 0.02

P = 0.446

Data indicates no benefit or additional opportunity to demonstrate knowledge for students with SLD

Sub-question #2Is the accommodation useful on all types of tests

and across content? Type of test:

The instrument of the current study was exclusively multiple choice in format.

The current study was not designed to examine the effect of the accommodation on different types of tests.

Content of test: 2 sections comprised of math and reading comprehension

Provided no benefit for students with SLD in either content area Math P = 0.446 Reading P = 0.5

Provided no benefit for students without SLD in either content area Math P = 0.410, Reading P = 0.011*

*This is a significant difference but it was in a negative direction

No other content areas studied

Sub-question #3

Is the accommodation only useful to those with specific learning disabilities and if not does it at least help those with specific learning disabilities

more? Students without SLD were negatively

impacted Math test scores were lower but not significantly Reading test scores were significantly lower

P = 0.011

Students with SLD scores were higher but only by a small amount

The accommodation was more useful to students with SLD, but only marginally

Sub-question #4

If it is found that the accommodation is a benefit to all who receive it, would giving the

accommodation only to students with specific learning disabilities be unfair?

The current study cannot answer this question

This study’s results indicate that it would be unfair to

read the test aloud to students without SLD.

Survey results indicate that many of the students

without SLD recognize the need for accommodation

Sub-question #5

Does the accommodation relieve test stress and increase the students’ beliefs in their

skills? Do the students believe the accommodation works?

All student with SLD thought accommodation increased test taking ability

Majority of students without SLD, 80%, thought accommodation increased test taking ability

Majority of both groups liked the test accommodation 10% of students without SLD strongly disliked the

accommodation

Results Related to Previous Research

Results contradict those of previous research

All students benefit from having reading tests read

aloud. McKevitt and Elliot (2003)

All students benefit from having math tests read aloud.

Meloy, Deville, and Frisbie (2002).

In the current study neither group benefitted from

having the test read aloud on either subtest.

Relation to previous research continued

Survey Results fall in the middle of previous research

Nelson, Jayanthi, Epstein, and Bursuck (2000) found

that the read aloud accommodation was the least liked

by students.

Majority of students with disabilities and the majority

without disabilities preferred the accommodated test.

McKevitt and Elliot (2003)

In the current study a majority of both groups preferred

the accommodated version

Implications for Use of the Read Aloud Accommodation

No clear indication as to when to use accommodation

Test administrators, including classroom instructors, need to be careful Should not base decision on solely whether a student

have a specific learning disability Should not base decision solely on the students’ beliefs

that the accommodation will help. Need to assess each student and make case by case

decisions Potential for harm

Implications for Further Research

Results don’t support prior research on the effect on

test scores.

A replication on a larger scale to examine the

negative effect of the accommodation.

Further examination of the reasons behind the

students’ beliefs that the accommodation allowed

them to perform better despite the actual results.

Further examination of the stress relieving aspects of

the accommodation.

Summary

The read aloud accommodation did not provide any

positive effect for either subpopulation on either subtest.

The read aloud accommodation negatively impacted

scores of student without SLD on the reading

comprehension test.

Both groups liked having the test read aloud. The reading

more than the math.

Both groups believed they did better on the read aloud

test though they did not.

What does this tell us

This research indicated that the use of some accommodations may be harmful to student performance

We need to be careful with applying accommodations indiscriminately

Checklist boxes are easy to check We need to develop accommodation plans

based on each individual student We cannot depend on student perceptions when

determining effectiveness or even benefit

Determining appropriate accommodations

Need for a comprehensive, individualized approach

Specific accommodation may only be effective on certain tests at certain times

Others are effective at all times i.e. extended time, Braille, scribe

Data, data, data to drive decisions Accommodation plans develop over time

Reference

Thurlow, M., & Bolt, S. (2001). Empirical support for accommodations most often allowed in state policy (Synthesis Report 41). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Retrieved from the World Wide Web: http://education.umn.edu.NCEO/OnlinePubs/Synthesis41.html

Fuch, L, Fuchs, D., Eaton, S., Hamlett, C., & Karns, K. (2000). Supplementing teacher judgments of mathematics test accommodations with objective source data. School Psychology Review, 29(1), 65-85.

Elbaum, B. (2007). Effects of an oral testing accommodation on the mathematics performance of secondary students with and without learning disabilities. The Journal of Special Education, 40(4), 218-229.

Meloy, L., Deville, C., & Frisbie, D. (2002). The effect of a read aloud accommodation on test scores of students with and without a learning disability in reading. Remedial and Special Education, 23(4), 248-255.

http://education.umn.edu.nceo/OnlinePubs/Synthesis41.html

References Continued

Phillips, S. (1994). High-stakes testing accommodations: validity versus disabled rights. Applied Measurement in Education, 7(2) 92-120

Rieck, W., & Wadsworth, D. (2005). Assessment accommodations: Helping students with exceptional learning needs. Intervention in School and Clinic, 41(2), 105-109.

Ketterlin-Gellar, L., Yovanoff, P., & Tindal, G. (2007). Developing a new paradigm for conducting research on accommodations in mathematics testing. Council for Exceptional Children, 73(3), 331-347.

McKevitt, B., & Elliot, S. (2003). Effects and perceived consequences of using read aloud and teacher-recommended testing accommodations on a reading achievement test. School Psychology Review, 12(4), 583-600.

Nelson, J., Jayanthi, M., Epstein, M., & Bursuck, W. (2000). Student preferences for adaptations in classroom testing. Remedial and Special Education, 21(1), 41-52.

Wisconsin Department of Public Instruction Accommodation Matrix, retrieved from the web at http://dpi.wi.gov/oea/pdf/accom09.pdf

Lang, S., Elliot, S., Bolt, D., & Kratochwill, T. (2008). The effects of testing accommodations on students’ performance and reactions to testing. School Psychology Quarterly, 23(1), 107-124.

Bolt, S., & Thurlow, M. (2004). Five of the most frequently allowed testing accommodations in state policy. Remedial and Special Education, 25(3), 141-152.

Cawthon, S., Ho, E., Patel, P., Potvin, D., & Trundt, K. (2009). Multiple constructs for effective accommodations on accommodated test scores for students with disabilities. Practical Assessment, Research and Evaluation, 14(18), 1-9. Retrieved from http://pareonline.net/getvn.asp?v=14n=18




http://pareonline.net/getvn.asp?v=14n=18

accommodation: lang, elliot, bolt, and kratochwill (2008) stated that testing accommodations are...

Documents