This article was downloaded by: [University of Regina]On: 18 November 2014, At: 23:11Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH,UK
Journal of Educational andPsychological ConsultationPublication details, including instructions forauthors and subscription information:http://www.tandfonline.com/loi/hepc20
Teachers' and Students'Preferences for MathematicsInterventions: Implicationsfor Teacher Acceptability inConsultationChristopher T. Arra & Michael W. BahrPublished online: 09 Dec 2009.
To cite this article: Christopher T. Arra & Michael W. Bahr (2005) Teachers' andStudents' Preferences for Mathematics Interventions: Implications for TeacherAcceptability in Consultation, Journal of Educational and Psychological Consultation,16:3, 157-174, DOI: 10.1207/s1532768xjepc1603_2
To link to this article: http://dx.doi.org/10.1207/s1532768xjepc1603_2
PLEASE SCROLL DOWN FOR ARTICLE
Taylor & Francis makes every effort to ensure the accuracy of all theinformation (the “Content”) contained in the publications on our platform.However, Taylor & Francis, our agents, and our licensors make norepresentations or warranties whatsoever as to the accuracy, completeness,or suitability for any purpose of the Content. Any opinions and viewsexpressed in this publication are the opinions and views of the authors, andare not the views of or endorsed by Taylor & Francis. The accuracy of theContent should not be relied upon and should be independently verified withprimary sources of information. Taylor and Francis shall not be liable for anylosses, actions, claims, proceedings, demands, costs, expenses, damages,and other liabilities whatsoever or howsoever caused arising directly orindirectly in connection with, in relation to or arising out of the use of theContent.
This article may be used for research, teaching, and private study purposes.Any substantial or systematic reproduction, redistribution, reselling, loan,sub-licensing, systematic supply, or distribution in any form to anyone isexpressly forbidden. Terms & Conditions of access and use can be found athttp://www.tandfonline.com/page/terms-and-conditions
Dow
nloa
ded
by [
Uni
vers
ity o
f R
egin
a] a
t 23:
11 1
8 N
ovem
ber
2014
Teachers’ and Students’ Preferencesfor Mathematics Interventions:
Implications for TeacherAcceptability in Consultation
Christopher T. ArraNorthern Virginia Community College
Michael W. BahrUniversity of Missouri – St. Louis
Teacher and student acceptability of academic interventions is an importantfactor for school- based consultants to consider in determining the use and ef-fectiveness of academic interventions. This study compared the acceptabilityof 3 theoretically distinct mathematics interventions: a cognitive, a behav-ioral, and a traditional intervention. The study lasted 8 weeks. A total of 18teacher-candidates (TCs) and 55 fourth grade students were exposed to 1 of 3mathematical interventions and were asked to rate the acceptability of eachintervention. Results showed that both TCs and students rated the interven-tions as equally acceptable. These findings, though useful to bothschool-based consultants and trainers, are in contrast with previous findingssuggesting that teachers prefer cognitive and cooperative interventions overbehavioral interventions (de Mesquita & Zollman, 1995).
Effective classroom intervention strategies are one way to help preventand remediate difficulties in mathematics (Skinner, Shapiro, Turco, Cole,& Brown, 1992). School-based consultants and university trainers canwork to develop and implement reliable and valid interventions in the
JOURNAL OF EDUCATIONAL AND PSYCHOLOGICAL CONSULTATION, 16(3), 157–174Copyright © 2005, Lawrence Erlbaum Associates, Inc.
Correspondence should be sent to Christopher T. Arra, Northern Virginia Community Col-lege, Division of Business and Social Sciences, 15200 Neabsco Mills Rd., Wood-bridge, VA 22191. E-mail: [email protected]
Dow
nloa
ded
by [
Uni
vers
ity o
f R
egin
a] a
t 23:
11 1
8 N
ovem
ber
2014
area of mathematics (Rhymer, Hennington, Skinner, & Looby, 1999) thatare sensitive to both teacher (de Mesquita & Zollman, 1995) and studentpreferences (Elliott, Witt, Galvin, & Moe, 1986; Fink, 1995). The need forsuch information is great, given that there is little research on the accept-ability of math interventions. This article briefly reviews general issuespertaining to treatment acceptability and presents findings from a studyin the area of mathematics.
TREATMENT ACCEPTABILITY: TEACHER ANDSTUDENT PREFERENCES
Models of Treatment Acceptability
Treatment acceptability is a judgment by laypersons, clients, and othersof whether treatment procedures are appropriate, fair, and reasonablefor the problem or client (Kazdin, 1981). Several models of treatment ac-ceptability have been developed. The first, developed by Witt and Elliott(1985), stressed the interrelationship of four elements: treatment accept-ability, treatment use, treatment integrity, and treatment effectiveness.Reimers, Wacker, and Koeppl (1987) expanded on Witt and Elliott’swork and focused on the importance of understanding a treatment be-fore acceptability can be assessed. Accordingly, a treatment perceived aslow in acceptability will likely be low in compliance or teacher imple-mentation, whereas a treatment rated as high in acceptability will likelyresult in high compliance.
Nastasi, Varjas, Schensul, Silva, Schensul, and Ratnayake (2000) pro-vided yet another model of treatment acceptability known as the Participa-tory Intervention Model (PIM). The goal of the PIM is to integrate previousresearch into a contemporary approach that promotes ownership and em-powerment of consultees. Increased ownership and involvement hope-fully results in sustained intervention use after consultation has ceased.
Teacher Acceptability
Although several models of acceptability exist, many effective classroominterventions are still unused by teachers due to low levels of acceptability(Martens, Peterson, Witt, & Cirone, 1986; Witt, 1986). For example, Witt dis-cussed four factors that have been linked to teachers’ continued use of anintervention: (a) perceptions of intervention effectiveness, (b) time and per-
158 ARRA AND BAHR
Dow
nloa
ded
by [
Uni
vers
ity o
f R
egin
a] a
t 23:
11 1
8 N
ovem
ber
2014
sonnel resources required, (c) theoretical orientation of the intervention,and (d) the degree to which the treatment is ecologically intrusive. Whenjudging an intervention’s effectiveness, teachers often do not have dataconcerning the effectiveness of a treatment, and they often rely on per-ceived effectiveness of an intervention. With regard to time and personnelresources, Witt found that teachers prefer interventions that require lesstime and fewer personnel resources.
Witt, Martens, and Elliott (1984) investigated the influence of time in-volvement, intervention type, and problem severity on teacher acceptabil-ity and found that interventions requiring high levels of time were lessacceptable for many classroom problems except when behaviors werevery severe. In a related study, Martens et al. (1986) assessed teacher per-ception of effectiveness, ease of use, and frequency of use for variousschool-based interventions. The highest rated interventions were redirec-tion, manipulation of material reward, alteration of classroom environ-ment, consultation, time-out, and removal from classroom.
de Mesquita and Zollman (1995) studied preferences for mathematicalinterventions with 62 elementary school teachers rating their preferencesfor either cognitive, behavioral, or cooperative interventions. The resultsindicated that the teachers significantly preferred the cognitive and coop-erative learning interventions over the behavioral approach, although nodifferences existed between the cognitive and cooperative approaches.The primary limitation of this investigation was that it was unknownwhether the participants had actually used these interventions in theirown teaching; thus, they may have rated interventions with which theywere unfamiliar.
Overall, previous research (Witt, 1986; Witt, Elliott, & Martens, 1984) ontreatment acceptability with teachers has suggested a preference for inter-ventions that are effective, easy to implement, and require short periods oftime to implement. Although several studies (e.g., Martens et al., 1986;Witt, 1986; Witt, Elliott, & Martens, 1984) have increased knowledge of in-tervention acceptability, the research, for the most part, has been analo-gous in nature with little emphasis on ensuring that participants havesufficient knowledge and use of the interventions they rate. Particularly inthe area of mathematics, research that directly exposes teachers to inter-ventions and examines acceptability is needed.
Student Acceptability
Relative to teacher preferences, even fewer studies assess student accept-ability of interventions. Goldberg and Shapiro (1995) assessed acceptability
PREFERENCES FOR MATHEMATICS INTERVENTIONS 159
Dow
nloa
ded
by [
Uni
vers
ity o
f R
egin
a] a
t 23:
11 1
8 N
ovem
ber
2014
by exposing sixth grade students to two group contingencies (interdepen-dent and dependent). Results showed that, prior to treatment, studentspreferred the interdependent condition, although after treatment, they pre-ferred both interventions equally. Elliott et al. (1986) examined sixth grad-ers’ acceptance of 12 common, school-based methods for dealing with mis-behaving peers. Findings revealed that the students rated privatestudent–teacher interactions, group reinforcement, and negative sanctionsas most acceptable, and they rated public reprimand and negative groupcontingencies as unacceptable interventions. Finally, Fink (1995) askedhospitalized children to rate two interventions—time-out and restraint—atadmission and discharge. Children rated time-out as significantly more ac-ceptable than restraint at both admission and discharge.
These studies represent a limited scope of research on student accept-ability of interventions. Relative to our investigation, no research has ex-amined student acceptability of mathematics interventions.
PURPOSE OF THE STUDY
What are the key research findings associated with intervention acceptabil-ity for students with academic difficulties in the area of mathematics? Asearch of the literature in this area resulted in one previously discussedstudy, de Mesquita and Zollman (1995), which specifically addressed mathacceptability. Though providing initial information on math acceptability,the study’s findings that cognitive instruction and cooperative learningwere preferred to behavior instruction should be interpreted with cautiondue to methodological constraints.
This study addresses the need for additional research by extending ourknowledge of the acceptability of three theoretically distinct, mathematicsinterventions (cognitive, behavioral, traditional) by comparing the judg-ments of students to those of teachers. It does so by focusing on acceptabil-ity judgments of cognitive and behavioral interventions byteacher-candidates (TCs; i.e., preservice teachers) who are preparing to en-ter the field of education. Specifically, this study examines four researchquestions. The first question replicates de Mesquita and Zollman (1995)and the remaining three questions extend their research. First, do TCs apriori prefer cognitively based math approaches over behaviorally basedmath approaches? Second, do elementary school students a priori prefercognitively based math approaches over behaviorally based math ap-proaches? Third, how are acceptability judgments by TCs influenced byexposure to (i.e., actual use of) a particular theoretically based math ap-
160 ARRA AND BAHR
Dow
nloa
ded
by [
Uni
vers
ity o
f R
egin
a] a
t 23:
11 1
8 N
ovem
ber
2014
proach? Fourth, how are acceptability judgments by elementary schoolstudents influenced by exposure to (i.e., instruction in) a particular theo-retically based math approach?
METHOD
Participants
TCs. A total of 18 TCs participated in the study. The TCs were elemen-tary education majors at a Midwest university. They were advanced under-graduates (seniors), with only their student-teaching requirement remain-ing. Sixteen women and 2 men, ranging from 18 to 39 years of age,participated in the study. There were 16 White and 2 African AmericanTCs. Each TC had completed two courses in mathematics for elementaryeducation. TCs were recruited through information that was distributed bytheir advisors and they voluntarily enrolled in a three-credit re-search-based practicum course entitled Teaching Mathematics for Elemen-tary School. The TCs agreed to participate in this research study.
Students. As a practicum assignment, each TC was assigned to afourth grade classroom in a school that was part of a university and schoolprofessional development partnership. The school district incorporatespractica experiences as part of the standard math curriculum for the dis-trict. Each TC worked with a small group of three students providing in-struction in mathematics. A total of 55 students participated in the investi-gation (27 male, 28 female). The students were from low- andmiddle-socioeconomic backgrounds and English-speaking families in theMidwest. Their ages ranged from 9 to 11 years old with an average age of 9years, 9 months. The ethnic composition of the students consisted of 82%White, 15% African American, 2% Hispanic, and 2% Native American.
Materials
TC demographic questionnaire. This questionnaire consisted of in-formation related to age, ethnicity, previous relevant coursework, andgender. It was administered to all TCs at the outset of the study by theprincipal researcher.
PREFERENCES FOR MATHEMATICS INTERVENTIONS 161
Dow
nloa
ded
by [
Uni
vers
ity o
f R
egin
a] a
t 23:
11 1
8 N
ovem
ber
2014
TC questionnaire. An acceptability questionnaire with three scenar-ios was administered to the TCs at the beginning of the study, and it wasre-administered 8 weeks later at the study’s conclusion. Each of the scenar-ios described a fourth grade student experiencing difficulty with addition,multiplication, and division. Depending on the theoretical approach usedin the scenario, a particular set of instructional strategies was described.The cognitive scenario focused on use of mnemonic and related strategies,and the behavioral scenario described step-by-step instructional strategiescoupled with reinforcement. The traditional scenario described task analy-sis and a drill/practice orientation.
Witt and Marten’s (1983) Intervention Rating Profile (IRP) is a com-monly used acceptability measure. Previous research (Witt, 1986; Witt,Elliott, et al., 1984; Witt & Martens, 1983; Witt, Martens, et al., 1984) indi-cated that the IRP has demonstrable reliability and validity; consequently,it was selected as the primary measure to assess TC acceptability. How-ever, each IRP is worded in a way that reflects intervention acceptabilityfor each instructional approach. Because this study investigated three dis-tinct approaches to instruction, the IRP was modified to reflect the nuancesof each instructional approach. The modified IRP (see Appendix A) con-sisted of 20 Likert-type items using a 6-point scale ranging from 1 (StronglyDisagree) to 5 (Strongly Agree), and it assessed acceptability in terms of in-structional behavior, instructional impact on students, and teacher skillneeded to implement the intervention. Using the TC sample from thisstudy, internal consistency analyses were conducted on the modified IRPsto assess reliability. The resulting Cronbach alphas were .72 for behavioralinstruction, .76 for cognitive instruction, and .92 for traditional instruction(Arra, 2001).
Student questionnaire. Similar to the IRP, a modified version of theChildren’s Intervention Rating Profile (CIRP; Witt & Elliott, 1985) was ad-ministered to the students along with three scenarios. Similar to the TC sce-narios, the children’s scenarios focused on a fourth grade student experi-encing difficulties in math. Three different instructional approaches (i.e.,cognitive, behavioral, traditional) were described, though worded for ele-mentary students.
The modified CIRP was a 7-item scale with 6 points ranging from 1(Strongly Disagree) to 6 (Strongly Agree) that assessed a child’s acceptabilityof a teaching approach and its perceived appropriateness and effective-ness (see Appendix B). Half of the items were reverse-keyed to control for
162 ARRA AND BAHR
Dow
nloa
ded
by [
Uni
vers
ity o
f R
egin
a] a
t 23:
11 1
8 N
ovem
ber
2014
response sets. Cronbach alphas were .79 for the behavioral scale, .71 for thecognitive scale, and .55 for the traditional scale (Arra, 2001).
Treatment integrity measures. Two treatment integrity measureswere developed by the researchers. The TC Accuracy of ImplementationChecklist was completed by all TCs at the end of every session. This check-list has three forms (behavioral, cognitive, traditional), each with separatesteps to which the TC checked “Yes” or “No.” The checklist was designedto assess if the TC (a) provided the group with instruction and examples, (b)monitored the group during practice time, (c) provided rewards when ap-propriate, and (d) gave the math assignment sheet at the end of the session.
The researchers developed the Observer Accuracy of ImplementationChecklist. The principal researcher observed 10 of the TC’s 216 sessionsand completed a checklist for every session observed. This 9-item checklistassessed if the TC (a) began instruction on time, (b) implemented the in-structional sequence appropriately, (c) provided examples and modeling,(d) monitored the group during practice, (e) refrained from giving or gavereinforcers as appropriate, and (f) gave the math assignment sheet at theend of each session (see Appendix C).
Procedures
At the outset of the study, the IRP was group-administered by the principalresearcher to the TCs. Order effect was controlled by dispersing the threescenarios to the TCs in different orders at pre- and posttest. The TCs wereprovided standard instructions to read each of the scenarios and provideacceptability ratings. Next, a demographic questionnaire was completedby all TCs. The researchers then randomly assigned the 18 TCs to one of thethree treatment groups. At the end of the first training session, TCs wererandomly assigned to a fourth grade classroom.
TC training. Training was provided for approximately 2 hr before theteachers implemented their interventions for each topic (addition, division,multiplication). All TCs were trained together as a group in one classroomat the university and instructed on the theoretical and applied componentsof each topic. A mathematics professor taught the intervention proceduresfor each of the three treatments. The training for each treatment is describedin the following sections.
PREFERENCES FOR MATHEMATICS INTERVENTIONS 163
Dow
nloa
ded
by [
Uni
vers
ity o
f R
egin
a] a
t 23:
11 1
8 N
ovem
ber
2014
Cognitive instruction training. All TCs met for 2 hr before the imple-mentation of each topic. The first 2-hr training session focused on thetopic of addition. During a portion of this session, TCs were taught thetheory behind cognitive strategies instruction. Topics such as informationprocessing, mnemonics, and self-talk were covered. Next, the professorprovided detailed explanation about the cognitive teaching approach.The professor discussed the use of a mnemonic strategy and how to applyit to addition problems. The professor then modeled several additionproblems using the mnemonic strategy. TCs then practiced applying themnemonic strategy to addition problems under the supervision of theprofessor.
Irrespective of their assigned treatment condition, TCs met with groupsof three students for four 30-min sessions. TCs demonstrated the mne-monic strategy to the students and applied it to the addition problems. TheTCs then modeled several addition problems for the students using themnemonic strategy. The students then practiced addition problems usingthe intervention. The TC monitored the students to ensure that the inter-vention was being appropriately applied. Corrective feedback was givento the students by the TCs, but TCs were instructed to minimize verbal re-inforcement (e.g., Good job! or Excellent!) because verbal reinforcementwas a component of the behavioral instruction treatment. Students receiv-ing cognitive instruction kept a journal in which they were given 5 min toanswer one self-reflective question at the end of each session. Some exam-ples of questions were, “Describe what the teacher taught today,” “Howwould you teach what you learned today to a friend?,” and “How did to-day’s lesson help you with math?”
Behavioral instruction training. Approximately 1 hr of the first train-ing session was spent teaching the TCs the theory behind behavioral inter-ventions. Topics such as operant reinforcement, schedules of reinforce-ment, and types of reinforcers were covered. Next, the TCs were taught, indetail, a three-step intervention: task analysis, tangible reinforcement, andverbal reinforcement. The professor provided examples of each. The TCswere then shown how to apply task analysis and reinforcers to additionproblems. The professor modeled these interventions for the TCs. The TCsthen practiced these interventions under the supervision of the professor.Next, the TCs were shown how to score a curriculum-based mathematicsprobe. They were taught the metrics used to score the probes (i.e., problemsattempted, problems correct) and were shown how to graph the two met-rics for each student.
164 ARRA AND BAHR
Dow
nloa
ded
by [
Uni
vers
ity o
f R
egin
a] a
t 23:
11 1
8 N
ovem
ber
2014
During each session with students, the TC modeled task analysis for ad-dition problems and provided examples. The TC analyzed addition prob-lems by teaching students a series of steps, each of which were taughtsequentially. The students then practiced addition problems using the taskanalysis while the TC monitored the students to ensure the intervention wasbeingappropriatelyapplied.Throughout thesessions, theTCprovidedtan-giblereinforcers tothestudents (e.g., stickers) foreffort.Effort includedpay-ing attention to instruction, working quietly when appropriate, and askingquestions in an appropriate fashion. The TCs in the behavioral group wereencouraged to provide substantial verbal praise and encouragementthrough the frequent use of verbal reinforcers (e.g., Good job!, Way to go!,Excellent!). Finally, during the last 2 min of each session, the students com-pleted the curriculum-based math probe. The TCs then corrected the probesusing two metrics: number of problems attempted and number of problemscorrect. At the beginning of the following session, the TCs assisted the stu-dents in graphing these two metrics onto separate charts. This allowed thestudents to record and track their math performance.
Traditional instruction training. The TCs were trained on how to usetraditional classroom interventions. For the purpose of this study, this con-stituted the control condition. In the area of addition, the interventions in-cluded task analysis and drill-and-practice. Because task analysis was alsoa component of behavioral instruction, traditional instruction differed intwo important ways: It emphasized more drill and practice than in the be-havioral treatment, and it minimized reinforcement (i.e., no external rein-forcers and limited use of verbal praise by the TC). The professor modeledeach strategy for the group as it applied to addition problems. The groupthen practiced task analysis and drill-and-practice techniques under the su-pervision of the principal researcher. The TCs in the traditional group thenimplemented the addition intervention in the classroom.
During the students’ sessions, the TCs analyzed addition problems andtaught the students addition problems in a sequential, step-by-step fash-ion. The students then practiced addition problems using these steps. Dur-ing this time, the TCs monitored their groups for accuracy of interventionimplementation.
Training in multiplication and division. When all groups of TCs im-plemented four sessions of the addition intervention, they then returned tothe university to receive a 2-hr training session on the second topic, multi-
PREFERENCES FOR MATHEMATICS INTERVENTIONS 165
Dow
nloa
ded
by [
Uni
vers
ity o
f R
egin
a] a
t 23:
11 1
8 N
ovem
ber
2014
plication. This cycle was repeated for the third topic, division. Subtractionwas not included in this component of the university curriculum; conse-quently, it was not included in this study.
Treatments. The TCs were in their school placements for a total of 1230-min sessions over a 6-week period. Because each of the TCs was ran-domly assigned to a treatment condition, their students received one of thetreatments twice weekly in their classrooms for 30 min per session. Eachmath topic (addition, multiplication, and division) was covered for foursessions. The CIRPs were administered at pre- and posttreatment by theTCs to the students. Standard instructions were given to all students. Thestudent demographic questionnaires were completed by the classroomteachers at the outset of the study. For the CIRP, order effect was controlledby dispensing the scenarios in different orders to the students at pre- andposttest.
Interrater Agreement
For both the IRP and CIRP, the researchers summed the individual itemscores tocreatea total score,whichservedas theunitofanalysis.Tenpercentofeachof thetwomeasureswasrandomlyselectedfor interrateragreement.An outside observer, unfamiliar with the study’s purpose, was recruited toserve as a blind rater. Percentage of agreement was calculated by dividingthe number of agreements by the total agreements plus disagreements andmultiplying by 100% (Sulzer-Azaroff & Mayer, 1977). Interrater agreementfor all measures was 100% on the acceptability checklists.
Treatment Integrity
Treatment integrity is the extent to which the treatment was imple-mented as planned. Two treatment integrity checklists were used in thisstudy. The TC Accuracy of Implementation Checklist, which had threeforms (cognitive, behavioral, traditional), was completed daily at the endof every session by each TC. Each TC completed separate checklists de-pending on the instructional treatment group to which they belonged.The TCs complied with requests to complete these checklists as evi-denced by high return rates: 93% of the sessions on cognitive instruction,
166 ARRA AND BAHR
Dow
nloa
ded
by [
Uni
vers
ity o
f R
egin
a] a
t 23:
11 1
8 N
ovem
ber
2014
85% of the sessions on behavioral instruction, and 98% on traditional in-struction included checklists.
Of these cognitive, behavioral, and traditional checklists, 82, 79, and97%, respectively, contained all “Yes” responses, thus indicating that TCsreported relatively high levels of treatment implementation. Inspection ofthe cognitive integrity checklist revealed nine “No” responses to items thatstated, “Did you give the math probe at the end of the session?,” and “Didstudents complete their journal?” Inspection of the behavioral integritychecklists revealed four “No” responses to items that stated, “Did the stu-dents graph their problems attempted and problems completed?,” and“Did you give the math probe at the end of the session?” Inspection of thetraditional integrity checklists revealed two “No” responses to the itemthat stated, “Did you check the group’s work for accuracy?”
The second checklist, the Observer Accuracy of Implementation Check-list, was completed by one of the researchers. Inspection of the checklistsrevealed “Yes” responses to all of the items.
Data Analysis
Two types of analyses were used to examine the data. First, means and stan-dard deviations of all items of the modified IRP and CIRP for each treat-ment group (cognitive vs. behavior vs. traditional) were examined to see ifpatterns emerged (acceptable vs. unacceptable) on individual items. Sec-ond, given initial evidence of psychometric adequacy for the modified IRPand CIRP, scores from all items on a measure were summed and totalscores were used as the unit of analysis to assess between and within groupdifferences.
RESULTS
Table 1 displays the means and standard deviations of TC responses to in-dividual items on the modified IRP and Table 2 shows IRP descriptive sta-tistics on the total score. Using the total scores as the unit of analysis, a 3 × 3× 2 (behavioral vs. cognitive vs. control × behavioral rating vs. cognitiverating vs. control rating × preintervention vs. postintervention) mixedmodel analysis of variance (ANOVA) with the last two factors treated as re-peated measures revealed a nonsignificant three-way interaction, F(2, 13) =.408, p = .673, Cohen’s f = .059. Because the interaction was nonsignificant,there were no reliable differences between means that merit further investi-
PREFERENCES FOR MATHEMATICS INTERVENTIONS 167
Dow
nloa
ded
by [
Uni
vers
ity o
f R
egin
a] a
t 23:
11 1
8 N
ovem
ber
2014
168
TABLE 1Means and Standard Deviations for Pre- and Posttest Teacher-Candidate Ratings on the Intervention
Rating Profile (IRP)
Cognitive Behavioral Traditional
Pretest Posttest Pretest Posttest Pretest Posttest
Question M SD M SD M SD M SD M SD M SD
1. 4.28 1.87 4.87 1.36 4.72 1.23 4.94 0.85 4.78 1.31 4.06 1.24
2. 4.33 1.50 5.00 0.97 4.67 1.03 4.88 0.89 5.00 1.03 4.69 1.203. 4.06 1.75 4.63 1.02 4.11 1.23 4.06 1.18 4.00 1.24 3.88 1.454. 4.17 1.50 4.75 1.13 4.50 1.38 4.69 0.95 4.28 1.36 4.13 1.265. 4.11 1.37 4.75 1.13 4.33 1.33 4.44 0.89 4.50 1.15 4.06 1.346. 4.00 1.41 4.73 1.18 4.56 1.34 4.56 1.09 4.39 1.42 4.09 1.487. 4.11 1.68 4.64 1.00 4.44 1.42 4.81 0.91 4.61 1.50 3.94 1.538. 4.33 1.61 4.81 1.33 4.28 1.49 4.94 1.34 4.83 1.34 4.50 1.469. 4.06 1.55 4.44 1.15 4.28 1.36 4.31 1.66 4.28 1.41 4.00 1.37
10. 4.44 1.46 4.50 1.21 4.39 1.33 4.50 1.15 4.56 1.20 4.06 1.3911. 4.28 1.71 4.44 1.32 4.67 1.50 5.00 1.15 4.78 1.44 4.31 1.7412. 3.94 1.26 3.31 1.54 4.44 1.10 4.56 1.21 4.39 0.98 4.94 1.0613. 4.06 1.30 3.45 1.40 4.33 1.19 4.13 1.36 3.83 1.42 4.56 0.8914. 3.94 1.26 3.69 1.30 4.33 1.28 4.25 1.39 3.94 1.30 4.29 0.9615. 4.00 1.14 3.63 1.31 4.17 1.25 4.19 1.38 4.17 1.29 4.31 1.0816. 3.56 1.42 3.31 1.54 3.83 1.47 3.31 1.49 3.06 1.66 4.50 1.4617. 3.56 1.20 3.63 1.50 3.78 1.83 3.25 1.65 3.39 1.54 4.62 0.8118. 3.89 1.18 3.31 1.30 3.61 1.42 3.38 1.59 3.50 1.62 4.38 1.1419. 3.17 1.34 2.81 1.47 4.50 1.58 4.69 1.25 4.39 1.61 4.81 0.9820. 3.39 1.69 2.88 1.36 4.67 1.33 4.56 1.32 4.39 1.33 4.73 1.12
TABLE 2Means and Standard Deviations of Total Scores for the Intervention Rating Profile
Behavioral Cognitive Traditional
IRP M SD M SD M SD
Prebehavioral 95.33 44.85 90.00 20.08 83.83 15.77
Postbehavioral 97.50 15.32 78.00 14.76 83.83 20.66Precognitive 69.67 25.72 108.50 44.52 76.17 22.22Postcognitive 74.83 14.54 95.00 25.99 78.50 6.50Pretraditional 83.67 21.32 92.00 12.19 82.17 21.19Posttraditional 81.00 15.07 89.00 10.80 92.83 17.90
Dow
nloa
ded
by [
Uni
vers
ity o
f R
egin
a] a
t 23:
11 1
8 N
ovem
ber
2014
gation via follow-up analysis. Consequently, pertaining to this study’s firstresearch question—Do TCs a priori prefer cognitively based math ap-proaches over behaviorally based instruction?—the lack of significance in-dicates that TCs perceived all instructional approaches (cognitive, behav-ioral, and control) as equally acceptable. Item scores on the modified IRPcould range from a low of 1 to a high of 6, with a midpoint of 3.5 (see Table1). Total scores could range from a low of 20 to a high of 120 with a midpointof 70 (see Table 2). Higher item and total scores indicate more acceptability.Inspection of the data in Tables 1 and 2 reveals that preintervention ratingsby TCs are generally favorable.
Table 3 lists the means and standard deviations of student responses onindividual items for the CIRP and Table 4 displays the total scores. Usingtotal scores as the unit of analysis, a 3 × 3 × 2 (behavioral vs. cognitive vs.control × behavioral rating vs. cognitive rating vs. control rating ×preintervention vs. postintervention) mixed model ANOVA with the lat-ter two factors treated as repeated measures revealed a nonsignificant,three-way interaction, F(2, 49) = .784, p = .462, Cohen’s f = .071. The conclu-sion about the study’s second research question—Do elementary studentsa priori prefer cognitive instruction over behavior instruction?—is thatthere is no difference in perceived acceptability. Further inspection of themeans in Tables 3 and 4 reveals that preintervention ratings by studentsare generally high and indicate positive assessments of the various instruc-tional approaches.
PREFERENCES FOR MATHEMATICS INTERVENTIONS 169
TABLE 3Means and Standard Deviations of Pre- and Posttest Student Ratings
on the Children’s Intervention Rating Profile
Cognitive Behavioral Traditional
Pretest Posttest Pretest Posttest Pretest Posttest
Question M SD M SD M SD M SD M SD M SD
1. 5.32 1.36 5.39 1.23 4.92 1.36 5.61 0.89 5.15 1.31 5.17 1.23
2. 1.79 1.46 1.56 0.75 1.94 1.47 1.39 0.78 2.17 1.63 2.13 1.583. 4.30 1.79 3.98 1.70 3.52 1.86 3.48 1.87 4.38 1.78 4.39 1.594. 4.21 1.39 4.52 1.88 4.19 1.81 4.26 1.76 4.40 1.80 4.43 1.705. 4.87 1.58 5.01 1.39 4.87 1.44 5.13 1.25 4.75 1.52 4.91 1.506. 5.17 1.34 4.72 1.27 4.81 1.58 5.13 1.36 4.19 1.92 4.96 1.197. 4.89 1.53 5.25 1.03 5.00 1.34 5.57 0.84 4.96 1.32 5.39 1.08
Dow
nloa
ded
by [
Uni
vers
ity o
f R
egin
a] a
t 23:
11 1
8 N
ovem
ber
2014
Research questions 3 and 4, respectively, examined how TC and studentacceptability are influenced by exposure to a theoretically based interven-tion. The failure to achieve a significant Treatment × Time × Rating interac-tion on either TC (see Table 2) or student (see Table 4) ratings indicates thatexposure to a theoretically based approach or control condition did not in-fluence acceptability assessments from pre- to postintervention forpreservice teachers or students.
DISCUSSION
This study assessed the acceptability of three remedial mathematics ap-proaches by teachers and students. The results showed that after the TCsimplemented one of the instructional approaches for a 6-week period,they did not indicate a preference for any one of the three interventions. Itis important to note that all three groups of TCs viewed each set of inter-ventions favorably. This finding is useful for both school-based consul-tants and trainers to consider when recommending a mathematics inter-vention. These findings are contrary to those of de Mesquita and Zollman(1995), who found that teachers preferred cognitive mathematics inter-ventions over behavioral approaches. There are, however, noteworthydifferences between this study and the de Mesquita and Zollman investi-gation. In this study, the TCs actually implemented and experienced atheoretically based intervention, whereas in the study by de Mesquita andZollman, teachers were only surveyed about their preferences for inter-ventions without having to implement them or report previous experi-ence, or lack thereof, in using them. The analogue nature of that investiga-
170 ARRA AND BAHR
TABLE 4Means and Standard Deviations for Total Scores on the Children’s Intervention
Rating Profile (CIRP)
Behavioral Cognitive Traditional
CIRP M SD M SD M SD
Prebehavioral 32.63 5.60 29.24 7.10 30.56 9.14
Postbehavioral 31.42 8.26 28.24 7.99 27.56 8.30Precognitive 31.58 6.29 30.18 4.93 30.69 8.03Postcognitive 30.53 7.56 30.71 6.71 29.25 6.16Pretraditional 28.84 7.83 30.18 6.63 28.00 7.89Posttraditional 29.79 6.19 28.35 9.05 29.00 5.42
Dow
nloa
ded
by [
Uni
vers
ity o
f R
egin
a] a
t 23:
11 1
8 N
ovem
ber
2014
tion limited the utility of the findings. The findings by de Mesquita andZollman reflect the preferences of elementary mathematics teachers withan average of 15 years of experience for interventions with which theywere familiar and had previously implemented in the classroom. By con-trast, this study may reflect the preferences of TCs, early-career profes-sions at the preservice level.
An important finding, useful for both school-based consultants andtrainers, is that all students, irrespective of treatment or control group,rated the interventions relatively high, indicating high levels of acceptabil-ity. The results showed that the students did not demonstrate any prefer-ences for the three approaches even after having been exposed to only oneof them for 6 weeks. Overall, these findings suggest to consultants andtrainers that students find cognitive, behavioral, and traditional ap-proaches to mathematics instruction acceptable. One explanation for thesefindings could be that mathematics instruction in the classroom sharessimilar aspects—such as cognitive strategies instruction or the use of socialor tangible reinforcers—with the approaches in this study. It is possiblethat the classroom teacher’s mathematics instruction may incorporate sig-nificant aspects of these interventions, and this may have influenced howthe students rated the interventions. Upon rating each intervention, thestudents may have viewed each approach as both acceptable and effectivebecause it shared similarities with mathematics instruction in their ownclassrooms.
In a similar fashion, it is also possible that high acceptability was in-fluenced by the training the TCs received. This study ensured that TCsexperienced didactic and experiential learning with one of three ap-proaches. The implementation of at least one of the theoretically basedapproaches, coupled with ongoing instructional support from a univer-sity professor, may have contributed substantially to the TCs’ relativelyhigh level of acceptability.
The study’s findings should be considered relative to its constraints andimplications for future research. The first limitation pertains to the trainingof the TCs. In this study, the TCs, as part of their university curriculum,were taught all three interventions, but they implemented only one. There-fore, carry-over effects were possible, and to eliminate this threat, a futurereplication might provide TC training in separate classrooms. A secondimportant limitation is that the elementary students involved in the studywere not randomly assigned to the TCs. The TCs, however, were ran-domly assigned to a classroom, and students in each classroom were thenrandomly assigned to a TC. A preferable design would involve completerandomization of assignments. This limits the further generalizability of
PREFERENCES FOR MATHEMATICS INTERVENTIONS 171
Dow
nloa
ded
by [
Uni
vers
ity o
f R
egin
a] a
t 23:
11 1
8 N
ovem
ber
2014
the results. Finally, for this study, only 10% of the data were analyzed forinterrater reliability.
These constraints notwithstanding, this study adds important informa-tion to school-based consultants’ and trainers’ understanding of treatmentacceptability related to both teachers and students. Given the novelty ofthis study’s findings in relation to de Mesquita and Zollman’s (1995) work,intervention researchers are advised to heed the call for investigations thatfurther our understanding of treatment acceptability.
REFERENCES
Arra, C. T. (2001). CBM, IRP, and CIRP reliability analyses. Unpublished raw data.de Mesquita, P. B., & Zollman, A. (1995). Teachers’ preferences for academic intervention
strategies in mathematics: Implications for instructional consultation. Journal of Educationaland Psychological Consultation, 6, 159–174.
Elliott, S. N., Witt, J. C., Galvin, G. A., & Moe, G. L. (1986). Children’s involvement in interven-tion selection: Acceptability of interventions for misbehaving peers. Professional Psychol-ogy: Research and Practice, 17, 235–241.
Fink, A. D., (1995). Children’s perceptions of treatment acceptability. Dissertation Abstracts In-ternational: Section B: The Sciences and Engineering, 56, 2862.
Goldberg, R., & Shapiro, E. S., (1995). In-vivo rating of treatment acceptability by children: Ef-fects of probability instruction on students spelling performance under group contingencyconditions. Journal of Behavioral Education, 5, 415–432.
Kazdin, A. E. (1981). Acceptability of child treatment techniques: The influence of treatmentefficacy and adverse side effects. Behavior Therapy, 11, 329–344.
Martens, B. K., Peterson, R. L., Witt, J. C., & Cirone, S. (1986). Teacher perceptions of school-based interventions. Exceptional Children, 53, 213–223.
Nastasi, B., Varjas, K., Schensul, S., Silva, K., Schensul, J., & Ratnayake, P. (2000). The participa-tory intervention model: A framework for conceptualizing and promoting intervention ac-ceptability. School Psychology Quarterly, 15, 207–232.
Reimers, T. M., Wacker, D. P., & Koeppl, G. (1987). Acceptability of behavior treatments: A re-view of the literature. School Psychology Review, 16, 212–227.
Rhymer, K., Hennington, C., Skinner, C. H., & Looby, E. J. (1999). The effects of explicit timingon mathematics performance in second-grade Caucasian and African-American students.School Psychology Quarterly, 14, 397–407.
Skinner, C. H., Shapiro, E. S., Turco, T. L., Cole, C., & Brown, D. (1992) A comparison of self-and peer-delivered immediate corrective feedback on multiplication performance. Journalof School Psychology, 30, 101–116.
Sulzer-Azaroff, B., & Mayer, G. R. (1977). Applying behavior analysis procedures with children andyouth. New York: Holt, Reinhart & Winston.
Witt, J. C. (1986). Teachers’ resistance to the use of school-based interventions. Journal of SchoolPsychology, 24, 37–44.
Witt, J. C., & Elliott, S. N. (1985). Acceptability of classroom management strategies. In T. R.Kratochwill (Ed.), Advances in school psychology (Vol. 4, pp. 251–288). Hillsdale, NJ: Law-rence Erlbaum Associates, Inc.
172 ARRA AND BAHR
Dow
nloa
ded
by [
Uni
vers
ity o
f R
egin
a] a
t 23:
11 1
8 N
ovem
ber
2014
Witt, J. C., Elliott, S. N., & Martens, B. K. (1984). Acceptability of behavioral intervention usedin classrooms: The influence of amount of teacher time, severity of behavior problem, andtype of intervention. Behavioral Disorders, 9, 95–104.
Witt, J. C., & Martens, B. K. (1983). Assessing the acceptability of behavioral interventions usedin the classrooms. Psychology in the Schools, 20, 510–517.
Witt, J. C., Martens, B. K., & Elliott, S. N. (1984). Factors affecting teachers’ judgments of the ac-ceptability of behavioral interventions: Time involvement, behavior problem severity, andtype of intervention. Behavior Therapy, 15, 204–209.
PREFERENCES FOR MATHEMATICS INTERVENTIONS 173
APPENDIX AIntervention Rating Profile (IRP)
Directions: Please answer the following questions using a scale rangingfrom 1(Strongly Disagree) to 6 (Strongly Agree).
1._____ Most teachers would find this approach to math instruction helpful for studentswith learning challenges.
2._____ Most teachers would find this instructional approach appropriate for variousmath problems.
3._____ The student’s math problem is severe enough to warrant the use of this instruc-tional approach.
4._____ This instructional approach should prove effective in helping the student withtheir math difficulties.
5._____ This would be an acceptable instructional approach for the student’s math diffi-culties.
6._____ Overall, this type of instruction would be beneficial for the student.7._____ I would be willing to use this instructional approach in the classroom.8._____ This instructional approach would be appropriate to use before making a refer-
ral.9._____ This instructional approach would not negatively affect a student’s math per-
formance.10.____ This instructional approach would not result in risk to the student.11.____ This instructional approach would not be considered a last resort.12.____ This instructional approach is practical in the amount of time required for par-
ents who may assist the student in their math assignments.13.____ This instructional approach is practical in the amount of time required for
teachers.14.____ This instructional approach is appropriate in the amount of time necessary for a
teacher to record the student’s progress.15.____ This instructional approach is practical in the amount of out-of-school time re-
quired for the student to use the intervention.16.____ This instructional approach would not be difficult to implement in a classroom
with 30 other students.17.____ This instructional approach would not be disruptive to other students.18.____ It would not be difficult to use this instructional approach and still meet the
needs of other students.19.____ Teachers are likely to use this instructional approach because of its simplicity.20.____ Teachers are likely to use this instructional approach because of its ease of use.
Dow
nloa
ded
by [
Uni
vers
ity o
f R
egin
a] a
t 23:
11 1
8 N
ovem
ber
2014
Christopher T. Arra, PhD, is an Assistant Professor of Psychology at Northern Virginia Com-munity College. His research interests include cross-cultural school-based consultation andacademic interventions.
Michael W. Bahr, PhD, is an Associate Professor in the School Psychology Program at the Uni-versity of Missouri – St. Louis. His primary professional interests are in the areas ofconsulation/interventions and multicultural diversity.
Action Editor: Emilia C. Lopez
174 ARRA AND BAHR
APPENDIX BChildren’s Intervention Rating Profile
Directions: Please answer the following questions using a scale rangingfrom 1(Strongly Disagree) to 6 (Strongly Agree).
1._____ This is a helpful way to teach math.2._____ This way to teach math is too hard.3._____ This way to teach math may be hard for other kids.4._____ There are better ways to teach math to kids than this way.5._____ This way to teach math is a good way to use with other kids.6._____ I like this way of teaching math.7._____ I think that this way to teach math will help kids do better in school.
APPENDIX CObserver Accuracy of Implementation Checklist
1. Did the TC begin group instruction on time? YES NO2. Was the instruction implemented properly? YES NO3. Circle the focus of instruction.
ADDITION MULTIPLICATION DIVISION4. Were the math problems modeled for the children? YES NO5. Were examples provided for the children? YES NO6. Did the TC monitor the group during practice time? YES NO7. Did the TC provide corrective feedback when necessary? YES NO8. Did the TC either refrain from or give reinforcers as required? YES NO9. Did the TC give the math probe at the end of the session? YES NO
Dow
nloa
ded
by [
Uni
vers
ity o
f R
egin
a] a
t 23:
11 1
8 N
ovem
ber
2014
