effects of treatment integrity failures during ...with children with developmental disabilities....

EFFECTS OF TREATMENT INTEGRITY FAILURES DURINGDIFFERENTIAL REINFORCEMENT OF ALTERNATIVE BEHAVIOR:

A TRANSLATIONAL MODEL

CLAIRE ST. PETER PIPKIN

WEST VIRGINIA UNIVERSITY

AND

TIMOTHY R. VOLLMER AND KIMBERLY N. SLOMAN

UNIVERSITY OF FLORIDA

Differential reinforcement of alternative behavior (DRA) is used frequently as a treatment forproblem behavior. Previous studies on treatment integrity failures during DRA suggest that theintervention is robust, but research has not yet investigated the effects of different types ofintegrity failures. We examined the effects of two types of integrity failures on DRA, startingwith a human operant procedure and extending the results to children with disabilities in aschool setting. Human operant results (Experiment 1) showed that conditions involvingreinforcement for problem behavior were more detrimental than failing to reinforce appropriatebehavior alone, and that condition order affected the results. Experiments 2 and 3 replicated theeffects of combined errors and sequence effects during actual treatment implementation.

Key words: autism, human operant, sequence effects, translational research, treatmentintegrity

_______________________________________________________________________________

Differential reinforcement of alternative be-havior (DRA), a commonly used treatment forproblem behavior, typically involves withhold-ing reinforcers following problem behavior(extinction) and providing reinforcers contin-gent on some appropriate, alternative response.For example, a DRA treatment for attention-maintained screaming might involve refrainingfrom talking to or making eye contact with theindividual while he or she is screaming andproviding attention following some appropriatebehavior, such as saying hello. When imple-mented with high levels of treatment integrity,DRA produces substantial decreases in problembehavior and increases in appropriate behavior(e.g., Carr & Durand, 1985; Tiger, Hanley, &Bruzek, 2008; Vollmer & Iwata, 1992).However, caregivers may implement DRA withrelatively low levels of treatment integrity

because those individuals may have a longhistory of reinforcing problem behavior (andmay therefore find extinction procedures diffi-cult), or they may have difficulty providingreinforcers consistently following alternativeresponses (Volkert, Lerman, Call, & Trosclair-Lasserre, 2009).

To date, the effects of treatment integrityfailures on DRA procedures have not beenexamined extensively. Thus, the level of integ-rity adequate for achieving intervention effectsduring differential reinforcement proceduresremains largely unknown. Studies examiningreduced integrity on differential reinforcementhave obtained mixed results (Mazaleski, Iwata,Vollmer, Zarcone, & Smith, 1993; Mueller etal., 2003; Worsdell, Iwata, Hanley, Thompson,& Kahng, 2000). These mixed results may bedue to experimenters combining different typesof integrity failures into a single measure ofoverall integrity. Some components of DRAprocedures are implemented with better integ-rity than others (e.g., Codding, Feinberg,Dunn, & Pace, 2005), which may have an

Correspondence concerning this article should beaddressed to Claire St. Peter Pipkin, Box 6040, WestVirginia University, Morgantown, West Virginia 26506(e-mail: [email protected]).

doi: 10.1901/jaba.2010.43-47

JOURNAL OF APPLIED BEHAVIOR ANALYSIS 2010, 43, 47–70 NUMBER 1 (SPRING 2010)

47

impact on treatment outcomes. For example,providing reinforcers following problem behav-ior (one type of failure) may have differenteffects than failing to deliver reinforcers follow-ing appropriate behavior (another type offailure). Collapsing these failures into a singlemeasure of treatment integrity would notcapture the differences between types of failures.Thus, single measures of treatment integritymight suggest that less than optimal levels ofintegrity (e.g., 80%) are acceptable in somecircumstances (when the majority of failuresinvolve noncritical treatment components) butare unacceptable in others (when the failuresoccur on critical treatment components). Inother words, poor integrity on noncriticalcomponents of the treatment may result in alow overall integrity score even though theeffectiveness of the treatment is retained.

Two (of many) possible types of treatmentintegrity failures involve either the failure todeliver an earned reinforcer according to thetreatment schedule (here termed an omissionerror) or the delivery of a reinforcer followingproblem behavior (here termed a commissionerror). These two types of errors reflect integrityfailures during different components of DRA(the programmed reinforcement schedule com-ponent and the extinction component, respec-tively).

An error of omission occurs when thecaregiver fails to deliver a reinforcer at thecorrect time. Northup, Fisher, Kahng, Harrell,and Kurtz (1997) examined the effects ofomission errors on a treatment involving bothdifferential reinforcement and punishment.Integrity failures involved intermittently omit-ting treatment components. For example,during evaluations of integrity failures, time-out was delivered on a variable-ratio (VR)schedule that successively doubled (e.g., fixed-ratio [FR] 1 to VR 2 to VR 4), andreinforcement was available according to vari-able-interval (VI) schedules that doubled induration (e. g., VI 1 min and VI 2 min). The

treatment retained its efficacy for all 3 partic-ipants when therapists implemented bothcomponents with 50% integrity. Slight increas-es in problem behavior occurred when integritydropped to 25% for 2 of the 3 participants.These results suggest that omission errors maybe detrimental to treatment outcome if integritydecreases to low levels. However, Northup et al.did not examine the effects of integrity ondifferential reinforcement independent of thepunishment procedure, so the effects of omis-sion errors on DRA remain unknown.

Research on the effects of ongoing reinforce-ment for problem behavior during behavioraltreatments may provide some evidence of thepossible effects of commission errors. Often, thefailure to implement the extinction componentof the DRA causes the treatment to fail entirely,with rates of problem behavior remaining highand rates of appropriate behavior low or zero(e.g., Kelley, Lerman, & Van Camp, 2002;Shirley, Iwata, Kahng, Mazaleski, & Lerman,1997; Worsdell et al., 2000). For example,Worsdell et al. examined the effects of ongoingreinforcement for problem behavior (commis-sion errors) by varying the reinforcementschedule for problem behavior while maintain-ing an FR 1 schedule for appropriate behavior.For all participants, response allocation eventu-ally shifted toward appropriate behavior as theschedule for problem behavior became thinner.These results suggest that errors of commissionmay be detrimental to treatments like DRA.

Vollmer, Roane, Ringdahl, and Marcus(1999) examined the effects of combinedcommission and omission errors, which in-volved both periodic reinforcement of problembehavior (commission errors) and failures toreinforce appropriate behavior (omission er-rors). Overall, the effects of DRA were resistantto decrements in integrity level. However, ratesof problem behavior increased somewhat whenthe experimenters reinforced both desirable andundesirable responses. These increases in re-sponse rate could be problematic when dealing

48 CLAIRE ST. PETER PIPKIN et al.

with severe problem behavior (e.g., self-injury,aggression, or property destruction) and againunderscore the potential negative effects oftreatment integrity failures. One limitation ofthe Vollmer et al. study was that almost all ofthe integrity failure phases followed a phase inwhich the treatment was implemented perfectly,raising the possibility that sequence effectsplayed a role.

The Vollmer et al. (1999) investigationrepresents one of the few studies that haveparametrically examined the types and levels ofintegrity failures on DRA. One potential reasonfor this lack of research may be due todifficulties associated with conducting paramet-ric studies with clinical populations. In partic-ular, these studies may require extensive periodsof time to complete, which may delay successfultreatment development. In addition, parametrictreatment research may not yield differentiatedresults without preliminary data to guide theselection of parameter types and levels that arelikely to affect treatment outcomes. Onepractical means of obtaining preliminary para-metric data is through translational research.

Translational research typically begins withcontrolled laboratory studies that are laterreplicated with clinical populations (Lerman,2003). Translational research may afford morecontrol over variables than is usually available inapplication, such as precise delivery of anteced-ent stimuli and reinforcers. Isolation of partic-ular variables may illuminate the most influen-tial factors, which can later be examined inapplication. Translational research may alsoallow more rapid manipulation of variablesthan is typically afforded with clinical popula-tions, for whom the influence of extraneousvariables such as therapist differences, limitedparticipant availability, or the participants’histories may affect the outcome of studies,particularly those that use relatively briefexperimental phases.

The purpose of the current investigation wasto study the effects of treatment integrity

failures on DRA, using a translational researchmodel. Experiment 1 used a laboratory proce-dure to examine the effects of errors of omissionalone, errors of commission alone, and com-bined errors of omission and commissionduring an analogue DRA treatment, at fivelevels of failure and with different sequences ofexposure. Experiments 2 and 3 replicated thecombined errors examined in Experiment 1with children with developmental disabilities.

EXPERIMENT 1: INITIAL EVALUATIONOF INTEGRITY FAILURES

Method

Participants and setting. Twenty-two under-graduate students enrolled in an introductorypsychology course at the University of Floridaparticipated. Students received course credit forcompleting the experiment, but this credit wasnot dependent on performance during theexperimental sessions. Each session involvedonly 1 participant. We conducted all sessions ina laboratory room that was equipped with acomputer desk, a computer, and a chair.Students participated for a total of 123 min,which included two 60-min blocks with a 3-min break between blocks.

Procedure. When the student arrived, weasked him or her to read and sign a consentform that stated that the experiment assessedthe effects of different contingencies of rein-forcement, but it did not specify the reinforce-ment schedules. The experimenter told theparticipant that he or she would be working at acomputer and should use only the mouse toearn as many points as possible during thesession.

During the sessions, the computer screen wasblank except for one red circle, one black circle,and a cumulative point score. The circles were127 mm in diameter and moved at a speed of25 mm per second in random directions.Participants earned points according to theprogrammed schedules of reinforcement, whichdiffered for both circles. We arbitrarily defined

TREATMENT INTEGRITY 49

clicking on the black circle as analogous toengaging in problem behavior and clicking onthe red circle as analogous to engaging inappropriate behavior. The background of thescreen was beige throughout all sessions;schedule-correlated stimuli were not used.

During baseline, clicking on the black circlewas reinforced on an FR 1 schedule and clickingon the red circle was not reinforced (extinction).During the full-integrity DRA, clicking on theblack circle was not reinforced (extinction) andclicking on the red circle was reinforced on anFR 1 schedule. The baseline and full-integrityDRA schedules were designed to replicate thosepublished in the majority of studies that haveevaluated DRA as a treatment for problembehavior (e.g., Shirley et al., 1997; Vollmer,Iwata, Smith, & Rodgers, 1992; Vollmer et al.,1999; Walsh, 1991).

During treatment integrity failures, wereinforced one or both of the available responsesaccording to random-ratio (RR) schedules. AnRR schedule is a type of VR schedule in whicheach response is associated with a particularprobability of reinforcement. For example, inan RR 10 schedule, each response would have a.1 probability of resulting in reinforcer delivery.In the current experiments, each level oftreatment integrity failure was associated witha particular probability. For example, 80%integrity with both omission and commissionerrors (hereafter, combined errors) was associ-ated with a .8 probability of reinforcement forappropriate behavior (80% of responses beingreinforced; an RR 1.25 schedule) and a .2probability of reinforcement for problem be-havior (80% of responses going unreinforced;an RR 5 schedule). Although the use of RRschedules is only one possible type of integrityfailure, the use of ratio-based schedules wasconsistent with prior research on treatmentintegrity failures during DRA (Vollmer et al.,1999; Worsdell et al., 2000). The programmedlevels of treatment integrity failures are de-scribed below, but in some cases, the pro-

grammed levels of treatment integrity differedslightly from the obtained probabilities ofreinforcement as a function of participantresponse allocation. We calculated obtainedprobabilities of reinforcement for each of theprogrammed levels by dividing the totalnumber of reinforcers delivered contingent ona response by the number of responses in thatphase. Programmed and obtained probabilitiesdiffered by a mean of .01 (range, 0 to .03)across all subsets and phases of the experiment(obtained probability values for each phase areavailable from the first author).

At the beginning of the experiment, theexperimenter instructed participants to earn asmany points as possible. Points were notexchangeable for any backup reinforcers. Clearreinforcement effects and appropriate changesin rates of responding following contingencychanges between at least one replication ofbaseline (FR 1 extinction schedule) and treat-ment (DRA schedule with perfect integrity,extinction FR 1) were required for a participantto be included in the experiment. Participantswho did not show differentiation in responserates between baseline and DRA phases wouldhave been excluded; however, all participantsmet the inclusion criteria.

Participants were assigned randomly to oneof four subsets (described below) that varied inthe type of integrity manipulation. For three ofthe subsets, condition sequence remainedconstant, regardless of subset assignment, tocontrol for the influence of sequence effects. Allparticipants were exposed to baseline, full-integrity DRA, and four phases of reducedtreatment integrity (80%, 60%, 40%, and20%), using a reversal design. The sequenceof conditions for each subset is shown inTable 1.

Participants in Subset 1 were exposed only toomission errors; for these participants, reducedlevels of treatment integrity meant that theschedule of reinforcement for appropriatebehavior became thinner (e.g., probability


decreased from 1.0 to .2). As treatment integritylevels decreased, these participants could earnfewer reinforcers for engaging in appropriatebehavior, but could never earn points forengaging in problem behavior (extinction).The purpose of this subset was to examine theeffects of missed reinforcer deliveries forappropriate behavior alone, without introduc-tion of reinforcement for problem behavior.

Participants in Subset 2 were exposed tocommission errors only. For these participants,reduced levels of treatment integrity meant thatsome reinforcers were introduced into theextinction schedule for problem behavior. Thatis, as treatment integrity levels became lower,the reinforcement schedule for problem behav-ior became richer, but appropriate behavior wasalways reinforced on an FR 1 schedule. Thepurpose of this subset was to examine the effectsof introducing accidental reinforcement forproblem behavior.

Participants in Subset 3 were exposed tocombined errors of omission and commission.For these participants, as fewer reinforcers wereavailable for problem behavior (fewer commis-sion errors), more reinforcers were available forappropriate behavior (fewer omission errors).The purpose of this subset was to examine thecombined effects of missed reinforcer deliveriesfor appropriate behavior and reinforcement forproblem behavior.

Participants in Subset 4 experienced only onelevel of treatment integrity failure (50%integrity on both the reinforcement andextinction components), but the conditionsequence was altered such that treatmentintegrity failures followed baseline twice andfull-integrity DRA twice. We designed thismanipulation to address the possibility thatcondition sequence affects the outcome oftreatment integrity failures.

Results and Discussion

Figure 1 shows the results for participants inSubset 1 (omission errors only). For all 3participants in this group, rates of problembehavior were high and rates of appropriatebehavior were low during baseline. Theyengaged in high rates of appropriate behaviorand low rates of problem behavior when weimplemented DRA with full integrity. Duringthe first exposure to treatment integrity failures,participants usually engaged in appropriatebehavior at somewhat lower rates when thetreatment integrity decreased to 40% or 20%.In general, rates of appropriate respondingremained high during the second exposure tothe integrity failure phases, with the exceptionof the final failure phase (80% integrity).Regardless of the level of errors of omission inplace, participants in Subset 1 did not engage inelevated rates of problem behavior. This finding

Table 1

Condition Sequence for Experiment 1

Condition type

Condition number Subset 1 Subset 2 Subset 3 Subset 4

1 BL BL BL BL2 DRA DRA DRA DRA3 80 omission 80 commission 80 combined BL4 60 omission 60 commission 60 combined DRA5 40 omission 40 commission 40 combined 506 20 omission 20 commission 20 combined BL7 BL BL BL 508 DRA DRA DRA DRA9 20 omission 20 commission 20 combined 50

10 40 omission 40 commission 40 combined BL11 60 omission 60 commission 60 combined 5012 80 omission 80 commission 80 combined DRA


suggests that errors of omission in isolation maynot be highly detrimental to DRA treatments.In fact, the low levels of appropriate behaviormay be desirable in some cases (cf. Hanley,Iwata, & Thompson, 2001).

Figure 2 shows results for participants inSubset 2 (commission errors only). For theseparticipants, commission errors did not becomedetrimental until the level of treatment integritydropped to 40% or lower. For Participants 206(upper right panel) and 222 (lower left panel),high rates of appropriate behavior and low ratesof problem behavior were observed during DRAat full integrity and during 80% and 60%treatment integrity. However, rates of appro-

priate behavior decreased, with a correspondingincrease in problem behavior at 40% and 20%integrity. The increase in problem behaviorduring 40% and 20% integrity was somewhatsurprising. Because participants could consis-tently earn a point for each appropriateresponse, it seems counterintuitive that theywould allocate responding to a thinner rein-forcement schedule (i.e., the treatment integrityfailures for problem behavior). Overall, howev-er, the DRA treatment seemed relatively robustwhen treatment integrity was greater than 60%with commission errors.

Figure 3 shows results from participants inSubset 3. For these participants, errors of

Figure 1. Results for participants in Subset 1, who were exposed to omission errors only. Each panel shows results fora participant. Filled circles denote rates of clicking on the black object (problem behavior), and open circles denote ratesof clicking on the red object (appropriate behavior). Condition labels show baseline (BL), DRA, and treatment integrityfailure phases. During integrity failures, condition labels show treatment integrity as omission integrity orcommission integrity.


omission and commission covaried. For exam-ple, during the 20% integrity conditions, wedegraded integrity on both the FR 1 andextinction components of the DRA; thisresulted in a 20% chance that appropriatebehavior would result in a point and an 80%chance that problem behavior would result in apoint. Results were consistent across the 3participants in Subset 3. They engaged inhigh rates of problem behavior and lowrates of appropriate behavior when treatmentintegrity was 20% or 40%. However, oncetreatment integrity reached 60%, responseallocations switched, with participants engagingin more appropriate behavior than problembehavior. Condition sequence did not affectthese results.

The results obtained from Subset 3 could beexplained based on the reinforcement rateavailable from each response type. Whenproblem behavior was more likely to result inreinforcement than was appropriate behavior(during the 20% and 40% integrity phases),participants engaged in more problem behaviorthan appropriate behavior. Conversely, whenappropriate behavior was more likely to resultin reinforcement than was problem behavior(during the 60% and 80% integrity phases),they were more likely to engage in appropriatebehavior than problem behavior.

Figures 4 and 5 show the results fromparticipants in Subset 4. In general, twopatterns of responding occurred across the 13participants: Responding during the error phase

Figure 2. Results for participants in Subset 2, who were exposed to commission errors only. Each panel shows resultsfor a participant. Filled circles denote rates of clicking on the black object (problem behavior), and open circles denoterates of clicking on the red object (appropriate behavior). Condition labels show baseline (BL), DRA, and treatmentintegrity failure phases. During integrity failures, condition labels show treatment integrity as omission integrity orcommission integrity.


carried over from the previous phase (i.e.,participants continued to allocate respondingtoward the most recently reinforced response;Figure 4) or responding was almost completelyallocated to the first reinforced response(Figure 5). Figure 4 shows the results forparticipants whose response allocations duringbaseline and DRA carried over into thetreatment integrity phases. For these 5 partic-ipants, the contingencies in place during thepreceding phase (baseline or full-integrity DRA)influenced responding during the subsequent50% integrity phases. For example, all of theseparticipants engaged in higher rates of problembehavior during the 50% integrity conditionsthat followed baseline and usually displayed

higher levels of appropriate behavior during the50% integrity condition that followed the full-integrity DRA.

The remaining 8 participants in Subset 4 didnot show consistent carryover during the 50%integrity failure conditions. Figure 5 shows theresults for these participants. Carryover oc-curred in some phases and seemed particularlylikely to occur in the first exposure to theintegrity failure condition following DRA (inwhich carryover occurred for 7 of 8 partici-pants). However, as the experiment progressed,these participants responded in an unexpectedmanner during the integrity failures: Responseallocation completely switched from the previ-ous phase. In other words, they primarily

Figure 3. Results for participants in Subset 3, who were exposed to both omission and commission errors. Eachpanel shows results for a participant. Filled circles denote rates of clicking on the black object (problem behavior), andopen circles denote rates of clicking on the red object (appropriate behavior). Condition labels show baseline (BL), DRA,and treatment integrity failure phases. During integrity failures, condition labels show treatment integrity as omissionintegrity or commission integrity.


engaged in appropriate behavior during 50%integrity phases that followed baseline and inproblem behavior during 50% integrity phasesthat followed DRA. This change in allocation

occurred for the second, third, and fourthexposures to 50% integrity for Participants 001and 007, the first and fourth exposures for 019,the second and third exposures for 005, the

Figure 4. Partial results for participants in Subset 4, who were exposed to 50% integrity following baseline and fulltreatment. Each panel shows results for a participant. All of these participants showed consistent carryover betweenbaseline or treatment and error phases. Filled circles denote rates of clicking on the black object (problem behavior), andopen circles denote rates of clicking on the red object (appropriate behavior). Condition labels show baseline (BL), DRA,and treatment integrity failure phases. During integrity failures, condition labels show treatment integrity as omissionintegrity or commission integrity.


Figure 5. Results for some of the participants in Subset 4, who were exposed to 50% integrity following baseline and fulltreatment. Each panel shows results for a participant. All of these participants showed some evidence of switching betweenbaseline or treatment and error phases. Filled circles denote rates of clicking on the black object (problem behavior), and opencircles denote rates of clicking on the red object (appropriate behavior). Condition labels show baseline (BL), DRA, andtreatment integrity failure phases. During integrity failures, condition labels show the treatment integrity percentage.


second and fourth exposures for 013 and 020,and the fourth exposure for 003 and 004. Inalmost all of these conditions, the participantengaged in high rates of one response and lowrates of the other response throughout the phase.

The changes in response allocation may havebeen due to the reinforcement schedules that weused during the baseline and DRA conditions.As mentioned previously, we chose to use FR 1and extinction schedules in these conditionsbecause those schedules were used most com-monly in previous DRA research. However, it ispossible that the continuous reinforcementschedule exerted sufficient control over re-sponding, such that the introduction of the50% integrity condition signaled participants toalter response allocations. For example, it couldbe that once the condition changed from DRAto 50% integrity, the first unreinforced appro-priate response resulted in switching to problembehavior. Alternatively, it is possible that thefirst reinforcer delivery for one response resultedin a change in response allocation to thatresponse (i.e., reinforcer delivery altered re-sponse allocation).

The switching of response allocation ob-served for those participants in Subset 4 alsomay be due to the experimental arrangement.Because the ideal response pattern was alterna-tion between problem behavior and appropriatebehavior every 10 min for the first four phases,it is possible that participants generated a ruleabout responding, such as ‘‘switch every10 min.’’ Although they did not have anyexteroceptive means of timing the conditions, itis possible that the passage of time exertedcontrol over responding.

In sum, the results from Experiment 1demonstrated that the efficacy of DRA treat-ments decreased based on different kinds oftreatment integrity failures. Errors of commis-sion had a greater impact on responding thandid errors of omission, but only at relatively lowlevels of treatment integrity (20% and 40%).Thus, the DRA arrangements presented in this

experiment appeared to be a relatively robust‘‘treatment’’ overall, in that generally lowerlevels of integrity continued to support treat-ment efficacy (decreased levels of problembehavior and maintenance of appropriatebehavior). This outcome may be becauseparticipants who experienced a DRA treatmentwith errors of commission only can maximizereinforcement by continuing to respond on thericher reinforcement schedule (i.e., the FR 1schedule for appropriate behavior).

Combined errors seem to have the sameeffects on behavior as commission errors only.This suggests that, when both types of failuresare combined into a single measure, commis-sion errors may be more responsible fordetrimental effects than omission errors. Toillustrate, at 50% combined treatment integrityfailures, both the reinforcement schedules ineffect and the participant’s recent reinforcementhistory influenced problem behavior. This isevidenced by the differences in respondingduring the 50% integrity phases that followedDRA implemented with full integrity and thosethat followed baseline (Figure 4).

Combined errors may have detrimentaleffects because participants attempt to maximizereinforcement. As mentioned previously, theparticipants exposed to combined errors (Subset3) allocated the majority of their responses tothe richer reinforcement schedule. This resultmay have important implications for applica-tion because caregivers may be likely to makecombined errors (commission and omission)when attempting to implement DRA treat-ments. For example, caregivers with a longhistory of reinforcing problem behavior may beprone to reinforcing problem behavior on arelatively rich schedule and appropriate behav-ior on a relatively lean schedule, similar to thelower levels of treatment integrity experiencedby Subset 3. If this is the case, those caregiversmay see little to no improvement in thebehavior, despite their attempts to implementthe treatment as it was designed.


Results from Subset 4 highlight the possibil-ity that treatment integrity failures, particularlythose that do not clearly favor one response overanother, may be particularly detrimental ifimplemented following baseline probabilitiesof reinforcement. For example, if a caregiverimplements the DRA with less than optimal(e.g., 50%) integrity following training, theremay be no improvement in the problembehavior, at least initially. Alternatively, if thecaregiver can create a reinforcement history foralternative behavior initially by implementingDRA with high integrity, later treatmentintegrity failures may not be as detrimental.Recent reinforcement history influences theimpact of integrity failures, thereby emphasiz-ing the importance of initially implementingthe DRA with high integrity.

Although the results of Experiment 1 haveimplications for application, the generality ofthese results was unclear. In particular, weobtained the results with a nonclinical popula-tion responding on an analogue task, whichmay not have generality to the treatment ofproblem behavior displayed by individuals withdisabilities. Thus, Experiments 2 and 3 extend-ed the results of Experiment 1 to clinicalapplication.

EXPERIMENT 2: APPLIED EVALUATIONOF COMBINED ERRORS

The purpose of Experiment 2 was to evaluatethe effects of combined errors on the occurrenceof problem and appropriate behavior duringDRA, in an attempt to replicate the results ofSubset 3 from Experiment 1. Thus, we designedthis experiment as one means of validating theresults from the human operant laboratory. Wechose to replicate the results of Subset 3 becauseit seemed probable that naturally occurringintegrity failures during DRA included bothcommission and omission errors and because ofthe detrimental effects of these errors on DRAefficacy observed during the combined integrityerrors in Experiment 1.

Method

Participant and setting. Helena was a fourth-grade student who had been diagnosed withautism. She spoke in complete sentences,independently requested items and activities,and participated in a regular education class-room for the majority of her school day.Helena’s teacher had referred her to a school-based program for the assessment and treatmentof off-task behavior (defined below).

All sessions were conducted in an emptytherapy room (4 m by 4 m) in Helena’s schoolthat was equipped with a table, chairs, andleisure items appropriate for a variety ofdifferent ages and skill levels. Sessions wereconducted 2 or 3 days per week, with two tofour sessions per day. All sessions were 5 min induration.

Data collection and interobserver agreement.Data for all sessions were collected usinghandheld computers. Observers were under-graduate or graduate students in behavioranalysis who had been trained to masterycriterion previously for data collection byattaining interobserver agreement scores of90% or above for three consecutive sessionswith a previously trained observer. Observerstypically sat in the corner of the therapy room,about 1 m away from Helena and the therapist.

Trained observers collected data on studentand therapist behavior. Student responsesincluded off-task behavior, on-task behavior,and task completion. We defined off-taskbehavior as engaging in an alternative activity(not related to the task). Examples of off-taskbehavior included putting her head on her desk,playing with pencils or a pencil box, or drawing.We defined on-task behavior as having herpencil in her hand and her eyes oriented towarda worksheet. Both off-task and on-task behaviorwere scored as duration measures. Observersalso collected frequency data on the number oftasks that Helena completed as a secondarymeasure of on-task behavior. In addition tothese responses, observers collected data on


attention delivery from the therapist (thetherapist looked at Helena and made a vocalstatement) that was also scored as a durationmeasure.

A second observer simultaneously and inde-pendently collected data during 43% of rein-forcer assessment sessions and 49% of treatmentevaluation sessions. Interobserver agreementwas calculated by dividing each session into10-s bins and comparing the number ofresponses scored within each bin across observ-ers by dividing the smaller number of responsesin that bin by the larger number of responsesand converting the ratio to a percentage (Shirleyet al., 1997). The percentages were thenaveraged across all bins in the session to yieldan overall percentage. During the reinforcerassessment, mean interobserver agreement was100% for on-task behavior, 100% for off-taskbehavior, and 82% (range, 74% to 87%) fortherapist attention. During the treatmentevaluation, mean agreement was 92% (range,73% to 100%) for on-task behavior, 94%(range, 80% to 100%) for off-task behavior,and 83% (range, 63% to 100%) for therapistattention.

Observers also collected data on treatmentintegrity, which we defined as the degree towhich the therapist implemented the contin-gencies as designed. During the phases in whichtreatment integrity was manipulated deliberate-ly, integrity scores were based on the therapist’sadherence to a randomized computer output, asdescribed below. Treatment integrity for allsessions was 100%.

Reinforcer assessment. The therapist conducteda reinforcer assessment to determine possiblereinforcers for Helena’s behavior. During thereinforcer assessment, the therapist providedHelena with two identical worksheets thatdiffered only in that one had the word BREAKprinted across the top, and the other had theword TALK printed across the top. Bothworksheets were available concurrently through-out all reinforcer assessment sessions. When

Helena oriented toward the break worksheet,the therapist prompted her every 5 s tocomplete a task. Completion of a task on thebreak worksheet resulted in the removal of bothworksheets for 15 s. When Helena orientedtoward the talk worksheet, the therapist avoidedeye contact with her and did not speak until shecompleted a task. Completion of a task on thetalk worksheet resulted in eye contact andpraise, lasting approximately 15 s. If Helena’sbehavior was influenced by escape, we expectedher to either work on the break worksheet (toobtain programmed breaks) or to orient towardthe talk worksheet but not complete any tasks(to maintain therapist silence and lack ofprompts); in other words, either of theseresponses would result in the termination ofeither demands or therapist attention. IfHelena’s behavior was influenced by attention,we expected her to either complete tasks on thetalk worksheet (to maintain access to attentionin the form of praise) or to orient toward thebreak worksheet but not complete any tasks (tomaintain access to attention in the form ofprompts). Thus, both worksheets were associ-ated with potential positive and negativereinforcement contingencies.

The therapist conducted forced exposuretrials before the start of the reinforcer assess-ment. During the forced exposure, only oneworksheet was available. The therapist vocallyprompted Helena to complete a task on thatworksheet and provided the associated conse-quence. The therapist provided one exposure toeach consequence before the start of thereinforcer assessment.

Before the start of each session, the therapistinstructed Helena that she could work onwhichever worksheet she wanted and couldswitch between sheets at any time. We initiallyselected a double-digit addition task as theworksheet content. After four sessions using thedouble-digit addition task, we changed the taskto writing vocabulary words to determine thegenerality of the reinforcer assessment findings


across tasks. Vocabulary tasks were used only forthe reinforcer assessment; double-digit additionwas used for the treatment integrity evaluation.

Treatment integrity evaluation. To makeHelena’s duration-based measures more similarto the rate measures in Experiment 1, weconsidered each consecutive 15 s that Helenaspent on task or off task as a response. Duringbaseline sessions, the therapist attended toHelena in each 15-s interval that she was offtask and ignored her on-task behavior. Atten-tion during baseline took the form of coaxingher to work, including saying statements like,‘‘It’s math time, Helena,’’ ‘‘You have a mathworksheet and a pencil,’’ ‘‘Don’t you think youshould work on your math?,’’ and ‘‘You won’tget your math done if you keep playing.’’ Thesestatements were similar in form to the typicalreaction of Helena’s teachers when she engagedin off-task behavior in the classroom.

During DRA phases, the therapist attendedto Helena when she was on task for 15 s andignored off-task behavior. The form of atten-tion during DRA was changed to praisestatements and comments about Helena’s work,like ‘‘great working,’’ ‘‘You got that one right!,’’and ‘‘You’re so smart.’’ Again, these statementswere similar to the praise typically delivered inHelena’s classroom.

We evaluated four levels of combinedomission and commission errors: 80%, 60%,40%, and 20% integrity. These levels and theprogramming of the integrity failures weresimilar to Subset 3 of Experiment 1 (usingRR schedules). For example, 80% of on-taskbehavior and 20% of off-task behavior resultedin attention during 80% integrity phases.During all integrity failure phases, the therapistheld a clipboard with a computer-generatedsequence of which responses should result inreinforcement. The form of attention wassimilar to that provided in baseline and DRAconditions (i.e., coaxing and praise followingoff- and on-task behavior, respectively). Wecounterbalanced the order of the integrity

failure phases across replications in a reversaldesign.


Helena selected the talk worksheet during96% of the reinforcer assessment (range, 70%to 100%; results available from the first author),suggesting that praise served as a reinforcer foron-task behavior. Anecdotally, Helena beganscratching out BREAK at the top of the breakworksheet and writing TALK on that worksheetbefore the third session. This did not affect theconsequences that the therapist provided con-tingent on working on the two sheets, butprovided additional evidence that attention wasa reinforcer for Helena’s behavior. She chose towork on the talk worksheet regardless of thespecific task (double-digit addition or copyingvocabulary words).

Figure 6 shows the results of Helena’streatment integrity analysis. The top panelshows the percentage of the session time thatHelena spent on or off task. The bottom panelshows the rate at which she completed double-digit addition problems. She initially engaged inon-task behavior. Over the course of thebaseline, however, on-task responding decreasedand off-task responding increased; by the lasttwo sessions in baseline, Helena was engagingexclusively in off-task behavior. During the firstexposure to DRA implemented with fullintegrity, off-task behavior decreased and on-task behavior increased.

Helena’s responding seemed to be affected bythe degree of the treatment integrity failureduring the integrity failure phases. She engagedin more on-task than off-task behavior duringthe 80% and 60% failure phases. However, sheallocated more time to off-task behavior thanon-task behavior during the 40% and 20%integrity conditions. These results replicatethose obtained in Experiment 1, which alsoshowed that DRA lost its efficacy when imple-mented at less than 50% integrity with combinedomission and commission errors. The conditionsequence did not influence Helena’s behavior


Figure 6. Treatment integrity assessment results for Helena. The top panel shows percentage of the session spent onor off task by the open and filled circles, respectively. Condition labels show baseline (BL), DRA, and treatment integrityfailure phases. During integrity failures, condition labels show the treatment integrity percentage. The bottom panelshows the rate at which Helena completed math problems during the assessment.


strongly during the integrity failure phases,insofar as her behavior during the replicationsmatched the results obtained from the initialexposures. Taken together, these results suggestthat when the reinforcement schedules select for aparticular response, the sequence of conditions isless important (as was the case with Helena);however, when the schedules do not select for aparticular response (as in the case of 50%integrity from Experiment 1), sequence maybecome a key factor in response allocation.

The secondary measure of on-task behavior(the number of double-digit addition problemscompleted; Figure 6, bottom) supports theanalysis based on time allocation. Helenacompleted a mean of 2.8 and 3.0 tasks perminute during the 80% and 60% integrityfailure phases, respectively. When treatmentintegrity decreased to 40%, she completed amean of 2.1 tasks per minute. When treatmentintegrity was 20%, she completed a mean ofonly 0.9 tasks per minute, equal to the meanrate of responding during baseline.

The results of Experiment 2 suggest thattreatment integrity failures may be detrimentalto DRA interventions if the integrity failuresoccur often. However, DRA seems to beresistant to detrimental effects of integrityfailures as a whole, in that treatment integritydecreased to 40% before detrimental effectsoccurred. Yet the results from Subset 4 ofExperiment 1 suggested that condition sequencemay play an important role when DRA wasimplemented with 50% integrity. Thus, thepurpose of Experiment 3 was to examine thepossible influence of sequence effects onintegrity failures by replicating the results ofSubset 4 from Experiment 1 in a school settingwith a child with developmental disabilities.

EXPERIMENT 3: APPLIED EVALUATIONOF SEQUENCE EFFECTS

MethodParticipant and setting. Jake was an adolescent

who had been classified as trainable mentally

handicapped; he was enrolled in a center schoolfor children with disabilities. Jake’s schooladministrator referred him to a school-basedtreatment program for the assessment andtreatment of aggression (described below). Jakecommunicated using gestures or single-wordvocal utterances and spent his school day in aspecial education classroom for students withmoderate mental impairment. He was able tofollow one-step directions and complete avariety of self-care tasks independently or withminimal prompting.

We conducted all sessions in an emptyclassroom (approximately 6 m by 10 m) inJake’s school. The classroom was equipped withtables, chairs, and leisure items appropriate for avariety of different ages and skill levels. Weconducted sessions 2 to 4 days per week, withthree to five sessions per day. All sessions were5 min in duration.

Data collection and interobserver agreement.The data-collection system and interobserveragreement arrangements were identical toExperiment 2. Student responses includedaggression (physical contact between Jake’sopen hand and the therapist’s body) andgreetings (i.e., saying, ‘‘hi’’). Observers scoredaggression and greetings as frequency measures.Therapist responses included attention delivery(the therapist looked at Jake and made a vocalstatement) and escape delivery (the therapistsaid ‘‘take a break,’’ removed instructionalmaterials, and turned away from Jake).

A second observer simultaneously and inde-pendently collected data during 32% of func-tional analysis sessions and 44% of treatmentevaluation sessions. Interobserver agreementwas calculated as described for Experiment 2.During the functional analysis, mean interob-server agreement was 98% (range, 75% to100%) for aggression, 100% for greetings, and97% (range, 81% to 100%) for therapistbehavior. During the treatment evaluation,mean agreement was 93% (range, 60% to100%) for aggression, 92% (range, 64% to


100%) for greetings, and 84% (range, 72% to95%) for therapist behavior. Treatment integ-rity, as defined for Experiment 2, was 100% forall sessions except Session 13, in which thetherapist inadvertently reinforced one instanceof problem behavior.

Functional analysis. We conducted a func-tional analysis to determine possible reinforcersfor Jake’s aggression. Initial functional analysissessions were similar to those described byIwata, Dorsey, Slifer, Bauman, and Richman(1982/1994) and consisted of play, attention,and escape conditions. During the play sessions,Jake had access to leisure items and continuoustherapist attention. During attention sessions,he had access to leisure items, but the therapistignored him until he engaged in aggression.Contingent on aggression, the therapist provid-ed a brief reprimand, such as ‘‘Don’t hit me,Jake.’’ During escape sessions, the therapistasked Jake to engage in an academic task, suchas letter sorting, using a three-prompt instruc-tional sequence (verbal prompt, model, physicalguidance, with 5 s between prompts). Contin-gent on aggression, the therapist allowed Jake totake a 30-s break from the task, signaled by thetherapist saying ‘‘take a break’’ and removingthe instructional materials.

When low or decreasing rates of aggressionoccurred in the initial sessions, additionalconditions were included in the functionalanalysis. For example, Jake’s teacher reportedthat he was most likely to engage in aggressionwhen she was talking with another person orduring transitions, when people were frequentlyin close physical proximity to Jake. Based onthese reports and classroom observations, threeadditional functional analysis conditions wereincluded: neutral attention, proximity, anddiverted attention. The neutral attention con-dition was identical to the attention conditiondescribed above, except that the form ofattention was changed from a reprimand to aneutral statement, such as ‘‘What’s going on,Jake?’’ During the proximity condition, the

therapist sat within 0.5 m of Jake and movedaway from him contingent on aggression.During the diverted attention condition, twotherapists talked with each other. Contingenton aggression, one therapist ceased conversationwith the other therapist, turned to Jake, andmade a brief neutral comment, similar to thoseused in the neutral attention condition.

Treatment integrity evaluation. The baselineconditions were similar to the diverted attentionfunctional analysis condition. Two therapists satacross the table from each other, with onetherapist seated next to Jake (within 1 m). Thetherapists talked with each other and ignoredJake until he engaged in aggression. Contingenton each aggression response, the therapistsstopped talking, and one therapist turned toJake and made a brief neutral comment. Thetherapists did not reinforce greetings.

We conducted an evaluation of DRA afterthe conclusion of baseline. Before the first DRAsession, the therapist prompted Jake to greet herby saying, ‘‘Jake, if you want to talk to me, say‘hi.’’’ When Jake said ‘‘hi,’’ the therapist turnedto him and made a brief neutral comment. Thetherapist repeated this prompting procedure 10times before the start of the initial DRA phase.No additional prompts were used after theseinitial 10 trials. During DRA sessions, twotherapists sat talking with each other andignored aggression (extinction). When Jake said‘‘hi,’’ one therapist turned to him and made abrief neutral comment, such as ‘‘What are youlooking at, Jake?’’

We used a 50% integrity condition toexamine the effects of sequence effects onresponding during treatment integrity failures,which was similar to the treatment integrityfailures experienced by Subset 4 of Experiment1. During this condition, 50% of aggressionresponses and 50% of greetings resulted in briefattention, according to RR 2 schedules. Twotherapists talked with each other and ignoredJake. The therapist seated across the table fromJake held a clipboard with a computer-generat-


Figure 7. Results of the functional analysis (top) and treatment integrity evaluation (bottom) conducted with Jake.Responses per minute of Jake’s behavior are shown along the y axis, with sessions shown on the x axis for both panels. Inthe bottom panel, condition labels show baseline (BL), DRA, and treatment integrity failure phases. During integrityfailures, condition labels show the treatment integrity as a percentage. Filled circles denote rates of aggression, and open


ed sequence of which responses should result inreinforcement and cued the therapist sittingclosest to Jake as to which response the therapistshould reinforce. Reinforcers consisted of brief,neutral statements, similar to those provided inthe baseline and DRA conditions. The 50%condition followed baseline twice and DRAtwice to assess possible sequence effects using areversal design.


Figure 7 shows the results of Jake’s function-al analysis and treatment integrity evaluation.Although rates of aggression were elevatedinitially in the play, attention, and escapeconditions, rates eventually decreased to near-zero levels across all functional analysis condi-tions (top). Following the introduction of thethree additional test conditions, Jake engaged inelevated rates of aggression exclusively in thediverted attention condition, suggesting thatadult attention served as a reinforcer foraggression, particularly when two adults weretalking to each other but were ignoring Jake.

Figure 7 (bottom) shows the results of Jake’sintegrity analysis. During all baseline phases,rates of aggression occurred at moderate to highrates (M 5 10.2 responses per minute), andgreetings occurred at low or decreasing rates (M5 2.2 responses per minute). During the firstexposure to DRA implemented with fullintegrity, rates of aggression increased initiallybefore decreasing to low levels, and rates ofgreetings increased to moderate, stable levels.During subsequent DRA phases that followedbaseline, aggression decreased to low rates (M 5

4.4 responses per minute across all DRAconditions), and appropriate behavior increasedto moderate rates (M 5 4.9 responses perminute across all DRA conditions).

During the 50% integrity phases thatfollowed DRA, a mixture of aggression andgreetings occurred, with some bias towardaggression. Jake engaged in a mean rate of 5.4aggression responses per minute and 3.9greetings per minute during these two phases.This result contrasts with previous studies (e.g.,Vollmer et al., 1999) that showed that DRAhad relatively robust effects when implementedat reduced levels of integrity following DRAwith full integrity. The bias toward problembehavior observed in the current experimentcould be due to Jake’s extraexperimental history(i.e., caregivers attending to aggression beforeexperimental sessions). Unfortunately, we wereunable to collect data to evaluate this possibility;future studies could investigate the extent towhich extraexperimental histories influenceresponding during integrity failures.

The results in the 50% integrity conditionsfollowing DRA differ from the results of the50% integrity conditions following baseline.During the integrity failures following baseline,rates of greetings remained low or near zero (M5 0.5 responses per minute), and rates ofaggression remained high and stable (M 5 12.4responses per minute). For Jake, integrityfailures (at least the 50% integrity conditionused in the current experiment) were moredetrimental to the treatment when they fol-lowed baseline than when they followedtreatment with perfect integrity. These resultssuggest that the initial implementation of thetreatment may affect responding during laterintegrity failures. Specifically, if caregiversinitially implement DRA with a high level ofintegrity, later integrity failures may not be asdetrimental. However, if a caregiver whopreviously provided a rich reinforcement sched-ule for problem behavior initially implements a

r

circles denote rates of greetings. The asterisk above Session 13 (bottom) denotes an accidental treatment integrity failure,in which the therapist reinforced one instance of aggression.


treatment with moderate or poor integrity,problem behavior is unlikely to improve, as inthe 50% integrity conditions following baselinein the current experiment. These findingsreplicate those of Subset 4 in Experiment 1.

Exposures to treatment integrity failures alsoseemed to exert influence on a later implemen-tation of DRA at 100% integrity. In particular,treatment effects during 100% integrity weremore difficult to regain following a 50%integrity phase than following baseline. Thiseffect can be seen in Sessions 79 to 102(Figure 7, bottom). These results suggest thatinitial implementation of a treatment at reducedlevels of integrity may impede treatment effectsduring later implementation of DRA with highlevels of integrity. Evaluation of these effectswas not a focus of this study, so we did notmanipulate condition sequence in a way thatpermitted conclusions about the consistency ofthis effect. However, the effect of integrityfailures on subsequent conditions should beevaluated in future research.

GENERAL DISCUSSION

The present collection of experiments exam-ined the effects of two types of treatmentintegrity failures (failure to reinforce appropri-ate behavior and reinforcing problem behavior,here termed omission and commission errors,respectively) on responding during DRA. Wefirst investigated the effects of treatmentintegrity failures in a human operant procedure,in which participants responded on an analoguetask (Experiment 1). Commission errors inisolation and omission and commission errorsin combination resulted in increases in arbi-trarily defined problem behavior and decreasesin appropriate behavior. In addition, conditionsequence played a role in the effects of integrityfailure phases (as evidenced by results obtainedfrom Subset 4). Next, we replicated the resultsof the combined errors phase with a studentdiagnosed with autism and with a studentlabeled as trainably mentally handicapped. In

both cases, the treatment-based replicationsreproduced results obtained in the humanoperant laboratory. Specifically, errors of com-mission and omission decreased treatmentefficacy, and problem behavior was more likelyto recur when treatment integrity failuresoccurred following baseline conditions thanwhen otherwise identical integrity failuresoccurred following full treatment conditions.

Overall, DRA procedures seem to haverelatively robust effects despite the occurrenceof treatment integrity failures. Errors of omis-sion in isolation did not affect treatmentoutcome detrimentally, regardless of the levelof treatment integrity (Subset 1, Figure 1).However, participants engaged in somewhatlower rates of appropriate behavior during thefirst exposure to 40% and 20% integrity. Thisdecrease in response rates as the density of thereinforcement schedule was reduced was some-what unusual and should be examined further.

Detrimental effects of commission errors(Subset 2, Figure 2) and combined error types(Subset 3, Figure 3) occurred only whenintegrity levels decreased to 40% or below.When 50% integrity was evaluated (Subset 4,Figure 4), the detrimental effects were attenu-ated when the error phase followed a perfectlyimplemented DRA. The relatively robust effectssuggest that DRA may yield favorable treatmentoutcomes even when implemented with de-creased integrity.

The present experiments extend prior re-search by examining the effects of bothomission and commission errors on differentialreinforcement procedures and by providing anexample of translational research spanning thebasic-to-applied continuum. Prior research ontreatment integrity failures in DRA treatmentsshowed that the treatment generally was robust,but that sequence effects could be a problem(Vollmer et al., 1999). In the current research,treatment integrity failures seemed to havefewer detrimental effects when the level ofintegrity descended (the sequence of integrity


failures from 80% to 20%) than when levelsgradually ascended (the sequence of integrityfailures from 20% to 80%). These effects can beseen with Participants 205, 222, 307, and 319.For each of these participants, rates of problembehavior were greater in the 40% and 20%integrity conditions during the ascendingsequence than during the descending sequence.Unfortunately, the order of sequence presenta-tion is confounded because we exposed allparticipants to the ascending sequence beforethe descending sequence. Results for partici-pants in Subset 4 provide additional support forthe idea that sequence plays a role in treatmentintegrity failure phases. In general, treatmentintegrity failures appeared to have less of animpact on treatment efficacy when failurephases followed DRA implemented with fullintegrity than when they followed baseline.

In addition to the results relating directly totreatment integrity failures, the current studiesalso have implications for the utility of humanoperant methods in examining problems ofapplied significance. In Experiment 1, wecompleted each participant’s data set in just over2 hr, resulting in rapid generation of completedata sets. Although limited by brief exposures tothe contingencies, the results from the presentexperiments suggested which variables influ-enced responding and which do not. Using asimilar approach, the most influential variablescan be evaluated further using more extendedexposures either in the human operant laboratoryor in application (as in Experiments 2 and 3).The use of human operant methods in thecurrent study informed Experiment 2 by sug-gesting that further information on the effects ofcombined errors, but not omission errors alone,would be likely to help inform treatmentdevelopment. Thus, when replicating the resultsof Experiment 1, we focused on combined errortypes instead of omission errors. In addition, thereproducibility of the effects obtained in Exper-iment 1 permitted replication using only 1participant during each of Experiments 2 and 3.

Finally, human operant procedures permitresearchers to identify likely functional relationswithout the research constraints associated withapplied contexts (e.g., time restrictions ordifficulties associated with persistence or delib-erate exacerbation of problem behavior). Thisfreedom may allow researchers to identifyrapidly which variables may affect appliedoutcomes and further explore the mechanismsresponsible for behavior change during treat-ment procedures. The current experimentsprovide a model for this type of translationalresearch method. In these experiments, weobtained similar results across both humanoperant and applied procedures. To illustrate,the results from Experiment 1 were used toinform subsequent applied replications, whichfocused only on variables that influencedbehavior in the human operant laboratory.Similarly, Experiment 2 examined only com-bined treatment integrity failures (as opposed tothe three different types of failures examined inExperiment 1), in that combined errors weredetrimental to treatment effects during thehuman operant evaluation. In addition, theresults of Experiment 1 suggested that sequenceeffects might be most pronounced during 50%integrity conditions. This result shaped theprocedures for Subset 4 and for Experiment 3,both of which demonstrated effects of sequenceduring 50% integrity. In general, the humanoperant results allowed a more informed,targeted evaluation in clinical contexts thanwould have been otherwise possible.

There are limitations associated with the useof an analogue procedure in the human operantlaboratory. First, Experiment 1 involved collegestudents as participants, which may limit thegenerality of the results. Despite this limitation,the results of Experiment 1 seemed to generalizeto participants with disabilities (based on theresults obtained in Experiments 2 and 3).Second, the response chosen, a mouse click,was a simple response that existed in allparticipants’ repertoires. Although we did not


specifically instruct participants to click themouse buttons, we told them to ‘‘use only themouse to earn as many points as possible.’’ Thereinforcement contingencies in place, in con-junction with this instruction, frequently re-sulted in very high response rates, which oftenexceeded 200 responses per minute. These ratesare well above those typically observed whentreating problem behavior and may haveinfluenced the outcome of the human operantexperiments. This limitation could be addressedin future research by changing the form of theresponse to a topography that more closelyapproximates problem behavior (e.g., increasingthe effort or duration of the target response).

Third, the duration and sequence of exposureto the contingencies in Experiment 1 may haveinfluenced the outcome. We exposed partici-pants to each phase for only 10 min, kept thesequence of conditions constant, and changedphases at set points in time, regardless of thepattern or rate of responding. Therefore, phasechanges often occurred when behavior was on anupward or downward trend. Researchers shouldattempt to replicate these results using moreextended exposures and conducting phases untilresponding stabilizes. Extending the exposures tothe contingencies also may help clarify therelative lack of within-subject replication ob-tained during the human operant experiments;often, response rates or patterns obtained duringthe first exposure to a particular level of integrityfailure were not replicated in a subsequentexposure. This lack of within-subject replicationnecessarily tempers the conclusions that can bedrawn from the data. Increasing the duration ofeach phase and conducting phases until behaviorstabilizes may increase the degree to which levelsand allocations of behavior are reproduced acrossreplications.

Other limitations are based on the specificprocedures used in these experiments, indepen-dent of the human operant procedure. Forexample, the baseline and differential reinforce-ment schedules were restricted to only one set of

parameters: FR 1 and extinction schedules.Different parameters of the baseline andtreatment schedules may have resulted indifferent patterns of behavior during theintegrity failure phases. For example, intermit-tent baseline and DRA schedules may havealtered the effects of treatment integrity failures.We chose to examine FR 1 schedules because oftheir frequent use in prior DRA research (e.g.,Shirley et al., 1997; Vollmer et al., 1992, 1999;Walsh, 1991).

We examined only two types of possibleerrors in the current set of studies: errors ofomission and commission that occurred accord-ing to probabilistic (RR) schedules. Thisfrequently resulted in very high reinforcementrates during commission phases, which proba-bly contributed to the detrimental effects ofthose errors. However, probabilistic schedulesare only one way that errors of omission andcommission might occur. These errors couldalso occur based on a variety of different othertypes of schedules, including interval-basedschedules. The use of interval-based treatmentintegrity failures would limit the degree towhich participants could maximize reinforcersby responding at high rates during errors ofcommission. In addition, errors of omission andcommission are only two types of possibleerrors that may occur. Other errors mightinclude differential delays to reinforcement,differential reinforcer magnitudes, or alternat-ing periods of full treatment and no treatment.Future research should examine parametricvariables associated with errors of omissionand commission carefully, as well as differenttypes of treatment integrity failures.

Despite the limitations, the results of thecurrent experiments have important implica-tions for application. First, the effects of errorson differential reinforcement procedures couldbe used to inform caregiver training. Currently,many caregiver-training procedures stress theimportance of delivering reinforcers followingappropriate behavior or following some speci-


fied period of time in which no instances ofproblem behavior occurred. However, theresults of the current experiments suggest thataccidentally failing to deliver an earned rein-forcer (an error of omission) probably is nothighly detrimental to the overall treatmenteffects. Caregiver training procedures mayinstead focus on the importance of theextinction component to differential reinforce-ment procedures. If caregivers do not imple-ment this component with a high level ofintegrity (in other words, if they make com-mission errors), the treatment effects could bedegraded. Unfortunately, extinction is notalways possible, such as with dangerous behav-ior reinforced by social consequences. Thus,evaluations of differential reinforcement proce-dures that minimize the reinforcement ofproblem behavior, rather than eliminating it,are warranted.

Second, the results imply that initial moni-toring of a caregiver’s implementation ofprocedures may improve the chance thattreatments later have robust effects. Monitoringand feedback during initial implementationcould have two overall effects on the treatment.One, it could increase the overall levels ofintegrity with which the caregiver implementsthe procedure over time by providing a solidtraining foundation. Therefore, treatment in-tegrity failures may never become an issuebecause the caregiver consistently implementsthe treatment with a high level of integrity.Two, monitoring may help ensure a high degreeof treatment integrity during the initial imple-mentation of the treatment. The results ofExperiment 3 suggest that high levels of initialintegrity may reduce the detrimental effects ofsubsequent treatment integrity failures. Unfor-tunately, this implies that caregiver-trainingprocedures may be time intensive initially toensure the best long-term outcome. Futureresearch should examine the types and levels oftreatment integrity failures made by caregiversfollowing a variety of training procedures.

In conclusion, the methods used in thecurrent studies seem to be useful for studyingcomplicated questions like those related totreatment integrity. Across experiments, re-duced levels of treatment integrity had detri-mental effects on responding. The currentstudies may help inform future treatmentintegrity research, which could focus more onerrors of commission and combined errors thanerrors of omission alone. In addition, moreresearch is needed on the effects of treatmentintegrity failures on other types of behavioraltreatments, such as noncontingent reinforce-ment procedures. Finally, the types and levels ofintegrity failures that occur in applicationshould be examined through descriptive studies.The values obtained through these descriptiveanalyses then could be replicated and manipu-lated in the laboratory.

REFERENCES

Carr, E. G., & Durand, V. M. (1985). Reducing behaviorproblems through functional communication train-ing. Journal of Applied Behavior Analysis, 18,111–126.

Codding, R. S., Feinberg, A. B., Dunn, E. K., & Pace, G.M. (2005). Effects of immediate performance feedbackon implementation of behavior support plans. Journalof Applied Behavior Analysis, 38, 205–219.

Hanley, G. P., Iwata, B. A., & Thompson, R. H. (2001).Reinforcement schedule thinning following treatmentwith functional communication training. Journal ofApplied Behavior Analysis, 34, 17–38.

Iwata, B. A., Dorsey, M. F., Slifer, K. J., Bauman, K. E.,& Richman, G. S. (1994). Toward a functionalanalysis of self-injury. Journal of Applied BehaviorAnalysis, 27, 197–209. (Reprinted from Analysis andIntervention in Developmental Disabilities, 2, 3–20,1982)

Kelley, M. E., Lerman, D. C., & Van Camp, C. M.(2002). The effects of competing reinforcementschedules on the acquisition of functional communi-cation. Journal of Applied Behavior Analysis, 35,59–63.

Lerman, D. C. (2003). From the laboratory to communityapplication: Translational research in behavior analysis.Journal of Applied Behavior Analysis, 36, 415–419.

Mazaleski, J. L., Iwata, B. A., Vollmer, T. R., Zarcone, J.R., & Smith, R. G. (1993). Analysis of thereinforcement and extinction components in DROcontingencies with self-injury. Journal of AppliedBehavior Analysis, 26, 143–156.


Mueller, M. M., Piazza, C. C., Moore, J. W., Kelley, M.E., Bethke, S. A., Pruett, A. E., et al. (2003). Trainingparents to implement pediatric feeding protocols.Journal of Applied Behavior Analysis, 36, 545–562.

Northup, J., Fisher, W., Kahng, S. W., Harrell, R., &Kurtz, P. (1997). An assessment of the necessarystrength of behavioral treatments for severe behaviorproblems. Journal of Developmental and PhysicalDisabilities, 9, 1–16.

Shirley, M. J., Iwata, B. A., Kahng, S. W., Mazaleski, J. L.,& Lerman, D. C. (1997). Does functional commu-nication training compete with ongoing contingenciesof reinforcement? An analysis during responseacquisition and maintenance. Journal of AppliedBehavior Analysis, 30, 93–104.

Tiger, J. H., Hanley, G. P., & Bruzek, J. (2008). Functionalcommunication training: A review and practical guide.Behavior Analysis in Practice, 1, 16–23.

Volkert, V. M., Lerman, D. C., Call, N. A., & Trosclair-Lasserre, N. (2009). An evaluation of resurgenceduring treatment with functional communicationtraining. Journal of Applied Behavior Analysis, 42,145–160.

Vollmer, T. R., & Iwata, B. A. (1992). Differentialreinforcement as treatment for behavior disorders—procedural and functional variations. Research inDevelopmental Disabilities, 13, 393–417.

Vollmer, T. R., Iwata, B. A., Smith, R. G., & Rodgers, T.A. (1992). Reduction of multiple aberrant behaviorsand concurrent development of self-care skills withdifferential reinforcement. Research in DevelopmentalDisabilities, 13, 287–299.

Vollmer, T. R., Roane, H. S., Ringdahl, J. E., & Marcus,B. A. (1999). Evaluating treatment challenges withdifferential reinforcement of alternative behavior.Journal of Applied Behavior Analysis, 32, 9–23.

Walsh, P. (1991). The use of differential reinforcement inaffecting multiple behavioural change in a womanwith severe mental handicap. Irish Journal ofPsychology, 12, 382–392.

Worsdell, A. S., Iwata, B. A., Hanley, G. P., Thompson,R. H., & Kahng, S. W. (2000). Effects of continuousand intermittent reinforcement for problem behaviorduring functional communication training. Journal ofApplied Behavior Analysis, 33, 167–179.

Received March 17, 2008Final acceptance June 25, 2009Action Editor, Henry S. Roane


effects of treatment integrity failures during ...with children with developmental disabilities....

Documents