journal of early intervention - eric - education resources ... · kristie pretti-frontczak and...

http://jei.sagepub.com

Journal of Early Intervention

DOI: 10.1177/105381510002300204 2000; 23; 92 Journal of Early Intervention

Kristie Pretti-Frontczak and Diane Bricker Objectives

Enhancing the Quality of Individualized Education Plan (IEP) Goals and

http://jei.sagepub.com/cgi/content/abstract/23/2/92 The online version of this article can be found at:

Published by:

http://www.sagepublications.com

On behalf of:

Division for Early Childhood of the Council for Exceptional Children

can be found at:Journal of Early Intervention Additional services and information for

http://jei.sagepub.com/cgi/alerts Email Alerts:

http://jei.sagepub.com/subscriptions Subscriptions:

http://www.sagepub.com/journalsReprints.navReprints:

http://www.sagepub.com/journalsPermissions.navPermissions:

unauthorized distribution.© 2000 Division for Early Childhood of the Council for Exceptional Children. All rights reserved. Not for commercial use or

by M Peterson on May 28, 2008 http://jei.sagepub.comDownloaded from

http://www.dec-sped.org

http://jei.sagepub.com/cgi/alerts

http://jei.sagepub.com/subscriptions

http://www.sagepub.com/journalsReprints.nav

http://www.sagepub.com/journalsPermissions.nav


92

Enhancing the Quality of IndividualizedEducation Plan (IEP) Goals and Objectives

KRISTIE PRETTI-FRONTCZAKKent State University

DIANE BRICKERUniversity of Oregon

Individualized service is a cornerstone of early childhood special education. A primarymechanism used to individualize services is the Individualized Education Plan (IEP).Unfortunately, many studies report that IEP goals and objectives tend to be poorly written andquestion the individualized nature of the IEP. The purpose of this study was to validate astrategy for improving the quality of written IEP goals and objectives. A diverse group of earlychildhood special educators was trained on writing quality IEP goals/objectives and using acurriculum-based measure called the Assessment, Evaluation, and Programming System (AEPS)for Three to Six Years. Results indicate the quality of written IEP goals and objectivesimproved following the IEP goal writing training and use of the AEPS Test.

A guiding premise of Early Childhood Spe-cial Education (ECSE) is that young childrenwill prosper when intervention efforts focus

on their individual strengths, interests and

emerging skills (Bricker, 1989, Bricker, Pret-ti-Frontczak, & McComas, 1998). Since themiddle of the 1970s, the Individualized Ed-ucation Plan (IEP) has been at the center ofefforts to individualize services, particularlythrough the construction of child goals andobjectives (Bagnato, Neisworth & Munson,1997; Bailey & Wolery, 1992; Bricker et al.;Davis, Kilgo, & Gamel-McCormick, 1998;Espin, Deno, & Albayrak-Kaymak, 1998).Continued reliance on IEP goals and objec-tives as a means of individualizing servicesis supported by recommended practice foryoung children with disabilities (Odom &

McLean, 1996). Recommended practice sug-gests that learning and development can beenhanced when IEP goals and objectives ad-dress children’s individual needs and are op-

erationally defined making them useful

across team members (Bricker et al., 1998;

Grisham-Brown & Hemmeter, 1998). Wenote here that our definition of a team in-

cludes family members and professionalsworking together to meet the needs of indi-vidual children, thus whenever the term teamis used, family involvement and participationshould be assumed. When goals and objec-tives are directly linked to other key programcomponents such as assessment, interven-

tion, and evaluation, recommended practicesuggests that services can be improved (Bag-nato et al., 1997; Bricker et al.; Davis et al.,1998). In other words, many ECSE personnelbelieve that children’s individual needs are

better met when comprehensive assessmentinformation, goals and objectives, interven-tion, and evaluation efforts are directly linked(Bagnato et al.; Bricker, 1989; Bricker et al.).Thus, higher quality IEP goals and objectivesthat are developed from a comprehensive as-sessment process, and directly linked to in-tervention and evaluation, are likely to con-tribute to the individualization of services

and improved outcomes for young children.




93

Review of IEPsExamining written IEP goals and objectiveshas been a focal research topic since the 1980s(e.g., Billingsley, 1984; Hunt, Goetz, & An-

derson, 1986; Lynch & Beare, 1990; Morgan,1981; Smith & Simpson, 1989; Weisenfeld,1986). Findings from the past two decades ofresearch indicate that IEPs often (a) containmissing mandated components (Carri, 1985;Schenck & Levy, 1979; Smith & Simpson;Michnowicz, McConnell, Peterson, & Odom,1995); (b) target non-functional skills, such asstacking blocks (Downing, 1988; Weisenfeld);(c) contain little information regarding howgoals will be generalized or what performancecriteria will be used (Lynch & Beare); (d) em-phasize pre-academic skills versus real-lifeskills (Goodman & Bond, 1993); and (e) in-clude goals and objectives that do not addressa child’s area of identified delay or need

(Smith & Simpson). Given repeated evidencethat IEP goals and objectives are often poorlywritten it is not surprising to find that the IEPdoes not appear to strengthen the link betweenprogram components. In fact, many IEPs donot appear to serve as an intervention guide,assist in the decision making process, or tohave a direct impact on outcomes for youngchildren (Dudley-Marling, 1985; Fiedler &

Knight, 1986; Margolis & Truesdell, 1987;Morgan & Rhode, 1983; Reiher, 1992;Schenck, 1980). Additionally, teachers andschool personnel often report that the IEP haslittle utility, is unnecessary paperwork, and isfiled away until end-of-the-year evaluations.A gap between recommended and actual

practice in writing IEP goals and objectivescurrently exists (Bricker et al., 1998; Grisham-Brown & Hemmeter, 1998). Without strate-

gies for closing the gap between recommend-ed and actual practice, continued use of theIEP as a primary mechanism for individual-izing and improving outcomes for young chil-dren is questionable. In fact, some writershave suggested an adverse relationship be-tween IEPs and child outcomes. For example,Goodman and Bond (1993) maintain that poorintervention outcomes for children are a resultof poorly written IEP goals and objectives.While little empirical evidence is available to

support an adverse relationship between IEPsand child outcomes, strategies for ensuring apositive relationship are needed. Specifically,strategies that assist in the development ofquality IEP goals and objectives, and in turnenhance the link between program compo-nents and improve outcomes for children, areneeded (Grisham-Brown & Hemmeter; Mich-nowicz et al., 1995).

Selection of a Curriculum-BasedMeasureOne potential strategy for linking key programcomponents is the use of a curriculum-basedassessment and evaluation measure that con-

tains meaningful skills (Bagnato et al., 1997;Bricker et al., 1998; Notari & Bricker, 1990;Notari & Drinkwater, 1991). Curriculum-based assessment and evaluation measures,sometimes referred to as CBM, are &dquo;a formof criterion-referenced measurement whereincurricular objectives act as the criteria for theidentification of instructional targets and forthe assessment of status and progress&dquo; (Bag-nato & Neisworth, 1991, p. 97). Such mea-sures are designed to generate comprehensiveand detailed information about children’s per-formance (strengths and emerging skills) andwhen composed of functional skills, can beused to develop meaningful IEP goals and ob-jectives and intervention content (Bagnato etal.; Notari, Slentz & Bricker, 1991).The purpose of this study was to determine

if a training package that included (a) infor-mation on writing IEP goals and objectivesand (b) use of a curriculum-based assessmentand evaluation measure called the Assess-

ment, Evaluation, and Programming Systemfor Three to Six Years (Bricker & Pretti-Front-

czak, 1996) would improve the quality ofwritten IEP goals and objectives. The AEPSwas specifically developed to link the assess-ment process, goals and objectives, interven-tion, and evaluation activities. Further theAEPS is a comprehensive system with pro-cedures to promote (a) full collaborationamong team members, (b) a focus on chil-dren’s strengths and emerging skills, and (c)intervention within natural environments.




94

Rationale for the AEPS Three to SixThe AEPS Three to Six is a curriculum-based

assessment and evaluation measure designedto assist interventionists in (a) determiningchildren’s present level of functioning, (b) de-veloping meaningful IFSP or IEP goals andobjectives, (c) planning intervention, and (d)evaluating child performance. It was devel-

oped for use by teams composed of familymembers, direct service personnel (e.g., class-room interventionists, home visitors, child

care providers, assistants) and specialists (e.g.,speech language pathologists, occupationaland physical therapists, psychologists). TheAEPS Three to Six can be used to assess andevaluate the skills of young children who areat risk and who have disabilities. Further, theAEPS Three to Six was specifically developedto link assessment, goals and objective writ-ing, intervention, and evaluation activities.Test data are collected through observationsof children engaged in routine and play activ-ities in familiar settings. The AEPS Three toSix covers six broad curricular domains: fine

motor, gross motor, adaptive, cognitive, socialcommunication, and social.The AEPS Three to Six has four specific

features that validate its use in writing chil-dren’s goals and objectives and for use in thepresent study. First, most items are written toreflect conceptual or generative response clas-ses (e.g., initiates and completes age-appro-priate activities) rather than singular, specificresponses (e.g., remains seated during circletime). The AEPS Three to Six, however, con-tains hundreds of skills that can be combinedin an effort to both individualize and make

skills more functional, generative, and mean-ingful. For example, if a team determines thata child needs to learn skills such as colors,shapes, sizes, and to use 3 word utterances, agoal of using words, phrases, and sentences todescribe, inform, and direct may be targetedrather than targeting four separate goals. Sec-ond, the AEPS Three to Six targets functionalskills and abilities essential for young children

to function independently and to cope with en-vironmental demands (e.g., walks to avoid ob-stacles). The focus on functional skills andabilities ensures that each test item is poten-

tially an appropriate intervention target(Bricker, 1993). Again, items can be modified,adapted, and individualized, making them ap-propriate for teams to use and to &dquo;teach from

assessment results.&dquo;The third validating feature of using the

AEPS Three to Six for our study is a sectionin the test manual containing model examplesof high quality IEP goals and objectives.These models serve as templates that can bemodified and individualized by parents and in-terventionists. For example, one item reads&dquo;responds to topic changes&dquo; but is rewrittenin the manual as &dquo;The child will respond toconversational topic changes initiated by oth-ers with comments, answers, or questions re-lated to the new topic (e.g., The child says, ’I

want to play outside some more,’ and the

adult says, ’We need to go inside now to fix

snack.’ The child responds, ’What are we gon-na eat?’).&dquo; A team using the AEPS Three toSix would take the rewritten model and fur-

ther individualize it for a particular child.Fourth, using the AEPS Three to Six for

IEP goal and objective development is sup-ported by documented utility and growing ev-idence of treatment validity (e.g., Bagnato etal., 1997; Bricker, 1993; Bricker & Pretti-

Frontczak, 1996; Hsia, 1993; Slentz, 1986). Inparticular, support comes from research on theAEPS Birth to Three. For example, two stud-ies have reported that the AEPS Birth to Threehelped teachers develop goals and objectivesin a timely manner, and provided accurate andcomprehensive information regarding chil-dren’s level of functioning (Bailey & Bricker,1986; Bricker, Bailey, & Slentz, 1990). Fur-thermore, Notari and her colleagues (Notari &

Bricker, 1990; Notari & Drinkwater, 1991)found that teachers who used the AEPS Birth

to Three made noticeable improvements in thequality of their written goals and objectives.In particular, Notari and Drinkwater reportedthat teachers wrote goals and objectives thatwere &dquo;more functional, generic, easy to inte-grate within the instructional content, and

measurable, as compared to those based oncomputerized lists&dquo; and &dquo;the AEPS enabled

teachers to sequence goals and objectives ac-cording to a hierarchical teaching sequence&dquo;




95

(p. 101). Additional evidence regarding the

utility of the AEPS Birth to Three comes fromStraka (1994) who reported that speech-lan-guage pathologists found the AEPS Birth toThree to be more useful than the Communi-cation and Symbolic Behavior Scales (Weth-erby & Prizant, 1993) in developing IEP goalsand objectives. Finally, a study by Hamilton(1995) reported that goals and objectives writ-ten by teachers for children with visual im-pairments were of higher quality when theyused the AEPS Birth to Three than the OregonProject Curriculum for Visually Impaired andBlind Preschool Children (Anderson, Boigon,& Davis, 1991).

Although data from studies of the AEPSBirth to Three offer support for its usefulnessin developing quality goals and objectives; nostudies have examined the impact of using theAEPS Three to Six on the quality of writtenIEP goals and objectives. Previous experienc-es, however, suggests to us that training onhow to write IEP goals and objectives shouldaccompany training on how to use the AEPSThree to Six. Therefore, we developed a com-bined training approach for writing IEP goalsand objectives and using the AEPS Three toSix. In addition, we evaluated the quality ofIEP goals and objectives written by trainedparticipants.

Quality DimensionsTo evaluate written IEP goals and objectivesbenchmarks or criteria are needed. From a

thorough review of past research on IEP goalsand objectives and recommended practice de-rived from values and beliefs shared by manyECSE personnel, five quality dimentions wereidentified: functionality (Davis, 1989; Lynch& Beare, 1990; McWilliam et al., 1998; Wei-senfeld, 1986; Wolery, 1989), generality(Bricker, 1989; Lynch & Beare; Notari &

Bricker, 1990; Notari & Drinkwater, 1991), in-structional context (Grisham-Brown & Hem-

meter, 1998; McWilliam, et al.; Notari &

Bricker; Notari & Drinkwater), measurability(Downing, 1988; Michnowicz, et al., 1995),and hierarchical relationship (Michnowicz, etal.; Notari & Bricker; Notari & Drinkwater;Tymitz, 1980).

Functionality. A goal or objective is con-sidered functional if it is useful to children in

successfully negotiating their daily environ-ments (Davis, 1989; Lynch & Beare, 1990:

Weisenfeld, 1986; Wolery, 1989). Functionalskills are those that allow the child greater in-

dependence and give the child the ability toadapt to changing environments. A functionalskill is one that is &dquo;likely to be necessary forsuccess in everyday functioning or for en-hancing development...versus those that haveno immediate function&dquo; (McWilliam et al.,1998, p. 78). For example, learning to walkor feed oneself is more functional than learn-

ing to put pegs in a pegboard or labeling pic-tures on flash cards.

Generality. A second quality dimension,generality, is when a goal or objective repre-sents a general concept or class of behaviorsthat is not specific to a particular item or set-ting (i.e., the skill is generic) (Bricker, 1989).For example, stacking a variety of objects orplacing and releasing many different objectsis more useful than stacking three 1/2 inchblocks or releasing beanbags into a can. Whengoals and objectives are written in more ge-neric terms, interventionists and caregivershave the flexibility to practice the skill acrosssettings, materials, and events. Learning moregeneric skills (e.g., to stack a variety of ob-jects versus to stack 3 blocks) also allowschildren multiple opportunities to use the tar-get skills within their natural environment

(e.g., to stack books, cups, clothes, chairs, car-pet squares), thus enhancing generalization ofthe skill.

Instructional Context. A third quality di-mension relates to whether or not the goal andobjective can be frequently and easily targetedacross daily routines (i.e., instructional con-text). Skills that can be addressed within thenatural environment (i.e., places such ashome, daycare, preschool, and the grocerystore) or skills that can be easily elicited byparents and interventionists meet the qualitydimension of instructional context. For ex-

ample, balance in standing can be addressedor elicited while standing in line for lunch orwhile waiting for a turn on the swing, versusskills such as stands on one foot or walks on




96

a balance beam. McWilliam and his col-

leagues (1998) further suggest that &dquo;When

goals and strategies are written in such a waythat intervention can occur in the normal

home or classroom routines, in normal places,with normal objects and with regular caregiv-ers, they are helpful to families&dquo; (p. 79).

Measurability. A fourth quality dimensionis whether goals and objectives are observable(i.e., they can be seen, heard, counted) andtherefore measurable (Downing, 1988;McWilliam, et al., 1998; Michnowicz, et al.,1995). When goals and objectives include

vague terms such as demonstrates, knows, re-alizes, and increases or undefined terms suchas appropriate social skills or 4-year-old motorskills, it becomes difficult for team membersto (a) know when the behavior(s) have oc-curred and (b) exactly which behavior(s) arebeing targeted for intervention (Bricker et al.,1998; McWilliam, et al.). Goals and objectivesthat meet the measurability quality dimensionare ones that can be agreed upon as to whetheror not the skill was demonstrated and onesthat allow teams to determine if the child hasmet the targeted skill (e.g., independentlywalks for 15 feet for 8 consecutive days).

Hierarchical Relationship. The fifth qual-ity dimension addresses the hierarchical rela-tionship between a goal and its associated ob-jective(s). Objectives should be directly relat-ed to their goal by serving as either a precur-sor step or building block skill. For example,precursor skills to the goal of walking inde-pendently might include pulls to a stand,cruises, and walks with one hand support. Alltoo often objectives are written as restate-

ments of the goal or are unrelated to the goal.For the purposes of this study, IEP goals

and objectives were considered to have highquality when they contained all or most of thefive quality dimensions defined above. That is,high quality goals and objectives that were

functional, generative, addressed in a child’snatural environment, measurable, and hierar-chically related.

METHODS

Participants .

ECSE state coordinators and program direc-tors from California, Kansas, Louisiana,

Oregon, and Washington were contacted andasked to identify personnel who met the fol-lowing criteria for participation in the presentstudy: (a) currently working with children age3-6 years; (b) no prior training or experiencewith the AEPS; (c) willing to participate in a2-day training on how to write quality goalsand objectives and how to use the AEPS

Three to Six; and (d) willing to provide writ-ten goals and associated objectives for a targetchild before and after the training.

Eighty-six participants from five states metthese criteria and were included in the study.Participants for the study were somewhat geo-graphically diverse representing the middle,southern and western regions of the country.The sample was composed of 54 intervention-ists (teachers), 9 therapists, and 2 persons inan administrative or coordinator role (21 par-ticipants did not indicate their professionalrole). Although the literature clearly indicatesthat quality goals and objectives should in-corporate family priorities, values, and culture(e.g., Bricker et al., 1998; McWilliam et al.,1998), we did not directly address family is-sues in this study. We acknowledge the im-portance of determining the skill level of allteam members when constructing and IEP, andwe likewise acknowledge the importance ofensuring the tone and language of the writtendocument is respectful of all team members.We were unable, however, to directly addressfamily priorities and involvement in construc-tion of the IEP document. This decision was

made for three reasons. First, limited resourc-es precluded participants from being able toassure timely family input in writing both pre-training and post-training goals and objec-tives. Second, since the children already werereceiving services (i.e., pre-training goals andobjectives were written prior to the study),family input in the development of the writtenIEP could not be measured. Finally, we onlyexamined the child goals and objectives sec-tion of the IEP or IFSP Nevertheless, the ex-tent to which a goal or objective could be in-tegrated by professionals or family memberswithin the context of daily routines, (e.g.,while cooking dinner, during bath time) wasexamined. Furthermore, because of limited re-




97

sources and diversity across states on how

goals and objectives are written, we invitedonly individuals (primarily teachers) to partic-ipate in the study rather than entire teams. Theextent to which training is needed for otherteam members (i.e., caregivers, therapists, andadministrators), however, remains an area offuture research.

Forty percent of the participants held abachelor’s degree, 40% held a master’s degree,and 20% of the participants did not reporttheir level of education. Participants’ experi-ence in working with young children with dis-abilities ranged from 1 year to 21 years witha mean of 8 years and a standard deviation of

6. Together, participants provided services tonearly 1,000 children in a variety of settings(Head Start classrooms, integrated preschools,non-categorical preschools). These children

ranged in age from 3 to 6 years, resided in

rural, suburban, and urban communities, andwere ethnically diverse (e.g., Caucasian, Af-rican American, Hispanic, Asian/Pacific Is-

lander, and Native American). Disability cat-egories of children being served by the 86participants included children at risk for de-lays, as well as those identified with mild,moderate, and severe disabilities.

MeasuresIEPIIFSP Goals and Objective Rating Instru-ment (GORI). To evaluate the quality of goalsand their associated objectives written duringthe pre-training and post-training phases, theGORI was used. This instrument was de-

signed to evaluate IFSP/IEP goals and objec-tives on five quality dimensions: (a) function-ality, (b) generality, (c) instructional context,(d) measurability, and (e) hierarchical rela-

tionship. Each goal and each objective is ratedindependently to determine the presence or

absence of 10 different quality indicators. Ifan indicator is present, a score of 1 is as-

signed ; if an indicator is absent, a score of 0is assigned. An additional indicator is used toexamine the hierarchical relationship betweenan objective and its corresponding goal. Anobjective written as a necessary step towardattainment of the goal is considered a highquality objective and scored 2. An objective

that simply restates the goal, it is scored 1,and objectives completely unrelated to the

goal are scored 0. Scores across the qualityindicators are summed to determine the over-all quality of a specific goal or objective.Higher GORI scores indicate higher qualitywritten goals or objectives. Shown in Table 1are GORI quality dimensions, associated in-dicators, and definitions of quality dimen-sions. The GORI has been used in previousstudies (Hamilton, 1995; Notari & Bricker,1990; Notari & Drinkwater, 1991; Straka,1994) and has been reported to be a valid,reliable and useful measure (Notari, 1988; No-tari & Bricker).An example of high quality and low quality

goals (and their corresponding objectives) ispresented in Table 2. How objectives can behierarchically related (i.e., as precursor skills)or unrelated to their associated goal also isshown in Table 2. Upon examination of theseexamples, one can readily see that high andlow quality goals and objectives often containsimilar features (e.g., antecedents, behaviors,and criteria). Differences in quality often arisein how the goals and objectives are selectedand written. For example, the conditions un-der which a child is expected to demonstratea skill (antecedents) should not be limited tospecific times of the day or locations (e.g., thegym, during group time). Rather, antecedentsshould promote the use of skills under con-ditions that simulate how and where the childwill need the skills within their daily routine(i.e., across environments such as the home,school, child care). Further, instead of select-ing behaviors that are broad and difficult tomeasure (e.g., improve gross motor skills),high quality goals and objectives contain in-dividualized behaviors that can be seen, heard,counted, and understood (e.g., moves aroundthe house and the classroom by holding ontofurniture). Finally, criteria that are selectedshould convince the team the child has metthe skill. Criteria such as &dquo;at a 4-year-old lev-el&dquo; or &dquo;80% of the time&dquo; are hard to interpret,making decisions more difficult regardingwhen a child has mastered a skill.

For this study, two coders were trained torate goals and objectives using the GORI.




98

Table 1.IEPIIFSP Goals and Objectives Rating Instrument (GORI) Quality Dimensions, Associated In-dicators, and Definitions

&dquo; - ~ J ..

During coding training, both coders reached88% inter-rater percentage agreement. Onecoder then rated all goals and objectives sub-mitted by the 86 participants. To ensure inter-rater agreement remained at or above 80%, thesecond coder randomly selected 33% of therated goals and objectives and recoded them.The order in which the goals and objectiveswere rated was sequenced randomly so coderswere unaware whether they were rating goalsand objectives from the pre-training or post-training phase. Total percentage agreementwas calculated by reviewing GORI forms forboth coders and counting each indicator inwhich there was agreement. The sum of agree-ments was divided by the total number of

agreements plus disagreements and multipliedby 100 (Tawney & Gast, 1984). Inter-rater

agreement consistently exceeded 80%. AKappa coefficient (Cohen, 1960) also was cal-culated across each of the 11 quality indica-

. _ .... _ . _ _

&dquo;&dquo;

tors to establish a more conservative estimateof inter-rater agreement. The average Kappawas .72 with a standard deviation of .06.

Procedures

Pre-Training Phase. During the pre-trainingphase, participants were instructed to select

and submit to us, three IEP goals and corre-sponding objectives they had written. (Al-though the focus of the study was on IEP de-velopment, some participants lived in states

where IFSPs were developed for 3 to 6 yearold children, thus we refer to both the IFSPand the IEP) Despite potential beliefs that cer-tain disability categories and certain develop-mental domains present greater or lesser de-

grees of difficulty when constructing IFSP andIEP goals and objectives, there is a lack of

empirical evidence to support such a notion.Thus, we did not restrict the scope or type ofskills participants could targeted, nor did we




99

Table 2.

Comparison between High and Low Quality Goals and Objectives ’

Note. Underlined portions signify examples of antecedents, behaviors, and criteria that contribute to the high qualitynature of the goal and objectives. Bolded portions signify examples of antecedents, behaviors, and criteria that contributeto the low quality nature of the goal and objectives.

specify the developmental domains fromwhich goals and objectives could be selected.Furthermore, participants could submit goalsand objectives for any child with any level ofability. We also did not specify a number ofobjectives, although we used no more thanthree from each participant.

Also during the pre-training phase partici-pants indicated the resources they used to de-velop the goals and objectives they were sub-mitting. A variety of resources were listed in-cluding (a) program-specific checklists, (b)computerized IEP programs, (c) the BattelleDevelopmental Inventory (Newborg, Stock,Wnek, Guidubaldi & Svinicki, 1988), (d) theEarly Intervention Developmental Profile(Rogers & D’Eugenio, 1981), (e) the HawaiiEarly Learning Profile (Vort Corporation,1995), and (f) Preparing Children to Learn: AFamily-Centered Approach to FunctionalSkills Assessment (McLean, McNay, & Kot-

twitz, 1995). None of the participants indicat-ed using either the AEPS Birth to Three orThree to Six to write their pre-training goalsor objectives.

Training Phase. After participants submit-ted their pre-training goals and objectives,they participated in a 2-day training session.The training session was conducted by expe-rienced trainers and divided into six majorcontent areas. Content Area 1 included a de-

scription of AEPS components: (a) the mea-surement, (b) curriculum, (c) child progressform, (d) family report, and (e) family interestsurvey. Content Area 2 provided rationale forusing the AEPS Three to Six and a descriptionof its utility. Content Area 3 included activi-ties for administering and scoring the AEPSThree to Six first with one child and one do-

main, then with one child across domains, and

finally with multiple children across domains.Content Area 4 provided practice opportuni-ties for participants to summarize AEPS Threeto Six results (i.e., obtaining percentage cor-rect scores and present level of functioningnarratives) and to interpret findings (i.e., de-termine a child’s strengths, interests andemerging skills). Content Area 5 emphasizedhow to use the AEPS Three to Six to select

and write IFSP/IEP goals and objectives. Con-




100

tent Area 6 consisted of training participantsto use a five-step process in developing highquality goals and objectives: (Step 1) admin-istration of a curriculum-based assessment andevaluation measure and gathering informationfrom multiple sources, (Step 2) summarizingresults, (Step 3) selecting skills that meet thefive quality benchmarks, (Step 4) prioritizingskills, and (Step 5) writing goals and objec-tives.

Post-Training Phase. After completing the2-day training session, participants were to ad-minister the AEPS Three to Six to a child theywere currently serving. We provided each par-ticipant with an AEPS Three to Six manualand data recording form and asked them toadminister the test, complete the data record-ing form, and independently write post-train-ing goals and associated objectives using theAEPS Three to Six results. Participants couldselect a new child for this activity or theycould use the same child whose goals and ob-jectives were submitted at the pre-trainingphase. Again, because empirical evidence

does not suggest that one child poses a lesseror greater degree of difficulty in writing IFSPsor IEPs, we were not concerned that writinggoals and objectives for different children

would pose a threat to the validity of thestudy. Participants were given 8 to 12 weeksto complete this assignment.

Analyses were conducted by comparing thepre-training goals and objectives (written be-fore training and use of the AEPS Three toSix) with the goals and objectives written fol-lowing training and use of the AEPS Three toSix. For this comparison, we used the IEP/IFSP Goals and Objectives Rating Instrument(GORI; Notari, 1988) that addresses the qual-ity dimensions described earlier. Only threepre-training and three post-training pairs ofgoals and objectives were compared for eachparticipant. If a participant submitted morethan three goals and three corresponding ob-jectives at either phase, we randomly selectedthree pairs for analyses. Eight participantssubmitted goals without objectives during thepre-training or post-training phases, thus finalanalyses were conducted on 86 participants’

pre- and post-training goals and 78 partici-pants’ pre- and post-training objectives.

RESULTS

This study was designed to determine if a 2-day training session on how to write IEP goalsand objectives and the use of a curriculum-based assessment and evaluation measure, the

AEPS Three to Six, improved the quality ofIEP goals and objectives written by ECSEproviders. Findings are based on a comparisonof pre-training goal and objective ratings withpost-training goal and objective ratings for 86participants from 5 states.

Participants’ pre-training and post-traininggoals and objectives were rated using the

GORI. For each participant at least one goaland objective, and no more than three goalsand objectives were rated during both studyphases. A mean percent score on pre- andpost-training goals and pre- and post-trainingobjectives was calculated for each participantand these mean percent scores then were ag-gregated across participants. For goals, therewere 86 aggregated mean percent scores at

both pre- and post-training. For objectives,there were 78 aggregated mean percent scoresat both pre- and post-training.

Using paired t-tests, aggregated mean per-cent scores for the two phases were compared.As shown in Table 3, there was a statisticallysignificant difference between pre-trainingand post-training aggregated mean percentscores for each of the 10 goal indicator com-parisons. Also shown on Table 3 are the dif-ferences between pre-training and post-train-ing aggregated mean percent scores for eachof the 11 objective indicators. Nine of the 11 1pre-training to post-training objective compar-isons were statistically significant.

DISCUSSION

Goals and objectives that are functional, gen-erative, understandable, measurable, and re-lated, as well as based upon family prioritiesand values are likely to strengthen the linkbetween key program components and consistof individualized content that will result in

positive child outcomes (Bricker et al., 1998).




101

Unfortunately, as noted in the introductorysection, investigations of IEP goals and objec-tives written by educational personnel oftendo not meet recommended practice guidelines(e.g., are not functional and meaningful).The purpose of this study was to examine

whether an independent variable composed oftraining on writing IEP goals and objectivesand training and use of the AEPS Three to Sixwould improve the quality of written IEP

goals and objectives. Study results are clear.The quality of written goals were rated as sta-tistically significantly higher on all 10 qualityindicators following the combined goal writ-ing training and use of the AEPS for Three toSix. The quality of written objectives were rat-ed as statistically significantly higher on 9 of11 quality indicators following training anduse of the AEPS Three to Six.A review of Table 3 reveals an interesting

finding regarding the quality of goals versusthe quality of objectives. First, the aggregatedmean percent scores for objectives were con-sistently higher at post-training, than were ag-gregated mean percent scores for goals. In

fact, aggregated mean percent scores for ob-jectives at pre-training were consistently high-er than the aggregated mean percent scores forgoals at post-training with standard deviationsbeing similar for objectives and goals. Thesefindings are consistent with data reported byNotari and Drinkwater (1991) and suggest thatinterventionists are able to write higher qual-ity objectives than goals. Interestingly, otherinvestigators (e.g., Billingsley, 1984; Tymitz,1980) have reported that objectives are moredifficult for interventionists to write than

goals. It is not clear, however, why the writtenobjectives were of higher quality. A replica-tion is necessary to determine if such differ-

ences would maintain across other participantsand what the implications of such findingswould be.

Although our study lacked a control group,the number of interventionists involved, theireducational, geographic, and experiential di-versity, the variety of intervention programsand range of enrolled children (disability, eth-nicity), all contribute to the robustness of theoutcomes. The use of the AEPS Three to Six

in combination with training on writing goalsand objectives did improve the quality of thegoals and objectives written by the participat-ing ECSE interventionists. Because the train-ing and use of the AEPS Three to Six wasinextricably linked within the study, we can-not draw conclusions about the individual

contributions of the AEPS Three to Six, or the

training on goal and objective writing. Our ex-periences, however, suggest that both use of acurriculum-based assessment and evaluation

measure that contains meaningful skills andtraining on writing goals and objectives arenecessary for learning to write quality IFSPand IEP goals and objectives.

Curriculum-based assessment and evalua-

tion measures generally offer the user an arrayof developmental domains, content areas, op-erational criteria, and scoring procedures thatrequire time and effort to learn, understand,and interpret if the measure and its outcomesare to be used correctly. Although some cur-riculum-based assessment and evaluation

measures do much to connect assessment to

intervention, it may also be necessary to pro-vide interventionists with specific training inunderstanding and using the assessment re-sults to develop quality IFSP and IEP goalsand objectives. Further, teams may requiretraining on how to select curriculum-based as-sessment and evaluation measures that will

help them develop quality IFSP and IEP goalsand objectives. Finally, it is important to re-mind teams that multiple methods and sourcesshould be used when developing IFSPs andIEPs.

If the field’s assumption is correct, that is,quality goals and objectives result in more ef-fective intervention, which in turn, producebetter child outcomes, then future work de-

signed to assist ECSE personnel in developingquality IFSP and IEP goals and objectives iscritical. Replication of the study’s findingswould support the importance of training andthe use of effective curriculum-based assess-

ment and evaluation measures to assure the

quality of IFSP and IEP goals and objectives.Future studies should validate the strategy of

training ECSE personnel on writing qualitygoals and objectives and use of the AEPS Test




102

Table 3.

Pre-Training and Post-Training Aggregated Mean Percent Score Comparisons for IEP andIFSP Goals (N = 86) and Objectives (N = 78) on GORI Quality Indicators

*p < .05. **p < .01. ***p < .001.

Three to Six or other appropriate curriculum-based assessment and evaluation measures asa potential first step in linking program com-ponents and improving outcomes for young

children. In addition, a related line of workneeds to be undertaken that addresses the

treatment validity of curriculum-based assess-ment and evaluation measures such as the




103

AEPS Three to Six. In this sense treatment

validity refers to whether child outcomes areimproved through the use of a measure’s re-sults (Bagnato et al., 1997; Messick, 1989).Finding such as these will do much to leadthe field forward in its quest for improved ser-vices to young children with disabilities.

REFERENCES

Anderson, S., Boigon, S., & Davis, K. (1991).Oregon project curriculum for visually im-

paired and blind preschool children (OPC).Medford, OR: Jackson Co. Education ServiceDistrict.

Bagnato, S. J., & Neisworth, J. T (1991). Assess-ment for early intervention: Best practices forprofessionals. New York: Guilford Press.

Bagnato, S. J., Neisworth, J. T, & Munson, S. M.(1997). LINKing assessment and early inter-vention : An authentic curriculum-based ap-proach. Baltimore, MD: Paul H. Brookes Pub-lishing Co.

Bailey, D. B., & Wolery, M. (1992). Teaching in-fants and preschoolers with disabilities (2ndEd.). New York: MacMillan.

Bailey, E. J., & Bricker, D. D. (1986). A psycho-metric study of a criterion-referenced assess-ment instrument designed for infants and youngchildren. Journal of the Division for EarlyChildhood, 10(2), 124-134.

Billingsley, E F. (1984). Where are the generalizedoutcomes? (An examination of instructional

objectives). The Journal of the Association forPersons with Severe Handicaps, 9, 182-192.

Bricker, D. D. (1989). Early intervention for at-riskand handicapped infants, toddlers, and pre-school children. Palo Alto, CA: VORT Corp.

Bricker, D. D. (Ed.) (1993). The assessment, eval-uation, and programming system for infantsand young children: Vol. 1 AEPS measurement

for birth to three years. Baltimore, MD: PaulH. Brookes Publishing Co.

Bricker, D. D., Bailey, E. J., & Slentz, K. L. (1990).Reliability, validity and utility of the Evalua-tion and Programming System: For Infants andYoung Children (EPS-I). Journal of Early In-tervention, 14(2), 147-160.

Bricker, D. D., & Pretti-Frontczak, K. L. (Eds.)(1996). The assessment, evaluation, and pro-gramming system for infants and young chil-dren : Vol. 3 AEPS measurement for three tosix years. Baltimore, MD: Paul H. Brookes.

Bricker, D. D., Pretti-Frontczak, K. L., & Mc-

Comas, N. R. (1998). An activity-based ap-

proach to early intervention (2nd Ed.). Balti-

more, MD: Paul H. Brookes.

Carri, L. (1985). Inservice teachers’ assessed needsin behavior disorders, mental retardation, andlearning disabilities: Are they similar? Excep-tional Children, 51, 411-416.

Cohen, J. (1960). A coefficient of agreement fornominal scales. Educational and PsychologicalMeasurement, 20, 27-36.

Davis, M. D., Kilgo, J. L., & Gamel-McCormick,M. (1998). Young children with special needs:A developmentally appropriate approach. Bos-ton : Allyn & Bacon.

Davis, P (1989). Effectiveness of inservice trainingon the functionality, technical adequacy, andgenerality of IEP objectives. Carbondale, IL:Southern Illinois University. Order #

DA9022780.

Downing, J. E. (1988). Active versus passive pro-gramming : A critique of IEP objectives for stu-dents with the most severe disabilities. Journal

of the Association for Persons with SevereHandicaps, 13, 197-201.

Dudley-Marling, C. C. (1985). Perceptions of theusefulness of the IEP by teachers of learningdisabled and emotionally disturbed children.

Psychology in the Schools, 22, 65-67.

Espin, C. A., Deno, S. L., Albayrak-Kaymak, D.(1998). Individualized education programs inresources and inclusive settings: How "individ-ualized" are they? The Journal of Special Ed-ucation, 32(3), 164-174.

Fiedler, J. E, & Knight R. R. (1986). Congruencebetween assessed needs and IEP goals of iden-tified behaviorally disabled students. Behavior-al Disorders, 22-27.

Goodman, J. F., & Bond L. (1993). The individu-alized education program: A retrospective cri-tique. Journal of Special Education, 26(4),408-422.

Grisham-Brown, J., & Hemmeter, M. L. (1998).Writing IEP goals and objectives: Reflecting anactivity-based approach to instruction for

young children with disabilities. Young Excep-tional Children, 1(3), 2-10.

Hamilton, D. A. (1995). The utility of the Assess-ment, Evaluation, and Programming System inthe development of quality IEP goals and ob-jectives for young children, birth to three, withvisual impairments. Dissertation Abstracts In-ternational, 56(08), 2994A. (University Micro-films No. AAI95-41906)

Hsia, T. (1993). Evaluating the psychometric prop-erties of the Assessment, Evaluation, and Pro-gramming System for 3 to 6 Years: AEPS Test3 to 6 Years. Dissertation Abstracts Interna-




104

tional, 54(04), 1317A. (University MicrofilmsNo. AAT93-22027)

Hunt, P, Goetz, L., & Anderson, J. (1986). Thequality of IEP objectives associated with place-ment on integrated versus segregated schoolsites. Journal of the Association for Personswith Severe Handicaps, 11(2), 125-130.

Lynch, E. C., & Beare, P L. (1990). The quality ofIEP objectives and their relevance to instruc-tion for students with mental retardation andbehavior disorders. Remedial & Special Edu-cation, 11(2), 48-55.

Margolis, H., & Truesdell, L. A. ( 1987). Do specialeducation teachers use IEPs to guide instruc-tion ? The Urban Review, 19, 151-159.

McLean, L., McNay, V , & Kottwitz, E. (1995).Preparing Children to learn: A family-centeredapproach to functional skills assessment. Tuc-son, AZ: Communication Skill Builders.

McWilliam, R. A., Ferguson, A., Harbin, G. L.,Porter, P, Munn, D., & Vandiviere, P (1998).The Family-centeredness of individualizedfamily service plans. Topics in Early Child-hood Special Education, 18(2), 69-82.

Messick, S. ( 1989). Meaning and values in test val-idation : The science and ethics of assessment.Educational Researcher1 8(2), 5-11.

Michnowicz, L. L., McConnell, S. R., Peterson,C., & Odom, S. L. (1995). Social goals andobjectives of preschool IEPs: A content anal-ysis. Journal of Early Intervention, 19, 273-282.

Morgan, D P (1981, May-June). Characteristics ofa quality IEP Education Unlimited, 3, 12-17.

Morgan, D. P, & Rhode, G. (1983). Teachers’ at-titudes toward IEPs: A two-year follow-up. Ex-ceptional Children, 50(1), 64-67.

Newborg, J., Stock, J. R., Wnek, L. Guidubaldi, J.,& Svinicki, J. S. ( 1988). Battelle Developmen-tal Inventory (BDI). Chicago: Riverside.

Notari, A. R. (1988). The utility of a criterion-ref-erenced instrument in the development of in-dividualized education plan goals for infantsand young children. Dissertation Abstracts In-

ternational, 49(07), 1767A. (University Micro-films No. AAT88-14193)

Notari, A. R., & Bricker, D. D. ( 1990). The utilityof a curriculum-based assessment instrument inthe development of individualized educationplans for infants and young children. Journalof Early lntervention, 14(2), 5-11.

Notari, A. R., & Drinkwater, S. G. (1991). Bestpractice for writing child outcomes: An evalu-ation of two methods. Topics in Early Child-hood Special Education, 11 (3), 92-106.

Notari, A. R., Slentz, K. L., & Bricker, D. D.

(1991). Assessment-curriculum systems forearly childhood/special education. In D. Mitch-ell & R. Brown (Eds.), Early intervention stud-ies for children with special needs (pp. 160-205). London: Chapman & Hall.

Odom, S. L., & McLean, M. E. (1996). Early in-tervention for infants and young children withdisabilities and their families. Recommendedpractice. Austin, TX: PRO-ED.

Reiher, T. C. (1992). Identified deficits and their

congruence to the IEP for behaviorally disor-dered students. Behavioral Disorders, 17(3),167-177.

Rogers, S. J., & D’Eugenio, D. B. (1981). Devel-opmental programming for infants and youngchildren (DPIYC): Vol. 2. Early InterventionDevelopmental Profile (EIDP). Ann Arbor:

University of Michigan Press.

Schenck, S. J. (1980). The diagnostic/instructionallink in individualized education programs.Journal of Special Education, 14, 337-345.

Schenck, S. J., & Levy, W. K. (1979). IEPs: Thestate of the art - 1978. Paper presented at themeeting of the American Educational Re-search Association. San Francisco, CA (ERICDocument Reproduction Service No. ED 175201)

Slentz, K. L. (1986). Evaluating the instructionalneeds of young children with handicaps: Psy-chometric adequacy of the Evaluation and Pro-gramming System—Assessment Level II. Dis-sertation Abstracts International, 47(11),4072A. (University Microfilms No. AAT87-05890)

Smith, S. W., & Simpson, R. L. ( 1989). An analysisof individualized education programs (IEPs)for students with behavioral disorders. Behav-ioral Disorders, 14(2), 107-116.

Straka, E. A. (1994). Assessment of young childrenfor communication delays. Dissertation Ab-stracts International, 56(02), 456A. (UniversityMicrofilms No. AAT95-19689)

Tawney, J. W., & Gast, D. L. ( 1984). Single subjectresearch in special education. Columbus, OH:Merrill.

Tymitz, B. L. (1980). Instructional aspects of theIEP: An analysis of teachers’ skills and needs.Educational Technology, 20(9), 13-20.

Vort Corporation. (1995). Hawaii Early LearningProfile (HELP) for preschoolers. Palo Alto,CA: Author.

Weisenfeld, R. B. (1986, September). The IEPs ofDown syndrome children: A content analysis.Education and Training of the Mentally Re-tarded, 21(3), 211-219.

Wetherby, A. M., & Prizant, B. (1993). Communi-




105

cation and symbolic behavior scales-normededition. Chicago: Riverside Publishing Co.

Wolery, M. (1989). Using assessment informationto plan instructional programs. In D. Bailey &M. Wolery (Eds.), Assessing infants and pre-schoolers with handicaps (pp. 478-495). Co-lumbus, OH: Merrill.

Support for this research was provided by GrantNo. H133G40147 from the National Institute for

Disability and Rehabilitation Research, U.S. De-

partment of Education. The authors gratefully ac-knowledge Maura Schoen for her work on earlierdrafts of the manuscript and Misti Waddell andCraig Leve for data collection, organization, andcoordination of the research project.

Address correspondence to Kristie Pretti-Front-

czak, Ph. D., Kent State University, EducationalFoundations and Special Services, 405 White Hall,Kent, OH 44242. E-mail: [email protected]




journal of early intervention - eric - education resources ... · kristie pretti-frontczak and...

Documents