DOCUMENT RESUME

ED 240 421 CG 017 250

AUTHOR Gottfredson, Gary D.; And OthersTITLE The School Action Effectiveness Study: Second Interim

Report, Part II. Report No. 342.INSTITUTION Johns Hopkins Univ., Baltimore, Md. Center for Social

Organization o. Schools.SPONS AGENCY Department of Justice, Washington, D.C. Office of

Juvenile Justice and Delinquency Prevention.;National Inst. of Education (ED), Washington, DC.

PUB DATE Oct 83GRANT 80-JN-AX-0005; 82-JS-AX-0037; NIE-G-80-0113NOTE 435p.; Some tables are marginally legible because of

small print. For the Second Interim Report, Part I,see ED 237-892; for the First Interim Report, see ED222 835.

PUB TYPE Reports - Evaluative/Feasibility (142)

EDRS PRICEDESCRIPTORS

MF01/PC18 Plus Postage.Art Activities; Delinquency Prevention; DeliverySystems; Dropout Prevention; Educational Counseling;Educational Environment; Educational Improvement;Employment Counselors; *High Risk Students;Intervention; *Nontraditional Education; *Outcomes ofEducation; *Program Effectiveness; ProgramEvaluation; Program Implementation; RecreationalActivities; Secondary Education; *Secondary SchoolStudents; Student Development; Youth Problems

ABSTRACTThis report, the second part of a longer study

prepared by the Delinquency and School Environments Program, furtherdescribes interim results of the program's national evaluation of theOffice of Juvenile Justice and Delinquency Prevention's (OJJDP's)Alternative Education Program. The report consists of evaluations of14 specific projects: (1) Compton Action Alternative School; (2)Project STATUS; (3) Project RETAIN; (4) The Milwood AlternativeProject; (5) Project PREP; (6) The Jazzmobi-le Alternative ArtsEducation Project; (7) Otro Camino; (8) Project PATHE; (9) VirginIslands Alternative Education Project; (10) Academy for CommunityEducation; (11) Alternative Education for Rural Indian Youth; (12)Plymouth Alternative Education Project; (13) Educational ImprovementCenter--South Alternative Education Project; and (14) JewishVocational Services Alternative Education Project. The prOjectsprovide alternative education programs for high risk junior high andhigh school students and several include art, recreation, orvocational activities as well as education programs. Theeffectiveness of the school in promoting student development whilepreventing dropping out and delinquency is evaluated in thesestudies. (JAC)

***********************************************************************Reproductions supplied by EDRS are the best that can be made

from the original document.***********************************************************************

THE SCHOOL ACTION EFFECTIVENESS STUDY:

SECOND INTERIM REPORT

PART II

Gary D. Gottfredson, Denise C. Gottfredson, and Michael S. Cook

Editors

Report No. 342

October 1983

This work is supported primarily by Grants No. 80-JN-AX-0005 and 82-J5-AX-0037from the Office of Juvenile Justice and Delinquency Prevention, U.S. Depart-ment of Justice, and in part by Grant No. NIE-G-80-0113 from the NationalInstitute of Education, U.S. Department of Education. The opinions expresseddo not necessarily represent the position or policy of any agency.

Center for Social Organization of SchoolsThe Johns Hopkins University3505 North Charles StreetBaltimore, Maryland 21218

U.S. DEPARTMENT OF EDUCATIONNATIONAL INSTITUTE OF EDUCATION

EDUCATIONAL RESOURCES INFORMATION

CENTER 'ERIC/LI This document has been reproduced as

received from the person or organizationoriginating it.

7( Minor changes have been made to improvereproduction quality.

Points of view or opinions stated in this docu-ment do not necessarily represent official NIEPosition or policy.

"PERMISSION TO REPRODUCETHISMATERIAL HAS BEEN GRANTED BY

/

TO THE EDUCATIONAL RESOURCESINFORMATION CENTER (ERIC)."

The Center.

The Center for Social Organization of Schools has twoprimary objectives: to develop a scientific knowledge ofhow schools affect their students, and to use this knowledgeto develop better school practices and organization.

The Center works through three research programs toachieve its objectives.

The School Organization Program investigates how schooland classroom organization affects student learning andother outcomes. Current studies focus on parental involve-ment, microcomputers, use of time in schools, cooperativelearning, and other organizational factors. The Educationand Work Program examines the relationship between schoolingand students' later-life occupational and educational suc-cess. Current projects include studies of the competenciesrequired in the workplace, the sources of training_ and _experience that lead to employment, college students' majorfield choices, and employment of urban minority youth. TheDelinquency and School Environments Program researches theproblem of crime, violence, vandalism, and disorder inschools and the role that schools play in delinquency.Ongoing studies address the need to develop a strong theoryof delinquent behavior while examining school effects ondelinquency and evaluating delinquency prevention programsin and outside of schools.

The Center also supports a Fellowships in EducationResearch program that provides opportunities for talentedyoung researchers to conduct and publish significantresearch and encourages the participation of women andminorities in research on education.

This report, prepared by the Delinquency and School Envi-ronments Program, describes further interim results of theprogram's national evaluation of the Office of Juvenile Jus-tice and Delinquency Prevention's (0JJDP's) Alternative Edu-cation program. The first interim results were reported inCSOS Report No. 325, April 1982.

ot

School Action Effectiveness Study Staff

The Johns Hopkins University

Gary D. Gottfredson, Project DirectorDenise C. Gottfredson, Associate Research ScientistMichael S. Cook, Associate Research ScientistDonald E. Rickert, Jr., Research AnalystHelene Kapinos, Data TechnicianLeis Hybl, SecretaryAndrea Nuzzolo, Research AssistantRobert Kirchner, Student Research AssistantAbhijit Mazumder, Student Research AssistantStuart Gavurin, Student Research AssistantNorm Ringel, Community Psychology Intern

Social Action Research Center

J. Douglas Grant, Project AdvisorDeborah Daniels, Research AssociateJane St. John, Research Associate

Federal Project Officers

National Institute for Juvenile Justice andDelinquency Prevention

Barbara Tatem Kelley, Project Officer

Special Emphasis Division, Office of Juvenile Justiceand Delinquency Prevention

Roberta Dorn, Project OfficerCatherine Sanders, Project Officer

Technical Assistance Division. OJJDP

Mary Santonastasso, Project Officer

Technical Assistance

Polaris Research and Development

Noel Day, Project CoDirectorDavid Sheppard, Project CoDirectorMonseratte Diaz, Project AssociateMichelle Magee, Project AssociateJohannes Troost, Project Associate

4

,ble of Contents

Part II: Interim Evalua s of Specific Projects

Compton Action Alternative :hool: Second Interim Report . 143G. D. Gottfredson, D. C. Gottfredson, and M. S. Cook

Project STATUS: Second Interim Report 163D. K. Ogawa, M. S. Cook, and G. D. Gottfredson

Project RETAIN, Chicago Beard of Education:Evaluation Report 197J. St. John

The Milwood Alternative Project: Second Interim Report . . 227M. S. Cook

Project PREP: Second Interim Report 267

D. K. Ogawa and G. D. Gottfredson

The Jazzmobile Alternative Arts Education Project 285D. E. Rickert, Jr.

Otro Camino: Second Interim Report 323

J. St. John

Project PATHE: Second Interim Report 347

D. C. Gottfredson

Virgin Islands Alternative Education Project 393

J. St. John

Interim Quantitative Evaluation of the Academy forCommunity Education 407D. E. Rickert, Jr.

Alternative Education for Rural Indian Youth:Lac Courte Oreilles 415M. S. Cook

Plymouth Alternative Education Project: Second,Interim Report 439M. S. Cook

Educational Improvement Center--South AlternativeEducation Project: Second Interim Report 471D. E: Rickert, Jr.

Jewish Vocational Services Alternative EducationProject: Evaluation Report 505M. S. Cook

Campton Action Alternative School: Second Interim Report

D. K. Daniels and G. D. Gottfredson

Abstract

The Compton Action Alternative School (formerly the Compton ActionCenter for Youth Development, CACYD) is a delinquency prevention demons-tration project sponsored by the Office for Juvenile Justice and Delin-quency Prevention (OJJDP) as part of the OJJDP Program in DelinquencyPrevention through Alternative Education. CACYD has evolved over thecourse of its first two years in ways that appear to have. strengthenedit as a delinquency prevention project. Interim results based on stu-dent self-report suggest that the project has been remarkably effectivein altering a number of student'characteristics that delinquency preven-tion theory implies must be altered to prevent delinquent behavior, andstudent self-reports of delinquent behavior are significantly lower thanthe self-reports of a control group. The self-report data must beinterpreted with caution, however, because of some evidence of differen-tial validity for treatment and control group members. Problems withthe retrieval of some archival data on official delinquency and otheroutcomes limit the assessment reported here. New data have veryrecently become available to strengthen the present analyses, and thisreport should be regarded as tentative. It is subject to revision asnew results emerge.

The Compton Action AlternativeSchool (CACYD) has gone through a

developmental sequence that paral-lels the development its clientsmust go through. That is, just asits clients must first come to seethemselves as students and thenlearn the behaviors associated withbeing students, CACYD first had toestablish its identity as a school.Two years ago the CACYD was a youthand family counseling center calledthe Compton Action Center for YouthDevelopment (hence the acronym) witha mini-school component providingonly short-term services for ComptonUnified School District (CUSD) stu-dents. For the first year it con-centrated its energies on becoming aschool--and it attempted to teachits clients that one of the things astudent does is come to school. In

-143-

6

its second year, the Center placedemphasis on refining its interven-tions designed to achieve affectiveand behavioral objectives, and it

tried to teach its clients the

behaviors associated with being stu-dents, to have confidence in theirabilities in the school environment,and to have confidence in the

staff's concern about their educa-tional progress.'

This second interim report sup-plements an earlier report (Daniels,1982) which should be consulted fora basic overview of this project andan account of earlier activities.

1. In the third year (1982-83) theCenter is emphasizing academics.

CACYD

Programmatic Changes in the SecondYear

The Center's clients are youthswho have not adapted well in a trad-itional school setting. In alllikelihood they were regarded by

their teachers in the public schoolsas "slow" or "troublesome," studentcharacteristics generally regardedas undesirable in public education.The Center's approach to theseclients is to tell them that theycan learn. For those who are slowlearners, instructors provide as

much time and repetition of thematerial as it takes for them to

learn what is being taught. Theyalso aim to provide enough opportu-nities for them to succeed that theycan begin to realize they can learn.This process is intended to leadstudents to come to understand thatit is OK go be slow; and that it isimportant that they can learnnevertheless. To promote thislearning, CACYD implements its con-cept of personalized education,This term means individualized edu-cation with educational aims broadlyconstrued, i.e., not limited to aca-demic concerns but also to theaffective and behavioral domains.The Center assumes that differentmotivators, teaching styles, andmaterials may be most effective fordifferent students; and it assumesthat different interpersonal stylesmay be effective for different stu-dents. 2

2. Unlike the first and secondyears, each student took a batteryof tests in the third year (theGates and California AchievementTest). These tests were used to

place students in achievementgroups, a procedure also not used inthe first and second year.

As a part of the personalizededucational approach, IndividualizedEducation Plans (IEPs) were preparedfor all students by each instructor.The intention was for the plans toinclude an assessment of each stu-dent's skills at entry or the begin-ning of the school year, individualacademic and behavioral objectives,and plans to meet those objectives.The plans were to be revised andupdated every five or six weeks. At

each periodic update, the plans wereto specify whether the student wasmeeting the objectives and to updatethe objectives or the methods andmaterials to be used in the coming

,,period to assist the student in

achieving the objectives. Theinstructor was to inform each stu-dent of the plan. A review of a

sample of these plans showed that

the format did not include space forbehavioral objectives and that aca-demic objectives sought_and_plans_toachieve them often needed to be moreprecise and detailed. Most of theseplans were updated regularly as

planned. Administrators are awarethat additional attention is neededto improve the implementation of

these IEPs. A revised form which issomewhat more specific in the infor-mation requested was introduced nearthe end of the second semester foruse in the third year.

144

Academic credit was grantedthrough the Compton Unified SchoolDistrict. The curriculum included(a) reading, (b) English,(c) mathematics, (d) social studies,and (e) computer-assisted learning.Whereas in the first year survivalskills were included as modules ineach class, in the second year a

separate class was established thatexposed students to tasks such as

the completion of job applicationblanks, letter writing, completionof rental application forms, andunderstanding gas and electricbills. Unlike CUSD schools, whereclasses extend into the afternoon,

at the Center classes are held until1:30 p.m., including half an hourfor lunch.

Interviews with staff, and directobservation, indicate that mostinstructors attempt to individualizeinstrtction, but additional work maybe needed to achieve individualiza-tion in all classrooms. Some staffhave also expressed a desire forassistance in improving their teach-ing methods, and a desire to makethe curriculum more experiential.

The Typical School Day

The typical day starts off with ahalf-hour "morning meeting," or

assembly. This meeting was initi-ated as an incentive to get studentsto school on time (and is alsoviewed as a vehicle for promotingstudent participation and capacitiesto plan and identify problems).Staff members who rotate throughthis responsibility share responsi-bility for running the morning meet-ing with two students. It is gener-ally held as an open forum, wherestudents debate, make presentations,and plan for daily events. Issuesto be taken up by the studentgovernment are developed in themorning meetings. Students plannedand implemented several activitiesthis year that were based on ideasthat arose in these morning meet-ings. The largest such undertakingswere a prom and graduation whichthey initiated and helped to plan,and a week of debates known as "TheGreat Debate." All students andstaff members participated in thesedebates. The topics debated were"Decriainalizing Marijuana," "Pre-Marital Sex," and "The Criminal

3. In the third year a studentgovernment has assumed the responsi-bility for running the moriningmeeting.

-145-

CACYD

Justice System."

After the morning meeting, stu-dents begin their academic coursework. Some, for instance, beganwork on tLe Plato (computerizedinstruction) system. The on-linePlato system used in the first yearwas replaced with micro-Plato whichuses discs. This smaller systemoffers fewer programs than the lar-ger system, but eliminated problemswith phone lines that were experi-enced in the first yeai. A teachersupervises this activity, supple-menting work on the computer withclassroom instruction and practice

yin work books. Students spend timeworking with PLATO every day.

Teaching Approach

No set formula for instruction isprovided for staff (three full-timeinstructors, two half-time instruc-tors, and a teaching aide), but gen-eral guidance has been provided.Four elements of effective teachinghave been communicated to the staffby the project director:(a) "direct instruction," (b) timeto exercise the skills to be tran-smitted, (c) time spent in reviewand exercise of the skills, and(d) testing to check for mastery.In addition, several training ses-sions for the staff in the Kawaidamethod were held. The methodappears to be directed to enhancingself-esteem, identity, planfulness,and responsibility on the part of

students by encouraging instructorsto provide opportunities for themastery of skills, recognition andreinforcement of success, and accep-tance. The method was developed byGary and Maye Beale who drew heavilyon an unpublished dissertation by

Joseph White. The materials we haveseen seem to draw heavily on thepopular writings of Glasser (1969;1975).

CACYD

Additional Educational Features

Several kinds of activities occureither periodically or occasionallyto supplement the day-to-day rout-ine:

Career day. The Center's commu-nity advisory committee helped spon-sor a career day in which a numberof professionals, including a policeofficer, came to the school to dis-cuss their work with the stu' nts.The staff felt an important featureof this career day was that it wasthe first time that many profession-als had an opportunity to talk toso-called "bad" students in a posi-tive setting, and vice versa.

Informal student-staff inter.c-tion. As an approach to providingrewarding social interactions bet-ween students and teachers that arenot provided in traditional schoolstructures, staff are expected to

spend time with students by engagingin non - academic interaction. Anexample of this form of interactionis the potluck lunches and dinnersinvolving parents and students.

Formal interaction. Field tripsto an amusement park, a museum, somelocal colleges and universities, andto sports and entertainment eventswere carried out.

A partial token economy. Nearthe end of the second year, the Cen-ter began to implement a partialtoken economy system. Studentscould earn tokens for class atten-dance and participation that couldbe redeemed in a recreation area toplay pool or engage in other activi-ties.

Student government. A studentgovernment was formed with eightelected members. The governmentinitiated ad hoc committees involv-ing other students to plan and

implement activities such as a fund-raiser for a field trip and therevision of roles that govern stu-dent behavior,

Maintenancloyment_ activity.As one way to help solve a graffitiproblem during the second semester,two students rotated through jobs tokeep the Center clean. Studentsmust sign in, and they are expectedto learn conventional work habits.They are paid for two hours work andvolunteer for tWo hodrs of work.The students bolding these jobs arerecommended by the staff, and thiswork assignment Is sometimes used as

,a reward for achieving a goal or asan incentive to improve when a stu-dent is not achLeving goals.

-146-

Mentors. Attempts were madeagain to implement the mentor Pro-gram that was unsuccessful in the

first year._ Agaih,this attempt wasnot totally successful, althoughsome community involvement wasachieved. Otte UCLA student con-ducted two projects with students,but attempts to recruit mentors fromother closer colleges failed, par-tially because of cutbacks in workstudy funds and Partially because ofa reported lack of interest on thePart of the college students. Priorto cuts in CETA funding, some CETAworkers were employed by the Centerto work with ytudents in the mentorrole. Although students apparentlyrelated better to CETA workers thanto College students, and studentsundertook projects such as con-structing Easter baskets for disa-bled children, funding cuts pre-vented this activity from

continuing.

Task orientarion and atmosphere.Two additional features of the edu-cational environment were salient instudents' responses to an interviewquestion. When asked what theywould tell someone about the Center

9

if asked to describe it, the mostcommon response was that studentshave to get their work done every-day. Student responses to interviewquestions also emphasized that theyfelt good about themselves and thestaff, that they felt relaxed, as

though they could learn, and thatthey were sure the staff was con-cerned about them learning.

Parent and Community Activities

Parents. Formal parent trainingwas planned, and six workshops wereheld. The topics covered were "Howthe CUSD Functions," "A CandidatesForum for Persons Running for theCUSD Board of Trustees," "ParentingAdolescents," "Preventing Drug andAlcohol Use by Teens," "Gang-O-riented Youth," and a "Family After-noon." The last of these was apot-luck meal shared by parents,students, and staff at which theworkshop series was reviewed. Theresponse on the part of participantswas reportedly favorable, but onlysix or seven parents participated inall the workshops. (The first twoworkshops drew approximately 35 par-ticipants.) Considerable parentcontact and involvement was achievedin other ways. This included commu-nication with parents when studentswere absent, a parent council, andparent participation in Centeractivities.

Parents were contacted by tele-phone when students were absent.Although this is a difficult activ-ity to conduct, and although manystudents of the Center live in non-traditional families, this was a

regular and recurring activity.

A seven-member parent councilheld meetings about once per monthduring the year. This council wasprimarily a discussion and supportgroup for parents.

The project manager has observed

CACYD

that parents seem to respond betterto requests to participate in schoolactivities than to requests to

attend meetings. Accordingly,parents have been encouraged toengage in activities such as thecareer day and a car wash. This

appears to be a fruitful approach,and suggests that parent involve-ment in school activities might befostered by providing them withmeaningful roles in the conduct of aschool's activities. An attempt hasalso been made to -get parentsinvolved in the student contractsinvolving educational plans, butthis has not met with great success

,nor been attempted as regularly ashad been envisioned.

Community. A 13-member communityadvisory council has been formed.It includes a Compton attorney,business persons, representatives ofthe probation and narcotics paroledepartments, a clergyman, a Congres-sional staffer, and two students.We have not interviewed any of theseindividuals about their roles, butthe function of the committee is

reportedly to give advice, to pro-vide an indication of community sen-timent, and to help develop goodwill for the Center and help it tosolve problems.

The Center encouraged informalcommunity participation in otherways. For example, a community mem-ber dropped in, volunteered to

tutor, and did so for a number ofweeks. And a local liquor storeowner contributed to and partici-pated in student-sponsored activi-ties.

Other Project Improvements

Staff turnover was high in theCenter's first year. Staffing wasstable in the second project year.One teaching aide was dismissedbecause she was associated with a

-147-

10

CACYD

gang, and the Center wanted to beperceived as neutral with respect tocompeting gangs in the community.Quarterly staffing reports showthat, exclusive of CETA workers, theproject had 8.4 full-time equivalentworkers through May, 1982, with noturnover during the year until thattime. TLe project staff decreasedby one person in May, one full-timeand one 70% time ,worker June, andone person (the principal) in

August, 1982. Two full-time CETAworkers were lost in May when theirtime allotment in the programexpired.

The Center has engaged in plan-ning using the PDE method more fullyin its second year of operation. It

has developed its activities accord-ing to a more fully specified plan,and it has involved more staff mem-bers regularly in the formal plan-ning process. Although all staffmembers do not view the PDE methodas equally useful, nor do all use itin equal amounts, most staff membersknow about the Center's PDE plan andwere able to explain how planningfollowing this method works.

Difficulties with on-site evalua-tion activities experienced in the

first year have ceased, and the on-site evaluator is doing an excellentjob of implementing evaluation-re-lated activities.

Progress Towards Institutionaliza-tion

/

We define the institutionaliza-tion of a program as a condition inwhich no special effort is requiredto maintain its operation as a partof the status quo, and in which con-siderable effort would be requiredto end or modify the program. By

this definition, the Center is

clearly not yet institutionalized.It appears to be making progresstowards institutionalization in thesense that obstacles to

its operation are decreasing somew-hat and it is acquiring some of thetrappings of an institution.

The only noticeable opposition tothe Center's activities has comefrom the police who feel that addi-tional security is required for theCenter to remain in operation in itspresent location. (There was oneserious gang intrusion into the cen-ter this year.) The Center's advi-sory committee has been of some helpin interacting with the policedepartment. In addition, the Cen-ter's participation together withthe police in the. District's Clean

,Sweep Program has increased respectfor the Center on the part of thepolice, as well as other communityentitites. One CUSD school princi-pal has shown some opposition, butthis is not considered serious.

-148-

In the second _year, the_Center'sclients were carried on the CUSDrolls and received lunches through aCUSD program. There are additionalsigns that the Center's relationswith the district are becoming morepositive. Project managers now knowmore key people in the district andthe new acting superintendent (AS)is supportive of the Center'sefforts. An interview with the ASimplies that he views CACYD verypositively and sees the Center asworking closely with his schools.The Center's manager is a member ofhis Blue Ribbon Committee on SchoolChange (chairing its subcommittee oneducational change), and has beenactive in conducting surveys in CUSDschools intended to be helpful in

planning programs to prevent CUSDteachers from alienating students.The Center's principal is a memberof the district's disciplinaryreview committee. The AS plans toseek CACYD assistance in implement-ing and monitoring CUSD's efforts toincrease individualization in

instruction.

11

The Center managers and the ASboth see the Center as an entitythat should operate outside the

public school system. The Center isintended to have a role of its ownbut to operate in cooperation withthe CUSD to accomplish some commongoals. Although the CUSD has pro-vided some services and furniturefor CACYD there are no plans for thedistrict to provide any substantialfinancial support. The AS reportsthat he would help with funding ifhe had the funds, as it is he seeshis role as helping the Centerobtain outside funding.

Decreasing funding through theOJJDP grant may influence the pro-ject's ability to operate. Further-more cuts in work-study funding forcolleges in the area, and cuts in

CETA, have decreased the availabil-ity of low-cost supplementary work-ers. The poor economic climate anddecisions to reduce the Federal rolein the support of social programswill almost certainly work to reducethe likelihood that the Center willachieve institutionalization.

Interim Summative Evaluation

Problems experienced in imple-/menting the evaluation in the firstproject year have been resolved.Project personnel have zea-lously pursued evaluation activitieswith the result that the evaluationdesign is strong and has been wellimplemented. In all 80 studentsparticipated in the Center duringthe 1981-82 school year. Of these48 (60%) were still in attendance atthe end of the school year.4 Attri-tion is accounted for as follows:

4. Our best estimate of the percen-tage of participants who persisteduntil the end of the 1980-81 schoolyear is 58%.

-149-

CACYD

involuntary withdrawn (primarilyincarcerations), 15% of partici-pants; left Compton, 12%; voluntarywithdrawal (dropout), 4%; return toregular school, 4%.

Of the 91 students for whom wehave data who participated in CACYDin the first year, 41 (45%) returnedto the Center at some time duringthe second year. Of these 41 retur-nees, 66% were still in attendanceat the end of the second year, 22%were involuntary withdiawals (pri-marily. due to incarceration),5% left Compton, 5% voluntarilywithdrew, and 2% returned to a regu-lar school. This pattern of studentattrition implies that few studentsare voluntarily dropping out ofschool, and the largest source of

attrition is incarceration of stu-dents in this high-risk population.

Methods

Assignment to treatment and con-trol groups. Except for returningstudents (N = 41), students eligiblefor participation were assigned to

treatment or to an untreated controlgroup. Because of pressure from.OJJDP to rapidly increase the numberof participants in the school, how-ever, assignment was not straight-forward. Pressure to rapidly bringthe number of students served up toan acceptable level required thefollowing procedure: Between the

start of the school year and 14

December 1981, a computer generatedlist of pseudorandom numbers wasprovided to the on-site evaluationcoordinator. The list containedlines a random 80% of which carrieda code for treatment condition andthe remaining 20% a code for controlcondition. As youths became eligi-ble for the program, their nameswere alphabetized and written in

that order on the list. This proce-dure was i-tended to randomly assign80% of el 7e youths to treatment.

12

Some Ito Coster woe approachise thementor geettetleste required byShe ersomser. the esti* of treatises*to seletrel osetammeete om the listMtge rowersed=00% were aseigmed tothe owed grow nand 202 to the441104,404 gO0414 Is ell, $ teethewere smetgood from a pos4 et ellgti.Mae to perttsipste to the %eater.and 14 tote deetgeel to as seltreatedisootrs4 grasp Swim the first044410MOAt period described above, atotal et 23 paths wore wiped totreeivemt had 10 palm wereetstemod tae the mourn group, %r-ime the wooed aseteemeet period, 14paths gore aesigmed to treatises'anus 44 tore asetsmod to antral.Ibie depertwee free stool* realmesoLgomeist complicate* the deskseamewhet4

4041451112k0 toll to to oil::

4441004 t4 04f the peel 44 eligibles*my here differed be early andlate aseigameet portfolio. (Theseeherhe wee mesoosety beams* treat-meet Mateo to of comae associatedwith ported of assignmeet.) Tarte-hlee so ibteh molter end laterspeep* differed, eat me treatment

tem, ere give* (with g levelsto preethesee) to the folly/lagWets withdtawel from school (.02),Wise 6004$444 ditteth000t totweets (.00), SOW Wert (.05),when amino; (.05), Parental %de-salt** (.fl), a somposite lodes ofSibeel Ormatteadsoce that is par-

ttelly redeadoot with *then undue(454 Sehelliees Mom* (.0%),sod lavolvemoot (.01). fa plot*

iloglish, the pool el eligibles dif-fered is impertaat weys between theearly mod late *Mesons period.This mess that valets the assign -as period is Wife into account,treatimmet and mitre' groups wouldbe etas to differ due to thelays they were emaposed is additionto aoy differeesce due to trestmext.

-ISO-

A difference was also found

beton* tteatemot god control groupsfa umbers of police contacts bet-imes 16 September 19110 amd 22 August

(i.o., the year Woe assign-meet). The trestmeat group (N 38)

had a mese of .16 police contacts inthat year sod the control group (X32) bad a mesa of .60 police con-tacts. Treatment students had feverprior police contacts than controlyouths in both the early and lateassigemest periods, and the differ-ence is statistically significant (et .004) eve* when assignment periodis statistically held constant. Aclose examination of the diatribu-,tlone implies that this differencecan mot be accounted for by outl i-ers. for the treatment group thenumber of prior police contacts

rafted from 0 to 4, with 55.3% hav-ing no prior contacts Le the periodexamined. For the control group the*ember , of , prior, police _contactsranged from 0 to 7, with 47% havingso prior contacts in the period exa-mined. The treatment and controlgroups differed somewhat in ethnicconvention but this differenceappear, due to a difference in thecomposition of the eligible pools inthe early and late assignment por-ted.. The treatment group was 78%Stack, and the control group was 91%black. This difference is not sig -eificant when assignment period isstatistically controlled.

We do not know to what to attri-bute the observed difference in

pre - intervention police contacts.

One possibility is that insufficienttime was alloyed to elapse betweenthe dam a police contact may haveoccurred and the tine the police

records were searched for evidenceof arrests. It is possible that

because a disproportionate number ofcontrol youthi entered the experi-Mont in the late assignment period,more of their arrests were reflectedin the data base because a longer

13

period of time had elapsed betweenthe arrest and the time the fileswere searched. On the other hand,the difference was observed for

early and late assignment periods.We will continue to explore this

issue and conduct new searches of

the police contact data base tocheck on this hypothesis.

Analyses. To assess the effectsof participation in the Campton project in the most direct way possible, a comparison was made of students who were assigned to

participate and the control grouptaking into account the assignmentperiod during which they entered theexperiment.5 Where deemed appropriate, statistical adjustments werealso made for prior police record.

Measures. A single instrument,the School Action EffectivenessStudy Student Questionnaire was usedto collect information on both

treatment and control students.

5. In technical terms, the approximate equivalent of a 2 by 2 factorial design with unequal cells wasaccamplished using the SPSS ANOVAprocedure with option 10 on release9 of that package (Nie, Hull, Jenkins, Steinbrenner, & Bent, 1975;

Hull & Nie,, 1981). This procedureinvolves a regression approach,

where the statistical test for maineffects involves the incrementalcontribution of each factor net ofprior factors (in this case the

assignment period during which a

person entered the treatment or control groups) and covariates (prior

police record when appropriate).The interaction effect (treatment byassignment period) is tested aftermain effects are entered into the

model. If an interaction effectsignificant at the p = .10 level orless was found, data were analyzedseparately by period. It was neces-

CACYD

This instrument is described elsewhere (Gottfredf,on, Ogawa, Rickert,& Gottfredson, 1982), and readersshould consult that description foran account of some of its psychometric properties. This questionnairewas administered by the onsiteevaluation coordinator in smallgroups or individually as necessary.The administration was closelysupervised and resulted in high response rates' for both treatment andcontrol groups, and a low percentageof item nonresponse. Control groupmembers were offered $5.00 for coming to a questionnaire administration. A total of 74% of treatment

,and 87% of control students wereadministered the questionnaire.

-151

In addition, the onsite evaluation coordinator collected several

other pieces of information fromschool and police records. This

information included police recordsand school attendance and withdrawalinformation. The measures examined

sary to search for a main effect oftreatment only after adjusting for

assignment period because assignmentperiod was significantly associatedwith a number of outcomes even whentreatment was "controlled" out. Weregard this procedure as a conservative test of treatment effectsbecause, as a practical matter, theperiod during which students are

assigned to treatment is associatedwith the duration of exposure to

treatment. Thus, applying a statistical control for period of assignment tends to work against findingtreatment effects if effect siie"is---associated with duration of exposure, as might be expected. Dependable differences associated withboth treatment condition and period of assignment have been assignedto period in the analyses. This is

unfortunate, but protects the

believability of the statistical

14

CACYD

in this report are briefly describedbelow:

Withdrawal from school. Withdra-wal from school, either voluntary orinvoluntary, is coded 1; any otherobserved outcome (left Campton,returned to school, or persisted inCACYD) is coded 0.

Any police contact. Any evidenceof police contact for any offenseduring the period 16 September 1981through 1 August 1982 according to asearch of arrest records in theHall of Records auto-mated information 'reftieval systemis coded 1. No evidence of policecontact during that period is codedO. Access to these records wasgained through a court order.

Number of alleged offenses. Thisis the total number of offensesalleged during the period 16 Septem-ber 1981 through 1 August 1982.

Attachment to parents. This is aquestionnaire-based measure of

"attachment," i.e. feeling close andwanting to be like parents. It isintended to measure an element of

Hirschi's (1969) social bond.

Negative peer influence. This isa scale based on student reportsthat his or her friends get into

conclusions drawn.

Multivariate analysis of variancewas precluded because of excessivemissing data. Our inability to pro-tect the univariate tests by per-

forming multivariate, analysis of

variance first means that some dif-ferences may be expected by chancealone. We interpret_univariate dif-ferences nonetheless, and this len-iency seems to us to counterbalancethe conservative handling of nonor-thogonality in the design.

trouble, do not like school, etc.

Attachme4 to school. This is ascale based on student reports thatthey like the school. It is

intended to measure an element of

Hirschi's (1969) social bond.

Belief in rules. This is a scalebased on student reports that takingadvantage of others and breakingrules or laws is not OK.

Interpersonal competency. This

is a questionnaire-based measure ofpsychological health based on Hol-land and Baird's (1968) scale with

.,the same name.

-152-

Positive self-concept. Thisscale measures students' self-esteemand conception of themselves as pro-social, law-abiding citizens.

Self-reported delinquent behav-ior. This scale is based on 19

items asking whether the respondenthas committed various kinds of del-inquent behavior during the pastyear.

Self-reported substance use.

This is a subset of the previousscale, limited to the use of drugs,alcohol, and inhalants.

Self-reported serious delinquentbehavior. This is a subset of theself-report delinquent behaviorscale limited to the most serious ofthe offenses in that scale.

Schoolpunishment. This scale isbased on students' reports that theyhave been punished in various waysin school for their recent school

behavior.

School rewards. This scale is

based on students' reports that theyhave been rewarded in various waysin school for their recent schoolbehavior.

Victimization. This scalemeasures recent in-school victimiza-tion experiences.

Invalidity. This scale is

intended to detect nonveridical res-ponding. Reporting that one readsseveral whole books every day andthat it is easy to get along withnasty people earns a high score.

School effort. This is a questi-onnaire-based measure of effortexpended at school work.

Practical knowledge. This is a

7-item self-report measure of know-ledge of several life competencies,such as how to balance a check book,arrange a trip out of town, or applyfor a job.

Alienation. This is a self-re-port measure of feelings of

estrangement from the social order.

Self-reported grades. This sin-gle-item measure is self-explana-tory.

Self-reported reading ability.This single-item measure is self-ex-planatory.

Educational expectation. This isthe response to a questionnaire itemasking how far the person expects togo in school.

Work for pay. This

scored "1" if the

reported that he or shepay during the past week,the respondent indicatedshe did not.

measure is

respondentworked forand "0" ifthat he or

Regular bob. This indicator is

coded "1" if the respondent indi-cated that he or she holds a full-or part-time job, and it is coded"0" if the respondent indicates thathe or she does not.

-153-

Suspension.self-report of

school during theterm.

CACYD

This is a

suspension fromcurrent (Spring)

School non-attendance index.

This is a two-item measure of classcutting and school cutting.

Rebellious autonomy. This scaleis intended to measure an immatureattitude of autonomy. A high scoreis earned by reporting that it is noone's business how the respondentspends his or her money, for exam-ple.

Involvement. This is a question-naire-based measure of participationin same common school activities.

Results

Treatment-control group compariLsons are presented in Table 1. Of

25 comparisons based on question-naire measures, 18 significantlyfavor the treatment group.6 One ofthese significant differences favorsthe control group. In one case a

statistical interaction was found,

meaning that the results differdepending on whether the youths wereassigned to treatment or controlconditions in the early or lateassignment period.

6. These significance tests are notindependent. Indeed, some of themare largely redundant (specificallythe three self-report delinquencymeasures, two of which are based onsubsets of items in the overall mea-sure). Therefore the significancelevels shown in the table should beregarded as nominal rather than

actual levels. Some of these signi-ficant differences may arise due tochance alone.

16

CACYD

No significant difference occursfor any of the measures based onarchival evidence shown in Table 1

(withdrawals, police contact, numberof police contacts, suspensions, or

grades).7 The treatment and controlgroups were vastly different in

officially reported suspensions,however. CACYD reported only sixsuspensions in the entire year. In

contrast, 81.2% of control groupyouths were suspended at least onceaccording to school records.(Because of differences in data col-lection methods this result is nottabled, and we have not calculated asignificance test, but none is

necessary. The difference is strik-ing.)

A threat to interpretability.The one significant differencefavoring the control group presentsan important threat to the interpre-tation of the questionnaire results.Treatment students scored higherthan control youths on the invalid-ity scale, a measure of non-veridi-cal reporting. Accordingly addi-tional checks of the validity of theself-report measures were made sepa-rately for the treatment and controlyouths. For the control group,the self-reported delinquent behav-ior scale correlates .54 (N = 34, 2< .001) with the number of policecontacts in the 81-82 period. Forthe treatment group, this corela-tion is -.01 (N = 23, n.s.).° This

7. There is reason to believe thatsome errors may have been made thatled to an underestimate of the num-ber of withdrawals for control groupyouths. We are continuing to

explore the completeness of the datain this regard. We are unable to

compare treatment and control groupson archival attendance data at thistime. We are also continuingattempts to explore these data.8. The N's vary due to missing

result implies that the credibilityof the self-report delinquency mea-sure ia,highly questionable.

Several other checks on the val-idity of self-report for the treat-ment and control groups are not

reassuring. These checks are sum-marized in Table 2. Percentage ofdays in attendance calculated fromarchival data submitted for CACYDparticipants for the final quarterlyreporting period by the Center cor-relates .31 (N = 29, Q < .10) withthe School Nonattendance index basedon self-report. By comparison,total days of absence during the

,year based on CUSD records corre-lates .23 (N = 41, n.s.) with theSchool Nonattendance index. Self-reported reading ability correlates-.07 (N = 29, n.s.) with scores ona reading test the project used toassign students to levels in the

Plato system and_.05_(N = 24) withCACYD grades. No reading test wasavailable for the control group, butself-reported reading ability corre-lates .16 (N = 36) with CUSD gradesfor this group. Self-reportedgrades correlates with gradesaccording to official records -.21(N = 25) and .10 (N = 37) for treat-ment and control groups, respec-tively. Finally, CACYD reported noadministrative removals at all dur-ing its last reporting period, but28% of CACYD participants (N = 32)reported having been suspended dur-ing the last school term. It is not

data. Due primarily to selectivenon-response to these particularitems, delinquent behavior self-re-ports were available only for 59% oftreatment students (N = 39) and 64%of control group members (N = 54).This difference is nonsignificant.In contrast, police records wereavailable for almost all treatment(97.4%) and control group (98.1%)youths.

-154-

possible, of course to calculate acorrelation when there is no varia-tion in one of the variables. Of

control students, 81.2% (N = 48)

were suspended at some time duringthe 81-82 school year, and 78% (N =45) reported having been suspendedduring the last school term. Thecorrelation between these two mea-sures for the control group was only.04 (N = 43, n.s.).

In short, the evidence supportingthe validity of self-reports forboth treatment and control groups,but especially for the treatmentgroup, is weak. In the case of sus-pensions, both self-report and

archival data converge in implyingmuch lower suspensions among theCACYD participants, although the

level of suspensions implied by thetwo data sources differs.

DiscussionResults

of Interim Summative

According to the questionnairemeasures, participation in CACYD hasremarkably positive outcomes. The

results based on information from

archives are not so positive.

Limitations. Several limitationsare important in assessing theseresults. The first, and most obvi-ous, is that the administration ofthe questionnaire by a member of theCACYD staff may have created a sub-tle and unintentional but powerfuldemand for treatment youths to pre-sent themselves in a positive light.This demand may have been absent orless powerful for control groupyouths. Evidence suggests that thisis a plausible interpretation of theresults. Treatment students' scoreson the Invalidity Scale were signi-ficantly higher than control youthsscores. This scale is intended tobe sensitive to careless respondingor to an attempt to fake good. The

elevated scores do not seem to bedue to careless responding--visual

CACYD

inspection of the questionnairebooklets imply that they were care-fully filled out and few items areskipped or marked in obvious pat-terns. Furthermore, the differen-tial validity found for treatmentand control groups for question-naire-based measures for which thereexists a parallel in the archival

data accords with a hypothesis thattreatment students attempted to

create a good impression or other-wise responded in nonveridicalways.9 And, the questionnaires wereadministered by a member of the Cen-ter staff who had frequent contactwith the students and whom they may,not have wanted to displease. On

the other hand, respondents fakinggood might be expected to havereported fewer school punishments.The opposite result was found:Scores on the School PunishmentScale are significantly higher forthe treatment group. These,, and anumber of other interpretations, arepossible. CACYD staff, for examplehave suggested an explanation forthe low correlation between actualgrades and self-reported grades thatinvolves the way student perceptionsof their performande vary over time.Until further work is performed,many such interpretations are possi-

. ble. On balance, the self-reportresults should be interpreted withcaution.

-155-

9. Hindelang, Hirschi, and Weis

(1981) found differential validityfor self-report measures similar tothose examined here in their studyof Seattle youths. Specifically,they found near zero validity coef-ficients for Black males who wereofficially delinquent. The patternfound here suggests limited validityfor the control group (many of whomwere officially delinquent) and evenlower validity for for the treatmentgroup.

16

CACYD

A second limitation stems fromthe finding that treatment and con-trol youths differed significantlyin the number of police contactsduring the year prior to the studyperiod. The same pattern (controlyouths being more officially delin-quent) holds for both the early andlate assignment periods. This out-come could occur by chance givenrandom assignment, or it could be aproduct of a breakdown in the ran-domization process."

A third, and less threatening,limitation is due to the assignmentof different propOrtions of eligi-bles to the treatment condition atdifferent time periods. Despite theapplication of a statistical controlfor period of assignment, becausethe composition of the pool of eli-gibles'differed between the two timeperiods some residual spurious asso-ciation between youth characteris-tics and outcomes may exist in theresults. An examination of treat-ment and control means separatelyfor each time period suggests thatthis threat to interpretation is nothighly plausible as an explanationof the results.

10. The application of a statisti-cal control for prior police recordsdoes not alter the statistical con-clusion: net of assignment periodand of prior police record treatmentyouths report fewer kinds of delin-quent behavior than do controlyouths. Police records are not,however, stable covariates. Thecorrelation of 80-81 police contactswith 81-82 contacts is low. Theapplication of a statistical controlfor prior grades does not alter thestatistical conclusion: net ofassignment period and of priorgrade-point average, treatmentyouths report receiving highergrades.

Implications. Although the

interpretation of the self-reportresults is hazardous, the interimsummative results based on theself-report measures suggest notonly that CACYD participation hasbeen remarkably effective in pre-venting delinquency, but also sug-gest some explanations of its effec-tiveness. Several known correlatesof delinquent behavior were affectedby the CACYD treatment. Theseinclude belief in rules, attachmentto parents and school,"self-concept(esteem), and involvement (measuresof the social bond according tosocial control theory); negative

,peer influence (a measure of differ-ential association); and schoolgrades. Furthermore, the greateramount of school rewards and punish-ments reported by treatment studentssuggests that supervision and theconsequation of behavior may be a

major way in which_ these _outcomescan be achieved. Although sitevisit observations and interviewswith participants are by themselvesgenerally untrustworthy sources ofevidence, observations and inter-views with students participating inthe program nevertheless lend sup-port to this interpretation. Like-wise, observations of phone calls toparents when students are absent andof deliberate attempts of the staffto strengthen interpersonal tieswith the students also support thisinterpretation.

-156-

Discussion of Other Observations

Our observations suggest someways the project might be strength-ened. First, the individualizededucation plans we examined can bestrengthened greatly by (a) makingthem more concrete and specific,(b) reviewing and updating them morefrequently, (c) providing specificconsequences for meeting or failingto meet short-term objectives,(d) ensuring that the objectives andconsequences are clearly understood

19

by students, staff, and parents orguardians.

Second, although the principlesof effective teaching suggested bythe project manager accord with someliterature on effective teaching(e.g., Brophy & Evertston, 1974;Crawford & Stallings, 1978; Gage,1978) a formal structure for ensur-ing that these principles are fol-lowed precisely should be imple-mented to strengthen the academiccomponent.

Third, observations and inter-views with staff suggest thatimprovements in curriculum andteaching technology could strengthenthe academic components. The use ofwell engineered educational technol-ogies such as DISTAR (Becker & Car-nine, 1980; Becker & Gersten, 1982)or various approaches to increasingthe effectiveness of classroomreward structures (Slavin, 1977,1982, in press) could add strengthin this area.

Fourth, the intiation of a par-tial token economy appears to be auseful step. This step could be

more fully elaborated along thelines developed in Achievement Place(Wolf et al., 1972; Levitt et al.,1981).

Brief Summary of Achievements in theSecond Year

1. Staffing was stable duringthe second year, with no person

/ leaving the Center until May of thatyear.

2. Of students who participatedin the Center during the 1981-82school year, 60% were still enrolledat the end of the school year.

3. The collection of informationby the on-site evaluator was tho-rough and timely.

-157-

CACYD

4. The relationship between theCenter and the Compton UnifiedSchool District was strengthened.

5. The Center began the proce-dure of starting its school day witha youth initiated activity--a crea-tive and interesting innovation.

6. Student reports imply thatthey felt positive about the alter-native school and that they knewthey were expected to learn.

7.. Warm interpersonal relationsbetween students and staff weredeveloped.

8. Despite serious problems withgangs in the Compton community, stu-dents from rival gangs generallyco-existed peacefully in the Center.

Recommendations

Our observations and analyses todate suggest the following recommen-dations:

1. Greater use of previouslydeveloped educational interventionsand curricula to increase learningwould probably strengthen the pro-ject's academic component.

2. The development of additionalstrategies to retain clients in theprogram for a longer period of time.

3. More active steps to securesponsorship for the Center in thefuture must be, taken if it is to

continue to evolve.

4. Administration of studentquestionnaires in the Spring by a

person unknown to the respondents,especially for the treatment group.

5. Closer replication of tokeneconomy systems developed and testedelsewhere to strengthen this projectcomponent, if it is to be continued.

20

CACYD

6. Strengthening theindividualized education plans bymuch more frequent review andgreater specificity to increasetheir faithfulness to the character-istics of such plans described ear-lier.

7. In future efforts in thisarea, making arrangements in advancesuch that demands from a sponsor for

service provision do not conflictwith the demands of an orderly eval-uation.

8. In future efforts in thisarea, making arrangements for ran-domization to be conducted off-sitewithin a feasible time line, andallowing a longer period of time toelapse before searching archives forarrests.

-158-

21

CACYD

References

Becker, W. C., & Carnine, D. W. Direct instruction: An effective approach toeducational intervention with the disadvantaged and low performers. In B.B. Lahey & A. E. Kazdin, Advances in clinical child psychology (Vol. 3).New York: Plenum, 1980.

Becker, W. C., & Gersten, R. A follow-up of follow through: The latereffects of the direct instruction model on children in fifth and sixthgrades. American Educational Research Journal, 1982, 19, 75-92.

Brophy, J. E., & Evertson, C. M. Process-product correlationsin the TexasTeacher Effectiveness Study: Final report (No. 75-21). Austin, Texas:Research and Development Center for Teacher Education, University of Texas,1974.

Crawford, J., & Stallings, J. Experimental effects of in-service teachertraining derived from process-product correlations in the primary grades.Paper presented at the Meeting of the American Educational Research Associ-ation, Toronto, March 1978.

Daniels, D. K. Compton Action Center for Youth Development: Interim report.In G. D. Gottfredson (ed.), The School Action Effectiveness Study: Firstinterim report. Baltimore: The Johns Hopkins University, Center forSocial Organization of Schools, 1982.

Gage, N. L. An experiment on teacher effectiveness and parent-assistedinstruction in the third grade: Background and rationale. Paper presentedat the Meeting of the American Educational Research Ass6ciation, Toronto,March 1978.

Glasser, W. Schools without failure. New York: Harper & Row, 1969

Glasser W. Reality therapy: A new approach to psychiatry. New York: Harper& Row, 1975. (First published in 1965.)

Gottfredson, G. D., Ogawa, D. K., Rickert, D. E., Jr., & Gottftedson, D. C.Measures used in the School Action Effectiveness Study. In G. D. Gottfred-son (ed.), School Action Effectiveness Study: First interim report. Bal-timore: The Johns Hopkins University, Center for Social Organization ofSchools, 1982.

Hirschi, T. Causes of delinquency. Berkeley, Calif.: University of Califor-nia Press, 1969.

Hindelang, M., Hirschi, T., & Weis, J. Measuring delinquency. Beverly Hills,Calif.: Sage, 1981.

Holland, J. L., & Baird, L. L. An interpersonal competency scale. Educa-tional and Psychological Measurement, 1968, 28, 503-510.

-159-

CACYD

Hull, C. H., & Nie, N. H. (eds.). SPSS update -?721AXPAcedures and facilities for releases 7-9. New York: McGrawBi11, 1981,

Levitt, J. L., Young, T. M., & Pappenfort, D. M. Alievesiet_..qs1.ace:Theclteachin famil treatment model in a rou home setti.gg (Reports of theNational Juvenile Justice Assessment Centers). Washington, D.C.: NationalInstitute for Juvenile Justice and Delinquency PreventioA, Office of Juvenile Justice and Delinquency Prevention, 1981.

Nie, N. H., Hull, C. H, Jenkins, J. G., Steinbrenner, K., & Bent, D. H. Statistical acka e for the social sciences (Second edition). New York:McGrawHill, 1974.

Slavin, R. E. A student team approach to teaching adolegeetIts with specialemotional and behavioral needs. .PsysilsIcla_kn the Schools, 1977, 14,77-84.

Slavin, R. E. Critical dimensions of clapsruom organization: A strategy forresearchbased instructional improvement. Baltimore: Johns Hopkins University, Center for Social Orgnization of Schools, 1982. (An earlier version of this paper was presented at the annual meeting of the American Educational Research Association, New York, 1982.)

Slavin, R. E. Cooperative learning. New York: Longman's, in press.

Wolf, M. M., Phillips, E. L., & Fixen, D. L. The teaching Anewmodel for the treatment of deviant child behavior in the community. In S.W. Bijou & E. L. RibesIresta (eds.), Behavior modification. New York:Academic Press, 1972.

160

Table 1Overview of Tests for Differences between Treatment

and Control Conditions

Criterion Differencevariable favors F df

Archival sourcesWithdrawal control 0.08 1,67 .78 nsAny police contact control 0.23 1,89 .23 nsNumber ofpolice contacts

treatment 2.69 1,87 .10 as

Grade-point average control 0.84 1,89 .36 nsSelf-reportAttachment to parents treatment 6.84 1,68 .01 **Negative peer infl. treatment 0.27 1,68 .6O nsAttachment to school treatment 46.81 1,74 .001 **Belief in rules treatment 5.30 1,70 .024 *Interpersonalcompetency

treatment 5.60 1,72 .021 *

Positive self-concept treatment 2.04 1,65 .16 nsSelf-reporteddelinquent behavior

treatment 7.62 1,54 .008 **

Self-reportedsubstance use

treatment 5.46 1,56 .023 *

Self-reported seriousdelinquent behaviors

treatment 7.44 1,54 .009 **

School punishment treatment 6.00 1,73 .017 *School rewards treatment 6.94 1,74 .010 **Victimization control 1.83 1,72 .18 nsInvalidity control 4.21 1,71 .044 *School effort treatment 8.45 1,71 .005 **Practical knowledge treatment 10.55 1,74 .002 **Alienation treatment 4.88 1,67

..(;13Self-rep. grades treatment 1.28 1,74 .26 n:Self-reportedreading ability

interactionc

Educ. expectation treatment 1.69 1,74 .20 nsWork for pay treatment 6.59 1,74 .012 *Regular job treatment 1.94 1,74 .17 nsSuspension treatment 24.08 1,73 .001 **School non-attendanceindex

treatment 4.31 1,71 .042 *

Rebellious autonomyInvolvement

treatment 20.81treatment 17.31

1,701,71

.001 **

.001 **

Note. Based on analyses that test for an effect of treat-ment net of period of assignment to treatment or controlgroups (see text footnote 5). The withdrawal variable maynot be comparable for treatment and control groups (see textfootnote 7).

aPart of total self-reported delinquent behavior.Treatment group reports more school punishment.Nature of effect depends on period examined.* Statistically significant ns Not significant

** Highly statistically significant as Almost significant

-161-

24

Table 2

An Examination of Differential Validity of Self-Reportfor Treatment and Control Groups

Self-reportmeasure

Archivalmeasure r N

Treatment group

Delinquent behaviortotal

School nonattendanceindex.

Reading ability

School grades

Number of police -.01 23contacts

Number of unex-cused absences,

,,April-end of year

Plato readingpretest

CACYD grades

CACYD grades

. 31 29

-.07 29

.05 24

-.21 25

Control group

Delinquent behavior

School nonattendance

Reading ability

School grades

Number of policecontacts

Number of absencesCUSD, 81-82

CUSD grades

.54 34*

. 23 41

. 16 36

CUSD grades .10 37

*p < 01

Project STATUS: Second Interim Report

D. K. Ogawa, M. S. Cook, and G. D. Gottfredson

Abstract

Project STATUS (Student Training Alternatives Through Urban Strate-gies) is continuing its efforts at Eliot Junior High and Muir HighSchool in Pasadena, California to reduce class cutting, disruptivebehavior, dropout from school, and lack of student and parental involve-ment in the school. The project's five interventions: a) the Optionsclass, b) the Youth Committee and Leadership Training Class, c) projecttraining, d) parent involvement, and e) the Action/Advisory Committee,are designed to provide students with a more meaningful educational pro-gram. More specific project objectives are to decrease alienation,increase attachment to school, self-esteem, and academic success.

Non-equivalent control group designs are used to assess the effec-tiveness of the Option classes and Youth Committee at Eliot, and theOptions classes at Muir. No evaluations are currently available for theother program components. Results show students in Eliot's Optionsclass to be significantly lower in alienation, make higher ratings oftheir own reading ability, report more fairness and clarity of schoolrules, and have fewer withdrawals from school than students in a non-e-quivalent control group. It apparently also increases absenteeism forproject students. No evidence of .effectiveness was found for Muir'sOptions class or for the Youth Committees at Eliot or Muir. We cannotas yet judge the effectiveness of parts of the project; numerous prob-lems in implementing some project components may have weakened theeffectiveness of the planned interventions. Next year's efforts willinclude consolidating the Youth committee and Leadership Training Classto intensify the leadership treatment.

Goals and Rationale

Project STATUS (Student TrainingAlternatives Through Urban Strate-gies) is continuing its efforts toprovide targeted junior and highschool students in Pasadena, Cali-fornia, with a high interest curri-culum (Options Classes) and leader-ship training (YouthCammittee/Leadership TrainingClass). Three other

This description of the project'sgoals, rationale, and objectivesdraws on several sources of informa-

163

interventions--parent involvement,project training, and the action/ad-visory committee--are intended to

involve parents, community members,and project staff in project actvi-ties. All of these interventionsare designed to address five prob-lems found at Eliot Junior High andMuir High Schools: (a) class cut-ting; (b) disruptive behavior;

ton including the project's ProgramDevelopment Evaluation plan, conver-sations with the project director, areport by the Project Director(Reiss, 1982), and an earlieraccount of the first year's interim

26

STATUE

(s) lack of student involvement;(4) dropouts at Muir high school,aced (e) lack of parent involvement.

These problems are thought tosaid becauee students are notlovely,* in relevant and meaningfuleducational programs. Consequently,students experience academic ba-ler*, loss of eelf -esteem, aliens -ties and less commitment towardsschool. This accords with empiricalstudies, which have shown corrals -ties§ between academic failure, lowcommitment and attachment to school,aod student misbehavior (Stilt -

chcembe, 1964; lilberberg i Silber -berg, 1971; Elliot 6 Voss, 1974;

Mirschi, 1969).

to addition to experiencing aca-demic failure, many students withnatural or traditional leadershipabilities are seen by the project as14ckicg the appropriate leadershipskills to effectively influencetheir school environment. This,along with the perception that theschool rules are not fair and thatthere are no mechanisms for studentinput and involvement in decision -

making, is believed to alienate stu-dents from school.

Eased on STATUS' theory, severalobjectives have been specified to

achieve their goals. Table 1 sum-marize* each goal and the corres -pealing objectives. This tablemales clear the extent to which theevaluation of this project deviatesfrom the Program Development Evalua-tion model. The objectives includeprocess objectives or criticalbeochmarhs for the implementation ofthe project, and the measures of

goals and objectives are not madeontareevaluation of this project (Carlton,1982). The Carlton report providesa more complete description of firstyear activities and plans.

164

clear. This departure from the

standards set for the evaluation ofthe alternative education programresulted in a relatively weak evalu-ation of Project STATUS. For exam-ple, the major objective of theYouth Leadership Training Class, toincrease students' leadershipskills, is lost amid a number of

process objectives. No measure wasspecified for this important objective.

Planned Intervenkions

Project STATUS interventions arebased on the notion that intervening,only at the individual level or onlyat the school level cannot solve

student misbehavior. The overallapproach is designed to meet indivi-dual student needs as well as

involve students, school personnel,parents, community members andorganizations_ in_ affecting .schoolclimate.

aptions Ulassea

Sixty 7th graders and 60 9th

graders of all academic abilitiesvolunteer to participate in alterna-tive educational English and SocialStudies classes. These classes uti-lize a curriculum focusing on cop-ing with authority, responsibility,the school, the family, and friends.Audio-visual presentations, fieldtrips, guest speakers, role-playing,simulations, and action activitiesare used to enrich the curriculumcontent. This intervention, whichattempts to provide students withrelevant and appropriate student-centered materials, is primarilyaimed at STATUS' objectives of

decreasing alienation and increasingacademic achievement and self-es-

teem.

27

Youth Committee (YC)

Approximately 50 students identi-fied as traditional (e.g., studentcouncil members, team captains,etc.) and natural (e.g., gang lead-ers) leaders are referred by schoolpersonnel to participate in theYouth Committee. This interventionis designed to provide students withopportunities to identify problemsin the school and seek solutions inorder to improve the climate of theschool. By channeling the YouthCommitee members' leadership abili-ties toward effective organizationalchange, project implementers theor-ize that these students will notonly feel more involved with andcommitted to the school, but willalso positively influence other stu-dents.

Leadership Training Class (LTC)

A leadership training curriculumwas devised to provide students fromthe Youth Committee with intensiveindividualized training in develop-ing decision-making, conflict man-agement, and communication skills.Approximately half of the studentsfrom the Youth Committee participatedaily in this class for one semes-ter; then the rest of the committeetakes the class during the secondsemester. The purpose of this classis to allow students to developtheir leadership skills and applythem to the problems addressed in

Youth Committee meetings.

Staff Training

Project staff, school administra-tors, and teachers participate in

training. sessions prior to the fallsemester to help them facilitatestudent development and involvementin organizational change. Thetheory behind the training is thatschool staff must be aware that

positive student attitudes are

165

STATUS

essential for change and that as

representatives of the school, theirpolicies and actions are importantin improving student attitudes andperceptions (Reiss, 1982).

In addition to these trainingsessions, project staff participatein on-going informal trainingthroughout the year. Utilizingweekly observation, the projectdirector gives feedback to to pro-ject teachers, aides, and counselorsin the areas of individualizedinstruction, effective use of aides,and classroom participatory manage-ment.

Parent Training

Parents of students participatingin the project were to meet on a

regular basis to identify trainingneeds and to receive training in

maximizing parent input in schooldecision-making practices. Theintent of this intervention is to

get parents involved and to getparents to accept responsibility forwhat goes on in school. This inter-vention was not well implementedduring the 1981-82 school year (seebelow).

Action/Advisory Committee

The Action/Advisory Committee ismade up of parents, citizens, commu-nity service agencies, and otherorganizations from the Pasadena com-munity who identify and provideresources for project activities.In addition, members serve as rolemodels and provide project studentswith vocational guidance andresources. Examples of activitiesinclude making in-class visits, pro-viding vocational intern experi-ences, and organizing on-site visitsto various agencies. As with parenttraining, this intervention is

designed to get the communityinvolved with the school.

28

STATUS

Project Environment and Staffing

The project's environment, i.e.,the implementing organization, thetargeted schools, and the schooldistrict, has remained the same aslast year (see Carlton, 1982, for acomplete description). There havebeen, however, some changes in

staffing. At the school sites,three new teachers' aides replacedthe three aides from last year wholeft the program. One of the newaides also left during the semester.Consequently, one of the high schoolteachers did not. have a teachingaide.

At caF (Constitutional RightsFoundation), there was no replace-ment this year for the communitycoordinator position. One reasonfor no active recruiting was the un-certainty of the third year budget,and the inefficiency in trainingsomeone for a one-year position.However, a new administrative assis-tant was hired to replace the previ-ous one. Since there is no on-siteevaluator and since the school dis-trict's director of evaluation hasother commitments, it is importantfor the administrative assistant tohandle many of the evaluation activ-ities.

Program Expenditures

Figure 1 shows how STATUS appro-priated its $405,971 budget for the1981-82 school year. Salaries,including contractual services (con-sultants, stipend for training ses-sions, etc.) comprise the largestsingle expediture. This 60% of thebudget does not include the total

salaries of project teachers andstaff at the schools. A large per-centage of their salaries are paidby the Pasadena School District.Program materials and equipmentaccount for a small portion of the

budget, 7% and 2% respectively.Other expenditures include travel

166

(2%), office operating costs (7%),

and indirect costs (21%).

Implementation of Project STATUS

Last year's problems in imple-menting the program (see Carlton,1982) resulted in differentialphases of implementation of the

intervention components for thisyear. For example, the Optionsclass is more developed than the

parent training component. Whereasthe Options class is at the imple-mentation/fine-tuning phase, parenttraining is closer to an initiation/adoption phase. Although this situ-

,ation is true at both schools,Tables 2 and 3 indicate that the

degree of implementation of the

interventions varied at the twoschools. Table 2 is based on theSAES (School Action EffectivenessStudy) student questionnaire andshows that Options students_at. Eliotsignificantly differ from non-equi-valent controls on measures of indi-vidualized instruction, field trips,student-teacher interactions, and

student influence in decision-mak-ing. Their are no differences on

Student Degredation (a scale measur-ing negative behavioral treatment ofstudents). On the other hand,Options students at Muir differedsignificantly from their non-equiva-lent controls only in terms of the

number of field trips. This find-ing, however, contradicts projectreports that no field trips weretaken at Muir (see Table 3). Thedata in Table 3 should be inter-preted with caution since projectstaff recorded the extent and natureof program activities for each stu-dent at the end of the school year.Inaccurate records and retrospectivereporting may degrade the quality ofthe data. Other project-developedrating systems were also examined toprovide information as to the amountand quality of the Options and LTCclasses.

23

Options Classes

The Options classes revolvearound a curriculum that aims to beof high interest to the students.Each unit of the Options class cur-riculum is based on law-relatedmaterials and execises, and attemptsto deal with a significant aspect ofthe student's life, e.g., thefamily, school and the law. Eachtopic is intended to address impor-tant project objectives such as

increasing attachment to school anddecreasing feelings of alienationfrom school and society.1 However,because curriculum units were notready until the week of use in theclassroom, teachers had little timeto review the materials and preparefor class beforehand.

Eliot Junior High. The 54 stu-dents and two teachers involved inthe Options class rated each curri-culum unit after its presentation.Ten rating categories were used (seeTable 4). These ratings were car-ried out using project-developedmaterials. Table 4 shows the

results for the students' ratings.Two of the ratings most critical forthe assessment of project objectivesare ratings of curriculum relevanceand interest. For each unit, overhalf (54-70%) of the students feltthe material was relevant and bet-ween 38-70% felt the class was moreinteresting than others. We have nonorms for these ratings, no project-generated standard against which tocompare the ratings, nor are thereratings of curriculum made by a com-parison group experiencing commoncurricula. These ratings do notnecessarily demonstrate that the

curriculum that was implemented was

1. For a description of the curri-culum, send requests to ElizabethSalzman, CRF, 1510 Cotner Avenue,Los Angeles, CA. 90025

167

STATUS

a high interest curriculum. Is a

38% affirmative response to "moreinteresting" good or bad? We do notknow. As another example, about 40%of the students report doing workoutside of class--without some com-parison group it is impossible to

say whether the curriculum is

inspiring more students to do morehomework, or less. CA comparisongroup was available for the self-re-ported School Effort measure in theSAES questionnaire. No statisti-cally significant difference wasfound between treatment and compari-son students.) Project staffreport, however, that the results

,did provide feedback to the curricu-lum writer so modifications could bemade for the upcoming year.2

Other activities of the Optionsclasses included cooperation betweenOptions and YC/LTC teachers in theplanning of activities, and staff

coordination of a mock trial compet-ition for the Options students whichincluded non-project eighth gradestudents as jury members.

Muir High School. Implementationproblems at Muir began early whencounselors mis-assigned nonprojectstudents to the Options classes atthe beginning of the semester. Pro-ject staff had a difficult time cor-rectly rescheduling the project stu-dents because a) it was difficult tore-arrange the mis-assigned stu-dents' schedules and b) nonprojectstudents assigned to the Optionsclasses enjoyed the classes andwanted to remain enrolled in them.Ultimately, about 25% of the stu-dents who took the Options classesduring the first semester were ori-ginally designated as nonproject

2. Options class teachers alsorated the curriculum units, but res-ponses are based on only one or twoteachers.

30

STATUS

students, i.e., they did not meetthe original criteria for selectioninto the program.

Another implementation problemwas that one teacher did not have ateaching aide. In addition, the

project director reports that alter-native education techniques have notbeen completely adopted at Muir, andteachers there do not spend as muchtime planning together as teachersat Eliot.

Table 5 shows the results of thestudents' ratings. of each unit of

the Options curriculum as taught atMuir. Depending on the unit, bet-ween 65-79% of the students agreedthat the information presented wasrelevant. Of the students respond-ing, 77-89% felt the class was moreinteresting than others. Projectstaff reported that the ratingsproved useful in modifying the cur-riculum.

Youth Committee/Leadership TrainingClass

The YC/LTC was only weakly imple-mented in 1981-82. There are sev-eral reasons why. As was the caseduring the first project year therewere problems in scheduling YC meet-ings. Meetings could not be sche-duled after school because this con-flicted with other activities.Lunch period was too short. Teach-ers objected to having studentsattend YC meetings during classtime--despite these students beingidentified as leaders, many lackacademic skills and cannot afford tomiss class. However, for lack of abetter time, meetings at bothschools were held during class time.Consequently, YC meetings were heldinfrequently. A second problem wasthat few of the YC students gotinvolved in project activities: out

of 50 YC members at Eliot, only 4students participated in presents-

168

tions and 7 participated in schoolcommittees. At Muir, 17 out of 77members participated in presenta-tions and 15 participated in schoolcommittees. This suggests that stu-dents may have been attending the

meetings only to get out of class.

In addition to problems in sche-duling YC meetings, not all YC mem-bers could be scheduled into theLTC, an elective course. This

course was only offered one periodper day and often coalicted withother courses. Some of the YC mem-bers who did take the LTC subse-quently dropped out of the YC. The

,scheduling and drop-out problemresulting in only a few studentsreceiving training and experience inthe use of leadership skills, per-haps subverting the YC/LTC as a

school change intervention. Accord-ing to the project's theoreticalmodel, systematic _changes_ _cannotoccur within an organization withonly a small number of trained indi-viduals.

Another problem in implementingthe YC/LTC was that the two projectparaprofessional counselors, whocoordinated the YC/LTC activities,may have taken on too many responsi-bilities for their half-time non-professional status.

Besides overseeing the YC/LTC,the paraprofessional counselors alsoserved as school counselors for theOptions and YC/LTC students. The

program design called for the para-professional counselors to arrangeparent conferences and make homevisits if students were failing aca-demically, cutting class, or behav-ing badly, but almost no conferencesor home visits took place. Recordsshow that at Eliot only one homevist was made all year, and at Muir,two. At Eliot, 25 students receivedon-going counseling from the para-professionals. At Muir, the para-

31

professional counselor contacted allof the Options students at leastonce at the beginning of the semes-ter. During the course of the year,15 students at Muir were called inonce for behavioral problems;

another two students were called intwice.

A further problem with the imple-mentation of the YC/LTC is that thecounselors had little exposure to

the leadership training curriculum.

To summarize the efforts of the

paraprofessional counselors, they

only worked half-time, were unfamil-iar with the leadership trainingcurriculum, provided little or no

documented counseling, and made a

total of only three home visits. It

is likely that unless they engagedin other, undocumented activities,they had little impact on projectstudents.

Eliot_ Junior High School. Atotal of 50 students participated inthe YC during the year and addressedsuch issues as security and safetyat school, longer lunch periods, andmore elective courses for 8th grad-ers. Twenty-eight of the YC memberstook the LTC. Scheduling difficul-ties led to a late start-up date forthe class. As a result, first sem-ester LTC students remained in theclass for the entire year, with nonew students added the second semes-ter.

Table 6 shows the results for thestudent ratings of the leadershiptraining curriculum. Ratings of

relevance, interest, and leadershiptraining received are of particularinterest to project staff. Overall,83% of those responding felt the

information presented was relevant,77% found the class more' interestingthan others, and 91% felt they haddeveloped leadership skills. Unfor-tunately, we do not know whether thestudents did in fact improve their

169

STATUS

leadership skills.

One result in the ratings that

may be important for program devel-opment efforts is that the ratingsat Eliot or generally lower than

those at Muir: nineteen of twenty-five ratings favor Muir. Even thiscomparison should be interpretedvery cautiously because of the smallnumber of students reponding to thesurvey. Percentage agreement scoresare unstable when the n's are as lowas they are here.

Muir High School. The 77 membersof the YC addressed issues such asclass and school cutting and student

"pride. The LTC was held daily for16 YC members each semester. Eight-y-two percent of the second semesterLTC students rated the informationpresented in class as relevant; 85%found the class more interestingthan other classes at Muir. All ofthe students- responding--reportedthat they had developed the abilityto lead a group (see Table 6).

Again, without a control group,there is no clearcut way to inter-pret these ratings.

Staff Training

During the school year, severaltraining sessions, were held withproject/staff, teachers, and schooladministrators. A total of 17 for-mal training meetings with projectteachers, counselors, and aides wereheld. Over 40 individual meetingswere held with project teachers andthe project director to discuss thein-class observations made weekly bythe project director.

Parent Training

Two meetings at each school for

the parents of STATUS students wereheld this year. Approximately 50

parents were involved in these meet-ings which were run by the project

32

STATUS

director and principals of the pro-ject schools. An overview of theprogram was given and an descriptionof how parents could help the pro-ject and how the project could helpparents. The program design calledfor the holding of parent trainingworkshops, but none were held. Noparent committees were established.

Action/Advisory Committee

This year the action and advisorycommittees were combined into onecommittee. Meetings were held everysix to eight weeks with an averageattendance of 15. The committeerevised and distributed the commu-nity resource guide. The committeealso held a public relations meetingat the superintendent's house to

expand the project's base in thecommunity.

Summary

For the Options and YC/LTC inter-ventions, obstacles internal andexternal to the project influencedimplementation. A major externalforce was scheduling: project stu-dents could not be scheduled intoproject classes, non-project stu-dents were scheduled into projectclasses, and convenient meetingtimes for the Youth Committe couldnot be found. An internal obstacleto faithful project implementationwas the reluctance of project staffto adopt teaching and counselingstyles advocated by the projectdirector. In addition, projectimplementers were inexperienced withthe curriculum developed for theOptions classes and the LeadershipTraining Classes. Thus, projectstaff did not adopt project educa-tional techniques, and were notexperienced in project curricula.It appears, therefore, that thesecomponents were not implemented asdesigned.

A further implementation

170

difficulty was the ignorance of pro-ject staff about the goals of theproject. It is only with the startof the third project year that otherproject staff besides the projectdirector have begun to understandthe goals and objectives of the pro-gram. Finally, a lack of directproject control over selection andsupervision of in-:school staff, anda shortage of personnel to coordi-nate all aspects of the interven-tions also contributed to poor pro-ject implementation.

Interim Outcomes

Interim evaluation results havetwo potential uses: they can pro-vide information as to the effec-tiveness of a project to date, andmay serve as material for furtherprogram development. The extent towhich the information is usefuldepends in large part on the ade-quacy of the evaluation design. Forthe CRF project, strong evaluationswere agreed to, but were considera-bly weakened by non-random assign-ment to treatment and control, and

multiple treatment for project stu-dents.

Evaluation Design

Options Classes. PotentialOptions participants were recruitedby project staff prior to the fallsemester. Interested students wererequired to submit an applicationand acquire parental permission toparticipate in STATUS. From this

pool of applicants project staffselected students who were represen-tative of the school population,i.e., participants included studentsof all academic abilities, races andbehavioral dispositions. The pro-ject agreed to randomly assign stu-dents to treatment and control fromthis referral pool. Randomizationwas not adhered to at either Eliotor Muir.

At Eliot, students were randomlyassigned to treatment and controlfrom the pool of screened referrals.Roster updates collected in Octoberindicated little relationship bet-ween the original random assignmentand the students designated as

"treatment" and "control" by projectstaff. The reason for this is

unclear. What is clear is that theevaluation of Eliot's Optionsclasses involves a comparison bet-ween treatment students and a non-e-quivalent control group accompaniedby an unmeasured selection bias.

The experimental evaluation of

the Options classes at Muir brokedown when not enough studentsreceived parental permission to par-ticipate. The non-equivalent con-trol group at Muir therefore largelyconsists of students whose parentsdid not wish them to receive alter-native education. The differencesbetween such students and thosewhose parents did grant permissioncannot be known with any certainty.

The YC. As with the Optionsclasses, YC participation is on a

voluntary basis and is open to allstudents. Faculty, administrators,parents, and students referred stu-dents to the YC. The criteria forreferral was that the student wasidentified as a natural or potentialleader. LTC students were non-ran-domly selected from among those stu-dents recognized as the most promis-ing leaders in the YC. Selectedstudents also had to be able to fitthe LTC class into their schoolschedules. Thus the LTC studentsare a select group, and no convinc-ing control group for them could bedeveloped--LTC students will not beevaluated seperately from YC, stu-dents at large.

171

STATUS

We do not know how comparisongroup students were selected for theEliot and Muir Youth Committees:students from the referral pool andconsent pool had originally beenrandomly assigned, but differentstudents were ultimately designatedby the project as treatment and con-trols. Students assigned to controlbecame treatment, and new studentsappeared in both treatment and con-trol groups. To compound theseproblems, at Muir only 6 controlswere identified. No sensible com-parison is possible between approxi-mately 50 treatment students and 6controls. Therefore the Muir YC

,cannot be evaluated.

Summary of Evaluation Designs.For the Options classes, quasi-ex-perimental non-equivalent controlgroup designs describe the evalua-tions at both Eliot and Muir. Theevaluation of the YC at E_ liot can

also be characterized as a non-equi-valent control group design. No

evalution design was implemented forthe Muir YC, or for either school'sLTC.

Measures. Measures come fromseveral sources: a) the SAES ques-tionnaire, b) school records, c)

project questionnaires, and d) pro-ject records. The majority of mea-sures are scales and individualitems from the SAES student questi-onnaire. Test scores, attendanceand grades were taken from schoolrecords. Project questionnairesinclude student and teacher evalua-tions of the curriculum, studentdevelopment of leadership competen-cies, and a teacher observationchecklist. Information from projectrecords will be used to determinethe number of parent meetings held,and the number of policy changesinitiated by parents and students.

3

STATUS

Results

Equivalence of the ControlGroups. Tables 7 and 8 give pretestcomparisons between the varioustreatments and their available con-trols. The treatment and non-equi-valent control groups appear fairlysimilar on the measures taken (thesepretest measures were scored in a

different--but equivalent--metricthan similarly-named outcome mea-sures reported later). The EliotOptions students do not differ fromtheir controls on any of the 12 psy-cho-social measures, and the YouthCommittee students differ from theircontrols only on a measure of

Attachment to Parents: YC studentsare more attached to their parentsthan the non-equivalent controls areto theirs. At Muir, the Optionsstudents differ from their controlson measures of Practical Knowledgeand Rebellious Autonomy: The treat-ment students report more practicalknowledge, and are less rebellious.Both of these comparisons, there-fore, favor the treatment group.

The evaluation designs for the

Eliot Options classes and Youth Com-mittees, and the Muir Youth Commit-tees are non-equivalent controldesigns. They are not true experi-ments. Besides the measured differ-ences that do exist, other, unmea-sured, differences may well exist.

Project Goals. Tables 9 through11 show treatment and non-equivalentcontrol group comparisons on severalmeasures of STATUS' goals for bothEliot and Muir. Three measures wereused to assess class cutting (see

Table 9). Options students at Eliothad significantly more unexcused,i.e., non-illness and non-medical,absences than non-equivalent controlstudents. Statistically, the prob-ability of this difference occurringby chance is less than 5 times outof 100 (p< .05). Although not sta-

'tistically significant, thesestudents also reported themselves ascutting classes and school more thandid the controls. Options studentsat Muir do not significiantly differfrom their non-equivalent controlson any attendance measure. No sig-nificant effects on attendance werefound for the YC at Eliot.

No significant differences werefound in the amount of disruptive ordelinquent behavior between Optionsand non-equivalent control studentsat Muir and Elliot, nor between YCstudents and non-equivalent controlsat Eliot (see Table 10).

There was a significant differ-ence between Options and non-equiva-lent control students at Eliot in

terms of the number who withdrewfrom school (see Table 11). Signi-ficantly fewer Options studentswithdrew from school (p<.01). Par-ticipants in the YC at Eliot and theOptions classes at Muir did not dif-fer from their non-equivalent con-trols in the number of withdrawals.

With no design, we do not knowhow Muir's YC affected the project'sgoals.

Unlike last year, the Youth Com-mittees at Eliot and Muir were ableto get students involved in schooldecision-making. This year, therewere three student-initiated policyand procedural changes put intopractice. However, the project wasunable to increase parental involve-ment in school decision-making prac-tices at either school--no parentinitiated policy or proceduralchanges were put into practice.

Project Objectives. The resultsof treatment and non-equivalent con-

1 trol group comparisions on STATUS'objectives as measured by the SAESstudent questionnaire are shown inTables 12 and 13. Options students

172

35 op*

at Eliot are significantly less

alienated (p<.05) than controls, andare more likely to report thatschool rules are fair (p<.01) andclear (p<.01; see Table 12). Thereare no differences on scales measur-ing Self-esteem or Attachment to

School. Muir Options students do

not differ from their controls onany of the above measures.

The YC at Eliot produced no mea-surable effects on measures of

Alienation, Self-Esteem, Attachmentto School, or the Clarity or Fair-ness of School Rules. We cannotassess the effectiveness of the YCat Muir at meeting the program'spsycho-social objectives.

In terms of academic achievement,Options students at Eliot reportedthemselves to be higher in readingability than did the non-equivalentcontrols (p<.01; see Table 13); theydid not report, however, gettinghigher grades, or receive them. No

measurable effects on schoolachievement were found for theOptions program at Muir, or the YCat Eliot. We cannot judge the

impact of Muir's YC on academicachievement.

Outcomes of Theoretical Interest.Table 14 shows the results of treat-ment and non-equivalent controlgroup comparisons on outcomes notdirectly addressed by project STATUSbut which are of theoretical impor-tance to the prevention of delin-quency. These outcomes also providea check on any unanticipated nega-tive or positive effects of the pro-ject.

Options class students at Eliotdiffer from the non-equivalent con-trols only on a measure of Interper-sonal Competency: Options classstudents report themselves to be

more interpersonally competent thando the controls. They do not differon scales measuring Attachment to

173

STATUS

Parents, Parents' Emphasis on Educa-tion, Belief in Conventional Rules,Life Competency, Internal Control,Negative Peer Influence, SchoolEffort, Involvement in ConventionalActivities, Rebellious Autonomy,Victimization, School Punishments,School Rewards, or Invalidy (a mea-sure of truthfullness in reporting).

Eliot YC members report them-selves to be more Involved in Con-ventional Activities than do theircontrols. This is probably animplementation measure, and suggeststhat the Eliot YC members are becom-ing involved in a wider range of

,,exeriences. The YC students alsoreport signficantly lower attachmentto their parents. Why this shouldbe the case is unknown. Although itis conceivable that the program iscausing the students not to care fortheir parents, the more likelyexplanation_ is that_this is a pre-existing difference arising from thenon-equivalent nature of the design.None of the other measures on Table14 show any differences betweenEliot YC treatment and controls.

The Muir Options class does notdiffer from its designated non-equi-valent control on any of the four-teen psycho-social outcomes found inTable 14.

Summary. The interim outcomeevaluation may be summarized as fol-lows:

1. Comparisons between EliotOptions class treatment students andnon-equivalent controls suggest thatthe class results in fewer withdra-wals from school, decreased aliena-tion, greater interpersonal compe-tency, higher perceived fairness andclarity of school rules, but moreunexcused absences. This interven-tion may be also be increasing stu-dents' perceptions of their ownreading ability of students, but it

36 al

STATUS

is not affecting grades, or disrup-tive behavior.

2. The evaluation of the MuirOptions class provides no evidencethat it has any effects on its par-ticipants--no differences were foundbetween the treatment and non-equi-valent controls on any of twenty-seven outcome measures.

3. Comparisons between Eliot YCstudents and their non-equivalentcontrols show that YC studentsreport more Involvement in Conven-tional Activities (a program imple-mentation measure), but less attach-ment to their parents. The lattereffect is probably a pre-existing,between group difference. Theinterim evaluation found no evidencethat this intervention is affectingattendance, academic achievement,disruptive behavior, or psycho-so-cial adjustment for participatingstudents.

4. No convincing evaluation is

available for the Muir YC, or theparent involvement, project train-ing, and action/advisory committeecomponents of the project.

5. Implementation data impliesthat the Eliot Options class wasmore strongly implemented than theMuir Options class. Nevertheless,student ratings suggest that theMuir students enjoyed the classmore.

Discussion

Caveats to the Evaluation Results

As discussed earlier, there weresignificant breakdowns in the origi-nal experimental designs for the

Eliot Options classes and YC, and

the Muir Options classes. Despitetheir similarity on the pretestsavailable, unmeasured differences,especially motivational differences,may exist between the treatment and

174

resultant non-equivalent control

groups. All results (and non-re-sults) must be interpreted in thislight.

What Accounts for the Results?

Four factors may account for thelack of effectiveness of the MuirOptions class as compared to themore positive effects found for theEliot Options class. First, it maybe that the curriculum is moreeffective on younger students, i.e.,7th graders as opposed to 9th grad-ers.

Second, an undetermined number ofcontrol group students received pro-ject services at Muir. This is a

potential explanation of a lack ofdifferences between the control andtreatment students on the outcomemeasures. To examine this possibil-ity, analyses were conducted usingonly non-equivalent controls whoreceived no project services. Thisliberal analysis still uncovered nodifferences between the treatmentand "no-treatment" controls at Muir.

A third reason why Muir's Optionsclass appears to be less successfulthan Eliot's, is that Eliot's classmay have been somewhat better imple-mented (see Tables 2 and 3).

A fourth possibility, of course,is that the few significant findings(both positive and negative) for theEliot Options class are due to pre-existing differences between treat-ment and non-equivalent control stu-dents. Because we do know howtreatment and controls were selectedwe cannot conclusively attributeprogram effects (or the lack of

them) to the program.

The null findings for the YC onindividual level measures at Eliotmay be misleading because the empha-sis of this intervention is on

37

school-level changes, unlike the

Options class, which focuses on

individual-level changes. It is

possible, although unlikely, that ina school climate sense, the YC mayaffect the control students as muchas the treatment students.

The puzzling negative effect thatthe Eliot Options class appears tobe having on attendance may be theresult of the positive effect theprogram is probably having on with-drawals from school. The programmay be successfully retaining stu-dents in the program that would havedropped out; these students maystill skip school more than studentsthat would have remained in schoolwithout the program's support. In

other words, the "negative" effectson absences may be a paradoxicalresult of keeping Options studentsenrolled in school.

Implications of the Results

Plausibility of STATUS' Theory.Although we cannot unequivocallyattribute the positive effects foundfor the Eliot Options class to theclass, the program does appear to

have met certain project objec-tives--decreasing alienation, andincreasing the use of individualizedinstruction. It also appears to

increase students' perceptions of

their reading ability. It alsoprobably is reaching a project goalof decreasing withdrawals fromschool. Also, even the EliotOptions class has had no measurableeffect on delinquency. In short, wedo not know as yet whether the

theory behind STATUS as a delin-quency prevention project will ulti-mately prove useful.

The theory underlying the YC isuntested in results to date. No

evidence is available to suggestthat this portion of the project iseffective; it was not implemented instrong form.

175

STATUS

Program refinements for 1982-83.Because the results suggest that theYC and LTC interventions as imple-mented in 1981-82 did not affectstudents, program changes have beenmade for the 1982-83 yeaL The YCand LTC will be combined next yearinto a single intervention, whichwill serve only 30 students. Fewer

students will be served, but the

intervention will be intensified byhaving a small number of studentsmeet everyday to receive the leader-ship training and practice in appli-cation. We are hopeful that this

consolidation may eliminate the pro-gram implementation and evaluation

,design breakdowns encountered this

year.

Modifications were made for boththe Options and LTC curriculums,based on the feedback from the stu-dent and teacher ratings collectedby the project. A greater emphasiswill be made on addressing STATUS'objectives in each curriculum unit.

Evaluation Strategies for 1982-83

The structure of the PDE plan hasbeen modified this year to make itmore useful to the project staff.

In addition, clearer specificationof the project's action theory hasresulted in more coherent objectivesand helped guide intervention stra-tegies.

As with last year, a quasi-exper-imental design will be implementednext year to assess the effective-ness of the Options and Youth Lead-ership Training class components.We were largely unsuccessful at

changing the procedures for place-ment of students into treatment andcontrol conditions for the 1982-83school year, but an early assessmentof the equivalence of the groupsshows no major differences betweentreatment and control students on

the few outcomes we analyzed.

38

STATUS

Problems Project STATUS is Encoun-tering

As discussed earlier, ProjectSTATUS has experienced major prob-lems in implementation. A secondmajor problem concerned evaluationissues. One issue is that the pro-ject director feels there is a

trade-off between implementationactivities and evaluation activi-ties. STATUS's small staff makesthis problem particularly acute.Another issue is the District evalu-ator's attitude toward the nationalevaluation. He feels the evaluationeffort is larger 'than the programitself and has strongly questionedthe validity of the evaluation andthe use of the SAES questionnaire.This resistance may have divertedenergies from program implementationto evaluation issues. A final eval-uation issue is providing data to

the national evaluators in a timelyfashion. Besides a lack of manpowerto collect the data, lags in updat-ing school records creates a datamanagement problem. Plans have beenmade for the upcoming year to

address these obstacles.

Successes Project STATUS is Encoun-tering

The treatment and non-equivalentcontrol comparisons indicate thatthe Options class at Eliot is proba-bly successfully achieving some ofits objectives. School administra-tors, teachers, and parents havereported marked improvement inOptions classes participants; theyfeel that these students have newlyacquired academic leadership and

176

interpersonal skills (although, as

yet, we have been unable to verifysuch changes with hard data). Stu-dents have given presentations atboard meetings and have competed ina. Mock Trial Competition. Thesesuccesses have helped turn initialskepticism of the program by schooland district staffs to support. Asa consequence, STATUS has been writ-ten into the Muir School ImprovementPlan, funded by the state to improvethe school environment. In addi-tion, a description of the projecthas also been included in "A Compen-dium of Character Education Programsin California Public School..."

Institutionalization

Even before it was known whetherthe project would be funded by OJJDPfor the 1982-83 school year, theOptions classes at both Eliot andMuir were _included_ in_ the _master _ _schedule for the upcoming year. TheEnglish Department at Muir is con-sidering the adoption of projectmaterials to be used in otherclasses. Other schools in the dis-trict have also expressed interestin STATUS.

Although next year will be thelast year of federal funding forProject STATUS, there are some prel-iminary indications that projectSTATUS or some aspects of the pro-ject mey be adopted by the projectschools -nd other schools as well.The Options class shows potential.Careful implementation and monitor-ing of other aspects of the programwill be necessary to realize theprogram's optimal strength.

39

STATUS

References

Carlton, R. Student Training Alternatives Through Urban Strategies (ProjectSTATUS): Interim Report. In G. D. Gottfredson (Ed.), The School ActionEffectiveness Study: First Interim Report. Baltimore: Johns HopkinsUniversity, Center for the Social Organization of Schools, 1982.

Elliot, D. S. & Voss, H. L. Delinquency and dropout. Lexington: Heath &Co., 1974.

Gottfredson, G. D. & Daiger, D. C. Disruption in 600 schools: The socialecology of personal victimization in the nation's public schools (ReportNo. 289). Baltimore: Johns Hopkins University, Center for Social Organization of Schools, 1979.

Hirschi, T. Causes of delinquency. Berkeley: University of CaliforniaPress, 69.

Reiss, D. Project STATUS: An overview. Paper presented at the annual meeting of the American Educational Research Association, New York, 1982.

Silberberg, N. E. & Silberberg, M. D. School achievement and delinquency.Review of Educational Research, 1971, 14 (1), 17-33.

Stinchcombe, A. L. Rebellion in high school. Chicago: Quadrangle, 1964.

177

4;0

Figure 1

Projoct STATUS' Budgeted Expenditures for 1981-1982

ProgramMaterials

$27,439(7%)

4Total Personnel-CRF Staff

OfficeOperatingCosts $27,054 (7%)

$127,746

(31%)

IndirectCosts$86,057

(21%)

Contractual Services$119,595(29%)

-178-

41

Travel $10,135 (22

Equipment $7,945(2%)

Table 1

Project STATUS' Goals, Objectives, and Measures

Goal/Objective Measure

I. Coal: Decrease class cutting. I.A. SAES student questionnaireB. Attendance data

Objectives:

1. Increase use of individualized instruction plans. 1.a. Count of individualized educational plans for

studentb. SAES student questionnaires

2. Increase use of appropriate student centered materi- 2.a. Teacher evaluation of CRF material

als. b. Student evaluation of CRF material

3. Increase use of high interest curriculum. 3.a. Teacher evaluation of CRF materialb. Student evaluation of CRF material

4. Increase self-esteem. 4.a. SAES Student questionnaire

5. Increase academic achievement. 5.a. GPAb. CTBS scoresc. SAES student questionnaire

6. Decrease alienation. 6.a. SAES student questionnaire

7. Increase attachment to school 7.a. SAES student questionnaire

II. Goal: Decrease disruptive student behavior. II.A. SAES student questionnaire

Objectives:

1. See objectives 1-7 above.

42

Table 1 (cont.)

Goal/Objective Measure

III. Goal: Increase student involvement in decision making III.A. Number of student initiated policy and procedurelevels. changes put into practice

Objectives:

1. Decrease student alienation. 1.a. SAES student questionnaire

2. Increase classroom participatory management. 2.a. Student self-reportb. Observations of classroom procedures by project

director

3. Increase leadership competencies of youth committee 3.a. Student self-repOrtmembers.

4. Increase perceptions of fair, and clear school rules.

IV. Goal: Decrease drop-out rate.

Objectives:

1. See Goal II, objectives 1-7.

2. Increase parents' awareness of their child's capa-bilities.

4.a. SAES student questionnaire

IV.A. Attendance data

1

2.a. Number of times parents participate in meetingswith project teachers and counselors

d3

Table 1 (cont.)

Goal/Objective Measure

V. Goal: Increase parental involvement.

Objectives:

1. Increase number of opportunities for parents to participate.

V.A. Number of parent initiated policy changes put into

practice8. Number of parent initiated procedure changes put into

practice

1.a. Number of parent meetings heldb. Number of parent committeesc. Number of parents attending meetings

2. Increase parents' skills to participate meaning 2.a. Number of PET meetings heldfully. b. Number of parents attending training sessions

/1

3. Increase community involvement. 3.a. Number of Advisory/Action committees heldb. Number attending meetings

Note. Goals and objectives were extracted from the Program Development worksheets and Lrom, Project STATUS: An overview,a paper presented by David Reiss at the annual meeting of the American Educational Research Association in New York, 1982.

Table 2

F.ovoral

Trmtment and Control Comparisons on

l!Ip!tlionnairo Mvo,tow; a rniim hplmontallan Objectives

Treatment

and Croup

Individualizea d

Instruction

M Sd N

Field Tripsa

N Sd N

Student - Teacher

Interactionn

II Sd N

bDegradation

M Sd N

Influence

N

Studentb

Sd N

ELIOT JUNIOR HIGH SCHOOL

Options

Treatment 2.68* 1.90 59 1.59** 1.04 59 1.52** 1.37 60 2.18 1.57 57 2.71* 1.89 56

Control 1.84 1.86 51 .49 .81 55 .90 .99 51 1.88 1.90 51 1.83 2.04 48

YC 6 LTC

Treatment 1.89 1.86 71 .94** .96 50 1.48 1.05 48 2,80 1,75 45 2.36 2.00 45

Control 1.61 1.59 49 .47 .73 74 1.16 1.16 79 2.47 1.69 72 2.21 1.81 71

Co

tN)

MUIR IITCH SCNOOL

I

Options**

Treatment 2.33 1.93 54 1.61 .98 54 1.60 1.26 52 2.10 1.60 51 3.02 1.97 51

Control 1.67 1.75 54 .42 .66 55 1.17 1.19 53 1.57 1.45 54 2.34 2.22 50

Note: Due to small number of non-equivalent controls (n=6) meaningful comparisons cannot be made between Youth

Conatta members and controls at Muir High School

`Individual questionnaire items

hIndividual level sttident scales

*r.05

**p.01

46

Table 3

Average Number of Times Project Status

Students Participated in Program Activities

Semesters in Options Semesters in LTC YC Meetings Task Forces Field Trips

Group M. SD N M SD N M SD N M SD N M SD N

ELIOT JUNIOR HIGH SCHOOL

Options .91 .94 114 .09 .41 114 .45 1.92 114 .11 .49 114 .08 .36 114

YC/LTC .10 .35 140 .36 .74 140 2.70 3.97 140 .64 1.07 140 .43 .70 140

MUIR HIGH SCHOOL

H Options .91 .95 108 0 0 108 .39 1.29 108 .01 .10 108 0 0 108

w YC/LTC .03 .23 91 .25 .44 91 2.84 2.35 91 .07 .29 91 0 0 91

Note. The information presented here is based on end of the year project reports on the nature and extent of program

activities for each student. This includes both treatment and control groups since some control students participated in

other aspects of the program.

45

47

Table 4

Options Students

Ratings of High Interest Curriculum

at Eliot Junior High

Unit I Family

Percent

Item Agree N

Unit II - Law

Percent

Agree N

Unit III - Person

Percent

Agree N

Unit IV - School

Percent

Agree N

Understood purpose 94% 127 100% 42 93% 42 100% 40Information was relevant 70 125 54 42 66 38 61 41Eajoyed activities 82 129 88 41 93 40 93 42Did outside work 39 126 43 42 37 41 39 41Read and understood materials 94 123 95 41 92 42 100 40Speakers, films helpful 92 114 88 42 93 42 88 41Would recommend class

i

H

to friends 59

Class more interesting

than others 54

127

126

55

70

43

43

71

39

41

41

66

63

/41

41'42 Materials different than1

uther classes 90 123 93 43. 90 41 90 40

Class lived up to expecta-

tions 65 124 85 40 66 35 73 37

Note. Rating forms were given to both English and social studies Options students at the end of each unit.

49

Table 5

Options Students' Ratings of

High Interest Curriculum at Muir High School

Item

Unit II - Law

Percent

Agree N

Unit III - School

Percent

Agree N

Unit IV - Job

Percent

Agree N

Unit V Life

Percent

Agree N

Understood purpose 94% 112 93% 102 97% 33 100% 18

Information was relevant 55 110 66 101 79 33 65 17

Enjoyed activities 91 111 95 102 88 32 94 18

Did Outside work 27 107 23 104 52 33 56 18

Read and understood

materials 96 110 96 103 94 33 94 18

Speakers, films helpful 96 110 89 102 94 33 89 18

Would recommend class to

1

friends1-' 93 110 90 104 88 33 94 18

Class more interesting

1

than others 77 110 87 104 78 32 89 18

Materials different from

other classes 88 109 88 104 88 32 100 18

Class lived up to expecta-

tions 79 105 74 98 84 31 89 18

Note. Rating forms were given to both English and social studies Options students at the end of each unit.

52

Table 6

Student Ratings of LTC Curriculumat Muir High and Eliot Junior High

Item

Muir Higher Eliot Junior HighPercent N Percent NAgree Agree

Understood purpose 100% 13 96% 23

Information was relevant / 82 11 83 23

Enjoyed activities 100 13 91 23

Did outside work 85 13 52 23

Read & Understood materials 92 13 87 23

Speakers, films useful 92 12 86 21

Would recommend this class to friends 100 13 91 22

Class more interesting. than others .. 85 13 77 22

Materials different from other classes 92 12 91 23

Involved in rule development 83 ,, 12 71 21

Involved in class constitution dev't. 33 12 48 21

Involved in planning 50 12 76 21

Involved in making suggestions 75 12 82 22

Class lived up to expectations 100 13 45 20

Involved in selecting school issues 73 11 75 21

Expressed opinions 100 13 67 21

Class conducted by students 85 13 - _ _ _ __ _ _11_ _ _21

Different points of view heard 100 13 77 22

Decisions based on general aggreement 62 13 82 22

Students responsible for o n behavior 83 12 52 21

Students respect different opinions 92 13 50 20

Little wasted time 85 13 45 22

Students decide plan 100 13 73 22

Equal group representation 69 13 55 22

Developed ability to lead group 100 13 91 22

Note: Leadership training class students were given course evaluations at theend of the year.

aResponses are from second semester students only.

-186-

53

Table '1

Means and Standard Deviations

on Fall '81 Pretest Measures- -

Eliot

Treatment

oppamorjammirww.m..mm.

1!,

Scale

Options Class Youth Committee

Treatment Control Treatment Control

M SD N M SD N M SD N M SD N

Alienation

Attachment to Parents

Negative Belief (Non-Belief in

Rules)

School Effort

Negative Peer Influence

Interpersonal Competency

High Internal Control

Practical Knowledge

Positive Self-Esteem

Attachment to School

Involvement

Rebellious Autonomy

92

1,28

1,41

7.17

1,80

3,88

3.07

8.57

1.75

3.81

-.53

1.58

1.08

2.49

1.37

1.69

1.78

1.00

1.30

2.78

4.08

5.46

5.27

1.05

44

42

42

42

40

42

43

42

28

42

40

43

.84

1.94

1.81

7.42

1.81

3.91

3.30

9.11

1.54

3.04

1.00

1.64

.83

3.06

1.34

1.60

1.86

1.05

1.02

2.89

4.86

3.35

5.80

1.12

35

37

38

36

33

38

38

38

24

36

33

38

1,23'

.61*

1.60

6.79

1.89

4.02

2.84

9,53

.99

1.02

1.67

1.85

1.18

2.80

1.29

1.67

1.88

1.09

1.08

3.29

5.06

6.78

6.66

.92

48

48

48

46

44

49

49

49

29

47

46

48

1,14

-.71

1.55

7.07

1,65

4.05

2.90

9.81

1.77

1.91

1.41

1.81

.94

4,07

1.29

1.63

1.92

1.05

1.18

2.86

4.93

5.45

6,27

1.00

71

71

72

64

69

72

72

71

43

69

61

72

'fit -test for difference between treatment and control is statistically significant at the p<.05 level, indicating source

of non-equivalence of groups at pretest.

**cutest for difference between treatment and control is statistically significant at the p<.01 level, indicating source

4. of non-equivalence of groups at pretest.

Note, Refer to Char ter 4 of the First SAES Interim Report for description of scale construction. Item number 32 was

omitted from the 'Attachment to Parents" scale, item number 45 from the "Negative Peer Influence" scale. The scale

"High Internal Control" is not discussed in chapter 4. It includes the following three items from the pretest:

--Luck is more important than hard work.

--Getting what I want has little or nothing to do with luck.

--Every time I try to get ahead, something or someone stops me.

54 55

Table 8

Means and Standard Deviationson Fall '81 Pretest Measures- -

Muir Options Class

Scale

. Treatment Control

M SD N M SD N

Alienation 1.10 1.01 51 1.14 1.14 23

Attachment to Parents -.46 3.13 51 .15 2.10 24

Negative Belief (Non-Belief in Rules) 1.25 1.22 48 1.42 1.26 19

School Effort 7.14 1.61 50 7.33 1.56 24

Negative Peer Influence 1.82 2.07 47 1.17 1.68 23

Interpersonal Competency 3.99 .88 50 4.00 1.00 23

High Internal Control 3.38 1.02 50 3.09 1.20 23

Practical Knowledge 9.49** 2.51 51 7.69 .2.67 24

Positive Self-Esteem 1.77 3.39 40 2.08 3.58 21

Attachment to School 3.84 4.19 47 2.77 3.29 22

Involvement -1.11 5.18- 46 - _-1.34 - 5.69_ 23 _

Rebellious Autonomy 1.68* .98 49 2.23 .87 22

*t-test for difference between treatment and control is statistically signifi-cant at the p<.05 level, indicating source of non-equivalence of groups atpretest.

* *t -test for difference between treatment and control is statistically signif-icant at the p<.01 level, indicating source of non-equivalence of groups atpretest.

56-188-

Table 9

Treatment Control Group Comparisons on Several

Measures of the Project Status Goal,

"Decreasing Class Cutting"

Treatment

and group

Class Cuttinga

M Sd N

School Cuttinga

M Sd N

Unexcused Absencesb

M Sd N

ELIOT JUNIOR HIGH SCHOOL

Options

Treatment .61 1.19 56 .43 1.08 56 1.56* 3.15 64

Control .36 .86 33 .09 .30 32 0.58 1.33 55,

1 ,Youth Committee

M0 Treatment .71 1.11 42 .29 .74 42 0.43 1.07 54

Control .50 1.05 72 .36 .92 72 0.99 2.04 86

MUIR HIGH SCHOOL

Options

Treatment .91 .97 47 .57 1.16 47 1.42 2.83 55

Control .93 1.00 40 .68 1.10 40 1.95 4.55 55

Note: Due to the small number of non-equivalent controls (nr.:6) meaningful comparisons cannot be made between Youth

Committee members and controls at Muir High School.

aClass cutting and school cutting are individual items on the SAES student questionnaire.

bMeasure based on school attendance records.

r.

*p<.05.

vu57.

To1111, 10

Treatment and Control Croup Comparisons on

Several Measures of the Project Status Cool,

"Decreasing Disruptive behavior"

Treatment

and Croup

!,11.tep.o.ted

'01..petolon4

4 Sd N

Sen-reporte

Oelinquuncy

H Sd N

Sll-rcporied

Drug nor

8 Sd N

Sll-reported

serious delinquency

M Sd N M M

lii 1081v

ExpolsIons

Sd N M

'Fall 001,

Suspensions

Sd N

Classrnor

Disrupt i

Sd

ELIOT JUNIOR 4ICH SCHOOL

Options

Treatment .24 .43 49 .17 .22 48 .20 .28 48 .14 .22 49 0.0 0.0 64 0.0 0.0 64 .50 .51

Control .25 .44 32 .14 .18 31 .20 .27 31 Jo .14 31 0.0 0,0 55 0.0 0.0 55 .33 .47

Ymth Cc!':;'111u.'

Treltrav111. .11 .32 35 .18 .19 30 .29 .29 37 .11 .19 39 0.0 0,0 54 9.0 9,0 54 .40 .50

Control .2h .44 62 .19 .22 64 .27 .30 65 .13 ,24 64 0.0 0.0 86 0,0 0,0 86 .49 .51

V) 01 III 11104 5011001.

C)

I Options

Treatment .09 .30 42 .21 .20 41 .34 .31 40 .13 .18 41 ,:n.o 0,0 55 0,0 0.0 55 .35 .48

Control .03 .16 39 .16 .16 37 .33 .34 31 .08 .11 37 0.0 0.0 55 0,0 0.0 55 .27 .45

Note; Due to the small number of non-equivalent controls (n.6) meaningful comparisons cannot be made between

Youth Commilto.. rwrbers and controls at Muir High School .

individual Item on SAES student questionnaire.

bIndividual-level student scale

C,1

hosed on school records.

60

Table 11

Treatment - Control GroupComparisons on a

Measure of the Project StatusGoal, "Decreasing Dropout Rate"

Treatmentand group Withdrawal from School

a

M Sd

ELIOT JUNIOR HIGH SCHOOL

OptionsTreatment 0.13** 0.33 64

Control 0.35 0.48 55

Youth CommitteeTreatment 0.06 .23 54

Control 0.13 .34 86

MUIR HIGH SCHOOL

OptionsTreatment .09 .29 55

Control .20 .40 55

Note: Due to the small number of non-equivalent controls (n=6) meaningful comparisonscannot be made between Youth C.ommittee members and controls at Muir High School.

aWithdrawals were collected from school records and include reasons such as transferred,graduated, involuntary, and voluntary. The reported mean combines involuntary and voluntary

withdrawals.

**p<.01.

Table 12

Treatment and Control Comparisons on

Several SAES Questionnaire Measures of Project Status' Objectivesa

Treatment

and Group

Alienation

M Sd N

Positive

Self-esteem

M Sd N

Attachment

to School

M Sd N

Fair

School Rules

M SD N

Clear

School Rules

M SD N

ELIOT JUNIOR HIGH SCHOOL

Options

Treatment .31* .25 47 .72 .17 42 .71 .25 51 2.70** 1.81 53 3.42** 1.76 57

Control .45 .23 28 .72 .19 26 .67 .24 32 1.48 1.66 46 1.77 1.87 47

,110.11 CummitC0

I

f-i

Treatment .43 .30 34 .71 .15 31 .58 .28 37 2.09 1.66 46 2.83 1.69 46

UD Control .36 .27 55 .73 .18 53 .62 .25 63 2.33 1.69 67 2.79 1.77 73

NI

MUIR HIGH SCHOOL

Options

Treatment .36 .27 35 .72 .19 32 .74 .23 44 2.51 1.64 47 2.98 1,76 49

Control .31 .31 38 .73 .16 33 .75 .23 39 2.33 1.86 51 2.63 2.01 54

Note: Due to small number of non-equivalent controls (n=6) meaningful comparisons cannot be made between Youth

Gumittee members and controls at Muir High School. All measures are individual level student scales.

aSce section on objectives for more complete description.

*p<,05

**p<.01

6263

1.4

Table 13

Treatment and Control Group Comparisons on

Several Measures of the Project STATUS Objective,

"Increase Academic Achievement."

Treatment Self-reported Gradesa Self-reported Readinga Fall GPAb Spring GPAb

and Group Ability

M Sd N M Sd N M Sd N M Sd N

ELIOT JUNIOR HIGH

Options

Treatment 2.50 .84 54 1.96** .79 52 2.34 .66 53 2.29 .61 54

Control 2.41 .61 32 1.38 .80 26 2.28 .62 41 2.30 .68 36

Treatment 2.53 .83 43 1.70 .79 40 2.36 .77 46 2.39 .74 50

Control 2.80 .80 71 1,94 .87 69 2.53 .85 80 2.50 .89 74

MUIR HIGH SCHOOL

Options

Treatment 2.62 .74 47 1,74 .80 46 2.31 .91 48 2.50 .87 47

Control 2.88 .85 40 .1.92 .76 37 2.68 .91 43 2.72 .92 44

Note: Due to small number of non-equivalent controls (n=6) meaningful comparisons cannot be made between Youth Committ

members and controls at Muir High School,

aAn individual item on the SAES student questionnaire.

bMeasure based on school records.

**p4.01

6,165

Ireltnnott

and Croup

Talent'.

Ii

,rih.rik on

i ii

Sd N

Altarhment to

parents

Sd N

options

Treatment .71 .29 48 .64 .26 ' 55

Control .72 .28 29 .63 .25 33

Mall Con-ltice

Treatment .81 ,26 34 ,481* .31 42

1 Control .80 .32 98 .64 .29 70

I-4

V)

Options

Treatment .61 .29 37 .49 .29 46

Control ,58 .30 38 '11 .23 40

Thhlo 14

trpaimeni and rolorol cnnwri.mn1 un

Sevoral IllitrouloN of Theofeliedi 101 oreqt

IOW In II (Ii Internal Negative

Conventional folio; Compotonoy Competency Control Veer Influec

M Sd N N Nd N M Sd N M Sd N H Sd

EMT JUNIOR HICH SCUM,

.65 .24 46 ,84* .18 46

.59 .25 25 .73 .19 27

.70 .22 33 .79 .22 33

.64 .25 56 .80 .21 56

MUIR 111011 SCHOOL

.59 .26 33

.65 .26 38

.76 .23 34

.76 .22 38

1,40 .48 43 .47 .28 46 .23 .21

1.26 ;43 26 .53 .24 26 .19 .17

1.41 .50 32 .40 .26 33 .23 .21

1.38 ;43 '55 .46 .24 55 .24 .21

1.348 .32 33 .44 .26 36 .20 .20

1.22 .41 38 .37 .29, 36 .16 .17

Note: boo to mall number of non-mivalent Youth Committee controls (n4) meluingful conyarisons cannot be Made

between Tooth Committee members and controls at Muir HO School. All outcomes are individual-level student scales from

the SACS questionnaire.

aAt pretest, 'pt Ins studonts were significantly higher than controls (pc.01).

p

66 .

ET COPY ARILABLE

67

TABLE 14

cont'd

Rebellious

Treatment School Effort Involvement Invalidity Autonomy Victimization Punish Rewards

and Croup M Sd N M Sd N M Sd N M Sd N M Sd N M Sd N M Sd

ter,.`.,. ELIOT JUNIOR 11011 8Q14.Q1,

Options

Trest rraControl

' Youth tionitter

?wontf7nttel

I.`,

P**1 '

VI r "'

141,---.1

r Optimum

Treatment

Antrul

.60 .33 56 .26 .20 51 .16* .24 45 .41 ,37 43 .20 .25 51 .31 .30 53 .36 .32

.64 .24 31 .21 .19 33 .27 .24 27 .57 .28 25 .18 .21 30 .31 .25 30 .27 .27

.55 .34 41 .350 .17 40 ,18 .25 34 ".66 .32 30 .15 .22 36 .29 .31 36 .36 .33

.61 .32 72 .26 .18 66 .19 '.20 56 , .64 .35 55 .17 .23 64 .31 .34 67 .34 .29

.

i1UIR RICH SCHOOL

.

.51 .32 46 .246 .17 46 .21 .24 34 .63 .36 32 .14 .21 43 .17 .24 44 .27 .29

.66 .31 39 .19 .14 34 .11 .17 37 .63 .33 35 .11 .14 38 .11 .18 38 .21 .28

I

1When excluding controls who participated in the Youth Committee, Options were significantly higher than

1 controls (11(.01)

Project RETAIN, Chicago Board of Education: Evaluation Report

J. St. John

Abstract

Responsive Education through Alternative Instructional Networks(Project RETAIN) was an alternative education project of the ChicagoBoard of Education sponsored primarily by a grant from the Officefor Juvenile Justice and Delinquency Prevention (OJJDP) as part ofits Program in Delinquency Prevention through Alternative Education.Operating in nine schools in the complex Chicago public school sys-tem, RETAIN sought to use individualized learning plans and the

attention of special teachers to reduce -problems of poor school

attendance, disruptive behavior, and low student achievement. Theevaluation provides no evidence that this project achieved its goalsor the results sought by the OJJDP., This report describes the pro-ject and its evaluation.

RETAIN (Responsive EducationThrough Alternative InstructionalNetworks) was a project undertakenby the Chicago Board of Education

with support from the Office for

Juvenile Justice and DelinquencyPrevention (OJJDP) as part of thatoffice's Program in Delinquency Pre-vention through Alternative Educa-tion (OJJDP, 1980).

Project RETAIN Plans

Goals

RETAIN was aimed at the problemsof poor attendance, disruptivebehavior, and low achievement in

Chicago public schools. Attendanceproblems included class-skipping andabsenteeism in the high schools and

This report supercedes an earlierreport (St. John, 1982) on this pro-ject. It covers the entire two

years of project operation. Donald

E. Rickert, Jr., analyzed the datafor the second year of the project'soperation, and wrote the sectiondescribing those results.

-197-

absenteeism in the elementaryschools. Disruptive behaviorincluded minor classroom or school

disruption as well as serious

assaults. Achievement problems wereevident in the results of standard-ized tests. The project consideredpoor attendance and low achievementto be both individual- and school-level problems.

Rationale

The project's staff listed manycauses of these problems, and

divided them into five categories:(a) Parental and domestic causes,(b) school staff causes, (c) schooladministration causes, (d) communitycauses, and (e) student causes.Parental and domestic causes rangefrom child abuse, to keeping chil-dr.an home to babysit, to an inabil-ity to meet the school healthrequirements for enrollment. School

staff causes include ineffective andinappropriate teaching styles, cul-tural insensitivities and negativebiases, and self-fulling prophecies.School administration causes range

from attendance policies (locking

school doors when the tardy bell

70

RETAIN

rings) to unfair disciplinaryactions and the escalation of minorincidents into major episodes. Com-

munity causes include discrimina-tion, unsafe streets, and lack ofcommunity support for the school.This lack of support occurs becausethe school "does not value communityparticipation" and because the "com-munity does not view schools as a

community structure that they areinvolved in influencing and cooper-ating with." Student causes mightbe economic concerns that conflictwith school attendance, lack 'of

motivation, not . valuing school,negative attitudes toward theschool, poor peer relations, lack ofsocial skills and self-discipline,low self-esteem, or lack of basicskills.

Although RETAIN did not discussdelinquency as a problem or set thespecific goal of delinquency preven-tion, the project's theory containsexplanatory elements similar to

those in Cloward and Ohlin's (1960)and Cohen's (1955) theories of del-inquency.

Objectives

Although the project revised itslist of objectives several timesduring the 1981-82 school year, thelist eventually produced resembled alist of interventions more than itresembled objectives (intermediateoutcomes resulting from interven-tions, as we understand the term).In other words, the list of objec-tives produced may have resembledobjectives of the "management by

objectives" sort more than objec-tives in Program Development Evalua-tion sense.

Although the project's theory

specified the system's numerous con-tributions to the problems at hand,and the project's goals impliedschool-level outcomes, no school-level objectives were specified.

-198-

Interventions as Planned and Modifi-cations

First program year. During thefirst program year the RETAIN staffspecified the following major inter-ventions: (a) increasing parentalawareness, (b) linking families withneeded services, (c) developingindividualized curricula,(d) advocacy, (e) peer group coun-seling, and (f) continued in-servicetraining for.RETAIN teachers. Thesecomponents were discribed in an ear-lier report (St. John, 1982), and

were modifications of the interven-tions specified in the original pro-posal (Chicago Board of Education,1980).

The primary focus of the projectbecame the implementation of Indivi-dual Learning Plans (ILP's). The

project's staff reported that ILP'sdeveloped .in _the_ early_ _stages of _implementation were not used effec-tively. Accordingly, they decided

that more in-service training for

RETAIN teachers was needed to assistthem vita developing and using thisapproach to education. Plans for

the in-service training were carriedout in September, 1981, just as

school opened.

Second program year. The list ofRETAIN interventions continued to

evolve during the second programyear. By the end of the second pro-gram year, project managers listedthe following major project compo-nents: (a) parental awarenessactivities, (b) efforts to establishlinkages with Chicago Board of Edu-cation social service agencies andother community agencies and use

these services, (c) individuallearning plans for attendance,behavior, and achievement problems,(d) career education (individualizedand not for all participants),(e) computer-assisted instruction,

(f) summer '82 RETAIN program,

71es,

(g) in-service training for RETAIN

teachers, (h) staff interventionanalysis meetings, and (i) evalua-tion tasks including student selec-tion, questionnaire administration,data collection, and case study

reports. These components are

described together with informationon the degree to which they wereimplemented in a subsequent sectionof this report.

Many of the changes from year toyear were the result of attempts toreorganize plans according to Pro-gram Development Evaluation (PDE)

definitions of interventions, objec-tives, strategies (see Gottfredson,1982); some changes reflect the

amount of effort needed for success-ful implementation; and some reflectmajor changes in plans. These firstyear to second year changes can begrouped into two categorieg,(a) interventions dropped and added,and (b) shifts among PDE categories.

Interventions dropped or added.

Only the Peer Counseling componentwas dropped from overall project

plans. One factor contributing to

the discontinuance of the componentwas scheduling. RETAIN studentswere scheduled for project servicessuch that often only one or two stu-dents received services at a giventime. Not all of the participantswanted to be in peer counselinggroups and the chances of those whodid being scheduled to attend RETAINat the same time were slim. Alt-hough managers abandoned efforts toimplement this component at each

site, it remained the primaryapproach used by one RETAIN teacher.Seven of the remaining eight RETAINteachers reported using somethingthey called peer counseling withsome students in their school.

Managers added extensive plansfor the successful implementation oftheir complex evaluation. The ela-

-199-

RETAIN

boration of plans for evaluation wasperformed not because managers sawevaluation tasks as essential to

their project, but because the

national evaluators had observedevaluation tasks were not alwayssucessfully carried out in the firstyear (see St. John, 1982). It washoped that careful planning wouldresult in a stronger evaluation.

Intervention Analysis Meetingswere added by managers as a means ofincreasing the quality and effec-tiveness of specific actions takenfor individual participants.

Shifts among_ PDE categories. In

the first year, Advocacy (advocatingfor the rights and needs of partici-pants) was considered an interven-tion. However, a more detailedexplanation revealed that this was astrategy used in the implementationof three components: ParentalAwareness, Linking Families and Ser-vices, and ILP's.

Computer- Assisted Instruction waslisted as a strategy associated withthe ILP's in the first year, but inthe second year, was categorized asan intervention because of the con-siderable planning effort needed tomake the computers operational.

Career Awareness was shifted fromthe objective category to the inter-vention category because this was anactivity planned for participants.This could be considered another ofthe ILP strategies, but since majorefforts, especially in the summers,were devoted to this activity, man-agers decided to classify it as anintervention.

The Implementing Organization

Project RETAIN can best be under-stood in the larger context of the

Chicago Board of Education (CBE),

the organization that was awarded

7

RETAIN

the grant to conduct the project.Members of Board are mayoral appoin-tees. Through a long chain of com-mand, the Board oversaw implementa-tion of Project RETAIN (see theorganization chart in Figure 1).

The General Superintendent of

Schools, appointed by the Board, hasresponsibility for implementing allschool board policies. The projectitself was under the Deputy Superin-tendent for Field Services, whodelegated direct responsibilitiesfor overseeing implementation to theDirector of Field services (DFS),Central. The project was placed inField Services because of its plansto include non-academic services toparticipants and their families.The Department of Personnel hired astaff to work with the DFS to managethe project. The project managementstaff oversaw nine RETAIN teacherswho worked in nine Chicago publicschools. Eight of these teacherswere selected early in the projectaccording to Board guidelines, and

one, whom a principal especiallywanted in her school, was hired sev-eral weeks later. The third majordivision of the Board, InstructionalServices, Government Funded Projects(GFP), was involved because the pro-ject was funded by the government;this division oversaw budgetaryprocedures.

This arrangement was unusual forthe Board: Most government grantsare completely under the authorityof the Deputy Superintendent of

Instructional Services, GovernmentFunded Projects. The arrangementintroduced unexpected elements intothe project's forcefield as the twodivisions of the Board developedworking relationships. Both divi-sions had to be kept informed of theproject's progress. Initially pro-ject managers expected that projectplanning and implementation would bereported to and approved by FieldServices, and fiscal management mat-ters would be the purview of GFP.

However, GFP assigned a programmonitor to Project RETAIN in thesecond program year.

It was also unusual that theadministrators overseeing implemen-tation and project teachers haddirect contact. Ordinarily, projectmanagement would only have indirectcontacts with teachers. Due to thenature of the project's activities,contact between project managementand RETAIN teachers was arranged;and formal structures- were main-tained by communicating projectplans and activities to interveningdistrict superintendents and princi-

,,pals.

CBE Environment

Project RETAIN operated in a com-plex forsefield because of its

affiliation_with the: Board._ CBE iscurrently under pressure from theDepartment of Justice to implement asatisfactory desegregation plan.During the 1981-82 school year, thecourts and school board reached anagreement and the early stages of

the desegregation plan went into

effect. This public school systemis one of the largest in the countryand well over 50% of its studentsare minority. Declining enrollmentsfor the last several years haveforced school closings and teacherlay-offs. A sizeable budget deficithad to be immediately amelioratedbecause state laws make it illegalfor public institutions to operateat a deficit. The Board of Educa-tion is under fire from parents andcitizens groups, each with its ownparticular (and often conflicting)concerns. Very few days pass with-out news articles about the systemand its problems.

During the spring of 1981, a newGeneral Superintendent of Schoolswas appointed by the Board of

-200-

73

Education from outside the system,causing uncertainty about steps thatwould be taken to solve the problemsof the system. Many new administrators were appointed in 1982, including a new Deputy Superintendent forField Services. This change hadvery little effect on RETAIN operations.

A critical incident. The GeneralSuperintendent sought additionalfunding for special programs as a

means of addressing problems in thesystem. As part of this fundseeking activity she called on officialsat OJJDP to seek support for futuredelinquency prevention programs.During her visit, she was informedthat her system currently had a

large grant from the agency, and wastold that both the evaluation andproject activities needed strengthening before more funds could beconsidered. OJJDP officialsreported that she was apparentlyunaware of the project RETAIN--acircumstance that is not surprisingbecause the long chain of commandkept the project at some distancefrom her.

On returning to Chicago the Geri'-eral Superintendent met with projectmanagers, and subsequently appointedthe GFP monitor. She also approvedthe administration of the SAES student questionnaire from which theCBE had censored important items inthe project's first year. CBE managers were invited to brief highlevel CBE administrators about theirproject and its evaluation. For a

time after these activities, theRETAIN manager' interest in PDEplanning and project documentationincreased.

201

RETAIN

History of the Project

The proposal for Project RETAINwas developed in part in response topublic pressures on the school system to more adequately respond to

the needs of troubled students andschools. The Board of Educationassigned priority to alternativeeducation and commisFioned the Program Development Manager of the Center for Urban Education to developsuch a program and secure funds.Prior to this proposal, one otherunsuccessful attempt at funding hadbeen made.

The system had never beforeimplemented an alternative educationproject closely resembling projectRETAIN, but there have been otherprojects with similar goals. TheABC Project and the Family GuidanceCenter were both aimed at increasingstudent achievement and reducingabsente' 1-m and disruption. TAPIwas im to reduce truancy inschools plagued by this problem, andworked with younger students identifies as potential truants. Title Iprog. ams were planned to be implemented in east school, but implementatica did r3t reach the finalstages. TAP 1 was revitalized as

part of the desegregation plan in1981-2.

Project RETAIN Staffing andResources

Project RETAIN had four management positions, nine teachers, and ncomputer specialist. The Direcf,.....,r

(not funded by the grant) had prmary responsibility for communicating with other CBE administrators.The Project Coordinator oversaw allproject staff and the implementationof each aspect of the project. Thecoordinator worked directly with theteachers and kept the Directorinformed of progress and :.ssues.

The Management Specialist wa3 pri

RETAIN

marily responsible for fiscal

management and evaluative tasks, andreported to both the Coordinator andDirector. The Activity Specialistwas responsible for working directlywith the RETAIN teachers in develop-ing and implementing activities, andreported to the Coordinator. A Com-puter Specialist had responsibilityfor securing, installing, and pro-gramming a microcomputer system atthe project schools and for provid-ing in-service training in the useof the computers. Teachers reporteddirectly to the Activity Specialist.Total funding for two years of pro-ject RETAIN was $1.3 million, withmost being allocated to personnelcosts. (The two-year grant fromOJJDP accounts for about $1.1 mil-lion; the balance was a CBE contri-bution.)

The Chicago school system pro-vided Project RETAIN with nineteachers (whose salaries were paidfrom grant funds) who were to imple-ment the project at each of ninesites, including three high schoolsand six elementary schools. Theproject managers conducted pre-ser-vice training in 1980-81 to acquaintthose teachers with the goals andobjectives of the project and to

assist them in developing newapproaches to the students who areoften neglected by the system, andongoing inservice training in1981-82 for continued assistance.

The RETAIN Structure

Project RETAIN served three clus-ters of three schools each: TheNorthside (School District 3),

Southwest (School District 12) andSoutheast (School District 19) com-munities. These clusters were cho-sen because they represent threedifferent population groups from thecity and are located in high crimeneighborhoods. Each cluster is com-I-Jsed of one high school and two

feeder elementary schools (gradesK-8) or middle schools. Theseschools were identified as havingserious problems with one or all ofthe project's target problems.

Elementary school level. Grades7 and 8 are, for the most part,housed in elementary schools in theChicago school system. These gradesare organized into classes whichstay together for the entire schoolday mostly in one classroom with oneteacher. Some elementary schoolshave "walking" reading and math pro-grams, which mean the studentschange classrocms to attend classes

,,with specialty teachers. These

schools are usually small with 2 or3 'lasses in each grade level. In

some schools, each grade level fol-lows the same basic schedule duringthe day.

-202-

This highly structured _environ-ment helped structure the RETAINproject. At all elementary schoolsites, the RETAIN teacher had a roomwith a telephone set aside forRETAIN activities. The RETAINteachers generally visited otherteachers each morning before schoolstarted. This was done to check onparticipant attendance, achievement,and classroom behavior from the pre-vious day. They returned to theRETAIN classroom to stand in the

hall and greet students arriving forthe school day. When a participantwas not seen, other locations in theschool were checked, and if the stu-dent could not be found, the RETAINteacher contacted the parents by

phone. In this way, the projectteachers monitored the attendance ofand had daily contact with partici-pants.

Participants usually attendedRETAIN in small groups of three orfour during the school day. Small

numbers allowed RETAIN teachers to

work with students either in small

groups or individually and provideeither academic instruction or coun-seling. The results of some stu-dent's learning style inventoriessuggested that small group instruc-tion was appropriate for them; oth-ers needed individual attention orto work mostly on their own. RETAINteachers attempted to schedule stu-dents for whom small groups weremost appropriate at the same timeperiod. But scheduling constraintsdid not always allow for this--onlycertain time periods were availablefor attending RETAIN both because ofstudent schedules and because RETAINteachers had other project dutieseach day. RETAIN teachers had to

have large blocks of time each weekfor making home visits and forattending every-other-week projectstaff meetings. Project staffreported that these constraints alsoresulted in a catch-as-catch-canschedule resulting in varyingamounts of contact with RETAIN par-ticipants.

High school level. Chicago highschools are large schools that

include grades 9 through 12. Most

of these schools have staggeredstart-up times for each class level.The daily schedule involves changingclasses every 52 minutes for 7 to 10class periods each day. Studentsare not necessarily scheduled forinstruction each class period.

The school organization helpedstructure the high school, level

RETAIN project. Each RETAIN, teacherhad a phone and classroom, or areaof a classroom, reserved for RETAINactivities. Generally attendancewas monitored by the having partici-pants come by the RETAIN classroombefore reporting to their homeroom.Later in the day, some RETAIN teach-ers went on what they called"rounds." This activity involvedchecking the classrooms studentswere supposed to be in at that time

-203-

RETAIN

period. Students who had notreported in the morning and who werenot in their classes were thought tobe absent and thier parents were tobe called. Students who were in

attendance but not in their classeswere to be counseled for class skip-ping.

Participants were scheduled to

attend RETAIN either during theirstudy hall time or before schoolstarted in the mornings. The stu-dent's schedules determined whenthey could participate in RETAIN andhow often during each week. If a

student were selected and his or herschedule conflicted with the timesthe RETAIN teacher had available forclassroom work, arrangements weresometimes made to change the stu-dent's schedule. When all routeswere blocked the RETAIN teachers andparticipants worked out the bestpossible arrangement for early or

late in the day contacts. RETAINteachers tried to schedule studentswith similar problems and learningstyles at the same time, but becauseof scheduling problems they were notalways successful.

Participants. The project plansspecified that 15 students wouldreceive direct services at eachschool. Students were to be chosenrandomly frOm pools of referrals.School staff were asked to referstudents with attendance, achieve-ment, or behavior problems for anycombinations of these problems) tpthe RETAIN teacher. Parental con-sent was required for participation.

Length of participation. Parti-cipants were to remain in the pro-ject until the RETAIN teachersjudged that they no longer neededalternative education. The projectdid not specify how much contacteach student would have with RETAIN,nor did it establish guidelines forthe termination of participation.

76

RETAIN

Forcefields

Each teacher had responsibilityto clear with the principal anyalternative procedure or policy theproject planned to implement in theschool. For instance, in most pro-ject schools, the RETAIN teacherswere relieved of hall monitoringresponsibilities because the projectrequired them to leave the campus tocontact families. Regular responsi-

--bilities in the school would havehindered the implementation ofimportant project interventions.Project RETAIN managers reportedthat not all principals were suppcir-tive of the project efforts in theirschools.

Problems in implementing some ofthe program strategies at someschools, including the advocacy rolefor RETAIN teachers and release timefor RETAIN teachers' home visits,were never overcome. The principalat one of the schools reported thathe suspected the teacher originallyhired to implement RETAIN at hisschool was abusing the off-campusarrangements for making home visits.In the Fall 1981 school semester, heblocked implementation until theteacher was replaced with someonemore to his liking in January 1982.

This action (at Sheridan) createdanother problem. The project Activ-ity Specialist had to spend a greatdeal of time training the new staffmember. Furthermore, the principalcontinued to disallow home visits.

The special activities and statusof the RETAIN teachers could haveled to tense inter-teacher relationsif the project's emphasis on helpingstudents with special needs were notaccepted by all teachers. Facultycontacts were maintained as a meansof building support among peers forthe project.

-204-

Other school administrators andthe Board supported the project, butthe project managers spent much timeworking through bureaucratic proce-dures.

Cross site variations. Extensivecross-site variations in the imple-mentation of the project resultedwhen the project caved in to princi-pal demands. At Bowen and Gage Parkhigh schools and Bontemps ElementarySchool, the principals required thatRETAIN teachers service more thanthe 15 selected participants. AtThorp elementary school the princi-pal used the RETAIN classroom as the,site for an experimental in-schoolsuspension project. The RETAINteacher had responsibility for thesuspended students and generallyinvolved them in RETAIN activities.The necessity to be at school allday for this non-RETAIN activityprevented this RETAIN teacher frommaking home visits during workinghours. At Blaine Elementary School,the RETAIN teacher was often calledupon to counsel non-RETAIN studentsreferred to the office for discip-line reasons. Several RETAIN teach-ers reported that principals calledon them to contact the families ofnon-participants because all otherattempts had been unsuccessful. AtBontemps, the RETAIN teacherexplained that one or two studentshad been placed in RETAIN so thatthe families could be contacted togive consent to assigning the stu-dent to the Special Education track.

RETAIN teachers repOrted that thedifference between the RETAIN parti-cipants and non-participant reci-pients of services was that no ILP'swere developed for the non-partici-pants, and contacts were less regu-lar for most of these students. Atmost sites, principal demands werehandled as additional responsibili-ties. Managers reported these unan-ticipated variations did not harm

RETAIN implementation. Only at

Sheridan Middle School was it necessary to drop a component.

What Was Implemented?

The project did not participatein Program Development Evaluation(PDE) reporting procedures in the

way we had intended. A managementplan was not completed during the

first PDE planning conference, norwas much systematic planning documented. In the first year, generalreports of successes were oftengiven. These reports usually toldof necessary successes in proceduralmatters within the bureaucracy.Implementation in the schools was

not documented, and so can notreally be described.

During the second program yearproject staff struggled with'documenting programmatic efforts in

their PDE plans. Although weattempted to clearly describe expectations, managers reported a lack ofunderstanding of the expectationsfor planning and reporting. Severaltimes they commented on the bulkyplan that would result if they documented project activities at each

school. Such documentation wouldhave been necessary because each

component had some exceptional variation at one or more of the sites.PDE plans were never developed;other documents were substituted.

Site visits were conducted as ameans of gathering information aboutthe project operations at the nineschools. All RETAIN teachers wereinterviewed about how they executedtheir understanding of the projectplans, a sample of ILP's werereviewed, and teachers were observedwhile working with students. Sevenof the principals at the implementation sites were interviewed.

The following description of what

205

RETAIN

was implemented is based, then, on

the documents furnished by RETAINmanagers, observations at eachschool, and comments made duringinterviews with RETAIN staff.

Parental awareness. Conferenceswith parents continued to be theprimary method of making parentsaware of the educational problemstheir children experienced and

involving parents in the solution ofthese problems. The role of theRETAIN teacher in these-sessions wasthat of an advocate for the students. Many conferences took placein the homes of the students.

At Nightengale Elementary Schoolthe RETAIN teacher organized a regular weekly parents meeting. Parentsat this school were assisting in theproject and three parents were at

the school the day I visited. I

also saw pictures in the RETAINclassroom of social events sponsoredby the project for both participantsat that school and for three RETAINprojects in nearby schools. Projectmanagers indicated that the teacherwas successful with the councilbecause ths,1 neighborhood had longtime residents who had always beenactive in school activities. At

Blaine and Bontemps ElementarySchools, the RETAIN teachers hadimplemented individual counselingwith several of the students' families in an effort to improveparent/child relationships. Sometimes the teachers helped familiesbegin counseling at local communitymental health centers. RETAIN managers also developed three newsletters and sent them to the parents ofparticipants.

Linking families and services.The Chicago Board of Education is acity agency. This status could linkthe Board with other divisions of

city government such as law enforce-

78

RETAIN

ment, the judiciary, health and

human services, and political bodiessuch as the City Council and theMayor's Office. Project RETAIN is

therefore potentially linked tothese organizations through theBoard.

The Board of Education maintainsmany social and supplemental educa-tional services through its threePupil Service Centers, and by virtueof the project being a part of theBoard of Education, these serviceswere automatically linked. Projectmanagers reported making arrange-ments for quicker processing of

requests for services from thesecenters.

RETAIN teachers and staff estab-lished relations with community ser-vice organizations and businessesnear some of their schools. Threeschools reportedly establishedinformal RETAIN Parents Councilsthat included members from communityorganizations. RETAIN managers alsoacquired a list of service organiza-tions categorized by geographic areaand type of service and distributedthe lists to the RETAIN teachers.

Although project managers pro-vided little in the way of guide-lines for determining whether or notto refer families for services, theteachers apparently followed theirhunches about family circumstancesand consulted other people acq-uainted with the students. Clearlysome decisions needed little gui-dance--one teacher reported she hadreferred a family to the local men-tal health center because the fatherwas an alcoholic who Abused his

child.

Individualized Learning Plans(ILP's). RETAIN teachers gatiL.slred

social and academic histories forthe participants and devised indivi-dual plans for achievement, atten-dance, and behavioral objectives

based on this information. The

teachers were also expected to det-ermine their students' individuallearning styles and plan teachingmethods that matched student learn-ing styles. Once these data weregathered, students and teachers wereexpected to develop a contract thatspecified long term goals, shortterm objectives, the activitiesplanned for reaching the objectives,and the means of regular evaluationof the students progress. For exam-ple, a student who hadbeen chroni-cally Absent might have as a goal toattend school every day he or she isnot ill. The short-term objectives

,might be to first begin attendingthree days a week, then four, andfinally five, with attendancereported by the students' regularclassroom teachers as the means ofevaluating whether or not the objec-tives were reached.

-206-

Individualized 1-31-ani -were- to

cover any individual counselingneeds with which RETAIN teacherscould help. The plans were also tospecify individual career interests,plans for increasing awareness of

jobs in fields of interest, instruc-tion needed for particular jobs, anyjob seeking-skills that may be

needed, and appropriate job place-ments for eligible students. Compu-ter- assisted instruction was plannedas one aid to implementation of

learning plans.

The project managers encountereddifficulties in their attempts to

standardize the ILP recording formatacross schools. Despite the manag-ers' honest efforts to devise a com-mon format by the end of the yearthere were as many recording formatsas there were RETAIN teachers. TheProject Coordinator reported thatthis was because each teacher feltmore at ease with his or her own

recording format.

7(9.0

RETAIN teachers varied in the

degree of specificity of both long-term goals and short-term objec-tives. Several reported confusionover these details. Despite theseproblems, each current participantat each school had a file foldercontaining records of goals and

objectives contracted for, and listsof actions taken by each teacher,dates the action was taken, and somereport of results of the actions.

Students reported they knew abouttheir plans and contracts; severalcould state some of their goals andthe reasons they thought working toachieve the goals would be benefi-cial. Students reported they wereaware of most of the actions takenon their behalf by the RETAIN teach-ers, and many reported they appreci-ated the caring and help re-ceived--even when the help involvedthe dreaded parent contacts.

Career awareness. Project staffreported that records of specificcareer awareness activities for theindividual students appeared on

their ILP's. Some of the approachesincluded the use of CVIS availableon the Board of Education's downtowncomputer, speakers, visits to job

sites, study of occupation-specificpreparation units before visitingsites, help with job seeking skills,and the "World of Work" component ofthe 1981 summer program.

Computer-Assisted Instruction.

This component met with obstacleafter obstacle and, although plannedto begin in the first year of imple-mentation, did not get off the

ground until the fall of the secondyear. The first unanticipatedobstacle was that the Board of Edu-cation guidelines required that thepurchase of the equipment be con-

ducted through a competitive biddingprocess. This unanticipatedrequirement delayed obtaining the

RETAIN

equipment. Later, arrangements forelectrical outlets and overnightsecurity slowed down installing theequipment. Then difficultiesacquiring software (computer pro-grams) were encountered when projectmanagers found tha$: the Board of

Education also had guidelines for

what could and could not be

installed on the computers. The

computer expert on the managementteam developed a few small programsfor the computers so teachers andstudents could begin using the

equipment, and he conducted on-siteworkshops in the use of the

machines. The software was eventu-

I,ally acquired and installed, but oneof the schools had its computer sto-len. Parents at this school we're

working to replace the computer whenI visited the school in March.

-207-

At some schools the RETAIN teach-ers regularly used the computers togive additional instruction in mathand English. One student reportedthat the machine was fun to usebecause it divided complex problemsinto little steps and immediatelyreported to the student whether ornot each step had been properly exe-cuted. Another student had begunwriting simple programs that madethe computer print out geometricdesigns. The computer kept indivi-dual loge of student use thatincluded amount cf time, units com-pleted, and degree of mastery in

each unit. Teachers -4ho used the

computers recorded the computer loginformation on students ILP's.

In the beginning of the school

year project managers had plannedfor students to be tr:',1sported to

DePaul University for additional

computer instruction. These planshad to be abandoned because of the

high fees that would have beencharged, large amounts of time thatwould be 'needed r:o transportdents to the University and 't,ack

RETAIN

again, and because Universitypersonnel expressed some reserva-tions about having groups of youngstudents using their facilities.

Summer program. During the sum-mer of 1981, the project implementeda special program to work with allRETAIN students for the entire dayat the nine school sites. The sum-mer program emphasized career educa-tion and remedial instruction. Theyhad hoped to employ the RETAIN highschool students as tutors for theelementary students with CETA fund-ing but were unable to do so.

Early in the 1981-82 year, pro-ject managers had not decidedwhether they would implement the

summer program for a second year.The final decision rested on whetheror not the Board of Education plan-ned to have any schools open forsummer school. We do not know if asummer program was implemented in

the summer of 1982.

In-service training. The forego-ing list of c*ponents planned asdirect services to students is indi-cative of the variety or skills theon-site implementers must have hadin order to successfully serve stu-dents as planned. Because teachersselected to implement the projecthad, for the most part, been class-room teachers, project managers feltthat many hours of in-service train-ing would be required. Project man-agers began the training by acq-uainting the teaching staff with theproject goals and planned interven-tions prior to student selection forparticipation in the first programyear. They also thought in-servicetraining should occur periodicallythroughout the implementation periodso teachers could acquire skills andinformation needed to (a) implementthe career development curriculum,(b) improve student reading skills,(c). use computers as an instruc-tional aid, (c) counsel students,

-208-

and (d) determine individual stu-dents' optimal learning styles.Teachers were also to learn aboutfamily services available throughthe Board of Education and, othercommunity agencies.

For the most part, project manag-ers did not make specific plans forspecific types of in-service train-ing after the initial pre-servicetraining. The management style wasone of informally surveying teachersto find out what areat of respon-siblity they thought they could usesome additional training in thentrying to identify resources fordelivering the training. Often themanagers learned of particular work-shops that were planned elsewhereand spontaneously arranged forrelease time This lack of specif-icity became clear when project managers asked Polaris, the technicalassistance contractor for the OJJDPinitiative, to come and train teach-ers in effective peer counselingmethods. The impression of the TAcontractors was that the projectmanagers were unclear about how theproposed training in peer counselingfit into their project plans. Alt-hough peer counseling was initiallyplanned as a component of the pro-ject, it had, by that time, beenabandoned for numerous reasons atall sites. The project managers'decision to seek training in peercounseling when they did was somew-hat surprising.

Intervention analysis meetings.By January of 1982, project managersadded a new component for theirteachers called Intervention Analy-sis Meetings. This was planned tobe a means of helping teachers to,

solve problems through the presenta-tion of case studies at staff meet-ings. Other members of the staffwere expected to analyze the casebased on the presented materials andsuggest possible interventions. We

have boon unable to determinewhether this component was imple-mented.

sFYs'1jg1Lcti.vities. At theAugust 1982 planning conference,project managers developed extensiveplans for carrying out each task oftheir evaluation. They had beenunable to gain approval in the firstyear to administer the SAES questi-onnaire containing the self-reporteddelinquency items. The ChicagoBoard of Educations censoring theseitems from the questionnaireresulted in no assessment of theprimary result sought by the spontorof this delinquency prevention pro-ject because the Board also blockedattempts to acquire official delin-quency data from police or courtsources. Managers reported thatfailure in this area was primarilydue to a fear on the part of admin-istrators and principals thatparents would object or cause prob-lems for principals and the schoolsystem if they knew the project waspart of a delinquency preventioninitiative, or knew about the sensi-tive nature of some of the questionsbeing asked of their children.

In the second year, project man-agers set a timeline for contactinghigh-level administrators to gainsupport for the questionnaire, buttheir plans did not include any

strategies for helping the adminis-tration cope with irate parentsshould this unlikely event occur.

As it turned put, the matter becamemoot when the questionnaire wasbrought to the attention of the Gen-eral Superintendent after her meet-ing with OJJDP officials. We wroteto the Superintendent requestingapproval of the standard studentquestionnaire for administration inChicago. The Superintendent reluc-tantly approved the questionnaire.

Overall school response rates on

-209-

RETAIN

the questionnaires were low in thefirst year's administration. Manag-ers indicated that principals saw

the questionnaire as a disruption totheir school and as having little orno value. Managers therefore madeplans for helping principals by

developing special reports on someof the questions and reviewing theresults they thought principalswould find to be useful information.These tasks were executed, but it

had little effect on the responserates in the second year.

Another plaguing problem was thelikely breakdown of the experimentaldesign at some schools. Post -ran-

'domization checks showed that ran-domization had failed in severalschools in the project's first year.Managers decided to resolve thisproblem by using a random numbertable to make the selections at theRETAIN central office. This provedto be an unsatisfactory-solution-tothe problem when evidence of anotherbreakdown in randomization began toaccrue.

Because the project also hadschool-level goals, plans were madefor reporting school level data tothe national evaluation team. Theproject identified several controlschools to use in the comparativeanalysis, and decided to administerthe questionnaire in one of theseschools. These plans were not fullyexecuted. No school-level data werereported for any of the schools.

Problems also surrounded indivi-dual-level data collection activi-ties. During the first reportingperiod of the project's second yearstaff at t1 schools who collectedand reported data on the treatmentand control groups misunderstoodsome printed instructions. At theend of the first reporting periodthis problem was discovered, and by

82-4,.4

RETAIN

the last data reporting period codeswere being properly used and allinstructions were followed.

Other components. Managers gaveoral reports about special activities which some of the RETAIN teachers implemented at their schools.At Blaine Elementary School, theRETAIN participants wrote and performed a Play for their school. TheNorthside cluster took a field tripto Evanston for the experience ofattending a professional performance.

Effectiveness

Evolution of Evaluation Design

First Year. The project managersestablished procedures for randomselection of students at eachschool. More than 15 students werereferred at each school; thus selections had to be made in some fashion. Randomization was agreed uponbecause it seemed administrativelysimplest. Principals, counselors,and RETAIN teachers then made the

selections by drawing names of students referred to RETAIN out of

ListsL,ists of RETAIN participants,control students, and waiting students were developed. The projectreported that there were only a fewstudents in the nine schools whowere placed in the project by any

method other than drawing names fromhats.

This method of randomization didnot produce equivalent groups of

students. Post randomization checksrevealed significant differences inachievement (standardized testscores and grades) for the

1. Alas, they were told to "drawnames from a hat" by the evaluationfield worker initially assigned to

work with the project.

210

comparison groups at two schools.In one of these schools, the controlgroup contained a high proportion ofspecial education students. Randomization may also have broken down ina third school. A significant difference in the number of siblingsreported by treatment and controlstudents was found in that school.As a result, project managers committed themselves to conduct therandomization process and use a random numbers table for the selectionsin the second year, and the placement of any student in the projectby any other means was to be documented.

Second year. Although we urgedthat randomization be conducted atJohns Hopkins, project managersdemurred. Several indications implythat the experimental design alsobroke down in the second projectyear. Some students who had participated in the program previouslywere reenrolled in the program.And, project managers used randomnumber tables to select new participants for the second year before-obtaining parental permission for

participation in RETAIN and theevaluation. Many students selectedfor the control group were thenexcluded from the study when parental permission could not beobtained. We experienced considerable difficulty in determiningexactly what happened to the design.

When we discovered that implementation had been delayed in one ofthe project's schools (Sheridan), weagain requested to conduct randomization at Johns Hopkins. Thisrequest generated some ill will, butwas acceded to. A onesemester trueexperiment was definitely conductedat Sheridan, although project implementation was weak in this school.

Both years. As with all projectsin the Alternative Education Program, project staff were expected to

83

take all necessary steps to secureinformation pertinent to the evalua-tion from school and other archivalrecords, and to administer theSchool Action Effectiveness Studystudent questionnaire each spring.Archival data were to be collectedfor RETAIN participants and controlgroup youths, and questionnaireswere to be administered to theseyouths and to probability samples ofstudents from participating schools.Survey administration was to beguided by administration manualssupplied to the project.

Results First Program Year

The clear evidence of a breakdownin randomization in two schools, andthe possibility that it failed in athird, made us wary of assuming thatrandomization was implemented in theremaining schools. Because we didnot supervise the randomizatioa, wehave no direct knowledge of theprocedures used. Despite these wor-ries, some significant differencesbetween RETAIN and control studentswere found and merit discussion.Interpretation should be tempered byawareness that (a) randomization mayhave broken down in several addi-tional schools, if not in all ofthem, and (b) interventions wereimplemented in limited forms, and foronly a short period of time.

No statistically significant dif-ferences were found on any cognitivevariables, after the two schoolswhere evidence of non-random assign-ment involved cognitive variableswere excluded from analyses. Mea-sures of delinquent behavior werenot available for analysis: Nearlyall self-reported delinquency itemswere censored from the questionnaireby the Chicago Board of Education,and the Board refused to seek offi-cial delinquency data.

-211-

RETAIN

In one project high school

(Bowen) RETAIN students were absentsignificantly more often than con-trol students (6.11 days vs. 2.67idaa).on average, respectively,

In Blaine ElementarySchool, Interpersonal Competencyscores were higher for RETAIN thanfor control students (4.07 vs. 3.21,p<.05).2 (This is the school in

which RETAIN and control studentsdiffered in number of siblings.)Several differences between RETAINand control students were found inBontemps Elementary School. RETAINstudents were significantly lower inInterpersonal Competency than thecontrols (2.96 vs. 4.30, p<.05),higher in Rebellious Autonomy (2.21vs. 1.37, p<.05), higher in self-re-ported suspensions (.76 vs. .28 perstudent in the past year, p<.05).Finally, in Thorp Elementary, RETAINstudents scored higher than controlstudents in Attachment to School(4.19 vs. -3.69, p<.01).-

These interim results were inter-preted with caution for several rea-sons. First, we were uncertain thatthe RETAIN and control groups weretruly randomly equivalent. Second,the available data were dredged to

detect significant differences, andwhen many significance tests areconducted a few may be "significant"by chance. Third, the projectimplemented its interventions in

weak form. Although details weresketchy, ILP's were apparently notregularly updated, the parent

2. Descriptions of the measuresused in the School Action Effective-ness study during year one may be

found in the first interim .report(Gottfredson, 1982) and descriptionsof the slightly modified measuresused during year two may be found inthe overview section of the secondinterim report (Gottfredson, Gott-fredson, & Cook, 1983).

84

RETAIN

component did not go as expected,and the intervention was brief.Questionnaires were administeredonly 8 to 10 weeks after implementation of the project began. Theseresults apply only to the interventions, as implemented, not to the

interventions intended.

Second Year Results;

During the 1981 82 school year,163 students in Project RETAIN'Snine target schools were identifiedas being control students, and 145were identified as treatment students. Table 1 shows the number oftreatment and control students in

each school. Usable pretreatmentdata froze records for the 1980-81school year were provided on students' gender, age, fourth quarterEnglish grades, Iowa Test of BasicSkills (ITBS) scores in reading andmath (for elementary school studentsonly), number of high school creditsearned (for high school studentsonly), and suspensions. An unfortunate amount of data were missing--ranging from. 21% missing forEnglish grades to 35% missing forsuspension data. As Table 2 shows,there was considerable variationacross schools on the degree of

missing data.

Outcome data from records for the1981-82 schools year were providedfor attendance, suspensions, English, math, science and social studies grades, and withdrawal fromschool. The amount of missing or

incomplete data ranged from 0% forwithdrawals to 60% for attendance.As was the case for pretreatmentdata, there was considerable variation across schools (see Table 3).

Completed or partially completedStudent Questionnaires (administeredSpring, 1982) were returned for 65%of the treatment students and 52% ofthe control students. Survey

212

administration was poorly implemented in several schools. Proportions of questionnaire data missingfor each of the project's schoolsare shown in Table 4. The questionnaire, although a posttreatmentmeasurement instrument, containsmeasures on several measures, such

as age, sex, race and parental education that are considered to be

background measures or measures ofstable personal characteristics thatcould probably not be affected by atreatment. They are thus useful in

assessing the pretreatment equivalence of treatment and comparisongroups.

Analysis proceeded in severalsteps. First, treatment and controlstudents were compared on availablepretreatment and Student Questionnaire background "variables. Thesecomparisons were performed at the

project level (i.e. all projecttreatment students vs. all projectcontrols) and were attempted at theschool level (i.e., treatment vs.

controls in each of the targetschools). Comparisons at the targetschool level are largely uninterpretable due to small N's and manymissing data.

-If the project had substantialeffects, this should show up in theprojectlevel comparisons, where,despite the unknown bias due to

missing data, the N's are largeenough to allow a reasonable chanceof detecting statistically significant differences if they exist.

Comparisons of treatment and controlgroups were made for outcome measures from school records as well asoutcomes measured by the StudentQuestionnaire. For outcomes for

which the difference between treatment and control students was significant, subsequent analyses examinedthe data to determine whether treatment by school interactions were

present. A significant interaction

would suggest that the project wasmore effective in one school or secof schools than in others.

Because the randomization processhad broken down, for treatment-con-trol group comparisons in which ini-tial analyses found significant dif-ferences, statistical controls wereapplied in a stepwise regressionanalysis to "control" for pre-exist-ing differences between groups.

Post randomization checks. Whenall project treatment students werecompared to all control students onavailable pre-treatment and back-

ground data, no significant differ-ences were found (see Table 5).

Furthermore, differences between thetwo groups were small in comparisonto the individual differences withingroups (compare the differences bet-ween means in Table 5 with the stan-dard deviations, which are indica-tors of the extent of individualdifferences). Comparisons on pre-treatment and background variableswere also performed school-by-school. This time, several signifi-cant and near significant differ-

ences appeared, despite the small

N's (see Table 6). Because randomi-zation was to occur at the schoollevel, this set of results impliesthat the evaluation must be regardedas involving a non-equivalent con-trol group design, because randomi-zation was not effective.

Outcomes for treatment and com-parison groups. Post-interventioncomparisons between treatment and

control group students are summar-ized in Tables 7 and 8. These aresimple raw comparisons, with no sta-tistical controls. Three outcomes

favored the treatment group. These

were: dropout (p<.01), practicalknowledge (p<.05), and self-reportedemployment (p<.01). One result

favored the control group--parentalemphasis on education. When statis-

-213-

RETAIN

tical controls were applied in a

regression model,3 no significantdifferences between groups remained,except for seli'-reported employmentand dropout in school 1240 only. It

turns out thw. the difference bet-ween treatment and control studentson dropout rate in school 1240 is

due to a very high rate among con-trol students.4 These results may beinterpreted as suggesting that the

observed differences between RETAINand comparison youths can plausiblybe attributed to pre-existing dif-ferences; they may not be attributedto the program's influence with anyconfidence.

Discussion

The present results are subjectto several important limitations.The most important limitation is theoverall poor quality of the data.

Missing and incomplete data, and thevariations across schools in qualitymay introduce bias of an unknown

3. A control variable is a variablethat is correlated with the outcomevariable of interest which is some-times introduced into an analysis toadjust for pre-existing differencesbetween groups when the groups arenot randomly equivalent. The fol-lowing variables were controls forthe regression of dropout rate on

treatment: age, suspensions in

1980-81, and absences in 1980-81.

For the regression parental emphasison education on treatment, the fol-lowing variables were utilized as

controls: parental education, dummyfor school 4440, and the dummy forrace (1=white, 0=other). For the

regression of practical knowledge ontreatment, these variables were usedas controls: sex, dummy for race

(1=white, 0=others) and the dummy

for school 1240. The control varia-bles used for the regression of

self-reported employment were : sex

RETAIN

nature into statistical analyses.It also weakens analyses by decreas-ing statistical power, i.e.,

decreasing the chance that differ-ences that may exist will be

detected by a statistical test forsignificance. A second limitationis the non-equivalence of groups

prior to treatment (See Table 5).Groups were to have been randomlyassigned to treatment and control

conditions, but this was either notaccomplished or was unsuccessful.Finally, we judge the implementationas not particularly a strong one,and therefore statistical resultsshould probably be interpreted 'as

'representing the results for a

rather weak set of interventions.

Results of First Program Year

Despite the limitations, the

implications of the first year

results for the project were sev-

eral. Earlier (St. John, 1982) welisted several suggestions:(a) Steps should have been taken toassure that randomization was imple-mented. (b) The project should haveexamined activities at Bowen to det-ermine whether any aspects of RETAINactivities contributed to stude,v-

non-attendance. (c) The project

should have examined its activit.at Blaine to learn whether projectactivities that contributed to

interpersonal competency could be

identified and put into practice

and the dummies for schools 1240,

6180 and 5090.4. For control students in school

1240, the sample mean for dropoutrate is .55 with an SD of .51

(N=18). For the treatment group thesample mean is .13 (SD=.35, N=15).See Table 7 for the sample means andstandard deviations for the entiresample (i.e. all treatment vs. all

control students).

elsewhere. (d) The activities at

Bontemps should be have been care-fully scrutinized to s:L., if changescould have been made tc -,void whatappeared to be negative ,*sects oninterpersonal competency, lllious

autonomy, and possibly sui:p,sions.(e) The activities at Thori, ,bouldhave been examined to learn w;.,?.77project activities that contrit,...i.ito student attachment to sclloo

could hay.i. been identified and pvtinto practi . elsewhere.

Possibly, of course, thcstl

results were cttributable to

breakdown of r:ldmization. Ther-,fore, we sugge,t--- ,2 that the proj...ct

carefully review -bow students vereselected for participation in

RETAIN, and consider whether .theseresults may Lave been due to someartifact of the selection procedure.Although project managers made someattempt to improve their evaluation,there is little evidence that any ofthese suggestions were followed upwith much zeal.

Second Program Year Results

Taken Logether, the results forthe second year provide little evi-dence that RETAIN accom:Lishedeither its own goals or the resultssol,ht by OJJDP. This outcome maybe the result of (a) a weak evalua-tion, or (b) a weak program.

PDE and The Proiect

Project RETAIN managers did notfully participate in a ProgramDevelopment Evaluation of th,s kind

we had envisioned would be imple-

mented. They were coopera-ive in

the summative aspects of evaluati......,

electing to attempt implementationof a true experimental design and toreport school record data on indivi-duals in a timely and frequent fas%-ion. They did not, however, furnis:

-214-

87

systematic implementation data to usas requested. We do not fullyunderstand the reasons we were unable to obtain implementation data,the reasons for the extensive missing data problems, or the reasonsfor the failure of randomization.5

Goals. Project managers participating in the first PDE planningworkshop were clear about the problems the project was designed to

address. Therefore they were clearabout the goals of the project andthe target population that wasappropriate for the direct servicescomponent.

Theory and objectives. The project's rationale pointed to manysources of the problems RETAIN wasintended to address, and some of theimplied corrective activities wereincorporated into planned activities. For the most part, the plansdeveloped failed to clearly specifythe intervening outcomes that wouldhave to be achieved. We sensed somereluctance on the part of projectmanagers to make clear statements inthis area. For example, althoughmanagers stated that one cause of

the problems that RETAIN was to

address was ravaged selfesteem,they would not state this obvious

5. It may be of some significancethat, after extensive discussion ofthe advantages and disadvantages ofalternative evaluation designs at

the January 1981 workshop where theexperimental' approach was decidedupon, the reason project managerschose to implement randomizationrather than alternatives (e.g.

matched control groups, regressiondiscontinuity, time series) was thatthey judged it to be administratively simpler to accomplish. In

actual practice, maintaining the

randomized treatmentcontrol groupdesign was not simple.

215

RETAIN

outcome as an objective of theirinterventions because they said theydid not want to be held accountablefor raising selfesteem, a relatively stable characteristic that isdifficult to change.

Project managers appeared sometimes to use the word "objective,"in ways other than that intended inthe PDE method. In other planningmodels, "objective" is often usedsynonomously with a process or

activity. PDE requires abandoningthat .usage and learning a moreresearch oriented meaning of objective. Activities or processes arecalled "interventions" in the language of PDE.

The Match between Theory and Interventions

The project's theory describedthe contributi,us or students,

parents, community, and school to

the problems that students experience. It follows, therefore, that

objectives and interventions shouldbe aimed at system chwles, with atleast as much vigor as the individual focus. Although plans initially included several intermediateoutcomes and project activiti,:s thatwere related to chances in the system, nearly all were dropped. Onlyat Thorp was there an experimentwith an inschool suspension projectinitiated by the principal. Exceptinsofar es the use of ILP's and theattendance methodology were viewedas a pilot or demonstration of thesetechniques, RETAIN may then be characterized as a project aimed at

assisting individuals in ,arsisting

despite the system. If the project's theory was correct, RETAINwas probably not able to achievedramatic effects when no systemchanges were made. The focus on 15individuals may not be a strongenough interve,.:tion to affect a

whole school--even a small one.

88

RETAIN

Schoollevel. From the outsetproject staff stated schoollevelgoals of increasing attendance andreducing disciplinary incidents.Although the schoollevel interventions were dropped, the goals wereclung to as if the project's successwere riding on having reached thesegoals.

The project managers explainedthis was because Project RETAIN wasitself an experiment with new. policies and procedures. They-ft argued

that implementation of the' projectwas therefore evidence that thegoals were met. As for r1 creating

broad changes in school policies orpractices, the staff reported thatthis would have taken years of

effort, involving every level of theBoard in planning and approving newpolicies. Furthermore, they werenot policy makers, but projectimplementers responsible for a project serving 15 students at eachschool. They did indicate that theproject was being carefully watchedby administrators and that if the

alternative policies used in theproject appeared to be successful,policy makers would consider adoption in the system.

Conclusion

Project RETAIN suffered from management difficulties. The CBE is

large enough that these problemsapparently were undetected (just asthe Project itself operated withoutthe knowledge of the General Superintendent until her communicationswith OJJDP) despite each projectmodification being cleared by a DFS,an administrator placed rather highin the system. Although the managers had all worked witht this DFS, aformal hierarchy in decision makingappeared to be the rule, perhapsaccounting for the minimal awarenessof Project RETAIN at higher administrative levels. The assignment of a

216

Government Funded Programs monitorin the second year did not help toresolve the management difficulties.

Teachers in the schools triedvaliantly to implement the projectto the best of their ability, but

lacked guidance and training in themany skills needed. Students at allsites seemed to be attached to theRETAIN teachers and grateful fortheir help, but management practicesresulted in nine different programsevolving (one in each of the

schools) rather than the developmentof a single cohesive program.

The large Chicago Board of Education bureaucracy is a highly political organization. Managers weresensitive to the politics of the

system. They appeared less sensitive to the demands of developing astrong set of interventions, and haddifficulty in working with the project's complex forcefield to developthe project. The very nature of theproject meant that the system had tobe modified at several levels to

accommodate it. Project activitiesand organization required that newlines of administration be developedto assist in the direct contact between RETAIN teachers and managers.Without these arrangements, the

question of who was in charge--principal or project managerswas leftwide open, and the project took on adifferent character at each school.The project dropped entire components at some schools, and addedadditional activities at others, in

large part in accommodation to theprincipals of the several schools.New policies were needed for the

implementation of many plannedactivities such as campus releasetime for home visits.

It may have been helpful if project managers had been empowered towork with principals to plan andimplement experimental policies andprocedures that would have affectedtheir schools. Managers did notperceive they had this power.

SY4

RETAIN

References

Chicago Board of Education. Project RETAIN (A proposal to the ()ffice forJuvenile Justice and Delinquency Prevention). Chicago: Author, 1980.

Cloward, R. A., & Ohlin, L. W. Delinquency and opportunity: A theory of delinquent gangs. New York: Free Press, 1960.

Cohen, A. K. Delinquent boys: The culture of the gang. Glencoe, Ill.: FreePress, 1955.

Gottfredson, G. D. A theoryridden approach to program evaluation: A methodfor stimulating_ researcherimplementer collaboration (Report No. 330).Baltimore: The Johns Hopkins University, CSOS, 1982.

Gottfredson, G. D. (ed.). The School Action Effectiveness Study: Firstinterim report (Report No. 325). Baltimore: The Johns Hopkins University,CSOS, 1982.

Gottfredson, G. D., Gottfredson, D. C., & Cook, M. S. (eds.), The School.Action Effectivness Study: Second interim report. Baltimore: The JohnsHopkins University, CSOS, 1983.

Office for Juvenile Justice and Delinquency Prevention. Program announcement:Delinquency prevention through alternative education. Washington, D.C.:Author, 1980.

St. John, J. Project RETAIN: First interim report. In G. D. Gottfredson(ed.), The School Action Effectiveness Study: First interim report. Baltimore: The Johns Hopkins University, CSOS, 1982.

217-

Figure 1: Aspects of Chicago Board of EducationOrganizational Chart

IIState Legislature

Chicago Finance Authority(Budget Authority)

1Chicago Board of Education

GeneralSuperintendentof Schools

1

Superintendentfor Finance

j-------T------'4"."-I-61Deputy Superintendent Deputy Superintendent for

for Instructional Serviced Maintenance Services-,

Deputy Superintendent:for Field Services

Director,PupilServiceCenterSouth

Director,PupilServiceCenterCentral

Director,PupilService

'CenterNorth

GovernmentFundedPrograms

Curriculum

DistrictSuperintendentDistrict19

f 3 Principals

DistrictSuperintendentDistrict12

DistrictSuperinten-dentDistrict3

Pupil PersonnelServices andSpecialEducation

3 Principals

3 RETAINTeachers

3 Principals

3 RETAINTeachers

Personnel.]

Table 1

Numbers of Treatment and Non-EquivalentControl Students, 1981-82 School Year,in Each Project School--Project RETAIN

Treatment Status

School TreatmentNon-Equivalent

Control

la

1240 15 181340 14 15

1430 18 25

2300 16 20

4440 15 16

5090 18 17

5750 18 20

5880 16 17

6180 15 15

-219-

92

Table 2

Average Proportion of Missing Pre-Treatment Datafrom 1980-81 School Records in Each ProjectSchool, 1981-82 School Year, Project RETAIN

Treatment Status

School Treatment Control. Overall

1240 .33 .14 .23

1340 .30 .33 .32

1430 .27 .72 .53

2300 .08 .02 .05

4440 .05 .20 .13

5090 .17 .26 .21

5750 .06 .12 .09

5880 .58 .51 .54

6180 .02 .05 .03

Project as .24

a whole

Table 3

Average Proportion of Missing Post-Treatment Datafrom School Records in Each Project

School, 1981-82 School Year, Project RETAIN

Treatment Status

School Treatment Control Overall

1240 .32 .67 .51

1340 .65 .63 .64

1430 .60 .82 .73

2300 .16 .22 .20

4440 .11 ,, .36 .24

5090 .24 .10 .18

5750 .33 .39 .36

5880 .88 .88 .88

6180 .10 -.19 .14

Project as .44

a whole

-221:94

Table 4Average Proportion of Missing Student Questionnaire

Outcome Data for Each ProjectSchool, 1981-82 School Year, Project RETAIN

Treatment Status

School Treatment Control Overall

1240 .63 .79 .72

1340 .93 1.00 .97

1430 .35 .77 .59

2300 .14 ,, .24 .20

4440 .21 .48 .35

5090 .19 .16 .18

5750 .44 .38 .40

5880 .62 .46 .53

6180 .14 .36 .25

Project as .46

a whole

222

Table 5

Comparison of Background Characteristicsof Project RETAIN Treatment and Control Students

(Entire Sample) During 1981-82 School Year

Variable

Group

Non-equivalentTreatment Controls

M SD N M SD N

Sex (coded 1-2)

4th Quarter EnglishGrade--1980-81

1.36 .48 107 1.37 .49 105

.58 .73 117 .59 .68 130

1980-81 ITBS Reading Scorea 6.33 6.90 96 5.41 5.75 103

1980-81 ITBS Math Scorea 6.46 6.56 95 5.36 6.68 102

Credits Egrned Through 3.68 3.27 28 2.54 2.82 48

June 1981

Total Days Absent During 21.83 18.20 101 23.85 22.97 110

1980-81 School Year

Total Days Lost Due toSuspension: 1980-81

Age (from 1982Student Survey)

Grade Level (from 1982Student Survey)

Parental Education (from1982 Student Survey)

.18 .52 102 .27 .74 97

13.44 1.66 95 13.14 1.70 86

1.78 1.45 95 1.54 1.33 83

2.08 1.20 72 2.16 1.15 58

aVariable is available only for students who were in elementary school

bduring the 1980-81 school year.Variable is available only for students who were in high school duringthe 1980-81 school year.

-223-

96

Table 6

Background Differencesof Project RETAIN Treatment and Non-Equivalent Control Students

During 1981-82 School Year, by School

Variable

Treatment

SD

Group

Non-equivalentControls

N M SD N

School 1240

Prob.

Total Days Lost Dud to .25 .46 8 .93 1.07 14 .05Suspension: 1980-81

School 1340

Total Days Absent During 39.15 19.18 12 55.04 20.64 13 .061980-81 School Year

School 1430 _

1980-81 ITBS Math Score 12.08 13.23 7 .63 1.51 12 .06

Credits Earned Through 5.39 3.86 11 1.58 3.05 18 .01June 1981

School 5090

Age (from 1982 13.31 1.14 16 12.33 1.84 15 .09

Student Survey)

Grade Level (from 1982 1.53 .64 15 .78 .80 14 .01

Student Survey)

School 5750

Total Days Absent. During 16.28 18.73 16 6.42 10.62 18 .081980-81 School Year

School 5880

Age (from 1982 12.28 1.11 7 13.30 1.06 10 .08Student Survey)

Note. These comparisons suffer from problems of missing data.

-224-

Table 7

Comparison of 1981-82 School Jeer Outcome Variables fromSchool Records for RETAIN and Control Students

Outcome Variable

Group

Non-equivalentTreatment Controls

M SD N M SD N

Ratio of DaysAbsent to DaysEnrolled: 1981-82

Ratio of DaysExcused Absence to DaysEnrolled: 1981.-82a

Ratio of DaysUnexcused Absence to DaysEnrolled: 1981-82

Ratio of Days LostDue to AdministrativeRemoval to DaysEnrolled: 1981-82

. 10 .14 101 .10 .13 80

.03 .04 68 .03 .04 65

.03 .05 61 .02 .03 62

. 00 .01 78 .01 .02 73

Ratio of Times Suspended .00 .01 83 .00 .01 59

to Days Enrolled:1981-82

Average English Grad .89 .66 95 .86 .69 84

(1-4 scale): 1981-82°

Average Math Grade .88 .67 87 .94 .60 77

(1-4 scale): 1981-82°

Average Science Grads .96 .70 64 .91 .60 64

(1-4 scale): 1981-82

Average Social Studies .93 .64 70 .89 .57 67

Grade g-4 scale):1981-82°

Dropout . 15** .36 145 .35 .48 163

Note. The statistically significant outcome for dropout favoring thetreatment group dissappears when appropriate statistical controls areapplied.

aVariable is applicable only for project elementary schools.

bAverage of the four quarterly grades in the particular course.

** I) < 01

-225-

98

Table 8

Comparisons of 1981-82 School Year Outcome Variables fromStudent Survey for RETAIN Treatment and Control Students

(Entire Sample)

Variable

Treatment

M SD N

Group

Non-equivalentControls

M SD N

Parental Emphasis on Education .53* .31 87 .62 .26 78Attachment to Parents .54 .26 92 .58 .28 85

Negative Peer Influence .31 .23 88 .31 .21 86

Attachment to School .57 .27 87 .54 .30 81

Belief Scale Scored Positively .57 .24 83 .55 .28 74Interpersonal Competency .74 .23 84 .77 .23 74Positive Self-Concept .66 .17 75 .65 .19 59Self-Reported Delinquency .18 .23 81 .19 .25 70

ScaleSelf-Reported Drug Use .28 .36 83 .21 .33 72Serious Delinquency .13 .21 81 .15 .25 71

Punishment Index .25 .29 86 .25 .25 77Rewards Index .28 .31 85 .27 .27 78Victimization .19 .25 86 .17 .21 78School Effort .49 .33 95 .47 .31 83Practical Knowledge 1.46* .35 83 1.33 .45 75Internal-External Control .55 .22 83 .53 .23 75Alienation .42 .26 81 .47 .22 75Self-Reported Grades 1.91 1.08 93 1.93 1.04 84Reading Ability, ..,elf Rating 1.11 .90 89 1.14 .81 77

Days of School Cut in4 Weeks Preceding Survey .56 1.09 95 .38 .72 85

How Often Cut One orMore Classes .98 1.41 95 .74 1.23 86

Educational Expectation 2.89 1.85 95 2.93 1.90 85

Did You Work for Pay LastWeek? .65** .48 95 .36 .48 86

Regular Part-Time orFull-Time Job? .37 .58 95 .28 .57 86

Suspended from School DuringTerm .43 .50 86 .33 .47 78

School Nonattendance Index .72 .84 94 .62 .77 85

Rebellious Autonomy .68 .33 83 .63 .30 73Involvement .27 .21 87 .23 .20 81

Note. Two of the statistically,significant differences in these rawcomparisons (i.e. Parental Emphasis on Education, Practical Knowledge)are not significant when appropriate statistical controls are applied.

* p<.05

** < 01P

-226-

The Milwood " 'native Project: Second Interim Report

M. S. Cook

Abstract

The Milwood Project is one of several projects fundedby the Office of Juvenil, Justice and Delinquency Prevention (OJJDP)as part its Program in Delinquency Prevention through AlthernativeEducation. The Milwood Project operates in Milwood Junior HighSchool in Kalamazoo, Michigan. Project components during 1981-82included an in-school suspension room, a home- school liaison, anattendance clerk, a Sk lls Lab class for low-achieving students, aschool-within-a-school for eighth grade students (the Milwood Alter-native Program), a student council, a project advisory, and anintramural/activities program.

School climate comparisons with South Junior High School (thecontrol junior high) indicate that between the 1981 and 1982 schoolyears school climate improved slightly at Milwood, and declined sub-stantially at South. Milwood is maintaining a positive school cli-mate in the face of financial cutbacks in Kalamazoo programs. Basicskills scores improved significantly at Milwood, while remainingstable at South. We cannot conclusively attribute these positiveeffects to the Milwood Alternative Project.

Chronic non-attendance--skipping several consecutive days of

school--is also down at Milwood as compared to South. The in-schoolsuspension room reduced the number of exit suspensions, but does notappear to have affected the overall number of suspensions. We areunable as yet, to judge the effectiveness of the other projectinterventions. For 1982-83 an Intensive Study Program has replacedthe Skills Lab, and a reading diagnosis and tutoring program hasbeen added.

Major Program Changes in 1981-82

The Milwood Alternative Projectcontinued implementing the majorprogram interventions described in

the first interim report (Carltonand Cook, 1982). Briefly, theseinterventions included (a) the Mil-wood Alternative Program, a school-within-a-school for sixty eighth

This report covers roughly the

period of September 1981 to June1982.

227

grade students, (b) the Skills Lab,a basic skills class, (c) a studentcouncil, (d) a student project advi-sory--a student board with inputinto project activities, (e) a pro-ject advisory utilizing communitymembers, (f) an attendance monitorand home-school liaison, and (g) anin-school suspension center. Theonly modification of note was in theteaching staff for the Skills Labclass. Because of teacher layoffsand teacher union issues related toseniority, the original teacher forthe Skills Lab (around whom the

class was designed), was replaced

i00

Milwood Project

first with substitute teachers, thenwith a school counselor appointed bycentral administration who had nevertaught in a classroom. Finally, inJanuary the position was filled by ateacher selected by Milwood stafffrom a pool available for seniority-based layoff recall. This persondid not have specific expertise inworking with low-skills children,and was not involved in program orproject planning.

Major Forcefield Changes in 1981-82

Four major forcefield changesoccurred between the 80-81 and 81 -82school year: a change in the Mil-wood assistant principal; the com-mitment of the Milwood principal andnew assistant principal to projectsuccess; the development of strongsupport for the project within thecommunity; and budget cuts forcingteacher layoffs.

Partly due to project efforts,the 80-81 assistant principal, whowas responsible for recommending thehigh number of out-of-school suspen-sions experienced at Milwood duringthe 80-81 school year, was replacedby an assistant principal who is

more accepting of alternative formsof discipline (i.e., referral toin-house rather than exit suspen-sion). Also, the building principalhas become an advocate for the pro-ject, both within the school and inthe community at large. Althoughnot involved in the early stages ofproject development, Milwood's prin-cipal has become actively involvedin project activities, and nowserves as an enthusiastic projectspokesperson before parent groups,central administration staff, andthe media.

Community support for the projectincreased sharply during the 81-82year. For example, The Citizens'Coalition on Discipline (a task

228

force set up to evaluate theKalamazoo Public Schools' disciplinepolicies) has praised the Milwoodproject's in-school disciplinarypolicies, and recommended that thepolicy be implemented system-wide.Favorable reviews of the projectappeared in the local newspaper, onthe T.V. news, and representives ofthe project were invited to discusstheir work on a local televisiontalk show. The Milwood project hasalso received favorable pressthrough a UPI news release.

A negative force acting on theproject was the financial situationof the state of Michigan and thecity of Kalamazoo. Although the

recession has not had as much of animpact on Kalamazoo as on the restof the state, budget cuts did resultin widespread teacher reductionsfrom 1981 to 1982. Besides gener-ally reducing personnel andresources froni the school (probablynegatively affecting school climate)the cuts had a particularly negativeeffect on the project's Skills Labclass.

The teacher who had been instru-mental in designing the class, andwho was to teach it was bumped downto the elementary school level dueto her lack of seniority. The posi-tion was covered by several diffe-rent teachers throughout the year,and was never filled by a personexperienced or trained in teaching abasic skills class.

In short, some major forcefieldchanges impinging upon the projectwere positive: Extensive supportfor the project has developed in theschool itself, in the school system,and in the community. One force-field change was negative: teacherlayoffs based on seniority ratherthan training or ability.

.TW1=. .MaIa

Evaluation Results

School Climate

Because the Milwood AlternativeEducation Project is primarily tar-geted at school-wide climate change,school level comparisons are of

major interest.

Milwood and South in 1980-81.The 1980-81 school year was predomi-nantly a planning year for the Mil-wood Alternative Project (see thefirst interim report). Most projectcomponents were not implementeduntil the 1982 year. Thus, the 1981questionnaire results may be con-strued as pre-treatment data. Thefindings in Table 1 support theclaims of project staff that Milwoodsuffered from a poorer school cli-mate than South at the initiation ofproject activities. There were sixsignificant differences on the 1981student psycho-social scales. Mil-wood students, as compared to Southstudents, had lower self-esteem andhigher rebellious autonomy. Milwoodstudents were also suspended moreoften, attended school less, andreported more delinquency and druguse.

It should be noted that Milwoodhad a lower percentage of minoritystudents than did South in 1981 (31%vs 45%, p = .005), and that race wascorrelated with several of the ques-tionnaire scales. Those that weresignificant at the .01 level or bet-ter may be found in Table 2. Beingwhite is associated with greaterbelief in the rules, but less prac-tical knowledge, and less involve-ment in conventional activities.Minorities are suspended more often,receive lower grades, and theirparents have completed fewer yearsof education. Whites report moredrug usage.

In order to control the effects

229

Milwood Project

of the differential racial composi-tion of the two schools, an analysisof covariance was performed.Because parental educational levelalso tended to differ between Mil-wood and South (see Table 1) paren-tal education was also introduced asa covariate. Table 3 presents thecomparisons that were significantafter controlling for race andparental education.

Striking differences remain bet-ween the two schools after correct-ing fpr the two demographic varia-bles, race and parental education.The adjusted means show that Milwoodstudents, as compared to South stu-dents, reported higher rebelliousautonomy in 1981. More importantly,perhaps, Milwood students were sus-pended much more often than Southstudents, reported more delinquencyand drug use, and skipped schoolmore often.

The foregoing analyses indicatethat the climate at Milwood was lessfavorable than that at Scuth at theclose of the 1981 school year. Mil-wood students were misbehaving more,getting into more trouble at school,and cutting school more often. Thequestion becomes what happened as

the project began implementation inearnest?

Milwood and South in 1981-82.

Table 4 documents the results fromanalyses of variance carried out onthe 1981-82 SAES questionnaire sca-les. Major differences can be notedbetween these results, and thosefound in Table 1. Table 4 indicatesthat by the end of 1982 Milwood stu-dents, as compared to South stu-dents, reported greater attachmentto school, found school less punish-ing, were less alienated, and werevictimized less. On the other hand,Milwood students said they were nowless involved in school activities

102

Milwood Project

than South students, and said thatthey were poorer readers. Milwoodstudents also reported greater prac-tical knowledge. ;,s contrasted with1981, there were no differences onthe SAES questionnaire drug use,

attendance, or suspension measures,and the trend was for Milwood stu-dents to report less delinquencyoverall, and less serious delin-quency. Student characteristic mea-sures show that Milwood studentsrated themselves as having lessreading ability than students fromSouth.

An analysis of covariance cont-rolling for race and sex producedessentially the same analyses, andwill not be reported here.

These results indicate that theclimate at Milwood had improvedrelative to South between the springof 1981 and the spring of 1982. Notonly did Milwood students like

school more than the South students,and report themselves to be lessalienated, but they no longer dif-fered from, the South students intheir self-reported attendance ordrug use. Importantly, in 1982,

Milwood students reported themselvesto have been victimized less thanSouth students and scored lower on ameasure of alienation. There arealso trends toward Milwood studentsreporting less delinquency overalland less serious delinquency.

But the positive changes at Mil-wood do not tell all of the story.Student involvement at Milwoodremains low (as compared to South),and in 1982, large differences in

self-reported reading ability exist,which appear to be the result of thejoint effects of improvement atSouth, and a slide in self-reportedreading ability at Milwood (see

Table 1). The 1982 differerce inreported reading ability should beinterpreted cautiously since analy-

230

sis of the Metropolitan AchievementTest of basic skills (MAT) indicatesthat Milwood students were indeedpoorer readers than South studentsin 1981, but this was not the casein 1982 (see details, below).

The finding of less involvementon the part of Milwood studentsimplies that the project has beenunsuccessful to date in signifi-cantly increasing Milwood students'involvement in school extra-curricu-lar activities. Both project staffand the school administration attri-bute great importance to extra-cur-ricular activities in the promotionof school attachment, thus the find-ing of low involvement in extra-cur-ricular activities is an importantoutcome for further program develop-ment efforts. Likewise, project

staff theorize that basic skills

proficiency is important for lifesuccess (and reducing the risk ofdelinquency),- so the -decieise inreported reading ability scoresfound at Milwood may also suggestprogrammatic change.

Climate over time. The next setof analyses was conducted to examinechanges in the SAES scores from 1981to 1982. The first set of analysesdetermined that in 1981, Milwood hada. less favorable school climate thanSouth. This set of analyses are anattempt to ascertain whether the

generally more favorable climate atMilwood in 1982 as compared to Southwas due to Milwood developing a morepositive school climate, South a

more negative, or some combinationof the two.

In the first attempt to investi-gate changes in school climate fromthe 1980-81 to the 1981-82 school

year, change scores were calculatedfor those students for whom therewere completed questionnaires for

both years. These, of course, wouldbe for the most part students that

103

were seventh graders in 1980-81 andeighth graders in 1981-82.

Table 5 gives the results forthis series of analyses of varianceon the available SAES scales (Inter-nal Control was not scored for1980-81). The only change scorethat differed significantly for Mil-wood and South was in self-reportedreading ability: Milwood showed adecrease of .13 in self-reportedreading ability, while South showedan increase of .34, i.e., Su ,h stu-dents reported themselves to be

somewhat better readers, while Mil-wood students reported themselves tobe slightly poorer readers. Thisresult is consistent with earlieranalyses.

There is also a trend toward Mil-wood students becoming more alien-ated over the 1981-82 year (p=.07).No other comparisons approached sig-nificance.

This analysis is difficult to

interpret due. to the relativelysmall numbers of individuals on

which we have data for both years;N's range from 52-84 for the changescores as compared to 191-676 for

the simple between-groups, 1982 com-parisons. The change scores do notinclude any student who had spentonly one year at either school, andlargely exclude seventh graders (andtherefore the samples differ in agealso).

The effect of the more select(and older) sample size can be seenin the alienation measure. Whilethe change scores indicate a bigjump at Milwood and a very smalldecrease at South, means for theentire yearly sample at each school(Tables 1 and 4) show that, overall,alienation was up somewhat at bothschools.

A further consideration is that

231

Milwood Project

the Milwood students included in thechange score sample had only experi-enced one year in which projectinterventions were fully opera-tional. Project staff hypothesizedthat the eighth grade class of1981-82 was particularly resistantto influence, largely because oftheir early (seventh grade) negativeexperiences in the school environ-ment; as a school Milwood was lessharmonious in 1980-81 than in1981-82. Students exposed to pro-ject activities in 1981-82 may havealready "opted out" of school.

Summary of school climateresults. The pattern of resultsfrom the two sets of data--the bet-ween school analyses, and thechanges from 1981 to 1982--suggestthree conclusions concerning schoolclimate at Milwood and South:

1. Milwood exhibited a slightimprovement in school climate from1981 to 1982. The school climate atSouth showed negative changes duringthis saws period.

Comparing Tables 1 and 4 it is

apparent that from 1981 to 1982 Mil-wood and South reversed their ove-rall standing on school climate.Whereas South had a better schoolclimate in 1981 than Milwood, thereverse is true in 1982. The mea-sures of attachment to school,alienation, rebellious autonomy,practical knowledge, self-reportedschool non-attendance, suspension,delinquency, drug use, and victimi-zation all support this interpreta-tion.

2. Measures of Involvement in

conventional activities and self-re-Ported reading ability favor Southover Milwood. These results arepuzzling. Involvement in school-re-lated activities was a major Milwoodproject focus in 1982. Students atMilwood apparently did not feel its

104

Milwood Project

impact, or not enough students wereinvolved in project activities latein the school year (at the time ofthe SAES questionnaire administration) to differentially affect the

Involvement measure. The selfreported reading ability measure is

highly suspect in these databecause, although Milwood's selfreported reading scores declined between 1981 and 1982, and South'simproved, test scores show preciselythe opposite pattern.

3. It is impossible to unequivocally attribute the pattern ofschool climate results (relativeimprovement at Milwood, and declineat South) to Milwood project activities.

It is reasonable to argue thatthe less favorable climate at Southin 1982 as compared to 1981 is a'

result of restricted resources andcuts in programs resulting from therecession. These cuts translatedmainly into staffing cuts, and cutsin extracurricular programs. Theslight positive climate changes witnessed at Milwood in the past yearare impressive because they havecome during a time of generallydeclining resources for education inthe area. In other words, thisargument assumes that without thealternative project efforts, theclimate at Milwood would have deteriorated as it did at South.

An alternative explanation wasoffered for the decline in schoolclimate at South by its principal.She feels that the negative climateeffects in 1982 were due to a particularly "bad" eighth grade class;that particular cohort at Southcaused the problems. This would notexplain why the effects did not showup until these students' eighth

grade year, since they had been atSouth during the previous schoolyear.

232

A third explanation for the

effects might lie in regressionartifacts. Milwood was chosen forproject activities because it wasjudged to have the poorest, schoolclimate of the three junior highschools in Kalamazoo. If Milwoodwas below the mean in school climateat the start of the project it wouldtend to move toward the average climate of the area schools, and theresults over time would be expectedto look much as they do. At thisjuncture, we can only say that Milwood is showing promising improvements in school climate, but we mustwait for the third year results formore definitive assessment of ove

rall school climate change.

Metropolitan Achievement Test Scores

A further way of examining organizational changes is by analyzingthe performance of the students ofthe two schools on the MetroPOlitanAchievement Test (MAT), a standardized basic skills test. The MATprovides scores on reading, use oflanguage, basic mathematics, and anoverall composite (total) score.

Table 6 presents the spring 1981raw and percentile scores for the

three subtests and the compositescore for Milwood and South. The

mean scores presented are adjustedfor grade, sex, race, and age. On

all three subtests and the compositescores, the South students performedsignificantly better than the Milwood students. The differences arenot only statistically significant,but are of a fairly substantial substantive size also. The compositepercentiles indicate that Milwoodstudents (after controlling forbackground differences) scored at alevel that was eight percentilepoints below that of South students.

The following table, Table 7,

gives the results for the analysesof the spring, 1982 MAT scores.

105

These results are dramaticallydifferent from those of Table 6.

Milwood students no longer differedfrom South students in Reading, Lan-guage, or on the overall compositescore, and actually showed signifi-cantly better performance on themathematics test. Tables 6 and 7

indicate that the 1982 effects arethe result of virtually no change inperformance at South between 1981and 1982, but of large increases atMilwood--eleven percentile points onthe composite score.

In order to examine the hypothe-sis that changes between the yearsare due to cohort differences, ananalysis of variance was conductedon both the 1981 and 1982 MAT scoresthat included school, grade, sex,

and race as factors, and controlledfor the age of the student. Thisanalysis indicated that, in general,whites perform better than blacksand other minorities, and girls per-form slightly better than boys.These effects were similar for bothschools, and for both years. Table8 gives the adjusted means for therace and sex main effeCts for theoverall composite score. The dif-ference between the overall gradelevels within a school are largelyuninterpretable because both theseventh and eighth graders take theseventh grade version of the MATs,and the eighth graders would beexpected to score more highly on thetests.

More importantly, the only con-sistent interaction effect was bet-ween school and grade in 1981. Asignificant interaction effect indi-cates that the results for one vari-able (in this case, the student'sgrade in school) depend upon anothervariable (here, the school in whichthe student was enrolled). Table 9shows the means for this interac-tion. These means, adjusted forsex, race, and age, show that the

7,-.

Milwood Project

poorer performance of Milwood versusSouth in 1981 was entirely due to

differences in performance of stu-dents in the two seventh grades.The two groups of 1981 eighth grad-ers scored very similarly on thetests.

Table 10 gives the same resultsfor the 1982 scores, and Table 11lists the average percentile changefrom 1981 to 1982 for the school bygrade groups. As Table 11 docu-ments, both the 1982 Milwood seventhand eighth grade cohorts scoredhigher than their 1981 counterparts,with the seventh grade cohort show-ing the biggest increase (and

accounting for the disappearance ofthe interaction effect found in the1981 data). The 1982 eighth gradestudents (who are essentially thesame students as the 1981 seventhgraders, and who do not constitute adifferent cohort) gain eight percen-tile points on the composite score.The overall change from 1981 to 1982on the composite index was +12 per-centile points for Milwood students,and +1 point for South students. As

seen in Table 11, the interaction ofschool by year (controlling for sex,race, and age) on the composite per-centile score is significant(p<.001).

To summarize the MAT data, Mil-wood students (particularly seventhgraders) performed much better, on

the 1982 than the 1981 MATs. Southstudents showed little change bet-ween the two years. As was the casewith the school climate findingsderived from the SAES questionnaire,we cannot unambiguously attributethe positive gains at Milwood to

project efforts. This is especiallytrue given that there was not a

great deal of attenti.on given to

basic skills education by the pro-ject during the 1981-82 year (see

the analysis of the Skills Lab,

below). Changes in administration

233

Milwood Project

(or other, unknown changes) mayaccount for Milwood's improvement intest scores. The third year resultswill be more definitive, but, atthis point, the basic skills testoutcomes are promising.

Specific Project Components

The next section of the reportwill discuss data pertinent to anevaluation of the individual treat-ment components of the MilwoodAlternative Education Project.

The Milwood Alternative Program.No a priori control group Wasdefined for the MAP. Because ofthis the rest of the eighth gradeclass was used as the nonequivalentcontrol group for the MAP. This isa conservative comparison. The MAPparticipants were invited to parti-cipate, largely on the basis of hav-ing exhibited behavioral and aca-demic problems, so one would expectthem to have poorer outcomes.

The results for comparisons bet-ween Milwood Alternative Program(MAP) students and the rest of theMilwood eighth grade (all MAP parti-cipants are eighth graders) areshown in Table 12. Table 12 givesthe results for analysis of variancecomparisons between MAP participantsand the rest of the eighth grade onthe 1982 questionnaire scales- Theresults are mixed. Three variablescentral to the efforts of the MAPintervention, School Rewards, Inter-nal Control, and Interpersonal Com-petency show significant differencesthat favor the MAP students. TheInvolvement in Conventional Activi-ties scale also shows positiveeffects for the MAP. On the otherhand, MAP students report lowerself-esteem and attachment to

parenti than the rest of the eighthgrade, and they report less parentalemphasis on education and greaternegative peer influence. The MAPstudents also admit to more drug

use, and to more serious delin-quency, and they are suspended more.There is also a tendency for them toreport greater overall delinquency,and to have lower grades. Finally,the NAP students, as compared to theeighth grade at large, have muchlower educational expectations.

The difficulty with this analysisis, of course, the probable grosspretreatment differences between theMAP students and . the "control"group: the rest of -the Milwoodeighth, grade class.

Table 12 suggests that pretreat-ment differences did exist: The MAPstudents are significantly olderthan the rest of the eighth grade(probably due to being held back inschool), and they indicate chattheir parents have much less formaleducation than the parents of thetypical Milwood eighth grader. TheMAP studentb Are 'also- SbmeWhat morelikely to be minority students thanare the rest of the eighth grade.

A second analysis was undertakento attempt to control for those pre-treatment, differences identifiedabove: age,'parental education, andminority status. Table 13 shows theonly significant differences thatremain after controlling for thebackgroud variables. MAP studentsreport receiving more rewards at

school than Milwood eighth graders,but they are suspended more often:Thus, most of the differences thatappeared in Table 12 can be attri-buted to the pre-existing differ-ences in the groups.

In a sense, the lack of differ-ences between MAP students andeighth graders can be construed as aproject success. The MAP studentswere selected for the program on thebasis of having behavioral and aca-demic problems. We would expectthem to differ negatively from the

234

0#*3 p4*4 . 9*91M0*i4 4f 4* 4

9*3 3*,. 4

3*o *t4 øft44,* 3 - WOiv

*$44I3p., 3* W 9** kI* 9*1,

4o 1w3OwL4w0,s3. 4 soso $I+t ft.. Ws 4ip144 w.(r*oq*041 *9* 4* 1W $* *P

4.so**o W*a4s4r*4 .4*. ra'

WM40 0051*04 44M'4$ i, ssi*.'$ØC404 4* soø4t* .41* b.0 o* 4 4W 4$) $P

4* 1*. *00*43*04 I**41**P *144*0 - 9* *4th *P.*

*t*** $1*t ***14&W *#*N 9* *P*$*44*: 4W *P $341*. s** is, S

*0*0 *l1i0 $* 94W0. 1I* N3*444,4*4,4 9*

so*i * $st 1*4* **# o' r*,44 *. ** is, 9*

4* IL 14

4*. W i41114*.4t is

i4M. *.**M*4 *1*44 *11* sri Is3 *54*414*4 *44l, JW4t1W'1*0*4 t4s K5*4*0* 9*4 44**$4 t4 9*

fl* 41441*04*.

4* 9*oe f*_4 444 I*$44 *I$w* *4oth 1$L- **)4.to, 4W

4 9*4 lb.** 1s4**4 *dy. Is

eo*4r, lb. ffi. iI 3b40** *P4 1W 0*4*4*4* ope**s14s 9*3 1*41

.o4* $**4i 4* i*4' 4

*4 4* 4ititssU t. 4041$44 4*444 44 0 p451 44 bad

Op. 00 9* It

4* lbs *su. Is*4*10* lb. t*ot* sri* Nil LUsol *m * 14*4* $ s*4 lb.*4*. 4*, t4* *epittms4*s4+ 4$*M**$ s r*44o*or .4 1*1øm34 *4* 4sc*flsssl

a"

*ttwsod ?r*j.cs

ibe bIjb.r rb.

no asessI t&sLf trait result of1*4. tLamie sos,. saslysts 1.. ibM44* $* t*dssts mousI aIseresi. Li *$0tte 3sit 1sf b*sc.ük*0 III lbs Otiwsod .Ls*4rb grad.aloes Is psrrM. If this La a pro-,rnm lusts, Li I. a sopit,.lust'.

lb. ibe.. sot. of data reported4* ?aM. 13, 13, aid' 14 coso.rg.loused Lb. (etloutag cs.c*wstos Vsdo set ssrrsssly 4*,. s desip ado-4ss1e 4* os.clsoLv.ly oralv.te Lb.

,ffsttiwoss.s .4 1b $P hi aasrtrgtie pils. V4*ti,ir Li. .ft.c*s.19*r p.*titw. or septic., ibly1 *et dremoUc .aoagb to be pithed*p .u*eIetsitIy wee. cart.,., types.4 rnsrly.e.. lbs "global" iupree'-*Iai Oil $041 4. that lbs pragrsey Is eeesfsl1y cr.aUag asebist esetesmisi Is which atudest.that i.sre blYiPig difficulty is

sibisi eeport aee.t,t.g sore raward.e4* ssbe.*. As the us. it... thoseaIuds.ti ottil set 4. trouble sore41*. thu lbs "scans." thUd, aid

y auffsr us, negativehush Ii deeIop1og a network oft,Issdobtpe cub other behavior or

(4*1IN stUdlits (as sea-er.4 by the Sagest,. Peer Tiflneacsseal .).

Th1iMitt4. The srt progrso be sutned is the

SkIlls Lab, a basic skills cussrern. As 1. ii.. ,va1atIos of the$P dasses, there La no equlvalaitossissi groop to serve as a basis of...puIass. Thi Skills Lab (EL)

etsdesls wore tvdeil. selectedIsisoso they bad relatively sever.dtfVls.lti.e Is roediag aid sub,(they bad either failed two or moreslaoa.e she previous year, or waretattIng dvrisg the current year), orlad Osw. attuduc.o or behaviorpsOhsa. those t,.dsnU who were

108u.f1, , s-.w*-r,-.. 't

111.....0.11

?Inwood Project

eighth graders had academic skills-that were considered to be at toolow a level for them to adequatelyfunction in the MAP program--wherethe instruction parallels that of

the regular school curriculum--orwere thought to be so disruptivethat they would be a negative influ-ence on the other MAP students.Seventh graders in the program hadacademic problems or disciplinarydifficulties, and did not qualifyfor special education or Title I

programming. Thus, the Skills Labstudents were a select group: theyhad weak academic skills, and wereusually presenting disciplinaryproblems.

Table 15 presents the results ofANOVAs comparing the Skills Lab stu-dents first to the rest of the Mil-wood population (Skills Lab studentscould be in either the seventh oreighth grade), and second, to theMAP students.

Looking at the differences bet-ween the SL students and the generalMilwood student population, as one

might expect given that the SkillsLab students are selected because ofhaving very poor basic skills, theydiffer from the other Milwood stu-dents on a host of measures. TheSkills Lab students are lessattached to school, find school morepunishing, expend less effort inschool, and report greater negativepeer influence. They are also lessattached to their parents and havelower self-esteem. Behaviorally,the Skills Lab students say thatthey are suspended more often, aremore Jelinquent, use drugs moreoften, and receive lower grades.They are also older, expect to goleas far in school, and (not sur-prisingly), report being poorerreaders. Finally, the Skills Lab

students report that their parentsare less educated than the typicalMilwood parent.

236

This analysis simply verifieswhat one would expect about a groupof students that were selected forhaving large deficits in basicskills, and says little about the

success of the intervention in meet-ing the needs of those students.The differences between this groupand the Milwood population as a

whole were considered too gross toconduct an analysis controlling forbackground variables.

Table 15 also gives' the resultsfor signficance tests for differ-ences between the SL and the MAPstudents. Again, the differencesbetween the groups are extreme, andmirror the results from the compari-sons between the SL and the Milwoodstudents. (See Table 12 to compareMAP means with SL means.) Althoughthe outcome differences were againlarge (and presumably, pretreatmentdifferences were large as well), a

series of analyiesi of covariancewere conducted on the scores of theSL and MAP students, controlling forparental education, reading ability,and age. Table 16 gives adjustedmeans and probability levels for

those measures that remained signi-ficant.

The psychosocial scales show thatat the end of the 1982 school year,the SL students were less attachedto school, expended less effort atschool, and were less involved in

school activities than were the MAPstudents (even while controlling forthe background variables). They

also found school more punishing.Not surprisingly then, they expectedto go less far in school. The SLparticipants also reported morenegative peer influence.

The behavioral self-report mea-sures indicate that the SL studentsengaged in more delinquency, drug

use, and serious delinquency, and

were suspended more in school. They

109...

reported receiving lower grades thandid the MAP students.

Skills Lab students finished the1982 school year with much lessfavorable outcomes than the MilwoodAlternative Program students.Because we do not have the data tocontrol adequately for pretreatmentdifferences, the outcome differencesbetween the SL and MAP students arenot very informative concerning theSkills Lab's success. There arelarge outcome differences, but therewere probably large input differ-ences between the two groups of stu-dents.

It is apparent though, that theSkills Lab did not have large posi-tive effects on its students; the SLparticipants finished the year toopoorly. In addition, project staffwere disappointed in the program.Partly in response to the requiredOJJDP budget cuts, project staffdecided to discontinue the interven-tion for the 1982-83 year.

To what do we attribute the

undistinguished performance of the

Skills Lab? Interviews with projectstaff indicate one fundamental prob-lem: the teacher around whom theintervention was designed, and whocontributed much toward the programin its planning stages in 1981, didnot ultimately teach in the lab.

School system budget cuts resultedin teacher layoffs and caused the

original teacher to be transferredto the elementary level. Initially,her position was covered by substi-tute teachers, and later by a

teacher selected by central adminis-tration, without consultation withthe project. This person was a

counselor and had never taught in

the classroom, much less taughtbasic skills to academically handi-capped students. After projectstaff complained about the appoint-ment, and refused to pay the

237

Mil wood Project

instructor's salary out of projectfunds, a search for a new teacherwas undertaken. The choices offeredto the project were still limited tothose aaliable for recall based onseniority requirements. It was notuntil January that project staffidentified a teacher that they hopedwould be suitable. The final

teacher was a former assistant highschool football coach, who had beenbumped "down" to the middle schoollevel. He had no. experience in

teaching low-skills students, andwas not trained in special education(the teacher around whom the programhad originally been designed held amasters degree in special educa-tion). Project staff understood,though, that he could be a patientteacher. Staff subsequently foundthat he was unexcited about teachingthe basic 'skills class. He hassince returned to football. Inshort, the Skills Lab had multipleteachers throughout the year, andnever at any time, had a teacherexperienced in or enthusiastic aboutteaching low-skills children.

Home-school liaison. The home-school liaison along with the atten-dance monitor (a secretarial-typeposition) are responsible forclosely monitoring daily attendance,and intervening by .letter, phone,and home visits--in a structured,sequential process--to insure thatstudents are present at school. Theattendance monitor collects dailyabsence reports, and prepares a

daily list of students absent, whichincludes the number of consecutivedays that the student has beenabsent. The home-school liaisonthen uses that information in makinghome contacts. (The home-schoolliaison is also responsible for

escorting students home that aregiven exit suspensions and whoseparents cannot pick them up.)

no

Milwood Project

The SAES self-reported attendancedata shows no change from 1981 to1982 for either Milwood or South(see Tables 1 and 4). Individuallevel attendance data are not yetavailable at the writing of thisreport, but the aggregated schoolattendance data (as reported by theschools to the school district) canbe examined. From 1981 to 1982 theaverage daily absence rate nearlydoubled at South from 11% to 21%.During this same period, the absencerate decreased at Milwood from 13%to 11%. This pattern of effectsresembles that found for the SAESschool climate measures: little(though positive) change at Milwoodaccompanying a a large negativechange at South.

The home-school liaison spends agreat deal of his time working withchronic non-attenders, students thatare very often absent from school,usually for several consecutive daysat a time. The records of Milwoodand South were searched to identifystudents who missed 30% or more ofthe days for which they wereenrolled to see if there was a

change in the number of such stu-dents. At South during 1981 74children (11% of the average atten-dance period enrollment) fell in thenon-attender category. In 1982there were 73 (11% of the averageattendance period enrollment)chronic non-attenders. In short,there was no change in the number oftruant students at South. At Mil-wood there was a drop in the numberof chronic non-attenders from 83

(12% of the average enrollment) to57 (9% of the enrollment). This 25%drop in the proportion of truants atMilwood is statistically significant(z = 2.14, p < .05).

These results suggest that thehome-school liaison is successfullyincreasing attendance at school,particularly by increasing theattendance of students who would

238

have been absent for several conse-cutive days.

The In-school Suspension Center.As is the case with attendance,besides the SAES questionnaire mea-sures, only aggregate counts of sus-pension are currently available.The questionnaire data indicatedlittle change in suspensions at Mil-wood, and an increase at South (seeTables 1 and 4). The aggregatedexit suspension information showsthat as one would expect with theinstallation of an in-school suspen-sion center, the number of exit(out-of-school) suspensions declined

,,dramatically at Milwood from 1981 to1982. In 1981 there were 510 exitsuspensions levied, while in 1982,there were only 105. This repre-sents an 80% decrease in exit sus-pensions. At South, exit suspen-sions showed a moderate decreasefrom 195 to 170.

It is impossible at this time todetermine how much of the dramaticdecline in the exit suspension rateat Milwood is due to a change in

assistant principals or due to thedevelopment of the in-house suspen-sion room. Milwood's previousassistant principal handed out manymore exit suspensions than the aver-age Kalamazoo disciplinarian; Mil-wood's exit suspension.*rate for the1980-81 year was over twice theaverage rate of the other two Kala-mazoo junior high schools. The Mil-wood project helped lobby for a newassistant principal, and the newperson does not resort'as quickly toout-of7school suspension.

At the same time, the 105 exitsuspensions at Milwood in 1982 wereconsiderably less than the 170 atboth South and Hillside junior highschools. Consequently, the avail-ability of the in-school suspensionroom appears to supress the numberof out-of-school suspensions.

J

The Student Council. StudentCouncil representatives for eachhomeroom class were chosen by a lot-tery from the students in that home-room who nominated themselves forstudent council membership. Thisprocedure produced a natural quasi-experiment. Table 17 gives theresults for the ANOVAs comparing thestudent council members to the ran-dom controls (those students notselected in the lottery). Perhapsthe three most relevant SAES mea-sures in terms of the goals of theprogram are attachment to school,school rewards, and involvement inschool activities. Of these,

attachment to school shows no effectof the program, but both schoolrewards and involvement in schoolactivities show significant positiveeffects for the student councilintervention.

The picture is clouded when oneexamines the rest of the results ofanalysis; several negative outcomesare apparent. The student councilstudents report more negative peerinfluence than the random controls.They also have lower self-esteem,and are less attached to theirparents. They say that theirparents put less emphasis on theireducation than the control students.Finally, the student council stu-dents have lower educational expec-tations than do the controls.

These results are puzzling, espe-cially those concerned with parentalvariables. There does not appear tobe any theoretical reason why stu-dent council membership would affectthese variables one way or another,much less negatively. Conceivably,since the student council ran someactivities involving parents, thestudents "learned" that theirparents did not care about theireducation, and therefore became lessattached to their parents. Thisexplanation seems a little far-

239

Milwood Project

fetched, and grants astonishingpower to a rather weak intervention.Likewise, it is hard to see why stu-dent council membership wouldadversely affect one's educationalaspirations.

This pattern of puzzling resultstends to support the hypothesisthat, despite an apparent randomiza-tion process, the student councilstudents were a somewhat differentpopulation than the random controls.Although none of the following dif-ferences are statistically signifi-cant, they consistently favor thecontrol group over the student coun-cil members. The student councilmembers are more likely to be malethan the controls (55% vs. 45%),i.e., girls were more likely to

volunteer for the student council,but boys were more likely to beselected for it. The student coun-cil members' parents tend to havehad less education (p = .20) thanthe parents of the control students,and the student council studentsalso report themselves to be

slightly poorer readers. In addi-tion, there is also a weak trend(p=.13) for the student council mem-bers to be suspended more often thanthe controls. Again, it is diffi-cult to imagine why being in the

student council would cause one toget suspended.

One possible explanation for theresults is that student council mem-bers were chosen in a random lotteryby homeroom, i.e., from each pool ofself-nominees from each home roomclass, a representative and analternate were drawn. Because theMAP and Skills Lab classes consti-tuted homerooms, students with theirqualities may have been overrepre-sented on the student council. Ananalysis of variance was carried outwhich excluded those student councilmembers and controls who wereenrolled in either the MAP or Skills

Milwood Project

Lab programs. Significant resultsfrom this analysis may be found inTable 18.

As can be seen, these results arevery similar to those reported in

Table 17. The non-MAP and non-Skills Lab student council membersreport lower parental emphasis oneducation, and less attachment totheir parents. These student coun-cil members have lower self-esteem,and suffer from greater negativepeer influence. They also work lesshard in school. The student councilmembers still report more schoolrewards (the effect is less signifi-cant because of the decrease in n,but the absolute size of the differ-ence is the same). Oddly, the stu-dent council members that were notin the MAP and Skills Lab did notfeel more involved in school activi-ties. To summarize this analysis,even removing the effects of a pos-sible overselection of students withbehavior and academic problems, the

student council apparently had nega-tive effects on the attitudes of thestudents that served on it.

There are three possibilities toexplain these anomalous results:(a) the randominzation process wasnot truly random, and some unidenti-fied selection bias was introduced(this possibility exists becauseeach homeroom teacher did his or herown random selection from thatroom's pool of nominees, and we donot know if this procedure was car-ried out properly); (b) the lotterywas random, and by chance, thegroups were not equivalent; and (c)the lottery was random, the groupswere initially equivalent, and thestudent council experience producedsome negative attitudinal effects inthe children.

The Student Proiect Advisory. At

the writing of this report, no

information was available concerning

240

the unique contributions or effectsof the student project advisory.

Summary of Second Year EvaluationResults

1. Milwood has shown some slightincrease in the quality of its

school climate, while the climate atSouth (the control school) hasdeclined.

2. We cannot as yet conclusivelyattribute the differences in the

school climate changes in the twoschools to the Milwood AlternativeEducation Project, but it appears

,probable that the project is favora-bly affecting the climate of MilwoodJunior High School under conditionsof declining resources available forKalamazoo schools.

3. Milwood demonstrated signifi-cant and substantial _improvement

andMAT peiformance between 1981 and1982, as compared to South whereperformance remained the same acrossthe two years. Both the seventh andeighth grades showed improvement atMilwood, with the seventh grade mak-ing somewhat stronger gains in

scores. We cannot judge the impactof the Milwood Alternative EducationProject on these basic skills gains.

4. The MAP program is probablysuccessful in creating a school

environment that is more rewardingfor academically marginal students.It is possible that the grouping ofsuch children together may lead tosome negative peer influence.

5. There is no evidence that theSkills Lab was successful in

increasing the positive schoolexperiences of its students.Whether this was due to a lack incontinuity and expertise in the

teaching staff for the intervention,due to some problem inherent to theclass, or due to pre-existing char-

113

acteristics of the students, we donot know.

6. The home-school liaison isprobably successfully improvingattendance at Milwood, particularlyfor students that are chronicallyabsent. This intervention has beenviewed so favorably by the schoolsystem that home-school liaisonshave been installed (at school sys-tem expense) in all Kalamazoo juniorhigh schools.

7. Most of the large decrease inexit suspensions at Milwood can beattributed to the change in assis-tant principals, but comparison withthe other Kalamazoo junior highschool& suggests that the availabil-ity of the in-school suspension roomhas also decreased the number ofexit suspensions.

8. The student council increasedschool rewards for its members, butdid not help them feel any moreinvolved in school. There is evi-dence that the student council mayhave had negative effects on thechildren's attitudes, perhapsthrough a negative peer influencemechanism. This possibility seemsunlikely, so we await replication ofthese effects in the coming year'sdata.

Changes Planned for 1982-83

Changes in the Milwood Alterna-tive Education Project that areplanned for 1982-83 fall into sixgeneral categories: (a) changes inthe student composition, use ofinstructor time in the MilwoodAlternative Program and time spentdoing academic work; (b) eliminationof the Skills Lab class and substi-tution of an individualized readingassessment and tutoring program; (c)addition of an intensive study pro-gram for selected failing students;

241

Milwood Project

(d) greater and more systematic useof college student and adult volun-teers; (e) elimination of the atten-dance clerk and counselor's secre-tary positions; and (f) increasedemphasis on student involvement in

school activities.

Changes in the Milwood AlternativeProgram

The MAP was originally designedto include some "positive" influencestudents: students who were havingsome difficulty in their academicwork, but who were not considered tobe behavior problems. It was hopedthat these students would serve to

'''set a positive example for the lesswell-behaved students. In planningfor the 1982-83 year, project staffdecided that there had been too fewof the positive influence students(only one or two in a class oftwenty), and that they had not been"strong" leaders in the class.Therefore, their positive modelingwas felt to have been ineffective.For the coming year, a larger numberof positive influence students havebeen included in each class, (alt-hough the numbers of positive stu-dents are still small) and studentshave been selected who are believedto be stronger peer leaders. It ishoped that the the greater numberand influence of the positive stu-dents in the class will decreasesome negative peer influence effectsthat project staff felt took placeduring the 1982-83 year.

In addition to the inclusion of agreater number of positive influencestudents a different selection cri-terion was used for the majority ofthe MAP students. Students wereselected who were judged to beenattending school enough to be ableto benefit from the treatment. In1981-82, several students withdrewor were dropped from the programthat never attended school regularly

114

Milwood Project

from the beginning of the year; theynever received the treatment.Chronic non-attenders were notselected for 1983. Also, it is theimpression of project staff thatthey did not select students thathad as severe discipline or academicproblems as some of the studentsincluded last year. In short, theselection criterion were moved "up."This selection change was madebecause project staff felt that manyof the students included in the pro-gram in its initial year were netresponsive to treatment because theyhad, in effect, already dropped outof school, or were'exhibiting disCi-plinary problems too severe to be

dealt with in a school-within-a-school. Staff believe that these"worst" students were adverselyaffecting the rest of the studentsin the class. Such students werenot invited to participate thisyear.

Two changes have been made in theMAP instructors' and counselors'time. Formerly during the firstperiod of each day (the "grouptime"), one of the three schoolcounselors assisted each of the

three MAP homeroom instructors in

carrying out the group activities.Building staff felt that this was aninefficient use of counselor time,

, and it did not provide any counselor"cover" for the rest of the Milwoodpopulation for that first periodeach day. Consequently, counselorsrotate duties during the first grouptime period, with one of the counse-lors always remaining in the coun-seling department office. One of

the MAP classes does not have a

counselor, then, during group time,but the MAP staff feels that theinstructors can carry out the activ-ities themselves since they now havea year of experience in facilitatingthe group time.

A more fundamental program change

a .

242

of the instructors' time involvessome extension of the MAP interven-tion into the afternoon, when thestudents attend regular classes. In

1982, the MAP teachers taught regu-lar classes in the afternoon, whenthe MAP students were taking classwith the rest of the Milwood stu-dents. For the upcoming year, oneMAP instructor's job has been res-tructured so that he is available inthe afternoons to help MAP studentsand their teachers in their main-stream classes. This "support" forMAP students in the afternoon, whenthey are not in the alternativeenvironment, was deemed necessaryfor two reasons. First, the MAPstudents still had difficulty in

some of their non-program classPes,and it was felt that they and theirteachers could benefit from some aidfrom a program teacher. Second,because the MAP students take all oftheir morning classes together,there is a Iendeney-forthem to endup in the same classes in the after-noon due to a limited number of

electives. In effect, some of theafternoon class teachers end up witha "MAP" class when they do not havethe training or desire to work withthis group of students. It wasthought desirable to have a MAPinstructor available to help teach-ers cope with this overabundance of"problem" students.

Another important programmaticchange involves the addition of an

after school study hall for MAP stu-dents. At weekly intervals, the

students' progress on their writtenassignments is reviewed, and stu-dents who are performing poorly orhave assignments outstanding arerequired to participate in an afterschool supervised study hall. Stu-dents and their parents are informedof tardy work on Friday, and if it

is not completed, they are requiredto attend the study period on thefollowing Tuesday, Wednesday, or

115

Thursday. The MAP teachers rotatein supervising this special studyhall.

The last important change in theMAP program involves the stucturingof the "group time." Group counsel-ing type activities are now heldonly three days a week instead offive. The other two days aredevoted to supervised study halls.Students work on their assignmentsor study for exams while receivingindividual attention from their MAPinstructor.

To summarize the major changes inthe MAP for the 1983 school year,they are of three kinds: (a) the

program participants are expected tobe better behaved, and to.have bet-ter academic skills; (b) programsupport will be provided to the stu-dents outside of the regular morningprogram, and (c) more time is beingspent on academic pursuits, and lesson group counseling.

Changes in Basic Skills Instruction

The Skills Lab class has beendiscontinued. It was not viewed byproject and building staff as havingbeen successful, and it was elimi-nated in reponse to the required 30%third year funding budget cut.

Because the upgrading of basicskills is theoretically important toproject staff, an alternative--lessexpensive--attempt to address the

problem has been designed. A read-ing specialist has been hired whowill be responsible for testingchildren identified by teachers andproject staff as having readingproblems. Academic success and MATscores are the primary referralsources. The specialist is a gradu-ate student at Western MichiganUniversity who holds a Mastersdegree in reading education. She

will work at the school 2-3 days per

243

Milwood Project

week. Students will be tested to

insure that they do have a readingproblem and to diagnose the natureof the reading difficulty. An indi-vidualized plan based on intensivetutoring will be drawn up for eachchild. It will be the reading spe-cialist's job to insure that the

child is receiving the readingtraining judged to be appropriate.

Importantly, the reading tutoringwill focus on the skills and aca-demic areas that the child is exper-iencing most trouble with, andtutoring materials will be takenfrom the class from which the childis drawn for tutoring. Because oftransportation problems, it wasjudged to be impossible to run anafter-school tutoring program. The

reading specialist therefore decidedto tutor children during classesthey were already failing (had

received an "F" in the previousreport card), reasoning that the

child could not do any more poorly,and it was therefore unlikely thatthe once-a-week removal for tutoringwould hurt the child's performancein that class. The child's assign-ment for the class he or she is

missing during the tutoring sessionsconstitutes the material for that

session. For example, if a child istutored during his science class, heis tutored in reading using the sci-ence material his class will be cov-ering that day.

Addition of an Intensive Stud/ Pro-gram

Eleven students that are experi-encing academic failure, who havebeen judged by project and schoolstaff to have the requisite abilityto perform successfully in school,

and are not enrolled in the MAP, areinvolved in an Intensive Study Pro-gram. In this program, they meetweekly with a project staffer to goover their assignments, plan their

110

Milwood Project

next week's academic work, andreceive tutoring. The staffer (whois in the building full time) checkson their academic progress almostdaily, and works with the studentsand their teachers to insure thatthe students keep up in their day-to-day assignments.

The program centers around a

"time management" approach, in whichthe timely scheduling and completionof assignments is emphasized. It ishoped that the students will betterknow what is required of them ineach of their classes, will schedulea particular time to do the assign-ment, and will complete it on time.It is believed that the focusedattention of a project staffer willmotivate these students to completetheir school work and improve theirgrades.

Additional Project Changes

In 1980-81 some limited use of

college student volunteers had beenmade in tutoring, and in supervisingthe in-school suspension center.

Student volunteers were not used in1981-82 because of problems insupervising the volunteers and ininsuring that they would be availa-ble on a regular basis. For the1982-83 year student volunteers willagain be recruited, and a moresystematic effort will be made tosupervise and coordinate theirefforts. Vol;,:nteers will be primar-ily used in helping with extracurri-cular activities, in tutoring, andin collecting and maintaining atten-dance data.

Due to the OJJDP budget cuts, theproject was forced to eliminate twosupport staff positions: the atten-dance clerk and the secretary in thecounseling center. The duties of

these persons are being covered byother project staff, and by the useof volunteers.

The final change in MilwoodAlternative Education Project activ-ities for the 1983 year is one ofemphasis. Project staff have alwaysbeen concerned with getting Milwoodstudents involved in school activi-ties and school decision-making.They were disappointed in the evalu-ation results that indicated thatthe students did not perceive them-selves to be any more involved in

the school or to be influencingschool policies. Project staff planto spend more time in the comingyear working with students in plan-ning and carrying out activities.They hope to increase the number of

,activities (through planning and theuse of volunteers) that students canbecome involved in.

Particular interventions that areplanned to involve students in

school activities are a pep club, a

choir, an intramural program, and aspring theatrical produition.- Theseprograms will be designed and imple-mented with the aid of the studentcouncil and the student advisoryboard, giving these students moresay in the events at school.

Recommendations

1. MAP teachers must be aware ofthe possibility of peer influencecontributing to a negative classroomatmosphere. Particularly, disrup-tive behavior that is reinforced bythe other students in the room candestroy the possibility of learning.This is a drawback to any programthat seeks to put a large proportionof behavior problem studentstogether in one class. Projectstaff have recognized the problem,and have attempted to rectify it byscheduling more "positive" students.

. It is probable though, that the num-ber of positive students (two or

three) in a class of twenty, is

insufficient to overcome the nega-

244

11?

tive peer influence of the rest ofthe class. The MAP teachers willhave to be aware of the potentialproblem, and should be prepared torespond quickly to contagious beha-vioral disruption.

2. The affective education cur-riculum for MAP class, organizedaround a three-day-a-week groupexperience, is not focused, and isoften carried out in an ad-hoc fash-ion. It is apparent that thecounselors and the teachers developmuch of the group material on the

themetare

spot. It is unlikely thatobjectives of the program can beunless specific interventionsdesigned to address.them.

3. Plans underway to documentthe actual curriculum taught in theMAP, should be examined. Thedescription of the MAP will only beuseful to future program implemen-ters to the extent that it is clear,and covers all activities and inter-ventions undertaken.

4. The teacher of the new remed-ial reading program should attendclosely to possible negative effectsof taking students out of theirregular classes. The program doesplan to organize the reading tutor-ing around the material the studentsare covering in their cldsswork, butit is probable that not as muchmaterial will be covered in thetutoring sessions as in the regularclasses. In addition, the negativelabeling that could result frombeing sent to this class could harmthe overall academic performance ofthe students. The reading special-ist should make every effort to pro-vide tutoring outside of the stu-dents' regular class schedule, and

allow the students to remain intheir mainstrean classes as much aspossible.

5. The Cambridge-Sommerville

245

1.;

Milwood Project

study (Powers and Wittmer, 1951)

inve6tigated the del inquency- preven-tion potential of assigning a case-worker to monitor the behavior andprovide support to lower class boysidentified at risk for delinquency.Long-term follow-ups to that study(McCord, 1978) indicate that the

program actually had severe negativeeffects on the life experience of

the participants. McCord attributesthe negative effects to a violatedexpectancy that "things are going toget better": children'in the pro-gram felt "special" and were led tobelieve that their lives would nowtake a dramatic, turn for the better.These children found that life didnot, in fact, get much better.Apparently, many of the participantsinternalized their failures, decid-ing that, "Even with all of thishelp, I still can't make it."Accordingly, the Milwood volunteersmust beware of setting up unrealis-tic expectations in their clients,that, if not met, produce negativefeelings of self-worth in the chil-dren.

6. The introduction of the in-school suspension room did not

decrease the number of self-reportedsuspensions (which includes both in-and out-of-school suspensions). Adifficulty with in-school suspensionis that it is a less severe sanctionthan out-of-school suspension, andthus, tends to be resorted to forless extreme disdiplinary infrac-tions. Ironically, this "progres-sive" procedure can result in moresuspensions. Fortunately, in thecase of the Milwood project, theinstitution of in-school suspensiondid not increase the number of

self-reported suspensions and did infact decrease the number of exitsuspensions--the program's primaryobjective. Nevertheless, projectstaff should make clear to buildingpersonnel the possibility of too

Milwood Project

frequent suspension, resulting fromthe decreased severity of the pun-ishment. Every effort should bemade to insure that suspensions,either in- or out-of-school, arelevied only for clearly specifiedextraordinary offenses.

7. A final caution concerns thethe Student Council. While it is

246

theoretically unlikely that the

negative effects associated with thestudent council in 1982 were actu-ally program outcomes, the possibil-ity exists. Project staff must paycareful attention to any processesthat may occur in the student coun-cil meetings or in the school at

large that could produce the nega-tive effects found in the 1982 data.

119

Milwood Project

References

Carlton, R, and Cook, M. The Milwood Alternative Education Project: InterimReport. In G. D. Gottfredson (ed.), The School Action Effectiveness Study:First Interim Report (Repoit No. 325). Baltimore: The Johns Hopkins Univ-ersity, Center for Social Organization of Schools, 1982.

McCord, J. A thirty-year follow-up of treatment effects. American Psycholo-gist, 1978, 33, 284-289.

Powers, E, and Wittmer, H. An experiment in the prevention of delinquency:The Cambridge-Somerville youth study. New York: Columbia UniversityPress, 1951.

247

120

Table 1

Comparisons of Milwood and South onSelected 1981 SAES Measures

School variables

ri

Milwood

SD N M

South

SD N p

Attachment to school .63 .27 206 .65 .26 196 .67School rewards .20 .27 217 .23 .27" 205 .37School punishment .20 .29 217 .25 .26 206 .06*Parental emphasis on education .68 .26 218 .68 .26 217 .92School effort .57 .31 223 .61 .29 219 .23Negative peer influence .21 .2P 219 .22 .23 217 .55Victimization .14 .17 211 .13 .17 202 .52Involvement .19 .16 214 .22 .16 210 .08*

Psycho-social variablesAttachment to parents .59 .28 222 .60 .27 218 .87Belief in rules .68 .25 205 .72 .24 187 .14Positive self-concept .69 .17 194 .73 .16 189 .03**Rebellious autonomy .73 .32 193 .63 .30 173 .003**Interpersonal competency .80 .23 204 .79 .21 181 .69Internal controla -- -- --Alienation .36 .30 202 .37 .29 193 .83Practical knowledge 1.37 .44 206 1.39 .45 191 .66Educational expectation 3.48 1.72 222 3.61 1.73 218 .44

Self-reported behavior variablesSchool non-attendance index .54 .74 224 .37 .68 218 .01**Delinquency .19 .20 216 .14 .18 210 .005**Serious delinquency .12 .19 216 .10 .19 211 .18Suspensions .25 .43 210 .14 .35 203 .007**Drug use .30 .32 215 .17 .24 210 .0000**Self-reported grades 2.67 .91 220 2.69 .84 219 .80

Background variablesSex (% male) 49% -- 221 51% -- 220 .74Age 13.10 .87 225 13.15 .95 221 .48Race (% minority) 31% 220 45% -- 209 .005**Parental education 2.28 1.15 196 2.49 1.17 189 .08Reading ability 1.82 .87 200 1.85 .86 201 .72

aNot available in 1981.

* Nearly significant.** Statistically significant.

-248-

121

figNititist CetrolatIoss Botwvimget, (Iiimecity vs. *ROI

*04 MI Quoottemostro halt's°

44000140,004,* 111040Vt*

tiolot le 04. W.* .13 .006 380

011141 too .15 .000 409

lovoltromosit .17 .000 409

tfsies4 .18 .000 422

Plitvotal tidoestom .22 .000 370

Ptittital taawledi,

footeoltioss

-.11

-.IS

.01

.000

385

399

mA.,26,4m4varwr1pairstmamm.wcamsmaimmawat

Stolottlp *mu, co6o4 "0" and vbite coded "1."

bigot mire 141amallee Samplo.

..29

9.111161121. Z. oiL111.17. . =11.11113111111.1.Y110.61:111111 4.1 ....,101111111MVIINENIMEMI*11..- 7.11NIMIWOM..

122

Table 3

Significant Differences between Milwood and Southon Selected 1981 SAES Measures

Controlling for Race and Fdi-ental Education

Milwood South

Adjusted Adjustedmean N mean N p

Suspensions .25 181 .12 165 .002

Rebellious autonomy .71 167 .62 145 .01

School nonattendance .52 4192 .38 176 .04

Delinquency .19 184 .15 169 .03

Drug Use .30 183 .18 169 .000

250

123

Table 4

Comparisons of Milwood and South onSelected 1982 SAES Measures

School variables

Milwood

M SD N

South

N SD N1

Attachment to school .64 .28 474 .57 .26 224 .002**School rewards .17 .23 496 .19 .24 221 .30School punishment .22 .25 497 .29 .30 221 .001**Parental emphasis on education .62 .28 407 .61 .30 210 .68School effort .58 .32 563 .59 .29 233 .68Negative peer influenceVictimization

.24

.16 3.22.20 5L91.714.25

.21

.24

.26

230220

.58

.005**Involvement .18 .16 550 .21 .17 231 .02**

Psycho-social variablesAttachment to parents .55 .30 558 .55 .29 232 .99Belief in rules .70 .27 332 .68 .26 191 .40Positive self-concept .74 .18 384 .73 .18 199 .53Rebellious autonomy .63 .34 302 .65 .32 183 .52Interpersonal competency .78 .24 328 .75 .25 188 .18Internal control .40 .26 346 .43 .26 191 .22Alienation .38 .29 342 .43 .25 193 .04**Practical knowledge 1.35 .46 295 1.25 .49 186 .02**Educational expectation .42 1.79 566 3.42 1.72 236 .72

Self-reported behavior variablesSchool non-attendance index .48 .73 566 .56 .62 234 .14Delinquency .17 .20 507 .20 .21 215 .07*Serious delinquency .11 .19 509 .14 .20 214 .06*Suspensions .21 .41 459 .18 .38 218 .36Drug use .28 :34 514 .27 .31 220 .72Self-reported grades 2.65 1.08 565 2.68 .95 230 .72

Background variablesSex (% male) 48% -- 565 54% -- 235 .001**Age 13.23 .93 573 13.22 .90 235 .89Race (% minority) 34% -- 676 46% -- 308 .001**Parental education 2.56 1.21 511 2.55 1.13 218 .92Reading ability 1.75 .88 516 2.04 .84 216 .001**

1 These are the actual N's. Means and SD's for the South sample are weighted by theprobability of sampling; all 1980-81 respondents were resurveyed.* Strong trend.**Statistically significant.

-251-

Table 5

Comparisons of Milwood and South on Changes in SAESScores from Spring 1981 to Spring 1982

Changes in school variables

Milwood

SD N

Attachment to school -.09 .33 69

School rewards -.07 .36 75

School punishment .04 .42 .75

Parental emphasis on education -.06 .38 69

School effort -...05 .42 81

Negative peer influence .07 131 75

Victimization .05 .29 71

Involvement -.02 .21 76

Change in psycho-socialvariablesAttachment to parents -.06 .43 83

Belief in rules -.02 .36 59

Positive self-concept .00 .27 58

Rebellious autonomy -.03 .42 47

Interpersonal competency -.03 .36 55

Internal controlaAlienation .11 .39 60

Practical knowledge -.03 .55 55

Educational expectation -.12 2.45 84

Changes in self-reportedbehavior variablesSchool non-attendance index .07 1.16 84

Delinquency .05 .31 70

Serious delinquency .03 .27 70

Suspensions .01 .63 70

Drug use .10 .49 73

Self-reported grades -.15 1.53 80

Changes in school popula-tion (background) variablesParental education .04 1.71 69

Reading ability -.13 1.23 67

South

M SD N

-.05 .37 68 .48

-.06 .37 68 .85

-.04 .41 69 .26

-.09 .37 71 .60

-.01 .40 74 .63

.01 .32 74 .31

-.06 .25 67 .82

.00 .25 70 .56

-.05 .42 74 .85

-.10 .34 55 .21

.00 -.24 60- .98-

.07 .43 52 .25

.00 .30 57 .68

-.01 .36 59 .07*-.14 .61 54 .32

-.33 2.63 72 .60

.19 1.14 73 .51

.05 .30 67 .95

.02 .30 67 .86

.12 .48 67 .27

.12 .41 68 '.84

-.04 1.36 70 .65

.12 1.62 61 .77

.34 1.34 65 .04**

allot available for 1980-81.

*Nearly significant.**Statistically significant.

-252-12 5

Table 6

South Students Performed Better thanMilwood Students on 1981 M.A.T.s

Milwood

Adjustedmeana N

South

Adjustedmeana N p

Reading 32.5b 586 36.3 571 .000*(44)c (52)

Language 33.6 586 36.6 571 .000*(44) (51)

Math 28.4 586 31.4 571 .000*(47) (56)

Composite 94.5 586 104.3 571 .000*

(45) (53)

Note. Results are the same whether using raw scores, percentiles,scaled scores, or normal curve equivalent scores.

aMeans are adjusted for grade, sex, race and age, using SPSS routineANOVA with classical analysis, controlling for school, grade, sex,race, and age.

bAdjusted raw score.

cAdJusted percentile score.

*Highly statistically significant.

Table 7

Milwood Students Perform as Well as South Students on1982 Reading and Language M.A.T.s, and Peform Better

than South Students on 1982 Math M.A.T.s

Milwood

Adjustedmeana N

South

Adjustedmeana

Reading 38.0b 5z6 38.3 580 .61

(56)c (56)

Language 37.7 576 37.5 580 .58

(53) (53)

Math 32.2 576 31.1 580 .01*

(58) (54)

Composite 108.0 576 106.8 580 .42

(56) (54)

Note. Results are the same whether using raw scores, percentiles,scaled scores, or normal curve equivalent scores.

aMeans are adjusted for grade, sex, race and age, using SPSS routineANOVA with classical analysis, controlling for grade, sex, race,and age.

bAdjusted raw score.

cAdjusted percentile score.

*Statistically significant.

254

127

Table 8

M.A.T. Composite Raw and Percentile Scores for Students ofDifferent Ethnicity, Adjusted for Age: Kalamazoo 1981 and 1982a

White Black Other

Adj. Meanb N Adj. Mean N Adj. Mean N

1981 105.56c 770 84.84 338 102.09 49 .000*(55)d (34) (51)

1982 114.31 765 82.69 358 96.05 33 .000*(62) (40) (55)

72

M.A.T. Composite Raw and Percentile Scores for Students ofDifferent Gender, Adjusted for Age: Kalamazoo 1981 and 1982a

Male Female

Adj. Meanb N Adj. Mean

1981 97.09 559 101.48 598 .008*(47) (51)

1982 105.81 566 108.89 590 .03(53) (56)

Note. Results are the same whether using raw scores, percentiles, scaledscores, or normal curve equivalent scores.

aRace effects are adjusted for grade, sex, and age. Sex effects are adjustedfor race, grade, and age.

bAdjusted means and 2. levels are from SPSS routine ANOVA, using classicalanalysis.

cAdjusted raw score.

dAdjusted percentile score.

*Statistically signficant.

255

128

Table 9

South Students Perform Better thanMilw000d Seventh Graders on the 1981 M.A.T.s

Milwood South

Seventh

Adj. Mean N

Eighth

Adj. Mean N

Seventh

Adj. Mean N

Eighth

Adj. Mean N

Reading 27.4a 312 39.8' 274 31.1 284 40.0 287 .000*

(35)b (59) (43) (58)

Language 28.8 312 40.3 274 31.5 284 40.5 287 .000*

(35) (59) (40) (59)

Math 23.6 312 35.2 274 26.2 284 35.4 287 .000*

(35) (65) (42) (66)

Composite 79.8 312 115.3 274 88.8 284 115.9 287 .000*

(34) (61) (40) (62)

Note. Results are the same whether using raw scores, percentiles, scaled scores, or normalcurve equivalent scores. Means are computed separately for each school, and adjusted for sex,race, and age. Adjusted means are from SPSS routine ANOVA, using classical analysis. 2 lev-

els are for the interaction of school and grade, controlling for school, grade, sex, age, andall other two-way interactions.

aAdjusted raw score.

bAdjusted percentile score.

*Highly statistically signficant.

-256-

Table 10

Milwood Students Perform as Well asSouth Students on the 1982 M.A.T.s

Milwood South

Seventh

Adj. Mean N

Eighth.

Adj. Mean N

Seventh

Adj. Mean N

Eighth

Adj. Mean N

Reading 32.7a 292 45.1 284 31.8 291 43.3 289 .39(47)b (69) " (44) (64)

Language 33.6 292 43.5 284 32.2 291 41.3 289 .31(46) (65) (43) (61)

Math 28.7 292 37.2 284 26.4 291 34.6 289 .13(50) (70) (42) (60)

Composite 94.9 292 125.8 284 90.3 291 119.2 289 .21(56) (69) (42) 62)

Note. Results are the same whether using raw scores, percentiles, scaled scores, or normalcurve equivalent scores. Means are computed separately for each school, and adjusted forsex, race, and age. Adjusted means are from SPSS routine ANOVA, using classical analysis.2 levels are for the interaction of school and grade, controlling for school, grade, sex,age, and all other twoway interactions.

aAdjusted raw score.

bAdjusted percentile score.

*Highly statistically signficant.

-257-

Table 11

Average Change in Adjusted M.A.T. Percentile Scoresfrom 1981 to 1982 for the Milwood and South

Seventh and Eighth Grades

Milwood South

Seventh Eighth Overall Seventh Eighth Overall

Reading +12 +10 +12'' +1 +6 +2

Language +11 +6 +9 +3 +2 +2

Math +15 +5 +11 0 +6 +2

Composite +22 +8 +12 +2 0 +1

Note. These percentile change scores are adjusted for sex race, and age. The

interaction of school x year, controlling for sex, race, and age, is signifi-cant, p<.001, for all four M.A.T. scores.

-258-

13

Table 12

Comparisons ofMilwood Alternative Program Students

to Milwood Eighth Graders

School variables

M.A.P.

M SD N

Eighth Graders

M SD N p

Attachment to school .62 .25 45 .61 .28 202 .73School rewards .20 .24 47 .13 .19 213 .02**School punishment .26 .27 .47 .24 .26 213 .53Parental emphasis on education .55 .26 42 .59 .28 182 .42School, effort .48 .32 51 .54 .32 230 .23Negative peer influence .34 ,p.29 49 .23 .22 231 .005**Victimization .19 .23 46 .15 .20 213 .19Involvement .17 .16 51 .17 .15 229 .77

Psycho-social variablesAttachment to parents .48 .24 51 .51 .30 231 .42Belief in rules .65 .23 35 .68 .28 147 .54Positive self-concept .68 .15 39 .72 .18 166 .19Rebellious autonomy .72 .31 36 .68 .32 127 .46Interpersonal competency .82 .24 37 .75 .24 140 .09*Internal control .51 .28 38 .41 .25 153 .02*Alienation .46 .26 38 .38 .28 150 .10*Practical knowledge 1.34 .37 36 1.40 .44 130 .43Educational expectation 2.76 1.89 51 3.47 1.75 235 .01**

Self-reported behavior variablesSchool non-attendance index .63 .75 51 .52 .74 234 .36Delinquency .27 .25 45 .21 .21 219 .10*Serious delinquency .19 .25 42 .12 .20 219 .04**Suspensions .44 .50 43 .20 .40 204 .001**Drug use .41 .40 43 .37 .40 219 .04**Self-reported grades 2.28 .82 47 2.59 1.10 234 .06*

Background variablesSex (% male) 49% 49 44% -- 233 .65Age 14.00 .81 51 13.70 .70 235 .009**Race (Z minority) 26% 51 20% -- 290 .08*Parental education 1.97 1.12 44 2.61 1.12 217 .001**Reading ability 1.72 .86 46 1.76 .86 223 .77

* Almost significant.**Statistically significant.

-259-

131,==.111410

Table 13

Significant Differences between M.A.P. and Milwood Eighth Graders,Controlling for Parental Education, Ethnicity, and Age

M.A.P. Eighth Graders

Adjusted AdjustedOutcome measure mean N mean

School rewards .23 38 .12 186 .002

Suspensions .41 35 .21 177 .01

-260-

133a

" 7 -.,:'..7',;;PW7?"',_,..4:::,;:trInaagrASPialefeERZEZAVEZIMICI:Zel,....2111ZENC.A.

Table 14

Significant Changes on SAES Scores from 1981 to 1982 forM.A.P. Students and Milwood Eighth Graders

M.A.P. Other Eighth Graders

Outcome measure Mean Change SD N Mean Change SD N p

Educational expectation -1.67 2.69 15 .22 2.29 68 .006**

Negative peer influence .25 .36 11 .03 .29 63 .03**

-261-

134

Table 15

Comparisons of Skills Lab Studentsto Milwood Population

and to M.A.P.

School, variables

Skills Lab

M SD N

Milwood

M SD N

M.A.P.a

p

Ati hment to school .43 .23 14 .64 .28 464 .02** .006*

School rewards .11 .18 9 .17 .23 487 .44 .15

School punishment .45 .32 9 .21 .25 488 .005** .03**

Parental emphasis on education .50 .29 7 .62 .28 400 .25 .77

School effort .28 .32 16 .59 .32 547 .001** .004**

Negative peer influence .56 .28 14 .23 .22 533 .001** .001**

Victimization .22 .34 9 .15 .19 482 .30 .77

Involvement .12 .14 15 .18 .16 535 .11 .17

Psycho-social variablesAttachment to parents .43 .32 16 .56 ".30 542- --.09*- .59- --:

Belief in rules .57 .19 5 .71 .19 327 .24 .35

Positive self-concept .55 .21 5 .74 .18 379 .03** .04**

Rebellious autonomy .73 .37 5 .63 .34 297 .49 .93

Interpersonal competency .83 .32 6 .78 .23 322 .61 .82

Internal control .52 .30 6 .40 .26 340 .28 .94

Alienation .51 .32 7 .38 .29 335 .21 .58

Practical knowledge 1.19 .28 6 1.36 .47 289 .38 .28

Educational expectation 1.35 1.70 17 3.43 1.76 549 .001** .001**

Self-reported behavior variablesSchool non-attendance index .94 .97 17 .47 .72 549 .009** .02**

Delinquency .47 .28 12 .17 .20 495 .001** .001**

Serious delinquency .39 .32 12 .10 .18 497 .001** .002 **

SuspensionsDrug use

.88

.56

.35

.37

8

14

.20

.27

.40 451

.34 500.001**.002**

.0l **

.06**

Self-reported grades 1.50 .94 14 2.68 1.07 551 .001** .001**

Background variablesSQX (% male) 69% 16 472 -- 549 .15 .23

Age 1414.29. 17 13.20 -- 556 .001** .09*

Race (% minority) 13% 24 23% -- 652 .46 - .13

Parental education 1.33 .86 15 2.60 1,20 496 .001** .008**

Reading ability 1.18 .81 17 1.77 .88 499 .006** .008**

aComparison of Skills Lab students to M.A.P. students.

* Strong trend.**Statistically significant

Tattle 16

Stkeificsot Conparisoos of Vills Lab and N.A.P. Students,Costrolliog (or forestal Education, Readirsg Ability and Age

IWOIWWWWMI.OWWW WWWWWW 0.111.1.11.WWWWwor

Skills Lab N.A.P.1111.1100.11

7110WW1VMS.WWwpwimmilm

Adjustedmean N

Adjustedmean

Altocksest to school .40 4$ .68 30 .03

tescatiseal espetcatios 1.14 15 3.13 1 .001

tovelwomost .08 13 .21 31 .03

Ative per infleesce .56 12 .28 30 .03

kiwi effort .36 14 .64 31 .03

Stbool renisboont .45 7 .24 31 .09

eel istoteay .47 10 .21 27 .02

kuj see .62 12 .29 27 .06

Serteus deliequesty .41 10 .15 27 .04

Grades 1.57 12 2.49 29 .004

Sosponsione .96 6 .34 30 .02

Table 17

Comparison of Quasi-Randomly Selected Student CouncilStudents to Students Not Selected for the Student Council

Student Council

Personal characteristic M SD N

School variablesAttachment to school .59 .27 25

School rewards .30 .31 28

School punishment .21 .20 28

Parental emphasis on education .53 .32 22

School effort .50 .32 28

Negative peer influence .32 .26 25

Victimization .19 .22 28

Involvement .27 .15 28

Psycho-social variablesAttachment to parents .43 .33 30

Belief in rules .69 .28 18

Positive self-concept .66 .23 20

Rebellious autonomy .63 .37 17

Interpersonal competency .76 .27 18

Internal control .49 .23 18

Alienation .39 .32 17

Practical knowledge 1.34 .47 17

Educational expectation 2.60 1.87 30

Self-reported behavior variablesSchool non-attendance index .53 .73 30

Delinquency .21 .17 24

Serious delinquency .10 .12 24

Suspensions .35 .48 26

Drug use .32 .38 24

Self-reported grades 2.69 .97 29

Background variablesSex (% male) 55% 29

Age 13.27 .87 30

Race (% minority) 27% -- 34

Parental education 2.31 1.51 27

Reading ability 1.70 .91 27

* Almost significant.**Statistically significant.

-264-

Control

M SD N

.64 .29 151 .40

.20 .23 158 .05**

.25 .26 159 .51

.66 .25 137 .04**

.58 .37 175 .21

.23 .23 170 .04**

.18 .22 159 .75

.20 .16 171 .03**

.56 .29 175 .02**

.70 .27 116 .91

__.75. .18. 133 .05**

.64 .33 109 .92

.79 .23 113 .61

.43 .27 121 .39

.38 .29 119 .89

1.40 .44 106 .58

3.58 1.76 176 .006**

.47 .73 178 .64

.19 .21 158 .70

.12 .20 158 .73

.21 .41 147 .13

.31 .36 159 .97

2.69 1.02 175 .98

45% -- 173 .42

13.31 .87 178 .78

22% -- 226 .53

2.66 1.23 158 .20

1.82 .87 161 .52

137

Table 18

Significant Differences between Student Council Members and Other StudentsWhen M.A.P. and Skills Lab Students are Excluded from the Analysis

Student Council Other Students

M SD N M . SD N

Attachment to parents .44 .35 25 .58 .30 151 .04

Involvementa .26,.15 23 .21 .17 147 .14

Negative peer influence .35 .24 25 .22 .22 146 .006

Parental emphasis on education .54 .33 17 .66 .25 117 .09

School effort .48 .32 23 .60 .32 151 .10

School rewards .29 .32 23 .19 .25 136 .08

Self-esteem .65 .27 15 .75 .16 114 .05

aIncluded because of its relevance to the project. Not statistically significant.

-265-

138

-266-

139

Project PREP: Second Interim Report

D. K. Ogawa and G. D. Gottfredson

Rationale

Project PREP is continuing itseffort to reduce high truancy, drop-out, and suspension rates, alongwith disruptive student behaviors in12 schools in District 9 in Bronx,New York. Ultimately, Project PREPanticipates that teachers and stu-dents will perceive school as a safeand viable place for learning. Alt-hough the statement of the project'stheoretical rationale remains essen-tially the same as last year's, thefollowing ideas have been added:

1. Lack of opportunities andpositive experiences createfear and frustration in stu-dents, lessens stakes in con-formity, lessens self-esteemand ultimately leads to angerand hositility. Creation ofopportunities, e.g., jobs, andpositive use of leisure timewill decrease delinquency anddecrease the chances of asso-ciating with those who doengage in delinquency.

2. Students lack caring, posi-tive, and strong role modelsand a consistent set of firmand fair guidelines to follow.

3. Students lack commitment tothe conventional norms andbeliefs of society.

This second interim report sup-plements an earlier report (Ogawa,1982) which should be consulted fora basic overview of this project andan account of earlier activities.

-267-

Although the theories elaboratedthus far deal with why the problemsexist in District 9, further elabo-ration of PREP's action theories,i.e., the project's theories whichwill guide their interventions, is

needed for the third year.

Objectives

The project objectives, gleanedfrom several sources, are summarizedin Table 1. Drawing from theupdated theoretical framework, thefollowing four objectives were addedto last year's: (a) reducing nega-tive peer influence among programparticipants when compared to compa-rable controls; (b) increasingopportunities for a successful tran-sition to high school and/or acareer; (c) increasing the use of

fair and consistent school rules forprogram participants; (d) increasingstudents' beliefs in conventionalrules. This year's "objectives"include processes or critical bench-marks necessary for implementation.Were the project's theoreticalframework sharpened, a clearer dis-tinction could be made betweenobjectives and critical benchmarksfor program implementation.

Intended Interventions

In addition to the three majorcomponents in the first year (i.e.,the Alternative School, the Citizen-ship Clusters, and the Youth compo-nent) a fourth intervention, studentcouncil, was added for the secondyear of operation. The studentcouncil is composed of eight electedrepresentatives from the AlternativeSchool and one representative fromeach of the eight cluster schools.

140

PREP

The Youth Director is primarilyresponsible for the student councilat the Alternative School and the

two teacher-coordinators are respon-sible for the student representa-tives at the Cluster schools. Thecouncil convenes at least monthly tolearn about decision-making proce-dures and developing an agenda forinitiating school policy changes.After a meeting at the AlternativeSchool, the Cluster representativemeets with the rest of the Clusterstudents at his or her home school.This intervention is directed at thefollowing objectives: (a) reducingnegative peer influence; (b) in-creasing the number of active, pro-ductive youths, (c) increasingself-concept, and d) in-creas-ingattachment to school.

Project Environment

A major change has taken place inSchool District 9, the implementingorganization. During the first yearDistrict 9's Deputy Superintendent(DS) acted as project manager. Fol-lowing the DS's resignation, the

administrative responsibility for

Project PREP was shifted to the Dis-trict's funded programs office, andthe Community School Board hasbecome more involved with monitoringthe project.

Staffing

The District 9 Director of Evalu-ation and Research and his assistantare no longer involved in performingevaluation activities. The Commu-nity School Board has assigned a

data collector to assist the projectdirector with major evaluationactivities such as the spring surveyadministration. Although the datacollector supervised the paraprofes-sionals who actually collected thedata, he did not adopt the role ofan on-site evaluation coordinator.Many of the on-going evaluationtasks were performed by the project

director, a teacher-coordinator andthe paraprofessionals.

In addition to the data collec-tor, a new Youth Director was hiredfor the second and third year to

coordinate the Youth Program and thestudent council.

Program Implementation

Project PREP has been servingstudents for two years; however, forthe first half year, the project wasrun by a director with a differentphilosophy than the present direc-tor. Although the present director

,,has been on staff since April 1981,it has only been recently, duringthis past year, that the project ismoving in the direction that he hasconceptualized. An example of thisis the development of curriculum toprepare students for options otherthan college. Pianos and other

_musical instruments were= acquired,as well as equipment for a photogra-pl,v class and an auto mechanics

-268-

The extent and nature of thisyear's implementation was ascer-tained primarily from projectrecords, the program developmentworksheets, supplemented with inter-views with the project's staff. No

formal comprehensive monitoring dev-ices were developed to record otherimportant dimensions of implementa-tion such as the degree and extentto which project staff have adoptedalternative education techniques(for example, see Hall & Loucks,1977; Leithwood & Montgomery, 1980).

Alternative School

Most of the project developmentoccurred with this intervention.The participants are still seventhand eighth graders referred fromfour local junior high schools forbehavioral problems. However, the

emphasis of the intervention nowincludes providing practical experi-ences as well as academics. Boththe career awareness element and thetransition to high school componentswere specified in the original pro-posal, but were not fully conceptu-alized or implemented. The careerawareness element originallyinvolved exposing students to guestspeakers of different professions.As implemented in year two, thiscomponent allows students to getexperience at the alternative schoolin various activities such as pho-tography, auto mechanics, carpentry,and electronics. In addition, by

the end of the year, students wereexposed to job resumes, participatedin mock job interviews, and, by thesummer, 25 students were reportedlyplaced in jobs around the communityfor minimum pay. In addition to

providing students with vocationalexperiences, the program providesstudents with practical experiencessuch as exposure to museums, restau-rants, ball-games, and taking a

train trip to other nearby cities.

English, math, social studies,and science are taught in the morn-ings. Teachers work with eight stu-dents per class for a one-hour per-iod. Individualized educationalplans, based on test scores, wereinitiated for students in December,1981, and were updated the followingApril for the remainder of the year.Students experiencing academic dif-ficulties are also provided withtutoring after school. For thoseexperiencing emotional difficulties,psychological and drug counselingare also available from the staff

psychologist and the local mentalhealth clinic.

Although an alternative setting,the school is structured as a juniorhigh or intermediate school where

PREP

students are prepared academicallyfor high school. The project direc-tor was concerned, however, thatProject PREP students would becomeso acclimated to the supportivealternative school environment thatthey would be unable to make the

transition to high school once theygraduate from the eighth grade. As

a consequence, arrangements weremade with local high school princi-pals to allow eighth grade PREP stu-dents to attend classes in the morn-ing. Five eighth graders wereplaced in high school last February.They took courses during the morningthen came back to the AlternativeSchool in the afternoons for tutor-

," .

ing and counseling, in addition toparticipating in the practical cur-riculum.

-269-

Both the academic and practicalcomponents of the Alternative Schoolare designed to increase academicachievement, self-esteem, attachmentto school, student involvement, andto decrease negative peer influenceamong the targeted students as com-pared to comparable control stu-dents. In addition to the academicand practical components, a thirdcomponent, informal counseling,designed to increase students'beliefs in conventional beliefs, is

primarily performed by the projectdirector during field trips or otheractivities whenever and wherever atopic comes up. Students also callthe youth director and teachersafter hours for help. In order toprovide consistent availability of

staff to students, arrangements arebeing made to provide a 24-hour"hot-line" to students. The projectdirector feels that the exposure todifferent experiences and a suppor-tive caring staff are the mostimportant aspects of the alternativeschool.

142

PREP

Citizenship Clusters

While the project director is incharge of the Alternative School,the two teacher-coordinators areprimarily responsible for the Citiz-enship Clusters, i.e., the teachersof project students, the paraprofes-sionals who operate the Time-OutRoom, and the students themselves.The teacher-coordinators arrangedweekly meetings with teachers andparaprofessionals, coordinated stu-dent council activities, providedthe prizes for the token economy andwere in charge of evaluation issuessuch as maintaining the design andsupervising data collection activi-ties.

Unlike last year, the token econ-omy went smoothly due to the readyavailability of the prizes. How-ever, the function of the Time-OutRoom changed from the behavioralmodel of a nonstimulating, "cooldown" environment for a misbehavingstudent to a microcosm of the "per-fect classroom." Once behavioralcontracts were established with pro-ject students, tne Time-Out Room wasutilized less. Some principals feltthat the paraprofessionals were notusing their time \efficiently andconsequently suu,sted that the par-aprofessionals oiler tutoring to

project students. The Time-Out Roomeventually became E supportive envi-ronment where ,Ludents receivedindivicivalized ' itorial and counsel-ing servic s .nn the paraprofes-sionals, as well as a place to "cooldown." It appears from subjectiveaccounts that the Time-Out Room metthe needs of students who neededpersonal attention, both academi-cally and emotionally, but who couldnot get this from a regular class-room.

One paraprofessional from a jun-ior high school describes her roleas first talking to a student who issent to the Time-Out Room' for

-270-

misbehaving. She ascertains why thestudent misbehaved and allows the

student to vent his or her emotions.She then explains why that behaviorwas inappropriate and draws up a

behavioral contract agreed upon bythe student, teacher, and parapro-fessional. If the student fulfillsthe contract, he or she receivespoints which are later redeemed forprizes. When there are no studentsin the Time-Out Room, the parapro-fessional goes to a Cluster class toget four students for'reading andmath tutoring. Each student is to

be seen by the paraprofessionaltwice each week for rap sessions or

,counseling. Whether visiting theTime-Out Room for tutoring or a

behavioral problem, students are inthe Time -Cut Room no longer than 45minutes before returning to class.

Although all paraprofessionals inthe eight cluster schools do essen-tially the same tasks, the intensityvaries across schools, as can be

seen in Table 4.' For behavioralproblems, students at one school hadas few as five contacts whereas atanother school students had an aver-age of 87 contacts. The averagenumber of rap sessions each studenthad ranged from two at one school toas many as 160 at another school.Part of this variation may depend onthe relationship between the para-professional and teacher. Someteachers initially felt threatenedby the presence of a paraprofession-al providing direct services to stu-dents. This was especially true ifstudents became attached to theparaprofessionals. Other teachersexcessively use the Time-Out Room tounburden themselves from certainstudents. Meetings with the teach-ers, paraprofessionals, and the

staff psychologist were held in anattempt to resolve these problems;but, attendance at these meetingswas minimal.

143

In summary, the "Time-Out-Room"is a misnomer for the model actuallybeing implemented in the ClusterSchools. Although it appears thatthe "Time-Out-Room" has moved awayfrom the behavioral model, the ori-ginal conceptualization of thisintervention, as outlined in theproject's proposal, includes indivi-dualized tutoring and counselingservices. Unclear role expectationsbetween teachers and paraprofession-als and haphazard referral of stu-dents at some schools threaten thestrength and integrity of thisintervention. Overall, the strengthof this intervention, according to

the project director, depends on therelationship between the student andthe paraprofessional.

Youth Component

This intervention was originallydesigned to bring Alternative Schooland Cluster students together to

engage in positive leisure timeactivities. In year two, however,the Alternative students and theCluster students became more inde-pendent of each other. Alternativestudents plan their own field tripsseparate from Cluster students andutilize separate night centers.There are 16 night centers in theDistrict sponsored by ContinuingEducation which are open from 7-9p.m. Only one of these centers isrun by the Alternative School Staff.Alternative students attend thiscenter while Cluster students attendother nearby centers staffed by someof the para-professionals who takeattendance. However, token pointsare no longer given to students whoattend the night center. Althoughstudents are not required to attendthe night centers, field trips, or

other activities, they are stronglyencouraged. Of all the activitiesof the Youth component, the nightcenters are utilized most often.Other components include wintercamp, summer camp and summer jobs.

-271-

PREP

Student Council

This was the first year thisintervention was implemented. Alt-hough council members were intro-duced to parliamentary procedures,took field trips to governing agen-cies, and addressed school and com-munity problems, the project direc-tor felt the council was not as

strong as anticipated. This was duein part to the fact that the councilonly met once a month and councildecisions were often temporary.

Parent Involvement

This has been the most difficultcomponent to implement. Ministers,technical assistance, and consul-tants have been utilized this yearin order to organize parents. Thevarious strategies were successfulin involving some parents.

Implementation Overview

Project PREP'S interventions havebeen implemented in varying degrees.Tables 3 through 4 provide an over-view of project implementation.Table 3 shows the average number oftimes alternative school studentswere reported to be involved in var-ious activities, and Table 4 pre-sents similar information for thecluster schools. Implementation isvastly different in differentschools.

Interim Outcomes

Methods

Design. With the first year'snon-equivalent control group design,little could be concluded about theeffectiveness of Project PREP'sinterventions. Consequently, anexperimental design was establishedin both the Alternative School andthe Cluster schools. A pool of eli-gible referrals from the four feeder

PREP

junior high schools were randomlyassigned to either the Alternativeschool or a control condition in

which the students would remain inhis or her original school. A simi-lar procedure was conducted for theCluster Schools. Cluster treatmentstudents were designated by the PREPstaff to utilize the Time-Out-Room.Cluster controls were to be treatedno differently from non-program par-ticipants.

Due to the mobility of studentsin the Bronx, it was necessary to

maintain a continuous pool of refer-rals. Because some treatment stu-dents on the waiting list subse-quently withdrew from school, it

became necessary to develop a treat-ment waiting list and a controlwaiting list. Each time a personfrom the treatment waiting list wasassigned to treatment, a person fromthe control waiting list was alsoadded to the control group. Post-randomization checks using the SAESpretest showed no sizable differ-ences between treatment and controlstudents. Larger pools of eligibleswere developed in some schools thanin others, resulting in some non-proportionality in the design.

The experimental design was ini-tially met with some resistance byprincipals who wanted to place cer-tain students in the AlternativeSchool and other principals whowanted all students served by theTime-Out Rooms. Over time, princi-pals accepted the randomization pro-cess; however, there were a fewoccasions when the project directoraccepted nonrandomized referralsinto the Alternative School. Thesestudents are excluded from our ana-lyses. Although control studentsreportedly do not receive any PREPservices, it is possible that theyreceive other services provided bythe District. For example, controlstudents can attend any of the nightcenters or they may be involved in

-272-

the Gates' program which offersremedial reading. In addition, oneCluster school has a drug programfor students and Special Educationhas resource rooms set up at someschools to deal with academic prob-lems. While Project PREP is uniquein that it offers both academic andbehavioral services, there are othersources from which controls canreceive help.

Subjects. Of the treatment andcontrol students, 70% were black and30% hispanic. Females made up 33%of the sample while 67% of the sam-ple were males. Fifteen students

I Ireturned to the Alternative Schoolfrom last year, leaving 25 openingsto be filled by students randomlyselected from a referral pool. Dur-ing the year, the project directoraccepted into the Alternative Schoolthree non-random referrals fromprincipals and one from the proba-tion department. Each ClusterSchool served 15 students, with theexception of School 148, whichexpanded its program later in theschool year to 30 students.

Measures. We intended to relyprimarily upon the SAES question-naire for measures of Project PREP'sgoals and objectives. Officialschool records and the Project'sparent questionnaire were also to beused. The project administered theparent questionnaire twice duringthe year to parents of PREP partici-pants.

Data quality. The utility of thedata available is severely limiteddue to extensive missing informa-tion. Required information was notprovided in sufficiently completeform to make most analyses meaning-ful. For completeness, tables showthe actual number of cases for whomdata are available. In Table 5 weillustrate the missing data problem.For the Alternative School, there

145

were 21 control and 31 treatmentstudents. For the Cluster schools,there were 96 control and 162 treatment students. Complete data onwithdrawal from school were available. Data on absences from schoolare also reasonably complete. Otherdata are so often missing that noconfidence may be had in analyses ofthe data that are present./ (Table 5is illustrative only. Other questionnairebased measures resemble themeasures shown in Table 5 in theircompleteness.)

Results

Project PREP's goals. Table 6

shows treatment and control groupcomparisons on several measures ofProject PREP's goals. Four measureswere intended to assess truancy anddropout. As noted earlier, however,the only sufficiently complete datafor meaningful analyses are forabsences (based on school records)and withdrawal from school. Students at the Alternative Schoolwithdrew from school more often thancontrols, but this result could haveoccurred by chance. Treatment students in the Cluster schools withdrew from school significantly (p <.02) more often than did controlstudents. This difference is

unlikely to have occurred by chance.

Alternative School treatment students were absent slightly and nonsignificantly less often than control students. Among cluster schoolstudents for whom data werereported, attendance was better (.10< p < .11) for treatment than control students. Attendance data werereported, however, only for studentswho did not withdraw from school.Recall that significantly moretreatment (13%) than control Students (4%) withdrew from school.'

1The randomization process for

273

PREP

The results shown for class cutting and school cutting in Table 6,although showing significant differences, are not very meaningful dueto the low response rates for thesequestionnairebased measures.

Project PREP's Objectives. Theremainder of the results shown inTable 6 relate to Project PREPobjectives. Due to extensive missing data, these results are undependable. Nevertheless, the resultssuggest that victimization may beslightly higher among Alternativeschool students than among controls(p < .05), and that negative peerinfluence may be higher in bothAlternative School and clustertreatment groups than in the controlgroups (p < .10).

A number of Project PREP objectives are specific to the Alternative School. Results relating to

these objectives are summarized inTable 7 (for objectives relating to

cluster schools was performed separately for each school. Because ofdifferences in the 'size of the poolof eligibles, different proportionsof students were selected for treatment in the eight schools. Thisimplies that school is 'associatedwith probability of treatment, andthat differences observed betweenthe pooled treatment and controlgroups could be due to school differences as well as to treatmentdifferences. Data are not analyzedseparately by school due to the verylow statistical power that wouldresult from the small sample sizesinvolved. When significant treatmentcontrol differences were found,statistical controls for school wereintroduced to determine whetherschool differences could account forthe result. Such additional analyses were performed for withdrawalfrom school and days absent. (These

146

PREP

students). As was true for mostmeasures summarized in Table 6,

missing data makes interpretation ofthese results difficult. No significant differences were found between treatment and control groupshowever.

One way to assess the objectivesfor the Alternative School itself(those relating to fair and consistent rules and to teacher competency) is to compare the school climate measures bearing on theseobjectives to those for otherschools and to the AlternativeSchool's standing on these measuresin the previous year. Chapter 5 andthe Appendix of Part I of thisvolume show results of the climatesurveys conducted in the spring of1981 and 1982. An examination ofthese results suggests the following: (a) Students report that therules are fairer than do students inmost other schools, and reports offairness increased somewhat in 1982over 1981. (b) In both years students reported that the school ruleswere clearer than did students in

other schools. (c) In 1981 studentsreported fairly typical levels of

victimization and safety; in 1982students reported significantly moresafety than in 1981. (d) In bothyears students reported high levelsof individualized instruction.(e) Students reported experiencingslightly less disrespect from teachers than did students in the typicalschool in 1981, and they reported

analyses were not performed forquestionnaire measures where missingdata were severe limitations.)Large and significant differences inwithdrawal across schools are present in the data. Even when controls for school are applied, the

observed difference in withdrawal ispresent and statistically significant (p< .025).

274

less still in 1982. (f) More studentteacher interaction than in thetypical school was reported bothyears, but students reported lessinteraction in 1982 than in 1981.(g) In 1981 students reportedreceiving about the same levels ofpunishment in school as did studentsin the typical school, but slightlymore than average in 1982. (h) Inboth years students reported receiving many more rewards in school thandid students in the typical school.

The, results of the parent questionnaire are shown in Table 8. Although more parents who responded to

,,the questionnaire reported visitingthe school and being aware of various opportunities in the communityin the second half of the sch:olyear, the sample may be biased dueto the low response rate (32 to44%). In addition, factors whichare not controlled for other thanProject PREP's interventions, mayhave produced the observed differences.

Discussion

Limitations

Problems of missing data ravagedthe utility of the evaluation.Satisfactorily complete data wereavailable only for two outcome measures: School absences and withdrawal from school. Failure to administer the questionnaire to treatmentand control ,students, and poor completion rates among students whowere surveyed'imade the questionnairemeasures only of very limited usefulness. The projectdevelopedandadministered parent questionnairesuffered from similar low responserates. Finally, information on disciplinary referrals was collected insuch a way that made it impossibleto distinguish between students withno referrals and students for whomno data were collected, and weretherefore unusable. Accordingly, we

14.7

regard only the absence and withdra-wal results as dependable, and focusour discussion on those results.

Withdrawal from School

A larger proportion of studentsparticipating in the AlternativeSchool than of control group stu-dents withdrew from school (19% vs.5%, respectively). This result,although not statistically signifi-cant, is worrisome. There is noevidence that participation in theAlternative School reduces the riskof dropout.

A larger proportion of treatmentstudents in the cluster schools thanof control students withdrew fromschool (13% vs. 4%) and this differ-ence is statistically significantwhen statistical controls for schoolof attendance are applied (p <

.025). Evidence suggests that par-ticipation in the Citizenship Clus-ters increases the risk of dropout.

Absence:;

Students attending the Alterna-tive School are absent about as

often as control students. There isno evidence that participation_ inthe Alternative School reducestruancy.

Treatment students in the clusterschools who have not withdrawn bythe end of the school year areabsent less than are control stu-dents, but this difference is notstatistically significant. There isno evidence that participation in

the Citizenship Cluster reducestruancy.

Evaluation Strategies for 1982-83

The Program Development Evalua-tion model has never been adopted byProject PREP. Clearer specificationof the project's action theory andmore careful attention to planning

-275-

PREP

and management would be useful.

The PDE managment plan will needto be specified more carefully nextyear in order to carry out data col-lection activities. PREP showedminimal improvement over the firstyear with respect to data collec-tion; however, and serious problemspersist. Identification of obsta-cles in this area can be fed intorenewed planning to strengthen theevaluation.

The process of identifying obsta-cles and formulating strategies toovercome the obstacles worked wellwith implementing randomization thisyear. Randomization will again beimplemented next year in the Alter-native and Cluster schools. Some ofthe Cluster schools will be expand-ing the number of students theyserve from 15 to 30, which willrequire maintaining a large pool ofreferrals. Due to the high mobilityrate of students, referral poolsmust be replenished and maintainedthroughout the school year. Thiseffort will be of limited use unlessthe completeness of data collectionimproves.

Problems Project PREP is Encounter-ing

Implementation. The projectdirector is skeptical about the use-fulness of the Time-Out-Room as anintervention. He feels the themesand prizes may be useful for elemen-tary school students but that tripsand various practical experiencesmay be more valuable with the olderstudents. As a consequence, theAlternative School and the Clustersare separate entities, with the pro-ject director responsible for the

Alternative School and the teachercoordinators responsible for theClusters. The project directordevotes most of his attention to theAlternative School.

148

PREP

Perhaps a more difficult task forPREP is getting parents involved intheir children's activities. It maybe, however, that the project is

dealing with an overwhelming obsta-cle in their forcefield--the commu-nity environment. Parents in thecommunity are often more concernedabout larger issues such as unem-ployment, poor public services, andhigh crime, rather than their chil-dren's activities. The communityenvironment may also make it diffi-cult for the project to reduce del-inquent behavior in the schools andamong PREP students if the surround-ing environment supports this behav-ior.

Evaluation issues. The projectdid not devote sufficient personnelresources and attention to collectthe data completely, despite the

hiring of an on-site data coordina-tor. Part of the problem also is

the quality of records kept at someof the schools. The way question-naire administration has been plan-ned for and executed have been seri-ous defects which must be remedied

if the effects of the project are tobe determined.

Successes Project PREP is Experienc-ing

Although the objective evidencedoes not depict Project PREP as suc-cessful, subjective evaluations of

the project are more promising. Theproject is praised in the District;other schools in the district wantto be served by PREP. After a visitto the Alternative school, one staffpsychologist was so impressed withthe students, she sought air time ona radio broadcast so students could

1,tell about their experiences.

Teachers and principals in PREPschools were initially skeptical,but now they want the programexpanded. Some students come to

school early, and do not want to

leave after school is out. Sane 8th_ _ _Some

graders do not want to graduate.Non-PREP students at the ClusterSchools have asked how they can joinProject PREP and participate in theactivities and in student council.

-276-

1.49

PREP

Aeforepeeg

Moll, tit, 4 Look*, S.?. A developmental model for determining whether thetsvommeti it attuelly implvmented. kgagrigAnikiggjaignALiumg),414404 1. I. :63.274.

Wit,wpo44 g.A. 4 Montgomtry4 D. S. tvaluating program implementation. IYA1-1044AM617410+ 1980. 2, 193-214.

Nowa, D.K. Project PRSIPt An interim report of its evaluation. In C.D. Cott-emboli (111.) nig > ti v 1122

*etiolate! Johns Sophia§ University, Center for the Study of the Social6visotsation of Schools, (Report No 325), 1982.

!0

Table 1

Project PREP's Goals and Associated Objectives

Goal I: Reduce truancy and dropout in the 12 participating schools.

Objectives:

1. Increase parental involvement and enhance the extent towhich they value education.

2. Increase the number of active and involved students.3. Increase academic achievement and success.4. Increase student commitment/attachment to school.5. Increase chances for a successful transition to high school

and career.6. Increase student self-concept.7. Reduce negative peer influence,8. Increase teacher competencies so thy can cope with problem

behaviors and meet students' needs.9. Increase students belief in conventional rules.

Goal II: Reduce the amount of disruptive behavior in the 12 schools.

Objectives:

1. See objectives 1 - 9 for Goal I, above.2. Increase :.he use of consistent and fair guidelines for

behavior.

Goal III: Reduce the number of suspensions in the 12 schools.

Objectives:

1. See objectives for other two goals (objectives are thesame).

aThis objective is specific to the Alternative School.

-278-

151

Table 2Average Number of Times

Time-out Room Utilized During the 1981-1982School Year for Each Cluster School

School Behavioral Rap Sessions,Problems Counseling

M SD N M SD

55 21.9 25.6,,

17 3.2 3.2 17

63 44.7 7.7 16 89.3 7.7 15

64 86.6 17.8 14 160.5 18.6 13

82 4.9 6.9 22 5.4 14.5 21

117 41.9 14.7 16 56.9 30.0 16

132 13.8 12.1 19 2.3 2.4 19

147 30.1 33.4 18 94.5 68.3 18

148 10.4 9.7 39 8.0 7.8 39

For all schools 26.62 28.4 161 41.5 26.4 158

Table 3

Average Number of Times Alternative School StudentsParticipated in PREP Activities During 1981-82 School Year

Activity N SD

Evening Recreation 28.0 21.7Tutoring 11.1 8.6Student Council 2.1 4.0Field Trips 17.3 11.9Drug Counseling 8.0 8.1Volunteer Work 10.3 19.3Parent Visits 4.1 .6Practical Curriculum 40.2 26.3Winter Camp' 0 0

Note. Non-randomized returning students are included;N = 46.

Table 4

Average Number of Times Cluster Students Participated in PREP ActivitiesDuring the 1981-82 School Year

SchoolEvening Recreation

M SD NField Trips

M SD NStudent CouncilN SD N M

TutoringSD /

55 0.0 0.0 17 3.1 1.2 18 0.1 0.2 18 0.0 0.0 :

63 53.9 12.5 15 1.4 2.0 17 0.6 2.4 18 24.3 7.7 :

64 82.1 20.0 14 1.0 0.0 14 0.0 0.0 14 102.1 30.2 :

82 1.4 3.6 22 0.2 0.4 22 0.1 0.6 22 0.3 1.1 :

117 a a a 1.0 0.0 16 0.1 0.2 17 49.9 16.1 :

132 0.5 0.7 2 2.0 0.0 .18 0.2 0.7 19 0.8 0.5147 46.3 27.5 18 0.9 0.3 18 4.4 2.5 18 57.9 45.2148 0.6 0.5 40 0.5 0.8 40 0.1 0.3 40 0.7 0.2 A

aInformation not available.

-280-

153

Table 5

Illustrative Percentages of Data Available

Measure

GPA

Withdrawal

Absences

Negative peer influence

Attachment to school

Interpersonal competency

Selfconcept

Delinquent behavior

Alternative School Cluster SchoolsTreatment Control

61 48

100 100

90 90

54 38

48 4 28

39 19

39 19

45 19

Treatment Control

76 71

100 100

84 78

55 53

41 41

31 29

25 26

48 44'

281--

154

Table 6

Treatment-Control Group Comparisons for Project PREP:

Measure

Truancy and dropout

Overall Project Coals and Objectives

Alternative School Cluster Schools

Treatment Control Treatment Control

SD N M SD N M SD N M SD

withdravalsa .19 .40 31 .05 .22 21 .13** .34 162 .04 .20 96

Days absents 9.91 13.15 28 11.53 14.98 19 18.40 26.97 137 25.48 34.50 75

Class cutting .72 1.41 18 1.11 1.05 9 .81* 1.29 107 .45 1.04 60

School cutting .44* .92 18 1.33 1.66 9 .50 .99 105 .38 .94 60

Disruptive behavior

Delinquent behavior .26 .18 14 .26 .15 4 .17 .21 78 .14 .16 42

Victimization .25** .27 15 .02 .05' 7 .22 .24% 64 .21 .25 39

Suspensions

Project records -- -- -- 2.40 5.22 15 3.12 7.36 8

Self-reported .47 .52 15 .50 .55 6 .38 , .49 60 .30 .46 37

Other objectives

Parental emph. educ. .54 .30 13 .42 .34 6 .63 .29 54 .64 .24 32

Negative peer infl. .43* .22 17 .25 .25' 8 .28* .21 89 .22 ,14 51

Involvement .24 .18 17 .23 .23 9 .25 .19 87 .24 .21 51

Self-concept .67 .15 12 .69 .22 4 .66 .16 40 .67 .18 25

Interpersonal competency .63 .30 12 .85 .19 4 .76 .20 50 .70 .26 28

Attachment to school .73 .20 15 .75 .18 6 .55 .28 67 .62 .26 39

GPAa 66.08 16.08 19 56.58 21.96 10 65.93 14.11 124 72.59 43.95 68

bGrades 2.44 1.15 18 2.00 .92 8 2.39 1.07 102 2.43 .84 58

Reading abilityb 1.69 .79 16 1.75 1.04 8 1.41 .78 94 1.50 .91 54

a Based on school or project records.

b Based on SAES student questionnaire.

* p < .10** p < .05

Table 7

Treatment-Control Group Comparisons for Project PREP:Objectives Specific to the Alternative School

MeasureAlternative School Control

SD N M SD

Increase successful transi-tion to high school orcareer

Worked for pay last week .59 .51 17

Hold full /part -time job .18 .53 17

Educational expectation 3.53 1.50 17

Practical knowledge 1.21 s'.34 11

Increase belief inconventional rules

Belief scale .59 .26 11

.50 .53 8

.25 .46 17

3.33 1.87 9

1.34 .48 5

.70 .22 5

Note. No significant differences between treatment and control group werefound on any measures summarized in this table.

156

-283-

.

Table 8

Responses on Project PREP'S Parent

Questionnaire Administered in Winter 1981 &

Spring 1982 for Alternative & Cluster Schools

Item

Winter

% responding

favorably

Alternative School Cluster School

1981

a

Spring 1982

% responding

favorablyb

Winter 1981 Spring 1982

% responding % responding

favorablyc favorablyd

Attended one or more parent association meetings 24% 67% 44% 72%

Aware of reading and math clinic 53 100 59 82

Aware of rec. centers 47 100 29 78

1

.L,

Aware of night centers 29 100 21 58

I

Aware of community agencies 18 33; 15 53

Visited school one or more times 65 67 71 88

Helped child with homework once/week or more 47 67 76 62

0,

School meets child's needs 76 100 78 82

Positive change in child's behavior 71 100 62 88

Positive change in child's attitude 65 100 53 90

Positive change in child's school atten. 71 100 60 82

Wants child to continue in PREP 76 100 75 93

Response rate: 39% 32% 44% 39%

157

Note: Respondents were parents who had children participating in Project PREP.

aN=17.

bN=12.

cN=68.

dN=60.

158

The Jazzmobile Alternative Arts Education Project

D. E. Rickert, Jr.

The Jazzmobile Alternative ArtsEducation Project's aim was to utilize the arts as a medium throughwhich juveniles can constructivelychannel their energies. The projectwas intended for juveniles in sixth,seventh, and eighth grades whoshowed disruptive behavior, who werechronic absentees and truants, or

who experienced academic failure.

Problems, Goals and TheoreticalRationale

Problems

The project's grant proposal

cited community crime and delinquency, as well as high rates of

school suspensions, truancy and

. dropping out as problems on whichthe project would focus. In its

proposed work plans the followingproblems were to be addressed:

1. Youth hanging out on the

streets and being involvedwith what project staff called"street activity." Studentswho hang out on the streetsrather than go to school wereseen as more likely to getinto trouble with the law.

2. Youth not having the opportunity to live up to their fullpotential to be successful in

life.

3. Lack of parental awareness oftheir children's schoolrelated activities and lack ofinvolvement in these activities.

4. Truancy and chronic absenteeism.

Goals

The project's goals were, therefore, to reduce truancy and chronicabsenteeism, reduce youth involvement in "street activity," increaseparental awareness and involvement,and increase positive utilization ofstudents' potential.. Table 1 summarizes these goals and indicateshow the first goal was measured.Measures of the other goals were notidentif ied.

4Theory and Objectives

285

The theory of action in the project's Program Development Evaluation (PDE) worksheets is less farreaching than that elaborated in theoriginal grant proposal and in earlymeetings between project and evaluation staff. The theory appearing inthe Program Development worksheetsappears to have been a compromiseresulting from staff discussion subsequent to the first meeting betweenproject staff and evaluation staff.Several staff members mentioned thatsuch revision took place becausesome of the wording of the originaltheory sounded belligerent. Forinstance, the theory originally contained a statement to the effectthat the school systems weredesigned for middleclass whiteyouths and that the potential of

minority children is not being realized because parents do not use

their power to change the school

system. Key staff made it clear onseveral occasions that the "official" action theory (i.e., the oneelaborated in the PDE plan) was nottheirs nor were they free to reviseor elaborate on it.

The grant proposal described a

comprehensive ecological theoretical

159

Jazzmobile

pespective. It stated that economicimpoverishment impinges on the families of many youths in Central Harlem. Unemployment, poor housing,community crime, health problemsassociated with poverty, and poorschools contribute to school failure. Dropout and subsequent unemployment can then introduce a hostof other problems--namely crime,drug abuse, and alcoholism.

Figure 1 illustrates my interpretation of the relation between(a) the theory described in the

grant proposal, (b) the theory as

laid out in the project's PDE worksheets, and (c) ideas discussedinformally by project managers andstaff. Elements enclosed in boxesare those included in the PDE worksheet. The central element is untapped youth potential. As Figure 1shows, youth potential may not be

realized for various reasons. But

if it is not, students become boredwith traditional schooling. Thisleads to school failure, which leadsto frustration. Frustrated studentsare more likely to be chronicallyabsent from school and hang out onthe streets. This leads to delinquency.

Figure 2 is an abbreviation ofFigure 1 that illustrates the goalsand objectives of the project as

laid out in the PDE worksheet. Theelements enclosed in boxes were considered goals by the project. That

is, they were regarded as valuablein their own right, and are not simply instrumental objectives.

Objectives

Table -2 shows the ojectives(i.e., intermediary outcomes) identified by project personnel and thecorresponding measures. As was thecase with goals, many of the project's objectives were not measureddespite some initial plans to provide these measures.

286

Interventions Intended

The art classes. The primaryintervention planned was the Jazzmobile Art classes. Most students inthe program were to attend dailyclasses in either drama, music,

dance or visual arts during the

regular school day. In the

intermediate school, six artistinstructors were to teach four classesdaily. An After School/Summer program was planned to serve 60

intermediate school students, who,

for one reason or another, could notattend classes during the regularschool day. The design of the Jazzmobile Alternative Education Artsclasses was consistent with Jazzmobile's theory of delinquency prevention: Attachment between studentsand artistinstructors would be

fostered by small class sizes (e.g.less than 10 students per class) andwarm, caring artistinstructors who

ecould relate to the students, specially in terms of cultural experiences. The artistinstructors wereto serve as conventional role modelsto the students.

The artistinstructors were to

make an effort to provide new experiences daily for students and

encourage creative expression to

keep students excited and stimu--

lated. Black cultural history wasto be incorporated into classes asseemed appropriate to the individualartistinstructors. Performancesand shows were seen by Jazzmobilestaff as alternative rewards (Li

grades) to help make 'school a moredesirable place for-students. Otheralternative rewards planned wereJazzmobile teeshirts, certificatesof participation, and artrelatedfield trips.

Perhaps most importantly,artistinstructors were to increasethe extent to which students valueeducation by teaching disciplinethrough the arts. Jazzmobile staff

160

maintain that art requires a discip-lined process that is transferableto academic matters.

According to the original planartist-instructors were to enhancethe school staff's approaches to

teaching. Conversations with Jazz-mobile staff members imply thatintuition was to play a large partin this enhancement effort and itsassessment.

Curriculum Guides

Jazzmobile intended to develop acurriculum guide for each of thefour areas for which there was a

Jazzmobile class: drama, music,dance, and visual arts. Theseguides were intended to assist theJazzmobile artist-instructors in

providing an artistic experience forstudents within the traditional aca-demic learning process. Specifi-cally, the guides used the specificart form which each guide deals withas a vehicle for teaching selectedcontent area, skills in languagearts, social studies, science andmath.

It is not clear what objectivesin the Program Development Plan thecurriculum guides sought to achieve,but presumably these guides would,through a "trickle- down" effect,raise students' self-esteem andinterest in academic subjects.

Counseling

A Jazzmobile counselor wasassigned to the intermediate school.This counselor's mission was to

counsel students regarding careersin the arts and to intervene whenbehavior problems arose. Thecounselor was also to act as a

resource consultant and take stu-

dents on field trips. The objec-tives of these activities were to

increase youth's career awareness,

-287-

Jazzmobile

primarily in the area of art,

secondarily in other areas, and toreduce youths' sense of hopeless-ness.

Parent Component

The parent component was intendedto promote increased attachment bet-ween parents and their childrenthrough their involvement in educa-tional activities. Home visits by aField Worker, and parent workshopsgiven by the staffs of 'the project,the school, and the police precinctwere to be included in this compo-nent.

Project Environment

Implementing Organization

Jazzmobile began in 1965 as a

non-profit organization giving freemusical performances, principally ofjazz.

Dr. William Taylor, better knownas "Billy" Taylor, the jazz pianist,was a leader in the formation of theHarlem Cultural Council, whichobtained an initial $10,000 grantfrom Ballantine Beer to put on freeconcerts in Harlem. Jazzmobile wasincorporated in 1966. Over theyears, Jazzmobile has becomeincreasingly involved in music andcultural education. The other finearts (drama, dance, and visual arts)are now included under Jazzmobile'spurview.

The overarching mission of Jazz-mobile is the preservation and pro-pagation of Jazz. This involvespresenting free musical performancesto.the community, particularly areasdeemed to be culturally deprived.

Secondary aims are the following:(a) to stimulate residents, espe-cially youth, into a more activeparticipation in musical perfor-

Jazzmobile

mances; (b) to demonstrate to

students the importance of communi-cation and cooperation through thearts; (c) to bring students into

personal contact with artists theycan relate to and emulate; and(d) to provide for students a

"built-in success factor," wherebythrough. the arts youngsters canexperience success, develop theirtalents, channel their energies,acquire self-discipline, developgood study habits, and developrespect for their school as a vitalpart of their community.

Internal Organization

The Jazzmobile Alternative Educa-tion Arts Program was one of severalactivities with which Jazzmobile,Inc. was involved, but during itsoperation was a substantial sourceof income for the umbrella agency.

Figure 3 describes the organiza-tional structure of, Jazzmobile.Presiding over all Jazzmobile, Inc.affairs is a volunteer Board of

Directors. An Executive Director(ED) has sole responsibility for theoperation and fiscal management ofall Jazzmobile activities, includingthe Alternative Education Arts Pro-gram. The ED consulted with anadvisory committee for the Alterna-tive Education Arts Program, whichheld monthly meetings.

Jazzmobile's Development Officer(DO) is directly under the ExecutiveDirector (ED) in the organizationalstructure. The DO authored the

alternative education project'sgrant proposal, and served under thethe ED as "Education Specialist" onthe project. The latter involvedconsulting as needed with the ED on"educational matters."

Also officially directly under

the ED were the Project Director(PD) and Evaluation Co ;rdinator

(EC). Informally, they appear to

-288-

have received ongoing supervision bythe DO. In practice, there appearsto have been some ambiguity abouthow much authority the ProjectDirector and Evaluation Coordinatorhad to make decisions without the

ED's approval. This arrangement mayhave caused difficulty in executingevaluation-related activitiesbecause of ambiguity about appropri-ate channels for organizing the

work. The official organizationchart shows no direct link betweenthe PD and the EC.

Directly under the Project Direc-tor were the project's GuidanceCounselor, Curriculum Coordinator,Field Worker, and the Artist-In-structors.

The Interorganizational Environment

Jazzmobile, Inc., is well known,locally and nationally, and hasinformal links with other organiza-tions, especially arts organizationsin the Harlem community. Theselinkages include those with the

local police precinct (i.e., Pre -.

cinct 28), the Upper West SideHealth Center, local banks, the Har-lem Urban Development Corporation,the Goddard Community Center, theDance Theatre of Harlem, Harlem Cul-tural Council, Upton Chamber of Commerce, Children's Art Council, andStudio Museum of Harlem. Represen-tatives of some of these organiza-tions serve on either the JazzmobileBoard of Directors or served on theAlternative Education Arts ProgramAdvisory Committee. Jazzmobile'sPresident was a Council Member of

the National Council for the Arts, aPresidential appointment. He wasalso President of the Arts and Busi-ness Council of New York City. Themembership of this organizationincludes representatives of 60

national and multi-national corpora-tions (e.g., Exxon, AT&T, Mobil) and60 arts organizations. Jazzmobile

also has longstanding links with theCommunity School Board in the targetschool district. These links havebeen an asset in Jazzmobile's previous arts and cultural projects.

With regard to the AlternativeEducation Arts Program, and theevaluation of the program, two critical links were weak: (a) thatwith the principal of the project'sprimary target school, and (b) linkswith parents and other residents inthe area of the school.

Staffing and Resources

The Jazzmobile Alternative Education Arts project consisted of the

following staff positions:Executive Director/Project Man

agerProject Director9 ArtistInstructors -- School

Day Component4 ArtistInstructors AfterSchool/Saturday and Summer Components

1 Evaluation Coordinator1 Guidance Counselor1 Field Worker1 Curriculum Coordinator1 ClerkTypist1 Accountant1 Secretary1 Audiovisual Production Special

ist

Job descriptions of the abovestaff were provided by the projectand are included in Appendix A.

Table 3 summarizes project staffingby primary work location and educational background. As of June 1979,15.11 fulltime equivalent (FTE)

workers were employed by the project. Of this, 7.00 FTE workerswere primarily stationed at the

Jazzmobile office and 8.11 FTE workers were primarily stationed in theschools. Of the 35 people everemployed by the project, 11 (31%)had left the project's staff by June1982.

289

Jazzmobile

The Community

The community targeted for services by the Jazzmobile AlternativeEducation Arts Program is calledCentral Harlem. The population ofCentral Harlem is almost entirelyblack. Reportedly, the unemploymentrate for black youths in CentralHarlem is about 60%, while it is 18%for adults (over 21). The incomelevel in Harlem is low: 1979 statist4cs indicate that 50% of families were earning less- than $5,000(grant proposal, p. 13) and a largepercentage of Harlem residentsreceive some type of public assistance (e.g., welfare).

Local health care facilities aredescribed as inadequate. Infantmortality is high. The housingstock is reportedly declining by3000 units per year due to arson andabandonment (New York Times, 1978).The specific target area as

described by Jazzmobile staff is

even worse than Central Harlem atlarge. Crime and delinquency ratesare among the highest in New YorkCity. Police data indicate that theprecinct within which the projectoperates ranked 11th out of 73 precincts for detention and arrests ofyouth between the ages of 7 and 20.

The Schools

Three schools in the school district were targeted by the Alternative Education Arts Program: Anintermediate school and two elementary schools. The intermediateschool was the primary target, RS

interventions in the elementaryschools were minimal. The elementary schools are two of four feederschools for the intermediate school.According to Jazzmobile administrators the intermediate school waschosen by the Community School Boardbecause of the serious problems withwhich it is plagued.

163

Jaarambile

Mister, of the Project

In 1V7A, Immobile was awarded agrant under the Emergency School AidAct (MA) Title VII to implement anArts Education Enricbuont Program insmother school district in East Har-lem. The iciest was to lend supportis the district's desegregation planby attempting to ameliorate theadverse effects of minority groupiselatiom. Jazzmobile received*mother IESAA grant in 1977 to workis Community School District 3

(Upper Vest Side) and A (East Har-lem), Itsobattan. The organizationreceived still another ESAA grant'in1971 to continue its activities.

is 1971. Jassmobile become inter-feted in delinquency prevention anddiversion for "first time offen-ders." It was awarded a grant from

the Law Enforcement AssistanceAdmiaistration MAO to initiateprevention and diversion program inan elomestary school in the districtwhere this project now operates.

The project conducted during the

1t71'79 school year was called YoungMises* of Modem Art Program, orCRAP. There were four basic inter-mac* compooentss An in-school

arts education enrichment program,an afteriwechool and Saturday programfor first-time offenders, workshopsfor permits, and a summer program.Studeote from grades 3 to 6 partici-

poled. Like the Alternative Educa-tion Arts program, major emphasiswas placed on helping children to"enperieece success by channelingtheir talents and energies into theacts"' (ANJOU. AlUattAllfit. MAU.April DSO. Police Precinct 28

participated by assisting in con -

ducting arts workshops and exhibit-lift students' art work in the police

statics. Students gave performancessod displayed art work at hapsitalssad churches.

An evaluation of Project CHAP

111

-290-

1

reported a reduction in drug-relatedproblems and absenteeism, and

parents reportedly became highlyinvolved in the program (JusticeAssistance News, April 1980).

Planning for the Alternative Edu-cation Arts Program began in earlyMarch, 1980. A meeting was held onMarch 3, in which Advisory Committeemembers from the Community School

District, key personnel from theSchool District, police, and commu-nity representatives met with Jazz-mobile staff. On March 29th, an

abstract of the proposed program byJazzmobile was submitted for review,discussion, and approval by membersof the Community School Board, Dis-trict administrators, and communityand business leaders. The proposal

was approved. The target schoolswere apparently decided upon at thefirst planning meeting.

Between the eime- the Coimunity

School Board approved the programand when the program began opera-tion, there was an election and theBoard composition changed. The newBoard fired the Superintendent andinstalled an acting Superintendent.The acting superintendent did not

achieve permanent status until

February, 1982.

The principals of the target

schools were apparently told that

the Alternative Education Arts Pro-gram would operate in their schoolsat the beginning of the 1980-81

school year. The principal of the

intermediate school was reported tohave resented the imposition of theprogram. As a result, the principalmerely tolerated the Jazzmobile pro-ject in her school. The evaluationOf the project fared worse, as theprincipal was opposed to it. The

basic issue seems to have been acomplete lack of trust in evaluatorswho are "outsiders." For a period

164

of time, this distrust extended toeven the Evaluation Coordinator, a

Jazzmobile employee.

The principal's distrust may havebeen fostered by accounts of the

evaluation of the earlier CHAP pro-ject. An article about CHAPappeared in the Justice AssistanceNews (April, 1980). It was entitled"Jazzmobile: A Ray of Hope in theCrime-Ridden Streets of Central Har-lem." The first sentence of thearticle read: "The deck is stackedagaint the kids who attend NewYork's Public School 113." In thebody of the article Jazzmobile wasdescribed as about the only positivething that children in the schoolhad going for them. This was seenas a personal affront to the princi-pal of the school as well as beinginsensitive to the positive aspectsof Central Harlem.

Because the Community School

Board in power when Jazzmobile beganoperation was not the same one thathelped design the program, there waslittle support from Board members inovercoming the obstacles that the

principal presented.

What Was Implemented?

The First Year

Basically, we know little aboutwhat was implemented by Jazzmobileduring its first year. We do knowthat classes were held, and that

artist-instructors were present in

the school. Access to all data wasdenied by the school. We wereadvised not to visit the intermedi-ate school during the project'sfirst year of operation, ald littlemonitoring of the implementation ofthe project's activities was possi-ble.

During the first year of opera-tion the Jazzmobile project faced

numberous obstacles to implementing

-291-

Jazzmobile

its program in the primary target

site. Many of these problems seemto have arisen from Jazzmobile's ownpoor relationship with the princi-pal. These problems were probablyaggravated by attempts to evaluatethe program.

Jazzmobile had problems withscheduling and even finding rooms toconduct classes. Also the under-qualification of some of the Jazzmo-bile staff fostered resentment of

them by regular teaching staff.

The Jazzmobile counselor employedduring the first year was unable toestablish cooperative arrangementswith the regular schOol guidancecounselor and was denied access tostudents' school records due to a

lack of proper credentials. Duringthe second year she was replaced byanother counselor.

The Jazzmobile field worker hadproblems getting the cooperation ofparents to participate in projectactivities. This lack of coopera-tion may have been related to

attempts to evaluate the project.The PTA president opposed an outsideevaluation and may have generatedsome distrust of the project by

parents.

Despite the foregoing problems,the project was able to implement

its program in some fashion:

classes were held, some field tripswere carried out, and some perfor-mances took place.

The Second Year

By the beginning of the secondyear agreement was reached betweenthe evaluators and key project staff

that they would make a seriouseffort to document plans to overcomeobstacles in their forcefield and

document implementation through theProgram Development worksheets.

165

Jazzmobile

This' agreement was made during a

conference in August, 1981.

In conversations with Jazzmobilestaff member:, it became apparentthat intuition rather than data wasto be the technique for measuringthe degree of implementation. We donot know whether artistinstructorswere "warm and caring," how rewarding the art shows and field tripswere, or to what extent artistinstructors worked with staff teachersto enhance their approaches to

teaching. Project personnelresisted the quantification of theseimplementation standards.

Shortly after the August, 1981,

conference, the project directorresigned. The Development Officerof Jazzmobile stated that this hadlittle impact on the program, as shewas quickly replaced by a projectdirector who was a social workerrather than an artist.

Training of ArtistInstructors

A problem during the first yearof program operation was that Jazzmobile management had little knowledge of what the artistinstructorswere actually doing in their classesand why. It became apparent at theend of the year that some of the

artist instructors were not doing agood job. Also, many of the

instructors were apparently notaware that they were employed by ademonstration project whose primarypurpose was to prevent delinquency.

To ameliorate these problems,

more attention was given to staff

training during the second year.

Training began with a formal orientation week in September, 1981.

Noel Day of Polaris Research and

Development (the technical assistance contractor) participated in

this orientation. Areas covered

were the following: (a) Personnelmatters (sick days, vacation, etc.),(b) system change strategies,(c) rationale of data collection,(d) the national scope and demonstration nature of the "DelinquencyPrevention Through Alternative Education" initiative, (e) the ProgramDevelopment Evaluation process and(f) long and short range goals of

the project weekly unit plans.

For the second year-, a requirement was implemented that artistinstructors submit "unit plans" on aregular basis. These plans weresupposed to describe what artistin

t,

structors planned to do and why.They were intended for quality control or monitoring purposes.

292

After the initial orientation,artistinstructors in the day program were required to attend weeklystaff meetings with the PrOjEct

Director, during which additionaltraining took place. Topics coveredthroughout the year included:

weekly unit plans, guidance concerns, special events scheduling andplanning (e.g. art shows, field

trips, etc.), curriculum, studentcouncil activities, protocol for

interaction with parents, protocolfor interaction and sharing ideas

with regular staff teachers, andstudent placement concerns (in secondary schools).

When the after school and Saturday program started in November,1981, the participating artistinstructors met twice weekly with theProject Director. Training focusedon adolescent development, behaviormodification techniques, special

needs of gifted and talented students and referral of students forspecial high school placements.

166

Diplomacy: Meetings with Key Actorsin the Forcefield

During the first year, many prob-lems arose that strained relationsbetween Jazzmobile and the adminis-trators of the target schools, par-ticularly the intermediate school.During the second year, Jazzmobilemade some effort to anticipate prob-lems before they occurred and to

approach key actors in order to

overcome obstacles. Many meetingsoccurred during the 1981-82 schoolyear, but the following appear to

have been most significant:

o Late August, 1981; The Exe-cutive Director and Evalua-tion Coordinator met withDr. Jerry Evans, Presidentof the Community SchoolBoard - District 3. Themeeting concerned mattersaffecting implementation ofthe second year program. As

a result of this meeting,the Community School Boardassigned a staff member toact as liaison between Jazz-mobile and itself.

o Sept., 1981: The JazzmobileEvaluation Coordinator metwith the principal of the

intermediate school in an

attempt to make peace, so

that constructive conversa-tion regarding the evalua-tion could take place.Reports indicate that the

meeting was cordial, but itdid not lead to data beingreleased for evaluation pur-poses. A decision was madeby Jazzmobile to wait untilthe principal retired in

November, 1981 before datacollection would be

attempted. (The principal

did not actually retireuntil February, 1982).

o September 17, 1981:

-293-

Jazzmobile

Jazzmobile Evaluation Coor-dinator (as acting projectdirector), the JazzmobileCurriculum Coordinator andthe newly assigned CommunitySchool District 3 liaisonmet with the assistant prin-cipal at the intermediateschool to schedule Jazzmo-bile classes into the schoolcurriculum.

o Early November, 1981: The

newly hired JazzmobileCounselor held an initial

meeting with the schoolstaff counselor to introducehimself and explain his rolein the Jazzmobile program.Reportedly, the new counse-lor was able to strike a

working relationship withthe school counselor and

they held regular meetingsthereafter. Unlike the pre-vious Jazzmobile Counselor,the new counselor wasgranted access to studentfiles.

o L cember 2, 1981: Jazzmo-bile Executive Director,Evaluation .Coordinator and

new Project Director metwith the Community School

Board to discuss data col-lection issues. At that

meeting, the Board decided(after 1 1/2 years) thatJazzmobile would need affir-mative parental permissionfor collection of data fromschool records.

o December 18, 1981: AnotherCommunity School Board meet-ing was held so that the

public could respond to thedata collection issues. At

this time the previouslypassed resolution that Jazz-mobile must obtain parentalpermission was upheld.

167

Jazzmobile

o January 31, 1982: Projectdirector met with school

personnel to coordinate second semester of the Jazzmobile Alternative EducationArts Program in order to

adjust to class shifts as aresult of the semesterchange.

o March 22, 1982: JazzmobileExecutive Director, ProjectDirector and new EvaluationCoordinator met with theCommunity School Board andschool administrators to

discuss objections that theschool personnel had to theSAES survey. The evaluationstaff's principal investigator and another staff member

f also attended that meeting.The Community School Boarddecided that the selfreportdelinquency section of thestudent survey must be

deleted in order for it to

be administered.

Public Relations

The Jazzmobile Alternative Education Arts Program suffered from alack of credibility during its firstyear. School staff reportedly sawthe arts as an unnecessary "frill."The artist instructors did notreceive the respect from otherteachers they felt they deserved,and this made it harder to do theirjobs.

During the second year Jazzmobiledevotee many resources to showingthat Jazzmobile did provide a worthwhile service and that it knew whatit was doing. In the case of

Project Director and Counselor, thisinvolved hiring people with moreimpressive credentials.

During the second year, a Jazzmobile Alternative Education Arts Program brochure was issued (late

294

November, 1981). The brochure gavea job description of each staff member and listed -111__e,r career accomplishments. The point was to establish the staff members as trainedprofessional educators. The brochure was distributed widely.

Another activity which Jazzmobileengaged in heavily during the secondyear was what staff called "creativedocumentation" of program implementation.

By midOctober, staff had prepared a slide presentation of yearone activities for showing at parentmeetings and workshops with regularschool staff. By the end of Novem-ber, 1981, a presentation was ready.A video tape of classroom activitieswas ready by January, 1982. These

slide and video presentations wereused for training purposes during

weekly staff meetings and were shownat parent and

_teacb-erWErksh4s:

Jazzmobile also organized an

alternative education resource bookcase, published one issue of a newsletter, and helped the intermediateschool develop a public relationsbooklet and a yearbook.

The Art Classes

Most sixth graders at the two

elementary schools participated in

arts instruction provided by Jazzmobile staff. In the intermediateschool, most (except those retained)who participated as 7th graders during the 1980-81 school year werecontinued in the program as 8th

graders. New 7th graders were

selected in cooperation withintermediate school staff accordingt-D scheduling convenience and perhaps other unknown criteria. Schedules for participating students

were completed by midSeptember,1981. Classes began September 24,1981, at the two elementary schoOls

168

and September 28, 1981, at the

intermediate school. While the sem-ester was in progress 33 studentswere lost from the program in the

intermediate school, owing to a Pro-motional Gates Program whereby stu-dents more than two years behind inreading or math were required to

receive remediation.

Six artist-instructors worked inthe intermediate school-day programand three artist-instructors in theelementary schools. Students in theelementary schools attended two tothree classes weekly. Participatingstudents at the intermediate schoolattended Jazzmobile classes daily.

During October, 1981, four

artist-instructors were hired to

teach after-school and Saturday

classes. Many of the after-schoolstudents were referred by Policeprecinct 28 or the intermediateschool as chronic absentees and

"first time offenders." Classesstarted on November 27, 1981 with 45students. Classes met two school

days a week from 3:15 - 5:00 p.m.and on Saturdays from 9:00 - noon.

By the third quarter of the

school year 60 students were in theafternoon and Saturday program.Students in the afternoon and Satur-day program were given the opportu-nity to continue during the summermonths (i.e. July 1, 1982-August 27,1982). Summer classes met daily

from 9:00 a.m. - 3:00 p.m. Seven

Artist/Instructors served 84 stu-

dents. Classes were held at one ofthe primary schools. Eight internsfrom the New York City Summer YouthEmployment division of the Depart-ment of Employment helped out.

Arrangements were made for students'lunches through the Free Lunch Pro-gram.

It is difficult to capture whatwent on in the Jazzmobile Art

-295-

Jazzmobile

classes except to say that the

artist-instructors taught arts. It

is apparent that much more actuallytook place but this was of an ad hocnature and depended greatly on thestyle of the individual artist-in-structor. Several artistic accountsof instructors' style have been pro-vided by the project and are

included in Appendix B.

Other Activities

Besides time in Jazzmobileclasses, target students partici-

pated in a number of out-of-classactivities such as field trips, artshows and performances.

Field trips. Attempts to conductfield trips during the first yearwere ad hoc. Lessons were learnedconcerning the steps necessary in

the planning of field trips:Licensed school staff must be pre-sent as chaperones' (this requiresfinding cooperative staff), and

parental consent forms are neces-sary.

A calendar for field trips wasput together by the JazzmobileCounselor for the second year. The

counselor also compiled a list of

students who would be participating.With this information in hand, the

counselor and project director wereable to make arrangements withparents and school staff.

Although an explanation of how

each field trip fit into individualstudents' treatment plans was not

available to the evaluators, an

effort was reportedly made to

arrange field trips that were a

meaningful supplement to each stu-dent's program. For instance, stu-dents who were interested in attend-ing city-wide arts high schools madefield trips to such places. In all,

31 field trips were conducted, withstudents going to the museum,

169

Jazzmobile

library, movies, plays, monuments,and parks.

Films were used during the secondyear for the purpose of reinforcingconcepts covered in Jazzmobileclasses. Some of the films dealtwith the arts and some with AfroAmerican history. Eighteen differentfilms about such topics as Paul

Robison, the Harlem Globetrotters,Adam Clayton Powell, Roy Wilkens,and sex education) were shown.

Productions and art shows. Students in the Jazzmobile classes were"rewarded" by having the opportunityto display their talents. Towardthis end, 10 productions and artsshows were held. Failure to obtainproper permits halted plans for a

street festival at which Jazzmobilestudents were to perform.

Special events. Several special,events were carried out as part ofthe Jazzmobile program. Theseincluded performances by the NanetteBeardon Dance Company and a lectureby Jazz musician Jimmy Owens.

Rewards

All Jazzmobile target studentsreceived a Jazzmobile teeshirt anda certificate of participation.Students involved in final productions had cast parties.

A Jazzmobile Youth Council was

established at the intermediateschool. Meetings of elected officers began in December, 1981, andtook place weekly thereafter. One

issue of a Newsletter was publishedby the council in January, 1982.

The newsletter covered informationabout the activities of Jazzmobileparticipants, as well as articles ofinterest to the general studentbody. Seven Jazzmobile participantsreceived scholarships to dance

classes at Harlem School of the

296

Arts. Three graduating students

from the Intermediate School

received the Chancellor MemorialAward, a citywide recognition formusical proficiency.

Jazzmobile participated in the

graduations held at the target

intermediate school and two feederelementary schools. Jazzmobile Special Art Awards were given to a number of graduating students. In

addition all students received certificates of merit.

Parent Component/Fieldwork

The Jazzmobile Alternative Education Arts Program emphasized parentinvolvement in their children's program activities. Many parent workshops were held throughout the second year. In these workshops,parents were told about the artrelated activities that their childrenlearned during the day. Videotapeand --';de presentations of performanc's lown et these meetings.The Jazzmobile Field Worker wasactive at soliciting parental partic'Lation. Flyers were sent homewi_h students, phone calls and homevisits were made, and appeals weremace to the respective Parent Associations. one occasion a healthfair was 'eld to attract the attenticn of ,arents. Also, in October,19E Jazzmobile staff held eveninghours during "OpenSchool Week" atall three schools so that parentwho could not attend workshops couldat least meet the artistinstructors. Narratives about the progressof the program were mailer' 'n

parents in December, 1981.

A total of 74 parent workshopswere held between September, 19L1,

and June, 1982. Each workshop hadbetween 30 and 35 participants. All

told, over 105 parents weta considered regular participants. Some

parents were enlisted to help carry

170

Jazzmobile

out some of the productions put on Evaluation of the Project

by Jazzmobile.

Counseling Component

The Jazzmobile management hiredan experienced and properly credentialed counselor in October, 1981.

The new counselor was better able toimplement the program in the schoolbecause he held a degree (i.e. MSW)and credentials that were respectedby regular school personnel. He wasallowed access to confidentialrecords and was effective in soliciting the cooperation of regularteachers for such 'things as chaperoning field trips.

Besides counseling students for

"behavioral problems" on an "as

needed" basis, he engaged some students in career counseling. Heorchestrated the previously mentioned field trips to artshighschools and prepared an arts careerdirectory. The new JazzmobileCounselor established a referralprocedure for onetoone counselingand organized a case record system.He designed special referral forms

and followed a set appointment schedule.

Curriculum Guides

During the life of the Jazzmobileproject, two curriculum guides werecompleted and a third started. The

first (drama) was completed at the

end of the first year. Little is

known about how this guide was actually used except that there was aformal presentation of the guide toschool administrators in Septemberand October, 1981. A second (dance)curriculum guide was developed during the second school year in cooperation with the Jazzmobile danceinstructors and some regular schoolteachers. Training workshops for

regular school teachers in the useof the dance guide were held in lateMay, 1982.

297

No evaluation design was

describe in the project's grant

proposal. Subsequent interviewswith project staff members indicatedthat they expected a simple prepostsummative evaluation involving readily available aggregate data with nocontrol group. Jazzmobile had beenexposed to this type of evaluationin its previous youth programs

(e.g., ESAA and Project CHAP). Several Jazzmobile staff members suggested that they would not haveaccepted the grant had they known ofthe evaluation requirements.

The rigorous evaluation--notanticipated by Jazzmobile or the

school system-- created problems forJazzmobile. The building administrator of the primary target schoolwas vehemently opposed tothe proposed evaluation activities, and

this may have created difficultiesin implementing other aspects oftheproject. A fair summary of evaluationrelated activities is that theywere unproductive. The Jazzmobileproject is a good example of a project that is not amenable to rigorous evaluation. Jazzmobile staff

did not see the value of rigorousevaluation. Evaluators were told

time and time again by key staffmembers that the Jazzmobile projectneeded a more "creative" evaluation.The fact is, however, that Jazzmobile was funded as a delinquencyprevention demonstration project andneeded to be evaluated as such, in arigorous manner. This.end was neveraccomplished. What follows is an

account of efforts to evaluate theJazzmobile project.

The- First Year

Because the Jazzmobile project

had already begun operation whenevaluation efforts began, the evaluation design was a nonequivalent

171

Jazzmobile

control group comparison (i.e.

Jazzmobile target students vs. the

rest of the students in the

intermediate school). Evaluationsof the after-school component and ofthe program in the elementaryschools were never accomplishedbecause problems with the evaluationof the major component--theintermediate school operation--de-voured the entire evaluation effort.

Although assignment to Jazzmobileclasses for the 1980-81 school yearwas not random, the assignment wasfairly haphazard, based on schedul-ing convenience. On all variablesfor which data were provided, the

treatment and non-equivalent controlgroup look similar. Differencesbetween groups on age, sex and raceare non-significant. Nevertheless,any attempt to interpret first yearresults must be done with cautiondue to potential bias of an unknownnature owing to the following:(a) only students for wham parentalpermission was obtained were allowedto be surveyed, (b) he differencein non-response rate to items on thesurvey between treatment and non-e-quivalent controls was highly signi-ficant (p<.001). Non-response to

questionnaire items was far greaterfor Jazzmobile treatment. studentsthan for non-equivalent control stu-dents.

The only data available for thefirst year cohort are survey mea-sures--no data from records wereever released to the evaluators.

Treatment-control differences onall measured outcome variables werenon-significant, suggesting that theprogram made little difference forthese outcomes. See Table 4.

-298-

Year Two

After attempts to create a stron-ger design for the second yearfailed, the design again defaultedto a non-equivalent control groupcomparison. The original assignmentto treatment and control conditionswas based on scheduling conveniencefor the school. Entire classes weredesignated as either Jazzmobile ornon-Jazzmobile classes. Then afterthe school year began, the schoolreceived the students' reading (CAT)scores from the previous school

year. Those who did not "pass" werereassigned to remediation programs,and they were not allowed to parti-cipate in the Jazzmobile classes.

Very late in the school year, theCommunity School Board decided to

release data only for students forwhom affirmative parental permissionhad been obtained. As expected,affirmative permission response ratewas low, 64% (116) for Jazzmobileparticipants, and 39% (67) for non-participants (i.e. non-equivalentcontrols). The extent of the biasintroduced into the evaluationdesign by the shuffling of studentsbased on CAT test scores and the

decision regarding parental permis-sion remains unknown.

The SAES Student Survey, withdelinquency items deleted, wasadministered in May, 1982 to treat-ment and non-equivalent controls forwhom affirmative parental permissionhad been obtained in the intermedi-ate school. 74% (135) of this

select group responded to the sur-vey. The 135 students for whom Stu-dent Questionnaire data were availa-ble represent 38% of the total

sample in the intermediate school

(N=351). No attempt was made to

survey the 214 seventh graders inthe two feeder elementary schools.

172

After the end of the school year,California Achievement Test scores,attendance and promotion data fromrecords were provided for the treat-ment and control students for whomaffirmative parental permission hadbeen obtained. Data on withdrawalswere provided for all students inthe intermediate school. Whateverbias was introduced, the treatmentand control students allowed to par-ticipate looked remarkably similaron background variables. There wereno statistically discernable differ-ences between treatment and controlson participation in special educa-tion (t=1.02, p<.31), sex (chi

squared =2.66, dF=1, p<.10), self-re-ported parental emphasis oneducation (t=-.13, p<.90), self-re-ported parental education (t=-.53,p<.59), and pre-treatment standard-ized achievement test (CAT) reading(t=-.68, p<.50) and math (t=.91,p<.37) scores. There was no signi-ficant difference in racial composi-tion between the treatment and non-equivalent control groups (t=1.42,p<.12). The treatment group diddiffer from the control group ongrade level. The non-equivalentcontrol group was split aboutequally between 7th and 8th graders.The treatment group was weightedheavily with 8th graders (t=3.44,p<.01).

Questionnaire measures. The onlysignificant differences found bet-ween treatment and non-equivalentcontrol groups were on the measuresof Attachment to School and StudentEmployment. Table 5 shows that thetreatment group was more attached tothe school than the control group(<.05), and members of the controlgroup more often reported holding aregular part-time or full-time job.If the 24 measures reported in Table5 were independent, one would expectto find one difference at the .05

level by chance alone. Among the

non-significant comparisons, the

differences between treatment and

-299-

Jazzmobile

control groups appear small relativeto the standard deviations. No

pre-treatment measures usable as

covariates for attachment to schoolwere available, so it was impossibleto attempt to adjust for potentialpre-existing group differences for

this outcome. For student employ-ment, grade level was correlatedboth with treatment status andemployment status. Using gradelevel as a covariate does not alterthe statistical conclusion.

Measures from school records.The project provided data on with-drawals from school (for reasonsother than transfer) for all stu-dents in the sample. Other data(standardized achievement testscores, attendance, and promotion)were provided only for studentswhose, parents consented, and onlyfor students still in school at

year's end._

No significant differences werefound between treatment and controlgroups on math or reading achieve-ment, or on school attendance.Furthermore, the differencesobserved appear small relative to

the standard deviations. (See Table6.)

A larger proportion of treatmentthan non-equivalent control groupstudents were promoted to the nextgrade (93% vs. 81%), and this dif-ference was significant at the p<.05level. But the treatment and non-e-quivalent control groups also dif-fered (p<.01) on previous promotionhistory and the control group con-tained more special education stu-dents. When previous retentions andspecial education status were used acovariates (to statistically "con-trol" for these pre-existing groupdifferences) no significant effectof treatment or promotions was found(p=.78).

173

Jazzmobile

Jazzmobile participants were lesslikely to withdraw from school forreasons other than transfer thanwere non-equivalent control groupstudents (5% vs 13%). No pre-treat-ment or other data were availablefor students who withdrew, making anattempt to apply statistical con-trols impossible. The observed sig-nificant difference therefore may bea treatment effect, or it mayreflect pre-existing differencesbetween the two groups. We have noway to disentangle these competingpossibilities.

Discussion

Limitations of the Evaluation

This evaluation, which was con-ducted under difficult circums-tances, has some important limita-tions. In the interest of clarity,the limitations themselves aredescribed here. Later, some of thefactors contributing to these limi-tations will be discussed.

This has been a quasi-experimen-tal evaluation of an interventionwhich, given the range of interven-tions possible in schools, was notparticularly strong on an a prioribasis. The non-equivalent controlgroup design used is imperfectbecause, despite some evidence thattreatment and control students werenot grossly different on availablemeasures of student background char-acteristics, the treatment and con-trol groups may have differed in

unknown ways. Furthermore, datawere generally available only for

students whose parents gave affirma-tive consent for the release of

data, and response rates to the sur-veys were not perfect. And, mostinformation about student outcomeswas available only for students whodid not withdraw during the schoolyear (the exceptions is studentwithdrawal, which was available for

all students). These limitationsmade statistical controls for pre-existing differences difficult or

impossible to apply. Biases of anunknown nature may influence thepattern of results. Finally, no

direct measures of delinquent behav-ior (the primary outcome of interestto the project's sponsors) was avai-lable in the second year of projectoperation.

Despite these limitations, it is

possible to say with -considerableassurance that if the project hadany sizable effects on any of the

outcomes studied, such affects would

11be detectable with the design used.This is so primarily because littledifference was found between treat-ment and control students on the

background characteristics that wereavailable for study, and because thesample sizes are sufficiently large.

-300-

Limitations of the Project

A number of influences in theproject's environment created limi-tations for the implementation andevaluation of the project. On sev-eral points the project's analysisof its forcefield appears to havebeen incomplete, leading to subse-quent problems. For example, thatJazzmobile staff working in theschool might be considered unquali-fied due to a lack of conventionaleducational credentials was notanticipated. The resulting non-ac-ceptance of staff in the school, andthe influence that the perception ofnon-acceptance had for the day-to-day work of the artist-instructorsand the counselor, was not thereforeanticipated.

A second example involves diffi-culties in implementing the evalua-tion component. The project tendedto focus on the immediate obstacleof the intermediate 'school princi-pal's reluctance to participate in

174

the evaluation, and planned to moveahead with data collection after theprincipal had retired (an eventual-ity that was considered imminent).No plans were made for an alterna-tive strategy to obtain data shouldthe principal not retire, nor werestrategies developed to move .head

with other aspects of evaluationactivities when the initial obstaclewas overcome.

The Puzzle of Community Support

Project managers viewed a lack ofparental concern with and commitmentto education as a problem, as'isillustrated by their inclusion of

these parental characteristics in

their theory statement. Indeed, onecomponent of their intervention wasthe series of parent workshopsdescribed earlier. Yet the commu-nity was not successfully mobilizedto support the evaluation of the

project. This lack of communitysupport for evaluation is illus-trated by an important meetingcalled by the chairman of the schoolboard to discuss evaluation planswith the Jazzmobile and national

evaluation staff. This meeting,which was to seek community input

for a school board decision aboutthe spring 1982 student survey, wasattended by relatively few parentsof children in the intermediateschool where the survey was to be

performed, and was dominated by

vocal individuals opposed to the

survey. An analysis of the force-field would have suggested the

advisability of ensurir.g that sup-porters of the Jazzmobile projectattend this meeting in large num-bers. That meeting led to the

school board's censorL;.,!, self-re-ported delinquency measurer.' from the

survey administration. We are puzz-led by the failure of the implement-ing organization to mobilize suffi-cient community support for the

project and its evaluation to over-come this obstacle, especially when

-301-

Jazzmobile

the project appears to have had a

reasonably strong program of parentworkshops and a full time communityfield worker.

Principal Support and the Level ofDelinquent Behavior

As noted earlier, one potentialexplanation of the intermediateschool principal's resistance to theevaluation activities was the por-trait of East Harlem contained in anevaluation report for an earlierJazzmobile project. That report hasdid the Jazzmobile grant proposalfor the present project) character-ized youth in the community as highin delinquency and from disorganizedfamily and community situations. Weare somewhat puzzled by the paradox-ical lack of agreement between somecommon stereotypes of East Harlemand the Jazzmobile grant proposal'sdescription on the one hand, and thelevel of self-reported delinquencyreflected in the 1981 student surveyresults on the other. The level ofself-reported delinquent behavior inour survey was not especially high.Although survey non-response in 1981was high, implying that substantialbiases may exist in the results, theresults do not support an impressionof an extremely delinquent popula-tion in the intermediate school. Wesee at least two potential explana-tions for this paradox. First,

either response bias or invalidreporting may have produced mislead-ingly low estimates of delinquentbehavior among members of this popu-lation. Second, the degree of indi-vidual delinquent behavior among theyouths of this community may be

exaggerated in popular stereotypes

of the community. Whichever of

these possibilities is nearer the

truth, clearly much remains to be

learned about the youth of this com-munity. The difficulties of con-

ducting evaluation research in the

community pose a challenge for

175

Jazzmobile

responsible service providers and

researchers of the future.

Protect Staffing, Personnel Plan-ning, and Management

Several difficulties in projectimplementation appear to have stem-med from problems of initial projectstaffing, personnel planning, andpersonnel management.

Staffing and personnel planning.The success of most human serviceendeavors depends heavily on the

staff involved in the delivery of

services. One difficulty in the

staffing for this project has

already been described: The lack ofeducational credentials necessary tocreate an impression of legitimacyin the school. An additional diffi-culty may have stemmed from a mis-match between the job demands andpersonal preferences or talents of

some staff. The artist-instructorswere, we assume, for the most partindividuals with interests and com-petencies in various artistic

endeavors. Their experience and

interest in the more conventionalaspects of their jobs--developingcurricula, keeping records, adheringto regular schedules, and so

forth--may have been less than wasrequired by the jobs. Our impres-sion is that a more careful analysisof the tasks required by the jobs ofproject director, instructor, andevaluation coordinator may have sug-gested the recruitment of a somewhatdifferent mix of staff. This

impression receives some supportfrom the unsystematic reports we

have received of job dissatisfactionon the part of some staff and fromthe relatively high turnover amongproject personnel. Both the projectdirector and evaluation coordinatorresigned, and turnover among staffoverall exceeded 30%.

Management. Midway through the

project, project managers discovereda need for a structure for holdingartist-instructors accountable for

their activities in the schools. At

that point they provided trainingfor staff on the goals of the proj-ect, and with the ,change in projectdirectors they began providing moresupervision. This increase in

structured expectations, training,and supervision may have come too

late in the life of the project.

Effects of the Intervention

Despite the limitations in the

evaluation described earlier, some

,reasonably confident conclusions arepossible. The following paragraphsdescribe our best summative judgmentabout the effects of the JazzmobileAlternative Arts project.

-302-

Empirical evidence. On the basisof the empirical evidence, the proj-ect had little effect on the stu-dents exposed to its interventions.Statistically signficant effects

favoring treatment students werefound on attachment to school (lik-ing for school) and on dropout. The

size of these effects were moderate,ard confidence in their interpreta-tion as actual effects of the alter-native education project is weakenedby the quasi-experimental nature ofthe design and the unavailabilitysufficient pre-intervention measures

for thorough application of appro-

priate statistical controls. Fur-thermore, because Co many differentoutcomes were examined, one or moresignificant results were to be

expected by chance alone. The proj-

ect does not appear, however, to

have done harm. The inly statisti-cally significant result favoringthe control group was on a measureof student employment: More controlthan treatment students reportedfull- or part-time employment.

17G

The evaluation design was,

despite its limitations, suffi-ciently strong that had the projectproduced large positive effects theyshould have been detected. We findno evidence of such effects in theempirical evidence.

Artistic evaluation. The imple-menting organization did not antici-pate an empirical evaluation of thekind we had hoped would be imple-mented (and of the kind that eventu-ally was implemented in substantialdegree). Indeed, the organizationindicated that it had hoped for a

more artistic evaluation and thathad it known about the kind ..of

empirical evaluation expected it

would not have undertaken the pro-ject in the first place.1

The descriptions of artist-in-structor approaches to their work,included in an appendix to this

report, are therefore of considera-ble interest. These descriptionssuggest that artist-instructors weresensitive, caring, individuals. Wehave no way of knowing how accu-rately these descriptions character-ize the workaday activities of theseinstructors, but we have no reasonto doubt that they convey an accu-rate impression. An iAterestingimplication of these descriptions,combined with the empirical evidenceabout project effects, is eitherthat (a) artist-instructors imple-menting the kinds of personal philo-sophies and approaches described inthe appendix is an insufficiently

1 A letter from S. David Bailey(dated 30 April 1983) commenting ona draft of this report is availablefrom Jazzmobile, 159 West 127thStreet, New York, NY 10027, or fromLois Hybl, Center for Social Organi-zation of Schools, The Johns HopkinsUniversity, 3505 N. Charles St.,

Baltimore, MD 21218.

-303-

Jazzmobile

powerful intervention to producemuch in the way of measurablechanges in student behaviors andattitudes, or (b) these philosophiesand approaches were not fully imple-mented on a day-to-day basis. Theempirical evidence does not allow usto chose among these two alternativeinterpretations.

Conclusion. The alternative artseducation intervention as imple-mented and studied here did not havea major influence on the psychoso-cial development of' youths in 'thecommunity in which it operated. It

may have increased liking for

school, prevented some dropout, anddecreased student employment, alt-hough the evidence is not ironcladthat these effects were produced bythe intervention. No evidence sug-gests that this intervention pre-vented delinquent behavior.

Recommendations for Future Projects- - - --- -

The conduct and evaluation of

this project have been instructive,and they suggest the followingrecommendations for future delin-quency prevention projects:

1. Grant awards for demonstra-tion projects should be made only toorganizations that understand andagree to fully implement a rigorousevaluation, and where concrete evi-dence of a commitment of relevantactors in the organization's force-field to fully cooperate with the

evaluation is available.

2. Expectations for evaluationshould be made concrete and explicitin advance of requesting applica-tions for grant competitions.

3. Demonstration projectsinvolving only arts education showlimited promise for delinquency pre-vention, and demonstration projectsproposing only to implement arts

hi nob it

educattoo *hook, be ovoided. Thisdee* ma moan that arts educaxion aspart of a IWO temples and conceptu-ally otroager *et of interventionsto to he Avoided, only that otherthrobs *tea equal a more complexand detenotble set of interventionsare to be preferred to a replicationof project* reoembling the onederottbed hero.

4. improokoionislic evidence is

no tutetitute to hard-nosed (spirt-Cal evaluation. The public deservespropose to prevent dolinquencywhich are demonstrably effective,and project sponooro should there-fore a** decielons about projecttendtma on (a) assessments of evalu.stability. and (b) the empirical andWeeretical literature on delin-'wary, rather thaw upon Impressions

about what seems appealing.

5. Prevention project implemen-ters should carefully consider (andshould be provided with assistancein considering) the potential obsta-cles to project implementation in

the project's environment.

6. Prevention project implemen-ters should carefully consider (andshould be provided with assistancein determining) the personnelrequirements of their interventionsand in selecting and training therequisite personnel.

7. Delinquency prevention pro-jects should initially and continu-ally demonstrate A clear and directcommitment to the prevention of del-inquency as a primary goal.

Jazzmobile

References

In Last Decade, Leaders Say, Harlem's DrTams Have Died. New York Times,March, 1978, pp. 1,13.

Jazzmobile: A Ray of Hope in the Crime-ridden Streets of Central Harlem.Justice Assistance News, 1980, p. 6-7.

-305-

179

Parents are

financially

poor.

Survival is

main

priority.

Schools do not

provide youth

with career

information.

o Educational

0 system denies

Black

accomplishments.

Male students

value athletics

but not

academics,

Low parental

awareness about

school

disciplinary

procedures

and practices.

Low student

career

awareness,

Education

not valued

by youths.

Figure 1

The Fully Elaborated Theory, Including Causal Factors Not . Idressed by Program

Low parental

concern and

support for

school

activities.

Low aspirations:

sense of

hopelessness.

Diverse talents

unexercised

and unrewarded

in school.

Failure

in

school.

Frustration

School not

stimulating.

Low student -

teacher

attachment,

1 30

Non - school

environment

relatively

more

stimulating

than school.

Truancy,

chronic

absenteeism.

involvement

in street

activities.

VInvolvement Delinquvol

with -7 hvbvior.

negative

role models.

181

..

Figure 2

Abbreviated Theory Including

Causal Factors the :!roject Hoped

to Addms

Increase parental Increase parental

awareness about concern and

school disci- support for

plinary procedures school activities,

and practices.

Increase

extent to

Increase students' Increase aspirations,which

career awareness.----)decreasediverse

----7hopelessness.

talents

are used

Increase student- Increase extent

teacher to which

attachment. youths value

education.

1S2

A

and

rewarded

7 in school,

1

2Increase

Increase -attendance

success 11

in

school.

, /

Increase

sense that

school is

stimulating.

\JDecrease

involvement

in street

activities.

183

Jazzmobile

Board of

Directors

and

President

0op

\.,

Jazzmobile Executive

Director

and

Project Manager

Figure 3, Organizational Arrangements

Alternate :e Arts Education Project

Advice and consultation

a Development

Officer

Evaluation

Coordinator

Project

Director

Community

School

Board

wr

Acting

Superindendent

rimm owe W...

Curriculum

Coordinator

Arts

Counselor

Artist Instructors

Field

Woikers41.=1.1-0M.M

Principal

Intermediate

School '

c -7 PTA

185

Table 1

Jazzmobile Alternative Arts Projects: Goalsand Corresponding Measures for Which Data

Were Actually Provided

Goal

1. Reduce truancy and chronicabsenteeism

2. Reduce youth involvementin "street" activity whichpromotes delinquent behavior

I'leasure................ .....

a. Data on absences providedby the school

b. SAES Student QuestionnaireNon-attendance Index

None

3. Youth potential will be Noneused in a positive way.

4. Increased parent awareness Noneand involvement

-309-

1 S 6

Table 2

aivesJazzmobile Alternative Arts Project: Obju

(i.e. Intermediary Outcorsqs) and CorrespondingMeasures

Objective

1. Increased student-teacherattachments

2. Student:: will becomeexcited and stimulated by in-school experiences

3. Increase youths' careerawareness, primarily in thearea of art, Seco ndarily inother areas.

4. Increase students' successin non-academic areas (i.e. thearts)

5. Increase parental awarenessconcerning anhool policies,procedures and practices withregard to disciplinary actionsand school referrals to thejuvenile justice system

6. Increase Parents' concernand support for youths' schoolactivities

..

..Measure

Student-teachurInteractiona

Attachment toScboola

None

None

None

Parental Empbaeis onEducation scaled

7. Youths previously unused None

potentials will be tapped

8. Increased Youth value of Educational aepirationsa

education

Alienation sOlea

........

9. Reduce youths' sense ofAlienationa

aThe project Personnel did not identify these tpeAsures;they seem to us to be relevant and were availaVik through theSAES questionnaire.

,310-1 8 7

Table 3

Educational Background of JazzmobileAlternative Arts Education Project

as of June, 1982

Work location and education

Jazzmobile officea

FTE

7.00

Number

9

M.A 2

4

A.A1

1

High school 2

Schools 8.11 15

M.A 2

B.A /11.S. 4

A.A 1

'Ugh school 8

Total 15.11 24

Note. Of the 35 people ever employed by the-project, 11 (31%)had left the project's staff by June, 1982 (Source: SAES quar

terly staffing report).

aIncludz project manager, project director, curriculum specialist, accountant, audiovisual specialist, field worker, and

secretaties.

311

188

Table 4

Jazzmobile Alternative Education Arts Project:Means and Standard Deviations for Spring 1981

Student Survey Outcomes for Students in Treatmentand Non-Equivalent Control Groups

Variable

Attachment to Parents

Treatment

ti SD

__a

Group

N

Non-equivalentControl

M -SD N

__a

Negative Peer Influence -.34 4.40 118 -.35 4.92 77

Attachment to School -.50 5.88 31 -.47 6.66 39

Negative Belief 2.68 '1.32 20 2.05 1.13 34

Interpersonal Competency 4.00 1.29 19 3.99 .90 35

Positive Self-Concept -.60 4.39 18 .85 5.28 32

Punishment Index -.75 2.02 43 -.08 2.54 52

Rewards Index .14 3.69 41 .37 3.07 48

Victimization .95 1.48 37 .53 .82 50

School Effort 2.05 .76 162 2.00 .73 95

Practical Knowledge 9.17 5.78 20 9.68 5.47 40

Internal-External Control 2.80 .95 20 3.06 .76 35

Alienation 1.24 1.24 22 1.22 1.43 34

Self-Reported Grades 3.01 .83 162 3.11 .79 94

Reading Ability Self- 1.39 .88 145 1.68 .80 78

ReportedSatisfaction with School 2.24 .78 167 2.16 .72 95

ProgressSchool Nonattendance 1.04 1.65 166 1.11 1.72 88

Index

Rebellious Autonomy 2.05 1.10 20 2.29 .92 36

Involvement .09 6.58 102 -1.06 5.69 69

Total Delinquency 1.88 2.55 78 1.75 2.54 57

Serious Delinquency .74 1.49 81 .44 1.33 62

Drug Use .63 .98 81 .76 1.17 60

Note. None of the outcomes presented above show statistically signi-ficant differences between treatment and control students.

aThe questions comprising this scale did not appear on the Jazzmobileversion of the 1981 Student Survey due to objections by the CommunitySchool Board.

-312-

189

Table 5

Jazzmobile Alternative Arts Project: Meansand Standard Deviations for Spring, 1982Student Survey Outcomes for Students in

Treatment and Non-Equivalent Control Groups

Variable

Treatment

SD

Group

N

Non-equivalentControls

M SD N

Attachment to Parents . .63 .26 81 .60 .25 42Negative Peer Influence .16 .19 75 .16 .15 39Attachment to School .81* 1, .18 72 .72 .23 39Belief Scale Scored Positively .71 .22 42 .75 .24 33

Interpersonal Competency .85 .18 45 .76 .26 32Positive Self-Concept .80 .14 46 .80 .18 31Punishment Index .20 .22 77 .18 .26 38Rewards Index .37 .32 77 .36 .31 38Victimization .09 .15 77 .13 .20 38School Effort .67 .26 _80. _ .59_ _ .26 _ _38

Practical Knowledge 1.31 .34 38 1.39 .29 30Internal-External Control .44 .21 48 .41 .24 31Alienation .27 .22 48 .30 .30 33Self-Reported Grades 3.18 .67 88 3.19 .92 42Reading Ability, Self Rating 1.56 .65 79 1.74 .92 38Days of School Cut in4 Weeks Preceding Survey .10 .30 89 .21 .68 42

How Often Cut One orMore Classes .39 .73 89 .25 .62 44

Educational Expectation 3.99 1.35 87 3.53 1.80 45Did You Work for Pay LastWeek? .46 .50 86 .65 .48 40

Regular Part-Time orFull-Time Job? .11* .31 84 .30 .56 43

Suspended from School DuringTerm .16 .37 69 .18 .39 38

School Nonattendance Index .38 .58 87 .31 .64 42Rebellious Autonrimy .60 .34 37 .52 .36 30Involvement .24 .18 77 .24 .20 37

*Treatment group significantly different from non-equivalent controls(p<.05).

-313-

190

__

Table 6Jazzmobile Alternative Arts Project: Means

and Standard Deviations for 1981-82Outcomes Measured Using School Records

Outcome

Group

Non-equivalentTreatment controls

variable M SD N M SD

April, 1982, CaliforniaAchievement Test(CAT)Grade Equivalent Scores:

Reading 7.69 1.99 105 7.53 2.93 51

Math 7.32 1.55 102 6.88 1.99 48

Number of days absentduring the schoolyear a

15.11 16.80 106 13.82 18.46 56

Promoted to next grade .93 .26 111 .81 .40 62(1=promoted, 0=not)b

Withdrawal for reasonsother than transferc

.05* .21 195 .13 .34 156

Note. N's vary depending on the availability of data. Data on with-drawals were provided for the entire sample. Parental permission wasrequired for the release of other information to the evaluation. Per-mission was obtained for a total of 183 students. Attendance, promo-tion, and test score data were missing for a few of these students.

aNo data were available for students who withdrew during the school

year.

b The unadjusted proportion of students promoted was significant atthe p<.05 level. The treatment and control group also differed at the..01 level on prior promotions. An analysis of covariance, treatingprior promotions and special education status as covariates, impliesno significant difference (p=.78) between treatment and controlgroups. Because prior promotion data were unavailable for many stu-dents, the N's for this analysis were 76 and 18.

c No pre-treatment data were available for students who withdrew.Therefore we were unable to perform analyses that control for pre-treatment differences.

-314-

19i

Appendix A Job Descriptions Provided by Jazzmobile

Executive Director/Program Mana-er

Manages program to insure that implementation and prescribed activities arecarried out in accordance with specified objectives and in accordance withO.J.J.D.P. guidelines: Plans and develops methods and procedures for imple-menting program, directs and coordinates program activities, and exercisescontrol over personnel responsible for specific functions or phases of pro-gram. Selects personnel according to knowledge and experience in area withwhich program is concerned, arts education. Confers with staff to explainprogram and individual responsibilities for functions and phases of program.Directs and coordinates through subordinate managerial personnel, activitiesconcerned with implementation and carrying out objectives of program. Reviewsreports and records' of activities to insure progress is being accomplishedtoward specified program objectives and modifies or changes methodology as

required to redirect activities and attain objectives. Prepares programreports for the Office of Juvenile Justice and Delinquency Prevention. Con-

trols expenditures in accordance with budget allocations. Meets with the

Advisory-Committee and District Representatives. Reports to the President andBoard of Directors of Jazzmobile, Inc.

Educational Specialist

No description available.

Project Director

Under the direct supervision of the Program Manager, plans, directs, andcoordinates activities of designated project to insure that aims, goals, or

objectives specified for project are accomplished in accordance with pre-scribed priorities, time limitation, and funding conditions: Reviews projectproposal or plan to ascertain time frame and funding limitations, and to de-termine methods and procedures for accomplishment of project and staffingrequirements. Establishes work plan and schedules for each phase of projectin accordance with time limitations and funding. Confers with staff to out-line project plans. Modifies schedules, as required. Prepares project status

reports for management. Confers with project personnel to provide technicaladvice and to assist in solving problems. Oversees daily project operationsin all three school sites.

Artist/Instructors

Art. Instructs pupils in art, such as painting, sketching, designing, andsculpturing: Demonstrates method and procedure to pupils. Observes pupils'

work to make criticisms and corrections. Directs planning and supervising of

student contests and arranging of art exhibits. Conducts workshops for

parents regarding the art discipline the student is learning.

Drama. Teaches acting principles and techniques to groups: Conducts read-

ings to evaluate student's talent. Adapts course of study and training meth-

-315-

4

Jazzmobile

ods to meet student's need and ability. Teaches enunciation, diction, voicedevelopment, and dialects, using voice exercises, speech drills, explanation,lectures, and improvisation. Discusses and demonstrates vocal and body

expression to teach acting styles, character development, and personality pro-jection. Directs plays for school and public performances. Auditions stu-dents to select cast and assign parts. Rehearses and drills students to

insure they master parts. Assigns nonperforming students to backstage produc-tion tasks, such as constructing, painting, moving scenery, operating stagelight, and sound equipment. Directs activities of students involved in con-structing, painting, and lighting scenery. Teaches elements of stagecraft,stage makeup, costume craft, play writing, and play direction. Conducts work-shops for parents of participating students regarding the art discipline thestudent is learning.

Dance. Instructs pupils in ballet, modern and ethnic forms of dancing:Observes students to determine physical and artistic qualifications and limi-tations and plans programs to meet students' needs and aspirations. Explainsand demonstrates techniques and methods of regulating movements of body to

musical or rhythmic accompaniment. Drills 'Pupils in execution of dance steps.

Teaches history of dance. Choreographs and directs dance perfumance. As

instructor, conducts workshops for parents of participating students regardingthe art discipline the student is learning.

Guitar. Teaches groups guitar in public school: Plans daily classroomwork based on teaching outline he/she has prepared for this special program.Evaluates student's interests, aptitudes, temperament, and individual charac-teristics to determine abilities. Instructs students in music theory, har-mony, score and sight reading, composition, music appreciation, and provides

group lessons using technical knowledge, aesthetic appreciation, and appropri-

ate teaching techniques. Conducts workshops for parents of participating stu-dents regarding the art discipline the student is learning.

Percussion. Teaches groups percussion in public school: Plans daily

classroom work based on teaching outline he/she has prepared for this specialprogram. Evaluates student's interests, aptitudes, temperament, and indivi-

dual characteristics to determine abilities. Instructs students in musictheory, harmony, score and sight reading, composition, music appreciation, andprovides group lessons using technical knowledge, aesthetic appreciation, andappropriate teaching techniques. Conducts workshops for parents of partici-pating students regarding the art discipline the student is learning.

Evaluation Coordinator

Directs research acitivities concerned with the program: Initiates proce-

dures to determine if program objectives are being met. Provides interpreta-

tion of research data gathered. Works as the liaison between the program andthe national evaluators concerning the management information system. Pro-

vides evaluators with the data as outlined in the evaluation design. Coordi-

nates data collection methods with the project director and school staff.

-316-

193

Jazzmobile

Counselor

Counsels individuals and provides grOup educational and vocational guidanceservices relating to the arts: Collects, organizes, and analyzes information

about individuals through records, tests, interviews, and professional

sources, to appraise their interests, aptitudes, abilities, and personalitycharacteristics, for vocational and educational planning. Compiles and studies occupational, educational, and economic information to aid counselees inmaking and carrying out vocational and educational objectives. Refers students to placement service. Assists individuals to understand and overcome

social and emotional problems. Assists gifted and talented students withplacement into special arts high schools. Provides arts activities and educational opportunities information for parents of the participating students.

Field Worker

Wor1s s the direct liaison between the program and the community. Under

the direction of the Project Director organizes and conducts workshops forparents of the participating students regarding the program goals, objectives

and their involvement. Also conducts parent workshops in conjunction with the

guidance counselor and artist/instructors concerning their subject areas.

Assists the Guidance Counselor with disciplinary problems requiring parentalinvolvement.

Curriculum Coordinator

Plans and develops alternative education curriculum materials integratingthe arts into basic subject areas for grades 6, 7, and 8: Develops materials

for inservice education of teaching personnel. Confers with local school

officials to develop curriculum materials. Confers with lay and professional

groups to disseminate and receive input on curriculum materials. Conducts

conferences designed to promote arts education activities related to core cur

riculum areas. Conducts research into areas, such as teaching methods and

techniques.

Accountant

Manages all fiscal activities for the O.J.J.D.P. program. Applies princi

ples of accounting to devise and implement accounting system. Maintains

accounts and records. Performs such bookkeeping activities as recording dis

bursements, expenses, and tax payments. Prepares quarterly fiscal reports.

Prepares cash advancement requests. Prepares weekly payroll. Represents

Jazzmobile before government agency upon certification by agency involved.

Audiovisual Production Specialist

Plans and produces audio, visual, and audiovisual material for communication, information, training, and learning purposes: Develops production ideasbased on assignment and generates own ideas based on objectives and personal

interest. Conducts research or utilizes knowledge and training to determineformat, approach, content, level, and media which will be -lost effective, meet

objectives, and remain within budget restrictions. Plans and develops prepro

duction ideas into outlines and scripts. Locates and secures settings, prop

317

194

Jazzmobile

erties, effects, and other production necessities. Sets up, adjusts and

operates equipment, such as cameras, sound-mixers, and recorders during pro

duction. Conducts training sessions on selection, use, and design of audiovi

sual materials and operation of presentation equipment.

Secretary

Schedules appointments, gives information to callers, takes dictation, and

otherwise relieves officials of clerical work and minor administrative and

business detail: Reads and routes incoming mail. Locates and attaches appropriate file to correspondence to be answered by employer. Types routine cor

respondence. Files correspondence and other records. Answers telephone andgives information to callers or routes call to appropriate official and places

out going calls. Schedules appointments for employer. Greets- visitors,

ascertains nature of business, and conducts visitors to employer or appropri

ate person. Arranges travel schedule and reservations. Compiles and types

statistical reports. Makes copies of correspondence or other printed matter,

using copying machine. May prepare outgping mail, using postagemetering

machine. Types all curriculum materials.

Clerk/Typist

Compiles data and operates typewriter in performance of routine clerical

duties to maintain evaluation data: Types reports, correspondence, application forms, and data for evaluation. Files records and reports, posts information to records, sorts and distributes mail, _answers telephone, and performs

similar duties.

Appendix B

The following promotional material, prepared by Jazzmobile staffer William

Harris, presents their artist instructors in a positive light. These "artis-

tic evaluations" are interest_ng and serve to give some of the flavor of the

project's activities. We present them without fUrther comment.

Wendell Williams

Wendell Williams, a guitar instructor, relates his first experience of per-

sonal counseling of one of his Jazzmobile students. A 7th grade student ten-

tatively indicated that he smoked marijuana. He later confessed that all of

the money from his after-school, part-time job was going toward purchasing the

drug which he realized had become "a problem." He did not feel that he could

talk to his parents about it, and fortunately Mr. Williams was an understand-

ing, but nonjudgmental adult with whom he could discuss his-dilemma.

Williams emphasized that although he was sympathetic, concerned and suppor-

tive, the solution had to come from the student himself. The student was able

to overcome his problem and thanked his instructor for his help and concern.

This incident taught Williams the importance of open communication with the

members of his class. He instituted loosely scheduled class periods for gen-

eral discussions of their problems and goals. Sometimes one or more of the

students would feel the need for thrashing out problems and would initiate the

talk sessions.

"The whole thing is getting them on your side," Williams says. During

these sessions, in addition to their problems, he discusses the positives and

negatives of the life of an aspiring artist, and also explores career alterna-

tives which most students had little knowledge of.

Earl Stewart

Percussion instructor, Earl Stewart, had his first contact with a neighbor-

hood family when one of their six sons entered his class. Although he was one

of Stewart's best students, he was eventually lost to the streets.

The three younger brothers, 12, 10, and 8, all good students, are now tak-

ing percussion. Stewart realizes that the street life and the "quick and

easy" money are as great a lure to them as it was to their brother. He also

realizes that without the positive attraction of Jazzmobile, it is a real pos-

sibility that they would already have fallen into "the life."

-319-

His method of combating the reality of that negative attraction is toconstantly remind them of the alternative. Jazzmobile, he points out, is more

than a day camp. To prove his point, he now has them involved in the projectto raise money through performances.

Self-discipline is constantly stressed as a necessary ingredient forremaining in his class, and for making positive progress in their own lives.

Stewart has appointed the oldest of the brothers as his drum captain. He

has taken the students to the rehearsals of his professional group. Althoughthe final verdict is far from being in, Stewart believes the brothers arebeginning to realize that they do not have to resort to the life style oftheir older sibling.

Elba Muley

Ms. Muley had an art student 12 or 13 years old. She found that he had ahistory of physical run-ins with his instructors and had spent time in a dis-ciplinary school situation as a result. t,*

His abilities with the visual arts, expressed primarily through wall graf-fiti,were obvious, but sports and physical aggressiveness held his interestmore than art.

Muley developed a project for a few of her students in various classes tocreate comic books which spoke to the problem of violence, stealing and drugs

as they affected the school environment. The student, working independentlyby his own choice, developed the storyline, characters, dialogue and art workfor the book. He worked formany weeks on the project. Though his interest

was evident, it was a constant struggle to motivate him on a daily basis.

When the work was nearly completed he found out that others were also

involved in similar pursuits. This angered him to the point that he tore upthe materials that he had worked so long and hard on. Ms. Muley asked him to

leave the class. He did, but returned a few days later. They discussed theirmisunderstanding both as it related to the nature of the project, and theirpersonal conflict which resulted from that misunderstanding..

With this new understanding the student set about recreating his manu-

script. He used his lunch periods and any free time he had to meet the dead-line for completing his submission. The student's aunt contacted his instruc-

tor to commend her for the marked change evident in his behavior.

The book was finished on time and was one of only three to receive an awardin an art competition. He later completed a second book about graffiti.

Miriam Bacote

A girl, who Ms. Bacote soon realized lacked self confidence, entered herdance class. She came as a member of a clique of girls who thought the danceclass was to simply involve learning the latest popular steps. When they

found what was actually expected of them they withdrew from the class. The

girl stayed, but was torn by an on-going conflict between her desire to followher friends, and her interest in learning to dance.

197

Ms. Bacote contacted the student's mother in the hope that encouragementfrom home would help to relieve the girl's anxiety about her choice. Unfortunately, the mother was more concerned with her own street activities and wasno help. The student obviously benefited from Bacote's concern, and came toher own understanding of the value of the discipline involved in an endeavorthat she loved. She developed to the point that she was featured as the leaddancer in one of the class's performances.

Roberto Reyes

Guitar instructor Roberto Reyes is well aware of the problem of providingtechnical instruction to students with widely varied levels of aptitude forthe instrument. Using a method of initially having them copy the things thathe plays, he begins to get a feel for their coordination and ability to memorize, as well as getting them all moving at the same time at their various levels. Step by step encouragement is most important at this stage, givingcredit for even the smallest acccmiplishment. Actual musical notation comesonly after everyone has a newly acquired sense of the confidence that theyhave some facility on the instrument.

Reyes' method of teaching his students to respect their instrument parallels his philosophy of dealing with them as human beings. The period leadingto the actual handson experience with the guitar is highly structured. The

question of who paid for the guitars is discussed. Once it is understood thattheir parents are, in essence, owners of the instruments, a sense or responsibility and privilege is connected with their use.- The students know that anyabuse of the instrument means revocation of the privilegeof using it. He

emphasizes that he has never lost a guitar through abuse or theft.

Felice Batiste

The image this art instructor tries to project is that he is his students'friend as well as their instructor. In addition to the specific techniques ofartistic expression, Mr. Batiste strives to instill in them an appreciationof art and its varied experiences.

He maintains an "open door policy," always providing students with afriendly, impartial ear. He attempts to explain the seemingly unfair contradictions they are faced with. Batiste, who does not insist that his studentscall him by his surname, says that he does not ever remember having a freelunch period; there always seem to be a number of students who want to talk.

Batiste initially concentrates on classroom assignments which can be completed in one session; both as an alternative to the ongoing pressures of therest of their school experience, and as an attempt to instill a steadily growing sense of accomplishment and selfconfidence.

An example of his oneonone contact with his students is a situation thatoccurred in a previous summer's program. Batiste had an introverted, buttalented student who should have been graduating but had been left back. She

wanted to go to an art school but was intimidated by the art test which was aprerequisite to entrance in the school of her choice. She was discouraged tothe point of wanting to quit school. Batiste helped her with the basics of

-321-

198

the school test and convinced her of the importance of staying in school. :Ale

eventually passed the exam, and is now doing well.

322

19I

Otro Camino: Second Interim Report

J. St. John

Otro Camino is a community-based program that functions as an exten-tion of the school day. Project staff work cooperatively with localschool administrators and faculty to assess student academic needs andschedule program activities. Besides the academic component, the pro-gram includes activities aimed at student social development and commu-nity participation in services offered.

Interim results suggest that the program may be increasing Otro Cam-ino participants' school grades, performance on standardized tests, andeducational expectations. Students' reports of influence in the schooland involvement in extracurricular activities may also be increased bythe program. Negative findings for Otro Camino's academic service com-ponent on arrest and reports of punishing experiences in school are mostlikely due to selection bias rather than to the program.

Major Program Design Changes

Changes to Community DevelopmentComponents

Advisory council. Otro Caminohad planned to institute an advisorycouncil during the first programyear. Although the council was

formed, the mission of the organiza-tion was not clear, nor was the com-position of membership specified.

The council met only once that year.In the second year, project managersdecided the council should have fourcommittees: (a) educators,(b) community members, (c) students,and (d) an executive committee com-posed of elected officers from eachcommittee and project management.The educators' committee was to

include the three school directors(principals), the district superin-tendent, each school's faculty liai-son to Otro Camino, regional univer-sity faculty, and others, up to 11

This report supplements an earlierreport on this project (St. John,

1982) which should be consulted foran account of the project's firstyear of operation.

-323--

members. Community members were tobe representative of many aspects ofcommunity life, including business,social services, police, and parentsfrom several barrios in the la Playacommunity, with the majority on this11-person committee being parents.The student committee was to includea representative of each grade levelserved (/ altogether), the presidentof the Presidents' Club at Otro Cam-ino, student council representativesfrom the school, and other studentswho were interested in being on thecouncil (up to 11 members total).

The council was charged by pro-ject managers to (a) assess whetherthe practices at Otro Camino wereeducationally sound and work to

improve the local schools (educa-tors), (b) assist Otro Camino in

meeting the needs of the community(community members), and (c) assesswhether or not Otro Camino studentactivities were adequate and suffi-cient (students). Their charge wentbeyond making recommendations andincluded assistance in implewentingrecommendations.

The committees met separately,

usually on the same evening, and

after the separate meetings, the

200

Otro Camino

total group met to hear the recom-mendations of each committee. Thecouncil did not begin functioningfully until January and reportedlymet two more times that school year.

Project staff reported on the

recommendations made by the parents'committee during the year. Commit-tee members were discouraged by thelow turnout for their meetings andsuggested that Otro Camino sponsorspecial community events to gainmore visibility in the grass-rodtscommunity. They suggested that stu-dents nominate their mothers forreceipt of.the "Mother of la PlayaAward" that would be given by OtroCamino. All nominees would be noti-fied and invited to the awards cere-mony at Otro Camino and thereby gainmore information about the programand their children's participation.Otro Camino staff and studentsimplemented this recommendation in

the spring. The community committeerecommendation resulted in the staffand students sponsoring severalother special community events as

well. Otro Camino sponsored obser-vations of International Women'sWeek, Holy Week, and Library Week,and hosted numerous celebrations ofstudent participant achievements.

School faculty support. SAES

Spring 1981 student questionnaireresponses indicated that students atall three schools were reporting lowself-concepts and that their abilityto interact with others was low (twoschools were below the fifteenthpercentile on both measures and theother school was below the thirtiethpercentile on the SAES Self-Conceptand Interpersonal Competency mea-sures). Otro Camino managers haddesigned their program to help stu-dents with their self-esteem and

interactive skills. They predictedthat as students improved in theseareas they would feel more comforta-ble in the school and become more

-324-

attached to school. But theyregarded current teaching practicesat the schools as an obstacle to

their effectiveness. They hypothes-ized that attachment to school wouldincrease only if teaching practicesin the regular school improved.

Furthermore, they thought that per-ceptions of the regular school mighteven worsen for students who parti-cipated in Otro Camino activities.The program managers decided to

institute teacher training coursesfor the target schools' faculties inhopes that some teaching methodswould 'change.

School administrators agreed to

turn over three half-day in-servicetraining sessions each semester toOtro Camino personnel. Otro Caminooffered workshops at two in-servicesessions in the fall semester, butlater abandoned the practice. They

felt that the competition betweenthe school - organizations- and -Otro - -Camino was being heightened by theirefforts, and that the competitionwas hindering program operations.

In the second semester, managersattempted to implement regular OtroCamino staff follow-up of partici-pants by contacting school faculty.This was to serve the purpose of

letting teachers know that goalsregarding students were shared, and

that Otro Camino's purpose was to

support the efforts of the schoolsdespite their different methods.

However, very little contact betweenOtro Camino staff and school facultyactually occurred. Otro Camino man-agers reported that their effortswere often thwarted because schoolfaculty schedules did not allow timeto meet.

Parent involvement. Project

staff planned to involve manyparents in the services offered to

students. Early attempts to solicitparent volunteers to assist with the

201

student service component met withonly limited success. Other strate-gies were developed to foster, parentsupport of students. A staff com-mittee went to the advisory counciland asked for suggestions on work-shop topics that staff could take toparents at community centers,recreation fields, government hous-ing project community halls, and thelike. A schedule set the beginningof the workshops in the second sem-ester of the 82-83 school year.

Staff planned for the public rela-tions club to help with publicity bybroadcasting workshop times andplaces by loud speakers from cars.The staff audiovisual specialist wasto arrange for electrical power atoutdoor sites .so that slides, videotapes, and movies could be part ofall workshops. At each workshop,staff planned to survey parents tofind out what kind of education top-ics they would like to have otherworkshops on, and these topics wouldbe added to the list for othertimes. This strategy was thought tobe responsive to the perceived needsof parents. Implementation data forthe 1982-83 year will show whetheror not the strategy was successfulat increasing parental participa-tion.

Changes to Student Services Compo-nents

Academic assistance at Otro Cam-ino. Academic assistance has been amajor component of Otro Camino sinceits inception. In the first yearthe component took the form of indi-vidual tutoring in subject areas inwhich students reported having dif-ficulty. This approach was usedbecause the program was a drop-incenter and students attended whenthey had time. In summer, 1981,

students participated for the fullday. To accommodate the schedulechange, the project adopted a smallgroup instruction model. Project

-325-

Otro Camino

staff and managers believed this

model was effective and implementedsmall group instruction for the

81-82 school year on a semesterbasis. Project managers believedthat the small group instructionapproach would make it possible toserve larger numbers of students.

Three distinctly different kindsof academic assistance were offeredusing the small group approach.During the first semester, groups

were involved in creative learningactivities to reinforce specificacademic skills. These activitieswere intended to be fun. For ins-tance, some students who reportedthey needed help with Spanish read-ing skills selected placement in agroup that read and discussed thedaily newspaper. During the secondsemester, groups were conductedusing more traditional materials andtraditional tutoring methods. Thatis, students brought in their schoolbooks and Otro Camino staff helpedstudents complete assignments. Stu-

dents were also tested for areas ofweakness and given supplementalmaterials and instruction in theseareas.

School administrators requestedthat Otro Camino provide intensivetutoring to a group of studentsidentified as potential dropouts.The schools sent 35 students in theseventh and eighth grades to theprogram site beginning in earlyMarch and ending in mid-May (theclose of the school year). OtroCamino staff divided the group intofive small groups. An elaborateschedule was established that pro-vided two 50-minute periods of

tutoring each school morning. Thedaily schedule included rotationthrough two subjects and allocatedfour 50-minute period4 to six sub-jects in a ten-day period.

202

Otro Camino

Library science. Otro Camino

managers arranged to train five highschool students in library proce-dures and techniques for high schoolcredit. These students worked in

Otro Camino's library learning bookcataloging methods, operating a booklending system, and assisting withlocating references for students.Four hours a week were spent workingin the library, and studentsreceived one school credit for thecourse.

Otro Camino mini-courses at the

schools. Each Otro Camino staffmember developed a mini-course dur-ing the first semester for possibleimplementation at the schools in thesecond semester. Staff contactedschool teachers to ask permission toincorporate the mini-course into theregular classroom courses. Minicourses were designed to last ten

days, had a limited number of learn-ing objectives, included daily les-son plans, and were to be given byOtro Camino staff with the teacherpresent in the room. The coursescovered special topics in Spanish,English, math, science, social sci-ence, and folk crafts. Each coursewas developed for a particular agegroup. Not all teachers agreed toparticipate. Project managersreported that three of these courseswere given during the 81-82 schoolyear, and that their course on thela Playa community was being givenin each tenth grade social scienceclass beginning in August, 1982.

Clubs. Otro Camino attempted topromote student social developmentthrough clubs. Otro Camino devel-oped and implemented 19 clubs duringthe 81-82 school year. Projectplans had called for the implementa-tion of five clubs beginning in thefirst program year, but none werebegun until the second year. The

implemented clubs covered personal

growth (career and psychological),recreation, arts and crafts,

-326--

investigative science, and drama andperforming arts. Two clubs wereorganized to address educationalissues in the community, and stu-dents organized a public relationsclub to help promote Otro Camino.Executives from each club composed aclub of presidents who coordinatedactivities and kept all participantsand staff up to date on each club.

One club had a special purpose.Several youths with discipline prob-lems were referred to. the project

from the high school. Repeatedcounseling efforts had proven unsuc-cessful except in eliciting somepersonal goals. Several students inthis group indicated they wanted tobecome martial arts experts. Arelative of an Otro Camino staff

person owned a karate school and

volunteered to donate one class perweek day for these students. In theclass, students learned the discip-line necessary to - become experts, - -and school personnel reported the

students' self-discipline improved

markedly at school. Once the stu-dents had mastered basic karatemaneuvers, they performed at a

school assembly, and appeared to

have become respected members of theschool community. I observed a

well-received presentation of skillsgiven by karate club members duringOtro Camino's week celebrating stu-dent achievements in April.

Learning resource center. Origi-nally Otro Camino planned for the

learning resource center to be usedby drop-in students (students whowere not registered in formal activ-ities) as an additional resource fortheir school assignments. The cen-ter included a library and audiovi-sual center. In the second year ofoperation, very few non-registeredstudents used the center, but largenumbers of registered students usedthe center facilities at times theywere not scheduled for activities.

203

44004.*** OOP144401.4, el 4

044*.o* OWNOSso. w sod

6444 *Simko.woes* sop*444* Ao to

04040 *Wise*6...4.* aidooroato owl4

.** aala istat***360$*44$4 444 040444, oit*1tt 4 4*400it4440so *MOO 400004 404,404*SO* ,44r okommow444t 404140.44$41044.04041* 401040410444.0 440 *WSW04* 0044040 4440,410i 04 O. sodSt Ow. 400* off. pro**WOO 6606004/04 onseits swoon

040 *000444* 464000 0404 14040440440 4040 44444**4 014 04 46.4tosta44 .441.41*

Woe*410 10141

4 4 4400100411444* 1140440 440P,4404400 414404w

4404 #4440444414 4444444+ 0404 44414,**MONO 0#40041 ott, Pro)00,otooma4amo 01*** *04440 o****** ot1000s4o maboo0* oo4a0mta 4oto

aoltooio*** arottoittoo oboe aokototoatitt44414 *so 44opowoctofie

*moo 40440044*, 40, oto*o41 at

4001 *4**,14 bmw bowl oo*bmood NG lbw* tkop baltpod

400mal otomoots**4 otoo WO to woodimm00 ***oily* atalmOvo aobottmlow.400 pia*****0 to ottioslet aotote**tamotoimgc,

41,04. 44 *4 *404. *vsSNOOM444 iit000lo ictiOmio ta

11.010.04* 4644014 44140410044014 44601044wv olotodoro.

ors omboolito. ts3w omen J.*ow *04 gloom**, 44wootor.ii41041 ato**Istowo tow instare

4444 tot vtatioreas roloosowtotitiatiftoo 400010 wolloqw **Wm tato, oa

44,64* 0444 ott$F4o1 tooth*

Ogre Cadge

fteopera-

ttoo *dew to Strop Contoes1404 tow jib poprottos aid tests-tag for otodonts. Jeb-sookies kelpwoe *flood to small WINO*. SCWdooto rootood iestrictios is boo totend aid al $y ter jobs, istorilaw-ten *tilts, istorporossal asoessica-Its* Atli% sad 'Moist 's* rossire-sos4* for wises oroors at

to sumo embers. Ibey

it** .*coiood assistingo is locaties

T6* mit ***pored*. odocotionsperistios woo osepostil is sem-iss 44 aides* pfsionost positiose.bet two 14ass000ts sir. acteally4ea444 m. staff diocevatol lbat lb*mild fakow limo wore very Witt is

tort* *tans lied promoted placingrosiest .tom. to Jobe tor pay.Aid, oho 00166 oppioyers *wood to4f *4411- ostaitoora sot t. *stoolswood to Om, or*osi credit for

oolowomor ptessoosts, stodosto fre-eway **stied est to take the

ovotiobto pottties. Appsrestly*tidos.* distded tboy mold be bet-ter aft pro/par ts* ter trusts* atter

ttsb stbsol by stayias is **beet.

otidonts did work as indicalcob olatotasto sad secretaries dar-

t** the Mot sm000tor. Mows ate-ilioSo were isperttsod by potpie attbs., OW et** a0 the 44re Cods*eisposetico sivestliniod ter rossiord sibool cultt.

to addition to tMoo positions tostinanity orssolsatties. the coops/r-etie* ciwesttos specialist tsusht a**arse to folk *rafts. Sbe took theportition4a Is Sae lass ober* stn -

s towed poop,* mere *M ins IbaUsdo of Was. they *ors nekins.lb** poporso41, Wiwi modest*WSW tho booboos. **testis) of

hooir effort,.

Otro Camino

Major Force Field Changes

A new school opened in August of1981 and caused unanticipated delaysin Otro Camino activities. Two tar-get schools changed from double-ses-sion to single-session days and re-distributed grade levels. Resultingschool enrollment and schedulingproblems lasted for about a month.Otro Camino managers had to waituntil school schedule problems wereresolved to establish their sche-

dule.

When Otro Camino finally began toschedule students into activities,project managers discovered that

their plan to serve students duringthe school day could not be imple-mented. The schools had added elec-tives to the school curriculum andstudents no longer had free periodsduring which they could participate.Project managers quickly assessedthe feasibility of after-school andweek end activities and received apositive response. The ensuingreorganization further delayed pro-ject start-up.

Changes in schedules meant thatlarge numbers of students were

served in a few hours each day.

This necessitated offering courses

that met two or three days a week atalternate times rather than every

day because of space shortages.

Project managers also gained schooladministrators' permission to

arrange with school teachers for

inclusion of ten-day mini-coursesgiven by Otro Camino staff in regu-lar school classrooms.

Some changes were salutary. Otro

Camino's good reputation spread

through the community and resultedin the enlistment of several commu-nity volunteers. A parent volun-teered to help with tutoring, areauniversity professors assisted in

diagnosing students' academic needsand designing treatment programs, a

-328-

local artist offered a class in folkart, 15 community businesses opened48 job training slots for partici-pants, the local school of karate

donated one class each day to a

small group of participants, the

Ponce Regional Office of the PuertoRico Department of Education sent

personnel to teach community inter-view and questionnaire design and

statistics to participants, and thepolice department sent a representa-tive to the advisory council. The

Puerto Rico Department.of EducationResearch and Evaluation office coop-erated in sending standardized testscores to the national evaluation

project.

Implementation Study

Project staff kept careful

records on student attendance at

Otro Camino by activity attended.

At the end of each semester the

daily hours were summed for -each

activity, then added across activi-ties for each student. This was theprimary means of reporting activi-ties offered, numbers served, and

total Otro Camino student service

hours.

These data are tabulated in

Tables 1 and 2. The tables show

that clubs are the most intensive

service provided by Otro Camino.

Students who participated in clubsspent 25-30 hours per semester, or

close to two hours per week, in

these activities. The intensity ofthe vocational and academic servicesincreased from semester one to sem-ester two, and the counseling ser-vices remained relatively low. The

main counseling service was group

counseling for 81 students who appa-rently met approximately once a

month.

Tables 1 and 2 show that, alt-

hough the absolute number of stu-

dents served declined from the first

205

to the second semester, the inten-sity of service increased. Duringthe first semester 74.8% of the stu-dents participated in only one act-ivity--clubs, most likely. But dur-ing the second semester more thantwo-thirds of the students wereactive in two or more differentactivities. The average hours of

participation per student alsoincreased somewhat during the secondsemester.

Outcomes

Evaluation Design

Otro Camino has an open dooradmission policy, meaning anyone whocomes for services is served. Ran-

dom selection of program partici-pants would have drastically alteredthis policy. A nonequivalent com-parison group design was estab-lished. This design enabled the

comparison of participating andnon-participating students from thetargeted grades at each of the threeprogram schools. Measures fromSpring, 1981 SAES surveys and schoolrecord data were used to statisti-cally control for the nonequi'.alenceof the groups.

Data

The project was specificallydesigned to help students in the

major transition grades 7, 9, and12, so data were collected on allstudents in the 1981 pre-targetgrade. Pre-target year and targetyear data include the SAES studentquestionnaire (administered in the

spring of 1981 and spring of 1982),school records data on school atten-dance and grades, Puerto Rico Stand-ardized Test Scores, and records ofpolice arrests for both years.

These data were also reported forsamples of all other grades between6 and 12.

-329-

Otro Camino

The SAES measures are describedin Gottfredson, Ogawa, Rickert, &

Gottfredsn (1982). Two distinctscoring procedures were used. The

1981 questionnaire scales mostlyinvolved sums of standard scores foritems in a scale. For the 1982 ver-sion, a new scoring procedure wasimplemented to make interpretationeasier: Most scale scores reflectthe proportion of items answered inthe desirable direction. Because ofthis scoring difference and becausethe 1981 scale scores were used onlyas statistical' controls, the 1981

scores are not reported here.

Analysis Procedures

Measures of intensity of treat-

ment were computed as follows:Records of treatment hours were col-lapsed into major categories: Aca-demic assistance activities, voca-tional development activities, clubactivities, counseling activities,and participation in activities dur-ing 1980-81 (the first year of pro-gram operation).

The number of participation hoursin each category was summed for bothsemesters and correlated with eachoutcome measure. The correlationmatrix was examined for correlati-ions significant at the p<.U5 level(two-tailed).

Then, regression equations wereused to estimate project effect

parameters--that is the part of thecorrelation that can be attributedto participation in the program

rather than selection effects.These equations included as statis-tical controls any pre-target yearmeasures that correlated with boththe activity category and the out-come.1 Since patterns of correla-tions were different for various

206

Otro Camino

activities, different control varia-bles were used in various tests.Notes on tables of results indicatethe control variables for activitycategories.

Results

Tables 3 through 6 present thecorrelation and regression coeffi-cients for all outcomes. Outcomesare organized according to projectgoals (Tables 3 and 4), projectobjectives (Table 5), and othertheoretically relevant objectives(Table 6). Appendix tables showmeans and standard deviations forall outcomes by activity categoryfor persons participating in eachkind of activity.

Academic Activities

Twenty-eight of thirty-nine cor-relations (72%) indicated the asso-ciation between participation in

academic activities and academicoutcomes was in the unfavorabledirection. (Read across the rows ofTables 3-6.) Participant achieve-ment was lower than nonparticipantachievement on 12 of 13 measures(Table 3); participants reportedskipping more classes (Table 4),

1 In technical terms, a model of theform y = n(-C4T.A,te was estimatedfor each outcome variable (y),

where 7 is the standardized regres-sion coefficent for hours of parti-cipation in activity category (x),

the/:: are the standardized regres-sion coefficients for the exogenouscontrol variables f , and e is ran-dom error. The specification of

each such model (i.e, the choice ofthe ) ) was made to include as

exogenous control variables any 1981measure correlated both with extentof participation in activity x andwith the particular y under consid-eration.

-330-

more alienation, lower self-con-cepts, less interpersonal compe-tency, and spending less effort onschool work than nonparticipants(Table 5); and participants reportedmore delinquent activity (self-re-port and record of arrests), morevictimization, more suspensions,less attachment to parents, morenegative peer influence, lowerbelief in the validity of conven-tional rules, less practical know-ledge, less internal control, andmore school punishment and fewerschool rewards than nonparticipants(Table 6). Most of the correlationswere small and nonsignificant, buteleven were statistically signifi-cant.

These uncontrolled associationsare not unexpected: Persons seekingacademic assistance in the programmay reasonably be expected to be inneed of it. These results do notimply that the participation/in OtroCamino's academic activities broughtabout these outcomes.

To examine whether these associa-tions remain significant when pre-existing differences among studentsare statistically controlled, thebetas in Tables 3 through 6 must beinspected. When controls are

applied, number of hours of partici-pation in academic activities is

associated only with arrest andschool punishment. In the contextof the model for the data examinedhere, this association could be

interpreted as an effect of partici-pation in Otro Cmmino's academic

activities. Two of these betas aresignificant.

Vocational Activities

Thirty-seven of forty-one corre-lations (90%) between participationin vocational activities and out-comes were in thefavorable direc-tion. Only on reports of having a

207

job (Table 3), amount of school

involvement, internal control, andschool rewards (Table 6) were the

nonparticipants favored. Twenty-oneof these uncontrolled correlationsare significant, but when statisti-cal controls are applied, only tworemain significant. In the contextof the models of the data examinedhere the results (shown in Table 3)could be interpreted as implyingthat participation in Otro Camino'svocational activities increasedscores on a standardized test of

Spanish language and increasedgrades (according to self-report).2

Club Activities

Thirty-four of forty-one correla-tions (83%) between participation inclubs and outcomes were in the favo-rable direction. Only for self-re-port of reading ability (Table 3),reports of victimization, suspen-sions, belief in the validity of

conventional rules, rebelliousautonomy, and internal control

(Table 6) were the nonparticipantsfavored. Fifteen of these correla-tions were significant. When sta-

tistical controls were applied,three remained significant. In thecontext of the models examined, par-ticipation in clubs increased socialscience grades (Table 3), students'reports about student influence in

the school3 (Table 5), and reportsof involvement in school activities(Table 6).

2 Although not significant when sta-tistical controls are applied, the

pattern of results for actual schoolgrades accords with self-reportresults.3 This measure is composed of itemsordinarily scored at the schoollevel (Gottfredson, Ogawa, Rickert,& Gottfredson, 1982) but combinedhere into an individual-level"scale" the psychometric properties

Otro Camino

Counseling Activities

Thirty of forty-one correlations(73%) between counseling activitiesand outcomes were in the favorabledirection. Nonparticipants werefavored on math grades, school grad-uation, having a job (Table 3),

self-reports of school non-atten-dance (Table 4), alienation (Table5), reports of victimization, sus-

pensions, belief in the validity ofconventional rules, and practical

knowledge (Table 6). .The applica-tion of statistical controls reducedthe number of significant correla-tions from six to two. In the con-text of the models examined here,

',the results imply that participationin Otro Camino counseling activitiesincreases educational expectations(Table 3) and involvement in schoolactivities (Table 6).

-331-

A re-examination of academicactivities. Project managers wereasked about the academic activitiesresults. They hypothesized that thestudents referred for intensivetutoring may have been markedlydifferent from students voluntarilyparticipating in Otro Camino's otheracademic activities. Furthermorethe tutoring came late in the yearand the nature of this interventionwas different from other academicactivities. They requested that

academic activities be re-examinedexcluding the tutoring activity, so

the results could be used in helpingdecide which way to approach aca-demic activities in the caning year.When tutoring is excluded and onlythose academic activities indicatedby the absence of an <a> in Table 1are examined, there is no dependableindication of any effect (positiveor negative) of participation in theremaining academic components.

of which have not been examined.

Otro Camino

Discussion

Except for results associated

with participation in academicactivities, the results are encour-aging. Most of the correlations arein the right direction and there issome evidence that participation hadpositive effects. However, the

actual difference between partici-pants and nonparticipants is small.For instance, less than 1% of the

variance in Spanish language stand-ardized test scores is accounted forby participation in vocational

activities. Most of the variance isaccounted for by grades and abilitymeasured by standardized test scoresin the previous year.

The findings cannot necessarily

be attributed to participation in

the specific activity categorylisted because many students (about78%) participated in more than oneactivity. The analytic proceduresused included only hours of partici-pation in one category at a time asa means of helping project managerslearn what happened for the variouscategories of activities imple-

mented.

Project participation was not

under experimental control; thus itis possible that the results reflectself-selection or differential matu-ration. Participants were plausiblymore highly motivated to betterthemselves and measures applied asstatistical controls may have beeninsufficient to control for this

possibility.

A final word of caution in the

interpretation of the significant

results from the regression equa-

tions: Some significant differencescan be expected to arise by chance.In these analyses, about two of the39 or 41 equations examined for each

category of participation would beexpected to reach significance at

the p<.05 level simply by chance.

This is precisely the number of sig-nificant differences observed.

In support of the argument thatthe results indicate, the effects ofparticipation, many 'of the resultsare sensible relative to project

activities. Project interventionwith students referred for intensivetutoring did not begin until near

the end of the school year, allowingplenty of time for academic and

behavior problems to occur before

participation and after pre-level

measures used as statistical con-

trols were collected. The reasonsfor their school referral included

recent disciplinary problems,

"declining achievement, and. atten-

dance problems.

Several of the vocational devel-opment activities emphasized lan-guage skills and exam preparation.It is not surprising that standard-ized test scores-improved-for -thisgroup. Students in this group werealso reported to have renewed theircommitment to education in prepara-tion for careers.

Many clubs emphasized the commu-nity culture and history. Others

were aimed specifically at develop-ing leadership skills. It is sensi-ble that students in these activi-ties had higher social science

grades and perceived more student

influence at the school than the

nonparticipants. On the other hand,people who are "joiners" in one set-ting are likely to be "joiners" inanother setting. Thus it is not

surprising to find Otro Camino par-

ticipants also participated in a

variety of school activities, and

this result may therefore reflect

primarily a selection artifact.

-332-

The counseling component includedhelping students decide about futureplans and determine the education

needed to successfully reach their

209

personal goals. Counselors

emphasized the need to stay in

school--both staying in school nowand pursuing education in the

future. Much of the counselingoccurred at the high school wheremany of the students who were goingto leave school had already withdrawn and at a time in students'lives when decisions about the

future must be made. It is sensiblethat this component is associatedwith higher educational expectations.

Finally, I have observed that theentire project staff are committed

333

Otro Camino

to the goals, theory, and objectives, and implement planned components with integrity. They have

encountered numerous obstacles, andimplementation of some componentshas, been seriously delayed. But

each delay and each obstacle has

resulted in more planning, and theseefforts appear systematically to

strengthen the component whileremaining true to the theory thatled to their development. Perhapsthe most serious implementationproblem the project would ever faceis delay while new strategies aredeveldped to insure implementationof components anchored in theory.

Otro Camino

References

Gottfredson, G. D., Ogawa, D. K. Rickert, D. E., & Gottfredson, D. C. Measures used in the School Action Effectiveness Study. In G. D. Gottfredson

(ed.), The School Action Effectiveness Study: First Interim Report. Baltimore: Johns Hopkins University, Center for Social Organization ofSchools, 1982.

St. John. Otro Camino. In G. D. Gottfredson (ed.), The Schoo_ Action Effec

tiveness Study: First Interim Report (Report No. 325), Baltimore: The

Johns Hopkins University, Center for Social Organization of Schools, 1982.

Table 1

Number of Students Served and Intensityof Treatment--by Activity and Semester

During the 1981-82 School Year

Semester 1

Number of Avg. hrs./ % TotalActivityb -students student effort

Semester 2

Number of Avg. hrs./ % Totalstudents student effort

Academic 157 11.2 13.3 115 22.7 27.0

Oral Communication-- 37 5.5 1.5 8 6.4 .5

EnglishReading--English 16 5.9 .7 7 7.1 .5

Speak, Read and 13 5.8 .6 0 0.0 0.0

Write--EnglishKeeping with the 1 2.0 t, 0.0 1 1.0 0.0

News--SpanishLearning to Read

and Write--Spanish

1 24.0 .2 2 8.5 .2

Growing with Music 16 8.8 1.1 0 0.0 0.0

--SpanishThe Why of Math- -Math 6 3.7 .2 0 0.0 0.0

Multiplying to 2 2.0 0.0_ 0_ _0.0_ 0.0

Divide--MathEveryday Problems 21 3.4 .5 6 2.8 .2

--MathPhotography--Science 30 28.9 6.6 20 29.8 6.3

Puerto Rico in the 2 30.0 .5 0 0.0 0.0

World--Social ScienceLearning More 14 13.0 1.4 0 0.0 0.0

About Puerto Rico- -Social Science

Civicsa 0 0.0 0.0 26 8.5 2.3

Natural Sciencea 2 2.0 0.0 37 8.5 2.3

Englisha 0 0.0 0.0 38 5.7 2.3

Spanisha 0 0.0 0.0 39 7.0 2.9

Matha 0 0.0 0.0 43 6.2 2.8

Social Sciencea 0 0.0 0.0 38 7.3 2.9

Investigation Club 0 0.0 0.0 5 71.0 3.8

Note. "Number of students" column reports the number of students who had at leastone hour of participation in a given activity. The "average hours per student"column reports the average number of hours for students who participated for at least

one hour in the activity. The "Z total effort" column reports the total number ofhours spent by all students as a ratio of the total number of participation hours.a Second semester participation hours in these activities represent time in Otro Cam-

ino intensive tutoring; all other courses in the academic group represent creative

gcademic activities.All 1981-82 activities except the Summer, 1982 "Mini-University" are included on

this table.

(continued on next page)

-335-

21_2

Activityb

Table 1 (cont.)

Semester 1

Number of Avg. hrs./ % Totalstudents student effort

Semester 2

Number of Avg. hrs./students student

% Total

effort

Vocational 216 17.9 29.4 43 26.0 11.9

Librarian Aides 10 48.8 3.7 7 36.4 2.7

Audiovisual 75 4.0 2.3 4 13.0 .6

TechniquesMy Career for the 4 14.0 .4 0 0.0 0.0

FutureGetting Prepared

for Work

20 29.1 4.4 32 - 25.3 8.6

Where Will I Study 8 23.3 1.4 0 0.0 0.0

College Entrance 103 21.1 16.5 0 0.0 0.0

Exam PreparationShort Term Careers 8 10.1 .6 0 0.0 0.0

Club

Clubs 269 24.7 51.3 156 30.0 49.7

Public Relations 10 2.0 ,2 12 39.2 5.0

Education 0 0.0 0.0 0 0.0 0.0

Sports 0 0.0 0.0 0 0.0 0.0

Model Building 13 4.9 .5 3 2.0 .1

Dramatic Arts Club 10 94.0 7.1 13 7.5 1.0

Aesthetic and 48 16.8 . 7.1 20 26.5 5.6

Modeling ClubAstronomy Club 23 55.2 9.7 14 20.9 3.1

. Gardening Club 17 14.9 1.9 12 11.9 1.5

Handicrafts for 57 20.6 8.9 28 16.3 4.8

GirlsHandicrafts Club 46 40.8 14.3 17 53.2 9.6

Folk Dances of 27 4.5 .9 8 60.1 5.1

La PlayaFolklore of La Playa 57 2.1 .9 8 28.0 2.4

Presidents Club 0 0.0 0.0 0 0.0 0.0

Karate Club 0 0.0 0.0 15 16.4 2.6

Vocal Group 0 0.0 0.0 10 82.9 8.8

Counseling 117 7.6 6.8 140 7.2 10.7

Self Development 14 16.0 1.7 0 0.0 0.0

ClubIndividual 22 12.5 2.1 33 2.0 .7

CounselingGroup Counseling 81 4.8 3.0 111 8.5 10.0

Total 653 20.1 100.0 397 23.7 100.0

Table 2

Number of Students Participating inOtro Camino Activities--by Semester

of the 1981-82 School Year

Number ofConcurrent Semester 1 Semester 2

Activities

1 493 74.8 136 31.9

2 142 21.5 122 28.6

3 21 3.2 56 13.1

More than 3 3 , 0.5 112 26.3

Total<a> 659 100.0 426 99.9

Note. Numbers do not include students who participated only in1980-81 school year activities.

<a> The unduplicated number of students servedfor the 1981782 _

school year is 797.

-337-

214

Table 3

Correlation Coefficients and Standardized Regression Coefficients for 1981-82

Academic Achievement Outcomes with Number of Hours in Each Type of Otro Camino Activity

ActivityType<a>

Academic

Span.

School Grades

goc.

Eng. Math Sci. Sci.

Project Goal

Standardized Tests

Span. Span. Eng.

Read. Lang. Read.

I

Self

Sch.

Math Grade

Report Self Report

Read. Educ. Have Paid

Abil. Expect. Grad.<b> Job Work

r -.041 -.027 -.053 -.005 .017 -.027 -.Ohl -.063* -.024 -.060* .016 -.048 -.013 .005 .020

beta --<c> --<c> --<c> --<c> --<c> --<c> --<c> -.020 --<c> -.020 --<c> --<c> --<c> --<c> --<c>

Vocationalr .164* .145* .043 .039 .100* .095* .200* .071* .201* .180* .044 .163* .442* -.045 .039

beta .055 .023 --<c> --<c> .009 .032 .083* -.025 .077 .084* --<c> .096 .050 --<c> --<c>

Clubsr .059* .072* .073* .140* .094* .029 .056* .042 .030 .087* -.014 .009 -.032 .020 -.009

beta -.014 .003 .017 .069** .023 --<c> -.014 --<c> --<c> .025 --<c> --<c> --<c> --<c> --<c>

Counseling

1

.046 .056* -.018 .088* .049 .008 .054 .054 .054 .072* .022 .118* -.038 -.034 -.033

to beta --<c> -.011 --<c> .031 --<c> --<c> --<c> --<c> --<c> .013 --<c> .090** --<c> --<c> --<c>

co

Participation1981

.7.'

r .010 .019 -.014 -.101* -.031 .092*<d> .038 .045 .091*<d> .036 .006 .043 .064* .028 .036

beta --<c> --<c> --<c> -.034 --<c> .145** --<c> --<c> .100* --<c> --<c> --<c> -.017 --<c> --<c>

Note. Participation 1981 is a dummy variable measuring whether or not students participated in 1981-.82 rather than number of

hours. N's range from 447 to 1006, depending on the patterns of missing data for variables in each equation. Analyses for eac

outcome variable uses 1981 variables that are significantly correlated with that outcome and with participation as statistical

control variables. Academic activity controls include 1981 standard test English reading score; English grade; and SAES friend

influence scale. Vocational activity controls include 1981 standard test' math and Spanish reading scores; school days absent;

school grades for Spanish, English, social science, and science; overall grade point average; and SAES belief, interpersonal co

petency, school rewards index, school punishment index, self-concept, rebellious autonomy, school attachment, total delinquency

serious delinquency, and victimization scales; self-report of gender; and 1981-82 school grade. Club activity controls include

1981 standard test Spanish language score; school days absent, and SAES self-report of grades, attachment to parents, interper-

sonal competency, school punishment index, attachment to school, victimization, and self-report of socio-economic level scales.

Counseling activity controls include 1981 standard test English reading score; school grade for math and social science; and SA

internal control and rebellious autonomy scales.Participation 1981 controls include 1981 school grade for English, and SAES

interpersonal competency, rebellious autonomony, attachment to school, and self-report of socio-economiclevel scales.

<a> See Table 1 for listing of specific Otro Camino activies for each category.

<b> Reported for 1981-82 twelfth-grade students only.<c> Regression coefficient not reported because activity type is not related to outcome.

<d> Beta unstable; interpret with caution.

* Coefficient is significant at the P<.05 level.**Coefficient is significant at the P<.01 level.

215;

Table 4

Correlation Coefficients and Standardized Regression Coefficientsfor 1981-82 School Withdrawal and Attendance Outcomes

with Number of Hours in Each Type of Otro Camino Activity

Project Goal II and III

Activity Type<a>

Academicr

betaVocational

School Report

PercentWithdrawal<b> Days Absent<c>

-.048 --<f>--<e> --<f>

Self Report

School SkippingAttendance<d> School

-.010 -.021--<e> --<e>

SkippingClasses

.022--<e>

r

beta-.057*--<f>

-.094*,,

.024

-.000--<e>

-.033--<e>

-.022--<e>

Clubsr -.063* -.090* -.072* -.052 -.076*

beta -.036 -.036 -.048 --<e> -.046

Counselingr -.049 -.040 .002 .010 -.035

beta --<e> --<e> --<e> --<e> --<e>

Participation 1981r .045 .021 -.035 -.031 -.054

beta --<e> --<e> --<e> --<e> --<e>

Note. Participation 1981 is adummy variable measuring whether or not students par-ticipated in 1981-82 rather than number of hours. N's range from 447 to 1006,depending on the patterns of missing data for variables in each equation. Each out-

come is controlled by 1981 variables that are significantly related to the type of

activity. Academic activity controls include 1981 standard test English readingscore; English grade; and SAES friend's influence scale. Vocational activity con-

trols include 1981 standard test math and Spanish reading scores; school daysabsent; school grades for Spanish, English, social science, and science; overallgrade point average; and SAES belief, interpersonal competency, school rewardsindex, school punishment index, self-concept, rebellious autonomy, school attach-ment, total delinquency, serious delinquency, and victimization scales; self-report

of gender; and 1981-82 school grade. Club activity controls include 1981 standardtest Spanish language score; school days absent, and SAES self-report of grades,attachment to parents, interpersonal competency, school punishment index, attachmentto school, victimization, and self-report of socio-economic level scales. Counsel-

ing activity controls include 1981 standard test English reading score; school gradefor math and social science; and SAES internal control and rebellious autonomy sca-

les. Participation 1981 controls include 1981 school grade for English, and SAESinterpersonal competency, rebelliOus autonomony, attachment to school, and self-re-

port of socio-economic level scales.

<a> See Table 1 for listing of specific Otro Camino activies for each category.<b> Reasons for withdrawal include voluntary and involuntary withdrawal, transfer toanother school, and reason unknown.<c> Percent based on number of days active.<d> Overall school and class attendance.<e> Regression coefficient not reported because activity type is not related to out-

come.<f> Not analyzed.

* Coefficient is significant at the P<.05 level.**Coefficient is significant at the P<.01 level.

-339- 216

Table 5 .

Correlation Coefficients and Standardardized Regression Coefficientsfor 1981-82 SAES Student Questionnaire Scales with Number of Hours

in Each Type of Otro Camino Activity

Activity type<a> Alienation

Academic

SchoolAttachment

Project Objectives

Self Interpersonal School StudentConcept Competency Effort Influence

r .031 .007 -.072* -.035 -.066* --<c>beta --<b> --(b.> -.044 --<b> -.039 --<c>

Vocationalr -.053 .054 .106* .059* .124* .092*

beta --<b> --<b> .060 .038 .067 .028

Clubsr -.031 .076* .059* .026 .077* .100*

beta --<b> .021 .013 --<b> .038 .104 **

Counselingr .009 .021 .055 .036 .072* .044*

beta. --<b> --<b> --<b> --<b>- - .044 --<b>

Participation 81

r -.005. -.027 .051 .016 .012 .:06*

beta --<b> --<b> --<b> --<b> --<b> .086**

Note. Participation 1981 is a dummy variable measuring whether or not students parti-cipated in 1981-82 rather than number of hours. N's range from 447 to 1006, dependingon the patterns of missing data for variables in each equation. Each outcome is cont-rolled by 1981 variables that are significantly related to the type of activity. Aca-demic activity controls include 1981 standard test English reading score; Englishgrade; and SAES friend's influence scale. Vocational activity controls include 1981standard test math and Spanish reading scores; school days absent; school grades forSpanish, English, social science, and science; overall grade point average; and SAESbelief, interpersonal competency, school rewards index, school punishment index, self-concept, rebellious autonomy, school attachment, total delinquency, serious delin-quency, and victimization scales; self-report of gender; and 1981-82 school grade.Club activity controls include 1981 standard test Spanish language score; school daysabsent, and SAES self-report of grades, attachment to parents, interpersonal compe-tency, school punishment index, attachment to school, victimization, and self-report ofsocio - economic level scales. Counseling activity controls include 1981 standard testEnglish reading score; school grade for math and social science; and SAES internal con-trol and rebellious autonomy scales. Participation 1981 controls include 1981 schoolgrade for English, and SAES interpersonal competency, rebellious autonomony, attachmentto school, and self-report of socio-economic level scales.<a> See Table 1 for listing of which Otro Camino activities are included in each cate-

gory.<b> Regression coefficient is not reported because activity type is not related to out-

come.

<c> Not examined.* Coefficient is significant at the p<.05 level.**Coefficient is significant at the p<.01 level.

-340- 217

*014 0-14 0+04 he.- ,

014444+ 4 r1 91,1 4 ,104=' ;$941t4 ;004 Alik ill zirnOtb /s1/10 AS6

40404 I-4 ooploon.

14444 VP6t ..01010

4461* t

4'74ri4v0ifcgo rt*10,#*/,40414 #04666460416 4160464464.16 446,444P-4W* I**410100,94P'; 04.44440 1610*-446i4,444 4:0-01,4 04 %Awl+ mi4 40104** tio tro

v# 46***46440h4o4014

*04004 +1014$4 o.kitki 44 4,4144w,$ *Ffiit:. Z.. t- r,t. U, SS It 9. scxn r. r le its

4, 4 09494-94*9099044t 4ioo04ti 44194,0271,14,M234,U.42-tiMU.S.111iiil

4i4e.00t 4#.6 4444441; 4%444%; 0444*444 6444,40,61 to4614*4411,,^4 46114* 40+ 4909, 400**090! 404401. 499 No I.044,40.4 -tootortowo,4*499. rop.44* trot/96i to OM 1 orsw*

'':1VZ.T.I.VISIlt111,,M7.1111TIT24,U.D8

.40.0141-44 op+t 400-$: AO* :4004 =)1066 =Altlo44

094140910.144411-

4099/0

0444 i.,1r14644+4* *10444

a 24. wooiii

1-)4016; A6100 4046z4,*4 az** 0 is.0,4 0

A0,01 al04t4 0041 AIN .01$!,:t001+ a,t Ike L,a,44

40ow *0.4,4400,4,*, 44*0 44, * 41***6 Av44,04o 4401%4416* 4464404 44 4094 *446644* 14t64414441444,4 44 1,0141 1444rt 1664

4,0043,4 -4****. +4466 0110,44. loth fis 404,4 4099000409, *0 04449009. b 4444094 WO 404 49411019144 10 444140.446er deig -OW ...*11* 4400,600414,0 .1%) 'Pe* 4t00+040491 +40919/ 41940 +4440.444044911* 000404 40 409/0 tillwr 64 WI 644.

*410'. Vit'Ve *10,"4104 #01*040 4400/ .i$'440111*. +WO 11%** +OM 00401.111 es**', 0404* +41.46.4 6441 1404* 144(.64 *

1414010. 44114+,000Vm 4401.0000.4 !010014114it 4.0041010 0111010 01.10400411 fokst *WA 104 11/064909019 94911130* 10/0004 40444 SOTO 111101490t49440kw itoi,*w. 4,4* 44040q4+,., tow4,004,4 Now+p4 0041o1ww) $99 popottis low4*0 44000 10404494 0044004/1 led IOU 191,44910/1694ois4400i roimplo:9**9+ *01,00 Op/0* 190410+;. 9040004 900044010404 4#4 P 44,4010P044, ~WA* 00400P407, bc090014:4041/400MW; 4/00114 41000ro09999wO, ow4g411940 4044.11000,1049; 604 4.14444,64666 64-64**6 **4444441 44 footer4 ii 1,61-41

104140, 60404.. 0** 1094490#0* 0400990+ 46f449 9$04 4400M09,4 4444 44004400 10006240 winch 606.4 66pa *Moat. 6641

004 06144A0m06* 40 110t04104- 10144004mon0 4* 4.000kt0-4 -04MAMP*0**Ii4 MNI405444*4. pom4ster64 11019004 .414091400494 10

0004900f".. -109941049,44 OOP ***44:4,w9p 1T or4,40o.mo..p009.0 owwwi *4.11.4.*4 4***40364 09144049, t40044014 404$I.,.44,0446006 4*-***441. 99.09Loo+4 4.0 OW; +44 090949 1,01 toreiho4 09604994/s *090 60* 440444041 **NW dot eirbei.

44666 049940044 *p/trOwt *WW4*44.494,90p 4444 9.1004449 0/466,6 4440 *****4 #4011101 11.9 MO4940k. 404 6160$ tatetperweeal

,01409000099494), 490910eti01/P0 1.p1rp09000904.1, 109/99,199019PPO 1** .90490044 914 .499/40'44#49999i 490+140,0144 $4,0000 Mane

/NO, -Ow .9t00,9 0 00* 40490444* ON 004+ 009**'4044100 119444404/099 #14«.4.114 09904 *11404010Pyo

Itub- #04.1 Oilaitioro lodpi04404'4+- 4+1104,14 104t 0449104000490 AI Ito** 40409101449:.Art: Olopmetowoos moww9,944,9wr 1* -if*, lo*u$-01 *movie*. ir!,-.44-014 40*. I. eiftsfi 444004 4* o4P44g**,

0 lutoW1m40* W 1019009040090 r 4104,

Table 6 (continued)

Correlation Coefficients and Standardised Regression Coefficients for1981-82 Student Questionnaire Scales with Number of Hours in Each type

of Otro Camino Activity

Belief in

Other Theoretically Relevant Outcomes

Attitutes and Social Development<b>

Practical Rebellioui Internal

School Experiences

Activity type<a> Rules Involvement Knowledge Autonomy Control Punishment Rewards

Academicr -.062* .027* -.055* w -.045 -.014* .111* -.006

beta -.047 --<c> -.043 --<c> --<c> .099** --<c>

Vocational

r .096* -.013 .086* -.044 -.008 -.107* -.009beta -.031 --<c> .063 --<c> --<c> -.058 --<c>

Clubsr -.012 .067* .007 .007 -.009 -.024 .051

beta --<c> .085* --<c> --<c> --<c> --<c> --<c>

Counselingr -.006 .081* -.024 -.009 .025 -.021 .029

beta --<c> .083* --<c> --<c> --<c> --<c> --<c>

Participation 81r .039 .083* .033 -.023 -.041 .043 .052

beta --<c> .085 --<c> --<c> --<c> --<c> --<c>

-- --

Note. Participation 1981 is a dummy variable measuring whether or not students participated in1981 -b2 rather than number of hours. N's range from 447 to 1006, depending on the patterns of missingdata for variables in each equation. Each outcome is controlled by 1981 variables that are signifi-cantly related to the type of activity. Academic activity controls include 1981 standard test Englishreading score; English grade; and SAES friend's influence scale. Vocational activity controls include1981 standard test math and Spanish reading scores; school days absent; school grades for Spanish,English, social science, and science; overall grade point average; and SAES belief, interpersonal com-petency, school rewards index, school punishment index, self-concept, rebellious autonomy, schoolattachment, total delinquency, serious delinquency, and victimization scales; self-report of gender;and 1981-82 school grade. Club activity controls include 1981 standard test Spanish language score;school days absent, and SAES self-report of grades, attachment to parents, interpersonal competency,school punishment index, attachment to school, victimization, and self-report of socio-economic levelscales. Counseling activity controls include 1981 standard test English reading score; school gradefor math and social science; and SAES internal control and rebellious autonomy scales. Participation

1981 controls include 1981 school grade for English, and SAES interpersonal competency, rebelliousautonomony, attachment to school, and self-report of socio-economic level scales.

<a) See Table 1 for listing of which Otro Camino activities are included in each category.<b) See SAES Student Questionnaire Manual for explanation of these variables.<0 Regression coefficient is not reported because activity type is not related to outcome.

* Coefficient is significant at the p<.05 level.**Coefficient is significant at the p<.01 level.

-342-

r

2 1 3

Table A

Means and Standard Deviations for 1981-82 Academic AchievementOutcomes for Each Type of Otro Camino Activity

ActivityTypes

Academic

Span.

School Grades

Soc.

Eng. Math Sci. Sci.

Outcomes for Project Goal I

Standardized Tests

Span. Span. Engl.

Read. Lang. Read. Math

Self-Reported

Schl. Read.

Grade Abil.

H.S.Crud))

Vocation

Have Pa

Job Wo

Mean 1.66 1.69 2.04 1.73 1.74 39.74 34.17 30.16 20.32 2.11 1.52 .12 .48 .2

SD 1.01 1.05 1.06 1.06 1.02 12.86 10.80 12.38 7.62 .93 .87 .33 .73 .4

N 231 230 213 202 195 197 195 196 203 201 200 236 199 19

VocationalMean 2.10 2.10 2.24 1.80 1.89 41.32 37.12 30.11 23.38 2.43 1.56 .52 .20 .2

SD 1.06 1.10 1.07 1.06 1.03 13.48 12.71 10.41 8.21 .98 .83 .50 .52 .4

N 229 226 182 124 129 203 202 203 209 194 '193 231 192 19

ClubsMean 1.70 1.86 2.17 1.86 1.83 39.73 34.07 29.59 18.76 2.25 1.47 .08 .32 .2

SD .95 1.00 .96 .92 .93 13.25 11.37 11.58 6.87 .85 .89 .28 .62 .4

N 354 355 337 326 309 313 302 304 308. 302 300 362 302 3(

CounselingMean 1.64 1.73 2.06 1.71 1.75 38.34. 33.59 29.82 18.32 2.14 1.47 .03 .32 .:

SD .95 1.01 .97 .95 1.00 13.74 11.30 12.23 6.78 .86 .92 .17 .62 .4

N 244 240 234 234 221 208 209 1208 214 210 207 246 208 2(

Participation1981

Mean 1.80 1.84 2.16 1.41 1.62 42.03 34.96 29:67 21.50 2.22 1.54 .21 .35 .

SD 1.08 1.05 .98 1.03 1.00 14.23 11.91 11.82 8.04 .86 .91 .41 .66 .1

N 149 148 131 110 115 117 116 1116 120 110 112 157 111 1

a See Table 1 for listing of specific Otro Camino activities for each category.

b Reported for 1981-82 twelfth-grade students only.

Table B

Means and Standard Deviations for 1981-82School Withdrawal and Attendance

for Each Type of Otro Camino Activity

Activity typea

Academic

School Report

PercentWith-daysdrawalb absentc

Schoolnonatten-

dance

Self Report

Skippingschool"

Skippingclassese

Mean .042 .024 1.116 .658 1.213

SD .202 .056 .740 1.084 1.012

N 236 235 199 199 202

VocationalMean .056 .033 1.163 .801 1.162

SD .231 .135 .767 1.140 .960

N 231 230 196 196 197

ClubsMean .044 .025 1.107 .610 1.107

SD .206 .080 .704 1.017 .883

N 362 362 307 308 308

CounselingMean .037 .023 1.165 .722 1.122

SD .188 .077 .745 1.090 .903

N 246 244 212 212 213

Participation 81

Mean .108 .055 1.133 .743 1.044

SD .312 .159 .773 1.132 .876

N 157 157 113 113 114

a See Table 1 for listing of which Otro Camino activities

Each category.Reasons for withdrawal include voluntary and involuntary

fer to another school, and reason unknown.Percent based on number of days active.Individual item ranges from 0 to 4 with 0 = no days cut

weeks and 4 = more than 10 days cut in the last 4 weeks.e Individual item ranges from 0 to 5, with 0 = never cutscuts classes almost every day.

-344-

are included in

withdrawal, trans-

in the last four

classes and 5 =

Table C

Means and Standard Deviations for 1981-82 SAES Student QuestionnaireScales for Each Type of Otro Camino Activity

Academic

Alienation

Project Objectives

School Self InterpersonalAttachment Concept Competency

SchoolEffort

Mean .346 .762 .645 .637 .583

SD .227 .212 .173 .268 .263

N 177 186 161 175 181

VocationalMean .299 .809 .700 .735 .623

SD .222 .198 .180 .234 .280

N 179 189 159 180 188

ClubsMean .314 .775 .671 .683 .615

SD .224 .209 .179 .278 .267

N 272 288 237 276 289

CounselingMean .316 .770 .684 .700 .584

SD .215 .214 .173 .247 .279

N 185 199 158 183 196

Participation 81

Mean .329 .762 .678 .700 .573

SD .216 .223 .168 .242 .282

N 93 101 84 102 106

Note. See Table 1 for listing of which Otro Camino activities are includedin each category.

-345-

Table D

Means and Standard Deviations for 1981-82 Student QuestionnaireScales with Each Type of Activity

ActivityTypea

Academic

Self Report of Delinquencyb

Total Ser. Drug SchoolDelin. Delin. Usage Victm. Suspn.

Other Theoretically Relevant Outcomes

Social Relations Attitudes and Soc. Dvlpmnt.

Attch. Neg. Belief Rebel. In-

Police to Peer in Pract. Auto- ternal

Cntct. Prnts. Influ. Rules Invol. Knowl. nomy Cntrl.

Schl. Expe

Punish Rew

Mean .069 .046 .119 .086 .158 .021 .642 .141 .778 .217 1.386 .345 .426 .101 .32

SD .117 .118 .192 .169 .366 .144 .262 .157 .209 .167 .479 .322 .209 .193 .28

N 182 181 179 182 183 236 197 177 175 179 179 171 176 183 185

VocationalMean .056 .034 .106 .060 .086 .004 .661 .103 .841 .185 1.520 .369 .398 .048 .29

SD .116 .109 .195 .143 .281 .066 .256 .141 .178 .139 .449 .319 .204 .145 .29

N 191 191 189 188 186 . 231 193 187 182 183 183 178 176 189 190

Clubs

Mean .053 .036 .082 .082 .156 .008 .669 .141 .779 .230 1.436 .388 .423 .092 .31

SD .110 .102 .173 .172 .363 .091 .266 .172 .203 .181 .456 .328 .219 .196 .32

N 293 293 268 287 276 362 303 290 277 284 281 273 273 286 287

CounselingMean .045 .031 .073 .072 .157 .008 .691 .145 .776 .226 1.465 .404 .429 .085 .29

SD .097 .090 .152 .156 .365 .090 .249 .171 .202 .192 .450 .325 .225 .176 .29

N 192 191 191 196 197 246 209 193 185 192 190 182 184 196 197

Participation1981

Mean .065 .043 .116 .078 .173 .019 .651 .144 .782 .234 1.507 .367 .395 .124 .3

SD .123 .109 .205 .180 .381 .137 .273 .184 .231 .166 .414 .360 .251 .233 .3C

107 107 105 100 98 157 112 107 101 105 101 98 99 102 101

a See Table 1 for listing of specific Otro Camino activities for each category.

b See SAES Student Questionnaire Manual for explanation of these variables.

223

Project PATHE: Second Interim Report

D. C. Gottfredson

Abstract

Interim results suggest that PATHE, a school-based program that com-bines organizational change, direct service to high risk youths, andstrong management, significantly increased academic performance for tar-geted individuals. In one of the four middle schools, target studentsreported significantly less delinquent behavior than did the controlgroup.

The program appears to have had a positive school-level effect on

student alienation, self-concept, and belief in rules as well as onschool safety, victimization, and teacher morale. Delinquent behaviorwas lower in all three PATHE high schools in 1982, although delinquent

behavior was higher in the control high school. A nonsignificant trendin favor of the program is evident on several additional PATHE goals andobjectives.

Implementation data show much variation across program components andacross schools in the fidelity of the implementation of the program.The expected association between level of implementation and success ofthe program was not, however, observed. Plans for strengthening theprogram and the monitoring procedures for the 1982-83 school year arediscussed.

PATHE--Positive Action ThroughHolistic Education--effectively com-bines school-wide organizationalchange, direct service to high-riskyouths and rigorous management andevaluation. The program was .devel-oped by the Charleston County SchoolDistrict, Charleston, S.C., and hasbeen operating in seven middle andsenior high schools for two years.This report updates the previous

interim evaluation report for the

PATHE program (D. Gottfredson,1982). The background informationincluded in the first report,

including descriptions of the

Charleston community and the PATHEprogram, will not be repeated here.This report summarizes major programchanges that occurred between Sep-tember, 1981 and August, 1982,

reports results of analyses of

implementation and outcome data forthe same year, and discusses impli-

-347-

cations of the data for programoperations.

Major Program Changes

Staffing

The PATHE staff remained rela-tively stable during the 1981-82school year. Both specialists at

St. John's High school were replacedin August, 1981, and the StudentConcerns Specialist at Rhett Middleschool was replaced mid-year... Thereplacement specialists resembledtheir predecessors in qualificationsand backgrounds and the staff

changes seem not to have been dis-ruptive to the program. A cut-backin funding resulted in the lay-offof five specialists at the end ofthe 1981-82 school year, and the

evaluator position also turned overat that point. The effect of the

On4.44

PATHE

reduction in staff at the schools

for the 1982-83 year will be

assessed in the final report for

this project. The turnover in the

evaluator position was smooth and

caused no disruption to the program.The new evaluator is knowledgeableabout research and evaluation tech-niques, and she is skilled in pro-gram management.

Program Changes

The goals, theory, and objectivesof the program remained unchangedduring the 1981-82 year. One new

program component-7Student TeamLearning--was added and three--theSpeakers Bureau, Summer Program toAccess Creative Experiences, and theM.U.S.C. Career Awareness and Train-ing Programs--were dropped.

In Spring, 1982, the program man-agers reorganized the 28 PATHEinterventions into five major cate-gories of interventions. This

reconceptualization of the program,which amounted to combining inter-ventions that had previously beenregarded as separate program compo-nent, helped to organize and focusprogram operations. A descriptionof the PATHE interventions as reor-ganized follows. Descriptions of

the components are provided only forthose interventions which changed

significantly or were added sincethe first interim report, or whoseoriginal descriptions required ela-bor ation.

PATHE Organizational Structure

This school level intervention isintended to establish and maintainan organizational structure whichfacilitates shared decision makingamong community agencies, students,teachers, school administrators andparents in the management planningfor the school. The intervention

provides training in assessing

needs, researching problems,

defining objectives, developing andimplementing plans, assessing pro-

gress and redefining strategies.The sub-components of the interven-tion follow.

Curriculum Review and Revisionprovides a structure for ongoingreview and revision of the curricu-lum policies. It identifies train-ing and material needs, provides forfaculty inservice training, and

operates a "resource room."

Didcipline Policy Review and

Revision provides a structure forongoing review and revision of the

¶'discipline policies. This includesthe establishment and maintenance ofa discipline referral system and

strategies to generate and publicizeschool and classroom rules. Thisactivity also identifies training

and material needs, provides for

faculty- inservice training, - andidentifies materials to be stockedin the "resource room."

-348-

Leadership /Support Teams receivetraining in leadership skills and

are given the opportunity to parti-cipate in the planning and implemen-tation of school improvement proj-ects. Each team member gains

management experience in assessingneeds, stating objectives, planningand implementing strategies, and

assessing progress toward goals.The four teams are the CurriculumSupport Team, Student Concerns Sup-port Team, Student Leadership Team,and the Parent Leadership Team.

Business/Education Partnerships(high schools only) are formedbetween the school administrationand local businesses. Businesses

offer management assistance and

other resources to the schools.

Faculty-Administration Team

Building includes strategies aimed

at involving administrators andteachers in school improvementactivities, and providing trainingfor school staff in areas identifiedas weak.

School-Wide Academic Innovations

Some academic services that hadpreviously been defined as school-wide services, such as specialisttutoring, "outside" and peer tutor-ing, and diagnostic testing/indivi-dualized instruction were focusedprimarily on target students during1981-82. Two other services, the

middle school Exploratory programand the Curriculum Guide developmentand distribution component weretaken over by the school district.PATHE continued to assist with theseprograms in several schools, but

they were not major foci of the pro-gram. The remaining school-wideacademic components are Study SkillsTraining, the Reading ExperienceProgram, the Test-taking Skills Pro-gram, the Field Trip Program, and

Student Team Learning.

Student Team Learning is a set ofinstructional techniques that trans-lates into classroom practice sev-eral well-known principles of learn-ing. Team members in STL classroomsstudy and drill together and preparefor quizzes or cross-team competi-tions. Rewards for academicimprovement are given to the hetero-geneous student teams based upon theprogress of the team. The tech-niques have received positive evalu-ations for enhancing learning, self-concept, liking of school and

increasing cross-race and cross-sexfriendships (Slavin, in press).During the 1981-82 school year spe-cialists in all PATHE schools weretrained in STL methods, as were

teachers in four of the sevenschools. Teachers in three schoolstried out the methods in their

classrooms before the close of the

PATHE

school year. Additional teacher

training and careful monitoring ofthe use of the techniques in class-rooms are planned for the 1982-83school year.

School-Wide Affective Innovations

These interventions are targetedat general school climate improve-ment. Staff counseling, previouslyconsidered a school-wide interven-tion, is now being directed primar-ily toward target students. The

subcomponents of the school-wideaffective intervention are the

School Pride Campaign, whichincludes activities aimed at improv-

,ing the overall image of the school,and the Extra-Curricular Activitiesprogram and the Peer Counseling or"Rap Sessions" program.

-349-

"Rap sessions" are sessions in

which adults present information ontopics of concern (such as drug useor teenage pregnancy) to students

and students react to and discussthe topic. These sessions werefound by the specialists and programmanagers to be an acceptable substi-tute for peer counseling sessions,which had proven difficult to imple-ment.

Career-Oriented Innovations

Programs to increase transitionsto college and work (high schoolsonly) include the Career ExplorationPrograms - -FACET and the Trident TechFall and Spring Programs, opened towomen during the 1981-82 school

year--end the Job-seeking Skills

Program.

Services to Target Students

This component was strengthenedin Spring, 1982. The followingparagraphs describe the redesignedprogram. The actual services pro-vided to target students during the

0

20'1.0

PATHE

1981-82 school were much less

intensive and less programmatic thanis implied by the following discussion. The implementation section ofthis report will discuss in detailactual services to target studentsduring the 1981-82 year.

The target student service component is intended to provide affective and academic services to a subpopulation of "target" studentsidentified as in need of special

services. The idea behind this

strategy is that each school has agroup of students that will not bereached by the organizationallevelPATHE components. These studentsneed highly individualized treatmentin order to get them to the pointthat they will benefit from the

classroom and school level PATHE

innovations. The direct servicecomponent is intended to identify

appropriate target students, diagnose their needs, define behavioraltreatment objectives, prescribe

appropriate intensive services,

monitor progress toward objectivesand frequently reassess the appropriateness of the treatment for

addressing the problem.

Program Management and Evaluation

The program managers implementeda true experiment, developed a database for management information, andestablished implementation standardsfor each program component during

the 1981-82 year. Each of these

activities enhanced the manageability and the evaluatability of the

program.

Evaluation design. StudenEs eligible for PATHE direct service wererandomly assigned to target and control conditions in September, 1982

from a pool of eligibles made up ofall students in each PATHE schooland the sixth grades of each feederelementary school who had been sus-

pended during the 1980-81 school

year, were referred to the PATHEprogram by a teacher, or whoseachievement test score (the CTBS

total test score) fell in the bottomtwentyfifth percent of scores forthat school. Teacher referrals wereobtained by sending the form shownin Table 1 to every homeroom teacherin grades five through eleven of thePATHE and feeder schools. The criteria listed at the bottom of the

form are the same selection criteriathat were established for the program at the time of the initial

grant 'proposal.

This procedure resulted in pools"of different sizes for the sevenPATHE schools. The pools for the

high schools, but not the middleschools, were of sufficient size toconstitute target and control groupsof equal size. In the worst middleschool case, 71% of the pool had tobe randomly- assigned-to- the -treatment condition in order to fill the100 treatment slots required as a

condition of the grant.

Tables 2 and 3 show the resultsof a postrandomization check. Only

one of the comparisons shows a significant pretreatment differencebetween target and control groups.

At least one significant differenceat the p<.05 level would be expectedto occur by chance if the groupswere, in fact, equivalent.

Although implementation data to

be presented later show a partialbreakdown of the experiment in someschools, the design was implementedwith enough integrity to enable thedetection of program effects. The

program managers have taken steps tomonitor the design of the experimentduring the 82-83 school year and

have established clearer expecta-tions for level and quality of services to target students. These

program changes, coupled with the

-350

decision to retain the 81-82 targetand control groups, should result inmore dramatic treatmentcontrol differences at the end of the 1982-83school year.

Management information system.

The evaluator established a management information system which organized data on level of services to

all students in the school. Datacollected in this way were used toreport on implementation levels during the Fall '82 semester and againfor the entire year (included in

this report). The midyear reportbased on these data exposed unexpected low levels of direct serviceto target students. It precipitateda major restructuring of the directservice component of the program toallow for more intensive and programmatic target student services.

Implementation standards. The

third major improvement in the management of the project was the

establishment of implementationstandards for each program component. The program implementersdefined the standards and used themto monitor the level of implementation of each program component. The

ratings derived from these standardswill be discussed in the implementation section of this report.

Effectiveness

The effectiveness of the PATHEprogram is assessed at two levels:Comparisons of 1980-81 to 1981-82school year school averages and comparisons of randomly assigned treatment and control students on measures of PATHE goals and objectives.

Measures are taken from the SAESstudent and teacher surveys (Gottfredson, Ogawa, Rickert, & Gottfredson, 1982; Gottfredson, Gottfredson,& Cook, 1983) and Charleston CountySchool District records of grades,

California Test of Basic Skills

351

PATHE

scores, attendance, suspension,expulsion, and disciplinary incidents. The item content and psychometric properties of all surveymeasures are described in Chapter 3of this volume.

Change from 1981 to 1982 wasassessed for all outcomes measuredby the SAES with the exception ofdelinquency for the middle schools(the Charleston project had the

selfreport delinquency itemsdeleted from the Spring, 1981 surveyfor their middle school students).Bar charts showing' the amount of

change on each outcome for eachschool are available (Gavurin, Hybland Gottfredson, 1982) and have beenused as a formative evaluation toolby the PATHE project implementers.This report discusses only outcomesdirectly tied to project PATHE'sgoals and objectives.

Tables 4a through 4c show differences between the 1982 and the 1981school scores on survey measuresrelevant to the project's goals andobjectives. Tables 5 and 6 showchange from baseline to 1982 on

measures of goals and objectives forwhich we have data from official

school records. Tables 7 through 10show means for target and randomequivalent control students on all

goals and objectives. The discussion which follows will summarizethe information in these tables.

Differences from 1981 to 1982 andbetween the target and control students are generally not large.

Although few reach statistical significance according to conventionalstandards, some consistent trends

warrant discussion. Throughout thefollowing pages I will consider thedirection of the observed differenceas well as the probability of

observing such a difference if project PATHE is not effective.

PATHE

tstatistics for comparisons of

target and control groups are basedon the observed means and standarddeviations for each group on Spring,1982 measures of goals and objectives. tstatistics for comparisonsof 1981 and 1982 student compositional measures (i.e., school averages on characteristics of the students) are based on the observedschool means and standard deviationsfor each year. For comparisons of1981 and 1982 school climate measures (i.e., student and teacherreports of the school environment),the "tstatistic" is the ratio of

the difference between 1982 and 1981scores to the standard error of

measurement of the 1982 scores forall but St. Paul schools in theOJJDP initiative. Climate measuredifferences that exceed twice theirstandard error of measurement may beregarded as dependable.

The following paragraphs useinformation about the direction ofthe difference score to assessschool level effects of the PATHEprogram. The reader should be awarethat some of the differences are

close to zero, but are still interpreted as improvements or regressions, depending on the sign.

Delinquency

School level. Measures of delinquent or disruptive behavior--suspensions and selfreported delinquency--show the same pattern of

results for high schools (Tables 4cand 5). Each of the three PATHEhigh schools experienced a declinefrom 1980-81 to 1981-82 in both thenumber of suspensions and selfreported delinquent behavior. For St.John's high school, the reductionsin Drug Use and Total Delinquencywere statistically significant. Forthe other two high schools, the

decline in suspensions was statisti-

cally significant. The high schoolcontrol school experienced an

increase in suspensions and in

selfreported delinquent behavior.

Progress in the middle schools onthis outcome is more difficult to

assess: The selfreport delinquencyitems were deleted from the Spring,1981 middle school survey, and

aggregate level suspension data fromofficial records (Table 5) do notaccord with selfreported suspensions from the survey. (Table 4a).Official records show an increase insuspensions for three of the fourPATHE middle schools as well as

their comparison school. Selfre" ports suggest that suspensions weredown in two of four PATHE middleschools as well as in the comparisonmiddle school.

-352

Alternative measures of schooldisruption shown in Table 4c show anincrease in- students'- and teachers'perception of safety and a decreasein student reports of victimizationin six of the seven PATHE schools.Several of these changes were statistically significant. Both comparison schools experiencedincreased student victimization butstudents in one and teachers in bothcomparison schools reported moreschool safety.

Individuallevel. Treatmentcontrol student comparisons on measuresof delinquency (Table 7) show littleevidence of a program effect on thisoutcome. Although one of the sevenschools showed a sig, ificant reduction in delinquency among targetstudents, the direction of the

treatmentcontrol difference favoredcontrol students about as often astarget students in the otherschools. Comparisons of suspensionsand disciplinary incidents in the

school show similar results.

229

Attendance

School-level. Attendance ratesin the Charleston County schools aretypically high (in 1980 the averagedaily attendance rates for the PATHEand control schools ranged from 89to 94 percent). Ceiling effects,the initiation during the 1981-82school year of a strict district-wide attendance policy, and the factthat most school personnel (includ-ing PATHE specialists) do not per-ceive nonattendance to be a problem,all reduce the likelihood of prograMeffects on attendance rates. Tables4c and 5 show that attendance did infact increase in all three PATHEhigh schools and one or two of thePATHE middle schools (depending onthe source).

Individual level. The target-control comparisons are promisingfor both high and middle PATHE

schools (Table 8): Although the

differences never reach statisticalsignificance, target students in

five of the seven schools have a

lower proportion of unexcused daysabsent to total days enrolled thando control students. The other twoschools show no difference betweentarget and control students.

Academic Achievement

School-level. All Charlestonschools, PATHE and control, improvedfrom 1981 to 1982 on California Testof Basic Skills scores, and this

improvement was more pronounced in

the middle than in the high schools(see Table 6). The extent to whichproject PATHE contributed to theimprovement is unclear, but the evi-dence favors PAM: By and largethe improvement in Charleston highschools on CTBS test scores was min-imal and did not reach statisticalsignificance. One PATHE highschool--Burke--was a clear exceptionto this rule. Table 6 shows a con-

PATHE

sistent statistically significantincrease in test scores for Burke.Evidence from many sources--ratingsof the strength and fidelity of

implementation and observations andinterviews with the Burkespecialists--show that Burke imple-mented the school-wide academic com-ponents of the program with muchmore fidelity to the program planthan did the other PATHE highschools.

Individual-level. Other evidencefavoring a positive effect of the

PATHE'program on achievement comesfrom target-control comparisons(Table 9). Comparisons of the per-

".,cent scoring in the bottom nationalquartile of the 1982 CTBS show thatin only one school did the controlstudents fare better than the targetstudents. The overall treatment-control difference reaches statisti-cal significance in Language, an

area that several PATHE specialistsfocused on during the year, as wellas for the total battery. The

direction of the difference favorsthe target students for all subar-eas. Target-control comparisons ofschool grades also favor PATHE in

six of the seven schools, and thedifference for percent receivingfailing grades averaged across allsubject areas reaches statisticalsignificance.

-353-

Transition to Employment and Post-secondary Education

This outcome will be assessed inthe final report, but see the "Edu-cational Expectations" objectivebelow.

PATHE Objectives

The PATHE managers identified anumber of intermediary outcomestoward which the program is aimed.

These intermediary outcomes, or

"objectives," are primarily student

230

PATHE

and teacher attitudes, but also

include certain teacher behaviors.Relevant comparisons for each of thePATHE objectives are shown in Tables4a, 4b, 4c, and 10. Because many ofthe objective measures are somewhatredundant, I will summarize only themajor dimensions here.

Teacher Morale (Teacher Reports)

Every PATHE school and the middleschool comparison school experiencedan increase in teacher morale. Thehigh school comparison schooldeclined in teacher morale.

Reports of Planning & Action andSmooth Aministration

Both teachers and studentsreported on the extent to whichtheir school is characterized by

planfulness and innovative atti-tudes, and teachers reported ontheir perceptions of the extent towhich the school is run smoothly andeffectively. According to studentsin four of the seven PATHE schools,the degree of planning decreasedfrom 1981 to 1982. Three of the

four schools that did not improvewere already above the 85th percen-tile for schools in the study,

though. Students in both comparisonschools reported a decline in plan-ning.

Teachers reported an increase inplanning and in smooth administra-tion in five of the PATHE schools.Both comparisons schools declined onboth of these teacher measures.

Teacher Expectations for StudentAchievement (Teacher Reports)

The direction of change was posi-tive in four of the seven PATHE

schools--two middle and twohigh--and in one of the two controlschools.

Student Reports of Rewards and Pun-ishments in School

Students in five PATHE schoolsreported an increase in rewardingand a decrease in punishing experi-ences in school from 1981 to 1982.Both comparison schools experienceda decrease in these intermediaryoutcomes. Of the two PATHE middleschools with increases in students'reports of school experiences as

punishing, one was already extremelylow on this measure.

Tre'atment/control comparisons ofrewarding and punishing experiencesin the middle schools showed differ-

t'ences favoring the treatment stu-dents in three of the four schools.One comparison reached statisticalsignificance. In only one of the

three high schools did the directionof the treatment/control differencefavor the treatment students.

- -

Students Reports of Fairness and

Clarity of Rules

-354-

Three of the PATHE schools

increased on both measures, and

another two increased on students'perceptions of the clarity of the

rules, though not the fairness. Two

of the seven PATHE schools declinedon both measures, and both compari-son schools declined on both meas-ures. The magnitude of the changesfrom year to year is noteworthy.The two PAM schools that declinedfrom 1981 to 1982 on the Clarity ofRules scale declined only slightly(in fact, the difference score

equals zero when rounded off). Thedecline experienced in the compari-son schools was greater. The majorPATHE activities aimed at increasingrule fairness and clarity occurredduring the 1980-81 school year. The

PATHE discipline referral system wasinstituted during that year, and

several PATHE schools had school-wide campaigns to generate and pub-

oho* 4.4 aligoornms rules.

*Lift op 410144 TOO. the 1114%a)01: :ileouarti samowdolaw,

1114414 4,10.4 Olkil 434)4'44 Oat144 tow iottliiapool as *efte1PIleapolmcmosi pests ersi54tion. tiff;;."

0000, Ots.* sose ors,

14911VONOLUMPitilliatt

444 Oet** WOW* echoed* am4 eve0440t itotaO reeses4 twood em thisootoose. Soh oemperteem school*thotShme4. t<hN two MAIMS highwho*. OA -4441 oot iSIKOW19 VOL!all P*44, d60100 the 144th peosetiletee scOoot4* to the study, awed

41h044$ 40c4 toe toe *light.

lfto Cropeemeorieeotrel comfort***'wore* 414ORC oedema, to three of

too *4444e old to two of Ole$40 *Opel*. eltheceb es'NOW etotittaeo.

tatimmit

#441 Nes 04 tut 00100iF

40,4101W016 thosimaraditmelrel cooper-MOP Slowed PAWS itedeote 4 ell*444., embeds mod to ese Mgt*40043

AtimosAms

44$ ISMS ottool topureed issottomor. la woo Oldie echo,*

OW Swim** toe oubstemttal.irke *Me 4444 cemosrteso ;Wheelotpofetoroo4 o sehootoottel Waves,to ei hooettas m00% eledesco sod toe6W1 efdonol conoorttpeo leet4i4400144.

%roe eledesole 4* hoe of the

beef OSAs ochred* soil tit goo allitktrite 44 11 loos oltomated than

Oitiote fOooles4o.

PAMIE

ILItti.20111

£11 but nee of as Charlestonschool sad controlscoredabove the Si* mai five scored abovethe 94th percentile on involvementfor all imbeds La the study on theSerino MI survey. Althoughimprove meet on this measure is

reliable for the Charleston4. stools* three ram and one cooper-isms school did improve on this

meovre. It is difficult to deter-s i** hew much of an effect the PAUL Zprogram had we this outcome.Although activities aimed attummies involvement were carriedout is the MUTE schools, poor docu-

vmentaties of the percentage of stu-dents Lavolito4 in tbe activities andthe fropeacy of students' partici-pates in extracurricular activitiespre vest a systematic evaluation ofthe program's effects in this area.thsfortumstely, attempts to improvethe sonitorieg of extracurricularactivities for the 1912-13 school

year are also failing.

Tercet-control comparisons on

oldest? reports of involvement are41460 iscoolusive. The comparisonfevers PAM stoniest's La three mid-dle schools aid one high school, butsever roams statistical signifi-cameo. Also, the pattern of differ-..cos deem net accord with the

implemestatiee data. The PAULschoots for which target students'reports of ievolvoneut exceeded

theft of control students by the

greolest margin were typicallyschnels whose records showed the

1.044 target-control difference in

actual participation in extracurri-cular activities. Several special-ists explained that their records ofextracurricular activities were

inaccurate.

-Mt

232

PATHE

Attachment to School

The program had positive effectson this key, intermediary outcome inthe middle schools. All improved onthis outcome while the comparisonmiddle school declined substan-

tially. Only one of the PATHE highschools improved on this outcome,

but of the two that declined one

started out and remained above the94th percentile. Middle school tar-get students reported higher levelsof attachment in every school, butonly in one high school did this

comparison favor the program (in asecond high school. there was no dif-ference). The high school with thehighest school-wide attachment had

an apparent negative effect on itstarget students (p<.01).

Educational Expectations

Students in two middle schools

and two high schools in the PATHEprogram reported increased educa-

tional expectations. Students in

one comparison school also reportedincreased expectations. In only onetreatment/control comparison did the

direction of the difference favor

the PATHE target students.

Belief in Rules

All but one of the Charleston

schools (a PATHE school) increased

on this measure. Roughly half of

the treatment/control comparisons

favored the PATHE students.

A Closer Look at Burke

The most surprising finding of

this interim evaluation is a consis-tent trend toward a negative effecton target students in Burke, one ofthe PATHE high schools. The resultis doubly troubling because the

Burke operation is regarded as theclosest approximation (among high

schools) to the ideal implementationof the PATHE program. The school

!

-356-

stands out on all measures of imple-mentation for school-wide activi-ties, and its improvement from 1981to 1982 on PATHE goals and objec-tives was second only to Haut Gap,the middle school which also imple-mented the program with excellence.How might Burke's negative resultsbe explained? There are several

possibilities:1) The control students received

more treatment than the tar-get students,

2) The "worst" control studentsdropped out of school whiletheir equivalent target stu-dents remained in school,thus making the groups none-quivalent (because dropoutswere not included in the datacollection), or the "worst"control students did not takethe spring 1982 survey, and

3) The treatment was harmful.Each will be discussed in turn.

- - - -

Table 11 compares the total num-ber of contacts with specialistsrecorded for the 1981-82 school yearfor Burke High School and all otherPATHE schools. It shows that con-trol students did not receive moreservices than treatment students at.Burke. This is true also for the

subset of target and control stu-

dents who took the Spring. survey.

The PATHE specialists in the schooladmitted that they may have beenmore careful to record contacts withtarget than control students because

they were concerned with "getting

credit for" their target student

contacts. The underreporting of

control student contacts would havehad to have been blatant in order toexplain the differences on outcomesfavoring control students, though.

The second hypothesis, that of

differential attrition from school

or from the survey administration,is also not supported by the data.The "original" and "final" columns

o33

in Table 2 show no pre-treatmentdifference between CTBS scores fortarget and control students who leftschool before Spring, 1982 when theCTBS test was administered. Accord-ing to these figures, dropout did

not change the equivalency of the

target and control groups. Similaranalyses comparing 1981 CTBS meas-ures for target and control studentswho took the Spring, 1982 surveyalso disconfirmed the differentialattrition hypothesis. Among stu-dents who took the 1982 survey at

Burke, control students did not havea pre-treatment "advantage" overtarget students.

Might the treatment at Burke havecaused target students to reportless attachment to school, score

lower on achievement tests, have

lower expectations for completinghigh school and have fewer jobs thanequivalent students who received onethird as much treatment? Observa-tions of the Burke specialists in

action imply that the oppositeshould occur. Both specialists areexperienced teachers, have excellentrapport with the students, teachers

and administrators in the school,

are perceived as extremely hard-working individuals, are advocatesfor the students while being firmdisciplinarians, and above all are

dedicated to their work. Theirteaching style seems to accord withthat suggested by research: Their

instructional pace is rapid and

their tutoring group size is alwayssufficiently small to allow for

individualized attention.

Two hypotheses about the way thePATHE treatment was administered atBurke were examined: Differencesbetween Burke and other schools in

the standard deviation for number ofcontacts (see Table 11) raises sus-picion. The frequency distributionsfor total number of contacts show

that at Burke a few students (bothtreatment and control) had extremely

-357

PATHE

high numbers of contacts with the

specialists. Although the distribu-tions for most of the students in

Burke and other schools are similar,the top 5% of the distributionreceived between 45 and 60 contactsat Burke and only between 18 and 45contacts at the other schools. Tar-get students were more likely to beamong those students receivingextreme amounts of service. The

hypothesis that a high amount of

contact with specialists causes tar-get students to dislike school wasexamined and disconfirmed. Numberof contacts with specialists is

slightly positively related to

attachment to school for target stu-dents (r=.07, p=.08) and slightlynegatively related for control stu-dents (r=-.09, p=.06) for all PATHEschools. The associations are

stronger at Burke, reaching statis-tical significance at the p<.05

level (target: r=.21; control:

r=-.28). Target students whoreceived extreme amounts of serviceliked school more, not less, than

those receiving less service.

One final difference between theway direct services are provided atBurke and other schools is notewor-thy. Tables 15 through 21 show thatBurke is the only PATHE school whichprovided more academic than affec-tive types of services to targetstudents. The difference may haveresulted from differences in record-ing practices across schools, but

observations and interviews with thespecialist at Burke also suggest

that the emphasis is on academics.The Burke specialists frequentlywork as a team with small groups ofstudents, tutoring and providingcounseling as needed. Tutoringseems to be the major focus in thesesmall group sessions. PATHE spe-

cialists at other schools work moreindependently and schedule entire

sessions solely for counseling.

234

PATHE

The hypothesis that the ratio ofacademic to affective services is

central to the success of the PATHEprogram is currently being tested.This ratio is a variable central tothe PATHE philosophy: The programrests on the notion that studentsneed both academic and affectivesupport. The program managers choseto compromise this notion when theyresponded to the federal budget cutby severely curtailing affectiveservices in half of the PATHEschools for the 1982-83 school year.

Summary of Outcome Evaluation

The effects of the PATHE programdiffer according to target (i.e.,

treatment-control comparisons vs.

school-level change) and level

(i.e., middle vs. high schools).

Target Students

Program effects on target stu-dents are more evident at the middlethan at the high school level. In

general, target students in the

PATHE high schools seem not to havebenefited from the program any morethan did the typical, nontargetedstudent in the school during the

1981-82 school year. The resultsfor academic achievement are an

exception, though: Target studentsin all PATHE schools save one highschool fared better than their ran-dom-equivalent controls on this out-come. Target students in all middleschools attended school more oftenand out-achieved the controls, and

they reported higher levels of

school attachment and more positiveself-concepts. Only for the

achievement outcomes do the differ-ences reach statistical signifi-cance, though.

-358-

School-level

Comparisons of 1981 to 1982

changes in school averages on

PATHE's goals and objectives must beapproached with more skepticism thancomparisons of randomly assignedtreatment and control students.

Even when improvement is observedfrom year to year we cannot be surethat the PATHE program caused the

improvement. Schools were not ran-domly assigned to treatment, and wehave only two Charleston schools

with which to compare the PATHEschools.

School-wide improvements on PATHEgoals and objectives are generallymore evident at the high than at themiddle school level, although the

"most improved" school is a middleschool. PATHE high schools improvedon 80% of the outcomes examined,

while the PATHE middle schoolsimproved on 69%.- -A1:1-three- PATHEhigh schools and two of the four

PATHE middle schools improved siAni-ficantly on several of the outcomesexamined in Tables 4a and 4b.

The Sign Test (Siegal, 1956) canbe applied to the results in Table4c to determine the probability thatthe observed pattern of changescould have resulted by chance. The

direction of 1981-1982 change is

treated as a binomial variable witha probability of improving of .5.

Several of the findings discussedabove are found to be improbablewhen this test is applied. For

example, the observed decrease in

student alienation in every PATHE

school and increases in teacher

morale and student positive self-concept in every PATHE school wouldoccur by chance only eight times inone thousand. The patterns of

increases and decreases for stu-

dents' and teachers' reports of

safety, student victimization andbelief have only a .06 probability

235

of occurring by chance. The PATHEschool improvements for teacherreports of smooth administration andplanning and action, student reportsof rewarding school experiences,clarity of rules, interpersonal com-petency, and attachment to schoolalso have a low likelihood of occur-ring by chance given that both com-parison schools regressed on thesemeasures. All of the above-men-tioned findings favor the PATHE pro-gram. None of the variables exam-ined in Tables 4a through 4c showdisadvantages for the PATHE schools.These probabilities and thosereported in the following paragraphare not corrected for possible ceil-ing effects which, as I mentioned inthe previous section, sometimes makeimprovement from 1981 to 1982 impro-bable.

A gross measure of program effec-tiveness is simply the ratio of out-comes on which a given schoolimproved from 1981 to 1982. If weassign a score of one to each out-come examined in Tables 4a through4c that moved in the positive direc-tion and a score of zero to thosethat did not, and then rank theseven program and two comparisonschools from lowest to highest onthe degree of positive change from1981 to 1982 there is no overlapbetween the average rankings for thecomparison and the PATHE schools.That is, the average improvement forthe highest-ranking comparisonschool is lower than the averageimprovement for the lowest-rankingPATHE school. PATHE schoolsimproved on 64% to 82% of the out-comes in Tables 4a through 4c withan average of 74% improvement. Thecomparison schools improved on 24%and 45% of the outcomes with anaverage of 34%.

-359-

PATHE

Program Implementation

Information on level of programimplementation comes from two mainsources. "Intensity" standards,(standards for the amount of ser-vices provided by PATHE personnel),and "fidelity" standards (standardsfor the quality of the services ren-dered) were developed by the programmanagers to assist in project man-agement and evaluation. Each schoolin the PATHE program was rated forintensity and fidelity -twice duringthe 81782 school year, once at mid-year and again at the end of the

school year. A five point scaleranging from zero (does not meetstandard/not implemented) to 4

(exceeds standard) was used. Theratings were based upon documentedcontacts with students, teachers andparents, minutes and agendas frommeetings, logs of resources used andinterviews with PATHE specialists.The standards are shown in Tables12and 13, and the averages of the tworatings for each school are shown inTable 14.

The second major source of pro-gram implementation data for 1981-82is information about the nature andduration of PATHE specialist-studentcontacts. These data come from a

management information system devel-oped by Ann Birdseye, the formerPATHE evaluator. Data from contactlogs and sign-in sheets were enteredinto computer files and reorganizedto facilitate analysis. Tables 15

through 21 compare levels of treat-ment for target and control studentsand for the entire school. Severalpoints can be drawn from these

tables:

Experimental design breakdown.The experimental design was notupheld in most schools. The mostsevere breakdowns occured in Courte-nay and Rivers middle schools, wherecontrol students received close to

236

PATHE

one half as much treatment as targetstudents. In both of these schools,control students received moredirect services than did other non-targeted students in the school. In

other words, it appears that controlstudents were singled out for treat-ment or that the specialists in

these schools had special difficul-ties withholding treatment from con-trol students referred for service.Control students in Burke alsoreceived more treatment than othernon-targeted students, and receivedabout one third as much direct ser-vice contact as the target students.All other schools except for HautGap treated control students one-fifth to one-third as much as thetarget students, but these ratioswould be expected if control stu-dents received only those servicesdirected at all students in theschool, which was in accord with theexperimental design.

Intensity of direct services.

The level of direct services to tar-get students did not reach accepta-ble levels in most schools. Accord-ing to the intensity standards setby the program managers, target stu-dents were to have at least eightcontacts with the specialists duringthe year. Rhett exceeded this stan-dard but was the only PATHE schoolto meet it. Haut Gap and Brown fellseriously short with less than halfof the required number of contacts.

Overall program accomplishments.On the average, PATHE schools metabout 75% of the program standards.The academic interventions wereimplemented most faithfully and theservices to target students leastfaithfully program-wide. The inten-sity and fidelity ratings suggestthat the team structure and the

career interventions were imple-mented almost up to standard in thehigh schools, and that the affective

interventions lagged behind.According to the ratings, the middleschools implemented the program withless fidelity than the high schools.

The most vigorously implementedcomponents include Student Leader-ship Teams and Curriculum Review andRevision for all schools, and ParentLeadership Teams, School Pride Cam-paigns, Field Trips, Reading Experi-ence Program, Job-Seeking Skills andFaculty Inservices for high schoolsonly. The least well-implementedcomponents included services to tar-get students and extracurricularactivities (all schools) and Busi-ness-Education Partnerships, Peer

1p

Counseling, Tutoring, Job-SeekingSkills and Faculty Inservices (mid-dle schools only).

-360-

Caution should be exercised in

interpreting the implementationdata. Only a moderate degree of

correspondence between outcome meas-ures and implementation levels was,:served: School-wide academic andaffective components were imple-mented best, and the outcome datashow most improvement in these

areas. By and large, however, the

schools which would be expected, onthe basis of the implementationdata, to show the most improvementin certain areas often did not. The

moderate level of correspondence is

explained by two factors: The proj-ect never established a workablesystem for collecting data on somePATHE activities--notably extracur-ricular activities and school-wideactivities such as the study andtest-taking skills programs. Also,the intensity and fidelity standardswere sometimes unrealistic and werenot well-understood by the PATHEspecialists. This problem was cor-rected for the 1982-83 year. The

standards are now more realistic andhave been communicated to all spe-cialists.

237

Discussion

The establishment of an experi-mental design, implementation stan-dards and a system for monitoringprogram activities during the

1981-82 school year strengthened thePATHE program and its evaluation.The program was more focused andsystematically implemented than in

its previous year. Specialists/reported that the monitoring systemfacilitated their jobs because it

helped them to organize and priori-tize activities and to manage theirtime better.

Educational expectations did notimprove. One objective central to

the PATHE program was not met duringthe 1981-82 year. The expectationsof students and teachers for the

students' academic success did notincrease. In 1981, teachers in

PATHE schools reported that between28 and 39% of the students they

teach are low ability students.

Between 55 and 77% of students in

these same schools reported thatthey expected to continue some typeof schooling after high school. The

1982 figures are roughly the same.Target-control student comparisonson this outcome also tended towardthe negative: Control students in

five of the seven PATHE schools hadhigher educational expectations thandid the target students, and the

difference reached statistical sig-nificance in one school.

Although the largest school-widegain in this area was observed in

Burke--the school which implementedthe school-level components of the

the program with most fidelity to

the PATHE theory, the absence of

evidence of overall program effec-tiveness in this area suggests theneed for better-defined program com-ponents aimed at increasing expecta-tions. The "individualized"approach taken by many educators andthe reliance of many counseling

-361--

PATHE

techniques on information about thechild's background may in fact serveto foster low expectations.

An example of subtle underminingof the program objective comes froman interview with one PATHE special-ist who reported frequently takingteachers aside and "filling them in"about the child's home life so thatthe teacher might react more appro-priately to the child's behavior.Although offering this type of

"assistance" is probably not an

uncommon practice among counselors,it certainly is not in accord withthe PATHE model.

Projected Program Improvements

The program managers have alreadytaken steps to improve the programin areas identified as weak duringthe 1981-82 school year. The mostdramatic change is in the targetstudent- service -component : Blue-prints for services to target stu-dents have been developed and sharedwith the specialists, and these ser-vices are monitored carefully by theprogram manager and evaluator on anongoing basis. Early signs from thefirst quarter of the 1982-83 schoolyear suggest that target studentsare receiving much more service of ahigher quality than they were lastyear. This improvement also seemsto be solving the problem of theexperimental design breakdown: Theincreased expectations for servicesto target students made it necessaryfor specialists to establish treat-ment schedules for students. In

order to keep up with the schedules,specialists are withholding servicesto nontarget students more often.

An improved system for monitoringprogram activities was developed andimplemented in Fall, 1982. The newsystem identifies problem areas

early so that the program managercan work with the specialists to

238

PATHE

correct implementation breakdowns ina timely fashion. Early evidencealso suggests that this system is

strengthening the PATHE program.

Conclusion

Interim results of the evaluationof Project PATHE suggest the follow-ing:

1. Program effects on high riskstudents targeted by the PATHE pro-gram are more evident at the middlethan at the high school level.

2. Target students in all middleand two high schools scored higheron standardized achievement testsand received higher grades than didrandom control students. Theseresults reached statistical signifi-cance.

3. Target students in all fourmiddle schools attended school moreoften, reported higher levels of

attachment to Pchool, and more posi-tive self-concepts than did the con-trols, although' these differencesdid not reach statistical signifi-cance.

4. Target students in one middleschool reported significantly lessserious delinquent behavior thanrandom-equivalent control students.

School-v.de improvements on

-362-

PATHE goals and objectives are gen-erally more evident at the high thanat the middle school level. PATHEhigh schools improved from 1981-1982on 80% of the outcomes examined.PATHE middle school improved on 69%on the average.

6. Every PATHE school improvedfrom 1981-1982 on student Alienationand Positive Self-Concept andTeacher Morale. Six of the sevenPATHE schools improved on students'reports of Victimization, andSafety., Belief in Rules, andteachers' reports of Safety.

7. Every PATHE high school exper-ienced a reduction in delinquentbehavior and suspensions from1981-1982. For one the reduction inDelinquency was statistically signi-ficant, and for the other two thereduction in the number of suspen-sions was statistically significant.

8. School-wide academic interven-tions were implemented with mostintegrity during the 1981-82 year.The high schools generally met or

exceeded program standards in this

area, and the middle schools metabout 75% of the standards.

9. Target student services weregenerally weak, compared to programstandards. Only one middle schoolmet the intensity standards estab-lished by the program managers.

23D

PATHE

References

Birdseye, A. T. Evaluating the Implementation of a Delinquency PreventionProgram: The PATHE Experience. Paper presented at the Annual Meeting ofthe American Educational Research Association, New York, March, 1982.

Gavurin, S., Hybl L., & Gottfredson, D. C. Project PATHE: Interim evaluationresults. Baltimore: Center for Social Organization of Schools, 1982.

Gottfredson, D. C. Interim evaluation of project PATHE--Charleston. In Gott-fredson, G. (ed.), The School Action Effectiveness Study: First InterimReport. Baltimore: Center for Social Organization of Schools, 1982.

Gottfredson, G. D., Gottfredson, D. C., & Cook, .M. S.(eds.). The SchoolAction Effectiveness Study: Second interim report. Baltimore: The JohnsHopkins University, Center for'Social Organization of Schools, 1983.-

Gottfredson, G. D., Ogawa, D. K., Ricker, D. E., & Gottfredson, D. C. Stu-

dents and teachers in context: The measures used in the School ActionEffectiveness Study. In, Gottfredson, G. (ed.), The School Action Effec-tiveness Study: First Interim Report. Baltimore: Center for SocialOrganization of Schools, 1982.

Siegal, S. Nonparametric Statistics for the Behavioral Sciences. New York:_ _

McGraw-Hill, 1956.

Slavin, R. E. Cooperative Learning. New York: Longman, in press.

-363-

4,4u0,1:1

Ch3rleston Table

Student Selection FormPA'

00 you think this'student should be I PATHS SELECTION CRITERIA13 PATH!: target

1(check all criteria that

Istudent next year?' apply to each student:-)

INAmo

I I'D I YES INO 1112131415 1 61'

4. « 4. 4-4.---4---«__-4.---4.---+-

(students' names were preprinted here);.-11 :::is., list ,t11 ctner stui,cnts in your homeroom not included in atove list:

I I I I 1 I I .1 I I I

* « 4.4., « 4.- 4. -«___«_.

I 1 I I ' I I I I I I I

« « « « - - 4.- « ---4._.

I I I I I

.

I I I I 1 1.

4. 4. « «.--4.---4._--4.---«---4.---4«.

I 1 I I I 1.1 I 1 1 I

« « 4.« « « -«---«---«---«-.

I I I . I I I I I I I I

,

I.

« « - « « « « -4. « « «-.

L'1,.:Ise list ..,,ny other stuents not in your homeraom who....should he selected as PAT(1-..:Arvi' stud,..ints next year:

.- - - -- -« « « « «- -4.---4.---«-.

II I I :I 1 1 1 1 ! 1

« « « « .: M +.1 I I I I I I I I I I

« « « « « « -4. _4.-__4__-4.-

I I c I I I I 1 I I I-I« « « « -4. « 4. «

I I I I 1.1 I I I I I

« « « « « «- « -« « 4.---4.-

I I I I I I 1 I I .1 I

« « «

40,!umbers one throu.p.seven reFer to the-following PATHEtsqlection'criteria:1* previous expulsion.' suspension. or involvement in a number- of .disciolinary

actions2. social promotion or previous retention3. decline in performance level over past :3, school years .

4. previous request for guidance orcounseling 'due to a s9ecial need or proble.m:

F. tendency to cut class (truant)6« difficulty with the .law (delinquent)7. referral by a school/community agency 241

Table 2

Spring 1981 California Test of Basic Skills Standard Scores

for Target and Control Groups, by School

Reading Math LLEPEL

Total

Original Final

x SD N x SD N x SO N x SD N x SD N

Courteney

Target (N=83) 376 68 79 389 56 78 402 65 79 363 68 78 365 68 74

Control (N=80) 374 62 77 388 53 76 397 60 77 361 60 76 362 60 73

Rhett

Target (N.62) 415 58 60 407 55 61 423 65 60 390 62 60 389 63 55

Control (N=54) 406 64 50 405 42 49 420 59 50 388 51 49 389 52 46

Rivers

Target (N=86) 400 65 77 396 50 79 414 59 77 380 54 77 382 53 68

Control (N=75) 399 70 65 400 43 66 419 63 66 383 56 64 382 59 56

Haut Gap

m Target (N=65) 407 69 59 401 46 58 422 60 59 387* 58 57 385 54 51

0Control (N=23) 378 48 21 378 51 21 392 57 21 354 53 20 358 47 18

Brown

Target (N=83) 449 77 73 456 70 70 466 69 73 445 75 70 448 74 k 58

Control (N=80) 450 68 71 455 66 70 457 66 71 440 69 70 439 68 4 61

Burke

Target (N=80) 442 82 79 424 63 74 444 78 79 414 69 73 418 72 61

Control (N=77) 436 81 75 424 62 71 i 441 71 75 414 71 71 420 68 57

St. Johns

Target (N=78) 469 87 65 458 78 64 466 74 66 450 83 62 449 85 43

Control (N=76) 464 69 64 453 71 62 454 72 65 442 '73 60 430 69 48

All Schools

Target (N=540) 422, 79 492 418 66 484 434 71 493 403 74 477 . 402 73 410

Control (N=464) 419 75 423 418 63 415 430 68 425 402 71 410 400 69 359

Difference between target and control groups is significant at the p<.05 level.

aFigures under the "original" column represent group averages for all 1981-82 target and control students who

took the CTBS test in Spring, 1981. Figures under the "final" column represent averages for all target and

control students who took the Spring, 1981 CTBS test as well as the Spring, 1982 CTBS test. Total N's (in 24iparentheses) are the number of cases initially randomized into treatment and control conditions.

242

Table 3

Means and Standard Deviationsfor Teacher Referrals to PATHE

and Number of Suspensions, 1980-81

Number ofReferralsto PATHE

Numbe'r of

of

Suspensions

Nr SD X SD

CourtenayTreatment (N =83) .61 .49 .06 .24

Control (N=80) .54 .50 .08 .31

RhettTreatment (N=63) .46 .50 .25 .57

Control (N=52) .58 .50 .17 .43

RiversTreatment (N=85) .53 .50 .25 .53

Control (N=75) .63 .49 .12 .40

Haut GapTreatment (N=64) .50 .50 .12 .33

Control (N=23) .39 .50 .00 .00

BrownTreatment (N =81) .63 .48 .63 .86

Control (N=78) .50 .50 .58 .75

BurkeTreatment (N=86) .20 .40 .24 .51

Control (N=80) .18 .38 .31 .61

St. JohnsTreatment (N=78) .46 .50 .77 1.02

Control (N=76) .43 .50 .88 1.36

All SchoolsTreatment (N=540) .48 .50 .34 .68

Control (N=464) .46 .50 .35 .78

Note--Total N's (in parentheses) are numbers of cases initially

randomized into treatment and control conditions.

-366- 2,4

Table 4a

1981 to 1982 Difference Scores for School Averageson Survey Measures of PATHE Goals and Objectives--Middle Schools

Courtenay RhettLaing

Rivers Haut Gap (control)

Goal or objective measure

Total Delinquency (-)Serious Delinquency (-)Drug Use (-)

diff t diff t diff t diff t diff

Suspensions (-) .01 .19 -.08 -2.62 .01 .37 -.01 -.18 -.03 -.96Victimization (-) -.03 -1:61 -.01 -.86 -.02 -1.33 .02 1.69 .00 .05

School Nonattendance (-) .02 .57 .03 .67 .00 -.00 -.03 -1.06 .01 .27

School Rewards (+) .02 .71 .02 .60 -.02 -.88 .02 .65 -.03 -1.19School Punishments (-) -.01 -.30 .02 .98 .02 .71 -.02 -.87 .05 2.45Interpersonal Competency (+) .02 1.29 .05 2.68 .02 1.15 .03 1.31 -.02 -.88Positive Self-concept (+) .00 .26 .02 1.04 .03 1.69 .04 2.18 .02 1.52Alienation (-) -.01 -.36 -.07 -2.97 -.03 -1.21 -.05 -1.03 .00 -.15School Attachment (+) .00 .05 .02 1.17 .01 .27 .04 1.76 -.05 -2.65Educational Expectations (+) -.06 -.46 .01 .11 -.14 -.92 .29 1.92 .10 .69

Belief (+) .00 -.11 .03 1.39 .03 -1.26 :05 2.32 .04 1.85

Low Expectationsa (-) -4.17 -.72 .30 .04 .90 .17 -5.51 -.97-10.99 -1.57Involvement (+) .02 1.41 -.00 -.10 -.03 -1.64 -.01 -.49 .01 .41

Victimizationa (-) -.01 -.25 .00 .07 -.01 -.32 .02 .29 -.00 -.10Safetya (+) .34 2.28 .32 2.17 -.03 -.19 .07 .49 .12 .83

Teacher Moralea (+) .07 1.26 .09 1.67 .05 1.03 .15 2.78 .00 .07

Planning and Actiona (+) -.02 -.24 .06 .87 .10 1.44 .03 1.17 -.07 -1.01

Smooth Administrationa (+) -.02 -.34 -.10 -1.68 .06 1.03 .15 2.46 -.09 -.31Rule Clarity (+) .00 -.10 .01 .23 .02 .51 .02 .47 -.04 -.90Rule Fairness (+) -.01 -.22 .02 .37 .00 .03 -.01 -.10 -.04 -.67Planning and Action (+) -.02 -.34 -.05 -1.01 .03 .73 .00 .10 -.06 -1.30Safety (+) .04 1.44 .03 1.01 .05 1.70 .00 .06 .02 .76

Pct. measures improved .68 .f8 .64 .82 .45

Note. The desired direction of change is indicated in parentheses after each scale name.t-statistics for compositional measures (Total Delinquency through Involvement) are basedupon 1981 and 1982 means and standard deviation's for each school. For psychosocial cli-mate measures, the "t-statistic" is the ratio of the difference between 1982 and 1981scores, to the standard error of measurement of the 1982 scores for all non-St. Paulschools in the OJJDP initiative. As a rule of thumb, psychosocial climate measure differ-ences that are twice the standard error of measurement may be regarded as dependable. Forcompositional measures, t values of 1.96 and 2.54 are statistically significant at thep<.05 and p<.01 level, respectively. All measures are taken from the SAES student surveysunless otherwise indicated. Dashes indicate that the goal was not measured in 1981.

aMeasure taken from SAES teacher survey

-367-

%Isle 4h

)$4 i* Oittotooto Stereo for School Averages044 Nolk-voe useeetes et tent Coale and Ohjetlives--1110 Schools

ttss s s.m.ssimissacusess

OratesMMUWW

644114 tithiswei is* lettMovie gift t31,40 7.42,1:ZiZAIAZtat,l,Z4,111141L11,111.3111,A1.711P.r,S,SMONaPaPoSultirligio

*stet Optiteheessev 4.1 a.415 -1.92

dot*** Opiteesesoba 4.1 ,00) 441Oesei tee 121 =.03 -1.04

Oft ehe 4.1 *4.20 -1.06

tleitstaettes 4.1 ,-.01 *.46

006smi Ossitivedsotio 4.1 .00 iii.O5

6sheei Opsittle 1.) a.02 ...le

*shoot Pestahmeets 4.) si.OS 'L40

404 otipoteseei feepeteeep is) .00 s..22

'settees thoitistossest is) .01 .44

ditssalthes is/ -.04 441*shoat Aelseemsed is) Al .83

8 esetiese) tasystattoso (s) is.10 s.069

6s44e1 (41.4 .01 .52

Cm lispeitetwo4 I ) .94 .20

Vitt MOM We 1- 0 -.02 -1.10ksmoissmeet is) is.02 -1.07

letete is) .02 .11

%Mot ilhorelea (a) .0) .12

Pismo. owl Art test tit) .01 .13

lOodt* ddishaistuttito0 '0 .04 1.25

bete elsettl is) .03 .10

Oells Pattspos ist .04 AOPisan* tad Attie* it) .03 .55

Weds is) .02 .76

P.m. imetesemse »a r me *4 .so

Berke1114G16110110.11114.

dttt t

It. Johns

dill t

Charleston(control)

diff t

-.01 .4.041 -.02 -2.05 .00 .09

-.01 -1.12 -.01 -.72 .00 .40

.01 .24 -.05 -2.27 .02 .92

-.10 -3.08 4.42 -.55 .01 .38

-.06 -4.01 -.01 -.99 .00 .21

-.04 -.94 -.05 -1.18 -.08 -1.77

.00 .16 .01 .25 -.02 -.75

-.02 .4.40 -.01 -.61 .01 .45

.02 1.46 -.01 -.31 .00 -.20

.04 2.97 .01 .70 .03 2.08

-.OS -2.31 -.02 -.76 .03 .98

- .02.1.36 -.01 -.63 -.03 -1.23

.25 1.88 .06 .38 -.23 -1.52

.02 1.21 .03 1.26 .01 .34

-5.10 -1.26 -8.23 -1.46 2.39 .33

-.Al -.32 -.01 -.26 -.05 -1.08.01 .73 .03 1.56 -.01 -.56

.21 1.40 .04 .29 .27 1.82

.01 .83 .11 2.12 -.08 -1.46

-.01 -.20 .01 .20 -.04 -.61

.00 .02 .04 .68 -.17 -2.88

.01 .25 .00 -.09 -.05 -1.13

-.03 -.57 -.01 -.15 -.03 -.52

-.10 -2.19 -.03 -.72 -.06 -1.17

.06 2.21 -.04 -1.49 -.01 -.49

.80 .80 .24

paw 1101 dotted direettes sit thefts is indicated in parentheses after each

'tele some. g*etettetite tee emmpeilti.sal messeres (Total Delinquency through

lismoiressosta me hosed epos 1101 sod 1102 moons old standard deviations for each

POW pepsheessis4 elisate feeieeeMI the "rstatistic" is the ratio of the

44fterssee issesess IOW end 10411 stores to the standard error of measurement of

"Ne 1444 sees** tar all seepOt. Peel schools is the WO" initiative. As a rule

of Wash, serichseettai el into moaaere differfoces that are twice the standard

*MR of 4.0441101004 1114,:be ffterded as dependable. For compositional measures,

4 ,444104 of 1.16 4011 1.54 are statistically sigaificant at the 1)4.05 and 1)(41

weepectiveiy. 611 sesames are takes tress the SAES student surveys

eidees meter tadicated.

tiseest tales ties 8418 teacher seem

-

Table 4c

Number of PATHE and Control Schools that Improved from1981 to 1982 on Survey Measures of PATHE Goals and Objectives

Goal or objective measure

PATHE Schools

Middle High Total

(n=4) (n=3) (n=7)

Control Schools

Middle High Total

(n=1) (n=1) (n=2)

Total Delinquency (-) __b 3 3 __b 0 0

Serious Delinquency (-) __b 3 3 __b "0 0

Drug Use (-) __b 2 2 __b 0 0

Suspensions (-) 2 3 5 1 0 1

Victimization (-) 3 3 6 0 0 0

School Nonattendance (-) 2 , 3 5 0 1 1

School Rewards (+) 3 2 5 0 0 0

School Punishments (-) 2 3 5 0 0 0

Interpersonal Competency (+) 4 1 5 0 0 0

Positive Self-concept (+) 4 3 7 1 1 2

Alienation (-) 4 3 7 1 0 1

School Attachment (+) 4 1 5 0 0 0

Educational Expectations (+) 2 2

:-

1 0 1

Belief (+) 3 3 1 1 2-

Low Expectationsa (-) 2 2 4 1 0 1

Involvement (+) 1 2 3 1 0 1

Victimizationa (-) 2 3 5 1 1 2

Safetya (+) 3 3 6 1 1 2

Teacher Moralea (+) 4 3 7 1 0 1

Planning and Actiona (+) 3 2 5 0 0 0

Smooth Administrations (+) 2 3 5 0 0 0

Rule Clarity (+) 3 2 5 0 0 0

Rule Fairness (+) 2 1 3 0 0 0

Planning and Action (+) 2 1 3 0 0 0

Safety (+) 4 2 6 1 0 1

Pct. measures improved .69 .79 .74 .50 .20 .34

Note. The desired direction of change is indicated in parentheses after

each scale name. All measures are taken from the SAES student surveys

unless otherwise indicated.

a Measure taken from SAES teacher survey.

b Delinquency was not measured in 1981 for middle schools.

-369-247

Table 5

Rates of Suspension, Expulsion, Attendance and

Withdrawal from School, 1979-1982

Suspensions Expulsions Average Daily

AttendanceL

Withdrawals

79-80 80-81 81-82 79-80 80-81 81-82 79-80 80-81 81-82 79-80 80-81 81-82

C. .rtenay M.S. .26 .13 .08 .00 .00 .00 .93 .93 .94 .03 .01 .01

Rhett M.S. .15 .09 .10 .00 .00 .00 .93 .95 .94 .03 .01 .01

Rivers M.S. .11 .18 .25 .01 .00 .00 .94 .94 .94 .02 .02 .03

Haut Gap M.S. .04 .10 .20 .00 .00 .00 .93 .93 .93' .01 .01 .01

Laing M.S.1

.08 .13 .15 .00 .01 .00 .93 .94 .94 .02 .01 .02

Brown H.S. .46 .66 .32 .01 .01 .02 .89 .90 .91 .14 .10 .14

Burke H.S. .09 .18 .14 .01 .01 .02 .90 .92 .11 .10 .10

St. Johns H.S. .49 .90 .87 .02 .02 .02 .93 .91 .92 .10 .09 .10

Charleston H.S.1 .24 .34 .48 .01 .01 .00 .92 .93 .94 .11 .05 .07

1Control School

2

Average daily attendance of average daily membership.

249

248

Table 6California Test of Basic Skills

PATHE and Control School Scores, 1981 and 1982

School

Scale Score

1981 1982

H SD 11

Reading

SD

scoring inbottomnationalquartile

1981 1982

Courtenay 436.1 80.8 459.5** 82.7 42.9 32.9Rhett 458.4 92.7 481.8** 92.2 31.0 19.1Rivers 446.2 75.8 466.6** 71.6 39.0 31.7Haut Gap 475.8 77.9 488.0* 73.5 34.1 27.2Laing<a> 514.4 101.6 536.7** 88.9 21.7 11.5Brown 483.2 86.3 488.9 87.7 57.6 57.1Burke 509.5 90.4 523.1** 91.8 50.2 45.4St. Johns 519.4 94.3 528.6 98.8 41.7 38.7Charleston<a> 567.0 90.5 575.7 53.2 36.2 30.0PAM Schools 484.3 91.9 495.3** 90.6 43.8 36.3Control Schools 543%7 99.0 554.9* 92.9 29.8 19.8

Language

Courtenay 452.9 76.5 470.4** 81.1 36.0 25.4Rhett 471.7 92.4 489.1** 85.6 26.4 18.4Rivers 462.1 76.4 474.1* 70.9 29.5 24.9Haut Gap 480.1 73.6 490.0 64.6 29.3 18.6Laing<a> 513.6 99.8 527.6* 81.0 18.3 11.7Brown 501.0 84.8 504.7 87.6 43.6 41.4Burke 516.0 90.1 527.8** 91.2 37.3 37.1St. Johns 520.1 89.2 522.1 93.3 39.8 39.3Charleston<a> 573.9 95.6 578.7 92;1 -22.4 21.1PATHE Schools 494.3 88.6 501.1** 86.9 36.0 30.4Control Schools 547.1 101.9 551.5 90.0 20.6 16.1

}lath

Courtenay 437.5 65.2 451.6** 70.2 37.6 28.8Rhett 448.1 75.3 465.5** 71.4 31.9 23.8

Rivers 440.4 56.9 448.5* 59.1 39.6 32.9

Haut Gap 450.2 65.4 463.5** 60.9 33.1 26.6Laing<a> 485.9 88.8 508.7** 82.8 23.0 13.4

Brown 473.8 77.9 477.2 80.4 53.6 52.1

Burke 488.7 81.7 504.5** 81.9 47.6 44.7St. Johns 509.1 85.8 509.0 94.9 41.5 37.1*

Charleston<a> 531.8 85.3 546.1* 88.3 36.1 33.8

PATHE Schools 472.0 80.0 478.6** 80.1 42.0 35.8 ,

Control Schools 511.2 89.8 526.3** 87.3 30.0 22.5

Total Battery

Courtenay 422.9 78.9 445.0** 83.0 46.3 34.5

Rhett 442.1 95.5 463.8** 89.1 32.8 23.4

Rivers 430.9 72.9 445.9** 71.8 44.8 35.8Haut Gap 453.1 76.7 466.1* 69.0 39.2 29.6Laing<a> 496.0 107.0 518.5** 93.6 25.5 13.0

Brown 477.7 86.4 482.1 90.8 60.4 59.4

Burke 499.1 91.3 515.1** 92.5 53.6 50.4.

St. Johns 513.2 94.0 519.0 102.9 46.3 44.3Charleston<a> 559.6 92.1 571.6 96.5 38.9 33.6

PATHE Schools 472.7 92.8 481.9** 92.3 47.7 40.3

Control Schools 530.8 104.0 543.3* 98.5 32.9 22.4

Note: Table entries are based on all cases in the schooleach year for whom a valid test score was provided. N's

range from 277 to 634 for 1981, and from 338 to 661 for1982.<a>Control School*Difference between 1981 and 1982 is significant at thep<.05 level.**Difference between 1981 and 1982 is significant at thepc.01 level.

Table 7

Means and Standard Deviations for PATUE

Target and Control

Croups on Pleasures of Goal 1: Reduce Delinquency

Courtenay

Target

Control

Rhett

Target

Control

) Rivers

Target

I Control

r I

1 uliaut Cap

1 "I Target

1\11

Control

C. A. Brown

Target

Control

Burke

Target

Control

St. Johns

Target

Control

All Schools

Target

Control

Number of Suspensions Number of

Expdlsions

Self-reported DelinquencyNumber of Disciplinary Infractions

Official Self-reportedTotal Serious Luna Major Minor

M SD N M SD .N M SD N M SD N M SD N M SD N M SD N M SD N

.12 .36 82 .35 .48 65 .01 .11 82 .15 .16 62 .13 .18 62 .14 .22 62 2.64 1.40 81 5.62 2.54 81

.09 .28 80 .30 .46 60 .00 ,00 80 .11 .15 55 .10 .18 55 .10 .21 55 2,82 1.48 80 5.39 2.52 80

.11 .51 63 .21 .41 43 .01 .33 63 .14 .21 42 .12 .20 42 .16 .26 41 2.28 1.27 61 3.92 1.51 61

.12 .43 51 .37 ,49 30 .00 .00 51 .13 .22 30 .11 .23 31 .12 .24 31 2.28 1.14 49 4.26 1.52 49

.37 .86 84 .38 .49 47 .00 .00 84 .13 .13 51 .09* .14 51 .15 .18 53 2.62 1:14 81 4.27 1,47 81

.50 1.20 74 .39 .49 41 .01 .12 74.19 .19 46 .16 .20 46 .21 .27 48 2.61 1.23 12 4.53 1.66 72

2.42 1,15 59 4.51 1.49 59

.32 .78 62 .17 .38 30 .00 .00 62 .09 .12 37 .05 .10 31 "2 .23 40 2.96 1.40 23 4.13 1.49 23

.17 .65 23.27 .46 15 .00 .00 23 .15 .11 13 .11 .16 13 .18 .22 13

..7 2.44 1.22 10 3.99 1.59 70

.43 .81 75 .33 .48 54 .00 .00 75.13 .15 53 .06 .12 53 .26 .28 54

. 2.26 ..97 70 4.41 1.43 70

.35 .60 75 .23. .43 56 ,03 .16 15 .14 .17 55 .09 .15 55 .22 .28 55

2.25 1.12 75 4.15 1.53 75

.17 .47 81 .25 .44 55 .01 .1181 .19 .22 59 .12 .21 59 .31 .33 60 2.44 1.28 72 4,24 1.73 72

.08 .27 78 .22 .42 46 .01 .11 78.14 .17 54 .08 .16 54 .25 .30 , 53

.971,67 16.52 .50 52 01 .11 76

.12 .13 52 .06 .10 52 .25 .26 51 2.26 1.39 69 4.23 2.00 69

1.26 2,19 16 .51 .50 51 .01 .11 76.11 .13 47 .06 .12 46 .20 .27 47 2.13 1.05 70 4.27 1.57 70

.36 .92 523 .33 .47 346 .01 .09 523 .14 .17 356 .09 .16 356 .20 .27 361 2.43 1,25 496 4.41 1.87 496

.40 1.15 457 .33 .47 299 .01 Jo 457 .14 .17 300 .10 .17 300 .19 .27 302 2.47 1.24 436 4.53 1.84 436

*Difference between target and control groupsis significant at the p<J1C level.

Measures not available.

Note - -The base N on which computations of official suspension::, expulsionsand withdrawals are based is all cases randomly

assigned to treatment and control condition; Iy1,0 to ro IP,I hlra led in the schools as of October, 1981. The base

N on which computations of allother outcomes are based is all cases

randomly assigned to treatment and control conditions minus

72 cases who had withdrawnfrom school as of 2/82 and for whom no 2nd semester data were provided,

fir,.

4,10.1. 25 9

Table B

Means and Standard Deviations for POE

Target and Control

1

.4

1.0)

1

Croups on Measures of Coal II: Improve Attendance

% Days Absent - String '82 % Days Tardy No, of Attendance

to Class -arinf, 182 related disc. infrar..

Number of

011hdrawals

Self-reported Non-attendance

Total Unexcused Cutting School Cutting Class Total Index

N SD N M SD N H SD N M SD N 11 SD N M S0 N ti 5b N II SD

Courtenay

Target

Control

Alit

Target

Control

;beers

Target

Control

Haut Cap

Target

Control

1., A. Brown

Target

Control

Burke

Target

Control

St. Johns

Target

Control

All Schools

Target

Control

.01

.01

.06

.08

.09

.13

.08

.09

.10

.11

.09

.11

.09

.09

.08

.10

.08

.09

.07

.10

.14

.18

.08

.11

.10

.14

.10

.11

.10

.08

.10

.13

81

77

56

47

80

70

59

22

70

69

72

69

62

61

480

415

.06

.07

.06

.06

.07

.10

.03

.04'

.09

.10

.08

.10

.07

.07

.07

.08

.06

.08

.06

.07

.12

.16

.04

.05

,10

.13

.09

.15

.09

.07

.09

.12

81

77

56

47

80

10

59

22

70

69

72

69

62

61

480

415

.04

.05

.01

.03

.04

.04

.02

.01

a

a

a

a

.09

.10

,05

.11

.08

.10

.04

.09

.06

.08

.03

.02

a

a

a

a

.10

.12

.07

.10

81

77

56

47

80

10

59

22

a

a

62

61

330

217

1.54 1.48

1.38 1.29

.67 .03

.82 .90

.91 1.04

.80 .04

1.07 .96

1.13 1.22

1.28 1.26

1.24 .98

.48 1.19

.12 1.17

1.38 1.35

1.60I.42

1.20 1.22

1.18 1.18

81

80

61

49

81

72

59

23

70

70

75

72

69

70

496

416

.11

.11

.22

.16

.17

.08

.08

.00

.19

.25

.20

.29

.18

.17

.16

.17

.31

.32

.42

.37

.37

.27

.27

.00

.39

.44

.40

.45

.39

.38

.97

.

82

'80

61

51

84

74

62

23

75

75

81

78

76

76

573

457

..47

.53

.37

.38

.20

.20

.10

..42

.33

.52

.39

.45

.30

.29

.32

.40

.96

1.06

.85

.95

.65

.56

.41

1.12

.60

.88

.58

.83

.78

.78

.71

.88

75

72

51

40

64

55

51

19

61

62

61

62

56

55

419

359

.49 1.04

.36 .83

.85 1.23

.68 1.47

.401.01

.25 .82

.24 .55

.63 1.30

1.10 1.32

1.19 1.28

1.27 1.05

1.04 .99

.59 .95

.87 1.39

.11 1.11

.12 1.18

15

72

53

41

65

55

53

19

61

62

63

56

56

54

426

359

.53

.46

.65

.45

.27

.28

.26

.42

.78

.90

1.03

.93

.54

.56

.58

.59

.67

.69

.80

.68

.60

.60

.49

.77

.78

.84

.80

.79

.71

.71

.74

.77

74

72

51

40

61

54

55

1

60

62

8r

55

56

54

415

356

aMeasures not available.

bWithdrawals are cases that left school before the close of the 81-82 school year for reasons other than graduation from the twelfth grade.

Note-The base N on which computations of official suspensions, expulsions and withdrawals are based Is all cases randomly assigned to

treatment and control conditions minus 24 cases who were not located in the schools as of October, 1981. The base N on which computations

of all other outcomes are based is all cases randomly assigned to treatment and control conditions minus 72 cases who had withdrawn from

school as of February 1982 and for whom no second semester data were provided.

253

1 of 3

Table 9

Means and Standard Deviations for PATI1E

Target and Control

Groups on Measures of Coal IV: Decrease Academic Failure

Percent Receiving Failing Grades During Spring '82 Semester Self-reported

Grades

Number of Academi.

Related Disc. Infra,English Math Social Studies Science Average

M SD N M SD N M SD N M SD N M SD N M SD N M SD N

Courtenay

Target .40 .49 81 .36 .48 81 .31 .46 81 .38 .49 81 .42 .50 81 2.37 .80 76 .25 .54 81

Control .40 .49 78 .36 .48 78 .26 .44 78 .27 .45 78 .44 .50 78 2.62 .97 72 .28 .59 BO

Rhett

Target .24 .43 55 .07 .26 55 .36 55 .17 .38 53 .22 .42 55 2,75 * *,74 52 .23 .50 61

Control .38 .49 45 .15 .36 46 .49

.0

.50 45 .20 .40 45 .35 .48 46 2.36 .62 41 .14 .35 49

Rivers

I Target .37 .49 73 .42 .50 73 .46 .50 72 .26 .44 73 .44 .50 73 2.35 .72 65 .16 .37 81

L.; Control..,1

.45 .50 65 .42 .50 65 .44 .50 64 .40 .49 65 .54 .50 65 2.27 .85 55 .18 .39 72

4s

Illaut Gap

Target .38 .49 55 .11 .31 55 .38 .49 55 .18 .39 55 .16 .37 52 2.56 .75 52 .25 .54 59

Control .45 .51 22 .14 .35 22 .41 .50 22 .23 .43 22 .27 .46 22 2.52 .90 19 .13 .34 23

C. A. Brown

Target .52 .50 62 .38 .49 48 .12 .33 33 .32* .47 44 .47 .50 64' 2.13 .50 61 .06 .23 70

Control .67 .48 63 .38 .49 55 .13 .34 30 .49 .50 45 .58 .50 65 2,21 52 62 .06 .23 70

Burke

Target .39 .49 64 .31 .47 55 .30 .46 53 .53 .51 36 .40 .49 65 2.36 ,66 63 .08 .27 75

Control .40 .49 60 .30 .46 47 .17 .38 47 .30 .46 40 .31 .47 642.43 .60 56 .11 .36 72

St. Johns

Target .20 .41 59 .35 .48 49 .41 .50 56 .43 .50 30 .40 .49 60 2.32 .64 56 .14 .39 69

Control .31 .47 58 .48 .50 52 .27 .45 55 .37 .49 30 .41 .50 i61 2.34 .67 55 .11 .32 70

All Schools

Target .36* .48 449 .29 .46 416 .35 .48 405 .31 .46 312 .37 .48 453 ' 2.40 .71 425 .16 .42 496

Control .44 .50 391 .34 .48 365 .31 .46 341 .33 .47 325 ,43* .50 401 2.39 .75 360 .15 .40 436

*Difference between target and control groups is significant at the pe..05 level.

AmDifference between target and control groups is significant at the p,..01 level.

Note--The base N on which computations of official suspensions, explusions and withdrawals are based is all cases

randomly assigned to treatment and control conditions minus 24 cases who were not located in the schools as of Octo-

ber, 1981. The base N on which computations of all other outcomes are based is all cases randomly assigned to treat-

ment and control conditions minus 72 cases who had withdrawn from school as of February, 1982 and for whom no second

semester data were provided.

256

5

2 ot 3

Table 9 (cont' d)

Standard Scores Elpliag '82 CTBS Test

LanguageReading Math Total

M SD N M SD N M SD N M SD N

Courtenay

Target 408.37 73.68 79 405.32 60.61 78 416.94 78.87 79 387.45 75.37 78

Control 406.90 60.85 75 406.25 53.62 7 415.36 64.16 76 386.73 62.10 75

Rhett

Target 441.05 70.36 58 436.83 55.26 58 444.71 69.07 58 421.83 67.96 '58

Control 422.11 77.72 47 429.81 50.90 47 430,36 57,57 47 407.72 62.22 47

Rivers

Target 432.74 66.06 72 421.74 47.73 72 445.85 58.25 72 411.51 56.41 72

Control 437.12 69.52 64 423.89 48.66 64 440.67 57.64 64 412.80 61.48 64

1

Haut Cap

L.) Target.1

437.36 60.21 55 420.35 59.18 47 437.67 57.71 55 409.41 59.51 54

ui Control 410.19 49.72 21 402.00 49.26 21 432.14 52.64 21 392.57 46.47 21

C. A. Brown

Target 474.26 76.09 62 465.58 76.82 62 493.63 73.94 62 467.56 78.32 62

Control 475.34 80.50 64 457.66 65.71 64 479.47 72.48 64 459.50 74.97 64

Burke

Target 473.66 82.24 65 459.11 75.56 65 466.94 82.16 65 454.54 79.75 65

Control 482.21 85.70 58 470.96 62.06 '57 478.86 83.03 58 466.16 74.82 57

St. Johns

Target 492.10 90.02 50 465.94 94.52 154 478.76* 82.14 50 470.34 102.60 47

Control 465.85 81.74 52 453.98 78.22 148 458.62 76.06 52 441.57 74.78 47

All Schools

Target 448.64 78.72 441 437.21 70.67 436 453.06 76.06 441 429.04 79.87 436

Control 445.04 79.44 381 436.53 63.70 377 448.64 71.97 382 425.50 73.32 375

*Difference between target and control groups is significant at the p<.05 level.

**Difference between target and control groups is significant at the p<.01 level.

Note--The base N on which computations of official suspensions,, explusions and withdrawals are based is all cases

randomly assigned to treatment and control conditions minus 24 cases who were not located in the schools as of Octo-

ber, 1981. The base N on which computations of all other outcomes are based is all cases randomly assigned to treat-

ment and control conditions minus 72 cases who had withdrawn from school as of February, 1982 and for whom no second 258semester data were provided.

0101311111Vahr4M

TO14. $ (coned)

the Settee larti1e on the Spring 82 CTBS Test

lOta

0

1/2 .411 PI .61 .47 71 .66 .48

,7t 15 44 .41 76 .70 .46

ot4 ,41 .40 59 .40 .49 54 .43 .50

1404 .04 .40 41 .50 47 .57 .50

*1-64 .4$ .411 72 .00 .49 72 .42 .50

4S.01 .$4 .$0 64 AO 40 64 .52 .50

04 At' .40 $$ .54 .50 34 .49 .50

444 .14 . 21 .76 .44 21 .57 .51

40464

04 .02 .21 61 .72 .45 62 .69 .46

Owl .04 .40 44 .6) 31 64 .76 .43

.44 .2$ 0$ .$2 .29 6$ .80 .40

,40 54 .14 .44 57 .69 .41

Omo

rio4 ,f) .6$ $0 JO .46 47 .66.411

0104 .11 ,44 51 .77 .42 41 .46 .34

4044*.0 .44 441 .66 .41 4)6 .5944.49

,",,f1111r POP efir2

114.00040:0001000, 41:144,

01, .47

3 of 3

Self-reported

Reading Ability

N M SD N M SD N

79 .69 .46 78 1.40 .98 63

76 .79 .41 75 1.45 .99 62

58 .48 .50 58 1.62 .85 42

47 .68 .47 47 1.60 .97 38

72 .64 .48 72 1.35 .80 54

64 .64 .48 64 1.30 .95 47

55 .70 .46 54 1.38 .79 42

21 .86 .36 21 1.44 .70 18

62 .77 .42 62 1.60 .75 57

64 .84 .37 64 1.38 .84 52

65 .88 .33 65 1.50 .84 54

58 .86 .35 57 1.39 .70 49

50 .77*

.43 47 1.33 .71 52

52 .91 .28 47 1.41 .84 49

441 .70**.46 436 1.45 .83 364

3112.-- .79 .41 375 1.42 .88 315

4 (antral grow Is Writ Icaot at the pc.05 level.

f4 44f04 *44 roam! imp* le significant at the oe.01 level.

ert* 4**0 $1 04 4404 00,4440404 of official olossioneloos, exclusions andwithdrawals are based is all cases

If 004004 40 100410041 and (Petrol 44440444 SOW 24 cases who wore not located in the schools as of 0cto-

016 tto boo limillAkh pompetstiomeof all other outcomes ate based Is all cases randomly assigned to treat-

hi WNW witittomo Woo if 4044 1414 hadvithdrown from school as of February, 1982 and for whom no second

win Ion fir. 260

Table10

'Means and Standard Deviations for PATE

Target and Control

Croups on other Survey Measures

1 of 3

Interpersonal

Competency Self-concept

Attachment to

Parents School

Educational

Expectations

M

School

Effort

SD N

Involvement In

Extracurricular Activ,

Practical

Knowledge

M SD N M SD N M SD N M SD N M SD N M SD N M SD N

Courteuy

Target .73 .25 62 .70 .21 51 .66 .22 76 .11 .20 69 2.44 1.11 73 .54 .26 63 .38 .21 60 I.24*.44 60

Control .78 .19 60 .67 .19 51 .66 .25 72 .16 .21 61 2.57 1.88 10 .55 .27 67 .36 .23 65 1.40 .37 57

Rhett

Target .78 .19 31 .75 .15 33 .65 .28 52 .16 .23 45 2.92 1.80 52 .63 .30 49 .32 .24 49 1.35 .36 37

Control .71 .22 27 .10 .14 27 .68 .19 39 .14 .24 34 3.41 1.82 39 .67 .24 34 .24 .25 32 1.40 .41 23

River;

Target. .75 .22 46 .73 .19 38 .62 .23 61 .10 .22 52 2.76 1.85 62 .61 .29 61 .31 .22 56 1.28 .42 46

Control .73 .28 39 .10 .19 32 .65 .25 54 .68 .24 43 2.76 1.90 55 .56 .29 48 .24 .20 43 1.31 .50 31

Haut Cap

1 Target .79 .22 30 .11 .19 27 .64 .21 50 .16 .27 34 2.56 1.91 52 .58 .29 46 .25 .22 42 1.17 .47 331.4

.4Control .10 .22 12 .67 .22 13 .68 .26 11 .63 .28 16 2.59 2.03 17 .56 .30 14 .28 .21 15 1.25 .35 14

.4

I C. A. Brown

Target .83 .18 51 .16 .18 45 .66 .24 60 ,77 .20 51 2.08 1.63 6'4 .65 .26 60 .22 .11 56 1.47 .40 54

Control .85 .17 52 .80 .14 46 .59 .22 59 .17 .20 58 2.20 1,38 56 .60 '.29 60 .26 .18 58 1.38 .48 51

Burke

Target .19 .21 44 .16 ,16 41 .60 .22 64 .75**,18 59 2,05 1.51) 64 .58 ,26 63 .21 .11 58 1.40 ,41 43

Control .71 .24 45 .79 .14 39 .62 ,27 56 .83 .13 48 2.70 1.65 54 .64 .28 55 .11 .15 55 1,46 .36 45

St. Johnsi

Target AO .21 47 .15 .18 41 .59 .25 55 .72 .22 53 2.41 1.55 54 .66 .27 52 .21 .18 44 1.41 .40 47

Control .79 .24 50 .73 .14 )45 .66 .23 55 52 2.07 1.59 54 .26 51 .22 49 1.40 .39 41.65 .281

.69 .25

All Schools

Target .78 .21 317 .74 .18 276 .63 ,2. 418 .74 .21 3691 2,1,5 1.13 417 .60 .27 394 .21 .21 365 1.34 .42 320

Control .18 ,22 285 .73 .11 253 .64 .26 152 .73 .23 112 2.58 1.76145 .61 .28 329 .26 .21 311 114 .42 274

*Difference between target and control groups Is significant at tie -.15 level,1

Note--The base N on which computations of official suspensions, explusions and withdrawals are bayed is all cases

randomly assigned to treatment and control conditions minus 24 cases who were nnt located in the schools as of Octo-

ber, 1981. The base N on which computations of all other outcomes are based is all cases randomly assigned to treat-

ment and control conditions minus 12 cases who had.withdrawn from school as of February, 1982 and for whom no second

semester data were provided.

262

26.;

1

2 of 3

Table 10(cont'd)

Parental Emphasis

on Education

N SD N

School

Punishments

M SD N M

School

Rewards

SD N

Perceptions of School Climate Disrespect

for Students

Control of

EnvironmentRule Fairness Rule Clarity St-Tch Interaction

M SD N M SD N M SD N M SD N M SD N

Courtenay

Target .63 .29 68 .40 .25 69 .37 .31 69 3.11 1,41 66 3,67 1.06 66 1.56 .95 68 ' 2.83 1.39 66 .56 .23 64

Control .63 .29 62 .45 .30 62 .43 .32 61 3.44 1.22 61 3.67 1.06 57 1.69 1.00 64 2.41 1.30 59 .62 .20 60

Rhett

Target .59 .26 39 .33* .29 45 .36 .33 45 3.40 1,47 42 3.79 1.14 43 1.61 1.00 46 2.77 1.34 43 .56 .24 37

Control .66 ,27 32 .49 .33 30 .26 .26 31 3.48 1.50 29 4.00 1.19 28 1.27 1.23 33 3.17 1.29 30 .62 .20 28

1

Rivers.ti

(.k) Target .64 .30 50 .36 .28 53 .36 .34 53 3.36* 1.20 52 4.04* 1.09 51 1.51 1.10 57 2.79 1.28 43 .54 .22 46.4

Control .60 .29 42 .37 .22 37 .35 .31 38 2.78 1.53 36 3.57 1,14 37 1.24 1.04 41 2.54 1.38 35 .57 .18 38

CO

1

Haut Cap

Target .58 .29 33 .24 .25 35 .34 .35 36 3,03 1.54 35 3.69 1.18 321 1.42 1.20 38 2.56 1.64 .14 .58 .26 31

Control .57 .30 15 .32 .27 15 .32 .29 15 3.00 1.57 14 3.60 1.18 15 1.50 1.10 16 3.00 1.51 15 .64 .21 14

C. A. Brown

Target .50 .33 54 .21 .25 56 .30 .30 56 3.29 1.28 51 3.61 1.06 51 1.53 1.24 57 2.25 1.15 52 .58 .24 53

Control .50 .32 55 .20 .26 55 .24 .28 56 3,36 1.24 50 3.47 1.38 53 1.53 1,04 56 2.48 1.42 52 .50 .23 53

Burke

Target .47 .32 49 .24 .26 58 .24 .28 59 3,10 1.38 57 3.85 1.18 53 1.50 1.18 59 2.65 1.36 55 .57 .22 43

Control .49 .29 50 .22 .21 47 .2 5 .30 61 3.51 1.10 43 4.09 1.09 46 1.67 1.15 46 2.26 1.42 43 .51 .21 45

St. Johns

Target .40 .26 49 .20 .28 52 .24 .25 52 2.85 1.38 48 3.86 1.11 51 1.63 1.07 52 2.88 1.42 50 .56 .23 47

Control .42 .28 51' .19 .27 53 .27 .32 54 2,61 1.50 51 3.54 1.26 52 1.42 1.15, 53 2.88 1.50 52 .54 .22 50

All Schools

Target .55 ,DI 142 ,29 .28 368 .31 .31 170 3.18 1.38 351 1.-1 1.11 14. 1.''i 1.10.371 2,6P 1,37 343 .56 .23 321'

Control .54 .111 307 .31, .29 299 J,19 1.33 284 3.69 1.20 2r. 1.11 1.10 309 2,61 1.41 256 .56 .21 288

Note-The base N on which computations of official suspensions, explusions and withdrawals are based is all cases

randomly assigned to treatment and control conditions minus 24 cases who were not located in the schools as of Octo-

ber, 1981. The base N on which computations of all other outcomes are based is all cases randomly assigned to treat-

ment and control conditions minus 72 cases who had withdrawn from school as of February, 1982 and for whom no second

semester data were provided.

264

3 of 3

Table 10 (Cont'd)

M

Belief in

Rules

SD N

Negative

Peer Influence

M SD N

Alienation

M SD N

Victimization

M' SD N

Working

for Pay

M SD N

Have

M

a

job

SD N

Rebellious

Autonomy

M SD

Courtenay

Target .62 .24 61 .27 .23 68 .37 .22 62 .25 .22 69 .60 .49 70 .36 .66 70 .50** .30 5

Control .57 .25 60 .24 .18 66 .43 .27 60 .24 .27 60 .58 .50 71 .48 .74 70 .68 .28 6

Rhea

Target .63 .30 39 .22 .21 49 .33 .26 38 ,19 .25 44 ,62 .49 52 .46** .61 . 50 .57 .35 1

Control .59 .19 29 .18 .16 36 .44 .20 29 .18 .20 31 .50 .51 40 .18 .38 40 .67 .31. 2

Rivers

Target .55 .26 47 .28 .22 58 .40 .20 44 ,27 .25 51 .61 .49 64 .29 .52 65 .57 .27 .4

Control .60 .22 38 .24 .19 46 .38 ,22 37 .22 .22 37 .47 .50 51 .19 .52 52 .60 .29 3

Haut Cap

Target .59 .30 30 .21 .22 46 .43 .26 29 .16 .21 37 .51 .50 47 .47 .65 49 .61 .36 3

Control .59 .24 13 .24 .24 14 .39 .35 13 .27 .28 15 .44 .51 18 .39 .78 18 .61 .34 1

C. A. Brown

Target .62 .27 54 .24 .22 , 59 .26 .20 51 .10 .17 54 .37 ,49 59 .28 .52 60 .59 .35 5

tAl

Control

Burke

.67 .24 53 .22 .19 59 .27 .23 52 .09 .16 56 .37 .49 60 .33 .54 60 .56 .30 4

Target .64 .20 43 .21 .20 60 .37 .22 45 . .10 .18 58 .44* .50 63 .26 .51 64 .64 .28 4

Control .68 .24 45 .21 .19 55 .30 .23 43 .10' .16 46 .62 .49 58.46 .60 57 .56 .34 4

St. Johns

Target .70 .20 49 .17 .14 54 .42 .24 48 .11 .14 52 .55 .50 56 .39 .68 56 .56 .34 4

Control .63 .21 50 .22 .20 53 .41 .24 50 .13 .22 53 .55 .50 54 .28 .60 54 .60 .33 4

All Schools

Target .62 .26 323 .23 .21 394 7' 317 .17 .22 365 .53 .50 411 .35 .60 414 .57 .32 30

Control .62 .23 288 .22 .19 329 .16 .22 298 .51 .50 352 .34 .61 351 .61 .31 27

*Difference between target and control groups is significant at the p<.05 level.

**Difference between target and control groups Is significant at the p .01 level;

Note--The base N on which computations of official suspen ,1, explusio:Is and withdrawals are based is all cases

randomly assigned to treatment and control conditions minus . . cases who vere not located in the schools as of Octo-

ber, 1981. The base N on which computations of all other outcomes are ba ed is all case!; randomly assigned to treat-

ment and control conditions minus 72 cases who had withdrawn from school ,1 of February, 1982 and for whom no second

semester data were provided.

,oJ

Table 11

%

Percent Students Receiving PATHE Servicesand Number of Contacts - Burke

High School and other PATHE Schools

recdService

Number of Contacts

Mean SD Media

Burke High SchoolAll Target Students (N=86) 84.9 7.2 12.4 3.0

Target Students who took survey (N=59) 94.9 8.6 14.1 3.4

All Control Students (N=80) 55.0 2.4 6.5 .7

Control students who took survey (N=48) 62.5 2.a 7.9 .9

Other PATHE SchoolsAll Target Students (N=454) 90.7 6.6 6.4 4.9

Target Students who took survey (N=310) 94.8 7.6 6.8 5.6

All Control Students (N=384) 60.4 2.2 .2 1.0

Control Students who took survey (N=264) 62.9 2.1 3.1 1.0

Table 12

Fidelity Scale

The rater compares implemented activities and tasks as documented on theImplmementation Report to the standards for implementation describe onthis and subsequent pages. After checking off which of the standards havebeen met, a rating is assigned to each intervention using the followingscale:

4=exceeds standards

3=meets standards completely orwith approved adaptation

2=meets SO-99% of standards

1=meets 1-49% of standards

0=does not meet standards/not implemented

Intervention Standards

Tutoring outside tutors involved- regular' schedule of tutoring contacts

by C.S.-target students included

Counseling -RAP sessions plan on file-evidence of RAP sessions-evidence of at least 10 parent.contacts-contact with at least 90% ofstudents suspended-system established to monitorattendance problems-target students included in counseling

Study Skills -plan on file-evidence of implementation

Job Seeking Skills -distribution of materials-presentations to groups on job seekingskills or career awareness

SerVices to TargetStudents

-individualized instructional plan/needsassessment or problem statement forevery target student

-letter sent to parents-positive parent contact for every targetstudent-at least one contact with each targetstudent-link to summer enrichment program

Resource Room -needs assessment memo used after eachgrading period-resource room open house conducted-evidence of teacher use

Faculty Inservices -participation in team building conference-teacher participation in Student Team LearningLearning training-inservice plan on file which addressesidentified needs-evidence of implementation of plan

.School Pride -plan on file for 0,,1e year

Campaign -evidence of implementation

Source: Birdseye, A. T. Evaluating the Implementation of a Delinquoncv

Prevention Proe,ram:: The PATHE Experience. Paper presented at the

Annual Meeting of the American Educational Research Association, New

York, March 1982.

Tlhlo !' Coont.)Intervention Standards

Curriculum ReviewCampaign

-CTBS analysis on file-plan to improve identified weakness on file-evidence of meetings with teachers re plansand weaknesses-failure data collected and used to identifystudents and teachers who need help-help provided identified teachers and students-mini-tests in use-instructional materials and teaching strategiesdistributed-assistance in use of curriculu:. guides-at least one .teacher has adopted S.T.L. inclassroom-S.T.L. monitoring forms completed .

-test-taking motivational activities implemented

Field Trips -plan on file-one trip for each of three areas (career,academic, cultural)-scheduled throughout the year-planned trips implemented

Reading Experie:7ce

Program

-plan for school-wide program on file-evidence of implementation-motivational activities planned and implemented

Exploratory Program -schedule on file-display of exploratory products in school-schedule implemented

Discipline PolicyReview and Revision

- school and classroom rules developed by teachersand students and posted-referral procedures and discipline referral formapproved by principal-referral procedure implemented so that includesS.C.S.-assistance given to teachers with classroommanagement problems (at least 10)

Peer Counseling -counselors trailed-plan on file- counseling' services provided on regular schedule

ExtracurriculuarActivities

High school:-club plans or charters on file-club mid-year activity reports on file-end-of-year reports received

Middle school:-clubs begun last year continue

Student LeadershipTeam

-meetings held regularly-statisfactory team plan on file-evidence of plan implementation-central leadership conference attended-participated in planning spring conference-attended spring conference

Student ConcernsSupport Team

-regular meetings held-contracts on file-satisfactory team plan on file-evidence of plan implementation-final evaluation meeting conductedMiddle school:=team member agrees to serve as liason-members agree to serve without pay next year

Curriculum SupportTeam

(same as above)

-regular meetings held-contracts on file-satisfactory team plan on file-evidence of plan implementation

Table 13

Intensity Rating Scale

The rater compares intensity of program service's as documented on the Imple-mentation Report to the standards given below and rates each intervention usingthe :allowing scale:

4=exceeds standard by more than 5t

3=meets standard + 50

2=meets 50-95t of standard

1=meets 5-490 of standard

0=meets 4% or less of standard

Intervention Intensity Standard

Tutoring 400 incidents of tutoring

Counseling 600 incidentsof counseling

.Study Skills 100 incidents of participation

Job Seeking Skills 1 all school activity or 50 incidentsparticipation

Services to target 8 contacts with every target studentstudents

Resource Room and # teachers checkouts=2 Xservices to teachers # teachers

Faculty Inservices 3 school level, 5 teachers attendS.T.L., and 5 attend-team building --

School Pride 6 all school activitiesCampaign

Curriculum Review incidents C.S. faculty contact=4 X# teachers

Field Trip Program 3 field trip conducted

Reading Experience entire school, monthly, for more thanProgram 15 minutes

Exploratory ?rogram entire school, weekly

Discipline Policy # discipline referral forms.# suspensionsReview and Revision + h # suspensions

Peer Counseling 100 incidents contact related to peercounseling

Extracurricular H.S. - 10 club plansActivities M.S. - 100 incidents contact

Student Leadership' incidents attendance = # team members X 6Tears

Studer,:: Concerns 40 incidents attendanceSupport Team

Cuv:iculum Support 40 incidents attendance

Team

Career Explmration 120 incidents attendanceprograms 1T.T_C.)

Business Education 6 meetings or documentedcontactsPartnership (assembly, planning session)

Parent Leadership 30 incidents attendanceTeam

Source: Birdseye, A. T. Evaluating the Implementation of a Delinquency

Prevention Program: The PATHE Experience. Paper presented at the

Annual Meeting of the American Educational. Research Association,

New York, March1982. . . . .... .-iJe44.., , _A.... ,]

.. el-383-'- P---i 0 .ze....,.

,o, ,

,1 le,nt.)

Intervention Standards

Career ExplorationProgram

- students recruited to attend TridentTechnical College fall program-follow-up meeting with students held-students recruited for spring program- students recruited for FACET

Business EducationPartnership

-team formed with school and business members-PATHE specialists included on team- plan developed to improve school management-implementation of plan evident- plans for summer meetings on file

Parent LeadershipTeam

- regular meetings held-satisfactory team plans on file-evidence of implementation of plan-leadership training attended-participation in .spring training

Draft Table 14

Average of Intensity and Fidelity Ratings for Each PATHE Intervention

Courtenay Rhett Rivers Haut GapAllMiddle Brown Burke St. Johns

All

HighAIL

' Schoo

Team Average 1.7 2.7 1.9 2.2 2.1" 2.4 3.3 2.9 2.9 2Student Leadership 3.0 3.5 3.5 3.0 3.2 4.0 4.0 3.5 3.8 3.Student Concerns Support 1.5 3.5 2.0 2.5 2.4 2.0 2.0 3.5 2.5 2.Curriculum Support 1.5 3.5 2.0 2.5 2.4 2.0 3.0 3.0 2.7 2.Business-Education Partnership 0.0 0.5 0.0 0.0 0.1 1.0 4.0 2.0 2.3 1.Parent Leadership 2.5 2.5 2.0 3.0 2.5 3.0 3.5 2.5 3.0

Affective Average 2.8 2.3 2.6 1.9 2.4 2.2 3.2 2.2 2.6 .,

Counseling 3.5 2.5 2.5 1.5 2.5 1.5 3.0 2.5 2.3School Pride Campaign 3.5 0.0 3.5 2.0 2.2 3.5 S.5 2.5 3.2Field Trips 2.5 2.5 2.0 3.0 2.5 3.5 3.0 3.0 3.2Discipline Review & Revision 3.0 3.0 3.0 2.0 2.8 2.0 3.0 3.0 2.7 .2.Peer Counseling 2.0 2.0 0.5 1.5 1.5 0.0 3.5 2.5 2.0 1.Extra-curricular Activities 2.0 4.0 4.0 1.5 2.9 3.0 3.5 , 0.0 2.2 2.

..1

Academic Average 2.8 3.0 1.5 2.8 2.5 2.7 3.6 2.7 3.0 2..

Tutoring 1.5 3.0 2.5 0.5 1.9 2.5 3.5 2.0 2.7 2.Study Skills 3.5 3.5 1.5 2.5 2.R 2.5 4.0 2.0 2.8. 2.Resource Room 1.5 2.5 2.5 3.0 2.4 2.0 3.5 2.5 2.7

. 2.Curriculum Review & Revision 3.5 4.0 2.0 4.0 3.4 4.0 4.0' 3.5 3.8 3.,

Reading Experience ProgramExploratory

3.53.0

2.5

2.50.50.0

3.53.0

2.52.1

2.5a2a--

3.5__a a

3.0--

2.

2.

Career Average 2.0 0.0 0.0 3.0 1.2 2.8 3.0 3.0 2.9Job-Seeking Skills 2.0 0.0 0.0 3.0a 1.2 3.0 3.5 3.0 3.2 2.i

Career Exploration Programsa a a a

2.5 2.5 3.0 2.7

Other Average 1.8 1.0 1.0 2.0 1.4 2.5 2.5 1.8 2.2Services' to Target Students 1.5 1.0 1.0 1.5 1.2 1.5 1.5 1.5 1.5 1.,

Faculty Inserviees 2.0 1.0 1.0 2.5 1.6 3.5 3.5 '2.0 3.0

Total Implementation Average 2.4 2.4 1.8 2.3 2.2 2.5 3.2 2.6 2.8

aThis intervention was not implemented at this level.

fah*. 1$

thlki 0410044101 Osamu, 1911412toletterep 044411, Wool 14.464)

L.vstiv suwitizmvmmuvle-somoszsgswoalp-watwaswasigus

011410* 00001 taut Stwitott CoMful fitutl.ntbMIIZZZZ-.1:2A:W:Z-Xsa4aZWAkSoSt.*WWIDWIt, ,r&salm=s=sc aaaaaaa am*

Mg. Avg. Avg.

00s NO. osI OW 0010 twitting contacts

Poppet* 9110* et otroloate pot Se. at per No. of per

04 **0440 Memiatt4 0ottotted etedtet oottott Hotfoot contacts st,wous-austuu.,42,4,=1-1,-muzzosupwww*mamemwwwwelm.....w...................................

IOW 414441i4,timeroolia4

14**4044or144404 toodet0

lea

Ore1044010$

.47

017

AO*14

/094A/

1.54,k,

475611//

39

5.72**Al

2.13.47**

243

53

146

7

t

. ,,

1.62

.09

21444 *4444414,

hot romeeetle0

12

001

.01

A/.03

3

0.04

OA1

1

.01

.01

644$ P. 56 AO .00 30 .36** 2 .02

imeteeerre POI 0/5 ,41 ISO IM0* 12 .15

146644401***iielae 44 A/ *13 10 .I: 12 .15

414#.14*lOokkil pttetto 0 *SO *OS 0 .00 0 .00

'wrote& 4 .16 .16 14 .19 9 .11

04404 .1600.044 161 ,10 40 47 .51** 16 .20

Opeehatiol tesivire 01 .10 .19 SI .37 11 .14

Peet me. 0 .00 *OS 0 .00 0 .00

PW104 00440 t .01 .01 1 .01 1 .01

4144111440 ta4 et4es It .04 .06 2 .02 3 .04

*ear milt, 040116104 0 .00 .00 0 .00 0 .00

0004 0 .00 .00 0 .00 0 .00

1464004 Sechrtral 0 *SO .00 .0 .00 .0 .00

6444401444ca1 kelt It *SS *04 13 .16** 1 .01

00000014441 3.19 517 6.35** 26$ 3.35IMMO 1 20110410=e1:02231tiiiisagaiodamaistse4usoAmmossaima.spowalossamaroa. 111411141.411.1114

*141Swerree Seemerft tempt sr il 4444441 stedeste to sissifitaat at the p4.0$ level.440411444444 66,46606 Wen art 4464,44 strie444 tr eigattiaaat at the 14.01 level.

273

Table 16

Purposeof contact

No. ofcontacts

PATUE Specialist Contacts, 1981-82nett Middle School (N=510)

Entire School Target Students

Avg. Avg.no. no.

% contacts contactsstudents per No. of percontacted student contacts student

Control Students

Avg.

no,

contactsNo. of per

contacts student

Total Affective 1986 .65 4.04 498 7.90 ** 130 2.50Counseling 480 .32 .98 220 3.49** 37 .71

Discipline 218 .29 .44 53 .84 28 .54

Student Leadership 234 .19 .48 25 .4r 8 .15

TeamPeer counselee 0 .00 .00 1 .02. 0

Peer counselor 58 .03 .12 9 .14 0 --

Field trip 0 .00 .00 0 .00 0 .00

Conference 174 .20 .35 98 1.56** 17 .33

Clubs/extracurricularactivity

179 .06 .36 2 .03 0

Special program 565 .18 .15 72 1.14 32 .62

Rap session 78 .14 .16 18 .28 8 .15

Total Academic 641 .22 1.30 , 359 5.69** 17 .33

Specialist tutoring 639 .22 1.30 359 5.70** 17 .33

Peer tutee 0 .00 .00 , 0 .00 0 .00

Peer tutor 0 .00 .00 0 .00 0 .00

''Outside" tutoring 0 .00 .00 0 .00 0 .00

Study skills training 0 .00 .001

0 .00 0 .00

Test 0 .00 .00 0 .00 0 .00

Trident Technical 0 .00 .00 0 .00 0 .00

CollegeCentral Staff 0 .00 .00 0 .00 0 .00

program1

Total 2637 .68 5.36 861 13.67** 148 2.85

*Difference between target and control students is significant at the p<.05 level.**Difference between target and control students is significant at the p<.01 level.

274

Table 17

PATHE Specialist Contacts, 1981-82

Purposeof contact

No. ofcontacts

Rivers Middle School

Entire School

Avg.

no.

contactsstudents percontacted student

-

(N=530)

Target Students

Avg.

no.

contactsNo. of per

contacts student

Control Students

Avg.

no.

contacts

No. of per

contacts student

Total Affective 1216 .61 2.26 402 4.73** 160 2.13Counseling 429 .36 .80 166 1.95** 77 1.03Discipline 108 .13 .20 25 .29 25 .33

Student Leadership 312 .18 .58 124 1.46** 32 .43

TeamPeer counselee 1 .00 .00 0 .00 0 .00

Peer counselor 0 .00 .00 0 .00 0 .00

Field trip 0 .00 .00 0 .00 0 .00

Conference 49 .05 .09 44 0 .00

Clubs/extracurricularactivity

176 .06 .33 24

,...52**

.28 5 .07

Special program 141 .25 .26. 19 .22 21 .28

Rap session 0 .00 .00 0 .00 0 .00

Total Academic 369 .25 .68 169 1.99** 53 .71

Specialist tutoring .311 .19 .58 162 1.90** 47 .63'

Peer tutee 0 .00 .00 0 .00 0 .00

Peer tutor 22 .03 .04 0 .00 2 .03

"Outside" tutoring 0 .00 .00 0 .00 0 .00

Study skills training 2 .00 .00 0 .00 1 .01

Test 0 .00 .00 0 .00 0 .00

Trident Technical 0 .00 .00 0 .00 0 .00

College.

Central Staffprogram

34 .06 .06 7 .08 3 .04

Total 1588 .65 2.95 572 6.73** 213 2.84

*Difference between target and control students is significant at the p<.05 level.**Difference between target and control students is significant at the p<.01 level.

27i

Table 18

Purposeof contact

PATHE Specialist Contacts, 1981-82Haut Gap Middle School (N=446)

Entire School Target Students

Avg. Avg.

no. no.

% contacts contactsNo. of students per - No. of per

contacts contacted student contacts student

Control Studer

Avi

n(

cons:

No. of IN

contacts stu(

Total Affective 510 .36 1.10 237 3.70** 2

Counseling 112 .14 .24 57 .89** 0

Discipline 70 .08 .15 7 .11 1

Student Leadership 53 .04 .11 5 .08 0

TeamPeer counselee 4 .01 .01 0 .00 0

Peer counselor 43 .04 .09 21 .33 0

Field trip 33 .07 .07 17 .26** 0

Conference 154 .14 .33 120 1'.88** 0

Clubs/extracurricularactivity

0 .00 .00 0 .00 0

Special program 37 .08 .08 8 .12 1

Rap session 4 .01 .01 2 .03 0

Total Academic 113 .07 .24 7 .11 0

Specialist tutoring 86 .02 .19 0 .00 0

Peer tutee 0 .00 .00 0 .00 0

Peer tutor 0 .00 .00 0 .00 0

"Outside" tutoring 2 .00 .00 0 .00 0

Study skills training 23 .05 .05 6 .09 0

Test 1 .00 .00 1 .02 0

Trident Technical 1 .00 .00 0 .00 0

CollegeCentral Staff

program

0 .00 .00 0 .00 0

Total 623 .38 1.35 244 3.81** 2

*Difference between target and control students is significant at the p<.05 level.**Difference between target and control students is significant at the p<.01 level.

276

Table 19

Purposeof contact

PATHE Specialist Contacts, 1981-82Brown High School (N=659)

Entire School Target Students

Avg. Avg.no. no.

% contacts contactsNo. of students per No. of per

contacts contacted student contacts student

Control Students

Avg.

no.

contactsNo. of per

contacts student

Total Affective 760 .38 1.15 197 2.43** 34 .36

Counseling 210 .18 .32 55 .68** 16 .20

Discipline 27 .04 .04 8 .10 2 .02

Student Leadership 276 .08 .42 33 .41** 2 .02Team

Peer counselee 0 .00 .00 1 .01 0 .00

Peer counselor 0 .00 .00 0 .00 0 .00Field trip 103 .15 .16 20 z..25* 9 .12Conference 138 .14 .21 77 .95** 5 .06

Clubs/extracurricularactivity

6 .00 .01 3 .04 0 .00

Special program 0 .00 .00 0 .00 0 .00

Rap session 47 .05 .05 2 .02 7 .09

Total Academic 424 .20 .64 116 1.43 28 .36

Specialist tutoring 257 .10 .39 100 1.23 18 .23

Peer tutee 29 .02 .04 10 .12 9 .00

Peer tutor 60 .03 .09 0 .00 1 .01

"Outside" tutoring 2 .00 .00 2 .02 0 .00

Study skills training 0 .00 .00 0 .00 0 .00

Test 4 .00 .01 1 .01 0. .00

Trident Technical 72 .10 .11 3 .04 9 .12

CollegeCentral Staff

program0 .00 .00 0 .00 0 .00

Total 1203 .46 1.82 314 3.88** 62 .79

*Difference between target and control students is significant at the p<.05 level.* *Difference between target and control students is significant at the p<.01 level.

277

Table 20

PATHE Specialist Contacts, '1981-82Burke High School (N=1005)

Purposeof contact

No. ofcontacts

Entire School

Avg.

no.

% contactsstudents percontacted student

Target Students

Avg.

no..

contactsN.:., of per

corts student

ontrol Students

Avg.

no.

contactsNo. lf per

cont. -4tv..1.ent

Total Affective 1106 .49 1.22 22, 2.64** 1 0,) .:.".:5

Counseling 165 .14 .18 73 .85** ll, .18

Discipline 494 .33 .54 59 .69 52 .65

Student Leadership. 182 .06 .20 16 .19 1:i .16

TeamPeer counselee 68 .03 .07 8 .09 5 a .06

Peer counselor 60 .03 .07 14 .16 4 .05

Field trip 28 .03 .03 4 °.05 3 .04Conference 62 .06 .07 51 .59** 2 .02Clubs/extracurricular

activity0 .00 .00 0 .00 0 .'00

Special program 0 .00 .00 0 .00 0 .00

Rap session 47 .05 .05 2 .02 7 .09Total Academic 883 .15 .97 391 4.55* 94 1.18

Specialist tutoring 810 .13 .F9 360 4.19* 94 1.18Peer tutee 0 .00 .00 0 .00 0 .00

Peer tutor 3 .00 .00 0 .00 0 .00

"Outside" tutoring 54 .02 .06 31 .36* 0 .00

Study skills training 0 .00 .00 0 .00 0 .00

Test . 0 .00 .00 0 .00 0 .00

Trident Technical 16 .02 .02 0 .00 0 .00

CollegeCentral Start

program0 .00 .00 0 .00 0 .00

Total 1996 .53 2.20 621 7.22** 194 2.42.

*Difference between target and control students is significant at the p<.05 level.**Difference between target and control students is significant at the p<.01 level.

1278

Table 21

Purposeof contact

PATHE Specialist Contacts, 1A1-82St. Johns High School (N=808)

Entire School Target Students

Avg. Avg.

no. no.

% contacts contacts

No. of students per No. of per

contacts contacted student contacts student

Control Students

Avg.

no.

contact.

No. of per

contacts studen.

Total Affective 1660 .70 1.99 375 4.181 ** 113 1.49

Counseling 411 .27 .49 47 .60 48 .63

Discipline 69 .06 .08 8 .10 7 .09

Student Leadership 151 .08 .18 53 .68** 3 .04

TeamPeer counselee 6 .01 .01 1 .01 1 .01

Peer counselor 81 .02 .10 4 .05 0 .00

Field trip 0 .00 .00 0 .00 0 .00

Conference 290 .12 .35 205 2.63** 0 .00

Clubs/extracurricularactivity

5 .01 .01 0 .00 0 ;00

Special program 59 .02 .07 4 .05 11 .14

Rap session 588 .49 .70 53 .68 43 .56

Total Academic 491 .31 .59 105 1..75** 29 .38

Specialist tutoring 250 .16 .30 72 .92* 20 .26

Peer tutee 4 .00 .00 2 .02 0 .00

Peer tutor 47 .02 .06 11 .14 1 .01

"Outside" tutoring 0 .00 .00 0 .00 0 .00

Study skills training 33 .04 .04 i 5 .06 2 .03

Test 62 .07 .07 7 .09 2 .03

Trident Technical 94 .08 .11 I 8 .10 4 .05

CollegeCentral Staff

program

1 .00 .00 0 .00 0 .00

Total 2279 .77 2.73 495 6.34** 155 2.04

*Difference between target and control students is significant at the p<.05 level.

**Difference between target and control students is significant at the p.01 level.

273

Virgin Islands Alternative Education Project

J. St. John

Abstract

The Virgin Islands Alternative Education Project is one of a num-ber of demonstration projects sponsored by the Office for JuvenileJustice and Delinquency Prevention as part of its Program in Delin-quency Prevention through Alternative Education. It is seeking toimplement two interventions disseminated by the National DiffusionNetwork (NDN)--Focus and PATL. Interim evaluation results suggestthat Focus is being implemented largely as intended, but with somechanges from the original Focus model. PATL is being implemented toa more limited degree, due largely to a limited number of teachersimplementing the model. Despite some weaknesses in the evaluationdesign and measures, results also ,suggest that the modified Focusintervention resulted in students receiving higher grades than theyotherwise may have received. Although project implementers believethey perceive positive results as a result of their activities, noother consequences of the Focus intervention were detected usingmeasures obtained in the evaluation. This outcome may be a resultof the small number of teachers implementing the intervention or toincomplete implementation. Some positive changes in the school cli-mate were observed, but there is no evidence in a comparison of stu-dents exposed in differing numbers of teachers trained in PATLindthat the limited implementation of PATL has had detectable effectson students. Program development is continuing, as are efforts tostrengthen the evaluation.

Major Program Design Changes

The Virgin Islands AlternativeEducation project continued its

efforts to implement the NationalDiffusion Network (MN) Focus (FocusDissemination Project, 1979, 1980)

and PATL (Pelan, Zirges, Scroggins,& Arterberry, 1975) programs duringthe 1981-82 school year. Majorchanges in the program included theopening of an Alternative Educationbuilding on the Elena ChristianJunior High School campus. Althoughthe building had only two of the

four needed classrooms, the

This report covers the period ofSeptember 1981 to June 1982.

-393-

structural beginnings of a school-within-a-school were launched forthe Focus Program. In addition, a

teacher coordinator was appointedfor the PATL Program and peer train-ing among teachers began anew.

The staff could not attend theAugust, 1981, planning workshop.Consequently their Program Develop-ment Evaluation (PDE) worksheets didnot detail plans for the 1981-82academic year. Without these plansit was difficult for us to knowwhich program components they

intended to implement or the degreeof implementation success. Forexample, an intensive public rela-tions campaign was a component dur-ing the first implementation year.The campaign took on a new slant thesecond year because the school

250

Virgin Islands

system incorporated alternative edu-cation and there was no longer a

need to lobby for survival. Thisdevelopment resulted in a change inproject priorities.

Major Forcefield Changes

In the fall, the director of theproject, representing the Commis-sioner's Office, recommended the

advisability of the services of a

clinical psychologist on the proj-ect. He suggested that psychologi-cal testing (including intelligencetesting) be used as a part of the

selection process. and that indiVi-dual behavior management programs bebased on the use of psychologicaltests. The underlying philosophy ofthis approach is counter to the phi-losophy of project staff and wasregarded as counter to the philoso-phy of Focus as well. In addition,the cost of implementing this recom-mendation was prohibitive.. There-fore, after negotiations with the

ps -Thologist, the project decidednot to implement most of the recom-mendations, although changes in

record-keeping procedures wereadopted.

The State Department of Educationcreated a division of AlternativeEducation in the summer of 1981.

During the school year the depart-ment recruited staff, chose cl.rricu-lum, and began program planning withseveral junior high schools in thesystem. The funded in-school coor-dinator helped the department estab-lish its new division and consultedwith other schools.

By late Spring, the in-schoolcoordinator had been promoted to

Director of Alternative Educationfor the Virgin Islands school sy.3-

tem. One of the program's teaches._took on the project coordinatorresponsibilities. The spring alsobrought faculty requests for trans-

-394-

fer to a n- school expected to openin the fall. All but one of the

Focus-tra. ted staff requested trans-fer, and by Augu....t. the new in-school

coordinator knew she would have atleast three untrained teachers join-ing the staff. Fortunately, new

teachers who were acquainted withFocus and were committed to its

ideals joined the staff. New train-ing sessions were held with NDN

trainers during the first weeks ofthe 1982-83 school year while stu-dent recruitment was in progress.Staff .began the upcoming school yearwith excitement and optimism.

Implementation

We attempted to assist in programplanning and PDE updating during1981-82 by phone and mail. This

method was time consuming, and

resulted in incomplete documentationof project plans and their execu-

tion. As the -school" year- pro-

gressed, the Focus staff devoted

considerable time to discussion ofthe Project Director's recommenda-tions (discussed earlier), and thismay have slowed further developmentof the Focus intervention.

Our obF-lvations suggest that

more monitoring of the Focus inter-vention in the cl ssroom; more in-

school cool,inator attention to

attendance, discipline; and moreattention to seeking family assist-ance with school discipline problemswould have been useful.

PATL Component

PATL is a teacher training pack-age intended to a) help teacherslearn alternative teaching and dis-ciplinary methods, and b) confronttheir attitudes toward students andthe effects these attitudes have onstudents. The training program pro-ceeds in several steps. First,

teachers who volunteer to teach the

28

curriculum participate in anintroductory four-day workshop.After the workshop, trainees formgroups of about five members. PATLhas four training "kits"--traininggroup members work together to com-plete the kits. Each kit has learn-ing objectives that must be masteredsequentially. Mastery is demon-strated by a 90% proficiency ratingon an in-classroom observation of

the techniques. The PATL programhas four stages of implementationthat correspomi with the completionof each kit. In addition, there arethree levels of implementation foreach kit: (1) training completed,(2) in process of implementation,and (3) implemented. PATL develop-ers expect the program to beginaffecting those students havingPATL-trained teachers in at leastthree of their classes; minimally,those teachers must be implementingPATL at the first of the four imple-mentation stages.

Plans to strengthen' the 1981-82PATL component were made during theprevious year when project managersrealized the component was not beinggiven sufficient attention. Manag-ers planned for the appointment of aTeacher Coordinator. The coordina-tor was to oversee the recruitmentof new teachers for peer trainingand structure the program. The NDNPATL trainer was scheduled to returnfor a refresher workshop in Septem-ber. Each of these steps wereimplemented. The Teacher Coordina-tor reported, however, that peer

training was not going as expectedbecause peer trainers and traineeshad difficulty finding time to meet.This was largely due to the school'sdouble session school day resultingin rigid schedules for teachers.

Consequently, by late spring manytrainees had reportedly covered onlyone or two of the objectives in thefirst ,training kit. During an

August, 1982, planning workshop,project managers reported that not

Virgin Islands

all teachers who had participated inthe PATL training were implementingthe PATL techniques due to insuffi-cient knowledge. Therefore, imple-mentation was probably at level 3 ofstage 1 for some teachers.

Focus Component

In March, 1982, I visited the

project site to learn what had beenimplemented. The assistance of an

NDN consultant on focus was soughtto prepare to assess implementationof the major program components ofFocus before the visit. The consul-tant supplied a checklist of 11 com-

,ponents that included ranges of

acceptable implementation. The fol-lowing on-site observations are out-lined according to NDN's checklist.

1. Family group meeting. Focuscalls for students to engage in

daily problem-solving sessionstogether with Focus teachers. Stu-

dents in the project were groupedinto four "family" units. One classperiod each day was set aside forthe family groups to meet sepa-

rately. The observed family groupwas participating in a values clari-fication exercise and not observedresolving the problems of members inthat particular session. Staff mem-bers interviewed reported that the

family meetings were handled differ-ently by each staff member, but theydid not fully describe what occurredin the meetings.

2. Student leadership board.

Focus calls for a student leadershipboard to help resolve problems notsolved in family meetings. Project

staff thought this component shouldnot be implemented until the projecthad been operating for several yearsand effects had taken hold.

Instead, students were invited to

attend staff meetings when problemsneeded attention. This change is

viewed by the State Facilitator as

-395-

282

Virgin Islands

an adaptation of the program modelrather as an interim strategyrequired while other aspects of theproject develop. Indeed, the facilitator views the entire Focus modelas open to adaptation rather than asa prescriptive model for intervention design.

3. Mini school. The Focus modelcalls for a designated Focus area beestablished within the school, andfor Focus staff to have their ownoffices. At the project site, a

campus Alternative Education building had been opened but two of thefour Focus classes were being heldelsewhere in the school building.Division of the two Focus classroomsinto four rooms was scheduled forcompletion during the summer of

1982.

4. Parental involvement. Focuscalls for "some kind of positivesupport" on the part of parents. At

the Virgin Islands project, parentsof project participants must sign

permission slips before their children can be considered for selection. In addition, project planscall for teachers to report to

parents their students' progress,and to seek parents' help with discipline problems. Records reviewedon site showed that at least some

parents were contacted by someteachers, but no systematic reporting occurred.

5. Program contracts. Focuscalls for students to set individualcontractual goals. At least one

project staff person had contractedwith students in her group. Thecontracts served as a means of monitoring and measuring progress.

6. Student ownership of the program. Focus sets up the expectationthat students "will demonstrate loyalty to the program in some way."Project managers reported that stu

dents seemed to be exhibitingloyalty and cited student statementsof support. At the end of the

school year, however, some studentsrequested they not be listed as

Focus students on the school promotion list.

7. Use of reality feedback andconfrontation. Focus calls for theconsistent application of realityoriented feedback, and staff and student confrontation of inappropriatebehavior. Such an apprOach seeks toinsure that students receive specific and immediate feedback on

their behavior and are given an,opportunity to change. Some of theteaching staff were using thesetechniques and others were not. In

one Focus classroom a studententered late and was wearing a hat.The teacher confronted his latenessand the student gave the teacher alate pass. Wearing a _hat wasagainst school rules and the teach-erasked him what the hat was doing onhis head. He promptly removed thehat. In another Focus classroom thesame behaviors by the same studentwent without comment.

396

8. Scheduling student's day.

Focus calls for at least four hoursof instruction a day in the program.Focustrained staff should teach

Focus students a Focusdevelopedfamily curriculum, and three of thestudents' regular academic subjects.Each project family group was scheduled in five Focustaught coursesand in regular school courses forthe rest of the day. The Focuscourses were occupational relations,family, English, math, and social

science. The occupational relationscourse was designed to expose students to careers of all types, helpstudents develop job skills, andhelp students gain everyday practical skills.

283

9. Community involvement. Focusrecommends that junior high studentsbe involved in one community serviceproject each semester. Senior highstudents should either participatein jobs or community service projects. Over the course of implementation, five project students havehad jobs. A public relations campaign implemented by project staffwas substituted for the recommendedstudent community service projects.

10. Assessment. Focus recommends that Focus participants be

pre and posttested in "three mainacademic areas". Project managersimproved on this simple prepostevaluation design by attempting to

implement a more rigorous quasiexperimental design. They added to

the assessment measurements relevantto a delinquency prevention demonstration project. Next year the

project will be implementing a trueexperiment and readopting the prepost approach as a method of monitoring student progress.

11. Admission policy. Focusrecommends that current participantsand staff decide which new applicants should be admitted. The project staff substituted an admissionspolicy that both identifies thetroubled students the program is

intended to serve, and allows forthe formation of participant andcontrol groups necessary for an

evaluation (see below for a moredetailed description of the selection process).

Other observations. The Focusprogram does not prescribe curriculum or classroom methods beyond theuse of reality feedback and confrontation; consequently each teacherwas using either the required VirginIslands curriculum, or curriculum oftheir own design for the familyclasses. Teaching methods were upto the teaching staff. For

397

Virgin Islands

instance, the English teacherreported using a highly individualized approach that was structured bytest results, while the math teacherreported using a model where students studied when they wanted. In

the English classroom, the teacherand student developed learningobjectives, and students worked onindividualized materials. In the

math classroom, the teacher reportedthat students were given a specifictime period to master assignedmaterials from the regular mathbook, .and were tested after eachunit. In both classrooms the teachers gave students individual attention.

On the day I observed the

classes, attendance was low. Whileobserving one class with about 10students present, students from theother class in the alternative education building were milling abouton the porch. In a later Focusclass, only three of the studentspresent in the class of 10 observedearlier were on time. One more ofthe students came to class later.

Observation limited to a singleday is not, of course, a dependableway of describing a school. Thetypical day may have looked eitherbetter or worse than the oneobserved.

Effectiveness

Focus Evaluation Design

The national evaluation staff andproject staff agreed to study Focuseffects by using a regression discontinuity design (Cook and Campbell, 1979). This design was chosenbecause it allowed services to be

given to students most in need ofthe program. The project staff

developed a screening device on

which referring teachers were asked

284

Virgin Islands

to rate students' academicabilities, and indicate attendanceand behavior problems. Students'scores in the first two waves of

eligible referrals were ranked, andthe median score determined the

selection pool. Those scoring abovethe median became Focus participants, and all others became the

control group. Students referred tothe program later had to score abovethe original median score to beselected as participants. Thisscore was intended to serve as preparticipation information to be usedin statistical analysis.

Project staff executed each stepfor design implementation as

planned, but the design broke downbecause of a weak relation betweenscreening device scores and outcomes. (Interpretable regressiondiscontinuity designs depend upon astrong relation between pretreatment measures and outcomes.)

Selection procedures for the

regression discontinuity design hadresulted in nonequivalent groups.Table 1 illustrates that small differei.ces between Focus and comparison students are apparent for gradesin the first quarter of the schoolyear (before the Focus interventionswere begun in earnest) and a moderate difference in attendance. Wehad forseen the possibiliry of

design failure and had administereda brief version of the School ActionEffectiveness Study (SAES) studentquestionnaire in early January,about five school weeks after moststudents were admitted to the program.

Administration of the SAES. questimnaire pretest, coupled with' thecontrols originally identified forthe regression discontinuity designcreated a quasiexperimental nonequivalent control group design. Thepretest included SAES scale3 measur-

-398

ing Alienation, Belief, SchoolEffort, Practical Knowledge, Positive SelfConcept, School Attachment, Selfreported Total and Serious Delinquency, and SelfreportedDrug Involvement. Items on selfreported school and class skipping,and suspensions were also included.These scales and items were usedeither because the project wasexpected to affect these outcomes,or because they are known to be

strong correlates of delinquency.Scores were derived as they had beenfor SAES scales in the Spring of

1982--summing the standardizedscores of the items comprising thescales (Gottfredson, Gottfredson,Ogawa, and Rickert, 1982).

School data from the first schoolquarter were also used as preparticipation measures because the project staff were recruiting and scheduling students at that time, andformal amplementatiOn-did not beginuntil the second quarter of the

school year. Although the questionnaire pretest did not occur untilJgnuary, it was also used as a

source of "control" variables in theanalyses because of the nonequivalence of the treatment and comparison groups. Its use, is a conservative procedure. Anypreparticipation measures (eitherschool or questionnairebased) thatcorrelated with both treatment andoutcome were used as statisticalcontrols for outcomes measured at

the end of the year.

Outcome Measures

The SAES questionnaire was administered in late April. The projectevaluator reported school and classroom attendance, school grades at

the end of quarters one and two, andagain after the second semester wascompleted. School discipline datawere reported at the end of first

285

and second quarters. MetropolitanReading Test scores from the falland spring administrations andpolice contact information for the

previous year were reported in June,1982. Unfortunately, 47% of the

Spring achievement test data for theFocus students were missing, makingthese data of little value.

Results

Focus. Mean scores on the questionnaire administered in Spring,1982, and means for outcomes measured through archival records areshown in Table 2. The grade pointaverage (CPA) for Focus participantswas significantly (p<.01) higherthan the grade point average for thecomparison students. No other difference shown in Table 2 is statistically significant. Two other outcomes measured by springquestionnaire items differed significantly. Selfreported gradesfavored the Focus group (p<.05), andFocus students reported less classskipping than did the comparisonstudents (p<.01).

Recall that the comparison groupwas not equivalent to the Focusgroup at the outset, however. In

other words, the treatment and comparison group differed systematically at the outset in ways thatwould lead them to differ at a latertime in systematic ways regardlessof whether or not they were exposedto any intervention. In such circumstances it is necessary to use

statistical techniques to adjustpostintervention differencesdifferences to be expected on the

basis of these preexisting differences. Table 3 shows the unadjustedassociation (r) of Focus participation with GPA, selfreported grades,and selfreported class skipping.

It also shows the association whenstatistical controls are used to

adjust for the measures of preex-

-3997

Virgin Islands

isting differences among students

(beta). When statistical controlsare applied, only the association ofFocus participation with GPA remainsstatistically significant. Theadjusted associations observed for

the other two outcomes, although inthe postive direction, are not statistically significant.

PATL. Table 4 describes PATLoutcomes grouped according to the

number of teachers exposed to PATL astudent had, That is, if a studenthad one teacher who had completed afourday workshop, he or she is

listed under "One PATL Teacher." If

the student had two such teachers,he or she is included in the "TwoPATL Teacher" group. The tableclassifies students in this way,

irrespective of the extent of continued peer training and of degreeof implementation of PATL methods byteachers. No differences are statistically signficant. There is noconsistent relation between the number of teachers exposed to PATL andstudent outcomes.

Discussion

Focus

. New project managers were askedwhether the improved grade pointaverage of Focus participants wasindicative of mastery of the subjectmatter, or reward for effort. They

believed that the students' gradeswere based more on effort than mastery because the Focus philosophycalls for the use of such rewards asmotivators for further improvement.Because of this, the results for GPAmay represent a measure of implementation, and may not address the

question of mastery.

The most direct way to assessmastery is through the use of standardized achievement tests. Unfor-

286

Virgin Islands

tunately, standardized achievementtest scores were available for only53% of the students, and could notbe used to address the issue of mas-tery. The evidence is clear thatgrades where higher than they other-wise would have been for treatmentstudents; the interpretation of thisresult is necessarily ambiguous.

Observations of the implementa-tion of Focus are encouraging.Although not yet implemented as perNDN specifications, 1981-82 Focusimplementation improved substan-tially over the previous year. For1982-83, the new in-school coordina-tor is committed to implementationimprovements. She is especiallycommitted to developing strategiesfor increasing class attendance.Finally, the managers' commitment toa stronger experimental designshould reduce evaluation problemsand give a better picture of 1982-83effects.

PATL

PATL implementation did not occuras planned. The project plans to

strengthen both the PAIL trainingcomponent, and the implementation ofPATL concepts after training is com-

-400-

pleted. The new in-school coordina-tor plans to work closely with thePATL teacher coordinator to removeobstacles to peer training. Peri-odic visits to classrooms of trainedteachers are expected to improve thethe match between program design andimplementation.

School-Level Results

Results reported elsewhere (Gott-fredson, Gottfredson, & Cook, 1983,Chapter 5) show some- changes inschool, climate between 1981 and1982. Specifically, increases wereobserved in teacher job satisfac-tion, teacher non-authoritarianattitudes, and students' reports ofschool rewards. Decreases wereobserved in self-reported schoolgrades, students' reports of commu-nity crime, and students' reports ofdrug involvement and delinquentbehavior more generally. Althoughthe detailed interpretation-of-theseresults is difficult, they areencouraging. Decreases in reportsof classroom disruption, which weexpected to observe because projectstaff believed this to be a conse-quence of placing disruptive stu-dents in the Focus program, were notobserved.

287

Virgin Islands

References

Cook, T. D., & Campbell, D. T. Quasiexperimentation. Chicago: RandMcNally, 1979.

Focus Dissemination Project. Focus curriculum manual (2nd ed.). Hastings,Minn.: Educational Growth Exchange, 1980.

Focus Dissemination Project. Focus training manual (3rd ed.). Hastings,Minn.: Educational Growth Exchange, 1979.

Gottfredson, G. D., Gottfredson, D. C., & Cook, M. S. The School ActionEffectiveness Study: Second interim report (Report No. 342). -Baltimore:The Johns Hopkins University, Center for Social Organization of Schools,1983.

Gottfredson, G. D., Rickert, D. E.. Jr., Ogawa, D. K., & Gottfredsdh, D. C.Measures used in the School Action Effectiveness Study. In G. D. Gottfredson (ed.), The School Action Effectiveness Study: First interim report.Baltimore: The Johns Hopkins University, CSOS, 1982.

Pelan, C., Zirges, J., Scroggins, F., & Arterberry, M. Positive attitudetoward learning. Washington, D.C.: Office of Education, Department ofHealth, Education, and Welfare, 1975.

401

28R

Table 1

Means and Standard Deviations for Focus and Comparison GroupFirst Quarter 1981-82 Pre-Intervention Measures

Measure

Grade point average-

Focus Comparisonti SD N M SD

all subjects .1.59 .76 51 1.41 .72 49

Grade point average-core subjects 1.43 .80 51 1.19 .73 49

Total school days absent 7.10 8.16 41 3.68 6.03 37

-402-

2RQ

Table 2

Means and Standard Deviations for Focus and Comparison GroupSpring 1982

Outcome measure

Questionnaire-based

Focus ComparisonN SD N M SD

Alienation .47 .24 25 .49 .25 28Attachment to parents .52 .28 30 .57 .28 31Belief .57 .28 28 .64 .26 28Interpersonal competency .68 .27 28 .72 .24 26Negative peer influence .28 .20 30 .21 .18 30Parental emphasis on education .52 .30 29 .49 .34 28Positive self-concept .61 .15 24 .56 .20 24Practical knowledge 1.31 .45 28 1.13 .45 26Rebellious autonomy .50 .30 26 .53 .28 27School attachment .58 ,:i .22 26 .66 .27 31School effort .52 .32 29 .61 .27 29School involvement .22 .19 29 .25 .17 26School punishment .27 .33 29 .26 .26 26School rewards .39 .32 29 .35 .36 26Self-reported delinquency-

total .12 .15 25 .10 .12 27Self-reported drug involvementself-reported serious

delinquency

.14

.10

.22

.15

26

25

.12

.07

.15

.13

27

27Victimization .22 .25t 29 .20 .26 26

School recordsDisciplinary infractions a a a a a a

Grade point average 1.44 .92 49 .97 .74 45Total absence 20.00 22.78 49 15.55 20.13 44

Police recordsPolice contacts

aData not reported

.09 .29 54 .12 .48 51

-403-

29n

Table 3

Correlation Coefficients andanti Standardized Regression Coefficients

in the Regressions of Outcomes on Focus ParticipationSpring 1982

MeasuresCoefficients

r beta

School report of gradesa .267** .274* 58

Self-report of gradesb

Self-report of class skippingb

-i300* .189 -45

.324** .220 46

Note. Table shows the three outcome variables withsignificant correlations. N shown is the lowestpairwise n.

aFirst quarter school reports of grade point averageand absenteeism, and January SAES measures ofAlienation and Serious delinquency were used asstatistical controls.

bFirst quarter school reports of absenteeism andclass skipping, and January SAES measures of Alien-ation, Serious delinquency, School attachment, andSelf-reported class skipping were used as statisti-cal controls.

*p<.05**p<.01

-404-

2q1

Table 4

Means and Standard Deviations for Students byNumber of PATL- Trained Teachers, Spring 1982

Outcome measure

No PATL One PATL Two PATL Three PATL

Teacher Teacher Teachers Teachers

SD N M SD N M SD N M SD

Questionnaire-basedAlienation .41 .28 18 .40 .22 47 .43 .21 44 .42 .27 24

Attachment to parents .71 .18 20 .63 .24 50 .64 .29 50 .57 .29 25

Belief .62 .22 19 .63 .26 50 .66 .24 44 .62 .25 23

Interpersonal competency .83 .17 19 .77 .24 49 .77 .28 41 .74 .24 24

Negative peer influence .15 .17 21 .17 .18 49 .21 .20 50 .20 .15 23

Parental emphasis on education .63 .29 21 .60 .29 50 .61 .31 44 .61 .34 24

Positive self-concept .72 .20 19 .68 .21 45 .71 .19 39 .67 .17 20

Practical knowledge 1.33 .36 19 1.24 .44 49 1.14 .41 42 1.19 .44 23

Rebellious autonomy .32 .36 19 .54 .37 49 .58 .33 43 .61 .27 22

School attachment .76 .23 20 .66 .23 49 .68 .23 49 .65 .21 24

School effort .68 .26 20 .66 .29 48 .64 .26.7 49 .62 .29 24

School involvement .23 .18 18 .21 .18 45 .23 .15 47 .28 .22 20

School punishment .17 .26 18 .20 .22 45 .25 .28 46 .25 .29 23

School rewards .24 .28 18 .39 .34 45 .25 .27 46 .38 .33 23

Self-reported delinquency- -total .08 .10 18 .07 .11 .46 .07 .10 47 .07 .07 18

Self-reported drug involvementself-reported serious

delinquency

.09

.05

.10

.09

16

18

.09

.05

.14

.11

46

.46

.08

.04

.15

.08

48

47

.10

.02

.14

.04

19

18

Victimization .09 .14 19 .16 .21 45 .14 .18 45 .19 .23 22

School recordsDisciplinary infractions a a a a a a a a a a a a

Grade point average 1.76 .94 42 1.49 1.01 193 1.72 .95 89 1.73 .82 46

Total absence 8.52 16.24 40 16.29 23.15 93 9.01 14.16 88 6.48 10.35 46

Police records I

Police contacts

a Data not reported

.02 .15' 45 .11 .49 100 .02 .14 95 .06 .25 47

.`4, 9 2

Interim Quantitative Evaluation of the Academy for Community Education

D. E. Rickert, Jr.

The Academy for Community Educa-tion is a small alternative schoolsponsored by the Office for JuvenileJustice and Delinquency Preventionas part of that Office's Program inDelinquency Prevention throughAlternative Education. The Academyuses a token economy system, aca-demic education, professional/voca-tional curriculum, and other inter-ventions described more fully byDaniels (1982, 1983) in providingservices to youths at high risk.ofdelinquent behavior drawn from theDade County (Miami) Public Schools.The present brief report summarizessome interim quantitative results ofthe evaluation of this project as itoperated during the 1981-82 schoolyear. Qualitative results of theevaluation are described elsewhere(Daniels, 1983).

Method

Plans were made to assess theproject using a non-random butclosely matched control group forwhom both pre- and post-interventiondata were to be available.1 Thisdesign was partially implemented:Pre-and post-intervention data onabsences, suspensions, tardiness,academic credits earned, standard-ized achievement test performance,and police contacts were availablefor analysis.2 The Academy expended

1. In its initial grant applica-tion, the Academy was to operate asa true experimental field trial,with eligible youths randomlyassigned to treatment and controlgroup conditions. For a number ofreasons described by Daniels (1982),this expectation was not realized.

2. No pre-treatment questionnaire

considerable effort in creating a

matched control group. Essentiallythe same screening procedures usedto select students for participationin the Academy were used (at a latertime) to screen students for thecontrol group (Daniels, 1983).

Analyses proceeded in severalsteps. First, treatment and controlgroup.students were compared on theavailable pre-intervention data.And, because many students partici-

tipated in the Academy for only ashort period of time (68.8%, or 75of 109, had at least six months'involvement in the Academy), compar-isons between long-term (six monthsor more) and control group studentswere also made on pre-interventionmeasures. Second, comparisons ofAcademy persisters and dropouts weremade to determine whether differen-tial attrition might bias theresults. (Only 2.2% of studentsever enrolled in the project droppedout during the year, a relativelysmall percentage in projects of thiskind.) Third, comparisons of treat-ment and control groups were madewith and without the application ofcovariates to increase the statisti-cal power of the analyses.3

-407-

measures were available for the con-trol group, and post-interventionquestionnaire measures were availa-ble for only 37% (21 of 57) of thecontrol group students. An accountof the difficulties in implementingthe planned design may be found inDaniels (1983). The present reportis limited to the most meaningfulcomparisons: those involving treat-ment and contr-1 group students onarchival information.3. A covariate is a variable knownto be correlated with the outcome

ACE

Results

No statistially significant dif-ferences were found between treat-ment and control groups on any of

the pre-intervention measures avail-able for analysis. Nor were anystatistically significant differ-ences found between long-term Aca-demy participants and the controlgroup on any of these measures.These results, summarized in Table1, imply that the matching procedureimplemented by the Academy resultedin a good match between treatmentand control groups. Furthermore,such differences among the groups asare observed are relatively small incomparison to the individual differ-ences within 'groups (compare thedifferences between means in Table 1with the standard deviations whichare indicators of the extent ofindividual differences).

variable of interest which is some-times introduced into an analysis toadjust for pre-existing differencesbetween groups when the groups arenot randomly equivalent. In thepresent case, as we shall see, thetreatment and matched control groupsdid not differ greatly on pre-inter-vention measures. Here the primarypurpose in the use of covariates wasto increase statistical power: Theprobability that a "true" differencebetween the groups will be detectedby statistical analyses. The logicof the analysis of covariance, asthis technique is called, is thatsome of the differences among indi-viduals in both the treatment andcontrol groups can be accounted forby pre-existing individual differ-ences. By taking these individualdifferences into account, analysescan sometimes focus in more closelyon group outcomes due to the treat-

A comparison between studentspersisting in the Academy for atleast six months and students whodropped out with fewer than sixmonths' participation is shown inTable 2. The table shows that per-sistence in the Academy is essen-tially unrelated to any of the pre-intervention measures examined.

Post-intervention comparisons,between treatment and control groupstudents are summarized in Table 3.Treatment students were suspendedsignificantly fewer times than con-trol group students, withdrew invo-luntarily significantly more times,

rand earned significantly more aca-demic credits. Although not quitesignificant by conventional stan-dards, treatment students were alsotardy less often than were controlstudents. No significant differ-ences were found for absences, num- .

ber of police contacts, or standard-ized achievement test scores. Table3 also shows that the same generalpattern of results is present whenonly the long-term Academy partici-pants are included in the analyses,except that (of course) the exclu-sion of students who withdrew invo-luntarily from the Academy from theanalyses alters the "result" forthis form of withdrawal.

-408-

A statistically more powerfulform of analysis that takes pre-ex-isting individual differences intoaccount is summarized in Table 4.In these analyses the results signi-ficantly favor the treatment groupfor number of absences, number ofsuspensions, number of times tardy,credits earned, and the Math Compu-tation and Reading sections of theStanford Achievement test. The dif-ference in number of police con-tacts, although favoring the treat-ment group (especially for long-term

ment.

persisters), is not significant.Even in these very powerful analy-ses, no significant differences werefound for standardized test perfor-mance in the areas of math conceptsor language.

Discussion

The present results are positiveand impressive, but before elaborat-ing on their implications, somelimitations must be described. Themost important limitation is that,with the exception of police recordsand standardized test scores, thesearchival measures 'may involve someunknown degree of instrumentationbias. That is, it is possible thatrecords of absences, suspensions,tardiness, and credits earned may bekept in somewhat different ways bythe public schools on the one handand the Academy on the other, orthat the public schools and the Aca-demy may differ somewhat in the waysthese outcomes are defined. Thislimitation is, of course, largelyunavoidable in a field trial such asthis one. A second important limi-tation is that these analyses bearonly on a part of the outcomes theevaluation seeks to examine. Theunavailablity of questionnaire meas-ures is a serious shortcoming.

The standardized achievement testdata are especially importantbecause, lacking questionnaire datacollected in parallel ways for bothtreatment and control groups theonly other data that were collectedin perfectly parallel ways for bothgroups (police contacts) did notshow significant differences. Asignificant difference favoring the

ACE

treatment group was found for two ofthe four test scores examined.

Despite these limitations, thefollowing interpretations appearreasonable.

1. Participation in the Academyresults in less absenteeism thanparticipation in the public schools.The difference, although not large,is significant.

-409-

2. Participation in the Academyresults in fewer suspensions thanwould occur in the public schools.The difference is of moderate size.

3. Participation in the Academyresults in less tardiness than wouldoccur if the students remained inthe public schools. The differenceis of moderate size.

4. Participation in_the Academy_

results in earning more academiccredit than would be expected if

students remained in the publicschools. The difference amounts toabcat 1.77 credits for students everparticipating in the Academy, andamounts to about 2.74 credits forstudents who persist in the Academyfor at least six months.

5. During the 1981-82 schoolyear, however, Academy participantswithdrew involuntarily more oftenthan students remaining in the pub-lic schools.

6. Participation in the Academyresults in higher scores on the MathComputation and Reading subtests ofthe Stanford Achievement Test.

ACE

References

Daniels, D. Academy for Community Education: First interim report. In G. D.Gottfredson (ed.), The School Action Effectiveness Study: First interimreport. Baltimore: The Johns Hopkins University, Center for Social Organ-ization of Schools, 1982.

Daniels, D. Academy for Community Education: Second Interim QualitativeReport. San Rafael, Calif.: Social Action Research Center, 1983.

/

Table 1

Pre - Treatment Comparisons of ACE and Control Students1980-81 School Year

VariableTotal ACE

SD N

Number of days absent 20.7 19.8 108

Number of times suspended 0.7 1.2 104

Involuntary withdrawal 0.0 0.0 103

Number of days tardy 7.2 7.4 82

Credits earned 7.4 3.4 109

Number of police 0.1 0.4 109contacts

Stanford Achievement Test, national percentile

Math computation 37.0 22.5 105

Math concepts 37.7 22.9 99Language 33.2 25.0 106Reading 34.9 23.3 83

Long-term ACE ControlN SD N N SD

21.0 19.7 75 19.9 17.7 5E

0.7 1.2 75 0.5 1.4 5E

0.0 0.0 75 0.0 0.0 5f

6.7+ 6.6 59 6.4 6.1 44

7.4 3.4 75 8.5 3.4 5E

0.0 0.1 '468 0.1 0.2 57

36.9 23.3 72 43.0 24.9 5]

38.1 23.4 70 40.5 28.8 5(

30.1 23.7 73 33.5 23.4 5]

33.0 22.7 64 35.8 26.4 4]

Note. N's differ among outcomes due to missing data. N's are as follows' disregardingmissing data: ACE total, 109; ACE long-term, 75; control, 57. Long-term ACE participarhad at least six months of treatment.

+This mean is almost significantly different from the control group mean: Q < .06.

% 9

Table 2

Pre-Treatment Comparisons of ACE Persisters andACE Dropouts

VariableACE Persisters ACE DropoutsM SD N M SD

Number of days absent 20.7 18.9 80 21.3 23.2 26

Number of times suspended 0.8 1.2 79 0.5 1.0 23

Involuntary withdrawal 0.0 0.0 75 0.0 0.0 26

Number of days tardy 7.0 7.0 60 8.2 9.0 20

Credits earned 7.6 3.3 80 6.6 3.5 27

Number of policecontacts

0.0 0.2 74 0.3 0.8 22

Stanford Achievement Test, national percentile

Math computation 38.1 21.7 76 33.3 24.8 27Math concepts 35.9 21.9 72 44.2 25.0 25Language 31.9 23.5 77 36.0 30.0 27Reading 34.9 23.6 70 34.5 22.6 13

Note. N's differ among outcomes due to missing data. N's are as fol-lows disregarding missing data: ACE persisters, 80; dropouts, 27.Long-term ACE participants had at least six months of treatment.

Oetiwtokele r 2011 end (esteil iliedmeitsUNA! Odes. War

DallettiSMI*0111141116100

allalaWagaMilltlfaNtiltrnetitoisa.auarsairtariamisahpnrroprpx*$0 o II SO

11100.11111.111.111411.041.1114MOMINIF

00 ilow* moos 1,40 10.2 *L4 ISA 13 21.11 21.$ 3

411000 410 0400, iosopisAliwo 10.410 0,f 100 OA*. 0.3 13 0.4 1.4 3

"441100144 06440 0.O 101 0,0 0.2 73 0.0 0.0 3

44,419 'jai LA 911 1,4. OA Hi 11.6 12.0 3

40104,441 SA* SA WI 10.0** 1.0 H 7.2 LI 3

"Wm tot 0J 0.1 10 0.0 04 6$ 0.1 O./*ammo

flowhoof fliotwoolo optimpos444

ft* ompsoaro. 4010 42.6 214 50 9.3 24.0 4$

gooshoo 40*.1 11.$ 144 41.3 211.1 *I 41.2 26.2 44

WINO 04 104 334 23.4 20 9.9 U.1 41

itoo*** IIVAA VOA MI WO NA 14 11.0 U.) 41

vairsmatoryorgiemv rnt,,trilpprinprorroppeorrximprrisrersortwunretaverisaersiagribeseuraiveimmoamerris.rn,...-a............

sio .44441_4041. Ora u* se follows dieropriltas140111114.09,14 3 cosuol. 91. lom,-tre i putitetpe

4, 0494$44144 tom 44. 44414-4.4 sm. wow s c .06.

oroop woe at tise $ Taut.

poop ems eit ilea .01 WM.

299

Table 4

Adjusted Post-Treatment Means for ACE and Control Students with Pre-TreatmentData used as Covariates: 1981-82 School Year

110011.111.

VariableSIIMMIITotal ACE Long-term ACE

Treatment Control Treatment Contr6

Member of days absent 16.82* 108 22.06 52 18.37 75 22.06

Member of times suspended .16** 103 .65 55 .05** 75 .64

Member of days tardy 7.86** 75 13.33 31 7.91** 55 13.32

Credits earned 9.25** 102 7.48 56 10.18** 73 7.44

number of police .17 98 .19 56 .03 68 .18

contacts

Stanford Achievement Test, national percentile,-.

Math computation 42.07** 96 35.46 44 46.08** 56 34.47

Math concepts 40.42 101 40.46 44 44.00 64 40.24

Language 35.35 111 38.67 44 36.59 69 37.93Reading 38.90* 83 32.60 41 38.90* 64 31.70

MONIIIMON11.

Nag. N's differ among outcomes due to missing data. N's are as follows disregardinimissing data: ACE total, 109; ACE long-term, 75; control, 57. Long-term ACE partici-pants bad at least six months of treatment. Analysis of covariance was performed usilthe 1980-81 variable corresponding to each 1981-82 outcome as a covariate. No expul-sions occurred for any control or treatment group student for whom pre-treatment datamere available.

*This mean differs from the control group meanlat the < .05 level.

**This mean differs from the control group mean at the 2 < .01 level.

300

Alternative Education for Rural Indian Youth: Lac Courte Oreilles

M. Cook

Abstract

The Alternative Education for Rural Indian Youth project (here-after referred to as the LCO project) was one of several projectsfunded by the Office of Juvenile Justice and Delinquency Prevention(OJJDP) as part of its Program in Delinquency Prevention throughAlternative Education. The LCO project operated on the Lac CourteOreilles Indian reservation, located in a remote part of.north-cen-tral Wisconsin. The project maintained two drop-in Youth Centers,installed PLATO computerized instruction. in the Youth Centers,developed a Youth Outreach effort, and partially supported PLATOinstruction in the LCO reservation schools.

The project sought to decrease delinquency on the reservation byproviding recreation services to youth, and by decreasing drop-outfrom the LCO schools through the support of PLATO instruction. Avariety of implementation difficulties including loss of funding forother reservation projects, staffing problems, unclear specificationof goals and objectives, and a scattershot approach to programdevelopment resulted in the program not being fully implementednearly two years after initial funding. The outcome evaluation isinconclusive due to a lack of data, but logical and theoreticalanalysis suggests that the project failed to implement a programthat might reduce delinquency.

Theory

The LCO grant proposal was a mas-sive 500+ page work. Despite itslength this volume contains no cohe-rent statement of what the projectintended to do, or why it intendedto do it. An intent to implement.PLATO, a proprietary computer-basedinstruction system, and to evaluateits effectiveness, was stated in theproposal; the objective of imple-menting PLATO is puzzling, however,because the LCO school had had PLATOinstruction available for threeyears prior to the submission of thegrant proposal.

This report covers the period ofSeptember 1980 to June 1982.

-415-

Letters of support for the grantapplication advocate for funds tomaintain drop-in youth recreationcenters (Youth Centers) to controldelinquency. Although the propo-sal's goals and objectives and theproposed workplan do not mention therecreation centers, the letters ofsupport suggest that the Youth Cen-ters were to be included as part ofthe,,program. (More importantly, theYouth Centers evolved into the pri-mary project intervention).

Vagueness about what the programwas about, and what it should doextended throughout the first project year. Week-long conferences inAugust 1980 and January 1981 wereinsufficient for evaluation staff todetermine precisely what the projectintended to implement. Over time,

301

LCO

project goals and objectives weredeveloped, but their theoreticalrationale remains sketchy. The fol-lowing account is constructed fromProgram Development Plans developedover the project's history, and fromimplications of the statements ofproject personnel. Bear in mindthat this account is constructedpost hoc, and was probably not usedto guide day to day project decisionmaking.

According to project staff,delinquency on the part of Indianyouth is due to 'having "insufficientopportunities for productive, rele-vant, and enjoyable activities" (LCOprogram development plan, datedMarch 16, 1982). There are "fewopportunities for adolescents to

engage in productive activity duringweekend and evening hours" (samesource).

Although alcoholism and drugabuse were frequently mentioned as

causes of delinquency by project andLCO personnel, the project onlyobliquely attempted to address theseproblems (see the discussion of theGroup Home, below).

Goals

The goals of the project were toreduce delinquency among reservationyouth, and reduce drop-outs from theLCO schools.

Objectives

Objectives of the LCO projectincluded reducing the suspensionrate among LCO high school students,reducing over the life of the proj-ect the number of students referredto the project from the reservationschool, decreasing in-school absen-teeism among project-referred youth,increasing the number of youth par-ticipating in Youth Center activi-ties, and increasing the self-esteemof tribal youth.

-416-

The foundation for strategies tomeet these objectives is the beliefthat youth on the reservation get introuble when they have nothing to dobut "hang out." Delinquency shouldbe reduced if the youth have oppor-tunities available for constructive(or least not destructive) activi-ties: school or supervised recrea-tion. Youth in school or at a YouthCenter should be protected fromdelinquency.

The LCO schools (secondary andelementary) were founded in 1976 as

an alternative to the Sawyer Countypublic schools which, it was felt,did not provide an education in

'keeping with the Ojibwa Indian cul-ture.

Early on, PLATO instruction wasemphasized in an attempt to provideindividualized instruction in a me-dium (visual and motor) that wasbelieved to mesh with Indian youths'behavioral and cultural style. Theproject proposed to expand the useof the PLATO system, and evaluateits effectiveness. PLATO's theoret-ical import for delinquency preven-tion is that increased PLATO availa-bility will result in decreasedabseenteism and drop-out from theschool, which would produce lessdelinquency. (The appropriatenessof computerized instruction is

called into question because(a) prior to the start of the proj-ect the LCO high school already hadeight terminals which were not in

constant use, and (b) the principalof the LCO high school stated thatabsenteeism and drop out from theschool were not really a problem.)

The project contributed to PLATOinstruction on the the reservationby placing four PLATO terminals inthe Youth Centers, and temporarilyputting four terminals in the LCOschools to supplement the eightthey already possessed. The four

302

project terminals in the schoolswere ultimately withdrawn and placedin the Youth Centers. Because PLATOinstruction was already a signifi-cant part of the LCO schools' cur-riculum, and the PLATO terminals inthe Youth Centers were almost exclu-sively used for computer games (seebelow), the importance of PLATOinstruction as a part of the projectis unclear.

The actual primary project inter-vention was the staffing and mainte-nance of two recreational Youth Cen-ters. The objective here was toprovide a place where youth on thereservation could engage in healthyrecreation. The explicit rationalewas that the reservation was iso-lated and that youth on the reserva-tion needed structured recreation toprotect them from delinquency (seeCarlton, 1982, for a more detailedanalysis of the social and geograph-ical environment of the LCO reserva-tion).

Is This Sensible?

Delinquency is a tough problem.It has not proven amenable to quickfixes or to ad-hoc programming.Evaluations of delinquency preven-tion projects have consistentlynoted that projects have not beenpowerful enough to solve the problemthat they were intended to address(Wright and Dixon, 1977, Lundman &Scarpitti, 1978). The program asdesigned and implemented by LCO doesnot appear to us to be a credibleapproach to the prevention of delin-quency. The primary theoreticalprinciple behind the project is themaxim, "Idle hands are the devil'sworkshop." Acting on such a theo-retical premise, one intervenes byproviding work or play for the idlehands: Youth Centers and computerterminals. Yet, in an exhaustivereview of delinquency preventionefforts, Dixon and Wright (1974)

-417-

LCO

conclude that "...there is noevidence that these (recreational)programs in any way alter delin-quency" (p. 37). Similarly, the,

background paper for the DelinquencyPrevention Through Alternative Edu-cation initiative (OJJDP, 1980) doesnot include recreational programmingas among possibly effectiveapproaches to the prevention ofjuvenile delinquency. More to thepoint perhaps, there is scant reasonwhy recreational programs ought toaffect delinquency. -Delinquency,and crime in general, takes littletime. Vandalism is usually animpulsive act taking only seconds.Shoplifting requires at most severalminutes. Unless one wishes recrea-tion personnel to supervise youthevery minute of every day (and call-ing such a program recreationalwould be strange), there is no rea-son to believe that youth will nothave time_available_to commit crimeif they so wish--no matter how muchbasketball or pool they might play.

Obstacles to Program Development

The LCO project suffered from asuccession of difficulties. Fromfunding to staffing to politics, theproject confronted a continuousseries of obstacles in the path ofprogram development. The followinglist of events, although not exhaus-tive of the trials that the LCOproject went through, illustratesthe project's struggles.

Project Unfocused

The project's fundamental imple-mentation problem was that its goalsand objectives, and the strategiesto achieve them, were diffuse. Asnoted above, the original grant pro-posal did not clearly outline a pro-gram. Neither the original projectdirector, nor the director thatshortly took over were involved inthe proposal writing or project

303

LCO

design. They were not sure whatthey were supposed to be doing.

After meeting with project per-sonnel in August 1980, and January1981, it was apparent to the evalua-tion staff that the project had noclearly specified aims. The projecthoped to help youth on the reserva-tion stay out of trouble--anythingthat seemed to address that goal wasfair game for project attention.However, after these conferences, wehad no clear picture of the proj-ect's implemented or intended inter-ventions.

Because the grant applicationstressed the importance of PLATO,and contained the stated objectiveof evaluating its usefulness withIndian children, a decision was madeto evaluate PLATO instruction in thehigh school. Although PLATO hadbeen an integral part of the LCOcurriculum for some time before theproject began, it was decided toadminister an abbreviated form ofthe SAES national questionnaire toall students at the LCO high schoolonce at the beginning of the schoolyear and again at the end of eachacademic quarter. This plan wouldhave allowed the project managers tomonitor the usage of the PLATO ter-minals over time, and would havemade possible a relatively weak qua-si-experiment of the effect ofPLATO. This quasi-experimentaldesign was not realized because ofpoor response on the early pretest.Early evaluation plans also calledfor monitoring the use of the youthcenters in order to learn about whatkinds of youths used the centers atwhich times.

At an August, 1982 workshop,project staff, along with evaluationand technical assistance staff,designed a new intervention: YouthOutreach (see below for a descrip-tion). An outreach effort seemed to

encompass the myriad of activitiesthat the project was involved in.

Youth Outreach became a major focusof program development efforts.Unfortunately, going from paper toprogram takes time--interventionsmust be specified, agreements mustbe made, and staff must be hired andtrained. For the LCO project, it

was a matter of too little too late.As of April, 1982, when refundingdecisions for the third project yearwere being made, Lca was still hir-ing new project staff-'-more than ayear and a half after the initialgrant award.

The unfocused nature of the ori-t.e

ginal project proposal was exacer-bated by the governing tribal coun-cil's view of the project. They mayhave seen it as another aspect of_general reservation activities andperceived little uniqueness to thedelinquency prevention project as aproject. Thus funds were spent on arange of things such as repairingthe school van, providing transpor-tation for school and Group Homeevents, staffing the Youth Centers,hiring alcoholism counselors, hiringoutreach workers, taking youth onfield and activity trips, rentingPLATO connect time, buying PLATOterminals, and developing PLATOOjibwa language software. While asubset of these activities mighthave been reasonable programefforts, the shotgun approachassured that no single interventionreceived enough attention to have awell defined impact.

-418-

Loss of Other Federal Funding

A serious obstacle to the projectwas the large cuts that the reserva-tion was receiving in other federalfunding concurrent with the start ofthe delinquency prevention project.Instead of being able to focus onadditional services to youth on thereservation, the project fell back

304

to maintaining some of the servicesno longer funded under other grants.

As the project began CETA fundsran out. CETA workers had beenstaffing the two reservation YouthCenters. Concurrent with thesecuts, the Youth Employment Trainingprogram, and the Youth CommunityConservation Improvement programwere discontinued. These federalprojects had also provided somefunding to maintain the Youth Cen-ters and their activities. At thestate level, the Wisconsin Gover-nor's Council on Juvenile Delin-quency Prevention; which, had pio-vided the funding to build the YouthCenters in the first place, alsowithdrew support funding. As aresult of these funding cuts, theYouth Centers were closed on eve-nings and weekends--the times thatthe project felt were most crucialfor delinquency prevention. There-fore, the project, instead of

extending the services of the YouthCenters as they had hoped, fell backto simply keeping them open.

Toward the end of the project,several other youth and educationprograms were being discontinued.As cuts in federal programs contin-ued throughout the life of the proj-ect, project staff became demoral-ized. They found themselvessubjected to requests to fund avariety of youth related events,whether they were linked to projectobjectives or not.

The situation deteriorated towardthe end of the project. Along withthe loss of the delinquency preven-tion project, the reservation lostfunding for a fine arts project inthe school, a youth athletic intra-murals project, and an adult educa-tion project.

-419-

LCO

Cuts in educational and socialprograms appear to have elicited abacklash on the part of the LCOreservation population toward whitesthat worked on the reservation. Atribal election in the winter of

1982 changed the composition of thetribal council. The new members hadcampaigned on an "LCO jobs for LCOIndians" platform. In April of1982, 11 of the 17 white teachers inthe LCO school system were fired,(with no replacements identified forthe following year) along with theprincipals of the elementary andhigh schools, the school superinten-dent, and the director of education(who was a Native American, but wasperceived as having let whitesbecome too involved in the educationof reservation youth). The alterna-tive education project director, whowas white, was convinced that theonly reason he was not fired wasthat the tribal _council knew thathis dismissal would be the laststraw as far as possible third yearfunding for the project was con-cerned. The project directorbelieved that if the project hadbeen refunded for the third year, hewould have been replaced shortlyafter the grant was signed.

Although the project director didnot give up on the project duringthese events, it is fair to say thathe was less invested in projectefforts past April 1982. The poli-tical shake-up particularly hurt theevaluation of the project, since thereservation firings were concurrentwith attempts to survey users of theYouth Centers, and survey Indianyouth who attended the local publicschools. Lacking the concertedattention of the project director,neither of these surveys met withmuch success (see below).

305

LCO

Succession of Project Directors

As late as December 1980, fourmonths into the project, noaccounting procedures had beeninstituted and the paper work forthe release of funds had not beencompleted. The project could notpay its bills, and was threatenedwith being dropped from the initia-tive. The project director's dutieswere assumed by the evaluation coor-dinator in December, 1980, after hehad been associated with the projectfor only one month. The initialproject director resigned in Januaryof 1981, and the evaluation coordi-nator became the acting projectdirector. The change in status ofthe evaluation coordinator producedthree management difficulties:(a) there was now no one with theprimary duty of overseeing the eval-uation, (b) having just been hired,the new director knew even lessabout the project's goals than theoriginal director, and (c) the newdirector was white, which may havelimited his ability to negotiatewith the tribal governing board.

Project Not Viewed as Distinct FromRegular School

Largely due to the unfocusednature of the project, there was noclear distinction made between thealternative education project andthe LCO schools. It may be that theoriginal intention of the grantwriters was that the LCO schoolsthemselves were the "alternative"education part of the project. Thisis certainly plausible given thatthe LCO schools were indeed estab-lished as a culturally-orientedalternative to the public schools.Yet, the project as developed andunderstood by evaluation and OJJDPstaff was an entity distinct fromthe school. The confusion (whateverthe source) resulted in the projecthaving to justify expenditures that

did not appear to OJJDP or theevaluation staff to be directlyrelated to the interventions thatthe project was trying to implement.

An example of this situation wasthe development of the Ojibwa lan-guage program for use on the PLATOsystem. The argument advanced forunderwriting the development of thiscomputer software was that learningOjibwa would put the reservationchild more in touch with his or herheritage, result in mbre self-es-teem, .and lead to less delinquency.This causal chain is plausible if aproject is making a full-scale

,effort to improve the children'sself-esteem. But that was not theproject's intention. The point is

that instead of concentrating on onetask or another, the project delvedinto a variety of things whose onlyinterconnection was that they hadsomething to do with Indian youth.

-420-

Not Expecting Rigorous Evaluation

A further organizational problemwas that LCO grant administratorswere familiar with a certain kind offederal evaluation. The LCO tribalorganization was accustomed to fed-eral evaluators making site visits,and judging projects on the basis ofinterviews and audits. The evalua-tion of the alternative educationproject was perplexing. Afterdecades of federally funded proj-ects, the project director commentedthat the evaluation of the alterna-tive educational project was themost rigorous that LCO had ever beensubjected to.

The consequence of the reserva-tion's past history with evaluationwas that evalution tasks were "justmore paperwork." When problemsdeveloped, or research designs brokedown, the response was to assumethat it didn't make any differenceanyway. One result: Evaluations of

306

the Youth Centers and of the LCOhigh school school fell apart due tononattention to data collectiondetails.

Change in Evaluation Staff

A final obstacle to program suc-cess was the change in evaluationstaff that the project had toendure. Project personnel were dis-satisfied with their original fieldworker. They were puzzled when heshowed up for a site visit without aclear set of objectives for thevisit. This event reinforced theproject's belief that evaluationswere generally meaningless. Thisattitude made evaluation effortssubsequent to the replacement of

that field worker in the fall of1981 more difficult. Then, thereplacement field worker resigned inJanuary, 1982. In the two years oftheir project, therefore, they hadthree evaluation field workers.This lack of continuity, coupledwith the project's previous experi-ence with federal evaluations,served to undercut attempts at for-mal, data-based evaluation.

Summary of Obstacles to Implementa-tion

The major obstacles to the imple-mentation of the LCO AlternativeEducation Project were (a) the proj-ect had unfocused goals and objec-tives and expended funds on a grab-bag of interventions, (b) otherfederally funded projects were beingdiscontinued, and the delinquencyprevention project picked up some oftheir expenses, (c) project adminis-tration was allowed to slip in theearly months of the project, (d) theproject was not viewed as an entitydistinct from the LCO schools,(e) project personnel did not expecta rigorous evaluation, and had noexperience in conducting one, and(f) there was a lack of continuity

-421-

LCO

in the evaluation staff assigned tothe project.

The result of all these obstacleswas that at the close of the LCOproject it was still in an earlystage of development. Staff wasstill being hired in March 1982, andall project components were notimplemented until May of that year.By this time the decision had beenmade not to renew the project'sgrant award.

Interventions

Youth Center

The fundamental intervention ofthe project was the staffing andmaintenance of the reservation YouthCenters. There are two Youth Cen-ters on the LCO reservation, locatedabout 15 miles apart in the two mostpopulous communities on the reserva-tion. The project initially hopedto expand the services offered atthe Youth Centers, but followingcuts in other sources of funds to

the Centers, had to be content withinstalling PLATO and paying thesalaries of the Youth Center staff.

The Youth Centers are drop infacilities that have grills, pooltables, video games, and meetingrooms. Space can be cleared to holddances or other large gatherings..The project hired persons to openand close the center, maintain thephysical plant, and serve as cooks(see descriptions of the SupportSpecialists and Youth Outreach Work-ers, below). Project personnel alsosupervised the use of PLATO at theYouth Centers.

The first direct project inter-vention at the Youth Centers (otherthan keeping the doors open) was theinstallation of on-line PLATO termi-nals. This intervention wasintended to provide another source

307

LCO

of recreation at the Youth Centers(on-line PLATO terminals allow oneto access a variety of computergames), and to make PLATO educa-tional instruction available foryouth not involved in the regularLCO school program.

Installation of the terminals gotoff to a slow start. The PLATO ter-minals had to be connected to a"parent" computer in Minneapolis,and the phone lines necessary tocarry the information are differentfrom the regular long-distancelines. New lines were installed,but this took fbur months. Thelines were connected directly to theReserve Youth Center, and regularlong-distance lines carried the sig-nal to the New Post Youth Center.

The necessity of sending the sig-nal by long-distance telephone linesfrom New Post to Reserve caused theproject quickly to remove terminalsfrom New Post. Long-distance rateswere too expensive to justify theterminals' use at New Post. There-after, until the end of the project,the New Post Youth center was not aprime focus of project activities.Project funds were used to staff itand keep it open, but PLATO andother project activities were una-vailable there.

Because rental of connect timewith the Minneapolis computer wasexpensive, the project eventuallypurchased self-contained PLATOmicro-computers. Four of these wereinstalled in the Reserve Youth Cen-ter in January of 1982. Althoughthey were less expensive to run, theself-contained units had one majordrawback: little software wasavailable.

The primary use of the PLATOunits in the Youth Center was game-playing. They were seldom used forinstruction. But game packages were

-422-

not available for the self-containedThe only software was

designed for the teaching of basicacademic skills--items of littleinterest to the Youth Center clien-tele. Project personnel estimatedthat use of the terminals dropped80% when the self-contained compu-ters were installed.

Project-sponsored Youth Commit-tees were developed in the summer of1981. These committees were organ-ized by project personhel and wereopen to all youth on the reserva-tion. They operated out of theYouth Centers. Participation in themeetings themselves was light, butthe committees--or the project per-sonnel running them--did design andcarry out a number of recreationalactivities for youth. From January1981 until April 1982, a total of 14events and trips were sponsored bythe project. These included dances,skating parties, field trips, skiingexpeditions, and canoe trips.

Youth Outreach

In August 1981 a new Youth Out-reach intervention was designed thatappeared to encompass a variety ofproject interventions. The LCOproject decided to locate thoseyouth on the reservation who wereunemployed and not enrolled inschool, or who were enrolled inschool but in danger of beingexpelled for discipline or atten-dance reasons. The Youth Outreacheffort was designed to identify suchyouth, offer counseling and tutor-ing, and involve these youth in con-structive activities. Two catego-ries of workers were responsible forthis effort: Support Specialists,and Youth Outreach Workers.

The first Support Specialistbegan work in November of 1980, anda second in April of 1981. Theywere originally responsible for

308

staffing and supervising the youthcenters, and the PLATO terminals inthem, and for providing drug andalcohol abuse counseling to YouthCenter clients. With the advent ofthe Youth Outreach effort, theyacquired the additional responsibil-ities for making home visits to Out-reach youth, and of supervising theYouth Outreach Workers.

It is difficult to say how quali-fied the Support Specialists were tocarry out their duties. Their sta-tus as tribal members was certainlyan asset; it would be difficult foroutsiders to understand the economicand social problems that the reser-vation youth faced. On the otherhand, although drug and alcoholabuse counseling was one of the Sup-port Specialists' primary responsi-bilities (indeed they viewed it astheir most important repsonsibil-ity), neither Specialist had anytraining in counseling. Althoughacademic training neither guaranteesnor prevents one from attaining theskills necessary to carry out effec-tive substance abuse counseling, alack of training in counseling is

not indicative of a strong counsel-ing component.

The Youth Outreach Worker posi-tion was created to establish a

liaison between the project and res-ervation youth. The Youth OutreachWorkers were to be the people in thefield identifying reservation youthin need of service, and organizingand supervising recreational activi-ties. The Youth Outreach Workersalso staffed the Youth Centers, andwere responsible for unspecified"peer counseling."

The Youth Outreach Workers wereall high school graduates or werecurrently attending the LCO highschool. Youth Outreach Workers ranthe Youth Committee meetings, andoversaw their activities. The Youth

LCO

Outreach Workers were alsoresponsible for conducting a censusof Youth Center users, administeringan evaluation survey to Youth Centerclients, and carrying out a fieldsurvey of reservation youth whoattended the Sawyer County publicschools.

By January 1982, three Youth Out-reach Workers had been hired, andfour more were hired in March 1982.

Alternative Education

Prior to the start of the alter-native education project, students

.at the LCO high school who exhibitedchronic absenteeism were expelledfrom the school. The project estab-lished a policy whereby students whoattained seven unexcused absenseswere suspended from the school, andrequired to attend PLATO tutoring atthe Youth Centers. If these stu-dents performed satisfactorily theycould be reinstated in the regularLCO curriculum at the beginning ofthe next academic quarter. Thisreferral and PLATO, tutoring compo-nent was called the "AlternativeEducation Program."

-423-

Other Interventions

Project funds contributed towardthe PLATO instruction offered at theLCO high school and elementaryschool. Project funds paid for partof the rental of the PLATO terminalsin the LCO high schcol. In addi-tion, in January 1982, the projectpurchased eight PLATO self-containedmicrocomputers. Two of these wereplaced in the LCO high school, andtwo in the LCO elementary school.In April of 1982, all of theseself-contained PLATO computers wereinstalled at the Reserve Youth Cen-ter.

The project also supported thedevelopment of an Ojibwa language

309

LCO

program for use on the PLATO system.The package describes in Ojibwa sev-eral traditional Ojibwa events (e.g."The Sugar Camp"), while on thescreen the scene is drawn andlabeled in Ojibwa. Headphones allowthe student to listen to the speech.Students respond by repeating thesentence they have heard (but withno feedback as to correctness), andengage in multiple-choice tests ofmatching written Ojibwa to drawingsof objects named in the previouslesson. A fundamental difficultywith the program is that Ojibwagrammar is substantially differentfrom that of English--the languageof all children on the reservation.For example, in Ojibwa, the verb isalways the last word in the sen-tence. Unfortunately, the programdoes not address grammar at all.Because they are not instructed tothe contrary, children make the rea-sonable assumption that the thefirst word spoken in Ojibwa corres-ponds to to the first, word of theEnglish translations.' It doesn't.Observations indicated that inter-acting with the program degeneratedto a guessing game of randomlyassigning Ojibwa labels to drawnobjects in the hope of matching themcorrectly. Any learning that tookplace appeared to be rote memoriza-tion of chance correct matchings.Project staff admitted that childrenwere a little "confused" by the lan-guage lesson.

A second Ojibwa lesson was sup-posed to be in development at theclose of the project. We do notknow whether it was completed ornot.

Project staff provided some ser-vices to the reservation Group Homeand to the LCO school. The LCOGroup Home is a residential commu-

-424-

nity for adolescents suffering fromdrug or alcohol dependency. Theproject provided transportation forGroup Home residents to Youth Centeractivities, and offered PLATO tutor-ing (although we do not know if

tutoring was utilized). The SupportSpecialists provided some counselingto these youth.

The project also provided trans-portation for LCO school children toproject activities.

A Human Services committee wasestablished. Chaired by the ProjectDirector, the committee was chargedwith the coordination of social ser-vice delivery agencies on the reser-vation. The project director feltthat his time as chair of the com-mittee was a project contribution.

Evaluation Results

At the outset,_

it should be madeclear that we have little data bywhich to evaluate the LCO project.The project was still hiring staffas late as March 1982, and had notyet fully implemented one of itsmost important interventions, YouthOutreach. Evaluation issues took onsecondary status to program imple-mentation. In addition, for reasonsoutlined above, at the close of theproject when summative evaluationdata were being collected, projectpersonnel were not invested in theproject itself, much less in itsevaluation. The project directorwas cooperative with evaluationefforts, but the data provided areinsufficient to make strong conclu-sions. Therefore, the informationthat follows is mostly descriptivein nature. It is offered more toaid other program implementers andevaluators than to evaluate the LCOproject.

310

11.

Youth Center

In March of 1982 the first taskassigned to the Youth Outreach Workers was to undertake a census ofYouth Center users. For four weeks,every user of the two Youth Centershad to sign in. From this signinsheet a roster of Youth Centerclients was developed. The ReserveYouth Center (the foremost site ofproject events) had 125 differentusers, and the New Post Youth Centerhad 81. Because the New Post YouthCenter had little to do with theproject, other than being staffedthrough project funds, it serves asa kind of nonequivalent control forReserve, the project Youth Center.

In consultation with the nationalevaluation staff,, a survey wasdesigned to identify the personalcharacteristics of the Youth Centerusers, and to assess their satisfaction with the Youth Centers. Thetwo Youth Centers were based about15 miles apart. Since transportation around the reservation is asubstantial problem for reservationyouth, it might be assumed thatattendance at one Youth Center orthe other would depend mainly ongeography. Differences in satisfaction with or use of the Youth Centers might be attributable to project effects.

Table 1 gives some descriptiveinformation on the clients of thetwo Youth Centers. The N's (thenumber of people responding to thesurvey) tells the most importantstory. Of the 125 identifiedclients of the Reserve Youth Center,78 were surveyed, for a responserate of 63%. Only 17 of 81 New Postyouth completed their surveys, for aresponse rate of 21%. Not only isan N of 17 too small for results tobe dependable, but a 21% responserate suggests that selection biasesmay exist. Nevertheless, Table 1

does not suggest any major

LCO

differences between the youth usingthe two Youth Centers, other thanthat the New Post Youth Center mightbe serving slightly older youth.With the poor response rate, it isimpossible to say with any certainty.

Table 2 suggests that youth visitthe Reserve Youth Center somewhatmore often than the New Post YouthCenter (but the difference is notsignificant). Satisfaction with theyouth centers does not differ much.

Table 3 gives the stated reasonsfor visiting the youth centers. Of

,particular interest here is the lowpercentage of youth who go the YouthCenters to use PLATO (by the time ofthe survey in the summer of 1982PLATO had been installed at NewPost). Youth mostly attend theYouth Centers to do what one mightexpect: play games, visit friends,and get something to eat. Theresults do not support the notionthat making computerized basicskills instruction available atrecreational sites will affect manyyouth.

425

Also interesting in Table 3 is

that almost no youths reportinvolvement in Youth Council Meetings.

One possibly positive effect of

the project's attention to theReserve Youth Center can be found inTable 4. Table 4 gives the place ofresidence of each survey respondent.Whereas all (N = 17) of the New PostYouth Center repondents were fromNew Post, Reserve served youth fromall 10 of the major reservation communities. It may be that the project and its Outreach efforts succeeded in involving more youth inYouth Center activities. Alternatively, the project may have chosento operate primarily in the morepopular center.

311

LCO

Finally, Table 5 gives descrip-tive information about the employ-ment and schooling status of theYouth Center users. No differencesare significant, although somewhatmore of the youth at New Post may beattending the LCO high school.

We can make no inferences con-cerning the delinquency preventionefficacy of the youth centers.Project personnel never providedcourt contact data on the Youth Cen-ter users as agreed.

What can we conclude about theYouth Centers? Reserve, the primaryproject site, seems to serve youthfrom a wider geographical area thanthe New Post Youth Center, whichreceived less attention from theproject. This may or may not be aproject effect. According to theproject director, the Reserve YouthCenter was more active than New Postprior to the start of the project.That was why Reserve was chosen asthe focus of the project.

To be fair, more important thanany differences between the youthcenters is their fundamental simi-larity: they were open for busi-ness. They would have been closedor opened on a much curtailed sched-ule without grant funding. Thus,their availability to youth was aproject success. For theoreticalreasons, however, their usefulnessin preventing delinquency is ques-tionable.

Youth Outreach

We have no direct way of evaluat-ing the effectiveness of the YouthOutreach component, including theYouth Outreach Workers, and the Sup-port Specialists. Yet, three com-ments are in order:

-426-

1. As just discussed, the

greater geographical distribution ofthe Reserve Center's users may bedue to Outreach efforts, since Sup-port Specialists and those YouthOutreach Workers that had been hiredwere based there. The evidence forthis in only circumstantial. ,

2. The Youth Center surveyadministration was the single mostimportant task of the Youth OutreachWorkers, and it .was performedpoorly. This suggests that highschool students require carefulsupervision to carry out such afield study. Similar supervisionwould probably be required for theother duties, including peer coun-seling and family visits, with whichthey were charged.

3. Support Specialists mayrequire more preparation in drugdependency counseling. Because theywere charged with supervising theYouth Outreach Workers in the YouthCenter survey administration, whichdid not go well, they may haverequired more preparation in projectmanagement or supervision as well.

Alternative Education

We were not provided with infor-mation needed to identify Youthreferred to "Alternative Education"during the 1981-82 school year.Both the principal of the highschool and the project directorindicated that the program did notwork: Youth suspended from highschool for nonattendance did notcome to the Youth Center for basicskills tutoring, or counseling indrug dependency.

For descriptive purposes, Table 6gives characteristics of the refer-rals to Alternative Education duringthe spring of 1981.

312

OfiltaAoglooksmo

MOM **ittht401144 xea forittI

to "mown Ise weo*Itoolii t s4.ce cles saw to pros

4400 4011,14x10101- 0444041r twti*itio, to140 oloSPote. ea Amp" to* sod* to046014,* i 04000 ees to *bow fever.

4ms *vote oto solfaiod the ere*OPOO*0 whoa .) tipot * ifeeipktiledoe

10400411 etvellsosi 00044s iatorsettea iratho p0000stoostose WON COW tfAeseelmksfail4 f3/44,46.444441104 talissa.4 *.I

tte 4 esevey se* etesialletfefdot4e001 ko eta* deltiodiag the

Oh 440 40hool* 4 #104444 et "poor=eiEl*Pe omit 4040064001 the Mowtomo4o 04434 *shoot* owe 64101001.*00 toottoeilb 001/4011 ertoeo aosigeedthe look et hativtastily feamister.bmg *Of *Aft ave*tioemales f theseefoOpoes oe pet* et thole thatoethWool.

$ oho la otbutol PO et $ sitsoaf.* attia4044 Use 016,0*, tee am04,0**4* foots** co* of 4)16 Onlyfa it it Ube pahlte *fleetmod; speo eerveye d. do ome thefees tee Ohs tooth Goatee foreey.

4000444011601*. tee OdibStateepito toe to* #04410110 tot sestideetimeolvretoefies it the dots.

toble t goofs f*Ileito etamsleito at Notloose soopa tos the00644a *ahead *toilet* to the U-*ago.* oo atoo 44411 sesaass46 TheTao re, mad the Attereettal re-*pew* Moe ease ilt44 ittiele Melte4044 he Sneffteelhi1 eStib gnatostestoso.

*woe the Woods Se botorpre-AMNON the psu et molt* mg-emote het the atedeets ettostimgOto pohlte wheel ere lee* embelli-eme mewl miehee MS 1011 delememeat60bosbotwObeekSebOt ptters* 'etasaoseetaiod ottb lop aft aabemee I.

LCO

Wool and esperteaces ofeaspiesika. het pohlie school sta-

te are mewled gag often and'waive twee muds is school thando psetbe *Medias the LCO schools.Thew **telt* fie, hear directlyes the 'some it whether the LCOwheel' teems* or reduce the riskef **wow behavior, Differencesebeerved say be setirely due toeelf..seleettee,

114 wereosaario preyed, the LCO high*sheet to raedomly assiga studentsto PLATO am Ammo strong claimslove hoes mode ter the of of

#PLATO as is isetrectiesal supdtuu, weeovertheless decided to. (lupine11411) see es best we could.' Tableproefs's staple eerrelatioos betweenhours et PLATO use is l'112 aod mess-uroe of academic achievement,attsSatest to school, psychologicalhealth, mad delisqueucy*

The only stgatlisont correlatesof PLATO use are sbossoss and disci-eliaory referrals. Tout% who bltipschool sore, and who get in troublemare* teed to use PLATO were. Thisof seerse is purely correlationdata* sod prohably indicates onlyinitial differences is *sere ofPLATO. More ie so Wailes to aca-demie achievement as measured byeither grease it self-reported*shod effortit is met ealy the

I* 'd'e were usable to do multipleregression amalysis oe PLATO usecontrolling for pretreatment differ-oases between individuals because oflaw salbers et 'taints surveyed insprigs ITU. Pretests administeredin the fell et OS: were also Londe-lust* became. (a) they were notadmisstered Datil November. makingtheir 'pretest" status opestiosaaIds, sad 00 there is a great dealof sissies data.

313

LCO

less able students who are usingPLATO. Neither is there any rela-tionship between PLATO use and anymeasure of psychological adjustment.Wo find no evidence here that PLATOuse increased the self-esteem of lowskilled learners.

Nothing in these results suggestsPLATO instruction had a measurableeffect on anything.

d the, Program Meet Itoals andObjectivts?

Probably not. We cannot assessthe project's impact on delinquency.Self-report delinquency was assessedin the LCO high school, but the re-sponse rate for students in the pub-lic school was extremely low, andbesides, the Youth Centers were theprime project intervention. Sincethe court contact data was neverdelivered, we are blind to theeffects of the Youth Centers ondelinquency. There is no reason tobelieve they had an effect on delin-quency.

We cannot say whether the projectaffected dropout from the LCOschool. Since the intervention inthe school was weak (paying for com-puters that were already there) wedo not know why dropout should havebeen affected. We do know that the"alternative education" interven-tion--referral to the Youth Centerfor tutoring after suspension--wasnot successful. No one came.

Similarly, we cannot evaluate theproject's effects on suspensions andabsenteeism from the LCO school, oron the self-esteem of LCO youth.Some weak evidence suggests thatyouths attending the LCO school aresuspended less than they might bewere they attending the publicschool. The project certainly didprovide recreational opportunitiesfor youth.

Recommendations

1. Proiects should not be fundedthat do not have a clearly specifiedtheoretical base supporting arational, planned set of interven-tions. Especially given the shorttime period (two years) by the endof which projects are required tohave demonstrated progress towardgoals and objectives, programs thatare not able to hit the ground run-ning are faced with an impossibletask.

2. Projects should not beallowed to expend funds on "scatter -

,shot" interventions. The LCO proj-ect spent time, money, and energy ona shopping basket of youth interven-tions. It would have better concen-trated its efforts on a emaller sub-set of interconnected interventions.The Youth Outreach program was anattempt to do but thereremained insufficient time for itsimplementation.

-428-

3. Personnel implementing anintervention should have the requi-site training and experience. Onlywhen counseling is done well is

there much hope of positive effects.Indeed, the evidence suggests thatcounseling can be of no help in pre-venting delinquency (Wright andDixon, 1977), and may have negativeeffects (McCord, 1978; Cook, 1983)that should be carefully guardedagainst. Under the circumstances,state-of-the art counseling inter-ventions implemented by well-trainedprofessionals are perhaps worthy oftest, but the, efforts of amateursare unlikely to be fruitful.

4. The requirement of rigorousevaluation should be made clearPrior to funding, and a clearly spe-cified experimental or strong quasi-experimental evaluation designshould be a requirement for funding.Federally funded crime prevention

314

LCO

projects should come to anticipate trol groups and the collection ofcareful evaluations involving con- adequate outcome data.

ti

-429-

315 ,

LCO

References

Carlton, R. The Lac Courte Oreilles Alternative Education Project: InterimReport. In G. D. Gottfredson (ed.), The School Action Effectiveness Study:First interim report (Report No. 325). Baltimore: The Johns Hopkins Univ-ersity, Center for Social Organization of Schools, 1982.

Cook, M. S. Plymouth Alternative Education Program: Second interim report.Baltimore: The Johns Hopkins University, Center for Social Organization ofSchools, 1983.

Dixon, M. C., & Wright, W. E. Juvenile delinquency prevention.programs: Anevaluation of policy related research on the effectiveness of preventionprograms. Nashville, Tenn.: Office of Educational Services, Box 60, Pea-body College for Teachers, 1974.

Lundman, R. J., & Scarpitti, F. DelinqueTcy prevention: Recommendations forfuture projects. Crime and Delinquency, 1978, 24, 207-220.

McCord, J. A thirty-year follow-up of treatment effects. American Psycholo-gist, 1978, 33, 284-289.

Office of Juvenile Justice and Delinquency Prevention. Program announcement:Prevention of delinquency through alternative education. Washington, D.C.:Author, 1980.

-430-

316

Table 1

Descriptive Statisticson Youth Center Patrons, Spring 1982

Characteristics

Reserve: New Post:Project Youth Center Comparison Youth Center

SD N Ti SD

Age in years 16.17 5.77 76 18.50 5.61 17

Male 63% -- 78 63% -- 17

Holding highschool diploma 31% -- 78 25% -- 17

Table 2

Frequency of Attendance and Satisfactionwith Youth Centers, Spring 1982

Patron's report of

Reserve: New Post:Project Youth Center Comparison Youth Center

M SD N M SD

Weekly visitsto youth center 3.93 1.75 75

Satisfaction withyouth center 2.81a .68 79

3.06 1.71 17

2.94 .43 17

aOn a fourpoint scale ranging from "Very Satisfied" (4) to"Very Dissatisfied" (1).

431317

Table 3

Reasons for Visiting Youth Center, Spring 1982(Column Percent)

Reason

Reserve: Project New Post: ComparisonYouth Center(n=79) Youth Center(n=17)

% Affirmative % Affirmative

Games (Pinball,video, pool)

Food

Social contacts

Use PLATO

Youth Councilmeetings

Other

90

68.

66

8

1

9

.41

94

47*

71

12

* < 10P

-432-

318

Table 4

Residence of Youth Center Patrons, Spring 1982(Column Percent)

Community of Reserve: Project New Post: ComparisonResidence Youth Center (n=78) Youth Center (n=17)

Bacon Square 4 0

Chief Lake 11 0

Dry Town 1 0

KTown 6 0

New Post 4 100

Reserve 48 0

Round Lake 6 0

SignorCouderay 9 0

Six Mile 4 0

White Fish 4 0

Other 4 0

-433- 319

Table 5

School and Work Status of Youth Center PatronsResponding to, "What are you Doing Mostly?", Spring 1982

(Column Percent)

School and Work Reserve: ProjectStatus Youth Center(n=78)

Attending LCO

New Post: ComparisonYouth Center(n=17)

high school 23 41

Attending LCOelementiry school 4 6

1.0

Attending Haywardpublic high school 4 6

Attending Haywardpublic elementaryschool 1

Attendingvocational school 1 0

Attending G.E.D. 1 0

Attending college 1 0

Working full orpart time 24 24

Doing tribalvolunteer work 1 0

Unemployed 15 18

Other 23 6

-434-

320

Table 6

Descriptive Statistics on Referrals to"Alternative Education", Spring 1981

Student Characteristics

Grade point average,

M SD

tig

N

Spring 1981 2.00 0.00 5

Hours PLATO use,Spring 1981 7.05 5.17 11

Absences, Spring 1981 9.63 3.07 8

Disciplinary referrals,Spring 1981 1.09 .70 11

Note: Six of the 11 referrals withdrew dur-ing the Spring of 1981. Three of these sub-sequently re-enrolled the following year,and one more dropped. At the end of 1982, 7of the 11 1981 referrals were still enrolledin school. None had graduated.

-435- 321

Table 7

Comparison of Self-report Measures ofIndian Youth Attending the LCO High School and theSawyer County (Hayward) Public Schools, Spring 1982

LCO Public SchoolQuestionnaire Measures

School variables

M SD N

Attachment to school .73 ..22 77School rewards.. .43 .33 78School punishments .10 .17 78Parental emphasis on education ,46 .30 76School effort .55 .29 77Negative peer influence .29 .19 79Victimization .10 .15 77Involvement .20 .16 77Reading ability 1.92 1.59 79

Psycho-social variables

.42 .24 79Attachment to parentsBelief in rules .61 .28 76Positive self-concept .69 .17 72Rebellious autonomy .86 .23 75Interpersonal competency .77 .25 73Internal control .44 .24 75Alienation .27 .24 76Practical knowledge 1.34 .40 76Invalidity .14 .18 77

Self-reported behavioral variables

1.29 .66 78School non-attendanceDelinquency--total .28 .19 52Serious delinquency .14 .17 52Suspensions .62 1.98 79Drug use .60 .32 51

N SD N

.70 .30 20

.10 .22 22***

.07 .19 22

.30 .21 22***

.51 .28 24

.27 .22 25

.06 .10 21

.15 .14 23

1.85 1.64 26

.55 .26 25***

.71 .28 22

.70 .17 21

.71 .36 22**

.76 .27 22

.42 .28 22

.24 .26 21

1.22 .61 22

.05 .12 22**

1.00 .82 25

.14 .15 25 ***

.07 .14 25*1.85 3.57 26**.28 .31 25**

*p<.10* *p<.05

***p.01

-436-

322

Table 8

Correlations between 1982 Hours of Plato Useand Measures of Academic Achievement, SchoolAttachment, Psychological Adjustment, and

Delinquent Behavior

Measures

Academic achievement

r N

4Spring 1982 GPAa .06 73School effort .09 77Absencesa .29* 89Withdrawals from schoola .16 95

Attachment to school

Attachment to school .09 77School rewards .12 78School punishments .03 78

Psychological adjustment

Positive selfconcept .09 72Alienation .10 76Internal control .07 75Interpersonal competency .18 73

Delinquent Behavior

.28* 951982 Discipline referralsaSelfreported delinquency--total .04 52Selfreported serious delinquency' .12 52Selfreported drug use .05 51

aSchoolbased data; other measures are SAES questionairebased.

* I) < 05

437-323

324-438-

Plymouth Alternative Education Project: Second Interim Report

M. S. Cook

Abstract

The Plymouth Alternative Education Project operated Student Ser-vice Centers, Student Activities Centers, and an out-of-schoolLearning Options program primarily for two high schools and two mid-dle schools. These interventions provide educational services,counseling, and recreation for students with disciplinary and atten-dance difficulties in this predominantly working and middle-classwhite community. Interim evaluation results imply that the highschool Student Activities Center is being implemented with care, andhas some promising positive effects on participants (increased aca-demic performance, decreased alienation, and increased practicalknowledge). Interim evaluation results raise some questions aboutthe Student Service Center implementation, and suggest that someunexpected negative effects of this counseling intervention may beoccurring. The Learning Options program, which appears to be vigo-rously implemented, did not participate in an outcome evaluation.

The Project

The Plymouth Alternative Educa-tion Project is one of several proj-ects funded by the federal Office ofJuvenile Justice and DelinquencyPrevention (OJJDP) as part of itsProgram in Delinquency Preventionthrough Alternative Education. ThePlymouth project operates in twolarge high schools that share thesame physical plant, and in twoseparated middle schools. All arelocated in a mostly white workingand middle-class suburb of Detroit,Michigan. The project seeks to pro-vide services to students exhibitingtruancy, disruptive behavior, or

This report covers roughly theperiod September 1981 through August1982. For an account of the projectduring its first year see Carlton(1982).

-439-

poor academic performance, and whoare considered at risk for drop-outand delinquency. The project is

composed of four discreet treatmentprograms:

1. The high school Student Ser-vices Center is a daily semester-long class for which studentsreceive credit toward graduation.Treatment involves four major inter-ventions: (a) facilitated grouppeer counseling, using the students'day-to-day problems as the source ofgroup discussion, coupled withaffective educational training in

transactional analysis and interper-sonal communication skills,(b) close weekly monitoring of thestudents' academic progress in theirother classes, (c) individual coun-seling outside of the class period,and (d) a high level of counselor-initiated parent contact.

2. The high school Student Activ-ities Center has three distinctinterventions: (a) a competency-based remedial writing and study

Plymouth

skills class for students showingpoor academic performance, (b) asupervised setting for student ini-tiated activities such as clubs anddiscussion groups, and (c) an organ-ized intramural program targeted atstudents who are uninvolved in con-ventional school extra-curricularactivities.

3. The middle school StudentServices Center is similar to thehigh school program of the samename. There are three major differ-ences between the two interventions,all of which are due to the matura-tional differences'of the respectiveclient populations: (a) studentsmee,. weekly on an individual basiswith their counselor for ten weeksprior to the formation of groups;this aids in developing bonding tothe counselor so that the counselorhas more leverage over negative peerinfluence effects that could appearin this age group; (b) the middleschool program is team taught; twocounselors facilitate the group dis-cussions so that they have moredirect control over the group pro-cess; and (c) the program is morestructured, relying less on the stu-dents themselves to bring up prob-lems as a source of group discus-sion.

4. Learning Options (LO) is acompletely self-contained alterna-tive school for highly disruptiveStudents from both the high and mid-dle schools (though most programparticipants in 1981-82 were middleschool students). It is character-ized by a high degree of individual-ized instruction, tutoring, groupand individual counseling, progres-sive attainment of responsibility,student input into rules and disci-plinary practices, and parentalinvolvement in their children'sschooling.

Besides the above programs, proj-

-440-

ect staff also seek to influence thegeneral school climate of the fourproject schools by serving as infor-mational sources for alternativediscipline procedures, and by sit-ting on appropriate building anddistrict-level committees concernedwith discipline, attendance, andschool rules.

Program Changes During 1981-82

The only major chanO in the.highschool SSC between the 1981 and 1982school years was the decrease in theamount of time that the class was

,team taught. In previous years, theproject co-directors had often teamtaught the class, particularly whenstudents were having difficulty, orsome special problem came up.Apparently due to time spent engagedin evaluation activities, the SSCcounselor was_forced_to work alonein the SSC class more than antici-pated.

In the SAC two changes were madein the program during the year.First, a pre- and post-testingprocedure for the remedial writingclass was revised to make it moresensitive. It was changed from afive-part to a seven-part examina-tion. The SAC instructor devoted agreat deal of time to developing anadequate diagnostic and evaluativeinstrument- -one that paid handsomedividends (see Evaluation Results,below).

A second change in plans for theSAC was the emphasis that was placedon the remedial writing class forthe SAC participants. In fact,"Student Activities Center" wassomewhat of a misnomer for theintervention during the 1981-82year. The primary intervention ofthe SAC was, in fact, the writingclass, and student activities took a

ilkes4 goo to SIP* develegmeet et ibismosimmemem. Dow that Use lately's-14me 111** two lstemed and easy ofrte Mom* totted imm, ewe time willIv cosstOWs for stftleet sett Studs.

*be v4A01* Wool. DOC cow a dee-init4c Aimee Om Om mr14044 pre-gx*mw OHO' apewleem4 firm ttmt, l4gbloissodi DOC smidel. Omrtog tic ,44-tso4 *toms to apply the Itieb

N*04 oedel to the middle embos4dote( Om owing el 1DOI, tt toeOmmed thee middle istibeol otudeetstemps*. mtgbit, mod sink evaders)446 OW bowie ties perms* maturity411 ts Ott, (members to dealseemly est* ta> Imterperomsw4 andtotomporosee4 tee** that were theombiovi if dhows is is the SOC411* **P00 leer tise 1,1142 yew. a01444144re moe developed is sibtcb

e4444e 40.04 WIC peritctemete metempetweee *40 their rep -tine SSC

oemosotet for 14+ twits prier to the4044Si it grew ettiettiem. It wee*mod Omit after the otudeste baddesedomod e permed Srls6time velem

WA* MOP at .eloop mouseelere,Obom semi. be able to VOS104 tbepoor prom ere to ""are owe durteeOw 4044 mod rhos ski gremps mould440 44111 1441$. this &age use em4y0404014 sesteeesfel.

40041tems4 shows tree tbe Ugh**Mod model that mote tosorporstedtato SO* middle *Owl 1K wore tbc*04Stbes of a meOtly limey day Is

milsimb the memmeelm, owed me es400441M41 Wore mod the use if seresiVtwttred aes4goetsia red saki-atm* Doris. tbe 11l 10. doe. goateedof totriog prtmerily es tbe *tediousmod tbete problem me the se ofsomtiv4.1 Oer weep disemestee, Op-ts* wee doimootoed beferebood bytie DOC mommelmee, aid promoted to301; groove Also, a greeter emptiest"was 'Wed so disci sod astemgoof. at Whe eobool rules dortme theO$C portbd. Is ~lbw satonm, torued Che pomp mind its bebevtor.

Plymouth

the class was always teamtemsbt--both Kiddie school SSCinstructors participated in eachclass meetimg.

12111aLliaSiliclidisaLl

tligklasstLissitar.litnisaLranurThe Sly mouth Alternative Educa-

tive project successfully imple-meted is tree experiment in the highsebcca Student Services Center(1K). This experiment was carriedset for both semesters of the11151-$2 school year. The experimen-tal design for the first semester*broke down sonevbet as a number ofthe masts randomly assigned totrosteemt refused the intervention.flisadominatiom was accomplished*elms a table of random numbers.) Amere adequate selection procedure,imvolvial a group meeting to providemore information about _the programto potential students, resulted infewer refusals by treatment studentsfor the second semester experiment.

Results for the year-long experi -moot are presented in Tables 1 and2. Table 1 pressing the results foranalyses of variance comparing therandomly assigned treatment and con-trol students on the measures ofPlymouth's seals as outlined in

their PIK plain. Table 2 gives theresult. for (limiter analyses ontheir PD2 plan's objectives. (Sepa-rate saalysea that are limited onlyto the second summer treatment andcontrol students yield essentiallyidentical results.)

As footnote I on Table 1 butt -

sates. too few students successfullyloopleted the Inpression scale forit to be analysed. The Expressionscale vas a self-report measure ofexpressive communication abilitythat was developed with projecthelp. It was added as an addendumto the regular SAES questionnaire.

32

Plymouth

A more extensive version of the

scale has been developed for1982-83, and will be included in theregular version of the SAES studentquestionnaire.

The G.P.A. measure reported inTable 2 is only a weak measure ofachievement. This is because it wasthe G.P.A. for the entire academicyear. Therefore, any effects of thesecond semester treatment will bewatered down by the first semesterpre-treatment G.P.A. All reportedhigh school G.P.A. data suffers fromthis defect.

Table 1 shows that out of the 14measures of the SSC goals, four aresignificantly different for treat-ment and control group students.Only one of those differences favorsthe treatment group: Treatment stu-dents attempted significantly moreclasses than the control group.This, in effect, is an implementa-tion measure. One of the programobjectives is to move the studentstoward graduation, and SSC partici-pants are required to take a fullload of classes, including the SSCitself, for which they receivecredit toward graduation. Impor-tantly, the treatment students do

not pass any more classes than thecontrol students.

The SSC results in three negativeeffects on goal measures. The SAESmeasures of Interpersonal Competencyand Parental Emphasis on Educationboth significantly favor the controlgroup. The treatment students alsoreport more drug use than the con-trol students.

The SAES measures of Alienationand Interpersonal Competency areperhaps the most direct SAES meas-ures of the attitudinal and intra-personal variables that the SSC

seeks to affect. In this data,Alienation shows no difference, and

Interpersonal Competency shows a

negative effect of the treatment.

The negative effect on drug useis particularly disturbing, since itsuggests that far from decreasingthe drug use of their clients, theproject is increasing SSC students'involvement in drugs.

Turning to Table 2, of the 12non-redundant measures of the SSC'sobjectives, there are two dependablyand two marginally significanteffects. Two of these effects favorthe treatment, and two the control.Treatment students are slightly less

,,likely to be suspended out of

school, and are also somewhat lesslikely to voluntarily withdraw fromschool. These are important, if

weak, positive program effects. Thecontrols, on the other hand, skipclass less, and report themselves tobe more involved in conventional(essentially school related) activi-ties. No other differences, includ-ing four other relevant measures(see the bottom of Table 2) approachsignificance.

-442-

Several hypotheses were advancedby the project co-directors to

account for these results. They areexplained more fully below, but oneincluded the observation that posi-tive benefits of the program may beof a more long-term nature; studentsmay initially have some negativeaffect as they are forced to con-front themselves and their problemsin the group context, but that theself-learning that occurs, alongwith the support provided by projectstaff, should better equip the stu-dents to cope with the large, imper-sonal school environment. In orderto examine this hypothesis, a sepa-rate series of analyses of variancewere run looking just at the firstsemester treatment and control stu-dents. The treatment studentsshould have had an extra semester

328

for the longer term effects of thetreatment to be manifested. Becauseof the decreased number of experi-mental participants, this analysiswill be less sensitive than thewhole year analysis.

Table 3 gives the results for allsignificant effects and nearly sig-nificant effects, for the same out-comes listed on Tables 1 and 2.

Also listed are the results of ananalysis of the Invalidity scale, ameasure of truthfulness in report-ing. There are six significanteffects, all of them indicating anegative effect of the treatment.First semester treatment students ascompared to controls reported lessInterpersonal Competency, lowerSelf- esteem, more Rebellious Auto-nomy, less Involvement in conven-tional activities, lower Attachmentto School, and less Parental Empha-sis on Education. No other measuresapproached significance.

The hypothesis that the SSC stu-dents would exhibit increased posi-tive programmatic effects over timeis not supported. If anything, thenegative effects on the interper-sonal and intrapersonal variablesmeasured by the SAES increase overtime.

Potential explanations of the

negative treatment effects. To whatdo we attribute the short- and

long-term effects of the SSC treat-ment? Analysis of program implemen-tation, and discussion with programpersonnel suggest eight possibleexplanations for the combination ofeffects found:

1. Composing the groups entirelyof students with academic and beha-vioral problems may have resulted innon-conventional group norms. Thiscould have produced in- and out-of-school friendship cliques that werepredominantly non-conventional in

nature.

-443-

Plymouth

The SAES questionnaire includes ameasure of Negative Peer Influ-ence--the extent to which the res-pondent's peers get in trouble andencourage the respondent to beanti-conforming. Analyses of vari-ance of this measure for the firstsemester treatment and control stu-dents shows that there is a veryweak trend toward treatment students(M = .48, N = 17) reporting morenegative peer influence than controlstudents (M = .40, N = 22; p =.19).For the comparison of all experimen-tal students for the whole year, wefind a similar, but even weaker pat-tern of effects. The treatmentgroup (M = .46, N = 46) reports veryslightly more negative peer influ-ence than does the control group (M= .41, N = 42; p = .27). This dif-ference is trivial and non-signifi-cant.

Evidence_more strongly _supportinga negative peer influence effectcomes from the difference in druguse and class attendance betweentreatment and controls in the yearsample. Adolescent drug use is

highly related to peer use and maybe subject to peer influence (Kan-del, Kessler, & Margulies, 1978).The treatment students may be influ-encing each other to increase druguse. Likewise, the increased classcutting is consistent with the nega-tive peer influence hypothesis. Ixall, the evidence for a negativepeer influence effect is equivocal.Project personnel should neverthe-less be attentive to its effects.

2. A second possible explanationfor the negative effects of the SSCtreatment lies in the possibiltiy ofa counselor-validated norm that,"The school stinks, but you needthat piece of paper." A primarygoal of the Plymouth project is toincrease the graduation rates of

potential drop-out students. One ofthe ways that the SSC addresses this

329

Plymouth

goal- is to validate the students'perceptions concerning the impersonal nature of the school and thelack of caring of some of theirteachers and parents, while at thesame time emphasizing to the students how important the high schooldiploma is. The students are taughtto cope with secondary education,and survive in the building, but notnecessarily to enjoy school. Thisexplains how the students can becomeless attached to school, decreasetheir involvement in school activities, and skip class more often, andat the same time ,reduce their rateof voluntary withdiawals. Given thestrong relationship between attachment to school and graduation rates(Bachman, O'Malley, & Johnston,1978; D. Gottfredson, 1982) this isa risky approach to the problem ofdropout. This is particularly truesince the students are attendingclass significantly less than thecontrols, despite programmaticefforts to closely monitor attendance.

3. An explanation for the consistent negative effects on theParental Emphasis on Educationselfreports comes from the datafound in Table 4. Table 4 shows thenumber of parent phone calls, parentvisits, and parent meetings recordedfor each of the four Plymouth project treatments. Collapsing parentvisits and parent meetings, it is

apparent that, although parents arehighly encouraged by program personnel to become involved in theirchildren's education, and are continually invited for conferences andprogram activities, the high schoolSSC treatment parents average lessthat 1.5 personal visits per semester. Throughout the semester, thestudents are behaviorally remindedthat their parents do not, in fact,support their educational efforts.They are given clear evidence thattheir parents do not care enough to

attend program functions. Whetherthis is a longterm negative effectof the program can only be knownthrough followup.

4. A hypothesis advanced by theproject codirectors for the negative selfreport measures (excludingselfreported drug use) is that thepositive intrapsychic effects ofthe. program will only occur overtime. This was the reason thatseparate comparisons were made ofthe first semester only treatmentand control groups. That analysisindicated, that if anything, thenegative attitudinal effects

.,increased over time. Yet, it is

still possible, although unlikely,that a still longer followup periodwill be necessary to demonstratepositive effects. It must be saidthat at the current time the weightof the evidence argues preciselyopposite to a delay of _positiveeffects explanation.

-444-

5. Another hypothesis concerningthe selfreport measures is that thetreatment students responded morehonestly. A large part of theaffective education instruction provided in the SSC class involvesteaching students to honestly evaluate their intrapersonal feelings andresponses. Students are encouragedto openly express their feelings inthe class. Indeed, one of the primary reasons that it became difficult to design a middle school SSCthat paralleled the high schoolmodel was that middle school students could not as easily trust oneanother to keep confidences so thatclose personal feelings could beexpressed without fear of ridicule.The project codirectors argued thatthis intrapersonal reflection andemphasis on honestly would contri'bute to differences between treatment and control students through atwo stage process: (a) because of

the emphasis on examination of them-

330

selves and their problems studentswould become more aware of the prob-lems they have--they would be carry-ing less "false autonomy," and(b) they would therefore be morelikely to report honestly abouttheir personal feeling and behaviorswhen responding to the question-naire. These are two subtly differ-ent processes, but they would com-bine to make the treatment groupmore likely to report negativeevents on certain measures, even ifthey did not actually differ; infact, they may not be as bad off asthe control students. Ironically,in the view of the project co-direc-tors such "negative" self-reporteffects of the treatment could actu-ally be positive treatment effects.

It is difficult to evaluate theaccuracy of this claim. Some datais available from the SAES Invalid-ity scale. Invalidity is a measureof truthfulness in reporting andcontains items such as "I read sev-eral whole books every day."Although this measure was designedto detect students who were notresponding seriously and honestly tothe survey, it shows no differencebetween the first semester treatmentand control groups (see Table 3).But, the argument advanced is thatthe control students may not be as"self-aware" as the treatment stu-dents, so the question is really notone of lying on the questionnaire.Therefore, the invalidity scale onlyobliquely addresses the "honest"reporting issue.

On the other hand, the "self-a-ware/honesty" explanation does notexplain the differences in self-re-ported drug use; the measure is a

factual report of use that does notrely on being more self-aware formore accurate reports. Particularlysince there were no differences onthe overall delinquency self-re-ports, it is difficult to hypothe-size Chat. the treatment students

-445-

Plymouth

were responding more honestly to thebehavioral measures. Why should thestudents honestly report their ille-gal drug use, and falsely reportother delinquent acts?

Obviously, differences in self-report style cannot account for thenegative program effects on classattendance where school records werethe source of the measures.

6. Another explanation for thepattern of negative effects is thatrandomization broke down, and thatthe experimental and control groupsdiffered at the outset of the evalu-ation. Table 5 gives the resultsfor analyses of variance comparingall program treatment groups andtheir respective controls on affec-tive pretest measures gathered priorto the experiment. Because many ofthe students failed to adequatelycomplete the pretest, N's for theseanalyses are very low. Neverthe-less, there is no indication fromthese data that the high school SSCstudents differed at the beginningof the treatment period. Only onecomparison out of nine approachessignificance, and this is wellwithin the bounds of chance.

7. A seventh possible explanationfor the results is that the negativeself-report effects are real, andthat .they may be the result of

inconsistent implementation of pro-gram components. Unfortunately, weonly have two sources of implementa-tion data. One is records of parentcontacts, the other is the actualtime, in days, spent in the treat-ment program. The correlations ofparent contacts and time in treat-ment with each other and with alloutcome measures were calculated.None of these correlations was sig-nificant. Table 6 shows the corre-lations between time in treatmentand parent contacts. It is apparentthat there is no relation between

331

Plymouth

the amount of time the treatmentstudents are in the class, and thenumber of parent contacts that aremade. If contact with the students'parents is an integral part of thetreatment process, then at leastthis component of the interventionis not being systematically applied.

Related to implementation issueis the significant increase in classcutting shown by the treatment stu-dents. Because class attendance isan important target behavior for theprogram, it is obvious that programimplementation is breaking down.One of the very behaviors that'isspecifically addressed is moving inthe wrong direction.

8. A final possible explanationfor the negative self-report effectscould lie in differential responseto the SAES, questionnaire. Table 7lists the reponse rates for the var-ious Plymouth project samplinggroups. Fewer of the high schoolSSC control than treatment studentscompleted the questionnaire. Sincethe students who are not surveyedmay show poorer outcomes than morecooperative students, some of thenegative effects may be due to thedifferential survey response rates.Of course, to explain the self-re-ported drug use results one wouldhave to argue that the non-surveyedcontrols differed from the surveyedtreatment students in their druguse, but not in their delinquency.In any case, the differential surveyrates make evident the need forstrengthened efforts to administerthe 1983 SAES student questionnaireto control group students to achievehigh response rates for both treat-ment and control groups.

Examining the entire pattern ofresults, and the possible plausibleand implausible explanations forthem forces us to conclude that thejury is still out on the high school

SSC. The self-report results areseemingly negative (especially fordrug use), and class attendance is

negatively affected, yet there is astrong positive trend in the wholeyear sample in the key behavioralmeasures of voluntary withdrawalsand out-of-school suspensions. Wewill have to wait for replicationresults and an examination of newdata with higher control group res-ponse rates to reach a final conclu-sion concerning the efficacy of thehigh school Student Services Centermodel-

One closing point should be noted,,concerning the high school StudentServices Center: the program is

very popular in the school system.It has been funded in the districtfor nine years, and will be sup-ported when federal funding is nolonger available. School systemadministrators, at least, view theprogram favorably. When confrontedwith the interim evaluation results,the typical administrator's responsereportedly is "Johns Hopkins mustnot be measuring the right things."Although we can hatdly imagine a

longer questionnaire, or more exten-sive data collection, the projecthas cooperated in helping us design"affective education" measures.They have been incorporated into the1983 version of the School ActionEffectiveness Study survey. Furtherspeculations await better data.

High School Student Activities Cen-ter

-446--

A true experiment was carried outduring the second semester of the

1981-82 year to evaluate the highschool School Acitivities Centerwriting class (SAC). Table 8 givesthe results for analyses of variancecomparing random treatment and con-trol students on measures of theSAC's goals.

332

The Capitalization, Punctuation,Words, Sentences, Sentence Struc-ture, Study Skills, and Essay meas-ures are all derived from pre- andpost-tests measuring the writingskills covered during the writingclass. Of the six test componentcomparisons, (the Essay comparisonis based on too few persons forinterpretation) three are signifi-cant, and one shows a marginallysignificant trend. In all cases thetreatment students perform betterthan the control students. It isapparent that treatment students areimproving their writing skills.

Four other measures of the SAC'sgoals are also significant. SACclients attempt more classes thanthe control students (and the per-centage of classes they passapproaches significance, p = .12).SAC students report themselves to beless alienated, and to have greaterpractical knowledge. Although theSAC students report themselves to bereceiving significantly highergrades, the results of the analysisof the actual grade data do not sup-port the self-reports. As notedearlier, however, the G.P.A. dataare for the entire academic year,and the experimental students wereonly in the program during the sec-ond semester. Any program improve-ments would be diluted by the firstsemester pre-treatment data. Thefourth effect is a non-significanttrend for SAC students to reportthat their parents put less emphasison their education.

Table 9 gives the results ofanalyses of variance comparing thetreatment and control students onmeasures of the SAC's objectives.None of the measures that do notoverlap with SAC goals are statisti-cally significant (Alienation is

discussed above).

The experimental evaluation of

-447-

"lymouth

the SAC writing classes suggeststhree conclusions:

1. The students are improvingtheir writing skills, and may beincreasing their overall academicperformance, at least so far as itis measured by self-reported grades.Follow-up of these students shouldresult in better information con-cerning general academic improve-ments.

2. Two psychosocial developmentoutcomes favor the SAC students:Alienation and Practical Knowledge.We do not know whether these effects

'thea direct result of the class and

the counseling that accompanies it,or an indirect effect of increasedacademic success. Since the SAC andSSC students appear to be very simi-lar by the pretest measures (seeTable 5), and the SSC program is

explicitly a counseling program,while the SAC is primarily a reme-dial academic program, one cannotbut be struck by the possibilitythat increased academic achievementresults in better "psychologicalhealth"--and that the SAC is a morefruitful intervention than the SSC.

3. The trend toward the SAC pro-ducing lower self-reports of Paren-tal Emphasis on Education is inter-esting because it parallels resultsfound for the high school SSC. It

was argued for the SSC that programpersonnel seek to involve parents intheir children's education, butreport that they are largely unsuc-cessful. This may serve to rein-force for the students the knowledgethat their parents do not care abouttheir educational progress. A simi-lar process may be operating in theSAC. The control students in boththe SSC and SAC may be in blissfulignorance of their parents rack ofconcern about their education.

333

Plymouth

4. A fourth conclusion is thatthe SAC seems to be well imple-mented. This conclusion flows fromtwo pieces of data. The first is

the observation that the SACinstructor has spent a great deal oftime developing curriculum materialsand pre- and post-tests to monitorthe progress of his students. It issignificant that he has revised hispre- and post-tests. This indicatesthat he is paying attention to thedata he receives from his test, andis responding accordingly. This isa positive sign for any programdevelopment effort.

A second piece of implementationdata can be found in Table 10. Thistable shows the correlations betweenparental contacts and time in theSAC class--measures of implementa-tion--with selected pretest and out-come variables. Parental contactsare highly correlated with severalof the outcome variables, notablythose having do with disciplineinfractions, and academic perfor-mance. Particularly strong (r =.60) is the correlation betweenclasses missed and parental con-tacts; the SAC instructor is notif-ying parents when their children arenot attending class. Also stronglycorrelated with parent contacts is

the Expression scale measured atpretest; tilt more expressive thestudent, the more the parents werecontacted. This pattern of correla-tions is in mrrked contrast to thatfound in th SSC data in which theparental itacts were not corre-lated -ay other measure. Table10 also shows that time in the SACclass is unrelated to parental con-tacts, so, like the SSC, parentalcontacts are not uniformly distri-buted over time. On the other hand,relationships between parental con-tacts and outcome variables showthat the SAC instructor seems to beresponding systematically to thebehavior of the students.

The Middle School Student ServicesCenter

Early in the 1981-82 year anattempt was made to implement a trueexperiment in the middle school Stu-dent Services Center (SSC). Thisdesign broke down when there wereinsufficient referrals to enablestudents to be randomly assigned totreatment and control groups.

As a fallback design, the projectco-directors identified- a non-equi-valent, control group of students inthe school, and administered theproject pretest to them. Table 5

,gives analyses of variance resultscomparing the SSC participants totheir non-equivalent controls onpretest self-report scales. Of the

nine comparisons, five are signifi-cant. Three of the pretest measures(Alienation, Attachment to school,and Expression) favor the controlgroup, and two (Belief in the rulesand Interpersonal Competency) favorthe treatment group. It is puzzlingthat some of the measures favor thetreatment group, and some the con-,trol, but the consequence of thegross differences between the groupsis that it will be very difficult toexplain any differences betweengroups on outcome measures becauseit will be difficult to disentangletreatment effects from pretreatmentdifferences.

-448-

Tables 11 and 12 give the resultsfor analyses of variance comparingthe treatment and non-equivalentcontrol groups on measures of themiddle school SSC goals and objec-tives. Of thirty-three outcomemeasures, fourteen show statisti-cally significant differencesbetween the groups and five othersshow strong trends. In all cases,the treatment group shows pooreroutcomes than the control group.Recall, however, that it is probablethat the groups differed before

334

treatment. The analyses imply thatthe treatment does not change thebehavior of referred students to

match that of non-referred students.(Analyses of covariance controllingfor pre-existing differences wereuninterpretable; too few studentscompleted both pre- and post-teststo allow for meaningful analyses.)

One positive note for the programmay be found in Table 13. Thistable lists selected correlationsbetween parent contacts and time inthe SSC with the various outcomemeasures. 'The table indicates thatthe counselors in the SSC areresponding to misbehavior andtruancy by contacting parents.

Learning Options

We were unsuccessful in elicitingan agreement from Learning Optionspersonnel to implement a true exper-iment or any other satisfactory sum-mative evaluation. Since LearningOptions serves adolescents with sev-ere behavior problems, identifying ameaningful non-equivalent controlgroup is difficult. Therefore, we

are left with no control group andno outcome evaluation of the Learn-ing Options program. This is unfor-tunate, in that the Learning Optiionsprogram appears to be a theoreti-cally based and well-implementedattempt to deal with problem stu-dents in an alternative educationformat.

For descriptive purposes Table 14gives the means and standard devia-tions of relevant outcome measuresfor the Learning Options students.

Other Proiect Accomplishments

A major accomplishment of thePlymouth Alternative Education Proj-ect in 1981-82 was the implementa-tion of a new Plymouth-Canton high

Plymouth

school attendance and withdrawalpolicy. The old attendance andwithdrawal policy was that studentswere automatically withdrawn fromthe high school if they accumulatedeight unexcused absenses. The proj-ect co-directors, while serving onthe high school's Attendance PolicyCommittee, were instrumental in get-ting this policy changed to one inwhich there are no automatic atten-dance related withdrawals. Instead,students who acquire. eight unexcusedabsenses are referred' to in-housesuspension upon their return.

The project staff was also pri-,,marily responsible for developingsystematic disciplinary record-keep-ing procedures at the the two proj-ect middle schools. Neither schoolhad adequate procedures (one kept norecords at all). In this case, thenational evaluation had an indi-rectly positive effect on implemen-tation. When the school with no

record-keeping procedures resistedthe development of such procedures,it was required to comply by thesystem superintendent after projectstaff informed the superintendentthat the national evaluationrequired discipline data.

-449-

Organized student activitiessponsored by the SAC included notonly the spring intramural volley-ball program, but a dance group, a

"Thursday afternoon discussiongroup," and a club called "MusiciansUnlimited." All of these groupswere student-initiated, and the SACprovided space and organizationalhelp. One unexpected positive bene-fit of the dance group is that it

developed into a support group forthe small number of blacks thatattend Plymouth-Canton high school.The center became a place whereblack students could meet togetherover lunch, dance, or just talk.

335

Plymouth

Major Forces Affecting the 1981-82Program

During the 1981-82 school year,two major environmental eventscaused difficulties for the PlymouthProject: (a) requirements of thenational evaluation, and relationswith the national evaluation staffat the Johns Hopkins University, and(b) a school district reorganizationthat moved ninth grade students fromthe high school to the middleschool. In addition to these twomajor forces, the project wasaffected in less dramatic ways by

the poor economic situation in

Michigan, and obstacles presented bythe physical education department inone of the high schools to theimplementation of the Student Activ-ities Center's spring intramuralvolleyball program.

Evaluation Issues

The relations between thenational evaluation staff and thePlymouth Project remained rocky formuch of the year (see the firstevaluation report), but showed muchimprovement by the spring of 1982.Efforts in the Fall of 1981 to

implement true experiments in thehigh school Student Services Center(SSC) and Student Activities Center(SAC) met with some success, butproblems developed because of (a) adrop in referrals (particularly at

the middle school SSC) when teachersand administrators found that accessto the program could not be guaran-teed (due to the randomization pro-cess), and (b) randomly assignedreferred students refusing to takethe treatment. At the high school,both the SSC and the SAC did under-take experiments during the firstsemester, but many of the studentsthat were randomly assigned to thetreatment did not receive it. This,coupled with the reservation of manyof the slots in the programs forcontinuing students and students

referred from the Learning Options(LO) component, resulted in few stu-dents actually being experimentaltreatment and control group members.

The result of these events at themiddle school was that an experimen-tal evaluation of the middle schoolStudent Services Center was notundertaken, and the project wasforced to identify a non-equivalentcontrol group of middle school stu-dents not referred to the program.This procedure was not successful(see above).

At the high school, the project,screened students more carefully forthe second semester randomizationprocess, and developed a group pre-sentation procedure whereby poten-tial program participants couldlearn about the program, and couldconsent to being involved in one orthe other program before randomiza-tion took place. This procedure,while not eliminating the problem oftreatment refusals, decreased it

sufficiently so that a true experi-ment of adequate power was carriedout in both the high school SSC andSAC during the second semester of1981-82.

-450-

In the meantime, project stafffound the randomization procedure,the core data collection require-ments, and the administration of theSchool Action Effectiveness Study(SAES) questionnaire to all SSC andSAC treatment and control students,LO students, and random samples ofthe two high and middle schools puta severe stress on their resources(see the first interim report). Afurther difficulty was a change in

the evaluation staff member assignedto the project. The Social ActionResearch Center (SARC) field workerassigned to the Plymouth Projectleft the evaluation in the fall, andthe responsibility was assigned to

the SARC Coordinator of Formative

336

Evaluation, who subsequentlyresigned in January of 1982. Theauthor was assigned to work with thePlymouth project in mid-January1982.

The project staff attributes sev-eral negative effects to the demandson their time that the evaluationimposed in 1981-82: (a) they hadless time to devote to "systemic"goals, to working within the schoolsystem at large to humanize the edu-cational process, (b) the projectco-directors could not provide the

personnel support to the SSC and SACprograms as they had intended,(c) staff coordination and planningdecreased as staff meetings werecancelled to make time for evalua-tion activities, (d) program refer-rals decreased, (e) and the per-ceived roles of the projectco-directors changed from that of

direct service providers to grantadministrators, evaluators andpaper-pushers. The project co-di-rectors feared that this might nega-tively affect their future in theschool system once the Federal fund-ing runs out.'

It is difficult to assess howserious each of the foregoingeffects has been. It is certainlytrue that there were referral diffi-culties, particularly at the middleschool, but these problems coincidedwith a re-organization that placedthe projects primary source of

referrals, ninth graders, in the

middle schools where the project wasnew and in which referral sourceshad not yet been strongly developed.Concerning time for planning, if

evaluation activities did interfere

1 We have recently learned that thedirect costs of maintaining the Ply-mouth project will be picked up dur-ing the 1983-84 school year by thelocal school district.

Plymouth

with planning, this is unfortunate:A primary objective of the ProgramDevelopment Evaluation process is a

systematic increase in planning.

Systemic change activities,although an interest of the project(see "Policy Changes" above), havenever been a project priority.Project staff have repeatedly statedthat ,heirs is primarily a treatmentintervention, and in fact, havestressed to the national evaluationstaff their desire to keep a lowprofile in project schools. As

regards the change in the roles ofthe project co-directors, part of

,this was to be expected, becausethey are, indeed, grant administra-tors; the addition of a projectstaffer in charge of many evaluationtasks for the 1982-83 year shouldalleviate some of those servicedelivery vs. data gatherer distinc-tions.

-451-

Finally, to the extent that eval-uation activities contributed to a

lack of direct program support by

the project co-directors, (and thisissue was raised by virtually every-one connected to the Plymouth proj-ect), then the national evaluationhurt the project. The difficultiesderive in part from a lack of proj-ect staff available to plan for andaccomplish evaluation tasks--arequirement that was not clear in

the OJJDP request for applications.Future requests for proposals shouldmake more explicit (a) the timenecessary for evaluation activities,and (b) the requirement that thistime be budgeted for in the originalrequest for support. If rigorousevaluation is a requirement of a

grant, as it is in the AlternativeEducation Program, then it should bea major criterion of selection forfunding, as it was not (GeneralAccounting Office, 1982).

337

Plymouth

The difficulties the project washaving in implementing evaluationrequirements, the turnover in evalu-ation staff, plus the project'sfeeling that they were spending allof their time on evaluation activi-ties precipitated anger over a draftof the first interim evaluationreport sent to the project for com-ments. The report criticized theproject's compliance with thenational evaluation at a time (sub-mission of the third-year refundingrequest) when the project felt it

could least afford such criticism.The project directors thereforeresponded with a written detailedcriticism of the report and thenational evaluation staff responsi-ble for it.

This event and its aftermath sym-bolize the entire year's negotia-tions between the project and JohnsHopkins: evaluation difficultiesbuilding to confrontation, culminat-ing in compromise. The firstinterim report was clarified to

underscore that while there had beenmajor problems with evaluationactivities, the project was makinggood-faith efforts to correct them,and that improvements were occur-ring. The most important changethat occurred was that the Plymouthstaff began to see ways to get thevarious tasks done (if some compro-mises were made), and the evaluationstaff was willing to allow the com-promises and provide some help in

order to see that the most rigorousevaluation possible was performed.For example, instead of a adminis-tering the SAES to a large randomsample of the high school population(a task that project staff felt wasabsolutely overwhelming), ten Eng-lish classes were surveyed; this wasa compromise for both parties: theEnglish classes are not as strong aclimate assessment as a random sam-

ple would be, providing a weakerevaluation of the school change aimssought by OJJDP, but the project washeld to doing the survey, and meet-ing response rate benchmarks, whichthey did. Johns Hopkins facilitatedthe Plymouth survey administrationby pre-printing mailing labels forthe project's parental ccasent let-ters. Negotiations between the par-ties ultimately resulted in the Ply-mouth project requesting funds in

their third year budget for an eval-uation staff person, which JohnsHopkins strongly supported in dis-cussions with OJJDP staff. Thesefunds were awarded for the third

.,project year.

-452-

The working relationship betweenJohns Hopkins and the Plymouth proj-ect is currently much improved. For1982-83 project personnel havedeveloped a more adequate non-equi-valent control group for the middleschool SSC (collecting better pre-treatment data, and matching thenon-equivalent controls more care-fully to the treatment population).They have implemented strong experi-ments in both high school programs,and have aided in the developmentand pre-testing of an SAES scaledesigned to measure attitudinalchanges resulting from affectiveeducation. Continuing compromisesinclude the quasi-experimental eval-uation of the middle school SSC, thecontinuing use of the English classsample in the high school, withdra-wal of the requirement to follow-upthe 1981-82 English class sample,and the dropping or the LearningOptions component from the thirdyear budget (this was also due to

the 30% budget cut requirement, butthe lack of evaluatability of Learn-ing Options--see the first interimreport-- contributed to the deci-sion)..

338

School District Re-organization

The second major environmentalchange influencing the project was adistrict re-organization that tookninth graders out of the high schooland placed them in the middleschool. Ninth grade students werethe primary source of referrals forthe high school program, so thetreatment population shifted upwardone year to include mostly tenthgraders. More importantly, theninth grade population was now in

the middle schools in which theproject was new and little known,,sothat referral contacts were notstrong. It became harder to iden-tify the students the project feelsare most suitable for the SSC treat-ment. In addition, the re-organiza-tion plan diverted many of the stu-dents that the middle school programhad been working with in the springof 1981 to other non-project middleschools where they could not begiven the follow-up support that theproject feels is necessary to suc-cessful treatment.

The re-organization also led to aproblem for the students returningfrom Learning Options to the regularschool curriculum. Since most of

the Learning Options students havebeen eighth or ninth graders, it hadbeen relatively easy to return themto the large, impersonal high schoolenvironment. At the middle schoolsthough, the project ran into diffi-culties when principals of the twoschools were wary of allowing theLearning Options participants backin the school. The Learning Optionsstudents were eventually allowed to

re-enroll, but project staff feelthat they are subjected to undue"surveillance."

Two additional climate effects ofthe re-organization are that boththe ninth grade students and theirteachers were angry about the

-453-

Plymouth

change. The ninth graders wereanticipating beginning high school,with the autonomy it presents, andthe ninth grade teachers felt thatthey were "demoted" from the high tothe middle school level.

A positive result of the re-or-ganization is that the SAC programwas given more building space at thehigh school than it had expected toreceive. It had room to conduct itsstudent actvities, and to have a

"home" room for the 'SAC writing

Other Forcefield Events

A third negative force in Ply-mouth's field is the severe reces-sionary economy in Michigan. WhilePlymouth, Michigan was not as

directly affected as Detroit proper,unemployment was up, and school dis-trict funding_ is_ largely, dependentupon state funding. As a whole, thestate of Michigan is suffering a

major fiscal crisis.

The recession affects the projectin two ways: First, it makes uncer-tain future continuation of theproject after Federal funding is

discontinued. This affects theproject now, because time must bespent investigating and shoring upadditional sources of funds. Forexample, Learning Options was notincluded in the third year budget,but is funded in 1982-83 primarilyby local and school system funds(see Footnote 1).

A second recessionary effectreported by project personnel at

both the high and middle schoollevel is that the students are suf-fering more family/home problems,problems directly related to unem-ployment. During the year, unem-ployment in Michigan averaged about12%. Project staff estimate thatamong their client population (pri-

339

Plymouth

marily working class) that theunemployment rate may be as high as30 to 40%. While we cannot verifythe magnitude of either of thesenegative effects of the recession,they certainly seem plausible.

As previously noted, a fourthnegative environmental effect on theproject was the required 30% budgetcut for the 1982-83 year. Thisaffected the project in the 1981-82year, because hard decisions had tobe made about where the cuts shouldoccur. Because the director of theLearning Options program and theJohns Hopkins evaluation staff hadbeen unable to come to an agreementconcerning an adequate evaluationdesign by February, 1982, the proj-ect decided to drop Learning Optionsfrom the third year funding propo-sal. In this manner, the SSC andSAC programs could be funded at

their current level, and funds couldbe acquired for an on-site evalua-tion staff person. It is unfortu-nate that we were unable to imple-ment an adequate evaluation forLearning Options, perhaps the mostpowerful of the Plymouth interven-tions.

Two forcefield events had effectsprimarily on the high school SACprogram. In the spring of 1982,when the project tried to implementan intramural volleyball program forstudents not involved in otherschool events, the project ran intodifficulties from the physical edu-cation instructors. They were waryof lending space and equipment to a

venture designed for what they per-ceived to be marginal students. TheSAC instructor had to see four per-sons, finally returning to his ini-tial contact before he received per-mission to use the gymnasiumfacilities after school. Thisresulted in the program startinglater in the semester than intended.The project was also forced to pur-

chase their own volleyballs, as theP.E. department would not let themuse its equipment.

A related problem that the SACran into was the lack of schedulingflexibility in the high school. Itwas difficult to schedule activi-ties, and most had to be held afterschool.

Changes in the Force Field for1982-83

The only anticipated major changein the force field is the markedlyimproved relationship with the

,national evaluation. This shouldoccur for four reasons: (a) ahighly trained evaluation stafferhas been assigned to the Plymouthproject as their evaluation contactand better continuity in and a

smoother working relationship withevaluation personnel is expected,(b) a full-time person has beenhired to perform evaluation taskssuch as data collection, programassignment, and questionnaire admin-istration, (c) the procedures forcarrying out the experiments quasi-experiments, and SAES questionnaireadministration have now been "fieldtested" for a year, and (d) theGrowth Works component_, over whichmuch time had been lost trying to

develop an evaluation acceptable toboth parties, is no longer a part ofthe federal initiative.

-454-

Program Changes for 1982-83

A new intervention has been addedin the high school Student ServicesCenter to address the problem ofschool non-attendance. The datafrom the 1981-82 experiment indi-cated that that the SSC studentswere skipping school at-a high rate.A group competition has been set upin which each "group" (class period)competes with the other groups onincreasing attendance. This is an

3,10

*14 i*.0 4., 19+ tqw-ot rtopoior0 1044i thot,! 14,/* pietism, that *F4-4 ot. 4. «4 0414 1 14** wit is 4,40 *4

* *0./,440 .4****itkpi14* the vitaiii .004 414,0.144tki=4* t1 O4ts040414 4,* 4 4474 one ot444: 0444 *« m*«1140-**4 4 ietitelywit 44 tikktiit Slott 4 44iiikt RA% th* pro=

4*.04.4.4114) it *** bopoil ihrt,4.4+ +r 11+01 'J14 ,oloti 001.. ter *or=

44i 44.17* 1)44* .0#4,4110 f4 4404114041141404 4* 1101410- sorVO4 11 pullet.#104** 4* 14p Kw's fee a 1000* +1

'-44* *4*** bilk et4pt siellar str-440) ttiWti sort14 test the benefits

et.** onpvtience. sad **bid mat'. tr aespilit tops the poop pre-**se) 4041 felt Met this4..w *ii **it ****ttiott4Mi, 100.4$11044414,0 WO 414 *01 1,0000,11 from140; tt i w t to*, spooster showediitlip iisterspostet MA tom. TheyCeps leossiefrep diet limed thista8etiowd ties, fee 408241.lnets44, tone ttwatets will be4440*0.4 1* Off,* its pert facilite.toes in t40 'voter *C dewbose

* cott4016110 ttioPO in the highte4seol i* -*titli** of a taco'1041 401 Meet* **4 *tog, The $$Cclovristis* porteivied a weed is workitch 14.0 firma', of iiv deaths ofslow Meade and Motives, se adiettssima at these lassos willsaittard &setts 14p 4911243 semi-test:s

in lie high sp4me4 Stud/eat Attie-

tostet 9040 emphasle will beileee4 an the attivittes composest.Ste* ettlyittee will be pleased, andleiessestele will he orgamiaed 1 4

$444* 044 fill and oprimg aessestersone trill losolvie este parlitiptatit.iii* Floutth hove (the ',machretied) ei wits* of l..formaltheft is psaitm4d as topics each aseismitoolloo, 'lammed ptrosehood, andOelPitivo tore Early. Wimp s411 bedosSgmed te ostructere more of the

Plytoutn

target students' lunch tite--a time(Whit which they often get involvedin disciplinary incidents.

A further chew involves the 000Ot tutors. In 1981-42, non-programseniors were used as tutors, but theSAC ieetouctor frit that this proce-dure WO not satisfactory, becauuetk* SAC students resented the exper-tise of the tutors. In 1982-83 suc-cessful gritluAtcs of the programwill do tutoring.

Pinally, the SAC instructor wasthe senior class advisor during the19111-82 paT, and their representa-

itive poop operated out of hisoffice. or the following year. hewill be thL tenth grade advisorwhere he feels he can be more of acounseling aid. The SAC instructorwilt also advise the Student Forum(the high school student govern-ment), and that body will operateout of the SAC offices.

At the middle school SSC, lesstime will be spent on the one-to-onebonding, and the groups will bestarted earlier in the semester.The program's student evaluationinstrument, the Student BehaviorEvaluation (888) has been simplifiedfor 19112-83. The instrument was4eaigtied for use at the high schoollevel, and the middle school stafffeel that it is too detailed andcomplicated for middle school use.Data from an evaluation of the SSEsupports their assessment.2

2 Si. Report on utility for diagno-sis of student Behavioral EvaluationScales, April, 1982. Available fromauthor.

341

Plymouth

References

Bachman, J. G., O'Malley, P. M., & Johnston, J. Adolescence to adulthood:Change and stability in the lives of young men. Ann Arbor, Mich.: Insti-tute for Social Research, 1978.

Carlton, R. The Plymouth-Canton Alternative Education Project: InterimReport. In G. D. Gottfredson (ed.), The School Action Effectiveness Study:First Interim Report (Report No. 325). Baltimore: The Johns Hopkins Univ-ersity, Center for Social Organization of Schools, 1982.

Generel Accounting Office. The Office of Juvenile Justice and DelinquencyPrevention's Special Emphasis Program Has Not Realized its Full Potential(GGD-82-42). Washington, D.C.: Author, 1982.

Gottfredson, D. C. Personality and persistence in education. Journal of Per-sonality and So0a1 Psychology, 1982, 43, 532-545.

Kandel, D. B., Kessler, R. C., & Margulies, R. Z. Antecedents of adolescentinitiation into stages of drug use: A developmental analysis. In D. B.

Kandel (ed.), Longitudinal research on drug use: Empirical findings andmethodological issues. Washington, D.C.: Hemisphere, 1978.

7456-

342

Table 1

Outcomes on Measures of Plymouth High School

Student Services Center Goals

Goals

Treatment Control

M SD N M SD

Reduce alienation

Alienation .44 .27 50 .43 .25 42 .88

Improve academic skills

Self-reported schooleffort

.33 .29 52 .30 .27 42 .45

Self-reported grades 1.79 .76 43 1.91 .65 43 .419

8182 G.P.A. 1.66 1.53'' 45 1.82 1.57 45 .62

Classes attempted 5.34 .77 44 4.67 1.30 45 .004*

Classes passed 4.24. 2.06 46 3.91 1.99 46 .44

Percentage of classespassed

.77 .35 42 .76 .31 44 .93

Reduce delinquent behavior

Self-reported delinquency .42 .20 46 .37 .21 38 .27

Self-reported seriousdelinquency

.22 .23 46 .20 .23 39 .63

Drug use .89 .17 42 .78 .30 38 .05*

Truancy petitions 0.00 0.00 57 .02 .13 60 .33

Improve coping behaviors1

Interpersonal competency .73 .30 50 .84 .20 41 .05*

Rebellious autonomy .82 .27 51 .72 .35 42 .11

Practical knowledge 1.35 .40 51 1.34 .49 42 .91

Increase parental involve-ment in education

Parental emphasis oneducation

.32 .28 46 .42 .49 42 .007*

* Statistically significant.

1 The Expression Scale was completed by too few control students for analysis.

2 A very weak measure of achievement.

-457- 343

Table 2

Outcomes on Measures of Plymouth High School

Student Service Center'' Objectives

Objectives

Treatment Control.P.

SD N M SD

Increase attendanceSAES non-attendance index 1.52 .57 52 1.64 .57 44 .32Classes missed second 102.86 45.70

semester43 73.89 46.74 45 .004**

Classes missed perclass attempted

19.80 9.22 44 15.95 10.21 43 .07*

Reduce official voluntarywithdrawalsVoluntary withdrawals

9% -- 57 21% 61 .10*Voluntary and invol-11% 57 21% 61 .18untary withdrawals ..

Increase sense of belong-,,

ing and purposeAttachment to parents .41 .24 52 .41 .29 44 .98Attachment to school .46 .25 50 .47 .27 43 .86Alienation See Table 1.Involvement .07 .10 52 .13 .13 42 .02**

Decrease expulsions

-00-1:60Expulsions 0.00 0.00 54 0:00- -0:06Administrative with-

drawals.56 1.31 55 .70 1.74 54 .64

Increase self-esteemSelf-esteem .58 .18 49 .62 .18 41 .35

Increase participants'ability to own respon-sibility for their ownbehaviorNo measure

Reduce out-of-schoolsuspensionsOut-of-school suspensions .08 .27 .53 .29 .89 52 .10*

Increase parental involve-ment in schoolsParental emphasis on

educationSee Table 1.

Develop more positive par-ental attitudes towardschool and alternativeeducationNo measure

Other theoretically rele-vant measuresBelief .64 .25 51 .66 .24 42 .74Internal control .48 .29 51 .49 .27 42 .88Self-reported suspensions .31 .47 51 .28 .45 43 .72In-school suspensions .34 1.00 53 .25 .58 53 .55

* Strong trend.** Statistically significant.

-458--. 344yu

Table 3

Plymouth High School Student Services Center Has Negative

Effects on First Semester 1981-82 Students

Goals

Treatment Control .E

M SD N M SD N

Interpersonal Competency .76 .27 21 .91 .14 17 .05*

Self-esteem .52 .19 20 .73

t

.12 17 .0005*

Rebellious autonomy .92 .14 21 .73 .36 - 17 .03*

Involvement .06 .06 23 .12 .12 17 .05*

Attachment to school .41 .27 22 .62 .25 17 .02*

Parental emphasis oneducation

.31 .31 21 .54 .32 17 .03*

Invalidityl .10 .17 21 .11 .14 17 .98

1A measure of truthfulness in reporting.

*Statistically significant.

459=

345

Table 4

Parental Contact Data, Plymouth

Second Semester, 1981-82

Parent Phone Calls Parent Visits Parent Meetings

M SD M SD M SD

High School SSC 2.26 2.59 .48 .72 .87 1.108

Middle School SSC 5.32 4.11 .81 1.16 .41 .99

High School SAC 1.76 1.72 .19 .50 .33 .12

P.

1

a,Growth Works 9.68 7.91 4.46 5.52 .89 1.15

0

346

Table 5

Pretest Measures on Plymouth Program

Participants and Controls

High School SSC High School SAC Middle School SSC Growth Work

T C T

Attachment to parents 6.6 5.8 .14 6.8 6.7 .92 6.6 6.8 .49 6.7

Affection from parents 6.0 6.2 .77 5.9 6.2 .69 6.1 6.2 .81 6.5

Alienation 1.7 1.9 .55 1.7 1.6 .94 2.0 1.4 .001** 1.4

Attachment to school 4.7 4.0 .26 5.9 5.04 .46 3.03 4.4 .01** 5.0

Belief 4.1 4.0 .95 3.6 2.8 .17 2.7 2.3 .07* 2.8.;

Expression 2.0 1.9 .92 2.7 3.6 .12 2.2 2.8 .04** 2.9

Interpersonal compe-tency

3.3 3.3 .91 2.8 2.5 .61 2.2 1.7 ..01** 1.8

Self-esteem 7.5 6.5 .07* 6.6 5.0 .03** 4.9 5.0 .72 5.2

Internal control 2.9 2.7 .63 2.' 2.6 .63 2.2 2.2 1.00 2.5

* Strong trend.'* Statistically significant.

3 Sic

Table 6

Correlations Between Time in SSC Classand Selected Client Characteristics:

Plymouth High School SSC Class

Client characteristic

Time in SSC Class

r p

Parental meetings -.11 .47

Parent phone calls -.13 .37

Parent visits -.09 .53

-462- 343

Table 7

Percentage Completing Student Questionnaire

Plymouth, Spring, 1981-82

Treatment Control

High School SSC 81% 71%

Middle School SSC 91% 79%

High School SAC 81% 80%

Growth Works 62%

High School English Classes 97%

Middle School Random Sample 99%

463350

Table 8

Results on Measures of Plymouth High School

Student Activities Center Goals

Goals

Treatment Control

M SD N M SD

Reduce alienation

Alienation .36 .23 32 .56 .31 17 .02**

Improve academic skills

Capitalization) 1.44 3.90 34 1.08 3.57 13 .77

Punctuation1

4.91 3.15 34 -.38 3.15 13 .0000 * *.

Words1

.4.29 3.54 34 .54 .3.27 13 .002**1

.,

Sentences 3.74 4.70 34 1.15 4.86 13 .10*

Sentence structure1

2.09 5.22 34 1.62 5.33 13 .78

Study skills1

5.29 4.19 34 .62 4.72 13 .002**

Es ay1

'

27.06 11.41 36 -12.50 10.62 6 .0004**

Self-reported grades 2.08 .55 36 _ 1-61_ _.78_ _18_ _.01" _

8182 G.P.A. 1.62 .89 34 1.52 .63 21 .66

Self-reported schooleffort

.29 .28 36 .29 .21 17 .94

Classes attempted 5.41 .82 34 4.73 1.28 22 .02**

Classes passed 4.27 1.50 34 3.64 1.43 22 .13

Percentage of classespassed

.80 .27 -; .65 .43 22 .12

Reduce delinquent behaviors

Self-reported delinquency .28 .20 32 .34 .34 14 .35

Self-reported seriousdelinquency

.14 .19 32 .21 .22 14 .28

Drug use .59 .39 34 .63 .35 14 .77

Truancy petitions 0.00 0.00 34 0.00 0.00 25 1.00

Improve coping behaviors3

Interpersonal competency .81 .21 33 .75 .27 17 .44

Rebellious autonomy .85 .24 33 .88 .27 16 .72

Practical knowledge 1.48 .47 32 1.13 .64 17 .03**

Increase parental involve-ment I, education

Parental emphasis on ed. .32 .29 34 .47 .30 17 .10*

1 This is the average change from pre- to post-test.2 Very low n's make this analysis inconclusive.3 The Expression scale.was not completed by any control students.* Strong trend.** Statistically significant. -464- 351

Table 9

Results of Measures of Plymouth High School

Student Activities Center Objectives

Treatment Control

M

p

Objectives M SD N SD

Increase attendance

SAES non-attendance index 1.53 .61 36 1.63 .50 < 16 .58Classes missed second 91.41 55.79 34 79.82 50.69 2 .43semester

--...,

Classes missed per 17.05class attempted

10.14 34 18.55 13.71 22 .64,

Decrease official vol-untary withdrawals

Voluntary withdrawals 16% 44 8% 25 .57Voluntary and involun-

tary withdrawals 16% -- 44 8% 25 .57

Increase sense of belong-ing and purpose

Attachment to parents .45 .27 36 .32 .26 18 .12Attachment to school .49 .30 35 .40 .31 17 ,36Alienation .36 .23 32 .56 .31 17 .02*Involvement .06 .09 34 .03 .08 16 .25

Reduce expulsions

Expulsions 0.00 0.00 44 0.00 0.00 25 1.00Administrative with- .52

drawals1.35 44 .76 1.33 25 .48

Increase self-esteem

Self-esteem .64 .17 30 .59 .24 13 .45

Increase ability to ownresponsibility

No measure

Reduce out-of-schoolsuspensions

Out-of school suspensions .11 .49 44 .08 .28 25 .75

Increase parental involve-ment in schools

No measure

Develop more positive parentalattitude toward school andalternative education

No measure

Other theoretically relevantmeasures

Belief .59 .24 33 .54 .27 17 .48Internal control .58 .23 33 .56 .28 17 .81Self-reported suspensions .23 .43 35 .29 .47 17 .62In-school suspensions .20 .55 44 .44 1.08 25 .24

*Statistically significant -465- ..,

Table 10

Correlations of Two Implementation Measures(Parent Contacts and Time in SAC Class)with Selected Client Characteristics

for the Plymouth Middle SchoolSAC Random Treatment

Implementation measure andclient characteristic

Total parent contacts

Administrative removals .40 .003

Outofschool suspensions .48 .001

Total suspensions .43 .004

Classes missed .60 .001

Classes passed .32 .0681-82 G.P.A. .32 .06

Expression Pretest .71 .004

Time in SAC class

Parental meetings .09 .58

Parent phone calls .00 .98

Parent visits .03 .85

466353

Table 11

Outcomes on Measures of Plymouth Middle School

Student Services Center Goals

GoalsTreatment Non-equivalent Control n

M SD .N PS SD

Reduce alienation

Alienation .46 .25 38 .50 .25 76 .57

Increase academic skills

Self-reported grades 1.53 .94 36 1.93 .77 81 .02*

Self-reported schooleffort

.15 .18 35 .37 .32 83 .00002*

Classes attempted 7.32 1.13 43 7.37 1.17 98 .83

Classes passed 5.72 2.19 43 6.65 1.49 98 .004*

Percentage of classespassed

.76 .25 43 .91 .14 98 .0000*

Reduce delinquent behavior _ _

Self-reported delin-quency

.60 .29 32 .40 .27 74 .0009*

Self-reported seriousdelinquency

.53 .31 32 .33 .28 75 .002*

Drug use .80 .30 49 .52 .39 77 .0000*

Truancy petitions .02 .15 45 0.00 0.00 105 .13

Detentions 1.60 2.17 20 1.31 1.71 52 .55

Improve coping behaviors

Interpersonal competency .63 .24 36 .70 .24 75 .18

Rebellious autonomy .65 .31 35 .70 .34 76 .43

Practical knowledge 1.33 .37 34 1.33 .51 78 .97

Increase parental involve-ment in education

Parental emphasis on .50 .29 36 .49 .27 80 .87

*Statistically significant.

?Ate 12

'.4t04410* 40 14.miuro, of Plymouth Middle School

itimelest Services Center Objectives

4

0411.410,r4,Is et, wpm .tt orm

11,.,q1",111i A 4 4144/1444.

Afro-mad *mac1tmomov4944 si,ofmrspopu

tufAttalaixitLeditatavtmotonta

; onrirp

6104motorf mfil towlomp.

(*f m4664cow416

ibmilleOrTroot000t

X SO

Non-equivalent Control

SD N

44(001*440000,44. *0444440mlop1 t.#41

limmtvoftofts

1-44LtellettAtit".-Ate,f4041004*Ailloot,tfottlio 040-

4rows$4

tivatz., tan

Ilmockettamtv,"

4iwirio

foxtbutt.441/4004114n4

W-gtOttf.441MtiALOVALY4'tan*

1.54 .735.02 .996.4$ 11.07

It%

It%

374343

45

.. 451

.17 .27

.29 .21

.51 .24

.09 .14)

0.005.20 6.09

.49 .14

1. 91 2.29

fibbf**14. 4040i1144444i 00. Tadao 5.

**woo-04os

4014M41440440,14.1101MA

Im4ofma psomotroll

46011twOopotto6 6m*poototon6

log064600i 406Opoo61006

)7

36

34)

36

4544

12

44

.44 .29 11

.11 .24 5$

$66 .49 11

$65 1.21 20

1.11 .88

5.02 .70

5.36 5.60

62

62

.42 .29

.36 .27

.46 .24

.15 .19

0.00 0.002.00 4.40

.58 .15

.66 1.23

.55 .25

.46 .27

.49 .50

1.08 1.34

84

9999

.01**

.61

.03**

109 .51

109 .51

8582

7980

.41

.09*

.25

.10*

106 1.00102 .0005**

75 .004**

106 .0000**

78

78

80321.

4001 140140.011161410+ 81,18$18844. .468 355

.10*

.22

.09*

.2808*

Table 13

Correlations of Two Implementation Measures(Parent Contacts and Time in SSC Class)with Selected Client Characteristics

for the Plymouth Middle SchoolSSC Treatment

Implementation measure andclient characteristic r p

Total parent contactstft

Excused absences .35 .02

Administrative removals .26 .09

Truancy petitions .47 .002

Court referrals .44 .004

Time in SSC class

Parental meetings .11 .69

Parent phone calls .09 .73

Parent visits -.05 .84

-469-

356

Table 13

Correlations of Two Implemei(Parent Contacts and Timewith Selected Client Chz

for the Plymouth Mid(SSC Treatment

Implementation measure andclient characteristic

tft

Total parent contacts

Excused absences

Administrative removals

Truancy petitions

Court referrals

Time in SSC class

Parpntn1 mpptinpq

Educational Improvement Center--South Alternative Education Project: SecondInterim Report

D. E. Rickert, Jr.

The Education Improvement Center-South (EIC-South) is a technicalassistance and resource center. Theprimary clients are educators, butmany community groups and individu-als use the resources of the Center.The catchment area for the Center issouthern New Jersey, which is com-prised of Atlantic, Camden, CapeMay, Cumberland, Gloucester, andSalem Counties.

The Alternative Education Projectis one of many programs sponsored byEIC-South. It is the only projectwithin the larger agency that pro-vides direct services on a largescale. The details of EIC-South andits, organizational arrangements aredescribed more fully elsewhere (Ric-kert, 1982).

During the 1981-82 school year,the Alternative Education Project'sdelinquency prevention efforts weretargeted at four schools: Pleasant-ville Middle School in Pleasantvilleand Cleary Middle School in Buena,both in Atlantic County, D'IppolitoElementary School in Vineland, Cum-berland County, and Pennsauken Jun-ior High in Pennsauken, CamdenCounty.

Problems, Goals and Guiding Princi-ples

During the 1981-82 school yearthe explicitly identified problemsand goals (i.e., those identified inthe Program Development Evaluationplan) were the same as described inthe first Interim Report, but dis-cussion with -program staff made itapparent that the project does. notoperate on a "theory" in the sensethat it is used in Program Develop-ment Evaluation (PDE; Gottfredson,

-471-

Rickert, Gottfredson & Advani,1983). Rather, the project is basedon a flexible set of ideas orhypotheses. In the EIC-South Alter-native Education Project, staff dowhat seems most appropriate at thetime.

The basic guiding ideas of EIC-South's Alternative Education Pro-ject are best presented in the pro-ject staff's own words; they appearas an appendix to this report (seeAppendix A).

Objectives

In the first Interim Report manyobjectives and suggested measureswere described. Measurement wasonly im-plemenied--for -sa-meof fhese-(see Table 1). Project staff mem-bers were uncomfortable with thestructure elicited using the PDEmodel, and felt a more realisticdescription of their project waspreferable. At the end of the year,th refore, project administratorsrevised the explicit statement of

the problems their program wasaddressing and the correspondinggoals and objectives (see Table 2).

Interventions During the 1981-82School Year

In the first Interim Report, fivegeneral components of 'the EIC-Southprogram were described: (a) aschool climate improvement compo-nent, (b) a community process compo-nent, (c) a youth participation com-ponent, (d) a public relationscomponent, and (e) a leadership andtraining resources (technical assis-tance) component.

358

New Jersey

In reality, the major thrust ofthe program, in terms of direct ser-vices provided, were alternativeeducation classes, called School/Community Relations classes, held ineach of the four target districts; a"self-contained" class in Penn-sauken;) and "Community-Process" intwo of the target districts, Vine-land and Pennsauken. Many of thetarget students also received indi-vidual counseling from EIC-Southpersonnel. 'A small number of stu-dents participated in "specialclubs" (e.g. dance club, etc.).Some tutored students in lowergrades and some received tutorial'services. (See Tables 3 and 4.) Astudent referred to the AlternativeEducation Project could receive asfew as one treatment or a combina-tion of as many as six of the listedtreatments. This depended upon whatproject personnel perceived the stu-dent's needs to be, as well as onscheduling convenience. The follow-ing sections describe the project'smajor interventions.

School/Community Relations Classes

The way in which this interven-tion was executed evolved during the1981-82 school year. There was not,at the outset, a clearly articulatedset of implementation blueprints,although training in preferredapproaches took place during thepreceding spring and summer. Thefact that there were no blueprintsfor implementation is related to anincremental approach to planningfollowed by this project. The

1. The "Self-Contained" class inPennsauken existed prior to theEIC-South Alternative Education Pro-gram as a special class for studentswith behavior problems. When theEIC-South program became opera-tional, the self-contained class wasadopted as part of the program.

project director explained that it

was important for Teacher/Counselors(those responsible for carrying outthis intervention) to gain experi-ence in their assigned schools andthen execute the program as seemedappropriate.

In short, the Teacher/Counselors'activities are not guided by a sin-gle model, but rather by a caseworkapproach where what seems intui-tively correct becomes the interven-tion.

The general aims of the School/Community Relations Classes were thefollowing: (a) teach students cora-

"munication skills, (b) help studentsimprove their self-image, (c) workat resolving students' family prob-lems, (d) improve study skills, and(e) aid students in career explora-tions.

-472-

One Teacher/Counselor wasassigned to each of the four targetschools. Students chosen to parti-cipate met several times a week withtheir Teacher/Counselor in a class-room setting. Teacher/Counselorsseemed to have a lot of flexibilityto do whatever they thought bestwith particulAr students, subject toguidance by the Educational Direc-tor. They were responsible forkeeping the Educational Director andProject Director informed of theirtreatment plans, attendance, paren-tal contacts, and schedules. Ser-vices provided included counseling,home visits, tutoring in basicskills, and arranged meetings withcommunity leaders.

In addition to class time withstudents, Teacher/Counselors devoteda considerable amount of time to

providing advice and consultation toother teachers in the school, parti-cularly teachers of participant stu-dents. They reportedly also metfrequently with vice-principals withregard to discipline policy.

359

Comments by principals (1982

Principal Questionnaire) at the fourtarget schools indicate that theproject was perceived as a programto improve students' attendance,grades, discipline, and self-imagethrough individual counseling andbasic academic instruction. Oneprincipal of a project target schoolstated that he would like the pro-ject to develop more of a blueprintfor operation such as a curriculumguide. The 1982-83 program is basedon a more explicit curriculum guide.

How students were selected forthe School/Community RelationsClasses. As early as the proposal-writing stage, nine selection crite-ria were agreed upon by AlternativeEducation Project staff. They arethe following: (a) poor attendancerecord, (b) high number of suspen-sions, (c) low academic performance,(d) perceived "drop-out potential,"(e) history of delinquency,(f) history of discipline problemsin school, (g) court involvement forstatus offenses, and (h) high degreeof parental commitment to partici-pate. Also listed as a criterionwas that students should be referredby school administrators, teachers,school counselors, or parents.Early discussions with the nationalevaluation staff led to a plan todevelop pools of at least 60 stu-dents meeting project criteria ateach school; 30 were to have beenrandomly assigned to the treatment,and 30 to the control group.

At the end of the 1980-81 schoolyear, copies of the criteria forstudent selection were distributedto all administrators, guidancecounselors, teachers, and principalswho had contact with sixth gradestudents. Meetings with the the

potential referring agents were heldin each district to determine refer-rals for the project. Each team ofproject staff, consisting of a

Teacher/Counselor and a Community

-473-

New Jersey

Process Consultant, then conductedstudent-family interviews to deter-mine which youths would volunteerfor the project. Teacher/Counselorsthen met with school guidancecounselors to schedule volunteers'project participation into theirschool schedules.

The project did not receiveenough referrals, to constitutetreatment and control groups asplanned.

Self-Contained Class in Pennsauken

At Pennsauken Junior High School,the EIC-South Alternative EducationProject also operates a Self-Con-tained Class. Students assigned tothis class stay in the same class-room for the entire day. The Penn-sani,en Self-Contained Class was inoperation several years prior to theEIC-South Alternative Education Pro-ject and was incorporated -into- theProject when it began.

An alternative-Education Projectstaff member is assigned full timeto the self-contained class. He

conducts the School/Community Rela-tions curriculum daily (i.e., commu-nication skills, self-image, familyprobems, study skills, and careerexploration), teaches history, andsupervises the students' study per-iod. In addition to the EIC-staffmember, three staff teachers, spe-cially selected for their ability todeal with "problem" students, visiteach day to give instruction in

mathematics, science, and English.

The same criteria as for theSchool/Community Relations classeswere used to select students for theSelf-Contained Class except thatcertain students were judged to bemore likely to benefit from theSelf-Contained Class. The criteriaunderlying these judgments are notspecified.

360

New Jersey

Community Process Component

Three Community Process Consul-tants were employed by EIC-Southduring the 1981-82 school year. Oneworker was, assigned to Vineland, oneto Pennsauken, and one to both Buenaand Pleasantville, since they are inthe same county. Apparently, onlyin Vineland and Pennsauken was thecomponent fully implemented to theextent that students actually parti-cipated (see Table 3).

The Community Process Consultantswere charged with primary responsi-bility for coordinating in-servicetraining for all teachers in thetarget districts, covering such top-ics as violence and vandalism, dis-pute resolution, parenting, teachereffectiveness, counseling services,the criminal justice system and dis-cipline. Another major function ofthe Community Process Consultantswas to call the homes of projecttarget students who were absent.They also provided counseling ser-vices for a small number of stu-dents.

The Community Process Consultantshad key responsibility for organiz-ing large committees of communityfigures and school personnel. Somestudents served on these committees(See Table 3). The purpose of thesemeetings was to attempt to put intoaction the Project Director's theoryabout how institutional changeoccurs (See Appendix A).

Generally, large meetings werecalled where key community memberswould discuss community problems andtheir causes. This process, in andof itself, was considered to betherapeutic. A hoped for outcomewas that people with power in thecommunities would become more awareof the problems that youth experi-ence and then do something aboutthem. The actual outcomes of thesemeetings are, by and large, unknown,

-474-

although the project director per-ceives that awareness was increased.

Project Process

We do not know as much about theimplementation of this project as wewould like. The ad hoc nature ofprogram operations makes it diffi-cult to monitor implementation. Ourearly attempts to get project manag-ers to be more explicit about whatthey planned to do and why may haveserved to alienate them. Much of

the early numeric feedback relayedto project personnel was seen asabstract and, therefore useless.Put another way, both the spontane-ous nature of the project and dif-ferences in the language -nd per-spectives of implementers andevaluators have sometimes made it

difficult for us to engage in usefulprocess evaluation.

Interviews with key project staffindicate that the, project experi-enced no substantial opposition dur-i- its first year of operation.taft r t that key school system

personnel have become more receptivc,to the project over time.' Thisaccords with the comments made byprincipals in the 1981-82 PrincipalQuestioillaires. But by and large,the EIC-louth Alternative EducationProject is a "black box." At theend o the 1981-82 program year itwas evident that the PDE

Jcess was not being used by pl.:-:-

ject personnel in the spirit that. itwas intended, and therefore was cotproviding useful information ah,:.?ut

project plans and implementation.

Fortunately, there is ri,,,s(%1 to

believe that by the end of the1982-83 school year, the piojectwill be less of a black box be.:ause

the project developed a' new planaccording to its own model, andmotivation to document implementa-tion is now higher. A clear

361

. u

workplan with milestones has beenworked out. Also, the project hasreworked its theory into a guide foraction that personnel see as waningful, and reduced its list ofgoals and objectives. Finally, a

fulltime Evaluation Coordinator washired by the project.

Interim Outcome Evaluation

Design Evolution

After earlier plans to conduct atrue experiment were abandoned (seethe first Interim Report), new planswere made in August, 1981 for a qdasiexperiment. The new designcalled for identification of nonequivalent control groups in each ofthe target schools. Also, at thesuggestion of the Project Director,control schools were to be chosenfor each of the four target schools,matched on demographic characteristics (i.e., SES, Race, school size,average achievement test scores andschool grade structure). A collegestudent was hired parttime to dothe work involved with collectingthe data for matching schools.Within each control school referralswere to have been solicited for seventh and eighth graders based on thesame nine criteria as in the targetschools.

Because of the nonequivalentnature of comparison groups the

design depended on the administration of a pretest survey in thefall of '81 to all treatment andcontrol students, the controls in

the target schools as well as in thecontrol schools (see Table 5).

By late October, 1981, the project was not ready to administer apretest. With the assistance of

475

New Jersey

the OJJDP project officer and a member of the Polaris (Technical Assistance) staff, a meeting was held atthe OJJDP office in Washington todiscuss the importance of theevaluation and obstacles to proceeding. The project director did notteel that the evaluation would beuseful and had reservations aboutthe appropriateness of collectinginformation about individuals in a

governmentsponsored demonstrationproject. The meeting did not resultin an immediate solution, but in

January, 1982, a full time Evaluation Coordinator was hired by theproject. In the weeks following themeeting and during a site visit made'to EIC`youth in January 1982, newarrangements for evaluating the project were made. The idea of "control" schools was abandoned as

/Inf0asible. Again, the agreed upondesign was a nonequivalent controlgroup comparison. Students who werein treatment as of -January-30-, -1982were to comprise the treatmentgroup. All other students on thewaiting list and new referrals wouldbe in the comparison group. Thisdesign, of course, depended on theprogram having reached a stablestage whereby new referrals andwaiting list students would notenter treatment during the year. In

lieu of a survey pretest, the project was to collect pretreatmentdata from records from the 1980-81school year for all treatment andcontrol students.

The expectation that treatmentstudents would remain. in treatmentand control students would remainuntreated was not met. By March,1982, virtually all students in thecontrol group had received some typeof service (see Table 6).

362 PO.

New Jersey

Relationship Between Known Back-ground Characteristics of TargetStudents and Amount of VariousTreatments Received

Because Fall, 1981, pre-testswere not administered as initiallyplanned, there were few pre-treat-ment data available for a thoroughexploration of the relationship bet-ween background characteristics andtreatment.

Several individual items and sca-les from the 1982 Spring StudentQuestionnaire can be viewed as mea-sures of background characteristics,for instance, the Parental Educationscale. Correlations, between thisvariable and the amount of treatmentreceived were non-significant. Thecorrelation between student self-re-ported sex (coded 0=female, 1=male)and the number of quarters in a spe-cial club was significant (r=-.21;p<.05). The correlations betweenstudents' self-reported age andnumber of quarters in community pro-cess (r=.22; p<.01), counseling(r=.18; p<.05), and self-containedclass (r=.48; p<.001) were signifi-cant, indicating that the older tar-get students tended to receive thesetreatments. Self-reported gradelevel was significantly positivelycorrelated with the number of quar-ters in the self-contained class(r=.40; p<.001) and tutoring (r=.19;p<.05). Self-reported race (coded1=white, 0=a11 others) was corre-lated significantly with the numberof quarters in tutoring (r=-.22;p<.05), school/community relationsclass (r=.19; p<.05), community pro-cess (r=.47; p<.001), counseling(r=.28; p<.01), and self-containedclass (r=.35; p<.001). The implica-tion is that the students partici-pating longest in school/communityrelations class counseling and theself-contained class tended to be

white, whereas students receivingtutoring tended to be non-white.

-476-

Several additional measures of

background characteristics wereavailable from school records forthe previous (1980-81) school year.These were: number of absences dur-ing the 1980-81 school year and num-ber of grade retentions in the stu-dent's history. Data on 1980-81suspensions were provided but weremissing for most students. Indivi-dual subject grades for the 1980-81school year were provided but werenot used because, for two schools,many of the correlations betweenindividual 1980-81 grades wereeither peculiarly low or negative(see Table 7) and correlations bet-ween 1980-81 and 1981-82 overall

4GPA's were low for Vineland (r =

.02) and negative for Pennsauken (r= -.19). This pattern of correla-tions suggests that the data mayhave been incorrectly transcribed.

Number of absences during the1980-81 school year was signifi-cantly positively correlated withcommunity process participation(r=.18; p<.05), counseling (r=.16;p<.05), participation in special

clubs (r=.23; p<.01), and participa-tion in the self-contained class(r=.28; p<.01).

The number of retentions in stu-dents' histories was significantlypositively correlated with communityprocess participation (r=.24;p<.01), participation in specialclubs (r=.18; p<.05), and participa-tion in the self-contained class(r=.34; p<.001). These correlationsare summarized in Table 8.

Implications of these correla-tions. The correlations justdescribed imply that students parti-cipating in the various program com-ponents are different in importantways from students who did not par-ticipate--i.e., the comparison groupis indeed non-equivalent. It willtherefore be necessary to attempt toadjust statistically for these

363

pre-existing differences in search-ing for the effects of participationin the various project components.

Analytic Methods

Because, by the end of the1981-82 school year, there were vir-tually no control students, a

regression approach to data analysiswas applied, using number of quar-ters in various treatments as pred-ictors in separate regression equa-tions. All target students in

schools where treatments were imple-mented were included in each analy-sis, but if a treatment was riot

implemented in a school, that schoolwas not included in the analysis.The project provided data on thenumber of quarters that students inthe target group spent in each typeof treatment. Number of quarters ineach treatment was coded from 0 to 4in increments of .25 quarters. Withthis coding the unstandardizedregression coefficient (slope) inthe regression equation can beinterpreted as the value on the out-come variable associated (i.e.added or subtracted) with one schoolquarter in treatment.

Measures from School Records

Results for a variety of measuresobtained from school records areshown in Tables 9 through 12.Because students participating invarious project components differfrom non-participants on a varietyof pre-intervention characterittics,it may be misleading to interpretsimple correlational results aseffects of participation. Theresults shown in Tables 9 through 12are results of simple correlationalanalyses of this kind.

The association between number ofquarters in cross-age tutoring andGPA and the association betweenself-contained class and GPA are

-477-

New/ Jersey

both statistically significant.These results should be interpretedwith caution because pre-treatmentGPA data were not available in theschools where these interventionswere implemented. Were dependablepie-treatment data available, theywould be used as statistical con-trols to probe the plausiblehypothesis that this correlation isdue to pre-existing differences bet-ween participants and non-partici-pants.

Measures from the Spring 1982 Admin-istration of the SAES Student Survey

The Student Questionnaire wasadministered anonymously to 7th and8th graders in two of the targetschools (Pleasantville and Buena),7th graders in Vineland, and a ran-dom sample of 7th, 8th, and 9thgraders in Pennsauken. Responserates and school level results arereported in-the oirerVieircif-the-SAESsecond interim report.

Questionnaire booklets for targetstudents were identified with a con-fidential survey number necessaryfor merging these survey data withinformation about participation inproject components so that regres-sion analyses could be performed.Unfortunately, students who were nolonger in treatment at the time ofsurvey administration were not giventhe assigned booklets; there weremany missing data. Fifty (31%) ofthe target students were not admin-istered their assigned booklets. In

addition, item non-response (a fai-lure of students to answer many ofthe questions in the booklets) madeit possible to score only about halfthe scales, for each target student(sample mean=.49). Biases of anunknown nature may have resultedfrom this attrition and non-re-sponse, so the survey data are ofmuch diminished value for 1982.

1P4

New Jersey

Because there were few studentsparticipating in either specialclubs or cross -age, tutoring andbecause of the previously mentionedmissing data problem, student ques-tionnaire outcomes were not analyzedseparately for these groups. Thesame goes for the Pennsauken Self-Contained Class.

The same regression approach thatwas described for the data fromrecords outcomes was applied here.In cases where the association bet-ween a type of treatment and an out-come was significant, a multipleregression approach, utilizing step-wise regression was applied.2 Thisprocedure yielded an estimatedregression coefficient (labeled "b"in the tables) that is adjusted forthe contribution of other variables.Of the adjusted associations betweentreatment and outcome, only twoapparently negative effects (theassociation between number of quar-ters in counseling and the score on

2. The SPSS (Nie, Hull, Jenkins,Steinbrenner, & Bent, 1975) stepwiseregression procedure with listwisedeletion of cases with missing datawas used. Stepwise regressionessentially allows the computer to

build a parsimonious statisticalmodel of the criterion variable. In

all cases, the computer run was setup so that it was possible for thefollowing variables to enter intothe model: race, grade level, age,school, number of quarters in coun-seling, community process, tutoring,special club and self-containedclass. In the current situation,the purpose for performing the step-wise regression was to determine thevalue of an estimated "adjusted"regression coefficient and to testits significance. Parameters in theregression procedure were chosen toset the tolerance at .5 and the p toenter at .05 (see Nie et al., 1975).

-478-

the Rewards Index, and the associa-tion between participation in theSchool-Community Relations Class andInvolvement) remained. See Table13.

An obvious interpretation of the

analysis of Student Questionnaireoutcomes would be that. the programdid not have negative effects but

neither did it help. The problemwith such a conclusion is that thestudents who had left treatment atthe time of survey administrationwere not administered an identifia-ble survey. This could mean that weessentially had no measurement of

the project's successes. (Alterna-,tively it could mean we have no mea-sure of the project's failures!) Wewill have to wait until next year tobe able to say anything definitiveabout program effects on outcomesmeasured by the Student Question-naire.

The Evaluation Design for 1982-83

Finally, a non-equivalent controlgroup design with appropriate num-bers of treated and untreated stu-dents has been agreed upon andimplemented. The following numberof treatment and control studentswere chosen at the beginning of the1982-83 school year:

Treatment ControlVineland 35 46Pennsauken 30 36

Pleasantville 38 39Buena 34 35

A Student Questionnaire pre-test wasadministered to all in October, 1982and the response rate was virtually100%. A variety of pre-treatmentdata from records have been providedfor two pre-treatment years (i.e.1980-81; 1981-82). These datainclude standardized achievementtest scores, the Tennessee Self-Con-cept Scale scores, number of timessuspended, GPA, attendance, promo-

365

tion record, age, race, grade level,and sex. The data appear on initialinspection to be of high quality andcontain few missing values.

The on-site Evaluation Coordina-tor, now a veteran of the difficultposition she took on, has made plansto administer the Spring 1383 Stu-dent Survey in a more careful man-

-479-

New Jersey

ner.

The marginal results of the eval-uation of 1981-82 outcomes drovehome the point about the need for amore intense level of interventionand a much stronger evaluationdesign. Students in the 1982-83cohort are staying in treatment muchlonger (most for the entire year).

366

New Jersey

References

Gottfredson, G. D., Rickert, D. Gottfredson,'D. C., & Advani,' N. Stan-dards for program development evaluation (Report No. 341). Baltimore: TheJohns Hopkins'University, Center for Social Organization of Schools, 1983.

Rickert, D. E., Jr. The Educational Improvement Center-lUuth's alternativeeducation project. In G. D. Gottfredson (ed.), The School Action Effec-tiveness Study: First interim report (Report No. 325). Baltimore: TheJohns Hopkins Univeristy, Center for Social Organization of Schools, 1982.

-480-

367

Table 1

The Original PDE Objectives of the EIC-South Alternative Education Projectand Actual Measures of Achievement Employed

Objective Measures Employed

Goal 1: Reduce (Prevent) Delinquency

1. Raised self-esteem of students in Program classes

2. Decreased negative peer influence

3. More mature attitude toward drugs and persons

4. More respect for public property (schools) and persons

SAES Student Questionnaire Postive Self Concept scale

SAES Student Questionnaire Negative Peer Influence scale

5. Awareness and "buying into" school rules by students and Awareness possibly measured by Clarity of Rules scale

teachers

Goal 2: Academic Success

1. Improved teacher attitudes with regard to student's aca-demic success

2. Improved physical environment

368

r 9C

es "2114 11 44 Yllrlf ,r, =1./177.17. , tt . , ;

I solimmist poittomplatpes mweseil444 er4V4 t1,10

'111+414941 1141,1 14614 1011 iteopp.14 pi 40.000P1*,044.1

WI* ems 14011,9evi mol mi1momou040 i040wilk 4411v,

sovto *.k 44.11111104 541.441 0404 4,1140*

0,40, yowl .4 1..414 **V.. ROA,- .4 *AU

,t,444,44, 4 po 4040. .4.40444

0. lt,..04 44 0,4 44.11.4} $4-1 0.440 V+ 4

% ..... 4,11%1[1,11T t I' r it " '1 :t ,- 101 t 44 1444.1* tsvoto

0.4441 44- 4 ,4,/, tt47141 ,44

'44t444!, .4 '4,4+ 40 0444-w0,0.

0Am#611m, 4+44 mA11,104 44911I.41t1t."0 t44f4i0 16,0* ..11,40 4+40444 0444.!.,'44

+40.1,.44 444«. ,.44, 0.. ot,i4r14 ,,f.jt 4

*44 440.40 14 400 p 4 9.4 44

04. NI A? ,

ow!' fitomw:t.11$11 ditilmootIkok4 14.vAt* *.t* .4,4 4 44-.4., 4 ja OW #r

ifirt*0 4 relp 4-1.4044 1.44.0,.44

151s ... XIS 32 Inc .1. !

ftW0414 1 .144*

Table 1 (Cont.)

Coal 5: Reduce (Prevent) Suspensions, Expulsions, Dropouts- ---

I. School district openness to re-evaluation and changing Judgment based on examination of policiesvolicies about suspensions and expulciens

School provide creative and diversified additionalresources for dealing with behaviors that would normallyresult in suspensions (outside of the school resources).:lomeone in the school will coordinate the requests forand delivery of these services.

3. Change in teacher and administrator attitudes towardusing suspension as a method of reducing disruption

Goal 5: With Regard to Dropout Specifically

4. Increase academic success - School grades- Standard achievement test scores- Self-report grades from SAES Student Questionnaire- Self-report reading ability from SAES Student Ques

noire

5. Increase self-esteem SAES Student Questionnaire Positive Self-Concept scale

Coal 6: Effect Institutional Change

l. Interagency cooperation

2, Comrunity input into planning

3. Youth input into planning and implementation

4. Standardization across agenciPR of Arnndards ef conduct

5. Reduce youth alienation and apathy

6. Positive community attitudes towards the AlternativeEducatiun Project

Documentation that representatives from various age!

attend community process meetings and with what freque

Documentation that representatives from various age'

attend community process meetings and with what freque

SAES Student Questionnaire Alienation scale

3 0

Table 2Revised Goals and Objectives of the EIC-South Alternative

Education Project (Summer, 1982)

Goal 1: To reduce/prevent delinquent acts by 10% for treatment group.

1. To limit disciplinary referrals for classroom disruption by teachers.

2. Improve/maintain attendance.

3. Decrease out-of-school suspensions in Pleasantville by 10%; maintainlevels in the other three districts.

4. Increase positive peer influence.

5. Establish an on-going process/structure to develop community stan-dards, resource use, problem solving, involving youth and other seg-ments of the community.

6. Establish/increase opportunities for summer youth employment.

Goal 2: To contribute to incxaased academic success.

1. Improve/maintain attendance.

2. Improve the study skills of project students.

3. Maintain high level of promoteion to next grade level.

Goal 3: To bring about the adoption of proven alternative education programsand policies.

1. To simplify school discipline policies to ensure comprehension byyouth and parents.

2. To expand and install concept of peer tutoring.

3. To achieve adoption of one or more units of the Alternative EducationCurriculum Guide by the participant districts.

4. To ensure effective involvement of youth.

5. To increase visibility of the New jersey Alternative Education Proj-ect.

6. To increase the TA capacity for the State Board of Education in thearea of alternative education and school climate.

-484-

37.

Table 3

Number of Students Receiving Each Type of Treatment by the EIC--South

Alternative Education Project -- Broken Down by School and Quarter

School

Treatment

Vineland Pennsauken Pleasantville Buena

Qtr Qtr Qtr Qtr Entire

1 2 3 4 year

Qtr Qtr Qtr Qtr Entire

1 2 3 4 year

Qtr Qtr Qtr Qtr Entire

1 2 3 4 year

Qtr Qtr Qtr Qtr Entire

1 2 3 4 year

Alternative ed class 8 11 22 15 23 20 19 30 31 37 15 15 33 26 39 7 8 12 11 14

Community process 0 7 0 0 7 5 0 34 30 31) 0 0 0 0 0 0 0 0 0

InIndividual counseling 5 9 18 12 25 14 13 34 31 37 0 0 18 12 23 8 9 24 22 26

' I

Special club 0 0 0 0 0 4 5 2 5 8 0 0 4 1 1 1 1 0 2 3 3

Cross-age tutoring 3 4 2 4 6 0 0 5 6 6 0 0 0 1 1 0 0 0 0 0

Self-contained class 16 16 14 14 18 --

Tutoring 0 0 0 7 7 0 0 0 1 1 0 0 0 7 7 0 0 0 0 0

Any treatment34 42 51 :1.6

No treatment0 1 2 1

1.

372 373

Table 4

Average Number of Quarters that EIC-South AlternativeEducation Project Students Spent In Each Type of Treatment --

1981 -82 School Year -- Broken Down by School

Treatment

Entire Project Vineland

N

Penns:.:: Pleasantville

SD N M SD AI SD SD,

Alternative educationclass

1.68 1.40 159 1.43 1.46 34 2.31 1.37 43 !..57 1.16 53 1.7.P- 1.56

Community process .47 .82 159 .20 .41 34 1.58 .83 43 0 ., 0 53 0 0

Individual counseling 1.39 1.28 159 1.18 1.14 34 2.10 1.28 43 .57 .72 53 2.07 1.32Special club .14 .50 159 0 0 34 .37 .82 43 .03 .21 .53 .14 .42

Cross-age tutoring .16 .58 159 .38 .95 34 .26 I .66 43 .02 .14 53 0 0

Self-contained class .36 1.08 159 0 0 .34 1.34 :1.75 43 0 0 53 0 0Tutoring .09 .29 159 .20 .41 34 .02 1 .15 43 .13 .34 53 0 0

a

374

Table 5

Representation of Non-Equivalent ControlGroups Design Agreed to In .ugust, 1981

PopulationTreatment November AprilStarts Pretest Posttest

4 target schools' September Yes Yes7th graders

4 target schools' No Yes Yes8th graders

4 Control Schools' No 4Yes Yes

7th graders

4 Control Schools' No Yes Yes8th graders

Table 6

Numbers of Treatment and Untreated ComparisonStudents at Two Points in the 1981-82 School Year

Total Number WhoHad Received Any

Treatment

Number ofComparison Students

School

Through Through

Jan '82 Mar '82 Jan '82 Mar '82

Vineland 18 33 16 1

Pennsauken 38 38 4 5

Pleasantville 19 50 34 3

Buena 16 26 4 10 0

376

Table 7

Zero Order Correlations BetweenGrades in Individual Courses for th. 1980 - 81

English

School Year - Vineland

Math Reading Social Studies

English

Math

Reading

Science

Social Studies

1.00 .53

1.00

.76

.06

1.00

.12

.09

-.26

1.00

.01

.10

-.22

.49

1.00

377

..

Table 8

Zero Order Correlations Between Background

Characteristics and Amount of Various Treatments

Received - EIC South Alternative Education Projects

0

Treatment

Days Absent

1980 - 81

Retentions In Age

Students History

Race Grade

Level

Parental

Education

Tutoring

School/Comm

Relations

Community

Process

Counseling

Special

Club

X-age tutoring

Self-contained

Class

-.10

(131)

.12

(131)

.18

(131)

.16

(131)

.23

(131)

-.02

(131)

.28

(131)

.04

(151)

-.03

(151)

.24

(151)

.00

(151)

.18

(151)

.05

(151)

.34

(151)

-.01

(109)

.09

(109)

.23

(109),

1(1089)

1(1049)

(10079)11

.481

(109)

-.22

(90)

.19

(90)

.47

(90)

.28'

(90)

-.02

(90)

.13

(90)

.35

(90)

.19

(95)

-.00

(45)

.13

(95)

-.10

(95)

.05

(95)

.00

(95)

.40

(95)

-.21

(40)

-.30

(40)

-.22

(40)

-.04

(40)

,-.26

(40)

-.05

(40)

-.33

1(40)

Note: Numbers in parentheses are n's. The n's vary duel to differential degrees of missing dat .

376379

Table 9

Regression of Outcome Measures on Degree ofParticipation in Treatment

Components Operating in All Four EICSchools

Quarters in

Days of .nonattendance

GPA on a1-5 scale

Number of timessuspended

b r2

r2

r2

Schoolcommunityrelations class

Counseling

1.36 .01 143 .07 .02 154 .32 .02 142

1.27 .01 143 .04 -.00 - 154 --.02 .00 - 142

Note. Each b is the unstandardized regression coefficient in a separate equationregressing the outcome var:able on the treatment variable. None of these regressions is significant.

Table 10

Regression of Outcome Measures on Degree ofParticipation' in Treatment Components

Operating Only in Vineland and Pennsauken

Quarters in

Days of non- GPA on a Number of timesattendance 1-5 scale suspended

,..,

b r2

r2

r2

r

Community process -1.49 .01 71 .08 .01 75 -.04 .01 73

Cross-age tutoring -4.20 .04 71 .23* .06 75 .00 .00 73

Note. Each b is the unstandardized regression coefficient in a separate equationregressing the outcome variable on the treatment variable.

* < 05p . .

Table 11

Regression of Outcome Measures on Degree ofParticipation in Self-Contained

Class Operating Only in Pennsauken

Quarters in

Days ofnon- GPA on aattendance 1-5 scale

Number of timessuspended

b r2 N b r2 N b r2

Self-containedclass

-1.63 .02 39 .20** .19 41 -.04 .03 39

.

Note. Each b is the unstandardized regression coefficient in a separate equationregressing the outcome variable on the treatment variable.

**p<.01.

382-493-

Table 12

Regression of Outcome Measures on Deg_ce of Participation in Special ClubsOperating Only in Pennsauken, Pleasantville and Buena

Quarters in

Days of non-attendance

GPA on a1-5 scale

Number of timessuspended

b r2

r2

r2

Clubs .91 .00 111 .04 .00 120 -.48 .01 1081.8

Note. Each b is the unstandardized regression coefficient in a separate equationregressing the outcome variable on the treatment variable.

383

-494-

Table 13

Regression of 1982 Student Questionnaire Scaleand Selected Item Scores on Degree of Part-icipation in Various Treatment Activities--

Ney Jersey

Quarters in Alt. Quarters in Com-Education Class munitv Process

Quarters inCounseling

raw :1Tdiusted raw adjusted raw adjusted

b N b N b N' b N b N b

Parental emphasis oneducation -.08 40 -.10 11 -.01 40 --

Attachment to parents -.06* 104 -.05 86 -.22** 47 -.05 86 -.10** 104 -.30

Negative peer influence .00 97 .18** 46 .04 53 .n3 97 --

Attachment to school .00 65 -.09 21 -.04 SO --

Belief scale scored .

positively -.03 34 -- -.16 21 -.04 34 --

Interpersonal compe-tency .01 34 -- -.33** 9 -.16 21 -.01 34 --

Positive self-cbncept .03 28 -- .02 6 -- -.00 28 --

Self- reported delin-

quency .01 88 -- ?20** 42 -.03 71 -.03 88

Self-reported druguse .04 88 .34** 45 -.02 62 .06 88 --

Serious delinquency -.01 91 .13* 45 .03 74 .01 91 --

Punishment index -.03 75 -.03 27 -.03 75 --

.rewards index -.07* 75 -.01 54 -.04 27 -- -.10** 75 -.10** 54

Victimization -.04 69 -.09 24 -.06* 69 -.03 54

Invalidity scale -.04 34 -.05 8 -.06 33 --_ _

School effort -.02 102 -.01 48 -.01 102 --

Practical Rnowledge .11 31 -.25 8 -.05 31 --

Internal- external

control .06 34 .:08 8 .02 34 --

Alienation -.01 34 -.07 8 .00 34 --

Self- reported grades -.03 107 -.02 48 -.01 107 --

Reading ability self-rating .11 99 -- .09 45 .04 99 --

Days of school cut lastfour weeks -.06 102 -.01 47 .05 102 --

How often cut one ormore classes -.06 108 .47 48 .18 108 --

Educational expectations .14 105 .43 46 -- .06 105 --

Did student work forpay last week? -.01 106 -- .02 48 -.03 106 --

Regular part-time orfull-time job -.06 104 -- .07 46 -- -.03 104 --

Suspended from schoolthis term -.02 56 -- .10 19 .:\ -.03 56 --

School nonattendanceindex

Rebellious autonomy

Involvement

-.05 102 -- .28 47 -- .08 102 --

.02 32 -- .34* 8 .18 31 .09 32 --

-.07** 98 -.03** 78 -.15** 47 -.05 78 -.06** 98 -.01 78

20

Note. Figures in the columns entitled "raw" are unstandardized regression coefficient,: forsimple regressions of each outcome variable on each of the Independent variables (i.e. Quar-ters in Alt. Ed. class, Quarters in Community Process, and Quarters in Counseling).Numbers in "adjusted" columns are unstandardized regression coefficients from multipleregressions. where, in addition to the independent variable of interest (i.e. the treatment)control variables were introduced into the equation. The purpose of such a procecI.-!.

Is to attempt CO control for the effects of pre-existing characteristics of the students.

The control variables utilized were the following: grade level, race, age, school ofattendance, participation in other treatments (i.e. counseling, class, tutoring, clubs,community process). Of these, only those whose contribution to a regression equationwas statisticaliy significant (i.e. p .05) with a tolerance of .5 were utilized.

<.05

**p <.01 -495-384

Appendix A: Guiding Principles of the EIC-South Alternative EducationProject

Working Theory of Delinquency

Delinquency is a means of expression both in a protesting sense andas a means of getting attention (youths don't know how to get around inthe system in a conventional way). Many things contribute to it.

1. Self-esteem--"Delinquent acts are the only thing I can successfullyaccomplish. This is the only thing I can be proud of."

2. Family relations--no communication, guidance, supervision, and youthsraise themselves. Family does not view itself as a means for provid-ing ,urnort for the total growth of the youths (provide food, shel-ter, and clothes, that's all) and in some cases is not a cohesiveunit. Negative contacts with community institutions ---> powerless-ness to do anything about things other than food, shelter, and cloth-ing.

3. Boredom--Youths aren't challenged in school. Curricula is irrele-vant. Youth activities provided by the community are irrelevant andthey are "provided" instead of decided and managed by the youths.This is a great and easy cop-out for the youths. They don't know howto get around in the conventional system and don't find out how to;thus they cop-out. Lack of programs to participate in. Transporta-tion is an obstacle. Parents are usually uncreative in thinking upactivities to sustain the family unit.

4. Negative peer and adult contacts--peers and some adults positivelyresponding to delinquency which creates more delinquency.

5. Poverty--no money which leads to delinquent acts which are gratify-ing. Youths don't know other ways of getting money.

6. Drugs and alcohol--to get these youths need money which means commit-ing delinquent acts to get it. Once they have the drugs and alcohol,youths do crazy (delinquent) things. Drugs and alcohol are delin-quent, and the youths have immature attitudes toward their use.These contribute to academic failure because youths are spaced out inschool and not responsive to any learning which may take place.

7. Depersonalized victims

This appendix is drawn from material prepared by project personnel incompleting the PDE plan. It is reproduced here with only very minorediting.

-496-

Youths don't have a sense of ownership of public property and donot feel a need for preserving the property

don't know their "neighbors" so hurting them or their property isnot personalized

8. Lack of jobs leads to no money which youths want (they don't knowhow to participate in a conventional sense). Transportation issometimes an obstacle to employment.

9. Lack of uniform standards of conduct and the application of thosestandards (vague school rules, without community input).

inconsistency in rules and the consequences for breaking a rulebetween institutions.

double standards (youths can't smoke but adults can)

youths as consumers of these institutions, do not have input inthe decisionmaking which governs them

Youths experience academic failure-because:

1. Youths enter school with low selfesteem

2. Schools are not responsive to individual needs:

a. Youths are unable to fit into the system due to measurement (usedas a means of tracking, labeling, not as diagnostic tools) andcultural differences which lead to dissaffected youth not "buyinginto" the system.

b. Youths don't feel cared for because of teachers, principals andschool policies.

3. Students have no input, particularly in curriculum, (thus it is boring), discipline, and policy making. Youths end up not caring

4. Teacher attitudes: (hopelessness, cynicism, etc.)

a. Teachers don't expect youths to achieve which leads to youths notbeing challenged. In many cases, the potential is there, but nomotivation to achieve.

b. Lack of communication between teachers and adminstration directlyaffects teacher ownership (they do not feel they have ownership).

5. Parental contacts with school are for negative reasons and "theydon't want to hear it Parents aren't supported by the school orencouraged to support their children.

-497-

386

6. Youths not feeling slit. by teachers and parents leads to teachers not feeling support,. Principals, superintendents, board, community, and student's pa leads to principals not feeling supported by superiors, boart, munity, teachers (depending on theschool), and parents (deper on the prinicpal) leads to superintendents not feeling suppol by: board, community leads to boardsnot feeling supported by community (superiors) leads to community notgetting involved.

The above problems die a slow and painful death and there is noresolution.

Principals

a. Are responsible for insuring that youths come to school.

b. Carry out the tracking, and labeling procedures because theyimpact scheduling.

c. Approve the curriculum/development implementation and canchange the whole curriculum.

d. Strongly influence budgets which don't end up buying neededbooks, sponsoring field trips, and providing custodial carefor afterschool activities.

e. Are responsible for teacher placement. ("Good ones" end up in"bad" classes); consequently, teachers don't want to performwell because there are limited rewards, if any.

f. Carry out disciplinary responses sometimes inconsistently dueto others' influences (board, superintendent, politicians).

g. Aie responsible for insuring that youths come to school, butthe various responsibilities deem that they leave coming toschool up to youths and parents.

h. Designate responsibility to the guidance departments and thelocal court (15 dollar fines for not sending youths toschool). Guidance department are busy with "successful"youths and pass the problems on to truant officers.

7. Boards of Education superintendents (and community) are not planningeducational goals and plans, budgeting for the whole district, andthe small minority of "bad youths" get lost in the plans.

8. Classified students

Youths who are legally classified present another significant problem in the theory of student academic failures. These youths arevery often scapegoats for an education system which uses labelingand tracking techniques to rid itself of these students who havenot conformed to the schools social and educational "norms."

498

387

Teachers are not trained to adequately deal with the multifacetedhandicaps of the classified student. Insufficient teacher training and limited experiences with this type of student lendsitself to low expectations from teachers and no motivation on thestudents' part.

Most frequently used levels of classification for the specialeducation student-

-neurologically impairedperceptually impairedemotionally disturbedsocially maladjusted

Theory: Violence and Vandalism

1. Selfesteem--"Delinquent acts are the only thing I can successfullyaccomplish. This is the only thing I can be proud of."

2. Boredom--Youths aren't challenged in school.Curricula is irrelevant.Youth activities provided by the community are irrelevant and theyare "provided" insteac of decided and managed by the youths.This is a great and easy copout for the youths.They don't know how to get around in the conventional system anddon't find out how to; thus-they copout.Lack of programs to participate in.Transportation obstacles.Parents not being creative in thinking up things to do or makingyouths responsible in the home.

3. Drugs and alcohol--to get these-youths need money which leads to delinquent acts to get it. Once they have the drugs and alcohol, theydo crazy (I1elinquent) things. Drugs and alcohol are delinquent, andthe youthikhave immature attitudes toward their use. These contribute to academic failure because youths are spaced out in school.

4. Depersonalized victims

Youths do not have a sense of ownership of public property and nostake in preserving the property

5. Negative peer and adult contacts--peers and some adults positivelyrespond to delinquency which leads to more delinquenct acts.

6. Lack of uniform standards of conduct and the application of thosestandards (vague school rules, without community input). Inconsistency in rules and the response to breaking a rule between institutions and double standards (youths can't smoke but adults can).Youths, as consumers of these institutions, do not have input intothe rules which govern them.

7. Gap not bridged: transferring the idea of private ownership to publicproperty so that youths feel as if they own it too. This leads to a

-499-

need for youths to become involved in and responsible for ownershipas a constituency of the community in partnership with others, spe-cifically peer and adults.

8. Rage--"I'm not worth anything!" These are often expressions of angerat not being recognized. These expressions may lead to attacks onteachers (and others) who become personalized victims of the rage.Many demeaning things have accumulated over time and culminate withthe expression of rage, hostility.

Theory: Absenteeism

1. School becomes a negative place to be.

a. The differences between youths (skin coloi, dress & other) areapparent to .youths/teachers. and some youths get a lot of negativeresponses.

b. Language barriers between: Hispanics & English, Black dialect &English.

c. Problem with curriculum and achievement leads to poor testingskills which leads to being "labeled."

d. Parents transfer their negative feelings about-the school- (see-academic failure, No. 5) to their youths.

e. Boredom--Youths are not challenged in school. Curricula isirrelevant. Youth activities provided by the community areirrelevant and they are "provided" instead of decided and managedby the youths. This is a great and easy cop-out for the youths.They don't know how to get around in the conventional system anddon't find out how to; thus they cop-out. Lack of programs toparticipate in. Transportation obstacles. Parents not beingcreative in thinking up things to do.

f. Students have no input, particularly in curriculum (thus it isboring), discipline, and policy making which leads to youths notcaring.

g. Teacher attitudes: (hopelessness, cynicism, etc.)

1) Teachers do not expect youths to achieve which leads to youthsnot being challenged and boredom becomes commonplace!

2) Lack of communication between teachers and administrationdirectly affects teacher ownership (they don't feel they haveownership too).

2. Due to family structure (responsibility for babysitting while parentswork), youths have to work (migrant workers). Migration of thefamily interferes with going to school. Education is not valued.

3. Social Success

5007

389

a. Youths who go to school and are not able to communicate with students, teachers, parents (no interaction between them) but whowant social success, often find it with others who aren't inschool.

b. Can't handle all that's thrown on them (not adults yet). Moonlighting in jobs and can't be succesful in both school and work.The job wins out (its more rewarding). They want immediate gratification so they get jobs and school is not an immediate gratification.

Theory: Suspensions, Expulsions, Dropouts:

Expulsions don't happen very often because it is a difficult process(due process requirements). Expulsions only happen in very clearcut andserious ci.rcumstances. Intervening at an earlier stage prevents expulsions.

Suspensions occur because they are a very easy way of coping (gettingrid of) with disruptive students. Teachers pressure administrators tosuspend disruptive students. They don't know what to do with youths.Suspensions, as a consequence (punishment), are based on the assumptionthat the youths want to be in school in the first place. In reality,being suspended can be a reward to some youths.

Guidance counselors work loads don't allow the time to deliver counseling services that disruptive youths need.

Dropouts (pushouts?)

Youths drop out because:

1. They are not academically successful

2. They have low selfesteem and no sense of worthiness

3. They want to earn money

4. Of boredom

5. Of family relatives, who appear to be "making it."

6. Of drugs and alcohol/problems.

OR disruptive youths are counseled out the door. Which means thatsome students are not encouraged to perform at any level.

They dont' fit in any program and therefore, in subtle ways; arepushed out the door.

501

Theory: Effect Institutional ChangeIn view of perceptions of fact and the stated problem definition, thefollowing hypotheses have been drawn:

I. Academic success (positive experiences at various levels of abil-ity), attendance rate, suspension, expulsion, dropouts, truancyare not subject to impact by schools alone.

2. The school system in any given community, is so reflective of theattitudinal make-up of that community that the schools' effective-ness in implementing change in areas such as violence, vandalism,suspensions and expulsions is limited to remediation at best,without a change in the community attitudes that the school systemreflects.

3. Schools, municipal governments, juvenile service providers, andindependent service groups are generally operating within theirown spheres to alleviate causal factors such as permissiveness insociety, lack of respect for authority and property, boredom, anddrug and alcohol abuse. This is primarily dealt with on a case bycase basis; this approach results in a haphazard individual remed-iation program, with little or no long range preventative effect.

4. Youth alienation and sense of powerlessness on the part of thecommunity; what we commonly refer to as "apathy" -in terms. of vot-- - _ers, community groups, PTA and school councils, etc., is morelikely a result of the sense of powerlesness than a demonstrationof the attitude of the community in general.

5. This "apathy" is a direct result of a feeling of alienation fromthe very institutions that have most direct impact on the commu-nity: schools, local government, social service providers. Theimplementation of a variety of programmatic solutions has createda confusing set of standards of behavior for children, parents,and the community. These standards are not reflective of what thecommunity desires, although the community feels powerless tochange them.

6. A problem-solving process must be designed and implemented whichwill allow the community direct input into analyzing the causes ofjuvenile problems in schools, home and community. These causesmust be sufficiently defined to allow for programmatic applicationand solution; the process must involve all agencies charged witheffecting the solutions along with the community; the solutionsmust be a cooperative effort so that schools, government, serviceproviders, and the community are not working at cross-purposes, asantagonists, or in competition with each other.

a) The process in each community must be credible, and subject toan intensive public relations campaign.

b) All resources, public and private, should be brought to bearupon the causes through systematic youth, parent, community and

391

educational programs, vocational training and placement,counseling, and individual service programs.

7. At a minimum, the process should result in:

a) a standard of conduct that the community deems appropriate inits schools and throughout the community;

b) widespread dissemination of that standard and its communitybasis, together with a rewards/punishment system;

c) fair and uniform application by schools, police courts, andother agencies charr,ed with these duties by the community;

d) positive attitude development throughoUt the entire school cur-riculum; and

e) on-going assessment by the community of its standards.

8. Community Process

A. Advisory councils are not necessarily what the project is hopingto formulate. Advisory councils are usually composed of:

1) Most times the same people who always get involved.

2) This means only 35-40% of any given community is directlyinvolved.

B. Expectations of community process

-Create climate for change (working with formal/informal leader-ship)

-Don't ask for big committee or time

-Ask for suggestions/recommendations

-Hopefully they will start thinking process for what the communitywants to happen

-Inter-agency cooperation

-Want change to occur from the insidewithout threatening anyone's territory

-Communication extremely important

-Agree with the "Rock-Throwers"

-Ask them to do something about their complaints

C. Kinds of people/organizations to invite to meetings

-503-

392

-Schools

Municipal Governments (Elected and appointed representatives)-ChurchYouth ("good" and "bad")

J -Business/Industry (Merchants Asso., Chamber of Commerce)-Kiwanis/Lions/Jay CeesPTO/PTA'sSocial service delivers paid professionals delivering ser-vices for youth-Ordinary PeoplePoliceCourts (municipal, courts)Senior CitizensFamilies

-Social/RecreationalJuvenile. Conference Committee (citizens, meetings, as needed,to handle 1st infraction for youths: punishment, etc., to beerased from youth's record)Knights of ColumbusFirefighters (volunteer)

JVS

Jewish Vocational Services Alternative Education Project: Evaluation Report

M. S. Cook

Abstract.

The Jewish Vocational Services Alternative Education Projectdeveloped and implemented three interventions. The Milwaukee YouthEmployment Center (MYEC) counseled dropout youth and attempted toplace them in employment. The Return Center, operated in coopera-tion with the Milwaukee Public schools, assessed and referred toalternative educational programming youth who were contemplatingdropping out, or who had already dropped out and wished to re-enrollin formal education. The Job Score class was a regular high schoolcourse developed by MYEC staff to teach employment skills to youthat risk for dropout.

An experimental evaluation of the MYEC program indicates that itwas not successful in increasing the employment opportunities of itsclients. Subsidiary analyses suggest that in general, clients didnot receive many services, although remedial education providedthrough the project probably had positive effects. Evaluations oftwo project components- -the Return Center and Job Score clas-ses--were not completed due to the early termination of the project.

In contrast to some recent reports, regression analyses suggestthat employment causes little, if any, pychological harm to youthwho have already dropped out of school. Returning such youth toschool may result in some negative outcomes.

The Job Score curriculum may be appropriate for use and evalua-tion by others interested in providing employment counseling to

youth. It is recommended that prevention projects be allowed tofollow a Program Development Evaluation cycle that provides moretime for program refinement and testing than was available to thisproject.

Overview

The Jewish Vocational ServicesAlternative Educational Project wasan attempt to provide alternativeservices to students who had droppedout of the Milwaukee Public Schools,or were at risk of dropping out.

Jewish Vocational Services (JVS) is

a large agency with a 40-year his-tory of operation in Milwaukee, andit has particular expertise in

employment problems faced by specialpopulations such as immigrants,youth, and the handicapped.

-505-

In the two year history of the

alternative education project, threedistinct interventions were devel-oped and implemented by JVS:

(a) the Milwaukee Youth EmploymentCenter, a job counseling and place-ment center for dropout youth;(b) the Job Score class, an employ-ment counseling curriculum for usein the regular public school curri-culum; (c) and the Return Center, anassessment and referral center foryouth seriously at risk of dropout.

394

Shownwslitau-slLakiessAits

Iko tollowitrg 4* me abbrovintedtorowoo Oro 0111 prOject'a

44004$44044 ,V44**03e. 44* 60410.000 is. 00404044w*. more enwtemm

eoitio 4.0004.400 44 004 %Mfrs soylire 0004 41* teneoeti end

04.40,4sitio Auts*. ***oily from01,0 eso4**4*.

13004:

#14 04; tbet

wow* Moe five t**0004 *tin Aims-A4***0 Liu meltfteetnees inch st per-44004 .10404 4(togeemol teesrel/. themitevossone.ii*e moo ot *ad Ihd01 WNW** ifotOn drer on* et oohs*,

*woof* Joey ne 404 joicet*o tradi-tions** *440&osimg 4410woot to

4044 444404. 441 Owy *44404011 to

0044,o 44*** nor4.**4eme by wow.40f 411101014miwt tot, 644, MO/.

For 444 ** oppertoai4ies forwooktopown*4 tbe *woo** et helitiog a

0100 owt Wimee 41 provides**Lid wo*** Adiamoomry tcf.

4breammitiorg4 Otter 'WOO soyMe rcovided emee twieveer educe-thomel ompotimmoei ow tbet Owl saetom. or tm arse mod oleo* motow. 3L 4001toottel* Ftr eachrmotb,, *vow* is * dittereot **ootefte remmemt mot **Witt*, for **s-*oertsO 4000 provemtteg

-004Mintienkor to COPAN O),mmomS Mismoreticel Oseie for

dm* e lime 0* tioeffective

404 i iNtdr m eisel imodemoste 004441

0400404 440 tro iy MOO. *DPends eso Miemolo4 ft mo oesiety, noel

*moo me food for otweferoily. 'bey

rMallowhod to etleificsat othu.or*. 444*Ohiml 1004 100 oaring role

frafFitexaleounestiore mod ter berruminhow441 del oeniewento tlr to

flehrffol emmlopmemt, and

10 flIMI*4c.4* 04W ageleig del is-

1011*Ffs

4011..

LESUSIILAIL12111

PS staff identified as problemto be addressed youth unemployment,youth es welfare. delinquency, lackOf teleran curricula. in the Mit-osis's., Fnblic Schools, and lack of a

method for youth seeking educationalservice to be referred to spprupri-41410 programming. Consequently,

their #0410 were to:

I. Seduce the uneeploynent ratedam dropouts.

2. Serrealut the proportion of

drepoots es welfare.

3. Decrease the incidence of dol-ing...et acts neon; dropouts.

4. tsetitutioealise the MYEC jobcomeseliag model within the Milwau-kee Public School Syetem by incorpo-

ratisgreties a competeery -based, work -re -lato4 curriculum into the existingprogram.

5. Provide a referral and assess-meet center where dropouts or poten-tial dropouts could receive guidanceeheni appropriate education and

Weer choice.

ailLUZIS

Objectives for the JV$ slterna-live education project include:

if 'armies MYSC clients' "jobreedioess" skillsthe skills nacos-eery to find, secure, and keep ajob.

2. Increasing WIC clients'

work nelited academic skills.

3. Deere/Wog rebellious autono -

sr.-feeling of false independenceand f net seeding anyone or any -0164.

395

4. Increase feelings of personalpower (internal control).

5. Provide youth with income

through employment.

6. Decrease youth's time spent

nonproductively.

7. Return dropout or potentialdropout youth to more adequatealternative education programming.

8. Develop and test a work-re-lated, competency-based high schoolcurriculum.

History and Organizational Contextof the Project

The JVS Alternative EducationProject was plagued with implementa-tion problems from its very incep-tion. Many of these will be

detailed below in conjunction with adescription of the particular inter-ventions. Presented here are gen-eral problems that affected the pro-ject as a whole.

Just as the project was funded inDecember of 1980, JVS underwent a

severe organizational shake-up. The

organization was undergoing federal,state, and local audits as a resultof alleged embezzlement of funds.

At the same time, federal cut-backsin social service funding had

resulted in layoffs of JVS union

staff. A new executive director washired to oversee the stabilizationof the organization during its fis-cal crisis. Thus the project wasborn in an organizational climate ofcrisis and distrust.

The layoffs in staff continuallysubjected the project to union

demands for priority in hiring,regardless of qualifications. Line

staff still on the job did not trust

the organization, and resorted to

strikes or strike threats to presstheir demands.

-507-

JVS

JVS had a history of poor rela-tions with local CBOs, social ser-vice agencies, and the MilwaukeePublic Schools. All of these agen-cies were eventually required to

serve as referral sources and inter-vention sites. Referrals thereforewere slow to develop, and negotia-tions for cooperative work on parti-cular interventions were long and

arduous.

An early error in staffing the

project occcurred whenJVS hired anon-JVS person to direct the pro-ject. This did not turn out wellbecause the project director neverfelt himself to be part of the

organization, or particularlyaccountable to it. The projectdirector, being an outsider, was notinvested in the project as a whole,but only in those parts that he felthe was hired specifically to attendto. Thus, for example, the schoolrelated parts-of -th-e- Froje-ct (th-at

OJJDP was especially interested in)were given short-shrift because theproject director felt that theirmanagement was beyond his purview.

JVS proposal writers, too, may

have promised more than could be

expected of the project. For exam-ple, in the original grant proposal,JVS promised to serve 750-800 youthin the MYEC job counseling programduring the first year--it actuallyserved fewer than 350, and only 79of these received the entire treat-ment. Similarly, the proposal saidit would serve 40-100 youth in theReturn Center during the initial

year. As it turned out, the ReturnCenter did not begin until the sec-ond year of the project.

JVS and the Sponsor

An additional organizational con-flict lay in JVS's relationship withits federal sponsors. JVS adminis-trators failed to quickly or seri-

396

JVS

ously deal with the concerns that

OJJDP had on budget, management, andprogram implementation. Two eventsthat ultimately may have contributedto a decision not to continue the

project for the third year will

illustrate the point. First, in

August, 1982, the project was stillunaware of its continuation statusfor the third project year. JVS

management chose to address this

ambiguity by not sending anyone tothe national conference--an OJJDPrequirement. Inexplicably nobody atOJJDP or at Johns Hopkins was evennotified of the JVS decision not toappear at the conference. They sim-ply did not show up. Hurried phonecalls, and the reminder that appear-ance at the conference was an

already-incurred contractual obliga-tion, produced a JVS delegation aday late.

Second, after the JVS staff

appeared, they were unable to suffi-ciently engage in program develop-ment efforts--the purpose of the

conference--because they were stilltrying to prepare a budget for thethird year proposal. Despite a

clear requirement that all projectssubmit a third year budget that was30% smaller than the previous year,the .JVS third-year proposed budgetwas 20% larger. Project staff spenta great deal of the time at the con-ference deciding whom they weregoing to lay off when they got back.Such thoughts are not conducive toprogram development efforts.

These JVS organizational problemssubjected the project to continuingupheaval over interpersonal rela-

tions, interorganizational rela-

tions, labor relations, and bugetarymanagement. Little organizational

effort remained to focus on the

interventions themselves.

-508-

The Interventions

Milwaukee Youth Employment CenterThe Milwaukee Youth Employment Cen-ter (MYEC) was the core interventionof the JVS project. MYEC was a "jobreadiness" training program, modeledafter the Jobs For Youth, Boston,program. Clients were trained in

the skills necessary to find,

secure, and keep employment, andwere referred to jobs identified bythe project. Youth ,served were 16-and 17-year-old, unemployed, offi-cial high school dropouts from thegreater Milwaukee area.

The MYEC program consisted of

three units: (a) a counseling unitthat trained youth in the job readi-ness skills; (b) an Educational Ser-vices unit that provided competency-based individualized instruction tothose clients needing or desiringit; and (c) an Employer Services

unit that located jobs available,and acted as the employers' advocatewithin the program.

The MYEC project went through twophases: March to December 1981, andJanuary 1982 to the end of the pro-ject in November, 1982. In the

first phase of the project, job

readiness skills were identified(based largely on the Jobs For

Youth, Boston, model), and youthwere assigned to individual counse-lors for training in the skills. In

the second phase of the the project,more detailed skills were identi-

fied, criteria for successful mas-tery were outlined, and a structuredseries of workshops were begun whichall clients had to attend.

There were three "core" job read-iness skills that had to be masteredbefore one was referred to a job:

job application, job interviewing,and job retention. In addition tothese core skills, clients were alsojudged for job readiness by their

397

counselor based on their appearance,punctuality, dependability, andattitude.

In phase one of the project,clients were assigned to an indivi-dual counselor at intake, whotrained them in the skills, andreferred them to a job when thecounselor felt that the client wasjob ready. In phase two, the skillswere taught in a structured workshopformat, and attendance at the work-shops was generally regarded as suf-ficient evidence of job readiness.Each client was also assigned a

counselor. An individual couldretake a workshop, or receive indi-vidual help from the counselor if itwas apparent that he or she had notmastered a requisite skill.

In both phases clients alsoreceived general counseling "asneeded." Also, those youth whodecided to upgrade their academicskills (either by their own wish, orat the suggestion of the counselor)were referred to Educational Ser-vices.

Educational Services providedcompetency-based individualizedinstruction in basic skills (readingand math), advanced instruction in

G.E.D. subjects, and functionalvocational training including typ-ing, cashiering, and driver's educa-tion. Educational Services was anoptional part of the MYEC program.Youth who did not need further edu-cational assistance did not receiveits services.

The Employer Services unit wasresponsible for developing jobs

(i.e., locating job openings andcreating good will among employers)so that positions to which MYECclients could be referred would beavailable.

An important element of their

(.

-509-

JVS

responsibility was ensuring that

employers were satisfied with the

quality of MYEC referrals. A criti-cal part of the program (particu-larly during a recession) was the

ability of Employer Services to havea.job bank available from which MYECyouth could choose appropriateemployment. While the number of

available positions fluctuated dur-ing the time the project operated,there were usually jobs availablefor the youth to be referred to.

The more common problem was thequality of the available jobs. Manywere part time, and virtually all

were very low level entry jobs suchas counter clerks, dishwashers, res-taurant workers, and maintenancepersonnel.

Youth who were referred to jobs,but did not land them, were continu-ally referred until they foundemployment or dropped out of the

program; Youth who -1-inde-d -a- jobwere followed up by their counselorsat irregular intervals to see howthey were doing (although this didnot happen as often as project planscalled for). Youth having diffi-culty on the job were encouraged tocome back to MYEC for additionalcounseling or training. Youths wholost jobs they had been placed in,either because they quit or werelaid off, could be re-referred if

the circumstances warranted it.

Employer services representativeschecked with the employers of youthwho lost jobs in order to assess thedifficulty the youth had withemployment. This often led to addi-tional counseling for the youth.For example, a youth laid off fortardiness would not be referred toanother job until some judgment hadbeen made that tardiness would notagain be a problem.

As alluded to earlier, the pro-gram began with an emphasis on one-

398

JVS

to-cne counseling provided to the

clients by their assigned counse-lors. In the fall of 1981, the pro-ject director became dissatisfiedwith this arrangement, believingthat the counseling provided to

clients varied too greatly dependingupon the counselor. In addition,the Employer Services division of

the MYEC program complained aboutthe "job readiness" of the gradu-ates. Employer Services was receiv-ing complaints from employers aboutthe quality of referrals from theprogram, and was frustrated by thenumber of clients who accepted jobsand then quickly quit them.

In order to insure that all

clients received training in the

three identified job readinessskills, and to gain some qualitycontrol over program delivery, a

workshop format for training wasdeveloped and implemented in Januaryof 1982. This change initiatedphase two of the project. In theworkshop format, MYEC clients werestill assigned to a personal counse-lor, but that personal counselor wasno longer solely responsible forproviding the job readiness train-ing, and certifying the client as

job ready. Instead, each clientwent through a structured series ofworkshops. Workshops were held atset times each week, and clientsprogressed through the workshops inserial fashion. MYEC clients firstattended an orientation workshop,then one on completing job applica-tions, followed by a workshop on

interviewing, and one on job reten-tion.

Satisfactory completion of the

four workshops resulted in certifi-cation as job ready. Although de-termination of "satisfactory perfor-mance" was still largely a judgmenton the part of the counselors con-ducting the workshops (except forthe job application skills workshop

where a post-test job applicationhad to be completed with 80% accu-racy in order to be certified), theworkshop format was an improvementover the previous procedures becauseit insured that every client wasexposed to the fundamentals of eachskill believed to be necessary foremployment success.

The switch to the workshop formatalso improved the program for a sec-ond reason: it forced project per-sonnel to commit to paper exactlythe skills and information necessaryto meet the criterion of "job

ready." In order to write a work-shop that would be presented to allclients, and could be facilitated byany of the counselors, project staffwere forced to come up with a set ofcurriculum materials that specifiedthe information to be taught, andthe methods and techniques to be

utilized. The technical assistancecontractor (Polaris) provided train-ing in group facilitation skills tothe staff specifically for the pur-pose of strengthening the workshops.Such training does not insure thatthe staff acquired the necessaryskills, but it at least forced theproject to engineer into the work-shop specific activities aimed. at

the job readiness criterion, and

gives us more confidence that allclients received instruction in thejob readiness skills.

At the same time that the work-shop format was developed and imple-mented, Educational Services wasmore strongly integrated into the

program. In the first year, clientswere referred to Educational Ser-vices at the discretion of their

counselor. In phase two MYECclients were automatically referredto Educational Services if they

showed low reading or math skills.A written recommendation of instruc-tional objectives was prepared anddiscussed with the client. Referred

510-399

clients were not required to receivetraining from Educational Services,but it was emphasized to them thattheir chances of success in the

employment market were remote with-out basic skills remediation.

The switch to a structured work-shop format, and to more formal uti-lization of Educational Serviceschanged the MYEC program from one ofproviding ad hoc one-on-one counsel-ing in vaguely defined job readinessskills, to one in which all clientswere provided with training in muchmore tightly defined skills. We cantherefore be much more confidentabout the job readiness skills of

the 1982 graduates of the MYEC pro-gram than we can about the 1981

graduates.

Problems in implementing MYEC.

The MYEC program suffered from man-agement difficulties and strainedlabor relations from its beginning.It was a large project. The projectdirector was responsible for hiringstaff and overseeing the design andimplementation of three complexinterventions. The project had a

full-time staff of about 20 personsand involved negotiations with theevaluator, OJJDP, the MilwaukeePublic Schools, the juvenile court,local CBOs, upper level JVS manage-ment, and the JVS employees' union.The newly hired project director hadlittle experience in management, andhis personal inclinations ran towardproviding direct services to

clients, not to running a large pro-ject. The project director did nothave the management experience, or

the exposure to program developmentto adequately oversee this complextask. The project director adopteda laissez-faire management style,

which was in keeping with his pre-ference for service rather than man-agement.

The project director appeared

-511-

JVS

often to avoid rather than confrontproblems. Three events involvingthe evaluation illustrate this avoi-dance of confrontation in uncomfor-table situations. The project hadpromised to secure court contactdata on treatment and controlyouths. Despite the obvious inter-est of both the evaluators and OJJDPin these important data in a delin-quency prevention project, and des-pite assistance in preparing to per-form this task, the court data werenot collected. Although the requestwas made in September, 1981, 15

months before the project was termi-nated, the project director did notnegotiate with the clerk of court toget the required data.

As a second example, the projectdirector did not inform the evalua-tion staff of an upcoming strike inthe spring of 1982. The result wasthat a follow-up of clients groundto an iaexpliCable (ta ES) Hart,- andwe were in the dark as to how to

provide assistance. We speculatedthat no mention was made of thelabor disputes in order to avoid theappearance of difficulties.

A third example involved the

development of a strategy to pay

experimental control subjects fortheir participation in a MYEC fol-low-up. The project director had

led evaluation staff to believe thatthe system for payment was completeand approved. It was not until theevaluation was over a month overduein starting that the directorinformed evaluation staff that, in

fact, he had not gotten the agree-ment of his comptroller to pay con-trol subjects.

A poor relationship between JVSand its employees' union made man-agement of the project difficult.The director, who had not been pre-viously associated with JVS was

400

JVS

immediately faced with the task ofhiring staff in an atmosphere of

distrust between JVS management andits employees. In the two years ofproject operation, there was oneunion strike, and a second was nar-rowly averted.

Union regulations severely con-strained the director's efforts to

select qualified staff. Immediatelyprior to project start-up, JVS hadbeen forced to lay off many of its

employees. When the AlternativeEducation Project began, the projectdirector was forced to hire stafffrom a pool of union employees avai-lable for recall. He had to hirefrom the list based on seniority,with little attention to qualifica-tions.

The problem of forced hiringbased on union seniority resulted inparticularly acute problems in

staffing MYEC's counseling unit.

The union felt that virtually anyonewas qualified to be a counselor, andpressed the issue. The result wasthat the backgrounds of the counse-lors who were charged with imple-

menting MYEC's primary interventionwere in diverse fields, but not incounseling. Two of the persons whoserved as counselors did not attendcollege; others had B.A.'s in

sociology, business, and physicaleducation. Another counselor had aB.A. in education, but had nevertaught, and her prior work experi-ence was as a buyer for departmentstores. The original supervisor ofthe counseling unit did hold a Mas-ters of Education with a major in

Counseling, but she was replaced bya counselor who held a B.A. in Cri-minal Justice. Although degrees donot guarantee successful counseling,the lack of counseling training inthe MYEC counseling staff underminesconfidence that sound counselingpractices were implemented.

-512-

Staff of the Employer Services

and Educational Services unit appearto have had more preparation for

their work. Most of the EmployerServices staff had backgrounds in

business, or had previously workedas job developers on other JVS pro-jects. All Educational servicesteachers were certified teachers:the requirement that they be certi-fied allowed the supervisor of Edu-cational Services to hire outsidethe JVS union.

High turnover and strikes by JVSunion (which included nearly all

MYEC employees) created other prob-lems for MYEC. By the end of the

project, in November, 1982, 11 per-sons had terminated out of the 26

different persons who ever workedfull-time on the project. Two

extensive staff re-organizationswere carried out by the projectdirector in the space of a year and,a half. The average length of stayon the two-year project was 13.6

months (and this does not includeseveral JVS personnel who spent con-siderable time on the project, butwere not paid out of grant funds).Project staff did not get along wellwith one another. As already men-tioned, there was one strike in

October of 1981, and a second wasavoided in April 1982 in a "finalhour" settlement: MYEC staff hadfully expected the employees to

strike. MYEC sought help in staffrelations from the technical assis-tance contractor (who suggested thateven more assistance in this areamight be helpful).

Poor staff relations were notconfined to problems between lineand supervisory staff: the projectdirector and his supervisors in JVSdid not work well together. Theproject director was hired from out-side the organization. He had pre-viously worked in a local CB O. Thedirector disliked the JVS organiza-

4 0.1

tion, and never considered himself apart of it; he felt he was hired torun MYEC, and that MYEC should be anindependent entity from the organization at large. In turn, JVS didnot trust the project director to

manage the project, and placed himunder the immediate supervision ofthe Director of Youth Services. At

one point during the summer we weretold that the project director had .

been or was about to be dismissed;shortly thereafter we learned thatthe project director had filed suitagainst JVS for discrimination in

pay. The project director was stillholding that job in November whenthe project ended.

Another problem for the projectwas the number of clients beingserved by the project. The proiect's proposal promised to serve alarge number of persons, and thesponsor pressed the project to meetits client load target. The programwas begun before developing strongreferral links to local CBO's,Juvenile Court, Milwaukee PublicSchools, or the county Department ofSocial Services. The result wasthat despite a fulltime staff averaging over 17 persons, the programserved fewer than 350 clients its

first year of operation, and most ofthese came in a burst at the end ofthe year; by December, 1981, only 79youth had completed the entire MYECjob readiness program.

In summary, MYEC's implementationdifficulties fall into four categories: (a) the project director hadlittle management experience, andhis personal management style wasincompatible with the demands of a

large, multifacited project, (b) theproject director did not get alongwith his supervisors, and did notconsider himself a real part of theJVS organization, (c) union hiringdemands resulted in placing workersin the counseling unit who had lit-

JVS

tle training or experience to workas counselors, (d) poor JVS managementlabor relations put the projectin continual conflict with its ownstaff, - even to the point of a

strike, and (e) the project wasbegun without first establishing anextensive client referral network.Thus the project careened from onecrisis situation to the next, withlittle time for planning, or

thoughtful program development.

The Job Score Class

The Job Score Classes were an

offshoot of the MYEC job counseling1,program, designed and directed by

the supervisor of MYEC's educationalservices unit. They were an attemptto provide to students still in

school the job readiness trainingavailable to dropout youth referredto MYEC.

-513

One of the prime motivationsbehind the instigation of the

classes was the desire by OJJDP tosee the JVS project more involved in"alternative education"--the basisof the federal initiative. OJJDPdid not regard MYEC, which was an

alternative to formal schooling as

"alternative education." The Officeinsisted that the JVS projectdevelop programs to serve youth thathad not already dropped out of

school, and the Job Score Classeswere an attempt to meet this

requirement.

A tentative curriculum wasdesigned in the summer and fall of1981, and a contract between JVS andMPS providing for the implementationof the class in six area high

schools was signed in October, 1981.Classes actually began in four of

the high schools in January, 1982.

Four more classes were held in theFall of 1982.

402

JVS

The Job Score class was a regularhigh school class that could be

taken for credit toward the highschool diploma in the MilwaukeePublic Schools. The curriculumrevolved around the MYEC job readi-ness skills. Topics included how tofill cut a job application, how tointerview for a job, what to expecton the job, how to locate jobs, andhow to get along with employers andfellow employees. Also included inthe curriculum was instruction in

basic life survival skills such ashow to balance a checkbook, shop forinsurance, and figure interestcharges.

Several weeks into the semester(optimally, around six weeks) thosestudents that were judged by theclass teacher to be "job ready"(using the same criteria as clientsin the MYEC program, above) werereferred to MYEC for job referral.Those students that MYEC also judgedto be job ready were then referredto part time jobs. Students that

landed jobs worked half time, and

attended school half time. They

received academic credit towardtheir diploma for the time theyspent working. The program designcalled for students to maintaintheir academic standing, but it is

not known if this requirement wasadhered to. Students who did notfind jobs continued in the regularcurriculum.

The curriculum package as devel-oped by the supervisor of EducationServices appears promising. She

wrote detailed day-by-day lessonplans with clearly specifiedinstructional objectives, andincluded methods of assessing suc-cess. All of the MYEC job readinessskills were covered and reinforced.The curriculum appears to be highlyrelevant to students, and was favo-rably viewed by the four teachersand principals involved in the ini-tial application of the course.

-514--

The Return Center

The Return Center was the focusof the original grant proposal to

OJJDP. It was envisioned as one ofa number of centers that would beset up statewide as a cooperativeventure of all Wisconsin agenciesresponsible for providing socialservices to youth.

These service delivery centerswere supposed to ,be centralizeddiagnostic, referral, -and record-keeping centers. Their purpose wasto serve as sites where youth withproblems could be assessed and thenreferred to an appropriate servicedelivery provider in the area. Cen-tralized record-keeping would pro-vide a method of follow-up and eval-uation, and would allow agencies tobetter plan for the needs of youth.JVS would focus on developing a

Return Center that would serve theneeds of Milwaukee area youth whohad dropped out of school, or whowere contemplating dropping out, butwho sought some form of structurededucation. The Return Center wouldassess their skills and interests,and then refer them to agencies(including the Milwaukee PublicSchool system) that provided appro-priate intervention.

It is perhaps symptomatic of thedifficulties JVS had in implementingits project that, despite the intentof the original grant proposal to

focus on the Return Center concept,the JVS-sponsored Return Center didnot begin operation until January of1982--more than a year after the

project was funded.

Three major problems contributedto the slow implementation of the

Return Center:

1. The project director believedthat he had been hired primarily tooversee the development and manage-

403

ment of the MYEC job counselingcomponent of the program. It washis belief that the Return Centerwas primarily the responsibility ofupper level JVS staff, and indeed,at the March 1981 planning confer-ence in Washington, D.C. a JVS staffperson not funded by the projectaccepted responsibility for negoti-ating the Return Center contractwith MPS. At the conference theproject director was assigned res-ponsibility for developing contactswith MPS in-school personnel so thatreferral sites would be available toaccept clients from the Return Cen-ter.

The project director had littleexperience in the negotiation of

contracts and agreements, and wasnot happy with working with the Mil-waukee Public Schools. In lact, theproject director (and MYEC staff ingeneral) were reluctant to referclients in the MYEC program to MPS.Also, the implementation of the MYECcomponent itself was not progressingrapidly, largely due to labor diffi-culties. The result of all of theseforces was that the project directorassigned top priority for his atten-tion to the MYEC job training pro-gram, and put the Return Center onthe back burner. Negotiations forthe establishment of the center wereleft to JVS staff outside of MYEC.The project director felt that if

JVS upper management thought the

Return Center was so important, thenthey could oversee it. In short,the project director felt that hewas director of the MYEC portion ofthe project, and that the ReturnCenter was an additional (relativelyunimportant) duty that took timeaway from what he saw as his centraltask.

2. The second major reason forthe slow implementation of the

Return Center was that JVS and MPShad poor inter-organizational rela-

JVS

tions. JVS feared that MPS couldnot be held accountable for the

funds, and would insist on control-ling the design and staffing of theCenter. MPS suspected that JVSwould really like to keep the funds"in- house" and would be reluctant torefer "JVS" youth to the Center, andback to MPS. Since neither organi-zation trusted the other, negotia-tions to set up a Return Center,staffed by MPS personnel, but paidfor out of JVS/OJJDP funds, becamelong and extremely complicated.

3. The third reason for slowimplementation of the Return Centerwas that the negotiations for the

1,1

establishment of the Return Centerwere tied to those for developingand implementing MYEC-sponsored JobScore classes in six Milwaukeepublic schools--for which curriculumdid not exist in 1981. EducationalServices personnel within MYEC wereassigned respOnsibiliiifoF deiign-ing the curriculum, and did so, butin the meantime, negotiations forthe Return Center could progress nomore rapidly than those for the JobScore classes (and vice-versa).

-515-

The result of the three majorobstacles to the implementation ofthe Return Center was that a draftof the contract with MPS was notcompleted until September, 1981. By

the time a site was found and staffwere assigned, it was January, 1982.

To summarize the primary causesof the slow implementation of the

Return Center: (a) the JVS projectdirector concentrated his energieson developing the MYEC componentwhile paying little attention to theReturn Center (and felt justified inso doing since other JVS personeltook responsibility for negotiatingthe Return Center contract withMPS); (b) the contract negotiationsprogressed slowly due to poor rela-tions between JVS and MPS; and

404

JVS

(c) the Return Center negotiationswere tied to negotiations for estab-lishing Job Score classes in the

Milwaukee public high schools.

As finally implemented, the

Return Center staff consisted of a

program coordinator, a psychologist,a social worker, a vocationalcounselor, three teachers and a sec-retary.

The intervention had three (some-times four) components:

1. A reentry conference was heldin which the youth (sometimes withhis or her parents) met with ReturnCenter staff to initially assess andinterest the youth in participatingin school activities.

2. An assessment phase followedin which formal assessment of theyouth's basic academic level, intel-ligence, and vocational interestswas made.

3. A referral conference washeld (sometimes with the youth'sparents in attendance), in which thechild was referred to either an MPSor Community-Based Organization(CB0) for instruction, or was re-ferred in-house to receive individu-alized instruction at the ReturnCenter itself.

4. Youth remaining at the ReturnCenter for instruction receivedindividual tutoring, usuallydirected at completing the G.E.D.

Most youths who stayed at the Centerdid so because they wished to returnto an MPS site, and could not do soimmediately because it was too latein the academic semester. They wereplaced in MPS programming at the

beginning of the next semester.

-516-

Evaluation Results

Results for the Milwaukee YouthEmployment Center

Two types of research were per-formed to examine the effects of theMYEC component of this project.First, straightforward analyses of

the experimental results were per-formed to describe the effects of

the project. Then, more complexanalyses were performed to unearthas much information -as possibleabout the relative effectiveness ofvarious elements of the overall MYECactivities.

Primary analyses of MYEC effects.JVS successfully implemented a trueexperimental evaluation of the MYECjob finding intervention. Beginningin August, 1981 and continuing untilFebruary,1982, 252 intake youth wererandomly assigned to treatment, and154 to control. Randomization wasaccomplished on-site by using anintake blank on which 62.5% of thelines contained a "treatment" desig-nation and 37.5% of the lines con-tained a "control" designation (a

five-to-three ratio of treatment tocontrol clients). These blanks wereprepared by the national evaluatorusing computer generated pseudo-ran-dom numbers.

Applicants' names were entered onthis list in the order they pre-sented themselves at MYEC. Controlyouth were referred to other pro-grams for youth in the Milwaukeearea. At intake, both treatment andcontrol youths were administered apretest that measured several of theobjectives of the MYEC program.

Seven months after intake, treat-ment and control youths were con-tacted by MYEC and JVS outreachstaff. Each was individually admin-istered a questionnaire that ass-

essed several personal characteris-

405

tics, past and current employment

status, and involvement in educa-tional activities. Control youthwere paid $10.00 for their partici-pation. A total of 203 treatmentyouth were surveyed, for a 81% fol-low-up rate; 118 of the controlyouth were administered the questi-onnaire for a 77% rate. The two

follow-up rates do not differ signi-ficantly (p = .41 by the chi-squaretest).

The pre- and post-test measuresof personal characteristics weresubjected to principal factor analy-sis with iterated communalities.Three factors had eigenvaluesgreater than 1.0 and were subjectedto VARIMAX rotation. Separate ana-lyses of the pretest and post-testitems yielded similar factors. The

three factors were labeled Psycholo-gical Health, Interpersonal Compe-tency, and Rebellious Autonomy.

Scales were constructed from theitems loading most highly on each ofthe three factors; a particular itemwas only included on the scale onwhich it had its greatest loading.Although the three factors weresimilar for the two testing periods,there were differences in a fewitems. Various combinations of

items were subjected to internalconsistency item analysis to developa set of items that worked well foreach of the three scales at bothtimes. Appendix A gives the item

content for the three psychosocialmeasures developed through theseprocedures. Psychological Health

had a reliability coefficient(Chronbach's alpha) of .61 at pre-test, and .59 at post-test, and a

test-retest correlation of .40.

Rebellious Autonomy had alpha coef-ficients of .49 at both pre- and

post-test and a test-retest reli-ability of .39. The third scale,

Interpersonal Competency had an

alpha of .45 at pretest, and .46 at

'controls, but the absolute size ofthe difference is very small--seeTable 1). The groups did not differon the time between intake and fol-low-up.

JVS

post-test. This scale shows lowertest-retest reliability than the

other two scales (.19).

Table 1 gives the results of

post-randomization checks on the

equivalency of the MYEC treatment

and control groups. Chi-squaretests on the sex of the subjects andthe percent who were of minorityethnicity (mostly Black and His-panic), show no differences betweenthe groups on these.measures. Ana-lyses of variance indicate that thegroups were similar in age, and inthe time of intake (there is a

trend, p = .10, toward the treatmentyouth being slightly older than the

-517-

Analyses of the pretest psychoso-cial measures show nb-diffeielices-onPsychological Health, or Interper-sonal Competency. The control groupreported significantly greaterRebellious Autonomy (p=.006) thanthe treatment group. This resultmeans that this pretreatment differ-ence will have to be accounted forin any analysis of outcomes, parti-cularly on Rebellious Autonomy as

measured at follow-up. Given the

procedure used to assign youths to

treatment and control groups, andthe quite close equivalency of the

groups, we are confident that a trueexperiment was implemented.

Changes in the psychosocial mea-sures were assessed by subtractingthe follow-up score from the pretestscore. Mean changes, and results ofanalyses of variance carried out onthe scores may be found in Table 2.No differences were statisticallysignificant. The treatment and con-trol youth showed no differentialchange in Psychological Health,Interpersonal Competency, or Rebel-

406

JVS

lious Autonomy; An additional

attitudinal measure not available atpretest was developed for the post-test. It was called Attitude TowardEducation. This was a scale com-posed of five items designed to mea-.sure the degree to which the respon-dent perceived formal education asuseful and relevant to his or herlife. This measure was developedbecause Educational Services stafffelt that one of tneir primaryobjectives was convincing youthsthat was indeed relevantto their lives and livelihood.Table 2 shows that there were nodifferences between the experimentaland control groups on this scale.

The analyses of the measures ofMYEC's intermediate objectives givesno reason to believe that they weremet. Nothing in these results sug-gests that any additional job exper-ience provided to client youth by

the MYEC program resulted in any

positive psychosocial change.

More critical for the MYEC evalu-ation is an analysis of the currentand past employment and educationalstatus of MYEC clients and theircontrols. Table 3 gives the resultsof chi-square analyses comparingtreatment and control youth on threemeasures of program success: cur-rent employment status, current edu-cational status, and a combined edu-cational and employment status

measure. For this composite mea-sure, a "positive outcome" was

regarded as a report of being in

school or employed. These outcomemeasures were derived from a ques-tion asking "What are you doing

mostly?" The respondent could onlycheck one of several categories ofeducational involvement or employ-ment. Analyses indicate that the

MYEC clients are no more likely toreport themselves to be currentlyworking on a job than are the con-trol youth. Approximately 80% of

-518-

both groups were not working at

follow-up. There is a tendency forthe control group to report moreinvolvement in school (either re-en-rolled in high school, or attendinga G.E.D. program). This tendency,when taken together with the verysmall difference in work status,

produces a trend (p = .07) for a

higher percentage of controls to

report a "favorable outcome" (eitherworking or it school) than treatmentyouth.

The difference in reported schoolattendance is difficult to inter-

pret. Project staff have repeatedlynoted that at intake clients invari-ably report that they are working ontheir G.E.D., even if they havenever enrolled, or have neverattended a G.E.D. class. Many youngdropouts believe that they willgraduate, "some day." MYEC clientsare faced with the fact that if theytruly want to earn their G.E.D., thetraining is available at MYECthrough Educational Services. It is

therefore more difficult for MYECclients to claim that they are work-ing on a G.E.D., if they are not,than for the control youth to makesuch a claim. Although we have noway of verifying the respondents',

reports it is doubtful that the

experimental and control groups dif-fered in participation in a G.E.D.program.

Further information on employmentand educational status can be foundin Table 4. This tabulates 'theresults of chi-squares comparingresponses to various categoriesavailable in response to a questionasking "What are you doing now?"Allowing multiple responses enabledstudents who were working part-timeand going to school to record bothactivities.

As the table shows, the resultsof the analysis of the questionnaire

407

item measure mirror those for the

previous item. About equal percen-tages of treatment and controlyouths report they are currentlyworking for pay, but more controlthan treatment youths report thatthey are attending a G.E.D. programand more report that they areenrolled and attending high school.More treatment than controls acknow-ledge that they are unemployed(which also connotes not attendingschool). A higher proportion of thecontrol youths also report that theyare "On temporary layoff from work."

Tables 3 and 4agree in implyingthat (a) the MYEC clients are nomore likely to be employed at fol-low-up than are the controls (andvery few of either are employed),and (b) a higher proportion of con-trol group members report themselvesto be re-involved in some kind offormal education. Of particularinterest is the difference in theproportion of respondents from thetwo groups who report they are"Enrolled in and attending highschool." We cannot be sure that thecontrols were simply "yea saying"more than the treatment stu-dents--although the lack of differ-ences in responding to work-relatedquestions would argue against thathypothesis. But both Tables 3 and 4suggest that participation in MYECmay have discouraged students fromreturning to formal schooling. Theresults offer no evidence that MYECwas successful in aiding clients tofind more stable employment thanthey could have found on their own.

Other follow-up questions con-cerned hourly wages, current hoursworked per week, and hours workedfull and part-time "in the last sixmonths." This insured that allrepondents would be reporting onlyon employment experience acquiredsince MYEC intake. Table 5 givesthe results for analyses of variancefor the wage and hour measures.

-519-

JVS

None of the nine measures revealany differences between the treat-ment and control groups. The groupsdo not differ on the current numberof hours working per week, currenthourly wages, current weekly wages,or total wages earned in the lastsix months. The groups are alsosimilar in the the number of weeksthat they had worked full time andpart time. Finally, a logarithmictransformation of the current hourlywages and total wages earned (a

transformation commonly carried outon wage data to decrease the influ-ence of extreme values in what is

usually a markedly skewed distribu-tion) did not produce any evidenceof differences between the treatmentand control groups.

There is no evidence in theseresults that MYEC improved the

employment situation of its clients.They are no more likely to be

employed at follow -up, to haveworked more in the recent past, orto be earning or have earned anymore money. There is therefore noevidence in these results that theMYEC program met its primary goal ofincreasing the employment opportuni-ties of its clients.

The MYEC project was funded pri-marily as a delinquency preventioneffort. Employed youth were theor-ized to have less of a need to com-mit crime out of a need for money,for status, or for ,excitement;Because the treatment group did notdiffer from the control group inemployment status, one would notexpect (on the basis of this theory)delinquency to be differentiallyaffected. Yet, personal counselingwas a largz part of the MYEC pro-gram, and might be expected to havesome positive effect on delinquentbehavior independent of employmenteffects. There is no evidence thatit did. The final outcome listed onTable 5 shows results for a question

408J.

m$40* 441-41y it*** Oat the last

os Otto tho teseottkeit Cad ette

+44e4 up 4)* che $tip. %et* erepie tiftwo**** **,*40o0 tioo it**trotisue oostto4 IP#44 00 WO 01044444.1

044 040010 fit sstetisottales-J*4*ft** of tlie Vi i« it townael..1440, 044140 400 4.401144+ MI, tofolke.wis*, OW, to* he 44414401444 44fleitos*i rhost is so ovideame thatthe Whe sepses Nod owp affect as4110441p4101 trilt44. psythatosialliese4almos44 oat detteguett beksist.Uwe ;* tome woe, tettdeset Om the#014404V6 edif hem Umelearaged poeth*t Monies to taloa.

AN4A4MUL.ANCWWL...g....fingillAtAp4 thete ate foss pesethle

aises* w4p, au evalwatioa may ukow44414.104* 4ivitoot treattoott eel

ivattel groupti

i* 'lbw tilimety atebtod the progtam

Is 4**04--110 0614* st *1* rearmstout* ust **to toot tot mom it itsobleciiee* Beare.

Po Mc eosmsom tide( addressthe theorpithe pees,. woe tart-tots* te the the tows wbtds itwet mappeeedly baseed.

2* fat program wee oet tople-00ated pempetlyn.,the tatervestiemeMat occurred mote eat the metWeeded*

444 tovelea44., desists wasiOneosillive 4. the tree effects of4Aom progrom.

al;WWwir.14402,41.114WW0,41,446M2.10.41.44

I* Est lees plao. called for the

project to immure of recordssof olive 08044itts. Partly becauseat the tarty temitatios of thiepeolect, sod pertly doe to the lowpetteity pawed ma evaluates maavOtos hp she project director (see110.0 ,), Oficial police recordstore erne obtsimool.

tech of the leer possible reason*tot falling to demonstrate programeffectiveness is important, and a

prtgrat or its evaluation may failfor any of these reasons.

sera cafteot easily assess the ade-

quacy of a program** theoreticalpremises, or its relation to its

theory, unless one knows that the

program was reasonably well imple-oonted. Table 6 offers strong evi-dent' that the HYIC job counselingprogram vas not implemented withenough fidelity to reach strong con-clueless about its ,gaggligl of

Table 6 gives simple descAptivestatistics of four implementationmeasures; the number of counselingcontacts, the number of educationalservices contacts, the number of

jobs to which the client was re-ferred, and the number of jobs thatthe client was actually placed in byWIC. Meet impressive is the lastcolumn in Table 6. It shows the

percentages of clients receivingAims of that particular part of theintervention. The bottom line, of

Mr.,. is job placementsplacingyouth is jobs is what MYEC was allabout. Yet, 72Z of the MYEC clientsbetween Moot 1981, and February,062 were not placed in a job. In

other words, only a little over aquarter of KYIC clients receivedMTEC's main service. Only half wereever even Wang to a job. For

each of the four implementation mea-sures, the modal (most common)amount of service provided to a par -titular client was aero.

Apparently the MYEC project was ajob placement project that did notplace many youths in jobs. In other

words, the job counselineplacenentprogram was not implemented as

Weeded. We cannot clearly addressthe question of the theoretical ade-quacy of the MYEC program, or its

relation to its theory (the first

409

and second possible reasons for pro-gram failure listed above) becausethe evaluated MYEC program was notthe Intended MYEC program. As to

whether the evaluation design wassensitive to possible programeffects, a well implemented true

experiment is a strong design (Cookand Campbell, 1979). The high res-ponse rate from a large number ofexperimental treatment and controlparticipants implies that thisaspect of the experiment was faith-fully implemented. Although ques-tions could be raised about the mea-sures examined, the job- andeducation-related measures appearstraightforward and highly relevantto the evaluation of this project.

A further question might be, "IfMYEC had systematically implementedthe program, would it have been ableto place more of its clients in

jobs?" In order to investigate thisa question, a series of regressionanalyses were undertaken. Statisti-cal models were built to account for(in separate analyses) individualvariation in the five primaryemployment measures, an indicator ofenrollment in school, and self-re-ported police contacts.

Two regression models wereemployed to investigate effects of

differential exposure to MYEC treat-ment. In the first model, for eachMYEC client a single indicator of

MYEC treatment intensity was enteredwhich reflected the total amount ofMYEC treatment received. In the

second model, each separate compo-nent of the MYEC program was indivi-dually examined. In both models, astepwise procedure was employed in

which background and personalitymeasures at intake to MYEC wereentered first; sex, ethnicity, age,month of enrollment in the program,Psychological Health at pretest,Interpersonal Competency at pretest,Rebellious Autonomy at pretest, andbasic skills proficiency were

JVS

allowed to enter at this stage if

they contributed to the explanationof the outcome of interest.

In the first set of regressionanalyses, MYEC treatment effectswere studied by entering at step twoan overall index of MYEC treatmentreceived. Any additional variancein the individual employment,schooling, and delinquency scoresthat was captured by this step in

the analyses might be attributableto MYEC treatment, net- of measuredbackground characteristics.

The MYEC treatment intensityscore was constructed by dichotomiz-

4.in the four MYEC implementationmeasures at the median. Each MYECclient received a score of 1 on eachmeasure for which he or she receivedmore than the median amount of

treatment, and 0 if he or she

received less than the median amountof treatment. These four scoreswere then summed to yield a singleindex of treatment intensityreceived. A client's treatmentintensity score could range fromzero to four.

-521-

Table 7 reports the results of

the MYEC treatment intensity regres-sion analyses on the primary employ-ment variables: hours currentlyworking per week, weeks workedfull-time in the last six months,weeks worked full-time in the lastsix months, log-transformed currentwages, and log-transformed total

earnings in the last six months.Listed are the effects of a) thosepredictor variables entered at stepone that significantly increased thevariance explained, and b) the MYECtreatment intensity variable(labeled "MYEC participation"). The

column headed "beta" shows the con-tribution of each variable, net of

any previously entered variables.The beta given for the MYEC partici-pation score is for the direct

410

JVS

effect of MYEC treatment, indepen-dent of all examined background mea-sures. That is, the beta representsthe MYEC treatment's contribution toexplanation that cannot be attri-buted to any other variable in themodel. The third column headed"Increment to R2" shows the propor-tion of variance in the criterionaccounted for by adding that predic-tor to the model, net of previouslyentered predictors. Thus, the "MYECparticipation" entry gives the addedvariance in the predictor explaina-ble by increased MYEC participation,holding constant the background mea-sures. Finally, the last colutin,

headed "p for increment" provides astatistical test for the added con-tribution (incremental validity) ofeach successive variable. The p'sfor MYEC participation are of parti-cular interest--significant incre-ments in explained variance (markedby an asterisk) suggest that parti-cipation in the MYEC program proba-bly contributed to an increasedscore on the criterion measureinvestigated.

Table 7 indicates that the MYECintensity variable was a significantpredictor of employment status forfour of the five employment depen-dent variables: receiving more MYECtreatment is associated with morehours per week currently worked,more part time work engaged in dur-ing the last six months, higher cur-rent wages earned, and greater totalwages earned during the last sixmonths.

Table 7 also indicates that ageand ethnicity are consistently asso-ciated with employment outcomes.Older youth show better employmentand earnings than blacks. There isalso some evidence that greaterInterpersonal Competency as measuredat pretest predicts less part-timeemployment, and, therefore, lesstotal earnings over the last sixmonths.

-522-

Table 8 gives the results for

regression analyses investigatingthe degree to which MYEC treatmentparticipation can predict the psy-chosocial outcomes of PsychologicalHealth, Interpersonal Competency,and Rebellious Autonomy as measuredat post-test, re-enrollment in for-mal schooling, self-reported policecontacts, and Attitude Towards Edu-cation as measured at post-test.Again, background variables wereentered first, and the MYEC partici-pation variable last;- the effectgiven.for the MYEC variable underthe heading "beta" indicates the

direct effects of MYEC participa-tion, independent of any variablesentered previously.

Table 8 shows that greater MYECtreatment has no statistically sig-nificant net contribution to psycho-social development (although thereare trends toward treatment beingnegatiVely associated WithRebelli-ous Autonomy, p=.08, and positivelyassociated with Attitude Toward Edu-cation, p=.10). Table 8 also sug-gests that increased participationin MYEC activities is associatedwith an increased likelihood of

being re-enrolled in some form of

formal schooling, i.e., MYEC clientswho persisted in the program weremore likely to enroll in school.Note that this result differs fromthat found in analyses of the treat-ment and control comparisons.Results of those analyses suggestedthat the MYEC program discouragedre-enrollment in school. This appa-rent discrepancy can be reconciledwhen it is recalled that most MYECclients received little or no MYECtreatment. It would appear that forthe majority of MYEC clients whoreceived only superficial exposureto the program, MYEC discouragedthem from becoming reinvolved in

formal schooling--yet the programhad the opposite effect on youththat persisted in the program andreceived the entire treatment.

411

These regression analyses show

that there is a mild negative pred-ictive relationship (beta=-.12,p=.13) between MYEC participationand self-report arrests ("picked upby the police during the last six

months"), i.e., there is weak evi-dence in these data of a directeffect of MYEC participation uponthe reduction of delinquency. Thisresult is slightly muddled in thatpolice contacts are measured somew-hat concurrently with the MYEC vari-able--the youth are reporting ontheir last six months experience,part of which would have been spentin MYEC. Some clarity is lent to

this analysis by the fact that mostclients were in MYEC for a shortperiod of time, and even long-termclients would seldom have been in

the program for a month. Since theaverage follow-up period was aboutseven months, the majority of MYECparticipation would have occurredprior to the period for reportingarrests. There is some legitimacy,therefore, in believing that the

MYEC effect found (such as it is) isa true "predictive" relationship:the longer a youth persisted in MYECtreatment the less likely he or shewas later to report being picked upby the police.

Table 8 also indicates that the

strongest predictor of future psy-chosocial development is presentpsychosocial development--personali-ty is fairly stable across the timeperiod covered by this study. Most

of the variance in PsychologicalHealth, Interpersonal Competency,and Rebellious Autonomy at post-testis accounted for by the same person-ality variables measured at pretest.

To summarize the effects of

increased exposure to MYEC treat-ment, Tables 7 and 8 show positiverelationships between the amount ofMYEC treatment on the one hand andemployment and formal schooling sta-

-523-

JVS

tus on the other, and some

suggestion of a weak effect on del-inquency.

In order to more closely examinethe MYEC effects, a second set of

regression analyses were carriedout. In this set of analyses indi-vidual measures of each of the fourMYEC treatment variables were usedto predict employment status (Table9) and psychosocial, schooling, anddelinquency status (Table 10). In

these analyses, the individual MYECtreatment indices were substitutedfor the overall MYEC treatment par-ticipation score in the second stepof the analyses, following the entryof the background variables. Countsof counseling contacts, educationalservices contacts, job referrals,and job placements were allowed toenter the statistical model in theorder of the amount of variance eachexplained. In all cases, statisti-cally significant effects listed arefor the direct contribution of the

significant MYEC treatment compo-

nent(s) toward the explanation of

variance of the outcome measure inquestion.2

Table 9 shows that the number ofeducational services contacts and

the number of job placements aregenerally associated with more favo-rable employment outcomes. Counsel-ing contacts show little relation-ship to employment outcomes, exceptfor a slight tendency for those whoreceived more counseling to be lessinvolved in in part-time employment(p=.09). This tendency contrastswith the result for number of educa-tional services contacts which showa moderately strong positive

2. Effects for the other variablesin the model--Lhe background varia-tion--are not given in Tables 9 or

10 because they are identical to

those given in Tables 7 and 8.

412

JVS

relationship with part-time employ-ment (p<.001). MYEC records indi-cate that most of the jobs in theMYEC job bank during the time of theevaluation were part time jobs.

These results suggest that clientsavailing themselves of the helpoffered by Educational Services weremore successful at obtaining workthan were other clients. The nega-tive correlation between counselingcontacts and part-time employmentsuggests either that the counselingstaff were actively discouragingclients from seeking part time work,or that youths who received counsel-ing also experienced more difficultyin getting jobs. There was no

organizational policy of eitherencouraging or discouraging part-time work, and different parts of

the MYEC project may have treatedtheir clients differently in theirapproach to part-time work.

Overall, Table 9 provides no evi-dence that counseling in the coun-seling unit was successful in pre-paring clients for employment.Controlling for background varia-bles, there is no significant rela-tionship between counseling and

employment. Recall that counselingwas considered to be the primaryintervention: educational servicescontacts were supposed to be anadditional treatment available forclients desiring upgrading of basicskills. The present analyses sug-gest, however, that training in

basic skills had more effect then

the prescribed counseling.

Table 10 shows the effects of theindividual treatment components on

psychosocial development, re-enroll-ment in school, and self-reportedarrests. Only the implementationmeasure of job placements explainsany additional variance in the psy-chosocial measures at post-test overthat explained by the pretest andbackground variables. Being placedin a job by MYEC is associated with

greater Psychological Health, and

with more positive Attitude TowardEducation.

The overall treatment intensityanalysis indicated that increasedMYEC treatment was associated withincreased re-enrollment in formal

schooling; Table 10 shows that thiseffect is largely the result of theefforts of educational services. It

is the only individual treatmentcomponent that is. a significantpredictor of re-enrollment in

school.

Finally, Table 10 suggests a

small positive effect for job place-ments on police contacts (p < .10).Clients who were placed in jobs wereslightly less likely to report them-selves to have been picked up by thepolice. Thus, the majority of the

trend toward decreased self-reportedarrests associated with persistencein MYEC (see Table 8) is attributa-ble to actually being placed in a

job. This is important for a theor-etical analysis of the relationshipbetween employment and delinquency(see Appendix B).

A final observation from this

analysis of MYEC treatment componenteffects is that the counseling com-ponent is essentially unrelated toemployment (although somewhat nega-tively related to part-time work),and is associated with significantlyless Interpersonal Competency at

follow-up, i.e., the few effectsnoted are in a 'direction opposite toprogram objectives.

In summary, the results of the

foregoing regression analyses sug-

gest that:

1. More exposure to the MYEC

treatment is associated with betteremployment status at follow-up, butthe primary evaluation--the true

e.hw ex-per-i-ment---in-di-cates no

-524-

413

overall positive benefit for the

MYEC program. It is difficult, evenwith the variety of pretest andbackground information available onthe clients, to entirely control forpre-existing motivational differ-ences among MYEC participants; youththat stuck to the MYEC program, andreceived more treatment, might showenhanced employability simplybecause they are harder workers, orare more persistent in a job search.

Nevertheless, because we do havea number of statistical controlvariables, the results suggest thathad clients received more systematicand thorough exposure to the treat-ment, the program would have beensuccessful.

2. Receiving assistance through theEducational Services component andactually being placed in a job byMYEC is associated with some posi-tive outcomes. No evidence suggeststhat counseling, MYEC's primaryintervention, was successful in aid-ing MYEC clients to find employmentor to improve their psychosocialskills. In fact, to the extent thatpre-existing client differences weresuccessfully controlled for by pre-test measures, the results suggestthat counseling produced a decrementin Interpersonal Competency. This

result is not necessarily an indict-ment of employment counseling in

general. Recall that the MYECcounselors had little training or

experience in counseling.

3. The educational services compo-nent of the program appears to havecontributed to the employment andeducational status of its partici-pants. Again, to the degree that

motivational differences are cont-

rolled for by the pretest and back-ground measures, basic skills educa-tion in an alternative out-of-schoolformat increases the employabilityof dropout youth. Because educa-

-525-

JVS

tional services offered in-house

G.E.D. instruction, it also contri-buted to the reinvolvement of MYECclients in formal education.

The positive educational servicesresults are important for the over-all evaluation of the DelinquencyPrevention Through Alterative Educa-tion initiative; this alternativeeducation intervention appears to

result in some positive outcomes.

4. Three other consistent pred-ictors, of employment status are theracial status of the clients, theirage, and the month the youth wasenrolled in the MYEC program.Whites are more likely to be

employed; older youth have betteremployment outcomes; and the laterin the year that a youth was refer-red to MYEC the less they workedfull- or part-time in the followingseven months.

The results for race imply thatdiscrimination in hiring exists inthe city of Milwaukee. Even whencontrolling for background differ-ences, including basic skills profi-ciency, there is still ,a residualnet benefit to being white. In

fact, in most analyses; the beta forracial status is larger than thatfor receiving MYEC treatment.

One might argue that some motiva-tional difference accounts for the

differential employability of whitesand minorities. This hypothesis canbe explored by looking at persis-tence in treatment. Table 11 showsthe simple correlations between thenumber of MYEC component contactsand white/minority status. Minori-ties stayed in treatment longer asmeasured by the number of counselingand educational services contacts.There is no evidence here to suggestthat minority youth were unmotivatedto attend program sessions.

414

JVS

The lack of correlation betweenminority status and job referrals inTable 11 indicates that MYEC person-nel referred proportionately as manyminority as non-minority clients forjobs. Yet the significant correla-tion between job placement andwhite/minority status demonstratesthat MYEC was unable to overcome thebasic white/minority difference in

employability. Although MYEC spentmore time with minority clients, andreferred both whites and minoritiesequally. to jobs, white youths werestill more successful in securingemployment.

The findings of enhanced employ-ability with increased age, anddecreased employability as the pro-gram progressed are also importantresults. The age of MYEC's servicepopulation was a continual source ofcontention both within the projectitself, and between the project andthe federal sponsor. MYEC personnelfelt they had been saddled by JVS

with an impossible task: findingemployment for 16- and 17-year-olddropout mostly minority youth duringa recession. Particularly disturb-ing to the staff was the constraintthey felt in being required to serveonly 16- and 17- year -old, youth.They felt that they could have beenmuch more effective serving olderdropout youth.3

The finding of poorer employmentstatus as the evaluation progressedis difficult to interpret. One pos-sibility is that the general Milwau-kee employment situation may have

been deteriorating over the courseof the evaluation as the recession

3. The Office for Juvenile Justiceand Delinquency Prevention's legis-lation limits its purview to juven-iles. Because the project was spon-sored by OJJDP, it had to servejuveniles.

-526-

deepened, and jobs may have becomeincreasingly hard to come by. Theeffect is not due to any increase intime between entry and follow-up.The time lag, which was measured formost of the follow-up population, isuncorrelated with any of the outcomemeasures.

Whatever is the source of the

decrease in full time employment forthe MYEC clients over the course ofthe evaluation, there is nothing inthe results to suggest that the pro-gram improved during the course of

the evaluation. Clients served in

the later months of the project wereless likely to work full-time thanwere clients served in the project'searly months. Project personnelargued that the recession made theirtask almost impossible; employmentwas so high in general, and in theirtarget population especially, that

there were simply no jobs availablein which to place the clients.Research indicates that youth unem-ployment is very sensitive to the

state of the economy--a one percentincrease in the official adult maleunemployment rate translates into a5-6% increase in the number of teen-age males looking for work (Clarkand Summers, 1980). In fact, Clarkand Summers (1980) argue that the

foremost reason for high teenageunemployment is a lack of jobs, andthat it is fruitless to set up

"employability" programs (such as

MYEC). Such programs (even whenthey work) may merely substitute oneyouth for another in the employmentqueue with no net positive result onunemployment in the target popula-tion. This problem would be parti-cularly acute during a recession assevere as the one that occurred dur-ing MYEC's operation.

Additional observations on MYEC.As discussed earlier, the MYEC pro-gram underwent substantial changes

during the winter of 1981-82.

415

Emphasis was shifted from a one-on-one personal counseling orientation,to a structured series of definedworkshops accompanied by increasedinput from Educational Services.Since the evaluation covered the

period of August 1981 to mid-Febru-ary 1982, the follow-up essentiallyevaluated the early version of theMYEC program that the program per-sonnel themselves had found unsatis-factory.

A second experimental evaluationof the MYEC project had been begun.A job readiness pretest covering theskills thought necessary for emplOy-ment success had been developed byproject staff. This instrumentwould have served not only as a

experimental pretest, but as a checkon the program's success in provid-ing the skills it deemed important.This second experiment was to beginin October, 1982, but was delayedseveral times. It had begun onNovember 30, 1982, but the discon-tinuation of the project by the Off-ice for Juvenile Justice and Delin-quency Prevention aborted the secondevaluation.

The loss of the second evaluationis unfortunate because of the majorchanges that the MYEC project hadundergone. Accordingly, the evalua-tion presented here is an evaluationof the program in development, notas developed. We know that a MYECprogram based on one-to-one counsel-ing was not successful; we do notknow if the project modificationsproduced a more successful set of

interventions.

Evaluation Results for the Job ScoreClass

In the spring of 1982, the JobScore class was begun in four Mil-waukee high schools. Together withthe project staff, we attempted toimplement two successive evaluationsof the Job Score Classes. A

JVS

preliminary experiment was attemptedin all four schools, but the designsbroke down in three of the fourschools: control students were notpretested (or identified), and ran-domization was not actually imple-mented because randomization tookplace before students in the pool ofeligibles were asked whether they

wished to be in the class; many didnot. In one of four schools random-ization did take place, but in thatschool the collection of follow-updata at the end of the-semester wasincomplete. Of the 21 studentsassigned to treatment and 18 to con-trol, complete data were only avai-lable on 12 (57%) of the treatment

'students and 7 (39%) of the controlstudents. Results of such a follow-up are virtually uninterpretable.The data were analyzed nonetheless,and the treatment and controls didnot differ on any outcome measure:Internal Control, Self-esteem,

_ _ _ _ _Interpersonal Competency, RebelliousAutonomy, Alienation, AttitudeToward Education, Employment Status,or Drop-out.

-527-

Having learned from their firstexperience, the experimental evalua-tion of the Job Score class in fourhigh schools in the fall of 1982 wasmuch better implemented. Randomiza-tion was carried out properly, and

pretests on personality and job

readiness skills were administeredto all treatment and control stu-dents. Unfortunately, the experi-ment was terminated along with theproject when OJJDP decided not to

continue funding. The MilwaukeePublic Schools were unwilling to

complete the follow-up without the

funding provided by OJJDP.

In short, we know little aboutthe effectiveness of the Job Scoreclass. This is particularly unfor-tunate because an enormous amount ofcareful work went into the develop-ment of its curriculum, and because

416

JVS

of the interest the project'ssponsor had shown in developing thisaspect of the project. One of thegoals of the Alternative EducationInitiative was to develop innovativealternative educational approachesfor youth; the Job Score curriculumseems to have been aimed at thatgoal.

Evaluation Results for the ReturnCenter

The Return Center did not beginoperation until January, 1982. At

that time MYEC personnel were begin-ing the follow-up of the MYEC exper-imental treatment and control sub-jects. This follow-up was not

progressing well, and we recommendedthat MYEC focus its energies on com-pleting the evaluation of the jobcounseling program aleady in pro-gress. One good evaluation wasjudged to be more worthwhile thantwo bad ones. In addition, sincethe Return Center was'a brand newintervention, we felt that it shouldbe allowed to operate for some timeand work the kinks out before beingsubjected to formal evaluation.Thus, no evaluation for the ReturnCenter was designed for the springof 1982.

During the August, 1982 confer-ence, an evaluation of the ReturnCenter was designed for implementa-tion in the fall of 1982. Thedesign involved tracking the first125 referrals to the Center for boththe first and second semesters of

1982-83 to see how many actually didre-enroll in some kind of schooling,and what kind of progress they hadmade by the end of that semester. Acomparison would be made between thesuccess of youth referred to MPSalternative education sites, andyouth . referred to CBO-sponsoredtraining (mostly G.E.D. instruc-tion). Since referral was largelybased on geography--youth were

referred to a program close to wherethey live--the design provided a

provocative comparison of "official"alternative education versus CB0-sponsored alternative- education.

The evaluation was never com-pleted. The evaluation began welland was progressing nicely when theproject's funding was discontinued.Pretest and background informationon the first 125 referrals had beencollected, and follow-up forms hadbeen designed and approved by JVS,MPS, and the CB0 referral sites.All information was being providedas requested. Because of the pro-

'4ject's termination, we know nothingabout the efficacy of the ReturnCenter.

Recommendations

The evaluation of the JVS Alter-_

native -Edudation-Pfoje-dt,--affd- ourinterpretation of it, lead to the

following recommendations:

-528-

1. The Job Score class appearsto be a carefully developed curricu-lar innovation worthy of dissemina-tion and testing. Procedures shouldbe established to examine its effec-tiveness.

2. Delinquency prevention pro-jects involving individual counsel-ing should be staffed with profes-sionally trained counselors. Suchcounselors should, at a minimum,meet the guidelines or standards forcounselors as laid outby the Ameri-can Personnel and Guidance Associa-tion, the American PsychologicalAssociation, the Association of Mar-riage and Family Counselors, or thelaws of the state within which theproject operates. Paraprofessionalsproviding counseling services shouldbe closely supervised by profession-als. Counseling interventionsshould be rigorously evaluated.Funding agencies should require that

417

such minimum standards be adhered toby projects they sponsor.

3. Experimental research, shouldbe conducted on the consequences ofthe employment of dropout and poten-tial dropout youth. We should nothave to rely on correlational meth-ods to answer policy questions thatwould be better answered experimen-tally.

4. Experimental research shouldbe conducted on the outcomes ofreturning dropout youth to school.We have long assumed that "the moreeducation, the better." Someresults presented here question thisassumption. Under what conditionsis it beneficial to return youth toan environment in which they met

with failure and frustration?

5. Experimental research shouldbe conducted on the net effects ofjob search assistance programs suchas MYEC. Are emplOyment slotsfilled more quickly, or with moreproductive personnel? Or do employ-ment assistance projects simplyrearrange the employment-seekerqueue? What happens to the personsthat would have landed the jobs? Is

there any net benefit to society

during a recession when there is alarge labor surplus, and competitionfor jobs is fierce?

Epilogue

MYEC and the JVS project in gen-eral were exemplary in their will-

JVS-

ingness to subject themselves to

rigorous evaluation. Evaluation isrisky--it exposes oneself to theverification of failure. But this

risk is mitigated when evaluationresults are used to produce programimprovement, and an ultimatelysatisfactory evaluation. This is

the essence of the Program Develop-ment Evaluation process. It is

unfortunate that the present reportis a summative evaluation--it wouldhave been better construed forma-tively.

It is much to ask of a newly con-ceived program that it demonstrate

'effectiveness in its first or secondyear of operation. Even privateenterprise ventures routinely oper-ate at a loss early in their his-tory; this is considered normal, andunnoteworthy. Yet social serviceprograms, addressing large problemsin small ways, are expected to

demonstrate efficacy almost immedi-ately. There can be no place forsloth or useless projects; there canalso be 'no place for pie-in-the-skyexpectations for instant success indifficult areas. What is needed issystematic attention to the improve-ment of programs over time. Our

nation must have the courage and

commitment to provide the supportthat will allow program developmentto take place in a spiral of

improvement over time. Projects todesign, build, and test ballisticmissle systems can take decades--whywould less time be required to pre-vent youth crime?

-529-

418

JVS

References

Alwin, D. F. & Hauser, R. M. The decomposition of effects in path analysis.American Sociological Review, 1975, 40, 37-47.

Clark, K. B. & Summers, L. H. The dynamics of youth unemployment. In YouthKnowledge Development Report 2.9. Research on Youth Unemployment andEmployability Development: The Youth Employment Problem--Dimensions,Causes and Consequences. Washington, D.C.: National Bureau of EconomicResearch, May, 1980. Available from the Superintendent of Documents, U.S.Government Printing Office, Washington, D.C., 20402.

Comer, J. The education of inner-city children.. Grants Magazine, 1980,3,20-27.

Cook, T. D., & Campbell, D. T. Quasi-experimentation. Chicago: RandMcNally, 1979.

Gold, M. Scholastic experiences, self-esteem, and delinquent behavior: Atheory for alternative schools. Crime and Delinquency, 1978, 24, 290-308.

Gottfredson, G. D. Models and Muddles: An ecological examination of highschool crime rates. Journal of Research in Crime and Delinquency, 1979,16 307-331.

. _ _ _ _______

Gottfredson, G. D., & Cook, M. S. A cognitive theory of person-environmentinteraction with implications for social control. In Program in delin-quency and school environments: Program Plans, 12/1/82 - 11/30/84. Balti-more: The Johns Hopkins University, CSOS, 1982.

Greenberg, D. Crime and the age structure of society. Contemporary Crisis,1977, 1, 189-223.

Greenberger, E. A researcher in the policy arena: The case of child labor.American Psychologist, January, 1983, 1, 106-111.

Greenberger, E., & Steinberg, L. D. Adolescents who r -ork: Health and beha-vioral consequences of job stress. Developmental Psychology, 1981, 17,691-703.

Hirschi, T. Causes of delinquency. Berkeley, Calif.: University of Califor-nia Press, 1969.

Yamasaki, C. The Milwaukee Youth Employment Center. In G. D. Gottfredson(ed.), The School Action Effectiveness Study: First interim report (ReportNo. 325). Baltimore: The Johns Hopkins University, CSOS, 1982.

Table 1

Post-Randomization Checks on the Equivalency of the MYECTreatment and Control Groups, 1982

MYEC Control

pSD N M SD N

Age (years) 16.81 .67 247 16.70 .72 149 .10

Sex (male=1) .67 -- 252 .63 -- 154 .30

Month enrolledain program

3.90 1.58 252 3.98 1.52 154 .60

Days from enrollmentto follow-up

219.79 34.94 184 215.63 44.00 98 .39

Interpersonal .89 .14 240 .88 .15 144 .58

Competency

Psychological .73' .21 245 .70 .22 145 .21

Health

Rebellious .61 .26 240 .69 .24 145 .006*

Autonomy

aThe first month of the experimental evaluation (August, 1981) is coded"1," the second month (September) is coded "2," etc.

* Statistically significant.

Table 2

Mean Post-Test Psychosocial Development Scoresfor '1YEC Treatment and Control Groups, 1982

Pi

MYEC

SD N M

Contol

SD. N

Interpersonal Competencya -.01 .19 176 -.02 .19 101 .65

Psychological Healtha .02 .43 180 .05 .26 101 .26

Rebellious Autonomya -.02 .30 176 -.06 .28 102 .27

Attitude towrdEducation pc 2.79 .70 185 2.75 .71 108 .95

aAverage change from pretest to posttest.

bAverage rating for each item; scale for items ran from 1 - 5.

ciThis is an after-only score, ie., no pretest is available.

-532- 421

Table 3

Percentage of MYEC Clients and Randomly EquivalentControls Reporting Favorable Employment

and Education Outcomes

Outcome MYEC Controls

Employed 18 21 .68

In school 17 26 .10

Favorable outcome 36 47 .07

(N) (172) (106)

Note. Source is response to the question, "Whatare you doing mostly?" Response rates to thisitem are nearly equal for the two groups: 68% for}NEC, 69% for the control group. Favorable out-come means the respondent was employed or enrolledin school (or both).

Table- 4

Iktrcontoge of KYZC Clients and Randomly EquivalentCoolest' Lporting Various Current Activities

at follow-Up

10.011WW,WWIDW MMMMM

ktivity MTEC Controls p21.0.WWMUM MM W MM MIIMIPWOW

% kiPG fir psy 13 30 .27

(166) (90)

TakCag vocatioast 8

courses (159)

Gradested free high school 9

or coopleted C.C.D. (196)

Astsodina a CEO program 25

(163)

Worollod is or attooding 5

high *shoot (157)

told off Iron work 1

(155)

Geld 69

WOWW0WWWWWWW.WWWWWW1

11 .57

(85)

.9 .99

(114)

39 .04

(88)

14 .02

(81)

10 .002

(81)

57 .05(175) (97)

Ipm. Source of most percentages are responses toa **Rills-option quostion, "What are you doing?"ilsopeodoits mad mark as many alternatives asoppiiod. In a 'sprat* question respondents wereasked about high school graduation. O's differ

free 'with* to quipstloe because not all respon-dents fellosted as instruction to *ark one res-pease for oath lt** in the multi-option question.

pa eostbesos *We tiro number of respondents to eachStart totst X for MEC 0 252, total M for controlgywp * 154.

-534

423 r

Table 5

Treatment and Control Group Comparisons on EmploymentOutcomes and Self-Reported Arrests, 1982

MYEC Control'

M SD N N SD -11

Current hours workedper week

3.00 9.89 239 3.84 12.24 146 .46

Current weekly wage 11.54 33.72 187 17.00 43.78 109 .23

Total earnings lastsix months

174.58 592.38 189 160.76 509.05 112 .84

Hours worked full-timelast six months

1.31 4.73 202 _ 1.27_ 4.12_ 117.._ _ .95_

Hours worked part-timelast six months

1.92 6.34 202 1.77 5.18 116 .83

Current hourly wage .58 1.50 192 .67 1.39 114 .57

Transformed currentahourly wage

.58 1.49 187 .72 1.69 109 .47

Transformed wagesalast six months

.75 2.22 189 .92 2.36 112 .55

Number of times pickedup by police,last six months

.27 .69 185 .39 .89 109 .18

aBecause of extreme skewness in the earnings data, the hourly and total wage

variables were subjected to a logarithmic transformation.

-535-

424

Table 6

NYEC Program Implementation Measures, August, 1982to February, 1982

Implementationmeasure Mean Median Mode SD Range

Received no treatment

Counseling contactsa 4.47 3.54 0,0 4.59 0-31 48 19

Educational servicescontact 3.43 0.20 0.0 8.96 0-60 171 68

Job referrals 1.58 0.43 0.0 2.69 0-21 129 51

Job placements 0.30 0.16 0.0 0.59 0-3 182 72

Note. N=252 for all measures.

aIncludes personal meetings, and phone calls

-536- 425

Table 7

Direct Contributions and Incremental Validity of a MYECImplementation Measure to the Explanation of Work Outcomes

According to Parsimonious Causal Models

Incremnt p forPredictor beta to R increment

Hours currently working per week (N = 198)

Ethnicity (white = 1) .19 .03 .009*

Age .14 .02 .05*

MYEC participation .16 .03 .02*

Weeks worked full-time in last six months (N = 169)

Ethnicity (white = 1) .24 .06 .001*

Month enrolled inprogram

-.21 .04 .006*

Age .03 .01*

MYEC participation .02 .00 .75

Weeks worked part-time in last six months (N = 169)

Interpersonal Competency -.31 .10 .001*

Month enrolled in program -.15 .03 .04*

MYEC participation .15 .02 .04*

Log-transformed current wages (N = 155)

Ethnicity (white = 1) .17 .03 .03*

MYEC participation .23 .05 .03*

Log-transformed total earnings in last six months (N = 157)

Interpersonal Competency -.23 .05 .003*

Ethnicity (white = 1) .21 .04 .008*

MYEC participation .16 .02 .04*

Note. The components of the overall MYEC intervention were implemented todifferent degrees for different members of the treatment group. The incre-

ment to R2 for MYEC participation shows the proportion of the variance ineach outcome accounted for by a composite measure of participation in vari-ous MYEC treatment components. MYEC was more fully implemented for stu-dents participating more fully in the various interventions. A positivebeta for MYEC participation implies that persons who participated morefully in MYEC had higher scores on the particular outcome measure in ques-tion, net of measured pre-existing personal characteristics. Variableslisted in each set above MYEC participation are those pre-test measuresthat significantly contributed to the prediction of each outcome.

* Statistically significant.

-537- 426

Table 8

Direct Contributions and Incremental Validity of a MYECImplementation Measure to the Explanation of Psychosocial Outcomes

and Arrests According to Parsimonious Causal Models

IncremntPredictor beta. to R

Psychological Health (N = 157)

p forincrement

Psychological health .47 .22 .001**Basic skills .20 .04 .006**MYEC participation .10 .01 .14

Interpersonal Competency (N = 156)

Interpersonal, Competency .18 .03 .02**

MYEC participation -.04 .00 .58

..,

Rebellious Autonomy (N = 155)

Rebellious Autonomy .40 .16 .001**MYEC participation -.13 .01 .08*

Re-enrollment in formal schooling (N = 141)

Psychological Health .24 .06 .005**MYEC participation. .16 .03 .003**

Times arrested last six months (N = 158)

Gender (female = 1) -.20 .04 .01**

MYEC participation -.12 .01 .13

Attitude towards Education (N = 156)

Psychological health .20 .04 .01**

Ethnicity (white = 1) -.17 .03 .03**

Basic skills .16 .03 .04**

MYEC participation .13 .01 .10*

Note. The increment to R2 for MYEC participation shows the proportion ofthe variance in each outcome accounted for by a composite measure of parti-cipation in various MYEC treatment components. A positive beta for MYECparticipation implies that persons who participated more fully in MYEC hadhigher scores on the particular outcome measure in question, net of mea-sured pre-existing personal characteristics. Variables listed in each setabove MYEC participation are those pre-test measures that significantlycontributed to the prediction of each outcome.

* Strong trend.** Statistically significant.

-538- 427

Table 9

Direct Contributions and Incremental Validity of Participationin MYEC Treatment Components to the Explanation of Work Outcomes

Controlling for Client Background and Pretest Measures

Incremnt p for

Predictor beta to R increment

Hours currently working per week (N = 198)

Counseling .06 .00 .44

Educational services .08 .00 .25

Job referral .00 .00 .99

Job placement .24 .05 .001**

Weeks worked full-time in last six months (N = 169)

Counseling .04 .00 .91

Educational services -.04 .00 .61

Job referral .0j .00 .65

Job placement .09 .00 .23

Weeks worked part-time in last six months (N = 169)

Counseling -.13 .01 .09*

Educational services .34 .12 .001**

Job referral .01 .00 .94

Job placement .11 .01 .13

Log-transformed current wages (N = 155)

Counseling -.06 .00 .48

Educational services .20 .04 .01**

Job referral -.02 .00 .83

Job placement .28 .08 .001**

Log-transformed total earnings in last six months (N = 157)

Counseling -.09 .00 .31

Educational services .22 .05 .004**

Job referral .13 .01 .08*

Job placement .15 .02 .07*

Note. The increment to R2 for participation in a MYEC component shows the

proportion of the variance in an outcome accounted for by participation inthat component. A positive beta implies that persons who participated inthat component had higher scores on the particular outcome measure in ques-tion, net of measured pre-existing personal characteristics.

* Strong trend.** Statistically significant.

-539- 428

Table 10

Direct Contributions and Incremental Validity of Participationin MYEC Treatment Components to the Explanation of Psychosocial

Outcomes and Arrest Controlling for Background and Pretest Measures

IncremantPredictor beta to R

Psychological Health (N = 157)

Counseling -.05 .00

Educational Services .05 .00

Job referral -.01 .00

Job placement .16 .03

Interpersonal Competency (N = 156)

Counseling -J.§ .03

Educational Services .02 .00

Job referral .09 .00'

Job placement .05 .00

p forincrement

.51

.49

..93

.02*

.02*

.83

.31

.55

Rebellious Autonomy (N = 155)

Counseling -.07 .00 .38

Educational Services -.06 .00 .44

Job referral -.02 .00 .80

Job placement -.10 .00 .18

Re-enrollment in formal schooling (N = 141)

Counseling .12 .01 .18

Educational Services .23 .05 .001*

Job referral .10 .01 .22

Job placement I .11 .01 .19

Times arrested last six months (N = 158)

Counseling .01 .00 .90

Educational Services -.08 .00 .33

Job referral -.03 .00 .74

Job placement -.13 .01 .10*

Attitude towards Education (N = 15(i)

Counseling -.13 .01 .12

Educational Services .07 .00 .39

Job referral -.09 .00 .34

Job placement .17 .03 .03*

Note. The increment to R2 for participation in a MYEC component shows the

proportion of the variance in an outcome accounted for by participation in

that component. A positive beta implies that persons who participated in that

component had higher scores on the particular outcome measure in question, net

of measured pre-existing personal characteristics. The results for Attitude

toward Education must be interpreted with special caution because this atti-

tude was not measured at pretest.

* Statistically significant. -540- 429 _

Table 11

Correlations between Extent of Participation inMYEC Project Components and Ethnicity (White = 1)

(N = 238)

.

Treatment component !ft r p

Counseling contacts -.12 .04*

Educational services contacts 7.11 .04*

Job referrals .01 _.43

Job placr.ments .16 .006*

* Statistically significant.

-541-

430

Table 12

Direct Contributions and Incremental Validity of ExplanatoryVariables in a Model of Psychological Health

at Follow-Up

Incrment p forPredictor r beta to R increment

Statistically significant prddictors

Psychological Health (pre) .41 .31 .17 .001*

Basic Skills Pretest .26 .23 .04 .01*

Rebellious Autonomy (pre) -.21 -.20 .03 .02*

Participation in formal -.23 -.17 .03 .04*

schooling (concurrent)

Concurrent variables not significant net of pre-test measures

Hours per week currentlyworking

.10 .07 .00 .37

Weeks worked full-timelast six months

.05 .04 .00 .61

Weeks worked part-timelast six months

-.01 .01 .00 .85

Logged current wages .08 .08 .01 .28

Logged total earningslast six months

-.03 -.04 .00 .58

Note. The model was constructed by allowing significant pre-test orbackground predictors of Psychological Health at follow-up to enter theequation first, and then entering each of the concurrent work andschooling variables into the equation. Only participation in formalschooling significantly adds to the explanation of Psychological Healthwhen pre-test measures are controlled for. Due to differential patternsof missing data, the models used to estimate the regression parametersfor the non-significant regressors vary slightly from those for whichthe parameters are shown here. The differences are trivial. N's rangefrom 132 to 156.

* Statistically significant.

-542-

431

Table 13

Direct Contributions and Incremental Validity of ExplanatoryVariables in a Model of Interpersonal Competency

at Follow-Up

Incrgment p forPredictor r beta to R increment

Statistically significant predictors

Interpersonal Competency .21 .21 :05 .01*

(Pre-test)

Participation in formalschooling (concurrent)

-.25 -.24 .06 .005*

Concurrent variables not significant net of pre-test measures

Hours per week currentlyworking

.09 .09 .01 .27

Weeks worked full-timelast six months

.01 .03 .00 .75

Weeks worked part-timelast six months

-.07 -.01 .00 .92

Logged current wages .04 .07 .01 .38

Logged total earningslast six months

.06 .11 .01 .19

Note. The model was constructed by allowing significant pre-test orbackground predictors of Interpersonal Competency at follow-up to enterthe equation first, and then entering each of the concurrent work andschooling variables into the equation. Only participation in formalschooling significantly adds to the explanation of Interpersonal Compe-tency when pre-test measures are controlled for. Due to differentialpatterns of missing data, the models used to estimate the regressionparameters for the non-significant regressors vary slightly from thosefor which the parameters are shown here. The differences are trivial.N's range from 131 to 155.

* Statistically significant.

-543- 432,1.' I

Table 14

Direct Contributions and Incremental Validity of ExplanatoryVariables in a Model of Rebellious Autonomy

at Follow-Up

Incrqment p forPredictor beta to R increment

Statistically significant predictors

Rebellious Autonomy (pre) .40 .40 .16 .001*

Weeks worked full-timelast six months

.15 .15 .02 .04*

Concurrent variables not significant net of pre-test measures

Hours per week currentlyworking

.14 .12 .01 .17

Weeks worked part-timelast six months

.01 .01 .00 .85

Logged current wages .09 .08 .01 .33

Logged total earningslast six months

.16 .12 .01 .12

Participation in formalschooling

.08 .12 .01 .15

Note. The model was constructed by allowing significant pre-test orbackground predictors of Rebellious Autonomy at follow-up to enter theequation first, and then entering each of the concurrent work andschooling variables into the equation. Only participation in formalschooling significantly adds to the explanation of Rebellious Autonomywhen pre-test measures are controlled for. Due to differential patternsof missing data, the models used to estimate the regression parametersfor the non-significant regressors vary slightly from those for whichthe parameters are shown here. The differences are trivial. N's range

from 130 to 154.

* Statistically significant.

-544-

433

Table 15

Correlations between Personal Characteristics Measuredat Pre-test and Follow-up with Number of Self-Reported

Arrests

Personal characteristic

Measured at pretest

r N

Age -.13* 285.

Sex (female = 1) -.22** 288

Ethnicity (white = 1) .03 291

Month enrolled in program .00 294

Basic skills test -.01 179

Psychological Health .00 283

Rebellious Autonomy -.02 284

Interpersonal Competency -.04 280

Extent of participation in MYEC -.12* 287

Measured at follow-up

Hours worked in past week .11 273

Full-time employment .04 292

Part-time employment -.01 291

Log-transformed current wages .00 269

Log-transformed totalearnings last six months

.01 274

Re-enrollment in school -.08 259

Psychological health -.10 280

Rebellious Autonomy -.04 277

Interpersonal Competency -.14* 278

-545- 434

JVS

Appendix A

Items in Scales Used to Measure Psychosocial Developmentin the MYEC Evaluation

Psychological Health

Scoring ItemOthers see me as a loser.It's pretty tough to be me.No matter what I do, it's not going to make any difference.I feel I do not have much to be proud of.These days I feel I'm just not a part of things.I don't know whom I can count on these days.I have little influence over what happens to me.Good luck is more important than hard work.

Pretest alpha = .61Posttest alpha = .59Retest reliability = .40

It

Interpersonal Competency

Scoring ItemI know how to get along with adults.I am the kind of person who can make it if I try.If I want to, I can explain things well.Others see me as successful.I like myself.My friends regard me as a person with good sense.I feel no one really cares what happens to me.

Pretest alpha = .45Posttest alpha = .46Retest reliability = .19

Rebellious Autonomy

Scoring ItemWhat I do with my time is my own business.I don't like anybody telling me what to do.I can get along just fine on my own.What happens to me is my own doing.Nobody has the right to tell me how to spend my money.

Pretest alpha = .49Posttest alpha = .49Retest reliability = .39

546 435