A Service Delivery System for Consumer Implementation and Evaluation James J. Reisinger, Ph.D. State Bureau of Research and Training - Mental Health Eastern Pennsylvania Psychiatric Institute Harrisburg, Pennsylvania and Children and Youth Services Pennsylvania Office o f Mental Health

Evidence supporting the feasibility of training nonprofessionals to administer therapeutic techniques has been collected for in excess of a decade. Included among successful demonstrations have been hospital attendant staff (Atthowe and Krasner 1968; Reisinger 1972), peers (Drabman and Spitalnik 1973; Cash and Evans 1975), and parents (Whaler 1969; Hall, Axelrod, Tyler, Grief, Fowler, and Robinson 1972).

In a participant capacity, parents might well be viewed as consumers of mental health, retardation, and special education services. And an accumulating literature testifies repeatedly that parents can be trained to perform as principle behavior change agents for their children (O'Dell 1974; Reisinger, Ora, and Frangia 1976). Still it is common knowledge that parents rarely have occasion to learn and to implement their skills within service programs. In fact, they have practically no control regarding the evaluation of service systems which are designed to meet the needs of their children.

Comprehensive systems of service delivery whose design includes active and direct parental involvement are relatively sparce. Perhaps the most notable one is offered by filial therapy (Guerney 1964; Andronico, Fidler, and Guerney 1967) which has made the transition from university to community (Ginsberg 1976)o While behavior modification studies abound in variations on the theme of parents affecting child behavior, model systems implemented with parent personpower have been slow to emerge. The distance between innovative technique investigations and applications of those procedures in community treatment contexts remains seemingly infinite.

This article1 describes one multifaceted behavioral system of service delivery operated primarily through parent-consumers for their preschoolers. The system, originally established within a college environment, 2 has been subsequently replicated within community surrounds. 3

23

Rationale The general design for the Holy Spirit Hospital Early Intervention

Program (EIP) originated in 1968 via its prototype, the Regional Intervention Project (RIP) of Nashville, Tennessee. Within a few years, that model was soon selected as one of a dozen exemplary programs for emotiohally disturbed children and has since received further recognition by the American Psychiatric Association as a 1976 Gold Medal recipient for innovative service delivery. The Holy Spirit Hospital program began in 1973 as a Pennsylvania "replica- tion" of the Tennessee model. Formulation of the system followed the strategies of measurement and accountability. That is, the objective measurement of parent-child progress, paralleled with the accountability of trained parent-consumers, provided the basic organizational development for EIP.

In addition to a data-based emphasis, the conceptual scheme of the program has attended to a variety of long-recognized issues within the purview of community mental health and special educa- tion service systems. A review of some issues and the EIP response to them follows.

1. Funding While children have commanded a high degree of attention

concerning their mental health needs, they have also historically retained a low priority when funding allocations were finalized. Therefore, on the programmatic level, with costs for intervention services on the increase, the survival of programs for children depends upon maximizing resources which limited funds can purchase. One alternative is for service programs to delineate tasks which require professional staff from those tasks which can be accomplished by nonprofessionals. Numerous technician functions could be organized and completed, thus saving funds. Professionals could also utilize their time more effectively in responsibilities which demand their levels of expertise. While costs for many one-hour professionally delivered therapy sessions may range from $15 to $60, EIP services can be delivered at a cost far less than that of the traditional format. Such cost reduction is feasible because the model utilizes a core professional staff whose responsibilities include training and supervision of parents. Parents themselves subsequently function as the primary service providers for their own children, and costs-per-family decrease mainly in an inverse relationship to the number of families receiving and completing the training program.

2. Technology The technology necessary to improve the delivery of adequate

mental health and special education services has appeared in professional literature for at least ten years (i.e., knowledge of behavior change procedure is a decade beyond the actual organized field applications of the techniques). Service delivery programs

24

continue to rise in the form of so-called "novel demonstrations," while the systematic utilization of previously tested technology remains unknown to many practitioners. The technology of EIP is an illustration of how clinical procedure, investigated in many clinics and universities ai;ross the country, can be incorporated into a service delivery system. Thus, EIP cannot claim innovation on the basis of new technology because these investigations were made years ago by applied behavior analysis researchers. The element of uniqueness to be attributed to EIP and to RIP lies in their successful transfer of research techniques from the laboratory to coordinated community applications.

3. Professional Personpower Shortage Another issue of longstanding concern revolves around the

professional personpower shortage. Albee and Dickey (1957) report- ed that the future training of mental health and health care profes- sionals would continue to lag behind actual need. The limited availability of training slots for professionals presently demonstrates that demand for services almost consistently exceeds ability to sup- ply them. Thus, despite warnings of nineteen years ago and numerous repetitions since (Albee 1959; 1965; 1968), relatively few widespread modifications have occurred within service delivery approaches to adapt the personpower shortage. Recently, with eco- nomic and political constraints added, the personpower issue has been made more complex.

The EIP model, in response to this problem, utilizes a core of professionals as trainers, consultants, evaluators, and supervisors instead of as immediate providers of service. Thus, professional time is allocated more efficiently and parent learning of child-rearing skills is facilitated through training applications. The prime source for personpower to deliver services becomes the trained parent: the mother who daily molds her child's development over the many hours they will share. The use of consumers to provide their clinical and educational services for their own children also virtually eliminates waiting lists. In practice, the EIP system has found that the more parents who seek service, the more who are trained, and consequently, the more resources there are for providing the services. This is predicative since as the population which demands service increases, so does the resource for that service provision.

4. Diagnostic Classifications The use of diagnostic categorizations to classify various emotional

and educational difficulties is well established throughout mental health and special education service systems. Indeed, in agreement with Hobbs (1975), categorization of problems is essential for the formulation of research leading to empirical conclusions for more appropriate treatment strategies. The issue is not whether diagnostic classifications can serve a purpose, but whether the current system

25

is pragmatically justified in that advantages exceed disadvantages to the child. The prevalent system has uses but one proves not to be its functional contribution to treatment decisions. More specific defi- nition is necessary than is provided by terms such as "'unsocialized aggressive reaction of childhood." Thus, while such diagnostic classifications may accompany a child upon entry into EIP, subse- quent treatment strategy avoids them and focuses upon the behavior which led to the child's identification. This is not to imply a dismissal of possible physical etiology for a portion of the population, but to establish a functional measurement approach for child progress. In no manner does utilization of this measurement approach proport to resolve difficulties associated with general formulation of a more beneficial diagnostic classification system. It simply decreases some negative aspects of the current system.

5. Therapeutic Technique The traditional approach to therapy involves the standard "one

hour" appointment scheduled from one to four times per month. By definition, intensity and in-situ application of therapeutic technique are relatively infrequent. Most likely, within one month, the parent may be given encouragement, little practical direction concerning ideas for remediation, and probably no opportunity to utilize problem-solving technique applications. Also, the time consumed in a traditional format for service delivery is very likely to exceed a few months and may continue for years.

EIP provides services four full mornings per week, both for individual and group sessions. The intensity of treatment applications within EIP is important for two principle reasons. First, parents can learn more efficiently under frequent contact with training materials. Second, effective modification of behavior patterns occurs with intensive application simulating real-life conditions (e.g., handling a tantrum). Most families can complete individual training in a maximum of four to six weeks. The gains of training then prepare parents to be of assistance to new parents entering the program.

6. Instructional Techniques Application of treatment procedures with mother-child pairs is an

integral part of parent training within EIP. The available literature suggests that there is a distinct difference between parent compe- tence on the knowledge level of techniques and the ability to apply those techniques. Training parents by book, lecture, and discussion has been minimally effective for dealing with actual problems. More useful training has incorporated observations of parent-child treat- ment applications, modeling such applications, and on-line feedback to correct and support proper technique applications. A recent study (May 1975), providing a comparison of four instructional techniques for parents, reported that ability to apply certain intervention procedure was superior employing modeling or modeling and

26

roleplaying in contrast to written or lecture teaching approaches. The EIP model thus adheres to the simulated application approach in training throughout the program.

7. Effective Treatment A most basic value of a service delivery program lies in its

maintaining treatment effects from the clinic to "real life" commu- nity settings. Studies (Brodsky 1967) and daily experience have shown that problems resolved in the therapist's office may not remain resolved beyond the office situation. Indeed, if a treatment procedure is effective only in the presence of a professional, it is practically worthless to the client. For EIP, the need for planned generalization of treatment effect is a priority. The training approach is expressly programmed to have parents learn techniques sufficient- ly to utilize them across situations and environments. Parent-child interactions are shaped and strengthened in the clinic with planned emphasis upon their transfer to the home, school, stores, and so forth. Assurance for generalization of training and interactions is based upon incorporating naturally occurring positive reinforcers in the family's daily life (i.e., training is designed to assist the family in realigning contingencies so that the family can know that it is purposefully self-controlling its own successes in parent-child interactions).

Program Operations The children composing the EIP population range in age from 1.5

to 6 years, the average being approximately 3.5 years old. They enter the program from a diverse referral base extending from pediatricians to self-initiated inquiries. A child may enter EIP with as many as five different and conflicting diagnostic classifications; none of which suggest treatment direction. EIP adopts a focus upon child behavior appropriate to and necessary in the community. By tracking behavior change throughout training phases, parent and child progress are continuously measured and documented for staff as well as for parents.

Three master-level persons and one bachelor-level person comprise the professional resource staff and represent the disciplines of psychology, education, and speech. Aside from a secretary, all other positions are part-time and occupied by trained nonprofessionals, volunteers, and mothers. In a continuing flow, parents first receive services and then assist in delivering similar services under profes- sional supervision. Therefore, the core resource staff functions to provide clearly defined structure in which trained parents help themselves. In summary, of the fifteen regular positions within EIP, five are full-time, five are half-time, and five are half-time volunteers.

The service structure of EIP involves the program's division into distinct elements termed "modules." A module is that element of

27

system which is intended to operate, with its own resources, in order to meed a specific need of the child and parents. Thus, a family may proceed through one or all modules as determined by need. The organizational model for the EIP system is abstracted from the approach known as "management by objectives" (Reddin 1970).

The Intake module provides families with an opportunity to discuss program requirements, to observe training sessions, and to discuss EIP with parents already receiving services. This allows staff to observe parent-child interaction and to secure further definition of the primary difficulty. After a decision is made as to a first priority for the child, the family will usually proceed to either the Individual Tutoring or Toddler Management module.

The Individual Tutoring module is intended to produce functional speech in the child by means of training his parents to teach him and also to directly increase his repertoire of adaptive behavior. Depending upon the child, treatment may begin with eye contact; motor, vocal, or verbal imitation; naming of objects; or more complex tasks. The child's mother begins by learning to record data on her child's progress. Within a few sessions, the mother becomes the teacher. After she has gained confidence in teaching her child, she may begin to teach him at home each day. She may also then teach her husband as she herself was taught by starting him on scoring home sessions. The child is now learning at home while his mother demonstrates progress and confers on procedures in the clinic as determined by her own and her child's needs. Once skilled in these procedures, many parents can not only teach skills like toileting and dressing, but habitually abide by the rule for generalization of learning: "any desirable behavior that the child learns anywhere is thereafter required and reinforced everywhere." The basis for technique applications was taken mainly from the research of Lovaas, Berberich, Perloff, and Schaeffer (1966), and Risley and Wolf (1967).

The Toddler Management module is designed for children whose behavior is characterized by tantrums and unmanageability. Although some of these children display biological influences, organic impairment is seldom the predominant causal factor of their behavior pattern. In this module's operation, parent and child participate in twenty-minute structured play situations. Through the stages of Baseline, Intervention I, Reversal, and Intervention II, parent-child interaction is analyzed daily by data collection. Parents are taught the use of differential reinforcement with additional techniques as are necessary. Procedures for clinic training and observation were adopted from the work of Wahler. 4

After completing the individual sessions the child enters the Intake Classroom module. The teacher assumes case responsibility for the family and conducts a nursery-level instruction program. However, while instruction is the format, emphasis is placed upon

28

appropriate peer interaction. As basic self-control is developed, a child is proven eligible for the next (~lassroom level.

The Deficit-Remediation Classroom module is operated on the assumption, allowing for maturation variations, etc., that these children either display parallel play and/or cooperative group behav- ior. The focus is educational, with individualized programs and objectives stipulated and measured daily for all children. The program is intended to improve skills in areas such as gross and fine motor control, color and form discrimination, speech, etc.

The final educationally based, oriented component is maintained in the Community Classroom module. Children are taught a mastery of basic skills in accord with age, maturation level, and existing preschool standards. The class employs minimal control techniques and is intended as a transitional parallel to interface with many preschool class programs available in the community. The technol- ogy for all classroom operations was taken from the work of investigators, including Brown and Elliot (1965) and Madsen, Becker, and Thomas (1968).

The Liaison module accepts responsibility for the family as the next step may include assistance in placing the child in the community. Liaison personnel maintain a listing of placements appropriate for various children. They consult with families on available systems and investigate possible placements, as well as coordinate continued home teaching programs begun by other modules. They may also actively assist a teacher in programming for the child placed in her. nurse'ry or kindergarten class. They conduct periodic surveys of children no longer primarily served by EIP. Should the family encounter further difficulties, Liaison is the link back to EIP support or intervention systems. EIP and RIP thus can maintain an open-ended, reciprocal commitment with all families served.

There are a number of other service modules including Nursery, Parent Theory Class, Public Relations, and Visitation. All of the latter program operations are primarily implemented by trained parents.

Evaluation EIP has a major concern with the systematic evaluation of its own

service delivery program. Yet, with its demonstration mission, categorical funding restrictions minimize any allocation for specific research. However, within such constraints, evaluation methodolo- gies still exist, based usually upon the functional analysis of behavior and designed as an integral part of the service-providing mechanism so as to further reduce costs.

The evaluation process is multifaceted. It includes: A, daily collection of data in every service module for every child; B, use of independent consultants from various professional disciplines; C, assessment by the Parent Advisory Committee; D, conduct, and publication of within program studies; and E, formulation of a

29

cost-benefit analysis approach. An elaboration of the evaluation facets follows.

A. The clinical work-record for EIP is composed of dally behavioral progress notes written by parents and professionals and of data from each training session. Figure 1 provides a summary presentation of a completed parent-child training case from Toddler Management. Across sessions and treatment phases, the data indicate defined oppositional or cooperative child behavior and respective attention by the parent. Changes in child-parent behavioral interaction thus serve as the basis for measurement of progress. Similarly, change and non-change displayed here provide a daily feedback system to inform parent and professional whether their intervention strategies are effective. Decisions and modifications in the system are based upon data in place of speculative judgment alone.

J

>

z

O tl.l o~

I o

t d L9 < F - Z li.,l 0

ILl 0...

" BOOTS"

I00 - i

90 - ~ l l

80 , # I

, o

~ , ,o :

' 14

30 '~'~ ,. \ I 20 " " " " <"'A l i

, , i . . . . ?-o ,

O 5 IO 15

CLINIC

r r OPPOSITIONAL CHILD BEHAVIOR o - - - o PARENT ATTENTION TO OPPOSITION

-* -= COOPERATIVE CHILD BEHAVIOR A - - - A PARENT ATTENTION TO COOPERATION

#/ , , ," l:,v,;"

0 - 0 I I I I I

, ,, o , a,,o \

20 25 30 35

2 0 - M I N O B S E R V A T I O N SESSIONS HOME

FIGURE 1: CLINIC DATA CHART

B. On the level of daily program operations, EIP relies upon outside psychological, educational, and medical consultations to best meet individual family needs. In addition, independent specialists study the program for the sake of general assessment of operations on administrative and clinical levels. To date, two such contracted evaluations have taken place; one by a Ph.D. clinical psychologist

30

and the other by an Ed.D. special educator. Their appraisals included information from on-site observations of parent-child training ses- sions to contacts with consumers themselves. Reports from these evaluations were shared simultaneously with staff, county monitoring agencies, and the Parent Advisory Committee.

C. One purpose of the Parent Advisory Committee is to provide recommendations based upon program evaluation data. At least half of the committee consists of parent-consumers, with the remainder being interested community persons. The committee meets monthly and may request any information of program staff, evaluate it, and render a binding opinion regarding the adequacy of program results. Within the context of day-to-day operations, program staff retain full authority for the manner in which data is collected for the committee. Thus, program personnel may develop their own methods for accomplishing final outcomes, but the entire system is nevertheless accountable for its results to the persons it serves. Consumers, with consultive professional guidance, judge the ade- quacy of services they receive.

In addition, the committee can and does collect its own data independent of program staff. For example, a follow-up question- naire is used for parents who completed or terminated the program. Results are received directly by the committee, with full respect for consumer privacy included. Therefore, parents communicate with parents regarding the pros and cons of operations, and this procedure simultaneously avoids potential biasing effects that could result were staff conducting the assessment. The advisory commit-

tee then periodically informs the professional staff of continuous consumer reactions, translating these into the form of programmatic concerns.

D. Within-program studies refer to applied research endeavors conducted upon specific modules and their effects. Such studies do not substantially modify existing operations but rather take micro- scopic samples of on-going programming and outcomes. They may be implemented by program staff and/or by professionals affiliated with universities, research facilities, and so forth. To date, two such studies have been completed:

1. Reisinger and Ora (in press) studied the interaction of parent-child pairs in the clinic and in the home during Toddler Management training. Conclusions were that in the presence of visible observers, parents were able to generalize training proced- ures across environments, and with less obtrusive observation (i.e., time delayed tape recordings), parents continued to func- tion as change agents for their children. 2. Reisinger, Frangia, and Hoffman (1976) tracked parent-child pairs to examine parental skills in technique application one year after clinic training was completed. Two groups were identified in terms of marital problems versus no such problems. Both groups were able to utilize clinic technique applications; however, the

31

group without marital problems quantitatively exceeded their counterparts. Thus, marital complications did not negate training effects, but were associated with less consistent use of training skills by these parents.

Studies as these are needed to continually examine and improve training programs to effectively facilitate interactions within families.

E. Finally, for evaluation of any and all service delivery models, an essential question is whether costs invested justify outcomes received. EIP has not been cost-benefit analyzed; however, its parent program was so studied by the University of Tennessee. Since EIP is a basic replication of its parent, a summary of RIP's cost-benefit data is deserving of mention.

The approach involved a simple ratio between dollar benefits and costs of the program. If the cost-benefit ratio exceeds one, benefits are greater than costs expended. Costs, over three years of program operation, were $275,000, including 6 percent interest. A total of 158 clients were served, with 64 children being high risks for institutional- ization. From the latter group, ten cases were selected as definite long-term institutionalizations. Given anticipated dates of commit- ment, a 60-year life expectancy, and a projected institutionalization cost of at least $4,000 per year, the ratio was calculated as 3.13. Therefore, using less than 7 percent of the program's population, there was a benefit of three times the cost investment.

Conclusion The EIP conceptual scheme and operations level address numer-

ous practical concerns which have persisted within community mental health applications. These include reducing costs while maintaining quality services, transferring innovative technology from the literature to daily living situations, training parents as change agents, providing intense and immediate services, and planning the generalization of intervention strategies use across environments. Refinement is a continual process, but currently a system exists for service delivery with the emphasis upon "community" mental health.

Still, the EIP model, with theoretical justification and demonstrated practical impact, continues operation for none of the preceeding mental health specific rationale. Instead, it has developed amidst bold administrative position, grantsmanship, and elementary "politi- cal" expediency (Reisinger 1975). Restated, program maintenance is dictated primarily by a trilogy of skills separate from those factors which should logically, in a mental health context, determine its future. Actual criteria, on the funding level, bypass consumer need, accountability, effectiveness, cost, etc., in favor of assumed good intention and "political" expediency. Final decisions for fund awards to service models are rarely influenced by informed consumer or trained parent input. Without user population inclusion at this level, the assurance that systems of service delivery will continue to be largely inadequate seems virtually guaranteed.

32

The problem of design and implementation for systems which actually respond to the needs of many people is vast and multifaceted. While not intended as exhaustive, some of the variables deserving more attention are as follows.

Contingencies which affect the development of prevention pro- grams require examination and realignment. Presently, advocacy efforts for such programs are obvious. However, on the applied level, the survival of a program is insured not by preventing disorders, b'ut by necessarily treating them, in volume. More problems generate more income for more treatment modalities. To conduct an effective program of prevention is therefore to provide additional service for less financial reimbursement. Serious efforts to utilize technology in the prevention of problems are, by definition, negated. Funding streams continue to increase yield to hospitalize persons and to decrease monies to prevent it.

A repeated threat to novel approaches of service delivery is characterized by the inverse relationship between innovation and status quo. Established organizations oppose change. Witness .the constant inabilities of mental health organizations in the face of demonstrated evidence identifying directions for change. Innovation in approach has often proven itself a necessary but insufficient condition for systems change. (Fairweather 1973; Graziano5). Innovative applications which find applause from many professionals may not find similar reception from host organizations who act as sponsors. Indeed, methodologies which even imply change in traditional operations, regardless of multiple benefits beyond the organization, are not likely to be tolerated.

Still, many community programs have found their way into existence to fill identified service gaps with the test of their worth yet to be investigated (Cowen 1973). Recently, there has been growth in the utilization of monitoring selected process variables. Unfortunately, by themselves, they may yield incomplete data followed by incorrect conclusions. Output measures are explained away due to cost factors, absence of suitable criteria, etc. Also, decision-makers of such programs are conspicuous for rarely being accountable to local consumers. Even the more than ten-year history of the community mental health center (CMHC) movement includes little explicit responsibility or accountability to the consumer. CMHC grants awarded to established organizations (e.g., general hospitals) have sometimes insured private controls, fiscal security, and in- creases in processionary puppets (Peter 1972) at the possible expense of consumer need, innovative programs, and output evaluations. Yet without objective assessment, including component analyses for target groups and problems, it is probable that one set of dubious techniques will supplant another (Lorion and Cowen 1976). In the absence of output variables in evaluation, service modalities and entire systems approaches to the delivery of services still await change based more upon whim than upon evidence.

33

At least by their specificity, professional training models may inadvertently establish territoriality (Sanders 1967). In addition, theory has led us to the point where, as in the case of parents, model-based assumptions may be ascribed to in spite of fact (Reisinger, Ora, and Frangia 1976). The potential advantages available in utilization of parents as change agents are documented. Apparently then, in part, a form of professional bias contributes to the exclusion of parents in programs for their own children. Without adjustment llere, the gap will widen between productive university- generated demonstrations of innovation and the relatively unproduc- tive replications of these within community service systems. Production of innovation will then continue in storage of libraries and clearing houses until knowledge exceeds application by twenty or thirty years. Verbiage presently fills the void for childrens' services and consumers' participation. However, formulation of comprehen- sive, functional systems with professionals maximizing personpower resources is yet to be accomplished. It is still true that professionals and consumers are needed in service functions (Hobbs 1964). The planner of such systems would do well to include and/or involve the consumers in all aspects of implementation and evaluation.

34

Footnotes

1 Development of this manuscript was supported in part through Grant #48-75005-21-300, Commonwealth of Pennsylvania Depart- ment of Public Welfare (Office of Mental Retardation) and Education, P.L 89-313, an amendment to Title I of the Elementary and Secondary Education Act.

20ra, J.P., and Reisinger, J.J. "The Regional Intervention Program: A Behavioral Service Delivery System for Preschoolers�9 Paper read at the annual meeting of the American Psychological Associa- tion, September 1971, in Washington, D.C.

3 Reisinger, J.J., and Hoffman, E.H. "Training Parents as Primary Service Providers�9 Paper read at the international convention of the Council for Exceptional Children, April 1976, in Chicago.

4 Wahler, R.G. "Behavior Therapy for Oppositional Children: Love Is Not Enough�9 Paper read at the annual meeting of the Eastern Psychological Association, April 1968, in Washington, D.C.

5 Graziano, A.M. "Clinical Innovation and the Mental Health Power Structure: A Social Case History�9 Paper read at the annual meet- ing of the Eastern Psychological Association, April 1968, in Wash- ington, D.C.

References Albee, G.W. Mental Health Manpower Trends. New York: Basic

Books, 1959.

�9 "Manpower Needs for Mental Health and the Role of Psy- chology." Canadian Psychologist 6 (1965):82-92.

"Conceptual Models and Manpower Requirements in Psy- chology." American Psychologist 23 (1968):317-320�9

AIbee, G.W., and Dickey, M. "Manpower Trends in Three Mental Health Professions�9 American Psychologist 12 (1957):57-70�9

Andronico, M.P.; Fidler, J.; and Guerney, L.F. "The Combination of Didactic and Dynamic Elements in Filial Therapy." International Journal of Group Psychotherapy 17 (1967):10-17�9

35

Atthowe, J.M., and Krasner, L. "Preliminary Report on the Applica- tion of Contingent Reinforcement Procedures (Token Economy) on a Chronic Psychiatric Ward." Journal of Abnormal Psychol- ogy 73 (1968):37-43.

Brodsky, G. "The Relation Between Verbal and Nonverbal Behavior Change�9 Behaviour Research and Therapy 5 (1967):183-192.

Brown, P~, and Elliott, R. "The Control of Aggression in a Nursery School Class." Journal of Experimental Child Psychology 2 (1965):102-107�9

Cash, W.M., and Evans, I.M. "Training Preschool Children to Modify Their Retarded Siblings' Behavior�9 Journal of Behavior Therapy and Experimental Psychiatry 6 (1975):13-16.

Cowen, E.L. "Social and Community Interventions." In Annual Review of Psychology, edited by P. Mussen and M. Rosenzweig, 24 (1973):423-472�9

Drabman, R., and Spitalnik, R. "Training a Retarded Child as a Behavioral Teaching Assistant�9 Journal of Behavior Therapy and Experimental Psychiatry 4 (1973):269-272�9

Fairweather, G.W. "'A Case Study in Change. Innovation: A Nec- essary But Insufficient Condition for Change." Innovations 10 (1973) :25-27.

Ginsberg, B.G. "Parents as Therapeutic Agents: The Usefulness of Filial Therapy in a Community Mental Health Center�9 American Journal of Community Psychology 4 (1976):47-56.

Guerney, B.G., Jr. "Filial Therapy: Description and Rationale." Journal of Consulting Psychology 28 (1964):304-310.

Hall, R.V.; Axelrod, S.; Tyler, L.; Grief, E.; Fowler, C.J.; and Robinson, R. "Modification of Behavior Problems in the Home with a Parent as Observer and Experimenter�9 Journal of Applied Behavior Analysis 5 (1972):53-64.

Hobbs, N. "Mental Health's Third Revolution�9 American Journal of Orthopsychiatry 34 (1964):822-833.

�9 The Futures of Children�9 San Francisco: Jossey-Bass, 1975.

36

Lorion, R.P., and Cowen, E.L. "Comparison of Two Outcome Groups in a School-based Mental Health Project." American Journal of Community Psychology 4 (1976):65-73.

Lovaas, O.I.; Berberich, J.P.; Perloff, B.F.; and Schaeffer, B. "Acquisition of Imitative Speech by Schizophrenic Children." Science 151 (1966):705-706.

Madsen, C.H.; Becker, W.C.; and Thomas, D.R. "Rules, Praise, and Ignoring: Elements of Elementary Classroom Control." Journal of Applied Behavior Analysis 1 (1968):139-150.

Nay, W.R. "A Systematic Comparison of Instructional Techniques for Parents." Behavior Therapy 6 (1975):14-21.

O'Dell, S. "Training Parents in Behavior Modification: A Review." Psychological Bulletin 81 (1974):418-433.

Peter, Laurence J. The Peter Principle: How to Make Things Go Right. New York: William Morrow and Company, Inc., 1972.

Reddin, W.J. Effective Management by Objectives. New York: McGraw-Hill, 1970.

Reisinger, J.J. "The Treatment of 'Anxiety-Depression' Via Positive Reinforcement and Response Cost." Journal of Applied Behav- ior Analysis 5 (1972):125-130�9

�9 "Selected Administrative Issues for Improved Mental Health Organization." Journal of Mental Health Administration 4 (1975):4-11.

Reisinger, J.J.; Frangia, G.W.; and Hoffman, E.H. "Toddler Man- agement Training: Generalization and Marital Status." Journal of Behavior Therapy and Experimental Psychiatry 7 (1976): 335-340.

Reisinger, J.J., and Ora, J.P. "Parent-Child Clinic and Home Interaction During Toddler Management Training." Behavior Therapy, in press.

Reisinger, J.J.; Ora, J.P.; and Frangia, G.W. "Parents as Change Agents for Their Children: A Review." Journal of Community Psychology 4 (1976):103-123.

Risley, T., and Wolf, M. "Establishing Functional Speech in Echo- lalic Children." Behaviour Research and Therapy 5 (1967):73-88�9

37

Sanders, R. "New Manpower for Mental Hospital Service." In Emergent Approaches to Mental Health Problems, edited by E.L. Cowan, E.A. Gardner, and M. Zax. New York: Appleton- Century-Crofts, 1967.

Whaler, R.G. "Oppositional Children: A Quest for Parental Rein- forcement Control." Journal of Applied Behavior Analysis 2 (1969):159-170.

38