beginning to “unpack” early childhood mental health consultation: types of consultation services...

A R T I C L E

BEGINNING TO “UNPACK” EARLY CHILDHOOD MENTAL HEALTH CONSULTATION:

TYPES OF CONSULTATION SERVICES AND THEIR IMPACT ON TEACHERS

NICOLA CONNERS-BURROW, LORRAINE MCKELVEY, AND LATUNJA SOCKWELLUniversity of Arkansas for Medical Sciences

JENNIFER HARMAN EHRENTRAUTSt. Jude Hospital

SKYE ADAMS AND LEANNE WHITESIDE-MANSELLUniversity of Arkansas for Medical Sciences

ABSTRACT: The goal of the current study was to examine the impact of the frequency of two types of early childhood mental health consultation(ECMHC) activities (time spent in the class and time spent meeting with teachers) on teacher–child interactions, use of positive classroom-managementtechniques, and the intent to quit the childcare profession. We addressed these questions with a sample of 115 teachers from private childcare settingsparticipating in a midlength (6–8 months) consultation partnership, using pre- and posttest data collected from structured classroom observationsand teacher surveys. Results suggest that ECMHC time spent in the classroom was associated with less teacher punitiveness, permissiveness, anddetachment, and more use of positive classroom-management strategies at the posttest assessment (controlling for baseline teacher behaviors). Thefrequency of meetings with teachers did not impact teacher–child interactions; however, in an exploratory analysis, the frequency of meetings with theteacher was associated with a reduction in teachers’ intent to leave the profession of childcare.

Abstracts translated in Spanish, French, German, and Japanese can be found on the abstract page of each article on Wiley Online Library athttp://wileyonlinelibrary.com/journal/imhj.

* * *

All young children can benefit from high-quality early child-hood programs, although children from disadvantaged back-grounds typically gain the most (Barnett, 1995). While thereare many aspects to “quality” in childcare and early education,recent studies have emphasized the importance of high-qualityemotional interactions between teachers and young children. Forexample, in a recent study of 2,439 four-year-olds enrolled in671 prekindergarten classrooms, researchers found that childrendeveloped greater social skills when they had higher quality emo-tional interactions with their teachers. In fact, the quality of theseinteractions was more predictive of social skills development than

This project was funded by the Arkansas Department of Human Ser-vices/Division of Child care and Early Childhood Education (Child Care De-velopment Fund/Quality Initiative) working in collaboration with the Divisionof Behavioral Health Services and the Arkansas Head Start Collaboration Of-fice. We also acknowledge the three community mental health centers thatparticipated in the project, and their dedicated staff members.

Direct correspondence to: Nicola A. Conners-Burrow, University of Arkansasfor Medical Sciences, College of Medicine, Department of Family and Preven-tive Medicine, 521 Jack Stephens Drive, Room 530, Little Rock, AR 72205;e-mail: [email protected]

were many other factors such as teachers’ level of education andfield of study, class size, and child-to-teacher ratio (Mashburnet al., 2008).

Given the importance of the emotional climate of the preschoolclassroom, researchers and policymakers have suggested that in-terventions should target teachers’ classroom-management strate-gies as a way to support young children’s healthy social andemotional development (Raver et al., 2008). In many states,early childhood mental health consultation (ECMHC) modelshave been developed to address this need. By most defini-tions, ECMHC in childcare programs is geared primarily towardcapacity-building for teachers and often focuses on two levels: pro-grammatic consultation and child-focused consultation (Cohen &Kaufmann, 2005). Programmatic consultation focuses on improv-ing the overall quality of a childcare program and addresses struc-tural issues that impact the climate of a classroom or teachers’ abil-ity to build nurturing relationships with the children in their care.This work may take place through observations, formal and infor-mal sharing of information, meeting with teachers or supervisors,team meetings, mentoring, and coaching. Child-focused consulta-tion occurs when teachers identify children about whom they are

INFANT MENTAL HEALTH JOURNAL, Vol. 34(4), 280–289 (2013)C© 2013 Michigan Association for Infant Mental HealthView this article online at wileyonlinelibrary.com.DOI: 10.1002/imhj.21387

280

Types of Early Childhood Mental Health Consultation Services • 281

concerned. Mental health consultants observe and assess the chil-dren, ultimately assisting the teachers to make a plan to address theissues that contribute to the children’s social and emotional difficul-ties (Cohen & Kaufmann, 2005). The level of family involvement inchild-focused consultation varies, although at a minimum involvesthe consent of the family. Many EMCHC programs simultaneouslyoffer both programmatic and child-focused consultation.

In recent years, ECMHC models have been promoted in agrowing number of communities. In fact, federal programs suchas Head Start have long mandated the provision of some formof ECMHC (i.e., Head Start Program Performance Standards,45 C.F.R. Part 1304.24). The need for EMCHC models that pro-mote nurturing interactions between teachers and children has beenmade more apparent with emerging research on the importance ofteacher–child interactions in predicting child outcomes. We nowunderstand that measures of teacher–child interactions show morerobust relationships with child outcomes than do structural (e.g.,teacher education, physical-space requirements, teacher–child ra-tios) and even global measures of quality, such as environmen-tal rating scales (Beller, Stahnke, Butz, Stahl, & Wessels, 1996;Mashburn, 2008; Mashburn et al., 2008). These findings are be-ginnings to influence the development of states’ quality rating andimprovement systems (QRISs), as they suggest reasons to moveaway from a heavy reliance primarily on structural features toincorporate specific measures of teachers’ nurturing interactionswith children. In fact, Louisiana has linked their ECMHC programto their QRIS system (the Quality Start Child Care Rating System)in recognition of the importance of teacher–child interactions asan indicator of quality and the need to help programs improve thisaspect of quality.

Beyond generally promoting nurturing interactions, interven-tions are needed to help teachers manage challenging behaviorsin appropriate ways. Approximately 20% of preschool childrenhave some type of emotional or behavioral problem (Lavigneet al., 1996). Without intervention, some children will “outgrow”these problems while approximately half will go on to have prob-lems in later childhood and adolescence (S. Campbell, 1995;Lavigne et al., 1998). Problems with behavior management arecommon, but some teacher and setting characteristics may con-tribute to the problem. Li-Grining et al. (2010) found that less expe-rienced teachers and teachers who reported dealing with more per-sonal psychosocial stressors tended to use less effective behavior-management strategies. These teachers had children who exhibitedpoorer social interactions within their classrooms (Li-Grining etal., 2010). The challenge that these behaviors pose for teachersis illuminated by the alarmingly high rates of preschoolers whoare expelled from their preschool settings. In a study of nearly4,000 state-funded preschool classes randomly selected across thenation, 10.4% of preschool teachers reported at least one expul-sion in their classes during the past 12 months (Gilliam, 2005).According to Gilliam (2008), results from surveys in several stateshave suggested that expulsion rates are considerably higher in theless regulated world of private childcare than in federally fundedor state-supported preschool programs.

OUTCOMES OF ECMHC

While mental health consultation has been suggested as an ap-proach to help teachers develop nurturing relationships with chil-dren and manage challenging behaviors, the research base forconsultation is still growing. In a comprehensive review of pub-lished and unpublished empirical research on ECMHC completedbetween 1985 and 2008, only five published studies were found tomeet inclusion criteria (Brennan, Bradley, Allen, & Perry, 2008).This analysis demonstrated that in general, ECMHC tended to helpimprove the overall quality of early care and education settings andthat consultation was related to reduced staff turnover. However,this study also highlighted the need for additional research to beconducted and published in the field of ECMHC.

In terms of research on teacher outcomes, Raver et al. (2008)found that ECMHC led to higher levels of positive classroom cli-mate, teacher sensitivity, and behavior management, and that gen-der, race/ethnicity, and exposure to poverty were moderating fac-tors with regard to the effectiveness of consultation. Alkon, Ramler,and MacLennan (2003) found early childhood centers that partici-pated in more than one year of mental health consultation servicesdemonstrated improvements in environmental rating scale scores,as well as in teacher self-reported self-efficacy and competence.Conners-Burrow, Whiteside-Mansell, McKelvey, Amini-Vermani,and Sockwell (2012) reported that teachers receiving consultationservices were observed to have greater decreases in permissivenessand detachment, as measured by research assistant ratings on theCaregiver Interaction Scale (CIS; Arnett, 1989), as compared toteachers not receiving consultation.

Several studies also have noted a positive effect of consultationon child outcomes. For example, in a randomized trial of short-term ECMHC in Connecticut, Gilliam (2007, May) found thatECMHC had a positive effect on child behaviors expressed withinthe early childhood classroom setting. Perry, Dunne, McFadden,and Campbell (2008) evaluated a program offering individualized(or child-specific) ECMHC and pre- and postevaluations of thechildren; results showed that social skills improved, problem be-haviors decreased, and more than 75% of the 200 children whowere at risk for expulsion were able to maintain their current child-care placement. Relatedly, Williford and Shelton (2008) found thatchildren whose teachers received individually based consultation,which presented the teachers with empirically supported ways toreduce problematic behavior, showed a greater reduction in ex-ternalizing behavior problems postintervention than did a controlgroup. Teachers in the intervention group also increased their useof appropriate teacher strategies to address problematic behaviors(Williford & Shelton, 2008).

WHAT CONSULTATION SERVICES MATTER, AND HOWMUCH IS ENOUGH?

Beyond new evaluations of ECMHC, the past decade has broughtmany changes to the field of ECMHC, including efforts to agreeon a common definition, and increasing efforts to “unpack” what

Infant Mental Health Journal DOI 10.1002/imhj. Published on behalf of the Michigan Association for Infant Mental Health.

282 • N. Conners-Burrow et al.

is meant by consultation. While progress has been made towardconsensus definitions, we are still a long way from understandinghow much of what kind of consultation activity is needed to makevarious sorts of desired changes in teachers and children. Whilethere is growing consensus about the definition of ECMHC (as de-scribed earlier), the reality is that ECMHC programs vary widelyin their approach, including the array of services offered, length ofpartnerships, and who can receive and who can provide services.In their cross-site analysis of effective ECMHC programs, Duranand colleagues (Duran et. al., 2009) highlighted these variationsin practice while also noting key commonalities among success-ful programs. An important area of future research in the field ofECMHC is to understand the potential mechanisms of ECMHC,identify the impacts of specific components of ECMHC on teacher-and child-level outcomes, and to understand how much consulta-tion is enough to produce good outcomes (Duran et al., 2009; Perry,Dunne, McFadden, & Campbell 2008). We hope that the presentstudy represents a contribution to the literature on this topic.

A few studies have touched on these research questions. AminiVirmani, Masyn, Thompson, Conners-Burrow, and Whiteside-Mansell (2013) utilized data collected in an evaluation of ECMHCin Head Start and state-funded prekindergarten programs to exam-ine the relationship between the regularity of consultation (e.g., fre-quency) and the approach to consultation and changes in teacher–child interactions. “Approach” to consultation involved the degreeto which the consultant engaged in specific practices (e.g., reflec-tive practice, offered the teacher new ways of interacting with chil-dren, etc.). Changes in teacher–child interactions were measuredby research assistant ratings on the CIS (Arnett, 1989). Results sug-gested that the approach and regularity with which mental healthconsultants and teachers met to discuss children, their families, andteacher issues were associated with gains in quality of teacher–child interaction. In exploratory post hoc analyses, Amini Virmani(2013) found some evidence that teachers made more gains to-ward higher quality interactions when mental health consultantsmet more frequently with teachers. Specifically, they reduced theiruse of punitive techniques. In terms of approach, improvementsin interactions were more likely when consultants offered teach-ers information about children’s age-appropriate capacities, needs,and feelings related to the child’s behavior and offered teachersinformation about resources and services for children.

In an evaluation of ECMHC in 25 urban childcare centers,Alkon (2003) used a consultant activity survey to record types andfrequency of services provided in the classroom and found thatthe frequency of consultation activities as a whole were associatedwith lower staff-turnover rates. In terms of duration, Alkon and col-leagues (Alkon, Ramler & MacLennan 2003) found that centersreceiving more years of consultation had greater changes in child-care quality, as measured by observers using the Early ChildhoodEnvironmental Rating Scale (Harms & Clifford, 1980).

Green, Everhart, Gordon and Gettman (2006) surveyed 655Head Start directors, staff members, and mental health consul-tants in an effort to learn what makes consultation effective fromthe perspective of the recipient. Results have suggested that the

ability of a mental health consultant to build positive collabora-tive relationships with early childhood staff members was seenas the most important factor leading to success of ECMHC. Inaddition, responders reported that consultants who provided morefrequent services tended to have more positive relationships withearly childhood staff members. This study has suggested that both“dose” of consultation and the skills of the consultant in devel-oping relationships are important for achieving improvements inboth staff and child outcomes (based on perception of the staff).These authors suggested that models in which consultants provideformal and informal training as well as coaching and mentoringand spend time in classrooms visiting children and staff are morelikely to be successful. In this study, we will be able to directlytest the impact of the frequency of these two types of activities onchange in teacher–child interactions in the classroom.

PURPOSE OF THE STUDY

The current study seeks to examine the relationship between thefrequency of specific consultation activities (e.g., “dose” of twodifferent types of consultation services) and teacher–child inter-actions and teachers’ use of positive classroom-management tech-niques. In a subsample of teachers, we also will examine the impactof consultation activities on the teachers’ intent to quit the child-care profession. More specifically, we will add to the literature byexamining two key aspects of consultation (time spent in the classand time spent meeting with teachers) and their relationship toimproved teacher–child interactions. We are not aware of studiesthat have examined the frequency of various types of consulta-tion services and their differential impact on teacher outcomes.We will address these questions in the context of a midlength (6–8 months) consultation partnership with teachers and children inprivate childcare settings. Because consultation models that havebeen evaluated vary greatly in length, from 8-week intensive mod-els (e.g., Gilliam 2007, May) to multiyear programs (e.g., Alkon,Ramler & MacLennan 2003), it is important to address questionsabout the effectiveness of different types of consultation servicesin context, and our study will contribute to our understanding aboutwhich services can have an impact on teachers over the course of atypical school calendar year. We also emphasize the context of thepartnership with private childcare partners, given that the majorityof ECMHC studies have been conducted in Head Starts (e.g., Raveret al., 2009; Williford & Shelton, 2008) or other publicly funded,low-income preschools (e.g., Alkon, Ramler & MacLennan 2003),and this body of research may not generalize to private programsthat may be more variable in quality.

Our study involves a partnership between community mentalhealth centers (CMHCs) and private childcare and early educationprograms. The purpose of the ECMHC pilot project in Arkansaswas to facilitate collaboration between CMHCs and early educa-tion programs. In 2004, three CMHCs were funded to serve aspilot sites and to partner with at least three early education pro-grams in their region. Original project partners included Head Start



centers and state-funded prekindergarten programs (Conners-Burrow et al., 2012). In Years 4 and 5 of the project, the focusshifted to private childcare partners, which we report on in thisarticle. While the specific activities differed somewhat from site tosite, each pilot site agreed upon common goals. The bulk of theproject activities and resources were focused on two key goals:(a) Enhance the capacity of early education centers/teachers toprevent and manage behavioral and emotional problems in chil-dren, and (b) improve the socioemotional outcomes of childrenenrolled in the collaborating early education programs. Throughthe use of a combination of programmatic and child-specific con-sultation, the project should have benefits on multiple levels; thatis, we theorized that children experiencing social or emotional dif-ficulties would be supported, and so would their teachers, leadingto the kinds of improvements in teacher–child relationships andclassroom climate that are beneficial to all children (not just thoseexperiencing difficulties).

METHOD

The ECMHC project was implemented by the staff of three CMHCsin three different regions of Arkansas. Next, we describe the designof the intervention, including how the CMHC teams were identi-fied and trained as well as how the early education partners wereselected. Then, we describe the design of the evaluation study,including recruitment of study participants, the assessment sched-ule and instruments used, the plan for consultation activities, andfinally, the analysis plan.

Intervention Design

Identification and training of CMHC teams. CMHCs were cho-sen by the funding agency, the Arkansas Department of HumanServices, Division of Child Care and Early Childhood Educa-tion, through a competitive grant process. Funding was availablethrough the Child Care Development Fund/Quality Initiative. AllCMHCs in the state were encouraged to apply, and three wereselected as a result of the review process. Each CMHC identifieda team of 2 to 4 staff members who were assigned to the project.Each team was led by a master’s- or doctoral-level mental healthprofessional, and each team also included at least one parapro-fessional staff person. Prior to the launch of the project, CMHCteam members participated in training sessions on ECMHC, class-room strategies for addressing challenging behaviors, and screen-ing and assessment. Specifically, they attended a 2-day workshopto help them understand the nature of consultation, the role of theconsultant, methods for facilitating a mutually respectful teacher–consultant relationship, and so on. To ensure that the team mem-bers had a good working knowledge of classroom practices andstrategies, they also attended a multiday workshop where theycompleted the curriculum “Promoting the Social-Emotional Com-petence of Young Children,” developed by the Center on the So-cial and Emotional Foundations for Early Learning (availableat www.vanderbilt.edu/csefel). Prior to the phase of the study

included in this article, all consultants were asked to read theJohnston and Brinamen (2006) book on ECMHC, one of the firstin-depth descriptions of consultation work and a strong influenceon our program model. Throughout the project, ad hoc trainingswere provided to address specific issues raised by the CMHC staff(e.g., identifying autism spectrum disorders, strategies for workingwith young children with attention deficit disorders).

Early Education Center Selection

As part of the competitive grant process, the CMHCs were in-structed by the funding agency to recruit three to five early edu-cation partners per year. In the first 3 years of the project, thesepartners were from Head Start and state-funded prekindergartencenters (modeled after Head Start standards and called “ArkansasBetter Chance,” or “ABC” centers) in their community. Resultsfrom this phase of the study have been summarized elsewhere(Conners-Burrow et al., 2012). In the latter 2 years of the study,the CMHCs transitioned to new partnerships with private licensedchildcare centers, with the goal of learning if the project couldhave success with a range of childcare programs that vary in qual-ity, resources, and size. Recruitment letters were mailed to licensedcenter-based childcare providers within the catchment area of theCMHCs. Centers were chosen by CMHC staff based on their in-terest and willingness to participate in the intervention and evalua-tion components. Some centers had classrooms that served infantsand young toddlers, and consultants occasionally worked in theseclassrooms; however, data collection for this project focused onclassrooms with children ages 2 to 5 years.

The centers were private childcare centers that varied in sizeand quality of care provided. Of the participating centers, the ma-jority (53%) were not participants in the state’s QRIS. Of thosethat were participants, all but one had a “one-star” rating, whichis the entry level into QRIS, and does not require an onsite reviewor completion of an environmental rating scale. They also variedin the makeup of the families that they served, serving private-paychildren and with three-fourths also accepting low-income child-care vouchers.

Consultation Activities

While the specific activities differed somewhat from site to site,the majority of the project activities and resources were focusedon two key goals: (a) enhancing the capacity of early educationcenters/teachers to prevent and manage emotional and behavioralproblems in children and (b) improving the socioemotional out-comes of children enrolled in the collaborating early educationprograms. Given the nature of the consultation approach, most ofthe activities were aimed at the teachers (as well as the parent,although to a lesser degree) rather than directly at the child. Ashas been articulated by others (Gilliam, 2005), one path to im-proved child outcomes is thought to be through improvements inteacher–child interactions and the classroom environment. It wasthe design of our project that the primary activities of the CMHC



staff would include teacher training (e.g., inservice training on ef-fective classroom-management strategies), screening children forbehavioral concerns, formal observations of the classroom envi-ronment, coaching and other support for the teachers, formal (e.g.,staff meetings) and informal meetings with teachers, developingbehavior plans with teachers and parents, and making referrals formore in-depth assessment or mental health services for the childand family if needed.

While the intensity of the services for any specific teacher orclassroom varied based on interest and perceived need, the mentalhealth consultants were present in each center at least 1 half-day perweek from approximately September through April. The length ofservice varied somewhat based on the particulars of each CMHC’ssummer schedule (i.e., some programs operated on a differentschedule during the summer while others maintained a normalschedule for the bulk of the year). While consultation serviceswere available to some of the participating centers for 2 years ofthe project, for this study we examine data from each teacher’sfirst year of consultation. This allows us to examine the impact ofservices offered in a 6- to 8-month partnership.

Study Design

The evaluation study was designed to provide ongoing feedbackto the funding agency on the implementation of the project andto evaluate change in the functioning of participants (includingteachers and children) receiving the intervention.

Recruitment of Teachers

Teachers were recruited to participate in the study by trained re-search staff from all participating childcare partners over 2 years.Most classrooms had only one teacher per classroom, but wheremore than one was present, both “lead” and “assistant” teacherswere recruited. Using an “intent to treat” approach, teachers wereconsidered study participants if they consented to participate andwere assessed at least once (n = 115), regardless of the level of theirinvolvement with the mental health consultant. While no teachersrefused to participate in the study, there was variation in the lengthof their involvement with the project (described later).

Data-Collection Schedule and Raters

The assessment schedule was designed to accommodate the factthat many of the participating childcare programs operated onschedules similar to the school year, with substantial changes tostaffing patterns and classroom composition and schedule duringthe summer months. Therefore, assessments of teachers, class-rooms, and children typically occurred in late August and Septem-ber (allowing approximately 1 month for teachers and childrento settle into their routine) and were generally repeated in Aprilor May (or just before the program transitioned to its summerschedule). Assessments repeated at the beginning and end of theyear included teacher surveys and independent observations of the

classroom (discussed next). When new teachers were hired afterthe assessments at the beginning of the school year, an off-cycleassessment would be completed when possible (The baseline wascompleted later in the year for 13% of the sample.) Teacher surveysand classroom observations were completed by trained researchstaff (Reliability is described later.)

Instruments

The CIS (Arnett, 1989). The CIS is an observational rating scalewhich consists of items that assess the teacher’s sensitivity (warm,enthusiastic, and caring behavior), punitiveness (hostility, harsh-ness, and use of threats), detachment (lack of involvement, disinter-est), and permissiveness (lack of appropriate rules or discipline).The CIS has been used in many studies of childcare programs,including in multisite studies such as the Cost, Quality, and Out-comes Study (Peisner-Feinberg et al., 1999). The CIS is completedon both lead and assistant teachers after a period of observation inthe classroom setting (in the present study, generally about 1 1

2 hr)by a trained research assistant. Each observer completed trainingwith the research team before conducting independent observa-tions. The training consisted of operationalizing each item on theCIS and watching videos of teachers and children interacting inthe classroom, followed by discussion of relevant CIS items. Next,observers went into classrooms in pairs and conducted practiceobservations, then discussed their scoring afterward to identifyand resolve any areas of disagreement. This process resulted inexact agreement between observer pairs on the score for at leasttwo thirds of the CIS items, and scores were more than 1 pointapart (indicating disagreement) on fewer than 10% of the items.We computed four scale scores (sensitivity, punitiveness, detach-ment, and permissiveness), with high scores indicating more ofthe behavior for which the scale is named. In the present study,the internal consistency coefficients (Cronbach’s α) for each scaleranged from .76 to .91.

Use of Positive Classroom-Management Strategies. As an adden-dum to the CIS, we added items about specific teacher and class-room issues that are often the focus of consultation with the teacher.For example, because behavior problems often emerge during tran-sitions between activities, we included a specific item about theteacher’s use of transition techniques. Examples of these items in-clude “Teacher facilitates smooth transitions between daily events(enough warning, no long waits, individualization if needed),”“Teacher effectively uses techniques to defuse escalating behav-ior,” and “The classroom is set up to avoid conflict and promotepositive interaction (ex. Duplicate toys available to avoid conflict,adequate space provided).” Training for this scale followed thesame process as the training for the CIS. The coefficient alpha forthis six-item scale was .88.

Teacher Survey. Teachers were asked to complete a sur-vey developed for this project, which was designed to collectdemographic information. In Year 2 of the project, three questions



TABLE 1. Frequency of Mental Health Consultation Activities basedon Teacher-Report (n = 80)a

M SD

Meetings/Trainings With TeachersI met with the MHP individually to discuss staff issues. 1.47 .90The MHP provided support for my own well-being. 2.00 1.33The MHP provided me with formal training. 2.17 1.34I met with the MHP to help me better understand achild/children’s behavior.

2.45 1.35

I met with the MHP individually to discuss childrenand/or families.

2.52 1.40

Classroom TimeThe MHP spent informal time in my classroom. 3.08 1.51The MHP conducted formal observations in myclassroom.

3.24 1.39

MHP = mental health professional.aItems were rated on a scale of 1 (rarely or never) to 5 (weekly or more).

were added to gather information about the teachers’ intention tocontinue on in or quit the childcare profession. These three ques-tions (“I frequently feel like quitting my job,” “I intend to leavework in childcare in the next 12 months,” and “I feel committedto my work as a childcare provider”) were rated on a scale of1 (strongly disagree) to 4 (strongly agree). We summed the threeitems (reverse-coding the positive item) to create an Intent to Leavethe Field scale (Cronbach’s α = .68).

Abridged Mental Health Consultant Activities Questionnaire–Teacher-Report. The abridged Mental Health Consultant Activ-ities Questionnaire is a retrospective assessment of the frequencyand type of activities that the mental health consultants engagedin with the participating teachers. This questionnaire was adaptedfrom the Mental Health Consultant Activities Questionnaire byGreen et al. (2006). Consistent with our program model, we re-tained items tightly focused around the goal of capacity-buildingfor the teacher. This form was administered to teachers at theposttest classroom visit near the end of the childcare year. Items areshown in Table 1, and response options were 1 (rarely or never),2 (1–2 times per year), 3 (monthly), 4 (every other week), or 5(weekly or more). As shown in Table 1, to address our researchquestions about the relationship between types of consultation ser-vices and teacher outcomes, we grouped the items into two scalesas shown in Table 1, hereafter referred to as meetings/trainingswith teacher and classroom time. The correlation between the twoscales was .37, p = .001.

Sample

The sample includes 115 teachers from 18 childcare centers in threeregions of the state. Participating teachers worked in classroomserving children ages 2 to 5 years.The average number of years ofoverall teaching experience for the teachers was 6.30 (SD = 6.59).In terms of education, 36.5% reported having an associate’s degree

in child development or other teaching credential. Most (70.4%)were lead teachers, as opposed to assistant teachers or “aides”(Hereafter, we use “teacher” to refer to both lead and assistantteachers unless we distinguish between the two.)

Analysis Approach

We first used t tests and χ2 tests to examine differences in teacherswho were retained for the follow-up, as compared to those who leftemployment. No teachers withdrew from the study, but a numberleft employment before their follow-up could be collected. Theirparticipation ended if they left employment at the participatingcenter, if they moved to a classroom that was not participating(e.g., an infant classroom in the same center), or if they were onextended sick or maternity leave. We collected baseline assess-ments on 115 teachers, with complete matched follow-ups on 76(An additional 4 teachers completed the follow-up survey of MHPactivities, but were unavailable for observation.) This resulted in a34% attrition/teacher turnover rate.

We compared teachers retained for a posttest with teacherswho were not retained and found that they were not statisticallydifferent on baseline measures of teacher–child interaction (totalArnett CIS and positive behavior management scores) or education(having a teaching credential). Turnover rates were similar acrossregions of the state. However, teachers who were retained had moreyears of experience, t(110) = 2.31, p = .02.

Given the significant difference between teachers who wereand were not retained for follow-up, we controlled for years of ex-perience in multiple regression models estimated for each outcomeusing full information maximum likelihood estimation (FIML; En-ders, 2001a, 2001b) in a structural equations modeling softwarepackage (LISREL 8.8; Joreskog & Sorbom, 2006). FIML doesnot actually impute values but uses all information that is avail-able for each observation to provide unbiased parameter estimatesin the presence of missing data (Acock, 2005). Simulation stud-ies that have examined FIML have found it an improvement overconventional missing-data methods such as listwise deletion (En-ders, 2006; Widaman, 2006), especially when variables that explainmissingness are included in analyses (Acock, 2005).

To examine the relationship between the dose of consultationservices of meetings with teacher and classroom time, we used aseries of regression models predicting teacher–child interactionsat follow-up (the CIS and the Positive Classroom Managementscale, as described earlier) while controlling for teacher demo-graphics and for baseline teacher–child interaction. For example,we examined the impact of frequency of meetings with teacher andclassroom time on the Arnett Sensitivity scale at posttest while con-trolling for the Arnett Sensitivity scale at pretest as well as years ofexperience and whether the teacher had a teaching credential. Thisanalysis allowed us to address our research question about whichtype of consultation services are associated with teacher change inthe context of a midlength consultation model in private childcare.

Finally, we conducted an exploratory regression analyses fora subsample of teachers. Specifically, we examined the impact of



meetings with teacher and classroom time on change in teachers’intent to leave the profession of childcare in a sample of teachersfrom the second year of the study (This questionnaire was addedat the midpoint.) This is exploratory both because of the smallersample size and because we can only examine change in teacherswho were still employed at follow-up, not that in teachers whointended to leave and in fact did so.

RESULTS

Teacher-Report of Services Received

Because we used data from each teacher’s first year of receivingconsultation services and participating in the evaluation study, theaverage length of services for teachers who were retained was 6.46months (SD = 2.89). Table 1 shows the teachers’ report of the fre-quency of the services that they received from the consultant (withitems sorted by mean score). Based on teacher-report, meetings todiscuss staff issues were the least frequent activities of the CMHCstaff (occurring somewhere between rarely and one to two timesper year). The most common activities included meetings withthe teachers to help them understand a child’s behavior or discusschildren and family issues, and spending time in the classroom ob-serving children or just being present informally. These activitiesoccurred more frequently for some teachers than for others, withthe average score reflecting about monthly frequency. As seen inTable 1, the standard deviation indicates considerable variation inreports of the intensity of services.

Relationship Between Consultation Services andTeacher–Child Interactions

Results of the regression models related to the Arnett CaregiverInteraction scales and our additional measure of Positive Class-room Management are shown in Table 2. Results indicate thatmental health professional classroom time is a significant predic-tor of follow-up teacher–child interactions on three of four Arnettscales (Punitive, Permissive, and Detached) as well as our mea-

sure of Positive Classroom Management (even after controllingfor baseline scores). All results are in the predicted direction, withgreater frequency of mental health professional consultation class-room time associated with more optimal teacher–child interactions.Specifically, more mental health professional classroom time wasassociated with significantly less punitive, β = −.22, t = −2.14,p < .05, permissive, β = −.34, t = −3.62, p < .01, and detached,β = −.38, t = −3.84, p < .01, teacher behavior, and more use ofpositive classroom-management strategies, β = .16, t = 2.02, p <

.05. On the contrary, results suggest that the frequency of consul-tant meetings with teacher is not a significant predictor of changein any observed teacher behavior.

Exploratory Analyses: Relationship Between Consultation Servicesand Teacher Intent-to-Leave the Profession

For a subsample of teachers whose participation began in the sec-ond year of the project, we also measured their intent to leave theprofession. We followed the same approach described earlier byexamining the relationship between posttest intent to leave anddose of services while controlling for pretest intent to leave anddemographics. In this analysis, greater frequency of consultationclassroom time did not predict the teachers’ intent to leave, β =−.08, t = .56, n.s. However, the frequency of consultant meetingswith teacher was associated with a decreased intent to leave at theposttest, β = −.32, t = −2.50, p = .01.

DISCUSSION

In this study, we sought to examine the relationship between thefrequency of specific consultation activities (e.g., dose of differenttypes of services) and teacher outcomes. We examined the teach-ers’ perception of how mental health consultants spent their time,including how often consultants met with the teacher to discusschildren or families and to help them understand children’s behav-ior and how often they spent time in the classroom to either makeformal observations or spend informal time there. Our findings

TABLE 2. Regressions Predicting Teacher–Child Interactions at Follow-Up

Arnett CIS ScalePositive Classroom

Variable Punitiveness Sensitivity Permissiveness Detachment Management

Model 1Teaching Credential (0 = no, 1 = yes) −1.84 (.80)∗ 1.67 (.89)† −.47 (.37) −1.27 (.46)∗∗ .13 (.09)Years of Teaching Experience .05 (.05) .07 (.06) .003 (.02) .003 (.03) .002 (.006)Baseline Teacher–Child Interaction (from same scale) .50 (.08)∗∗ .39 (.07)∗∗ .46 (.07)∗∗ .44 (.09)∗∗ .68 (.06)∗∗

MHP Classroom Time −.66 (.31)∗ −.20 (.35) −.53 (.15)∗∗ −.68 (.18)∗∗ .07 (.03)∗

MHP Meetings With Teacher .29 (.47) .17 (.53) .003 (.21) .04 (.26) .03 (.05)

Constant 8.15 16.91 6.61 6.24 .72Model R2 .41 .33 .51 .43 .66

Note. Unstandardized βs are presented with SEs in parentheses. MHP = mental health professional.†p < .10. ∗p < .05. ∗∗p < .01.



suggest that these two sets of activities may have different impactson teachers.

Overall, we found that it appears to be the time spent in theclassroom that is associated with many of the changes we hope tosee in teachers, including less punitiveness, permissiveness, and de-tachment and more use of positive classroom-management strate-gies. The frequency of meetings with teachers did not appear toimpact teacher–child interactions; however, the opposite was truein our subgroup analysis of teachers’ intent to leave the professionof childcare. In that case, our results suggest that meetings withthe teacher may be a key ingredient in reducing teachers’ intent toleave.

The items that comprise the scale related to consultant timespent in the classroom address classroom observations and “infor-mal” time in the classroom. While we did not collect additionalformal data about the nature of these observations or what consul-tants did in their informal time in the classroom, consultants weretrained to engage in certain types of specific activities as part of ourECMHC model. In terms of observations, consultants were trainedto observe the classroom as a whole, teacher–child interactions,and children experiencing difficulties. During their initial obser-vations, consultants routinely completed the CIS (Arnett, 1989) ineach classroom to help them think about the teachers’ strengthsand difficulties in interactions with the children. This observationhelped guide their consultation efforts (separate from the use ofthe CIS in the evaluation study). In addition, they observed to de-termine what might be triggering behavior problems in individualchildren or to identify issues related to “trouble spots” in the dayfor multiple children (e.g., difficult transitions between activities).They then used this information in their informal time in the class-room to suggest or model a new strategy to help teachers throughthese times or to “coach” teachers through implementation of anew strategy. For example, consultants taught teachers transitionactivities (e.g., transition songs, finger-plays) by modeling theiruse in the classroom; on subsequent visits to the classroom, theymight prompt the teacher to try the strategy or discuss any imple-mentation barriers. They used a similar approach when workingwith individual children experiencing difficulties, often suggestinga technique for working with a child, modeling the agreed-upontechnique, and then coaching the teacher the next time the behavioroccurred.

Our finding related to the impact of this time spent together inthe classroom could be viewed as consistent with evidence fromevaluations of other teacher trainings. There is evidence to suggestthat it can be difficult for teachers to implement new knowledgeor strategies that they learn about in training absent any opportuni-ties to apply the knowledge and to receive feedback, coaching, orsupport (Epstein, 1993; National Research Council, 2001; Spodek,1996; Zaslow & Martinez-Beck, 2005). Consultants may be moresuccessful when they are actually in the room and can coach theteacher through the way to properly implement a strategy—or canproblem-solve with the teacher regarding difficulties encounteredeither in the moment or shortly after an incident. Teachers maybe more cognizant of new strategies learned if the consultant who

suggested the strategies is in the room as a “stimulus” to triggerthinking about something in a new way. Further, we can speculatethat teachers may be more willing to try to implement new strate-gies if the person with whom they are working is present—simplydue to a desire to either be compliant or to appear compliant withthe consultation partnership. Finally, if teachers know that the con-sultant is present in case they get “stuck,” they may be more willingto practice a new strategy that is outside of their comfort zone.

It is interesting that time spent meeting with the teacher wasassociated with an increase in teachers’ commitment to the field ofchildcare, reducing their intention to leave the profession. Whilethis analysis was exploratory in nature, we could speculate thatencouragement and support provided in those meeting times mayhave impacted teachers’ willingness to continue with a very dif-ficult job. Early care and education teachers are generally under-valued (Rhodes & Huston, 2012), and it may be validating forthem to be “heard” and to be provided with empathic support. Thismay encourage them to “stick with it” a bit longer. Others havefound that the frequency of mental health consultation is associatedwith lower staff turnover (Alkon et al., 2003), although we are notaware of studies that have examined which aspects of consultationmight impact staff turnover. The impact of consultation on teacherturnover could be a by-product of reductions in teacher stress as-sociated with ECMHC. In their review of the ECMHC literature,Brennan et al. (2008) noted that several studies have suggested thatstaff who receive consultation have lower job-related stress. Futurestudy should test this link, including among teachers who actuallyleave the field during consultation.

In addition to deepening our understanding of how consul-tation works, our study offers promising evidence that even amidlength (M = 6 months) consultation model at relatively lowintensity can have positive impacts on teachers in the world ofprivate childcare. In our partnering centers, only about one thirdof the teachers had any kind of teaching credential (including atthe associate’s degree level). Anecdotally, we can say that whenour consultants first began working in private settings, as opposedto their prior experience in Head Start and state-funded prekinder-garten programs, they were concerned about their ability to make apositive impact because of the barriers to their work. Specifically,they voiced concerns about lack of basic structure (schedules, rou-tines), insufficient toys and materials, and staff with very littletraining. However, these results suggest that consultation can havea positive impact even under these difficult circumstances. Whilewe did not have a comparison group, our study illustrates the impactof dose of services on teachers, which offers some support for thismodel of relatively low intensity services offered over the courseof about 6 months. Note that for the services that made the mostimpact on teachers’ behaviors (related to time in classroom), themean frequency reported by teachers indicates that the “average”teacher received those services about monthly, with “high-dose”teachers (1 SD above the mean) receiving services closer to oncea week.

For states seeking to engage their private childcare providersin quality-improvement efforts, our study suggests that ECMHC



services could be helpful to these providers, although more re-search is needed with this group. Specifically, as states begin to in-corporate measures of teacher–child interactions into their QRISs,many private providers are likely to need support to meet thesenew measures of quality. Given the evidence suggesting that high-quality teacher–child interactions are critical to predicting goodchild outcomes (Beller et al., 1996; Mashburn, 2008; Mashburnet al., 2008), this seems to be a key target for intervention. Sincethe majority of ECMHC studies focusing on teacher outcomes havebeen conducted in Head Start or other publically funded preschoolprograms (e.g., Conners-Burrow et al., 2012; Raver et al., 2009;Williford & Shelton, 2008), our findings are an addition to theliterature.

A key limitation to our study is that teacher turnover was a seri-ous problem in these centers, and we were able to evaluate only theimpact of services on teachers who stayed in the partnering centers.More study is needed to understand the impacts of consultation onteachers who leave a center, especially those who stay in the fieldin some capacity. Note that our study utilized teachers’ report ofthe frequency with which they receive various services from theconsultant; it also was a retrospective report provided at the end ofconsultation. It is certainly possible that consultants thought thatthey had delivered a particular kind of service (e.g., feeling thata conversation was an “informal meeting”), but that the teacherdid not experience the service that way. Additional studies shouldexplore these questions using an activity log from the perspectiveof the mental health professional.

Additional studies also should explore the relationship be-tween specific consultation activities and teacher outcomes amongteachers in the infant classrooms. While our consultants did providesome consultation in infant rooms, directors more often encour-aged consultants to spend their time with toddlers and preschool-ers in classrooms where problems with behavior were beginning toemerge (e.g., toddlers biting or preschoolers exhibiting aggressionor noncompliance). We focused our evaluation of the consulta-tion in classrooms serving children ages 2 to 5 years because ourmeasures best addressed teaching in that context. However, webelieve that consultation is highly applicable to teachers in infantclassrooms and that it can be equally successful.

Even with these limitations, we feel that our findings offera strong contribution to the literature by increasing our under-standing of the process of achieving desired teacher outcomes ina midlength consultation model in private settings. By “pullingapart” aspects of consultation, we were able to shed some lighton which consultation activities may be most critical for achievingcertain teacher outcomes, at least in the context of a midlengthconsultation model. Further study is needed to continue to exam-ine various aspects of consultation and refine our understanding ofwhat it takes to achieve the various desired outcomes of consulta-tion. Future studies should examine these questions in the contextof other models of ECMHC, as one could speculate that as the re-lationship between the consultant and teacher deepens and evolvesover time, the impact of various consultation services could simi-larly evolve. Finally, future studies should examine the impact of

types of consultation services on children in the classroom, in-cluding children with normative behavior and children exhibitingchallenging behaviors.

REFERENCES

Acock, A.C. (2005). Working with missing values. Journal of Marriageand Family, 67, 1012–1028.

Alkon, A., Ramler, M., & MacLennan, K. (2003). Evaluation of mentalhealth consultation in child care centers. Early Childhood EducationJournal, 31, 91–99.

Amini Virmani, E., Masyn, K.E., Thompson, R.A., Conners-Burrow, N.,& Whiteside-Mansell, L. (2013) Early Childhood Mental HealthConsultation: Promoting Change in the Quality of Teacher-ChildInteractions. Infant Mental Health Journal 34(2), 156–172.

Arnett, J. (1989). Caregivers in day-care centers: Does training matter?Journal of Applied Developmental Psychology, 10, 541–552.

Barnett, W.S. (1995). Long-term effects of early childhood programs oncognitive and school outcomes. The Future of Children, 5, 25–50.

Beller, E., Stahnke, M., Butz, P., Stahl, W., & Wessels, H. (1996). Twomeasures of the quality of group care for infants and toddlers. Euro-pean Journal of Psychology of Education, 11, 151–167.

Brennan, E.M., Bradley, J.R., Allen, M.D., & Perry, D.F. (2008). Theevidence base for mental health consultation in early childhood set-tings: Research synthesis addressing staff and program outcomes.Early Education and Development, 19, 982–1022.

Campbell, S. (1995). Behavior problems in preschool children: A reviewof recent research. Journal of Psychology and Psychiatry and AlliedDisciplines, 36, 113–149.

Cohen, E., & Kaufmann, R. (2005). Early childhood mental health consul-tation (DHHS Publication No. CMHS-SVP0151). Rockville, MD:Center for Mental Health Services, Substance Abuse and MentalHealth Services Administration.

Conners-Burrow, N.A., Whiteside-Mansell, L., McKelvey, L., Amini-Vermani, E., & Sockwell, L. (2012). Improved classroom qualityand child behavior in an Arkansas Early Childhood Mental HealthConsultation Pilot Project. Infant Mental Health Journal, 33(3), 256–264.

Duran, F. et al. (2009). What works? A study of effective early childhoodmental health consultation programs. Washington, DC: GeorgetownUniversity, Center for Child and Human Development.

Enders, C.K. (2001a). A primer on maximum likelihood algorithms avail-able for use with missing data. Structural Equation Modeling, 8,128–141.

Enders, C.K. (2001b). The impact of nonnormality on full informationmaximum-likelihood estimation for structural equation models withmissing data. Psychological Methods, 6, 352–370.

Enders, C.K. (2006). Analyzing structural equation models with missingdata. In G.R. Hancock & R.O. Mueller (Eds.), Structural equationmodeling: A second course (pp. 313–342). Greenwich, CT: Informa-tion Age.

Epstein, A.S. (1993). Training for quality: Improving early childhoodprograms through systematic inservice training. Monographs of the



High/Scope Educational Research Foundation, No. 9. Ypsilanti, MI:High/Scope Educational Foundation.

Gilliam, W.S. (2005). Prekindergarteners left behind: Expulsion ratesin state prekindergarten programs (Abbreviated as Foundationfor Child Development Policy Brief Series No. 3, May 2005).Retrieved May 27, 2006 from http://fcd-us.org/sites/default/files/ExpulsionCompleteReport.pdf

Gilliam, W.S. (2007, May). Early Childhood Consultation Partnership:Results of a random-controlled evaluation [Final Report and Ex-ecutive Summary]. New Haven, CT: Yale University, Child StudyCenter.

Gilliam, W.S. (2008). Implementing policies to reduce the likelihood ofpreschool expulsion. Foundation for Child Development Policy Brief,7, 2–10.

Green, B.L., Everhart, M., Gordon, L., & Gettman, M.G. (2006). Char-acteristics of effective mental health consultation in early childhoodsettings: Multilevel analysis of a national survey. Topics in EarlyChildhood Special Educaiton, 26, 142–152.

Harms, T., & Clifford, R. (1980). Early Childhood Environment RatingScale (ECERS). New York: Teachers College Press.

Head Start Program Performance Standards. (2010). 45 C.F.R. Part1304.24. U.S. Department of Health and Human Services, Adminis-tration for Children and Families, Office of Head Start http://eclkc.ohs.acf.hhs.gov/hslc/standards/Head%20Start%20Requirements/1304/1304.24%20Child%20mental%20health.htm.

Johnston, K., & Brinamen, C. (2006). Mental health consultation in child-care: Transforming relationships among directors, staff, and families.Washington, DC: ZERO TO THREE Press.

Joreskog, K.G., & Sorbom, D. (2006). LISREL 8.80 for Win-dows [Computer software]. Lincolnwood, IL: Scientific SoftwareInternational.

Lavigne, J., Arend, R., Rosenbaum, D., Binns, H., Christoffel, K.,& Gibbons, R. (1998). Psychiatric disorders with onset in thepreschool years: I. Stability and diagnoses. Journal of the Amer-ican Academy of Child and Adolescent Psychiatry, 37, 1246–1254.

Lavigne, J., Gibbons, R., Christoffel, K., Arend, R., Rosenbaum, D., Binns,H. et al. (1996). Prevalence rates and correlates of psychiatric disor-ders among preschool children. Journal of the American Academyof Child and Adolescent Psychiatry, 35, 204–214.

Li-Grining, C., Raver, C.C., Champion, K., Sardin, L., Metzger, M., &Jones, S.M. (2010). Understanding and improving classroom emo-tional climate and behavior management in the “real world:” Therole of Head Start teachers’ psychosocial stressors. Early Educationand Development, 21, 65–94.

Mashburn, A.J. (2008). Quality of social and physical environmentsin preschools and children’s development of academic, language,and literacy skills. Applied Developmental Science, 12(3), 113–127.

Mashburn, A., Pianta, R., Hamre, B., Downer, J., Barbarin, O., Bryand,D. et al. (2008). Measures of classroom quality in pre-kindergartenand children’s development of academic, language, and social skills.Child Development, 79, 732–749.

National Research Council. (2001). Eager to learn: Educating ourpreschoolers. In B. Bowman, M. Donovan, & S. Burns (Eds.), Com-mittee on Early Childhood Pedagogy. Washington, DC: Commissionon Behavioral and Social Sciences Education.

Peisner-Feinberg, E.S., Burchinal, M.R., Clifford, R.M., Culkin, M.L.,Howes, C., Kagan, S.L. et al. (1999). The children of the cost, qual-ity, and outcomes study go to school [Executive Summary]. ChapelHill: University of North Carolina, Frank Porter Graham Child De-velopment Center.

Perry, D.F., Dunne, M.C., McFadden, L., & Campbell, D. (2008). Re-ducing the risk for preschool expulsion: Mental health consultationfor young children with challenging behaviors. Journal of Child andFamily Studies, 17, 44–54.

Raver, C.C., Jones, S.M., Li-Grining, Metzger, M., Champion, K.M.,& Sardin, L. (2008). Improving preschool classroom processes:Preliminary findings from a randomized trial implemented inHead Start settings. Early Childhood Research Quarterly, 23, 10–26.

Raver, C.C., Jones, S.M., Li-Grining, C., Zhai, F., Metzger, M.F., &Solomon, B. (2009). Targeting children’s behavior problems inpreschool classrooms: A cluster-randomized control study. Journalof Consulting and Clinical Psychology, 77, 302–316.

Rhodes, H., & Huston, A. (2012). Building the workforce our youngestchildren deserve. Society for Research on Child Development: SocialPolicy Report, 26, 3–26.

Spodek, B. (1996). The professional development of early child-hood teachers. Early Child Development and Care, 115(1), 115–124.

Widaman, K.F. (2006). III. Missing data: What to do with or without them.Monographs of the Society for Research in Child Development, 71,42–64.

Williford, A.P., & Shelton, T.L. (2008). Using mental health consultationto decrease disruptive behaviors in preschoolers: Adapting an empir-ically supported intervention. Child Psychology and Psychiatry, 49,191–200.

Zaslow, M.E., & Martinez-Beck, I.E. (2005). Critical issues in early child-hood professional development. Baltimore, MD: Brookes.


beginning to “unpack” early childhood mental health consultation: types of consultation services...

Documents