system-wide implementation of act in ontario: an ongoing improvement effort

Regular Article

System-Wide Implementation of ACTin Ontario: An Ongoing Improvement Effort

Lindsey George, MES, MD, FRCPCJanet Durbin, PhDChristopher J. Koegl, MA

Abstract

In the late 1990s, the government of Ontario undertook a province-wide implementation ofAssertive Community Treatment (ACT). Capacity grew to 59 teams within 6 years. This paperdescribes the implementation process, focusing on three phases—start-up, or the enabling phase;feedback, or the reinforcement phase; and response, or the corrective action phase. Keyimplementation supports include an active oversight committee with representation from both theministry and the field and the availability of the planning data on ACT performance. Three areas ofunderperformance were identified: lower than expected team caseloads, drift from the target clientgroup, and significant under-staffing in the teams. Likely causes were suggested, and correctiveactions developed, which centered on clarifying the ACT standards, especially related to intakecriteria, rate of intake and staffing, increasing team funding, and establishing expectations forreporting and accountability. While these corrective responses are promising, implementation ofinfrastructure and mechanisms for providing systematic practice feedback is still underdeveloped.

Introduction

Assertive Community Treatment (ACT) is a comprehensive community-based model fordelivering treatment, support, and rehabilitation services to adults with serious mental illness who

Address correspondence to Janet Durbin, PhD, Centre for Addiction and Mental Health, 33 Russell Street (3rd FloorTower), Toronto, ON, Canada M5S 2S1. Phone: +1-416-5358501; Fax: +1-416-9794703; Email: [email protected].

Lindsey George, MES, MD, FRCPC, Department of Psychiatry and Behavioural Neurosciences, Director of Adult MentalHealth Research, System Linked Research Unit Head of Service, Mental Health Rehabilitation Services, Clinical Director,Brant Assertive Community Treatment Team, 44 King Street, Suite 204, Brantford, ON, CanadaR Phone: +1-519-752-2927;Fax: +1-519-758-1971; Email: [email protected].

Christopher J. Koegl, MA, Institute of Criminology, Cambridge University, Sidgwick Avenue, Cambridge, CB3 9DA,United KingdomR Phone: +44-079-4270-6614; Email: [email protected].

Janet Durbin, PhD, Health Systems Research and Consulting Unit, Centre for Addiction and Mental Health, Toronto, ON,Canada.

Janet Durbin, PhD, Department of Psychiatry, University of Toronto, Toronto, ON, Canada.

Journal of Behavioral Health Services & Research, 2008. c) 2008 National Council for Community BehavioralHealthcare.

System-Wide Implementation of ACT in Ontario GEORGE et al. 309

have significant functional difficulties across a number of domains and have extensively usedservices or have other risk factors for poor outcomes such as criminal justice involvement,homeless, or substance abuse.1,2 The ACT model is based on an inter-disciplinary team of 10–12practitioners who accept shared responsibility for serving a caseload of 60–100 clients. Rather thanbrokering services from other agencies, the team delivers services directly, in community settingsto facilitate in vivo rehabilitation, crisis support, and treatment. Services are available on a 24-h,7 days a week basis, and clients are seen several times a week.

As it was first introduced to the field three decades ago,3 the ACT model has been widelystudied. Fidelity criteria have been defined,1,4 and a body of rigorous research has demonstratedsuperior outcomes on a number of domains for ACT compared with standard care and lessintensive care management. Most consistently demonstrated have been a reduction in hospitaladmissions and days and improved housing stability. As well, ACT clients are less likely to dropout from service and report higher rates of service satisfaction.5–8

ACT was one of six treatments selected for implementation in the Implementing Evidence-basedPractices for Severe Mental Illness Project.9 It has been endorsed as a priority treatment by a numberof US organizations, including the National Alliance for the Mentally Ill,10 the Health Care FinancingAdministration,11 and the surgeon general in his report on mental illness.12 The Canadian Senateidentified ACT as a key community treatment in a recent national report on mental illness inCanada.13 However, implementation of evidence-based practices (EBPs) in routine-care settings holdsmany challenges, especially related to initial model adherence and later drift.14 This is a concernbecause there is evidence linking overall adherence to the ACT model to better outcomes.4,8,15,16

While the mental health field is in its infancy in defining the principles of evidence-basedimplementation,17,18 an expanding scientific literature on the topic is beginning to inform ourunderstanding of factors that facilitate or inhibit faithful implementation of EBPs, including ACT.

Based on a literature review and interviews with ACT administrators, clinicians, and consumerinformants, Phillips et al.2 proposed a number of factors that facilitated replication of the ACTmodel and prevented unintended variation. Key strategies at the system level included strongleadership; adequate funding support; availability of standards; oversight capacity; team trainingand support, especially at start-up; and ongoing feedback on practice. Key strategies for teamleaders included development of clear team policies and procedures; selection and retention ofteam members; training; and organizational integration of the team with the broader system of care.

Similar system-level factors were identified in the US National EBP Demonstration Project onimplementation of five EBPs in eight states.9 Analyzing extensive site visit interview feedback onbarriers and facilitators, Isett et al.19 identified three areas where state authorities play a critical role.These included financing (e.g., creating a supportive fiscal environment); leadership (e.g., solicitingstakeholder support); and training (e.g., helping front-line workers obtain the necessary skills for agiven practice). Pertaining specifically to ACT, the study emphasized the importance of adequatefunding to support high initial start-up costs as well as ongoing service delivery (finances);interaction with stakeholders to review the model and address potential concerns about coercion(leadership); and retraining of front-line workers to align practice with model fidelity (training andquality). Implementation challenges related to targeting services to high need consumers, the longtime between ACT start-up and achievement of full caseload, and the difficulty of implementingACT in rural areas because of workforce shortages and frequent long travel time for face-to-facemeetings are faced.

In a separate study of the National EBP Project, Rapp et al.20 reviewed feedback from seniorsystem administrators and researchers and identified seven major supporting tasks used by statemental health authorities. These included (1) strategic planning: This task builds awareness andpromotes uptake and can be achieved through marketing to recruit early adopters and/or state-widemandates for wider implementation; (2) stakeholder involvement: This task uses strategies such assystem-wide task forces, networks, and conferences to build support from constituencies as diverse

310 The Journal of Behavioral Health Services & Research 36:3 July 2009

as unions, professional organizations, mental health providers, other sector providers (i.e.,employment sector), government staff, consumers, and families; (3) focus on outcomes that clientsvalue: This task is supported by ongoing outcome measurement and management by outcomes (i.e.,connection between incentives and outcomes); (4) regulatory standards: This task focuses ondevelopment of program specifications that can provide a basis for monitoring practice quality (e.g.,fidelity measures); (5) incentives and disincentives: This task is discussed mainly with regard to useof financial incentives tied to program fidelity or client outcomes; 6) adequate funding: This taskenables faithful implementation of a model and may entail moving funds from ineffective practicesto EBPs or promising practices; (7) workforce development: This task focuses on buildingknowledge, skills, and capacities through formal training (e.g., via university curricula) combinedwith on-site technical assistance and in situ supervision. Implementation toolkits may also be useful.

Moser et al.18 compared implementation of two EBPs—ACT and integrated dual diagnosistreatment in Indiana.18 Greater success in ACT implementation was attributed to a number offactors, including wide stakeholder familiarity with ACT and receptivity to its implementation;establishment of detailed, prescriptive standards tied to certification and reimbursement;availability of an adequate ongoing funding stream, along with incentive funding tied tocertification; and a comprehensive program of training and technical support provided through awell-funded, university-based technical assistance center. Even with these state-level supports,teams still faced challenges related to achieving the required multidisciplinary staff complementand developing good team cooperation, pressure to enroll clients that did not meet the ‘difficult toserve’ criteria, adequate service productivity especially at start-up, and use of program data forimprovement efforts. Teams typically required a year or more to reach the point where theiroperations were ‘sustaining the practice’.

Ganju17 argued that EBP diffusion occurs in waves and that the relevance of the above practicesvaries for earlier and later adopters. Specifically, demonstration projects with training may besufficient for first wave adopters, whereas late adopters may require standards and financialincentives, and resistors may require contractual mechanisms and intensive technical assistance.Ganju also emphasized the importance of integrating system-wide performance measurement andEBP implementation into an overarching and ongoing cycle of reporting and improvement.

Consistent with the notion of early and late adopters, Ohio used a combined ‘bottom–up’ and‘top–down’ approach for a state-wide implementation of the Crisis Intervention Team (CIT) model,an emerging best practice for pre-arrest diversion that involves mental health and law enforcementstaff.21 Implementation started with an initial presentation to engage local interest in early uptake,followed by establishment of a university-based technical support center and development of aconsensus document on core elements to support state-wide dissemination. The benefit of thisapproach was that local champions were identified who then played key roles in widerimplementation. Magnabosco22 noted that many states have used top–down and bottom–upapproaches as part of their EBP dissemination strategy, relying on an evolutionary framework ofimplementation characterized by interaction and negotiation over time.

In summary, recent research is relatively consistent in the system level strategies identified tosupport dissemination of EBP. These center on leadership, stakeholder engagement/commitment,availability of standards with related monitoring, adequate financing, training with reinforcingstrategies such as practice feedback, and staged roll-out using both top–down and bottom–upapproaches.

Beginning in 1998, the Ontario Ministry of Health and Long-Term Care (MOHLTC) undertooka province-side rollout of the ACT model.23 Ontario is the largest province in Canada, with apopulation of about 12 million people. Operating within a predominantly universal health caresystem, the province spends about $40 billion dollars annually to provide a wide range of healthservices, including inpatient and community mental health care. Hospital and community mentalhealth agencies receive a global budget for delivering service, which is updated annually.


Throughout the 1990s, Ontario was actively involved in implementation of mental healthreform,24 which broadly entailed downsizing inpatient specialized (tertiary) psychiatric beds whilestrengthening the community mental health system to assist people with serious mental illnesses tolive successfully in the community. ACT was introduced as a part of an ongoing provincial mentalhealth reform strategy to shift the locus of care for people with serious mental illnesses intocommunity mental health programs. The ACT model was intended as an alternative to lengthy andfrequent hospitalizations.

The present paper describes the evolution of the Ontario ACT program, considering the successfactors for dissemination of EBP identified in the literature. Specifically, the paper describes use offour strategies—standards, funding, technical support and training, and oversight—implementedwithin an improvement framework that included three phases: a start-up or enabling phase, afeedback or reinforcing phase, and a response or corrective action phase.

Start-Up Phase

The Ministry of Health recognized ACT as a best practice and lynchpin program for progressingmental health reform, but funding support was sporadic during the 1990s, and only a handful ofteams were funded before 1997. The final policy document of the decade25 explicitly endorsedACT as a means of providing community support for those most in need, and implementationaccelerated.

The ACT program was situated within a broader mental health system that included specialtypsychiatric hospitals, acute services within general hospitals, and community services (e.g., casemanagement, supportive housing, crisis services, and, more recently, court diversion and earlyintervention in psychosis programs). Community agency accountability was implemented primarilythrough regular reporting to ministry regional consultants (i.e., annual budget submissions and end-of-year summary reports). Service development and modification occurred mainly throughdirectives from senior ministry policy and operational staff, implemented via interaction betweenthe regional consultants and agencies in their portfolio.

The approach for the implementation and evaluation of the ACT program was atypical in thatprovince-wide responsibility was specifically assigned to a unit within the ministry. Between 1998and 2004, the number of ACT teams in the province grew from 10 to 59 teams. The rate of growthvaried from 2 to 14 teams per year. Rollout began using a bottom–up approach, with fundingallocations initially going to agencies that were most interested and prepared. At the same time, theunit implemented strategies to support standardization and to prepare the way for widerimplementation. A description of these strategies follows.

Standards

In 1998, the unit developed the first set of provincial ACT team standards.23 These were adaptedfrom American standards and formed part of the memorandum of agreement between the ministryand each ACT sponsoring agency. The standards described staff requirements, programorganization and operations, admission criteria, service capacity, and service components. Bothexisting and new teams were expected to adhere to the standards.

Funding

Within the community mental health funding stream, the Ministry of Health made a sizablecommitment to the ACT provincial program; however, the size of the team allocations varied, andoften, funding was only sufficient to support partial team implementation (e.g., half a team).


Technical support and training

In addition to the standards, the ministry created a technical support team of two very seniorclinicians from the field to assist with implementation. The technical team was expected to educatenew teams about the ACT mandate, provide didactic training at program start-up, and review teamfunction, especially related to team practice and professional boundaries.26 These functions wereimportant because, as noted by others,18,19 at start-up, staff may need to be convinced that apractice is worth learning and may need assistance to develop the required new skills and approach.The technical team also was expected to assist in developing a future ACT monitoring and fidelityassessment process. Unfortunately, the technical support team was short lived, a victim to limitedfunding in 2001. An accreditation mechanism was not developed.

The ministry also funded regular meetings and conferences for the ACT field, bringing togetherteam representatives from across the province. These meetings were an opportunity to provideexpert continuing education opportunities, share information and knowledge within the field,discuss problem areas and potential responses. Much of the content for these meetings wasdetermined by the needs of the field.

Oversight activities

The ministry combined oversight and stakeholder participation by convening a voluntarytechnical advisory panel (TAP) to implement the oversight function. TAP membership includedrepresentatives from ACT programs in each region of the province (team leaders, psychiatrists, peersupport workers), family organizations, consumers, the Ontario Office of the Patient Advocate, andsenior ministry staff. The TAP met an average of four times a year. A monitoring and evaluationsubcommittee was formed to work with ministry staff to monitor ACT implementation, usingexisting provincial data sources. The committee was expected to provide feedback to the ministryand the field on the degree to which “ACT outcomes in Ontario are consistent with the ACTliterature and evidence-based practice”.27 While the TAP lacked formal authority for monitoringand had limited funding to support the task, the subcommittee began some reporting in 2000 basedon limited data submission from the teams, and all but two teams regularly participated.

Feedback Phase

As the ACT program unfolded, there was criticism from various stakeholders that the programwas failing to meet its mandate. The ministry and local communities were concerned that a numberof teams were under-capacity (i.e., carrying fewer clients that expected) and were not serving thosemost in need (drift from target client group). ACT team leaders complained about staffing problems,which undermined the ability of the team to deliver care according to the other model criteria.

At that time, Ontario did not support routine data collection to monitor community programdelivery or clients served; however, a province-wide planning study funded by the MOHLTC andconducted during 2000–2002 developed a snapshot picture of mental health service systemoperation, including ACT, and provided an opportunity to assess the veracity and further explorethese concerns. Both program and client level data were collected to inform understanding ofprogram function and individual client need.28–30

Program data were collected through a self-report program questionnaire on staffing,accessibility of service, service delivery approaches, and partnerships. Questionnaire domainsand items were adapted from the Dartmouth ACT fidelity scale,4 the Case Management PracticesSurvey,31 and the Community Program Philosophy Scale—extended.32 Program respondents wereinstructed to use the best information available to answer questions. The questionnaire was pre-tested and revised. During implementation, data quality checks addressed missing, inconsistent,and outlier responses.33


Client level data were collected using the Colorado Client Assessment Record (CCAR), astandardized measure of client functioning related to symptoms, behaviors, social and communityfunctioning, substance abuse, and security/management. Trained staff completed assessments for across-sectional sample of randomly selected clients. Adequate inter-rater reliability and validity ofthe CCAR have been demonstrated in Ontario and other jurisdictions.34–36

A cross-sectional study design was used, wherein all community mental health agencies fundedthrough the ministry were expected to complete the program questionnaire and client assessments,unless they had been in operation less than 1 year. The planning study covered six out of sevenhealth regions in the province. Study data were available for 33 ACT teams, representing amajority of eligible teams during the study period. Data were analyzed to assess ACTimplementation in the three areas of stakeholder concern identified earlier: team caseload, staffing,and client profile. A summary of key findings follows.

Team caseload size

While the Ontario ACT team standards prescribed a target of 80 clients for an urban team and 60for a rural team, ACT teams in Ontario were performing well below this target. The averagecaseload was 36 clients with a range of 15–111. The average case ratio for teams was 1:5 (fiveclients per each front line ACT staff) well below the 1:10 ratio described in the literature.Explanations posed by the field to explain these results were tested with the data. Age of the teamdid not make a difference. Teams older than 3 years were just as likely to have low caseloads.Travel time also did not influence caseloads, with teams with low travel times no more likely tohave higher caseloads. The one feature that did impact on caseloads was the number of staff oneach team. Less than half (42%) of teams had the staffing levels recommended in the Ontariostandards, and the average caseload of these teams (37 clients) was substantially lower than themean caseload for fully staffed teams (56 clients).

Team staffing and service delivery

Ontario ACT standards related to staffing required representation from psychiatry, social work,and nursing on the team, as well as a vocational specialist, addiction specialist, and peer supportworker. Of the 28 teams reporting these data, only eight (29%) had the recommended mix ofdisciplines on staff. As already noted, less than half (42%) of teams had full staffing levels.

ACT teams are expected to provide 24/7 coverage; however, only 55% (18/33) of teams wereproviding such coverage. In addition, while 75% of contact with clients should be outside theoffice setting, only 46% of teams (15/33) reported achieving this target.

Team client profile

In Ontario, ACT was intended to serve as an alternative to hospitalization for persons withserious, long-term mental illness. The Ontario standards stipulated admission criteria to ACT thatincluded a diagnosis of severe and persistent mental illness that resulted in “significant functionalimpairments” and “high-service needs” as demonstrated by such indicators as high rates ofhospitalization, criminal justice involvement, and homelessness, among others. The planning datashowed that 83% of ACT clients had a diagnosis of schizophrenia or bipolar disorder and 72% hadmoderate to extreme overall problem severity based on the CCAR assessment. Related tofunctional impairment, however, only 45% of clients had moderate to extreme difficulty with day-to-day functioning. Older teams were more likely to serve clients with difficulties in functioning.Data on client status before program entry related to homelessness, legal involvement, and hospitaluse were not collected in the study.


The planning study results were discussed with TAP and senior ministry staff. In the ensuingdiscussion, the TAP raised a number of possible explanations for results, based on theirexperiences. These centered on funding, human resource supply, and system expectations.

Team caseload size

Regarding team caseload, the TAP agreed that the ability to achieve adequate staffing levels wasan important issue to address in improving ACT capacity. Teams cannot carry large caseloads ofhigh need clients while providing day, evening, and weekend direct services without a critical massof staff to cover the shifts.

Team staffing and service delivery

Several challenges to achieving the target staffing levels were identified. The most obvious wasdifferential funding among teams, which varied from $678,000 to $1,245,000 per team. Manyteams were sponsored by institutions where annual union negotiated wage agreements needed to behonored, putting further pressure on already inadequate budgets. Moreover, ACT programs had notreceived any annual increases to their base budgets since their implementation. Even ACT teamsthat began with full staff complements faced layoff situations as the buying power of their budgetsshrank over several years.

The TAP also commented on recruitment challenges to achieve a full range of disciplines onstaff. Shortages of professional staff such as nurses and occupational therapists particularly in ruraland remote parts of the province, the challenges of working in a high intensity environment, anddifferential pay scales between competing institutions have contributed to recruitment problems forsome teams.

Team client profile

Regarding the appropriate targeting of ACT services, TAP members suggested a number ofexplanations. These include the inappropriateness of referrals, system expectations, and psychiatrybias. In some areas, psychiatric facilities have been slow to make referrals to ACT. This may reflectpoor organizational or system integration of ACT teams within the larger system of care. Inaddition, ACT has been viewed by some local systems as the answer to a range of system problemsincluding lack of housing or crisis services and poor access to services for difficult to serve clientgroups, such as those with borderline personality disorder. As a result, some ACT teams haveexperienced pressures from system partners and their own sponsoring organizations to admitclients who may not be the most suitable for ACT services. The TAP also noted the changingreferral base for the teams, with older teams more likely to take clients from long stay inpatientunits while newer teams have a more diverse referral base, including clients with more experiencein community living. Clients with long periods of institutionalization and little opportunity to retainskills in daily living may have different kinds of needs than those who might come from thecriminal justice system, shelter system, general hospital programs, and other community mentalhealth providers. This may explain why older teams have clients who struggle more with day-to-day functioning. Finally, client intake is affected by the commitment of the ACT psychiatrist toadmit those who are most difficult to serve or who have the most significant risk issues.

Corrective Action Phase

While problems in implementing ACT were being informally discussed by the field, the studyfeedback provided objective information about the performance of provincial teams. The TAP saw


the data as a tool for advocacy and improvement. The ministry saw the data as an opportunity toimprove accountability structures and monitoring. The result is that the ministry, the TAP, and ACTproviders in Ontario are developing formal responses to these issues through revisions tosupporting structures of standards, funding, and monitoring, initially harnessed to facilitate ACTdissemination.

Revised ACT standards

Revised ACT standards were developed by the MOHLTC through the TAP and adoptedprovincially in late 2004 and presented to the field at an annual meeting in 2005. In the revised“Ontario Program Standards for ACT Teams”,27 expectations related to intake criteria, rate ofintake, staffing, and fidelity were strengthened or clarified.

Team capacity and related staffing issues have been addressed through a number of changes tothe standards. The rate of intake is now defined in the standards with a target rate of four to sixclients per month until a full caseload is achieved. This standard directly addressed the issues ofslow rates of intake. In addition, ACT teams are expected to have a minimum one full-time staffper 10 clients on a full/urban team and a minimum total complement of 11 clinical staff (notincluding the program assistant and psychiatrist). Smaller/rural teams are expected to have one full-time staff per eight clients and a minimum total complement of eight clinical staff.

With respect to the team client profile, the standards address targeting and intake criteria byrestating the need for ACT teams to serve clients with serious mental illnesses that are “complexand have devastating effects on functioning” to ensure that “clients with the most serious mentalillnesses have top priority for ACT services.” Functional impairment has been more clearly definedto include “significant difficulty consistently performing the range of daily activities required forbasic adult functioning in the community…significant difficulty maintaining consistent employ-ment or homemaking…or significant difficulty maintaining a safe living environment.”

Funding

At the same time as the revised standards were released, ministry staff was successful innegotiating new funding for the ACT program to address historical problems and support revisedstandard implementation and to develop new teams. Funding was to be rolled out over 3 years. Thefirst year of funding went to “topping up” existing teams to meet the full-time equivalent staffinglevels, case ratio, and other requirements of the revised standards. Thus, while the field now hasmore stringent requirements for performance, teams are also receiving enhanced funding to meettheir targets, a recognition that some of the problems with ACT performance are directly linked toinadequate funding.

Oversight activities

A new section in the standards describes the accountability requirements for performanceimprovement and program evaluation. Programs are expected to demonstrate fidelity to the ACTmodel and monitor client outcomes and satisfaction. Programs are required to regularly review theappropriateness of admissions. They are also required to participate in the central monitoring andevaluation procedures established by the MOHLTC.

The TAP has continued efforts to collect system data and produce an annual data monitoringoutcome report for the field and for the ministry. To date, four reports have been prepared. Eachyear, the TAP selects a specific issue and related performance indicators to address. For example, in2002/2003, data were collected on the number of ACT clients whose pharmacological treatmentwas consistent with Canadian Psychiatric Association guidelines for the treatment of schizophre-


nia, the percentage of clients with concurrent disorders and legal involvement, and programsources of referrals (related to concerns about the target population).37 Future report cycles mayfocus on critical model ingredients by assessing where local adaptations and departures from themodel can be tolerated without adverse effects on clients. Of interest, for example, are acomparison of hospital outcomes between teams that meet the 24/7 crisis fidelity measure andthose that do not and the association between vocational outcomes and the availability of avocational specialist on the team. While system-wide reporting has improved in the province, thesemonitoring effects are still limited by data availability and quality. The TAP is exploring the use ofstandardized functional measures.

Technical support and training

No changes were made related to these supporting activities for best practices dissemination.

Conclusion

The ACT model continues to be widely disseminated in Ontario as a part of mental healthreform and a shift in locus of care from hospital to community. Though heavily researched as aservice delivery model, experience from a number of jurisdictions, including Ontario, suggests thatthe widespread implementation of the model is challenging. In the Ontario experience, policy andimplementation strategies continue to evolve in response to research, evaluation, and critique. Theexperience, to date, reflects a continuous improvement cycle where initial feedback has stimulatedchange. The addition of strategies such as technical support, accreditation, and more sophisticatedsystem monitoring strategies are essential for ensuring continued feedback and the success of anexpanding provincial program.

Implications for Behavioral Health

The collective work of the Ontario Ministry of Health, the TAP, and the ACT field in Ontariorepresents a case example of the positive evolution of policy and implementation issues for ACTdissemination in Ontario. The ministry unit responsible for implementation of ACT has workedclosely with the field, both directly and through the TAP, to identify and address problems withACT performance. Regular dialogue through annual meetings and quarterly TAP meetings hasprovided a direct conduit to senior ministry staff, and the availability of the planning data on ACTperformance was critical in clarifying implementation problems. Concrete corrective actions havefollowed from this dialogue—the revision of the ACT standards and an expectation of adherence tothe standards, and increased funding to improve ACT team functioning. There has also been areiteration of the commitment to evidence-based programming. These changes should strengthensystem leadership by the ministry and increase accountability within the system.

Despite these successes, strategies to provide systematic practice feedback on model fidelity andteam functioning—technical assistance; accreditation; and performance monitoring—are stillunderdeveloped. The ministry’s attempt to provide technical support and develop an accreditationapproach was short-lived; however, these are key elements in the dissemination of EBPs, includingACT.2,17,18 In the early days of ACT rollout in Ontario, teams were implemented where there wasreadiness and a commitment to the ACT model. As noted by Ganju,17 teams that were rolled outlater as part of an overall provincial policy may not demonstrate the same readiness, commitment,and leadership as the early adopters. Technical assistance will be needed to ensure that teamsadhere to the model and the standards and are committed to the population they are intended toserve. An ongoing accreditation is useful both as an accountability tool and as a mechanism foridentifying common challenges across ACT teams.


The availability of the planning data on ACT performance focused the dialogue within the field.Evidence on low team capacity provided a rationale for increasing team-based funding. Regularmonitoring through annual comparative data can provide a basis on which to continue measuringsystem and individual team performance. While there is recognition of the need to improve systemmonitoring through additional data collection and focus on outcomes such as daily functioning, theMOHLTC will need to provide a sustainable resource base to accomplish the sophisticatedmonitoring and evaluation needed for such a complex intervention as ACT.

ACT is an expensive evidence-based program intended to serve a well-defined client group.Though this study describes a particular jurisdiction, the experience is not unique. ACT has beenwidely implemented in England and the USA where similar implementation challenges have beennoted.14,19 Much attention is being paid to the challenges of disseminating EBPs. The issues facedin Ontario, such as drift in target population, inadequate funding levels, workforce shortages inrural areas, and lack of ongoing system-wide monitoring, are consistent with those of otherjurisdictions.18,38 We would argue that it is necessary to attend to issues of leadership, funding,standards, oversight capacity, implementation support, and ongoing feedback on practice regardlessof the system in which ACT teams are situated.

References

1. Bond GR, Drake RE, Mueser KT, Latimer E. Assertive community treatment for people with severe mental illness. Disease Managementand Health Outcomes. 2001;9(3):141–159.

2. Phillips S, Burns B, Edgar E, et al. Moving assertive community treatment into standard practice. Psychiatric Services. 2001;52(6):771–779.

3. Stein LI, Test MA. Alternative to mental hospital treatment: I. Conceptual model, treatment program, and clinical evaluation. Archives ofGeneral Psychiatry. 1980;37:392–397.

4. Teague GB, Bond GR, Drake RE. Program fidelity in assertive community treatment: Development and use of a measure. AmericanJournal of Orthopsychiatry. 1998;68(2):216–232.

5. Mueser KT, Bond GR, Drake RE, Resnick SG. Models of community care for severe mental illness: A review of research on casemanagement. Schizophrenia Bulletin. 1998;24(1):37–74.

6. Marshall M, Lockwood A. Assertive community treatment for people with severe mental disorders. The Cochrane Database ofSystematic Reviews. 2003;3.

7. Ziguras SJ, Stuart GW. A meta-analysis of the effectiveness of mental health case management over 20 years. Psychiatric Services.2000;51(11):1410–1421.

8. Latimer EA. Economic Impacts of assertive community treatment: A review of the literature. Canadian Journal of Psychiatry.1999;44:443–454.

9. Torrey WC, Drake RE, Dixon L, et al. Implementing evidence-based practices for persons with severe mental illnesses. PsychiatricServices. 2001;52(1):45–50.

10. Flynn LM. Commentary. Schizophrenia Bulletin. 1998;24:30–32.11. News & Notes. President Clinton announces an array of initiative at First White House Conference of Mental Health. Psychiatric

Services. 1999;50:980–981.12. Centre for Mental Health Services. Mental Health: A Report of the Surgeon General. Rockville, MD: Substance Abuse and Mental Health

Services Administration, 1999.13. Kirby JL, Keon WJ. Out of the shadows at last: Transforming mental health, mental illness and addiction services in Canada. Ontario,

Canada: Committee Business, Senate, Recent Reports, 38th Parliament, 1st Session, 2006.14. Wright C, Burns T, James P, et al. Assertive outreach teams in London: Models of operation. British Journal of Psyc hiatry.

2003;183:132–138.15. McGrew JH, Bond GR, Dietzen LL, et al. Measuring the fidelity of implementation of a mental health program model. Journal of Consult

Clinical Psychology. 1994;62:670–678.16. McHugo GJ, Drake RE, Teague GB, et al. The relationship between model fidelity and client outcomes in the New Hampshire Dual

Disorders Study. Psychiatric Services. 1999;50:818–824.17. Ganju V. Mental Health Quality and Accountability: The Role of Evidence-Based Practices and Performance Measurement.

Administration Policy in Mental Health & Mental Health Service Research. 2006;33(6):659–665.18. Moser L, DeLuca N, Bond G, Rollins A. Implementing Evidence-Base Psychosocial Practices: Lessons Learned from Statewide

Implementation of Two Practices. CNS Spectrums. 2004;9(12):926–936.19. Isett K, Burnam M, Coleman-Beattie B, et al. The State Policy Context of Implementation Issues for Evidence-Based Practices in Mental

Health. Psychiatric Services. 2007;58(7):914–921.20. Rapp C, Bond G, Becker D, et al. The Role of State Mental Health Authorities in Promoting Improved Client Outcomes through

Evidence-Based Practice. Community Mental Health Journal. 2005;41(3):347–363.21. Munetz M, Morrison A, Krake J, Young B. Statewide Implementation of the Crisis Intervention Team Program: The Ohio Model.

Psychiatric Services. 2006;57(11):1569–1571.


22. Magnabosco J. Innovations in mental health services implementation: a report on state-level data from the U.S. Evidence-Based PracticesProject. Implementation Science. 2006;1(13):1–11.

23. Ontario Ministry of Health and Long Term Care. Program Standards for Assertive Community Treatment. Toronto, Ontario, 1998.24. Hartford K, Schrecker T, Wiktorowicz M, et al. Four decades of mental health policy in Ontario, Canada. Administrative Policy in Mental

Health. 2003;31:65–73.25. Ontario Ministry of Health and Long Term Care. Making It Happen. Toronto, Ontario: Queen's Printer for Ontario; 1999.26. Stovall , J. Is assertive community treatment ethical care? Harvard Rev Psychiatry. 2001;9:139–143.27. Ontario Ministry of Health and Long Term Care. Ontario Program Standards for ACT Teams, Second Edition. Toronto, Ontario, 2004.28. Koegl C, Durbin J, Goering P. Mental health services in Ontario: How well is the province meeting the needs of persons with serious

mental illness. Toronto, Ontario, Canada: Centre for Addiction and Mental Health; 2004.29. Lunsky Y, Bradley E, Durbin J, et al. The Clinical Profile and Service Needs of Adults with a Dual Diagnosis in Psychiatric Hospitals.

Psychiatric Services. 2006;57(1):77–83.30. Durbin J, Goering P, Cochrane J, et al. Needs-based planning for persons with schizophrenia residing in board-and-care homes.

Schizophrenia Bulletin. 2004;30(1):123–132.31. Ellison ML, Rogers ES, Sciarappa K, et al. Characteristics of mental health case management: Results of a national survey. The Journal

of Mental Health Administration. 1995;22(2):101–112.32. Jerrell JM, Hargreaves WA. Community Program Philosophy Scale (Expanded). San Francisco: University of California; 1996.33. Durbin J, Goering P, Streiner D, Pink G. Program structure and continuity of mental health care. Canadian Journal of Nursing Research.

2004;36(2):12–37.34. Durbin J, Cochrane J, Goering P, Macfarlane D. Needs based planning: Evaluation of a level of care planning model. Journal of

Behavioral Health Services and Research. 2001;28(1):67–80.35. Ellis RH, Wackwitz JH, Foster M. Uses of an empirically derived client typology based on level of functioning: twelve years of the

CCAR. Journal of Mental Health Administration. 1991;18(2):88–100.36. Schwartz RC. Reliability and validity of the functional assessment rating scale. Psychological Reports. 1999;84:389–391.37. Bursten J, Cooper C, Davidson B, et al. 2002/03 Ontario ACT Data Outcome Monitoring Report. Ontario Ministry of Health and Long

Term Care, Toronto, Ontario, 2004.38. Meyer PS, Morrissey JP. A Comparison of Assertive Community Treatment and Intensive Care Management for Patients in Rural Areas.

Psychiatric Services. 2007;58(1):121–127.


system-wide implementation of act in ontario: an ongoing improvement effort

Documents