Information in practice

Kaiser Permanente’s experience of implementing anelectronic medical record: a qualitative studyJ Tim Scott, Thomas G Rundall, Thomas M Vogt, John Hsu

AbstractObjective To examine users’ attitudes toimplementation of an electronic medical recordsystem in Kaiser Permanente Hawaii.Design Qualitative study based on semistructuredinterviews.Setting Four primary healthcare teams in four clinics,and four specialty departments in one hospital, onOahu, Hawaii. Shortly before the interviews, KaiserPermanente stopped implementation of the initialsystem in favour of a competing one.Participants Twenty six senior clinicians, managers,and project team members.Results Seven key findings emerged: users perceivedthe decision to adopt the electronic medical recordsystem as flawed; software design problems increasedresistance; the system reduced doctors’ productivity,especially during initial implementation, which fuelledresistance; the system required clarification of clinicalroles and responsibilities, which was traumatic for someindividuals; a cooperative culture created trade-offs atvarying points in the implementation; no singleleadership style was optimal—a participatory,consensus-building style may lead to more effectiveadoption decisions, whereas decisive leadership couldhelp resolve barriers and resistance duringimplementation; the process fostered a counter climateof conflict, which was resolved by withdrawal of theinitial system.Conclusions Implementation involved several criticalcomponents, including perceptions of the systemselection, early testing, adaptation of the system tothe larger organisation, and adaptation of theorganisation to the new electronic environment.Throughout, organisational factors such as leadership,culture, and professional ideals played complex roles,each facilitating and hindering implementation atvarious points. A transient climate of conflict wasassociated with adoption of the system.

IntroductionElectronic medical record systems have great potentialto improve the quality of health services. However, fewservice providers have adopted them. Cost, limited evi-dence linking adoption and improved outcomes, andconcerns about disruptions during implementationmay be important barriers.1–5

Electronic medical record systems may challengebeliefs about how health care should be organised;using physicians to enter data may be inefficient andperceived as demeaning, and clinicians and managersmay need to learn how to use specific software, causingfrustration. Formal evaluations of electronic medicalrecords rarely address these organisational issues.6 Thismissing information could enhance the viability ofexisting electronic medical records and greatly informthe implementation of new systems such as the UKNational Programme for IT (NPfIT) for the NHS.7–9

We examined the experience of implementing anelectronic medical record system in Kaiser PermanenteHawaii. Using semistructured interviews, we identifiedthe critical events in the system implementation, theimpact of organisational culture and leadership, andthe effects on clinical practice and patient careprocesses as perceived by the system’s users.

MethodsBackgroundKaiser Permanente is the United States’ largestnon-profit healthcare system, serving 8.2 millionmembers in eight regions. Kaiser Permanente Hawaiihas 26 primary care teams in 15 clinics, and one hospi-tal. It serves 234 000 members across three islands. Theaverage team has four doctors, three nurses, and nineother staff members.

Kaiser Permanente compared two electronic medi-cal record systems in the 1990s, EpicCare, developedby Epic Systems, and Clinical Information System(CIS), jointly developed by Kaiser Permanente andInternational Business Machines (IBM). In 1999, thesecond-generation CIS system was chosen for all itsregions, starting with Hawaii.

Hawaii began to prepare for CIS in December1999 with a planned starting date of October 2000.After a 12 month delay related to the operating system,the first site started using CIS in October 2001.However, in 2003 the company decided that the Epic-Care system had matured beyond CIS, and now was

The interview prompt sheet used in the study appears onbmj.com

This is the abridged version of an article that was first postedon bmj.com on 3 November 2005: http://bmj.com/cgi/doi/10.1136/bmj.38638.497477.68

School ofManagement,University of StAndrews, StAndrews KY16 9SSJ Tim Scottsenior lecturer

School of PublicHealth, Universityof California,Berkeley, CA 94720,USAThomas G Rundallprofessor

Kaiser PermanenteCentre for HealthResearch, Hawaii,HI 96817, USAThomas M Vogtdirector

Kaiser PermanenteDivision ofResearch, Oakland,California, CA94612, USAJohn Hsuphysician scientist

Correspondence to:J T Scottjts1@st-andrews.ac.uk

BMJ 2005;331:1313–6

1313BMJ VOLUME 331 3 DECEMBER 2005 bmj.com

better able to meet its needs. All Kaiser Permanenteregions halted implementation of CIS and began plan-ning for implementing EpicCare.

At that time, a third of the company’s Hawaiiansites had implemented CIS fully, and the rest had read-only access, some with order entry functionality. Formany in Kaiser Permanente Hawaii, the 28 monthexperience had been, to use a Hawaiian surfing meta-phor, truly a “wild ride.”

InterviewsJTS held semistructured, recorded interviews with12 clinicians and five managers in four teams, located infour clinics and the Kaiser Permanente hospital, andwith nine CIS project team members during March andApril 2003, about one month after the announcementto halt CIS implementation. Preparation for the newelectronic medical record system began during theinterview period. In the clinics with CIS implemented, itremained in use until replaced by EpicCare in 2004.

In the 60-90 minute interviews we sought respond-ents’ views on four implementation issues: (a) criticalevents in the implementation of CIS; the roles played by(b) organisational leadership and (c) culture; and (d) theCIS related changes in clinical practice (see bmj.com).Interviews were transcribed and analysed inductivelyand thematically. Responses were coded and collated tocreate themes. Unclear responses were clarified withinterviewees. We held regular discussions to reviewthemes and clarify facts. This paper reports only themesthat were mentioned by at least four respondents.

ResultsCritical processes in the implementation of CISSelection of the electronic medical record systemOnly one respondent expressed approval of the CISchoice at the time of the interview: 18 reported dissat-isfaction with the choice, 10 thought clinicians had notadequately “bought into” the decision, and seven iden-tified conflicting priorities between the organisation asa whole and individual clinicians.

“It was a bit of a turkey. I was actually on the group thatpicked it. So we had to go out and defend a product that weknew was flawed. And other people knew it wasflawed.”—Clinician, implementation team member

“With CIS it seems the highest priority was, ‘How can wedocument as completely as possible and then extract infor-mation easily so we can get paid and so maybe we can doresearch?’ but had very little to do with taking care of thepatient who comes to your office.”—Clinician

Nine respondents criticised the higher than antici-pated and escalating implementation costs, and feltthat Kaiser Permanente disproportionately bore thesoftware development costs.

“We had a 12 month preparation period. The productwasn’t delivered for another 14 months. That affects yourculture.”—Clinician manager

“The delay had multiple consequences to our wholeexperience with this. Lot of time goes by, lot of things happenin people’s lives. And my internist needed to go do otherthings. And my paediatrician also needed to move. So theimplementation date is shifting, and my players are(too).”—Clinician

“Any improvements we had to pay for. So we’re paying forIBM’s entire development costs.”—Clinician, implementa-tion team member

Respondents consistently recalled suspecting thatCIS was the wrong choice for Kaiser PermanenteHawaii, but may have been biased by the recent nega-tive experience. At interview, 24 respondents wereoptimistic that EpicCare would prove a better system.All respondents thought the hardest challenge—tochange from paper to computer—had been accom-plished with CIS and that the new implementationwould be easier. But at least eight respondents werealso cautious. Formal evaluations of the new EpicCaresystem are ongoing.

“Even [EpicCare] is going to take us more time. On a badday we might see 30 patients. There’s no way that you can dothat on a computerised system unless you’re going to do aslousy a note as possible.”—Clinician

Design and early testingTwenty three respondents reported substantial soft-ware problems.

“It was pretty clear that this product had a lot of problems—from our very first site.”—Clinician, implementation teammember

“The problem for internal medicine is that they go througha more complex process to arrive at a diagnosis, but CIS isn’treally designed to do that. CIS assumes that the diagnosis isknown at the beginning, or shortly after the beginning, ofthe visit.”—Implementation team member

Adaptation of the electronic medical record for widespread useFour respondents noted the difficulty and expense ofmodifying CIS software. Clinicians sought improvedusability and flexibility, whereas national headquartersprioritised business and executive functions such asaccurate coding and reimbursement.

“We wanted to make a whole lot of changes, and [IBM]would point out why it would take time and cost money, andthe smallest change would be a fight and it would cost tensof thousands of dollars.”—Implementation team member

Adaptation of the organisation to the new electronic medicalrecord environmentSeventeen respondents reported that CIS had reducedclinician productivity, primarily because of extra work,such as processing laboratory result reports, enteringorders, and navigating through the system. Fourteenclinicians reported that the extra time burdens fromusing CIS (estimated to be 30-75 minutes a day)persisted even after the initial learning period andaffected patient care (such as by making it moredifficult to fit in “overload patients”). The reasonsvaried: eight respondents said CIS required too manysteps to accomplish simple tasks; 12 felt the system wastoo cumbersome to accommodate the range of clinicalneeds or patients with multiple problems, requiringmultiple templates; and nine reported a lack of clinicalcapacity to absorb changes during implementation.

Although the CIS implementation included tem-porary provision of extra clinicians to help with theinitial additional workload, several respondentsdoubted whether they would be able to achievepre-CIS productivity levels.

“We’d started out in the high 90th percentile in terms of ourutilisation, so when we tried to add the constraints of thesystem, basically it broke.”—Clinician

Information in practice

1314 BMJ VOLUME 331 3 DECEMBER 2005 bmj.com

Sixteen respondents said that CIS preparation hadrevealed a variety of questionable practices and sharp-ened people’s sense of accountability. Eleven respond-ents believed implementing CIS had clarified staffresponsibilities for clinical decisions, but these tasksrequired much unanticipated work.

“Somehow it has changed the psyche of people, they aremore aware of what they are putting in the chart. It’s almostlike they didn’t really care what they wrote on paper, butnow it’s electronic and people can read everything.”—Implementation team member

Revised roles created shifts in work responsibilities,which led some doctors to complain that they werebecoming expensive order entry clerks. Otherswelcomed the greater accountability.

“No question in my mind, it’s forced me to be more organ-ised, more accountable. It’s forced me to do what I shouldhave been doing all along.”—Clinician

Respondents also reported innovative adaptations,such as allowing nurses to act as results managers toscreen laboratory results. Medical assistants started pri-oritising electronic task items. Some clinics starteddeveloped new appointment types, such as greater useof telephone consultations.

“We’ve seen more sharing of responsibility, having nurses dofollow-up phone calls, and understanding specifically what a[medical assistant] can do, and shifting some work off thephysicians on to others.”—Clinician, implementation teammember

Organisational culture and implementationRespondents characterised Hawaiians as averse toconflict and likely to interpret negative feedback aspersonal criticism. The culture minimised active resist-ance but also inhibited criticism before and duringimplementation, thus depriving decision makers ofimportant feedback.

“Hawaii’s culture is very non-confrontational, you know, ‘Justbe nice, agree’; so the feedback has been relative tothat.”—Implementation team member

Organisational leadership and CISFourteen respondents discussed the importance ofleadership. Respondents often stated that the electronicmedical record system selection process needed to bemore participatory, but in the implementation phasesome respondents valued decisive hierarchical leader-ship. Respondents described local leaders as consensusseeking, which at times exacerbated implementationchallenges or encouraged passive resistance.

ConflictAs these critical processes unfolded, a climate ofconflict associated with CIS arose. Five respondentsreported internal conflict between their commitmentto “going electronic” and scepticism about CIS, andsome initially receptive clinicians became alienated bythe implementation. A mingled feeling of relief andloss was associated with the withdrawal of CIS.

“What got docs here frustrated was nobody really seemed tolisten. And they had to compromise their values and ethicsto help the system work. That’s where I saw very amiable,nice, quiet people starting to talk stink behind the scenes.”—Clinician

DiscussionSeven key findings emerged on the implementation ofthe CIS electronic medical record in Kaiser Perma-nente Hawaii:x Many users perceived the initial selection of CIS tobe detached from the local environmentx Software design and development problemsincreased local resistancex CIS reduced clinicians’ productivityx CIS initially clarified and then changed roles andresponsibilitiesx Culture had varying effects: cooperative valuesminimised resistance to change early on but alsoinhibited feedback during implementationx Leadership had varying effects: participatory lead-ership was valued for selection decisions, but hierarchi-cal leadership was valued for implementationx An overall effect was a counter climate of conflict inthe company, which withdrawal of CIS resolved.

Implications for other healthcare organisationsA participatory process in selecting the system and infine tuning its capabilities is important. This grass rootsinvolvement is important to generate commitment;there may never be a perfect software system for allusers, or which will achieve specific local goals.Extensive software testing of the vendor’s claims forbaseline functionality and the system’s adaptability tolocal needs is important before implementation. Users’frustration with software problems can quickly escalateand result in resistance to implementation.

Organisational culture and leadership—Building asupportive, cooperative culture may pay dividendsduring the stress of implementation, but special effortsmay be required to obtain critical feedback. Differentleadership roles and styles are needed in the process:during selection of the system, participatory leadershipis needed; during implementation, however, decisivehierarchical leadership is needed.

What is already known on this topic

Electronic medical record systems have great potential to improve thequality of health services, but implementing such a system can beextremely disruptive

Better understanding of organisational contexts and responses toimplementation may help to minimise disruption, but few formalevaluations have been made

What this study adds

Introduction of an electronic medical record system to a non-profithealthcare organisation created several challenges: many users felt theselection of the system was detached from the local environment,sparking doubt and resistance; problems with software developmentincreased local resistance, as did clinicians’ reduced productivity

The system first clarified and then changed roles and responsibilities—often, though not always, for the better

The organisation’s cooperative culture minimised overt resistance tochange but also inhibited constructive feedback during implementation.Leadership also had varying effects: participatory leadership was valuedduring selection of an electronic system, but hierarchical leadership wasvalued during implementation

Information in practice

1315BMJ VOLUME 331 3 DECEMBER 2005 bmj.com

Roles, responsibilities, and relationships of clinical staff—During the early stages of implementation, changes inclinicians’ productivity may require extra staff and theability to make continuous adjustments. As the organi-sation adapts to the new electronic system, the capabil-ity is needed to document what care a patient received,who provided it and when, so processes may need toadapt to revised professional and legal standards.

Limitations of studyOur study captures only a snapshot view, during avolatile phase of implementation and transition fromone electronic medical record system to another. Therespondents’ perceptions should be seen in thiscontext. In fact, another Kaiser Permanente region hadimplemented an earlier CIS version successfully. How-ever, our overall findings highlight issues likely to befaced by organisations implementing or modifying anelectronic medical record system.

Contributors: See bmj.com

Funding: The Commonwealth Fund of New York supportedTim Scott through a Harkness Fellowship. The GarfieldFoundation provided funding for the data collection.Competing interests: TMV and JH work for Kaiser Permanente,the organisation implementing the system under study.Ethical approval: The Kaiser Permanente Hawaii InstitutionalReview Board gave ethical approval.

1 Garrido T, Jamieson L, Zhou Y, Wiesenthal A, Liang L. Effect of electronichealth records in ambulatory care: retrospective, serial, cross sectionalstudy. BMJ 2005;330:581-5.

2 Raymond B, Dold C. Clinical information systems: achieving the vision. Oak-land, CA: Kaiser Permanente Institute for Health Policy, 2002.

3 Gibbs W. Taking computers to task. Sci Am 1997;278:64-71.4 Kaplan B. Development and acceptance of medical information systems:

an historical overview. J Health Hum Resour Adm 1988;11:9-29.5 Littlejohns P, Wyatt J, Garvica L. Evaluating computerised health

information systems: hard lessons still to be learnt. BMJ 2003;326:860-3.6 Aarts J, Doorewaard H, Berg M. Understanding implementation: the case

of a computerized physician order entry system in a large Dutch univer-sity medical center. J Am Med Inform Assoc 2004;11:207-16.

7 Department of Health. Building the information core: implementing the NHSplan. London: Department of Health, 2001.

8 Humber M. National programme for information technology. BMJ2004;328:1145-6.

9 NHS Information Authority. NHS IA strategic plan for 2002-05. Birming-ham: Crown, 2002.(Accepted 20 September 2005)

Commentary: Trouble in paradise—learning from HawaiiSheila Teasdale

Scott and colleagues have set before us the sad story ofthe failed implementation of an electronic medicalrecord system in hope that readers can learn from themistakes.1 The English National Programme for IT,2 asthe largest implementation of an electronic medicalrecord system in the world, is singled out by theauthors as being a potential beneficiary of the lessonsin this report.

There are parallels between what Kaiser Perma-nente tried to do in Hawaii and what is planned for theEnglish NHS: Kaiser Permanente is a very largehealthcare organisation, covering a widely geographi-cally dispersed population of eight million patientsacross all health sectors (though this implementationcovered fewer than 250 000 patients). The overall goalwas to implement an electronic medical record for useby all clinicians, providing an integrated system. Thisevaluation looked specifically at the organisationalissues—consultation, communication, leadership, deci-sion making, education and training, changemanagement—as it is well known (though often sadlyignored) that getting these things right is crucial forthe success of any innovation that involves peoplechanging the way they do things in the workplace.

The reasons put forward for the failure of the imple-mentation will come as no surprise to those with experi-ence of working in health informatics: the initial decisionmaking was seen as remote from the clinical user base;resistance was increased by poor product design; clinicalproductivity was reduced (although this had beenplanned for in the implementation, many staff felt thatthey would be unable ever to return to their previouslevels of performance); roles and responsibilities wereunclear and were constantly changed; the cooperativeculture so prized by Hawaiians inhibited honestfeedback; leadership styles were not appropriate to thesuccessive phases of implementation; and a climate ofconflict was the result.

The authors suggest ways of avoiding suchoutcomes—all of them involving people, not technology.Their recommendations echo those of Nancy Lorenzi,president of the International Medical InformaticsAssociation and an expert on change management inhealth informatics. She recently enumerated strategiesfor effecting successful change, none of which are new,but without which failure is inevitable: set andcommunicate clear objectives and formulate a strategicplan (and modify when necessary); work at achievingownership of the plan by people at all levels; pay atten-tion to the organisational culture (“culture eats strategyfor breakfast”) and whether it supports the changesbeing implemented; develop leaders and champions forthe change (not just those in traditional positions ofpower); be patient and resist false urgency; stay involvedand keep communicating; evaluate; seek feedback (andact on it); plan ahead for the next phase of change.3

There are now encouraging signs of increasinginvolvement of clinicians proficient in informationtechnology within much of the National Programmefor IT and an increasing level of informed andconstructive debate, which is being listened to by NHSConnecting for Health. This is a positive and welcomedevelopment and one which must be fosteredthroughout the NHS; we simply cannot afford for thisimplementation to fail.

Competing interests: ST is strategic director of PRIMIS+, aneducational programme that is fully funded by NHSConnecting for Health. She is also editor of Informatics inPrimary Care.

1 Scott JT, Rundall TG, Vogt TM, Hsu J. Kaiser Permanente’s experience ofimplementing an electronic medical record: a qualitative study. BMJ2005;331:1313-6.

2 NHS Connecting for Health. National Programme for IT in the NHS.www.connectingforhealth.nhs.uk (accessed 26 Oct 2005).

3 Teasdale S. Shaping sands, shifting services [editorial]. Inform Prim Care2005;13:81-2.

Information in practice

University ofNottingham,NottinghamNG7 2RDSheila Teasdalestrategic director,PRIMIS+

sheila.teasdale@primis.nottingham.ac.uk

1316 BMJ VOLUME 331 3 DECEMBER 2005 bmj.com