foumal of Advanced Nursing, 1094,20,1148-1155

Beyond measurement: fourth-generationevaluation in nursingTina Koch RGN BA CHC PhDSenior Lecturer, School of Nursing, Flinders University of South Australia, GPOBox2100, Adelaide 5001, Australia

Accepted for pubhcation 23 February 1994

KOCH T (1994) foumal of Advanced Nursing 20,1148-1155Beyond measurement: fourth-generation evaluation in nursingWhoever believes that the 'scientific' method should not be the sole approach toquality assurance and evaluation research may be attracted to foxiith-generationevaluation The first three generations of evaluation have been categorized as'measurement-orientated', 'objective-orientated' and 'judgement-orientated',whereas fourth-generation evaluation is 'negotiation-onentated' In the presentpaper the author embarks on a descnption of these generations and makessuggestions for the implementation of fourth-generation evaluation in the healthcare setting The appeal of fourth-generation evaluation is that it argues for all'stakeholders' to have a right to place their claims, concems and issues on thenegotiating table The evaluator acts as a facilitator of the evaluation process'Stakeholders' refers to all people in the setting who are affected by theevaluation, mcluding managers, evaluators, medical and nursing staffParticipation of patients/chents is central in the negotiation process

EVALUATION RESEARCH IN NURSING } ^ ^^^ P"P^^ *\^ ^"^''^ "^^" '̂ ^^^"^^ "^f' generationsof evaluation as they apply to nursing, and make sugges-

Evaluahon research in nursing has emerged as one of the tions for the implementation of fourth-generahon evalu-most important achvihes in recent decades The term, ation m the health care settmgevaluahon reseeirch, has been applied to a wide spectrumof mveshgahve activihes that employ a problem-solving piRST-GENERATION EVALUATIONprocess In the mshtutional setting this type of evaluationIS ofien called quality assurance Evaluations are usually The first generation is seen m the emergence of scientificengaged with measurement, and their results are used by management m busmess and industry and is typified bymanagers to control and predict aspects of the health ser- early studies m educahoned research measunng the attn-vice. Its staff and pahents So far, however, this movement butes of school children, e g Bmet's IQ tests Evaluahonhas grown up uncnhcally is treated as a scientific process (Pfeffer & Coote 1991)

It IS suggested that quality assurance activities camed Guba & Lincoln (1989) have termed this first generahon ofout m the health care sector can fit into a four-generahon evaluahon the 'measurement generahon'evaluahon fi'amework denved firom Guba & Lmcoln's Three examples of first generahon approaches m nurs-(1989) work m the field of educahon Accordmg to Guba mg are pahent opmion surveys, the use of instruments in& Lincoln (1989), each succeeding generahon of evaluahon the measurement of quality of care (audit tools), and therepresents a step forward These authors assert that fourth- use of patient classificahon tools In all instances, bound-generahon evaluation is the culmmahon of generations anes and quahty cntena are selected by health carecharactenzed by measurement-, objechve- and judgement- professionals Pahents are rarely consulted m the devel-onentated approaches The aim of fourth-generahon evalu- opment of these tools, except m the testing ofahon IS to move toward a process of negohahon between psychometric dimensions What is bemg measured andall parhcipants

1148

Fourth-generation evaluation m nursing

the actual concerns of pahents often differ (Elbeck 1987,Kanar 1988)

SECOND-GENERATION EVALUATION

Second generahon evaluahon is 'an approach charac-tenzed by descnphon of pattems of strengths and weak-nesses with respect to certam stated objectives' (Guba &Lmcoln 1989} Guba & Lmcoln use the example ofR Tyler's work in educahon to expleun second generahonevaluahon Tyler, in 1933, inihated educahon programmeevaluahon m an 8-year study of secondary schools m Ohio,USA Tyler (1949) developed basic cumculum andmstruchon cntena, which were later taken up by nurseacademics in the USA and euround the globe The NahonalLeague for Nursmg (NLN) m the USA used the Tyler modelfor the appraisal of the 1983 baccalaureate and higherdegree programmes in nursing (Diekehnann et al 1989)The mfluence of this dommant paradigm of higher edu-cahon IS implicit m the NLN cntena However, this typeof evaluahon would be tietter termed 'objechve-orientated'as the term 'descnphve' used by Guba & Lmcoln (1989) istoo general and can be apphed across other generahons ofevaluahons

Nursing process

In the shift from educahon to nursing prachce, evaluahonof nursmg care is an essenhal component of the 'nursmgprocess' The nursing process, as first descnbed by Yura &Walsh (1973), mvolves the processes of assessing, plan-ning, lntervenmg and then evaluating nursing care Thenursing process applies 'the scienhfic method' to nursmgprachce Patient goals are stated in terms of measurableobjechves The nursing process requires that the pahent isassessed, that an objechve is stated in measurable terms,and that the care is evaluated agamst the formulated objec-hve Memy nurses, dunng the assessment phase, attemptto identify pahents' needs as opposed to pahents' prob-lems, how^ever, the nursmg process remains predomi-nantly a problem-onentated process For measurement tobe possible, stnngent nursmg documentahon is reqmredEvaluahon is bmlt mto the recording process The nohonof measunng quality of care against stated objechves andoutcomes cheuBCtenzes second-generahon evaluahon

The ways m which the nursmg process, nursing diag-nosis and their documentahon are used in second-generahon evaluahon have the potenhal to obscure theactual concems of pahents (Koch 1993) Pahent mvolve-ment m the nursmg process often depends on the individ-ual nurse, and the degree of nurses' collaborahon withpatients m plannmg care also hinges on the tjrpe of nursmgtheory dnvmg the nursmg process Nursmg diagnosis cat-egones pnncipally centre around the actual or potenhalharmfiil effects of illness on the health status of pahents/

clients Although nursing mtervenhons are not lunited tothe treatment of problems, the potential for some mdivid-ual concems to be ignored is probable when usmg nursingdiagnosis categones

THIRD-GENERATION EVALUATION

Third-generahon evaluahon may be viewed as the 'mana-genal' or 'excellence' approach (Pfeffer and Goote 1991)Guba & Lmcoln (1989) term this 'judgement-onentated'One of the problems of earlier generahons relates to theobjechves themselves

It required that the objectives themselves be taken as problematic,goals no less than performance were subject to evaluaUonJudgement requires standards against which the judgements canbe made But the mclusion of standards, that must, by deflmtionof the genre, be value laden into a scientific and putatively value-free enterprise was repugnant to most evaluators

(Cuba & Lincoln 1989)

It IS suggested by these authors that some evaluatorsfeared pohhcal vulnerability m taking on the role ofjudges, yet were encouraged to accept this obligation Theargument for acceptance was based on the claim that eval-uators were 'without doubt the most objechve' (Scriven1972) Green & Lewis (1986) review evaluation modelsdeveloped for the field of education, including decision-onentated models such as GIPP (context, input, processand product), an evaluahon developed by Stufflebeam(1983) and the Goal-Free model developed by Scnven in1972 The evaluator progressed to the role of judge 'whileretaimng the earlier techmcal and descnptive funchonsas well and judgement became the hallmark of thirdgenerahon evaluahon' (Guba & Lincoln 1989) Thisdevelopment m educahon can be discerned in nursmg

In the health care settmg, quality assurance committeesfill the role of the judges Providers of health care typicallycontract with the evaluator(s) or quality assuranceco-ordinator(s) for an evaluahon and together these groupsset the parameters and boundanes for a quality assuranceprogramme

Third-generahon evaluahon mcludes the developmentof nursmg standards/cntena, marketmg approaches andfinancied appraisal Primarily it mvolves the setting of stan-dards against which judgements are made Experts usuallydecide the level standards will take, excellent, achievable,desired and Tnimmnm acceptable standards (Koch 1992)Evaluahon based on these standards is usually developedmto measurable tools Experts, not pahents, decide whatwill be measured

1149

T Koch

Staff participation

Third-generation evaluahon highlights health staff parhci-pahon Redfem & Norman (1990) consider differentapproaches to measunng the quality of nursing care andquestion the value of managenal top-down approaches toquality assurance They conclude that 'bottom-up' parhci-pant approaches are appropnate if the comimtment andinvolvement of all staff is sought It is aigued that a'bottom-up' approach can maintain the integrity of thequality assurance cycle and that parhcipation of staff isthe key determmant of a successful programme Nursesare encouraged when they can see that their evaluationactivities have a positive impact on the ceire they provide,and being mvolved m the quality assurcince process facili-tates an overview of nursing care However, in third-generahon evaluation, no mention is made of lnvolvmgpatients in the quality assurance cycle

In summary, when judgement-orientated evaluation isapplied to nursing, stress is placed on quantifiable stan-dards of practice and the marketmg of the service Healthstaff participahon m quality assurance activities is a cru-cial development withm third-generation evaluation

PATIENT PARTICIPATION

In the case of the first three generahons of evaluahon,patient participation is unlikely Whilst there is movementtowards including another 'stakeholder', namely staff,patients still remam subjects and not parhcipants Theevaluahons of the first three generahons are unable to con-tain value-pluralism or the mulhple realities of personsinvolved m the evaluation Constructions patients haveabout their own situation are ignored Patients areexcluded from making decisions about evaluahons thataffect them

Disregard of pahents' expenences is part of our measure-ment-onentated health care system Morse (1989) com-ments that

administrators seek to control nursmg actions, to lunit canngtime, and to require concrete, measurable outcomes, while nursesheg for time for caring tasks (listening to the patient's concems)that do not have solid, quanbflahle outcomes other than patientsatisfaction

Patients bnng their realihes and concems to the healthcare sethng, but these are largely ignored m practiceDonabedian (1988) comments

if patient satisfaction plays so important a role in connoting, con-tributing to, and judgmg quality, why is it so httle used massessing the quality of care'' Perhaps the most fundamentalreason is that practitioners are mcomgihly biased m favour oftechnical care, havmg been warped hy the social institutions thatbreed them Then hy excluding consumers from the process of

standards settmg, they have insulated themselves firom influencesthat might prompt a broader view

It IS argued that nurses have not paid adequate attenhonto what really matters to patients because evaluations haverelied almost exclusively on 'the scientific method'Quality cntena are developed solely by health pro-fessionals Wagner (1988) wntes

professionals declared what quality services were based on com-bined professional experience, knowledge and what was felt tohe good for the client These declarations became standards tojudge whether or not services were of acceptahle quality Thiswas done because it was felt that the client was generally notcapahle of determming quality

The appeal of Guba & Lincoln's (1989) nohon is that allstakeholders must take part m evaluations

PATIENTS HAVE COMMON CONCERNS

The suggestion that pahents have common concerns recursthroughout several bodies of literature, such as that dealingwith focus groups (Petersen 1988), autobiographical work(Newton 1979), patients as consumers (Eck et al 1988,Allanach 1988, Holbrook & O'Shaughnessy 1988, Spicer1988), phenomenological studies explicating patients'experiences (Drew 1986, Brown 1986, Riemen 1986,Swanson-Kaufrnann 1986, Rempusheski et al 1988, Loos1989) and anecdotal expenences from nurses (Tweedley1985, Lightfoot 1990) When pahents are asked to expresstheir concems m open-style interviews, one most strikingfeature is the emphasis of the importance of beingacknowledged as an individual

Riemen (1986) records that patients perceive a great lackof canng when nurses hurry, belittle pahents, treat pahentsas objects, or do not respond Pahents feel Ccired for whenthe nurse is 'present' to the pahent (Riemen 1986) or, asForester (1980) suggests, when nurses listen to patientsactively Swanson-Kau&nan (1986) identifies several categ-ones of canng expressed by patients as affirmations ofenablmg and 'being with' pahents Drew (1986) sketchesthe nohons of confirmahon and exclusion The formerrefers to patients bemg acknowledged, whereas the latterdwells on the depersonalizahon of pahents SimilarlySpitzer (1988) claims that patients feel dehumamzed anddistanced from nurses when nurses appear unmvolvedand focus, for example, on technology

Older patients

In a recent study Koch (1993) aimed to capture the con-cems of older pahents who were admitted to the acutecare sector Listening to the voices of older pahents wasconsidered to be the first stage of fourth-generahon evsdu-ation The study took place m two W£irds for care of elderly

1150

Fourth-generation evaluation in nursing

people m a 1000-bed Nahonal Health Service (NHS) hospi-tal m the Umted Kmgdom Fourteen pahents were mter-viewed, each on several occ£isions, and their concemswere expressed in themes, namely routine geriatnc styleof care (from Baker 1978), depersonalizahon, care depn-vahon and genatnc segregahon Older pahents expen-enced bemg treated as less than responsible adults Otherconcems related to feelmgs of powerlessness and lack ofcare expenenced by pahents dunng the inflexible routm-lzed care procedures, segregahon and subsequent depn-vahon and discnmmahon based on age

However, common fears expressed by pahents of all agegroups indicate their feelings of powerlessness, the deper-sonalizmg experiences of health care, lack of mformahonand lack of consultahon (Kappeh 1984, Riemen 1986,Drew 1986, Rempusheski et al 1988, Spitzer 1988, Loos1989, Koch 1993)

The above hterature is not often a resource for qualityassurance achvihes or for evaluating care, preciselybecause the authors, whilst evaluatmg care, refrain frommeasunng care Yet these studies provide a nch source ofmatenal to be considered for improving care They alertnurses to pahents' concerns that are not captured by evalu-ahons from the previous generahons

FOURTH-GENERATION EVALUATION.BEYOND MEASUREMENT

Guba & Lincoln (1989) set the scene for participation inthe evaluahon process Parhcipants refers to all stake-holders Fourth-generation evaluation moves beyondmeasurement and

just getting the facts — to include the myriad human, pohtical,social, cultural and contextual elements that are involved

(Guba & Lincohi 1989)

Guba & Lmcoln (1989) devote a large proporhon of theirtext Fourth Generation Evaluation to defending their con-stmchvist paradigm emd comparing it with the seienhfiemethod and its preoccupahon with measurement Ratherthan repeat their debate the present paper adapts themethodological considerahons from educahon to health,outlines the process of this type of evaluation and makessuggeshons for its implementation

Guba & Lincoln's constmchvist paradigm aims todevelop judgemental consensus among stakeholders Theontological basis of the constmchvist paradigm demes theexistence of a smgle objechve reality Evaluahon outcomesare not bald descnphons of the 'way things really are 'but instead represent meaningful construchons that par-hcipants form to make 'sense' of the situahons m whichthey find themselves (Guba & Lincoln 1989) The outcomesare not 'facts' but are construchons created through anmterachve process that mcludes the evaluator as well asthe many stakeholders

Epistemologically, this paradigm demes subject-objectdualism In the creahon of construchons the process relieson the mterachon between evaluator and stakeholdersMethodologically, this type of evaluahon rejects the con-trollmg approach of the scientific method, and subshtutesfor this a hermeneuhc dialechcal process

Hermeneutical dialectic process

The main task of the hermeneutic dialechc process is totease out the construchons that parhcipants hold Theidea of hermeneuhcs accepts that parhcipants are self-mterprehng and bnng their construchons (stones) to thenegohahng table This process is dialechc as it contrastsand compares divergent views held by parhcipants In theprocess of mterpretahon, the reference pomts are theclaims, concems and issues that are the product of of self-interpretations The evaluator's aim is to produce a con-stmchon that provides a sjoithesis of them all

Another aspect of this approach is responsive focusmg,which means an emphasis on the importance of per-sonalizing the evaluahon process

Being responsive requires having face to face contact with peoplein the program and learning first hand about the stakeholders'concems

(Patton 1990)

This type of evaluation is descnbed as 'transachonal' byPatton (1990)

Claims, concems and issues

Evaluation is responsive to all those 'stakeholders' whohave a right to place their claims, concems and issues onthe negohatmg table for response

A claim IS any asserhon that a stakeholder may makethat IS favourable, for example, the claim that a parhcularway of teachmg the process of aseptic techmque to studentnurses reduces the number of wound infections m oneward

A concem is any asserhon that is unfavourable, forexample, patients may be concemed about tbe inflexibilityof the early 'hygiene' morning routme m the wards,whereas nurses are concemed that ward organizahon willbe disrupted if night staff do not carry out the morning'hygiene' procedures before the night shift terminates

An issue is a state of affairs about 'which reasonablepersons may disagree', for example, the numbers and skillof nursing staff, or the mtroduchon of non-nurse mane^ersm the wards The pomt is that divergent views will beheld by stakeholders and it is the role of the evaluator totease these out

1151

T Koch

Ckillaboration and participation

The appeal of fourth-generahon evaluation is that it hasan action onentation 'that defines the course to be fol-lowed', stimulates people mvolved to follow it, and gen-erates and 'preserves a commitment to do so' (Guba &Lmcoln 1989) There are two aspects to be consideredcoUaborahon and parhcipation

Action research and fourth-generahon evaluahon aresmular achvihes as both have parhcipatory and collabor-ative aspects Reinharz (1992) descnbes the change-orientated focus of action research as 'research m whichachon and evaluahon proceed simultaneously ' Reinharz(1992) IS refemng to femmist research but her observahonsare pertment to fourth-generahon evaluahon

To achieve an egalitarian relation, the researcher abandons controland adopt8 an approach of openness, reciprocity, mutual disclos-ure and shared nsk

Accordmg to Reinharz, achon research can emancipateparhcipants when approaches such as openness, self-disclosure, group mtemews and negohahon of mterpret-ahons are adopted The divide separatmg 'researched' and'researcher' dissipates

This type of evaluahon is co-operahve, that is to say, itIS 'with' people rather than 'on' people In addihon, m thisapproach stakeholders are welcomed eis partners m everyaspect of design and implementahon, mterpretahon andthe resulhng achon of an evaluation

Role of the evaluator

The evaluator facilitates the negohation, maintains a jour-nal, interviews mdividual stakeholders, leads the groupsand documents the evaluahon process The craft of evalu-ahon lies in the produchon of an acceptable agenda fornegohahon, mcorporahng diverse construchons, and pre-paring stakeholders in the negotiahon process The processIS educahve for all stakeholders in the construction andreconstruchon of their realities Further research isrequired on the advantages and disadvantages of usmg eva-luators mtemal or extemal to the settmg

The evaluator needs to be sensihve to the different per-spechves of vanous stakeholders and recogmze thatchange cannot be engineered mto the construchons ofthe parhcipants because 'it is a non-lmear process thatmvolves the mtroduchon of new mformahon andmcreased sophishcahon m its use' (Guba & Lincoln 1989)

The evaluator must be able to accept that this tjrpeof evaluation does not produce reassunng answersAmbiguity and doubt are pnmary features of the processbecause it takes its direction fi:om the parhcipants Controland predichon are set aside In addihon, the nohon ofconsensus requires further explorahon One of the mostdifficult aspects for the evaluator is to realize that these

emancipatory ideals can fail There is nothing m the nohonof emancipahon that guarantees success

Daily journal

The evaluators mamtam a daily joumal m order to exam-me mterachons with stakeholders and reflect upon theirown mohvahons and insights Mamtammg a joumal isan important part of most qu«ditahve research processesWays m which people make sense of their situahons areshaped by their values (Gadamer 1976) and this alsorefers to the achons of the evaluator who generates andmevitably participates m makmg these data (Drew 1989,Koch 1994) The evaluator's own values are incorporatedm the research process £ind these are documented m thejoumal Their analysis forms an important part of theresearch

Joumal data also offer an approach to thinkmg aboutand mteractmg with the data, and to understandmg thedecisions made m the process of parhcipatmg m the evalu-ahon The inclusion of joumal data m the final reportallows others to audit the evaluahon process and thisassists m the development of the study's ngour A decisiontrail can assist with the establishment of the trustworthi-ness of the evaluahon (Koch 1994)

THE PROCESS OF FOURTH-GENERATIONEVALUATION

Adaphng a process of evaluahon fi'om educahon to thehealth care setting entails considenng the differencesbetween sethngs, participants and reasons for the evalu-ahon Classrooms are clearly different to hospital wards orclinics, students emd pahents as parhcipants will havedivergent interests

Fourth generahon evaluahon is an achvity takmg in cul-tural, pohhcal, social and economic factors, so it wouldbe prudent for the evaluator to have an onentahon andobservational penod m the setting This mvolves ident-ifying local political factors and making logistical arrange-ments It mcludes assessmg the trade-offs and sanchonsoperatmg m the setting Guba & Lincoln refer to gamingaccess as contracting and organizmg the condihons neces-sary for an evaluahon process to take place

There are four phases of responsive focusmg

1 Idenhfymg stakeholders, commencmg open-endedinterviewing and soliciting claims, concerns and issuesfrom each group

2 Introducing these items to other groups for comment,refutahon or agreement It is expected that many itemswill be resolved at this stage

3 Explonng unresolved items The evaluator continuesmformahon gathermg m order to enlai^e construchonsand allow parhcipants to re-negohate

1152

Fourth-generation evaluation m nursing

4 Re-negohatmg among stakeholdmg groups guided bythe evaluator

Thus the product of fourth generation evaluation is nota set of conclusions, value judgements or reconunen-dahons, but rather an agenda for negotiation of thoseclaims, concerns and issues that have not heen resolved

ldentificahon of stakeholders mcludes assessing thosepersons who could potenhally be disenfranchised as aresult of the evaluahon Each stakeholder should be mter-viewed separately This aspect of the research processmeans working closely with parhcipants and entails recyc-lmg for validahon with them the descnphons of theclaims, concerns and issues ldenhfied Each stakeholderhas the nght to remove information which he/she feelsmay be attnbutable specifically to them Patients may feelvulnerable and fear exposure by the eveduation process(Koch 1993), and addihonal hme may be required toachieve trust and to build rapport

Ethical problems

In anticipahng ethical problems, the first nsk is associatedwith the relationships between stakeholders formeddunng the evaluation process Face to face contact pro-vides for intensive and fragile relationships, which aresubject to 'violahon of trust, to shading the truth, to misun-derstandings regarding the purposes or relationships'(Guba & Lincoln 1989) Second, confidentiality and pnv-acy are difficult to mamtam, given the emphasis on de-bate of mulhple constructions between stakeholdersStakeholders come to know each others' constructions andpositions on vanous issues This requires sensitive man-agement Stakeholders should be warned at tbe outset ofthe problems that are likely to anse

Pahents/clients should be part of the organization forlong enough to allow their parhcipahon It is envisagedthat a long stay or a rehabilitahon ward may be the mostappropnate m the hospital and, in the community, nursingservices such as district nursing The total number of par-hcipants should not exceed 50 if the evaluahon process isto remain manageable

It would be valuable if tbe wards/community settingsselected had m place a quality assurance programme sothat It would be possible to build upon previous gener-ations of evaluahon It would also be necessary for all staff,mcludmg medical and administrahve staff, to be consultedand to agree to participate m the evaluahon study

lmperahve for its progress would be structures to ensureparticipation of all stakeholders Most importantly the sup-port and co-operation of managers and administratorswould be needed Implicit m participatmg m this researchIS the willingness, commitment and support, from theDirector of Nursmg, managers and adnunistrahve person-nel, to facilitate change or corrective achon m the health

umt A work agreement would need to be developed andsigned by the evaluator and personnel The work agree-ment would specify procedures that protect staff andpatients' nghts Condition agreements for evaluationwould need to be formalized

The process

The process should move from working with mdividualstakeholders to working with groups such as nurses, dom-estic workers, doctors, and residents/patients/chentsWorking from an agenda prepared by the evaluator, itemswould be pnonhzed and discussion would take placearound common items denved from the individual partici-pant interviews Credibihty checi^ v«th stakeholders arepart of this process Members of the group must he able toconfront others, take account of other constructions offeredand be prepared to reconstruct their own agenda

Each group should define and elucidate unresolveditems and competing construchons If appropnate, eachgroup would select a representative to take the agenda ofresolved and unresolved items to the next phase, a her-meneutic circle enclosing group representahves The aimIS to form a connechon between the construchons frtimparticipants to allow mutual exploration by all parhes

The course of action is decided hy the participants

Stakeholder groups form a representative circle and shapejoint construchons of common and unresolved concemsof all parhes Their report produces constmchons provid-ing a synthesis of divergent views These construchons arebrought to the negotiatmg table to seek their resoluhonthrough consensus If consensus cannot he reached thenat least this process exposes and clarifies different pointsof view Hermeneutics mvites its parhcipants to an ongo-mg conversation Thus meetings are held regularly todiscuss compehng constructions and to shape jomt con-structions The course of achon is decided by theparticipants

CONDITIONS FOR A SUCCESSFULHERMENEUTIC PROCESS

Integrity

The first condihon for a successful hermeneuhc process isa commitment from all parhes to work with mtegnty Thisprocess relies on openness of exchange between stake-holders If lies are told or misleading constructions aregiven the process is at risk Sabotage of the process is nodoubt the greatest nsk Whilst the key to the success ofthis process is negohation between all parhcipants,undemimmg of such negohahon would signal its failureThe process relies on non-mampulahve trust between

1153

T Koch

stakeholders (mcluding the evaluator) The evaluahonrelies on the good will of all parhcipants in their desire tochange and improve their situahon

Communication

The second condihon for success is that all parhcipantsmust be able to commumcate effechvely Thus very youngchildren and severely mentally handicapped personswould be excluded Given the exclusionary nature of pre-vious generahons of evaluation it may be prudent to workwith pahents first The nohon of collaborahon may takehme to be fully appreciated, and the parhcipatory expen-ence may he novel to pahents/clients Working withpatients towards enhancement of their negohatmg skillsIS one of the first pnonhes of the research process but,rather than workmg solely with pahents, evaluators couldwork with groups simultaneously This would assist themutual educahon of pahents, staff and managers

Power sharing

The third condihon is for parhcipants to share power Oneof the difficulhes envisaged is the mequahty m currenthealth systems All research is polihcal, and the powerrelahons between the groups may restnct co-operativefunchomng The goal of this approach is to educate andengage stakeholders (including staff and managers) m anegohahon process which takes m the sociopohtical con-text of the sethng It is recogmzed that evaluahon canenfranchise or disenfranchise groups, and that an msh-tuhon could seek to mamtam the status quo It should berecognized that the negohation process could be subvertedand used to the detnment of patients and staff

Reconsideration of values

The fourth condihon is a wiUmgness of parhcipants toreconsider their values Evaluahon produces data inwhich

facts and values are inextricably linked Valuing is an essentialpart of the evaluation process, providing a basis for an attri-buted meaning

(Guba & Lmcoln 1989)

One of the most difficult aspects of the process is to recon-sider one's values The role of the evaluator is crucial mprovidmg the best possible merging of the construchonsgiven by alternative groups Based on a well-informed con-struchon, reconsiderahon of values may be possible

Willingness to change

The fifth condihon is a willingness on the part of all par-hcipants to change if they find the negohahons persuasive

Cktmmitment of time and energy

s of all parhcipants to make commitments oftune and energy is the sixth condition In a recent study(Koch 1993), which was a preliminary attempt at fourth-generahon evaluahon, shortage of time was the biggestproblem for nurses working m the wards The tune takenby sequenhal mterviewmg is tune away from the pahent'sbedside Involving all parhcipants m the evaluahon meansasking them to be committed to giving their time for theevaluahon process to work This appeared to be one of themost difficult problems to overcome Pahents, on the otherhand, found the sessions to be therapeuhc once lnihaldoubts were dispelled and, for them, time was not a scarcecommodity

CONCLUSION

If this process is to succeed, it relies above all on goodwill and trust between stakeholders In spite of the prob-lems inherent m power shanng, it is proposed that thenegotiation-onentation of fourth-generahon evaluahon hasmuch to offer all stakeholders, particularly clients/pahents, m the health care system It provides a proachveevaluahon process to shape the health services of thefuture

References

AUanach E (1988) Patients' expectations and values clarificationa service audit for the department Nursmg AdministrationQuarterly 12[3), 17-22

Baker D (1978) Attitudes of nurses to the care of the elderlyUnpublished PhD thesis University of Manchester,Manchester

Brown L (1986) The expenence of care patients' jmrspectivesTopics m Climcal Nursmg 8(2), 52-62

Diekelmann N , Allen D & Tanner C (1989) TTie MJV Cntena forAppraisal of Baccalaureate Programs A Cntical AnalysisNational League for Nursing, New York

Donabedian A (1988) Quality assessment and assurance unity ofpurpose, diversity of means Inquiry 25, 173-192

Drew N (1986) Exclusion and confirmation a phenomenology ofpatient's expenences with caregivers IMAGE Joumal ofNursmg Scholarship 18(2), 39-43

Drew N (1989) The mterviewer's expenence as data in phenom-enological research Westem Joumal of Nursing Research 11,431-439

Eck S , Meehan R., Zigmund D & Pierro L (1988) Consumensm,nursing, and the reality of the resources NursingAdministration Quarterly 12(3), 1-11

Elbeck E (1987) An approach to client satisfaction measurementas an attnbute of health service quality Health CareManagement Review 12(3), 47-52

Forester J (1980) Listening the social policy of everyday life (cnti-cal theory and hermeneutics m practice) Social Praxis 7,219-232

1154

Fourth-generation evaluahon m nursing

Gadamer HG (1976) Philosophical Hermeneutics (Linge Dtransl, ed), Umversity of Cahfonua Prras

Green L. & Lewis F (1986) Measurement and Evcduation in HealthEducation and Health Promotion Mayfield, Palo Alto,California

Guba E & Lmcoln Y (1989) Fourth Ceneration Evaluation Sage,Newbury Park, California

Holbrook M & O'Shaughnessy J (1988) On the scientific statusof consumer research and the need for an mterpretive approachto studying consumption behaviour Journal of ConsumerResearch 15, 398-402

Kanar R. (1988) The mfluence of a quality assurance program onpatient satisfaction Joumal of Nursing Quality Assurance2(3), 36-43

Kappeh S (1984) Towards a practice theory of the relationshipsof self-care needs, nursing needs and nursmg care m hospital-ised elderly Unpublished PhD thesis University Manchester,Manchester

Koch T (1992) Nursmg quality assurance a review Joumal ofAdvanced Nursing 17, 785-794

Koch T (1993) Toward fourth generation evaluation Listenmg tothe voices of older patients A hermeneutic mquiryUnpubhshed PhD thesis Department of Nursing, University ofManchester, Manchester

Koch T (1994) Establishing ngour m qualitative research thedecision trail Joumal of Advanced Nursing 19, 976-986

Lightfoot S (1990) Maisie's story doctors dismissed her symp-toms as 'just her age' Commumty Outlook Nursing Times(Suppl) June, 35-36

Loos C (1989) The client's view of hospiteilisation dunng preg-nancy Clinical studies Joumal of Obstetnc, Cynecologic andNeonatal Nursmg 18(2), 52-56

Morse J (1989) Qualitative Nursing Research A ContemporaryDialogue Aspen Publication, Rockville, Maryland

Newton E (1979) This Bed My Centre McPhee Gnbble,Melbourne

Patton M (1990) Qualitative Evaluation and Research Methods,2nd edn Sage, London

Petersen M (1988) Measunng patient satisfaction collectinguseful data. Joumal ATursing (^mlity Assurance 2(3), 25-35

Pfeffer N & Coote A (1991) Is Quahty Good for You^ InsUtute forPublic Pohcy Research Social Pohcy Paper Number 5, London

Redfem S & Norman I (1990) Measurmg the quahty of nursingcare a consideration of different approaches Joumal ofAdvanced Nursing 15, 1260-1271

Reinharz S (1992) Feminist Methods in Social Research OxfordUniversity Press, New York

Rempusheski V, Chamberlam S, Picard H, Ruzanski J &Collier M (1988) Expected and received care patient percep-tions as communicated m letters Nursing AdminisbntionQuarterly 12(3), 42-50

Riemen D (1986) Noncarmg and canng m the clinical settmgpatients' descnptions Topics m Clmical Nursing 8(2),30-36

Scnven M (1972) Pros and cons atK>ut goal free evaluation com-ment The Joumal of Educational Evaluation 3[4), 1-7

Spicer J (1988) A systems approach to customer satisfactionNursmg Administration Quarterly 12(3), 79-87

Spitzer R (1988) Meeting consumer expectations NursingAdministration Quarterly 12(3), 31-39

Stufflebeam D (1983) The dPP model for program evaluation InEvaluation Models Viewpoints on Educational and HumanServices Evaluation (Madaus G, Scnven M & StuSlebecun Deds), Kluwer Nijhoff, The Hague, pp 117-142

Swanson-Kaufinan K (1986) Caring m the instance of unexpectedearly pregnancy loss Topics in Clmical Nursing 8(2), 37-46

TweedleyJ (1985) Patient's page a lack of sensitivity poor qual-ity of nursing care given to her dying mother Nursing Mirror161(2), 45

Tyler R (1949) Basic Pnnciples of Cumculum and InstructionHarcourt, Brace and World, New York

Wagner D (1988) Who defines quality consumers or professions'Canng 7(10), 271-282

Yura H & Walsh M (1973) The Nursing Process Assessment,Planrung, Implementing and Evaluating, 2nd edn Meredith,New York

1155