‘having a say’: negotiation in fourth- generation evaluation

`Having a say': negotiation in fourth-generation evaluation

Tina Koch RGN BA PhD

RDNS Chair in Domiciliary Nursing, School of Nursing, Flinders University of

South Australia, Adelaide, Australia

Accepted for publication 13 April 1999

KOCHKOCH TT. (2000)(2000) Journal of Advanced Nursing 31(1), 117±125

`Having a say': negotiation in fourth-generation evaluation

In this paper, research guided by the principles of fourth-generation evaluation,

which has negotiation as its centre, will be scrutinized. The role of stakeholders

will be discussed with reference to negotiation issues. Throughout this

discussion, observations will be based on the actual process of conducting three

project evaluations in this tradition. Attention will be drawn to the conse-

quences of negotiating at an uneven table.

Keywords: fourth-generation evaluation, negotiation, stakeholders, voice,

nursing home residents, care of the elderly, power, nursing career structure,

industrial relations

INTRODUCTION

Several years ago I utilized a methodology termed fourth-

generation evaluation (FGE), adapted from the work of

Guba and Lincoln (1989), in care of the elderly wards in an

acute care setting in the United Kingdom (Koch 1993,

1994). A few years later I used the methodology in an aged

care setting, this time in a nursing home in Australia

(Mitchell & Koch 1997). In 1997, I was the chief evaluator

for another study utilizing the principles of FGE, entitled:

A new negotiated career structure for level three nurses:

RDNS pilot project study (Koch et al. 1998). Since then I

have accrued a range of experiences using or supervizing

higher degree candidates in the process of FGE. In this

paper I will raise some questions about the process, with a

particular emphasis upon negotiation. Ideally, FGE as a

methodology enables inclusion, involvement, co-opera-

tion and negotiation in an environment that aims to reach

all stakeholders (Swenson 1991). Missing from Guba and

Lincoln's evaluation model are documented case studies

(Fishman 1992). This paper provides some cases for

consideration.

Guba and Lincoln (1989) critique the positivist,

conservative foundation of traditional programme evalu-

ation. Three historical generations of evaluation are

critiqued, and the authors replace traditional programme

evaluation with fourth generation evaluation that is

located within the constructivist paradigm (belief

system). In the effort to guide the novice through the

constructivist belief system they developed a table (Guba

& Lincoln 1989) which contrasts conventional and

constructivist belief systems. In this table they sequence

ontology, epistemology and methodology. The

constructivist paradigm has a relativist ontology that

assumes that stakeholders invent reality. In a world of

multiple realities, each stakeholder is able to present his/

her construction of reality. In its epistemology, the

stakeholders create knowledge, and methodologically,

there is a continuing dialectic.

Although Guba and Lincoln (1989) give several interac-

tive steps guiding the implementation of FGE, it is

worthwhile exploring the literature for further assistance.

The literature around negotiation is from the 1970s, when

sociologists were debating whether a general theory of

negotiation was possible. Most of the literature around

that time concerned negotiation in business, industrial

and labour relations, and international relations. Strauss

produced an authoritative text entitled Negotiations in

which he (Strauss 1978 p. 99) describes the properties of

negotiation as:

· the number of negotiators, their relative experience in

negotiation and who they represent;Correspondence: Tina Koch, RDNS Business Centre, PO Box 247, Glenside

2056, South Australia. E-mail: tina.koch@¯inders.edu.au

Ó 2000 Blackwell Science Ltd 117

Journal of Advanced Nursing, 2000, 31(1), 117±125 Health and nursing policy issues

· whether the negotiations are one shot, repeated,

sequential, serial, multiple or linked;

· the relative balance of power exhibited by the respec-

tive parties in the negotiation itself;

· the nature of their respective stakes in the negotiation;

· the visibility of the transactions to others, that is their

overt or covert characters;

· the number and complexity of the issues negotiated;

· the clarity of legitimacy boundaries of the issues

negated; and

· the options to avoiding or discontinuing negotiation,

that is, the alternative modes of action perceived as

available.

Strauss gives a case study of an experimental psychi-

atric ward, and the above properties are used to frame the

analysis. I suggest that this framework may be usefully

combined with the FGE process, particularly if applied in

the health care delivery context.

The methodology of FGE has bypassed nurse

researchers, with the exception of Swenson (1991), who

argues for its congruence with a nursing paradigm. Rebien

(1996) argues for its application in development assis-

tance, in particular the American Aid Agency (USAID),

because it:

¼ includes the contextual facts of the project, allows a picture of

reality to be constructed by stakeholders, and rede®nes the role of

the evaluator as that of a facilitator assisting stakeholders in

constructing this picture. (p. 151)

THE PRINCIPLES OF FOURTH-GENERATIONEVALUATION

I will outline the principles of FGE as given by Guba

and Lincoln (1989). Elsewhere I have given a descrip-

tion of the three generations of evaluation as they can

apply to nursing (Koch 1994). However, for the purpose

of this paper, ®rst generation evaluation relies on

measurement, the second generation is driven by

measurable objectives, and the third generation is based

on the judgement of the evaluators. FGE, as the fourth

type of evaluation, is contingent upon negotiation.

The ®rst three generations may be categorized

as `measurement-oriented', òbjective-orientated', `judge-

ment-oriented', respectively, whilst FGE is seen as

`negotiation-oriented'. In other words, FGE leaves behind

evaluation based on measurement, evaluation driven by

objectives and evaluation programmes that are profession-

ally driven. The appeal of a `negotiation' process is that

the evaluation strives to give its stakeholders a voice or the

opportunity `to have a say' about things that affect them.

Stakeholders are de®ned as those who have a direct

involvement with the group or setting that is to be

evaluated.

Concerns, claims and issues

This type of evaluation argues for all stakeholders to have

a right to place their concerns, claims and issues (CC & Is)

on the negotiating table (Laughlin & Broadbent 1996).

Claims are de®ned as `favourable assertions', concerns are

ùnfavourable assertions' and issues relate to situations

where `reasonable persons may disagree' (Koch 1994).

Responsive focusing and hermeneutic dialectic

One important aspect of this approach is responsive

focusing. Interviewees are asked to tell the evaluator their

construction of concerns, claims and issues as they apply

to the situation being evaluated. These focus or reference

points are the CC & Is. Stories told to the evaluator are

taken to represent meaningful constructions that stake-

holders form to make `sense' of the situations in which

they ®nd themselves. This type of evaluation rejects the

controlling approach of the conventional research through

the use of the scienti®c method, and substitutes for this a

hermeneutic dialectical process. The idea of `hermeneutic'

accepts that participants in the evaluation are self-

interpreting and bring their constructions to the

negotiating table. Negotiation must be carried out under

circumstances where all parties are in possession of the

same level of information. This may mean that

stakeholders require assistance in understanding what

information means for their interests. Constructions are

created through an interactive process that includes the

evaluator as well as the many stakeholders. Teasing out

the constructions that participants hold is the main task of

the hermeneutic dialectic process.

Social action

FGE promotes an activist ideology. When common CC & Is

are agreed upon and negotiated, there is a moral imperative

that this form of evaluation will lead to action. Common CC

& Is can direct negotiation in the process of action

planning, action taking and back into iterative cycles of

evaluation. In this way, fourth-generation evaluation

represents the merging of an interpretist philosophy with

an òutside warrant' for social action. The aim of an activist

ideology is that action is taken by the participants to

advance or improve the situation at hand. Outcomes of the

evaluation processs are considered successful when

reform and/or emancipatory ideals are realized.

The negotiation process

In the staging of negotiations, stakeholders must ®rst have

reason to believe that they are dealing with the appropriate

party, that a desired outcome can be reached and that the

resulting agreements will be honoured (Levy 1982). The

T. Koch

118 Ó 2000 Blackwell Science Ltd, Journal of Advanced Nursing, 31(1), 117±125

negotiation process begins with an open-ended interview of

one of the stakeholders. This is followed by prolonged and

persistent engagement with stakeholders. As information is

collected, it is analysed immediately, and fed back for

comment, correction and revision. The process is `a contin-

uous, dialectic chain of questioning, answering, analysing,

and requisitioning, ultimately leading to the emergence of a

common construct of reality' (Rebien 1996 p. 155). As

suggested by Rebien, social change is seen to take place

when stakeholders themselves create their own solutions

based on their understanding of the problem.

The evaluation process

In the three case studies provided, the evaluator was guided

by the work of Guba and Lincoln (1989). In the section to

follow, I describe the way in which preparation for nego-

tiation was organized for the three case studies. Fourth-

generation evaluation is context bound and its ®ndings can

only be used in relation to the speci®c context in which they

were produced. An observational period in each of the

settings is a prerequisite to understanding the situation in

which the participants of the evaluation ®nd themselves.

Once the context is described (and this becomes part of the

evaluation report) and stakeholders are identi®ed, inter-

views can commence. Individual interviews are conducted

with each stakeholder in a private place. Stakeholders are

told that individual concerns will not be raised, nor

identi®ed as a result of the interview. Rather, the evaluator

will look for common CC & Is. The interviews are tran-

scribed and analysis is made for common CC & Is. These

common CC & Is are the basis for the negotiation agenda.

Before a stakeholder meeting can take place, it is important

to prepare stakeholders for the negotiation process. This

may require educational resources and/or role playing.

Eventually, the group is brought together for discussion and

negotiation round the `table'.

The evaluator facilitates the discussion and asks stake-

holders to prioritize the agenda items. Through dialogue

and interaction, the group will learn to understand each

other better. If possible, a few stakeholders will assume

responsibility for acting upon the negotiated agenda items.

Participants are led through a process of negotiation that is

a dialectic as it contrasts and compares divergent

constructions held by participants. If necessary:

the evaluator¼ should be prepared to act as the lesser group's

advocate, or, particularly in the case of important issues, the

lesser group may be aided by special advocates. (Guba & Lincoln

1989 p. 219)

Ideally, responsive evaluation seeks to reach a

consensus on all CC & Is, but that is rarely possible, and

instead the evaluator assumes that there will be competing

constructions. Not only do evaluators need skills to deal

with con¯ict and confrontation, but mutually educative

strategies for all stakeholders should be developed as part

of the preparation for the negotiation process. The

evaluator's aim is to produce an evaluation product that

provides a synthesis of all CC & Is. However, if the

participants cannot agree upon common CC & Is, or cannot

formulate an agenda for negotiation or even if a synthesis

is unrealizable, at the very least there will be an increased

understanding within the organization by highlighting

issues which stakeholders perceive as important.

Stakeholder meetings continue until most common

CC & Is are dealt with. The evaluator documents the

evaluation process, manages the group dynamics, ensures

that everyone has an opportunity to `have a say', is

mindful of power relations and predicts where potential

unequal relations will arise.

Rigour and authenticity criteria

Authenticity criteria refer to the extent to which different

constructions and their underlying value structures are

solicited and honoured within the evaluation process

(Guba & Lincoln 1989 p. 246). The rigour or trustworthi-

ness of the process can be evaluated in four ways: by

creating an appeal process should any negotiating party

feel that rules are not observed; through maintaining

accessible and audible records of the negotiating

procedures; being able to follow or audit documented

individual constructions; and making the negotiating

process open and transparent (Fishman 1992).

Role of the evaluator

The evaluator attempts to shift the control of the evaluation

to thestakeholders. Rather, the roleof theevaluatorbecomes

one of facilitating interpretive dialogue among stakehold-

ers. The objective of the dialogue is to obtain consensus

among stakeholders regarding an emergent construction of

the programme's effects and bene®ts. Re¯ecting on the role

of the evaluator, Papineau and Kiely (1996 p. 80), argue for

adaptation of evaluation practice to include considering

the social ecology of the evaluation site:

¼ the concerns and problems that stakeholders want to address,

as well as the relational and political context of the organization.

The evaluator assumes the role of a catalyst in bringing

about reform or change.

Power relations

Although the inherent political nature of evaluation is

acknowledged, Fishman (1992 p. 267) aptly points out:

The determination of whose view is more relevant to decision

making and practical action becomes a matter of previously

established political structures and ongoing political negotiation.

Health and nursing policy issues Fourth-generation evaluation

Ó 2000 Blackwell Science Ltd, Journal of Advanced Nursing, 31(1), 117±125 119

THREE CASE STUDIES USINGFOURTH-GENERATION EVALUATION

Project one

In the ®rst application (Koch 1993 p. 1994) the focus of the

study was the location of patients' voices as a preliminary

step towards FGE. The aim of the study was to express the

concerns of older patients who were admitted to the acute

care sector. The study took place in two care of the elderly

wards in a 1000-bed National Health Service (NHS) acute

care hospital in the United Kingdom (UK). In this study I

was concerned with listening to the way in which older

patients had constructed the realities of their health care

experiences. A methodology was needed that could open

the door to dialogue and give patients the autonomy to

speak openly about their experiences and concerns, hence

FGE as the choice to frame the inquiry.

Participant observation, conversationsand interviewsBefore I commenced interviews with the key stakeholders,

the older patients themselves, I felt it was important to be

in the role of a participant observer. So a great deal of time

was devoted to giving direct patient care in the care of the

elderly wards. In this way two months had passed and I

had not succeeded in recording one interview. Neverthe-

less, I had begun to follow through a few of the patients by

having daily conversations with them. I asked them, `What

is it like being in this ward?' or `Can you describe your day

in hospital?' Their ®rst comments were that `nurses are

wonderful' or that `nurses have very hard jobs' and they

were usually followed by the word `but¼ '.

I perceived these comments to be standard super®cial

statements, because in subsequent conversations, other

constructions appeared. There were stories about the

inadequacies of care but these were conveyed cautiously.

Patients were reluctant to raise their concernsIt became apparent in talking with patients that they were

reluctant to speak out about exposing their negative

experiences of nursing practice. I had conversations with

about 50 patients, and some of these conversations led to

interviews. By talking with everyone in the wards I made

an attempt to protect patients, and the 14 patients I was

actually following by interview were not clearly

distinguished from the others. Even so, patients needed

continual reassurance that their con®dentiality would not

be breached. Nevertheless, they stood the chance of being

victimized because their position was compromised by

lack of privacy (a private interview room was not avail-

able). Patients feared exposure and victimization by

nursing staff. Their involvement in interviews had the

potential to make things worse (dis-empowered) rather

than better (empowered).

Common concernsI followed a few discharged patients at home for more than

half a year and their stories were forthcoming and more

relaxed. Eventually, 14 patients were interviewed, each on

several occasions, and their concerns were expressed in

themes, namely: routine geriatric style of care, deperson-

alization, care deprivation and geriatric segregation. In the

7 months, seven patients had died. It was clear that those

remaining were unlikely candidates for taking their

concerns into the wider circle for negotiation.

Having `a say' was not importantDespite intensive preparation, this vulnerable group

would not confront others in a hermeneutic dialectic,

nor did they wish to have a voice in negotiation in this

health care setting. Even if I were to progress into the full

evaluation phase involving all stakeholders, I found that I

could not assume that my emancipatory ideal, that is,

`having a say', was important to these patients. In a

hierarchy of needs, `having a say' was not important, but

being acknowledged as a person was.

Project two

The aim of the second project (Mitchell & Koch 1997) was

to trial this methodology in an elder care setting and to see

if stakeholder groups, including older people, could be

brought together. It seemed important to complete the

process started in the ®rst project, and to see whether a

longer period of time and facilitative learning would enable

older peoples' participation. It seemed important to try to

involve this group, in particular, as there exists a large

literature to support the contention that residents in

nursing homes are particularly disenfranchised and power-

less (Reed & Payton 1997). We believed that the concerns of

older people are generally ignored and thus it seemed

important to facilitate an evaluation process that has the

potential to empower those individuals in particular.

Involving stakeholdersAs researchers, the appeal of FGE was that the process has

the potential to empower residents by enabling them to

drive the agenda and to contribute to the development of

the nursing home. The setting for the second project was a

32-bed nursing home in Australia. In this study,

stakeholders were the residents, signi®cant others, chief

executive of®cer, managers, evaluators, medical, nursing

(trained and untrained), domestic and other staff. The

project objectives were, ®rst, to identify the concerns of

nursing home residents using the case history approach,

second, to identify common concerns, issues and claims

held by staff in the nursing home, and, third, to identify the

structures and resources (human and material) required to

facilitate participation and negotiation of quality issues.

As for the previous study, we engaged in participant

T. Koch


observation before progressing with in-depth interviews of

all stakeholders. Whilst all nursing home staff were

engaged in the interview and negotiation process, only

seven residents (22%) were able to articulate their

concerns. The other 78% had varying degrees of dementia.

Suitability of the resident groupKnowing that nursing homes have high proportions of

residents with dementia, and knowing that these residents

were not well suited to an evaluation that relies upon

negotiation among its constituents, we were still deter-

mined to trial this approach. A strategy used to broaden the

study beyond the 22% able to participate was to recognize,

and include in the interview process, the signi®cant others

of residents, many of them spouses, daughters and sons

who visited regularly, some on a daily basis. We believed

that the inclusion of signi®cant others in this process was

appropriate. We were aware that this assumed that families

and friends could represent adequately the needs of the

resident. However, the signi®cant others selected for

interview were those who spent most of the days with

the residents in the nursing home, and had a reasonably

good understanding of the concerns as they affected the

residents. We chose to take their accounts on board as

legitimate constructions of the running of the nursing

home, and indeed they were able to share rich contextual

data. These data fed into the common pool of concerns.

Residents did not want to have `a say'in group negotiationsWhereas the interviewing of staff (n� 40) took place in 2

weeks, multiple interviews with the few residents and

signi®cant others took place over 1 year. As with the ®rst

project, time was required to build rapport with residents

before they were comfortable about sharing information.

One of the questions asked by the interviewer was `What

is it like here for you?' or `Tell me about your day'. These

open-ended questions were considered to be a better way

of asking residents to talk directly about their CC & Is.

Initially, residents were not able to articulate their CC & Is

until several months into the research and, as they began

to know and trust the evaluator, they haltingly con®ded

their concerns. Again, the notion of collaboration of all

stakeholders, particularly residents, took time to be fully

appreciated, given that the notion of `having a say' was

novel. Despite the increased preparation time, residents

still asked the evaluator to present their case at the

negotiating table, claiming that they lacked the con®dence

to confront others. Here we were prepared to act as the

lesser group's advocate.

Ethical concernAt the negotiation table, my colleague Trish Mitchell

brought residents' concerns to the group's attention. As

there were only seven residents to `have a say', nursing

home staff recognized their constructions and were able to

identify the residents to whom the constructions

belonged. Evaluation is related to the complexities of

practice and thus some loss of privacy, such as some data

being associated with individuals, is understandable. We

did not promise stakeholders con®dentiality or anony-

mity. Indeed, this is an ethical concern as the ability to

ensure privacy of respondents is not possible in this

evaluation process. However, through listening to

con¯icting accounts, it placed some of the residents at

risk of being marginalized and subsequently `punished',

particularly when those residents posed a point of view

that was recognized and not supported by the majority.

Residents concerns were unresolvedGiven the power differentials among stakeholders in this

nursing home, the idea that these stakeholders could

develop a common construction was doubtful, neverthe-

less we continued. Our task was to work with diverse and

contradictory constructions, tease out embedded values,

and raise these for open scrutiny. Concerns that were

registered in both the ®rst and second studies were with

routine geriatric style care. By that I mean a work practice

situation where two staff work together, beginning on one

side of the setting, a ward with 10 or so beds lined up, and

then give direct care to patients from beds one to 10,

providing hygiene, toilet and pressure area care. This

`geriatric routine' was challenged by older patients and

residents, respectively. However, this concern remained

an unresolved agenda item. Nurse assistants were not

persuaded that the routine should change to accommodate

residents' requests. There is still inaction, and the

abhorred routine still prevails.

Serious limitationsAlthough we believed that the FGE approach was feasible

in nursing homes, it was shown to have serious limita-

tions. Perhaps naively, we did not anticipate the dif®cul-

ties of involving stakeholders with different needs and

abilities to be actively engaged in the negotiation process.

Residents wanted their concerns heard, but they did not

want to `have a say'. Instead of the process working

towards empowerment of the resident group, here we

were faced with a reverse situation, one in which

unavoidable disclosure of their identity made them more

not less vulnerable. In addition, whilst there needs to be a

willingness for parties to change if they ®nd the negoti-

ations persuasive, here one group was unwilling to be

persuaded.

Project three

My interest in career structures stems from research work

undertaken in 1986 when a new clinical career structure

for nurses was planned, introduced, implemented and



evaluated in South Australia (Koch 1990). The structure

was based on the Royal Australian Nursing Federation

South Australian (SA) Branch Model, and its trial took

place in 11 health units, involving 5000 registered nurses,

one-third of the nurse workforce in SA. Central to the

model was a structure for the advancement of expert

clinicians that involved the separation of administrative,

teaching and clinical roles. The ®ndings of the evaluation

research were utilized by government, and the `new'

structure was implemented throughout South Australia

and other Australian states also adapted versions of the

structure.

BackgroundThe 1986 model's construction was largely theoretically

driven and imposed on nurses. Inherent in the model were

a number of features that have remained problematic.

These include dif®culties in separating advanced clinical

practice from the administrative role of the nurse

(McCoppin & Gardner 1994), role ambiguity, and lack of

equity in workload distribution (Bull & Hart 1995, Appel

et al. 1996). The imposed, rather than negotiated, char-

acter of the model has become increasingly problematic as

nurses pursue an ongoing process of professionalization

which implies increasing self-determination over the

content and character of their work (Davies 1995, Mascord

1992, Zadoroznyj 1997).

In the decade since the implementation of the new

career structure, there have been major changes in the

structure of health care organizations and in their ®nan-

cing, particularly with regard to purchaser±provider

agreements, which have signi®cant implications for the

way nursing work is organized. These changes make the

study of the nurses' career structure both timely and

necessary. The process of fourth generation evaluation

allowed us to explore whether a new structure could be

negotiated.

Aim to develop a negotiated career structurefor nursesThe third project involved about 70 stakeholders in a health

service organization in Australia providing district-nursing

services in the community. In this pilot study, a negotiation

of level three nurse roles in the Royal District Nursing

Service of SA Inc (RDNS) forms part of a broader objective

to develop a negotiated career structure for all RDNS

nurses. Level three nurses are those who work in middle

management as managers or are described at clinical nurse

consultant (CNC) level or with specialist portfolios, e.g. in

palliative care, HIV/AIDS, disabilities, diabetes, conti-

nence. Level two are the clinical nurses and level one are

the registered nurses, whereas the level four comprises

regional directors and level ®ve the director of the service.

The aim of the pilot study was to identify key stake-

holders' views on the level three nurse roles in RDNS in

the context of emerging RDNS directions and client needs.

There are currently 350 such nursing staff. The study

aimed to develop alternative models of the structure of

level three nurse roles so as to accommodate new

directions; this was achieved by means of a process of

negotiation with the key stakeholders utilizing the

methodology of FGE. Structures and processes were

established: an evaluation team, a project management

team and a steering committee, all with objectives and/or

terms of reference.

The evaluation time frameThe time frame for this study was 3 months. Two research

assistants were employed (Lewis and Bridge). Between

them they interviewed approximately 70 representatives

from stakeholder groups. Drawing upon observational

data, individual and group interviews, an agenda for

negotiation was developed. Discussion around, and

through, the hermeneutic circle was accomplished in a

very short time. Common CC & Is were identi®ed for each

of the stakeholder groups, as well as for stakeholders as a

whole, and these formed the basis for a series of stake-

holder group discussions. As a result of this consultative

process, ®ve alternative models of level three nurse roles

were developed, with all models incorporating the

upgrading of RDNS organizational infrastructure.

Bene®ts of the evaluationThe ®nal outcome of the study was model one, the most

conservative model. Whilst this may not appear as a

conclusive research outcome in traditional terms, it is one

which is consistent with the dynamics of the consultative

research process itself and one which has led to bene®ts

which go far beyond the project's terms of reference. These

bene®ts include a signi®cant increase in the involvement

of level three nurses in RDNS organizational matters

generally, and in their own career restructure in partic-

ular. The project recommendations incorporate the

consolidation of changes currently in train, including

the development of semiautonomous nursing care

delivery teams in each of the three RDNS metropolitan

regions and the mentoring of team members by CNC

specialists. These changes alone constitute a substantial

shift of responsibilities in the regions and should do much

to resolve current workload problems amongst level three

nurses. It remains to be seen, however, whether the

consolidation of current changes will be suf®cient to

address the broader issues of clinical management and

line responsibility with the tripartite regional structure.

An evaluation at the end of the consolidation phase

should provide some answers to this question.

Time to build trustIn my experience, trust is normally built between indi-

viduals only over a long period of time, and because only

T. Koch


3 months were allocated to the project (a management

decision), it could be envisaged that preparation and

sharing of information between groups would be rushed. It

was likely that trust would not have time to develop. This

will be revealed in the subsequent evaluation monitoring.

The consequences of negotiating at an uneven tableThe focus of this discussion is to show the way in which

the process of negotiation can be subverted. The question

to be asked again is how can anyone manage FGE in the

real world of power? Guba and Lincoln (1989 p. 267)

instigate the discussion:

Is it not naive to believe that any group in power would willingly

divest itself of that power in order to empower some other group?

Certainly that is a problem, and it bears no small resemblance to

the question of how the rich would consent to be taxed at a higher

rate in order that the poor would have more food stamps, have

better access to adequate health care or day care for their children,

or to be able to achieve higher levels of job training¼ In a

situation where there are vast power and information disparities,

one opening wedge may be a rede®nition of power, rather than to

regard it as a ®xed-sum commodity, for example, so that the only

way to acquire some of it is to take it away from someone who

already has it, we may wish to regard it as (potentially) ever

growing and enlarging.

Although there was more support for model three, in the

®nal vote at the project management team (with an

organizational representational body of 20 RDNS

personnel) a conservative model was selected, model

one. Model one has been fully implemented and its

evaluation is being progressed. In the section to follow

power relations are explored.

Some groups dominated the hermeneuticdialectical processFGE research aims to address the problem of political

exclusion by identifying those who have little or no voice

(the disenfranchised) and ensuring that they are given

one. It is important to strive for approximately equal

positions of power in negotiation. Not just in principle but

in practice. The evaluators observed the power differen-

tials and there were quite striking differences in the

negotiating strength of the various stakeholders within

this organization. This meant that some voices were not as

well heard as others.

Differences in inputFirstly, in the relatively informal discussion groups, there

were signi®cant differences in input between various

categories of level three nurses. For example, the CNC

specialists were both more numerous, and also more able

to plan their workloads, and thus formed the most

outspoken voice at general meetings. CNC generalists, on

the other hand, constituted a much smaller group and the

unpredictability of demands on their time made it more

dif®cult for them to attend meetings; hence their partic-

ipation in level three discussions was somewhat muted,

despite attempts by the project of®cers to facilitate their

contribution.

Domination of the process of negotiationSecondly, the process of negotiation at the more formal

level of the project management team (PMT) tended to be

dominated by representatives of senior management. This

was due in part to the greater negotiating experience and

con®dence of the individuals involved, but also due to the

fact that these individuals fell naturally into the roles of

bargaining agents in the context of the negotiation of the

career structure. Moreover, the pilot study formed part of

an ongoing enterprise Agreement negotiation process

between the two major parties. As a result, ®eld staff felt

somewhat alienated from the negotiation process, and

their attendance fell away. This problem of imbalance in

the negotiation process was not able to be resolved

successfully, partly because of the formal structure of

these latter meetings.

Negotiation with interests in mindThe CNC specialists opposed model three on the grounds

that it would restrict their clinical expertise and thus

lower the `expert' standing of the RDNS. Since this group

was both the most numerous and most vocal of the level

three nurses, and since one of their number was on the

Enterprise Agreement Committee which had responsibil-

ity for implementing the recommendations, their voice

was very powerful.

Industrial relationsThe project recommendations included the implementa-

tion of model one and incorporated the consolidation of

changes in train. However, the Australian Nursing Feder-

ation (ANF) opposed these recommendations on the

grounds that the project had side-stepped due industrial

relations processes, thereby undermining the authority of

the union and the interests of its members. First, the ANF

claimed it had not been informed of the project prior to its

commencement; second, the ANF felt that the pilot study

took no account of the union's overall strategy and

responsibility for the re-negotiation of the career structure

for nurses generally; and third, although management saw

the PMT as a working party of the Enterprise Agreement

Committee, this perception was not shared by the Austra-

lian Nursing Federation.

Honouring the principles of fourthgeneration evaluationAbiding by the principles of fourth-generation evaluation,

stakeholders were selected in ways that honoured the

principle of participatory (empowering, educative)



evaluation, but without being merely representative in the

statistical sense. This was not well understood as there

were concerns about the legitimacy of stakeholder repre-

sentation on the PMT, since stakeholders had been

selected by the researchers from `expressions of interest'

rather than being selected by their peers. Hence the ANF

saw the project as a management-driven exercise. On the

whole, RDNS senior management was disappointed with

the stakeholders' decision to vote for the most conserva-

tive model. However, they honoured the outcome of the

negotiation and model one has been fully implemented.

OVERVIEW: THREE APPLICATIONSOF FOURTH-GENERATION EVALUATION

In the ®rst application, older patients in care of the elderly

wards did not seek `having a say', but being acknowledged

as a person was important. In the second application, it

may be questioned whether the approach was the most

suitable for a nursing home setting given that many of the

residents had dementia. This meant that few residents

could `have a say'. The few residents and signi®cant

others who eventually shared their concerns were reluc-

tant to participate in the larger stakeholder group. Because

both residents and nursing staff had been in the nursing

home setting for many years, and knew each other's

speech peculiarities, residents were identi®able and

therefore at risk of being further disenfranchised. If we

recall that a major goal of fourth generation evaluation is

to work in an empowering way not a disempowering one,

this potential unintended effect is sobering. Meanwhile

residents still do not have a voice. What is the way

forward here? Should not an emancipatory project be

carried out where there is the greatest need?

In the third application, the 3-month time period for the

evaluation was too short. However, in the `real' corporate

world, time frames for evaluation are often unrealistic in

terms of research goals. It may be useful to apply the

properties earlier outlined by Strauss (1978), as evalua-

tions can be check-listed against these properties. The

issue of power, exhibited by the respective parties in the

negotiation itself, is clearly the most problematic in this

type of evaluation. The nature of their respective stakes in

the negotiation is interesting, and these questions need to

be asked at the outset of the process. When the process has

only 3 months in which to deliver a negotiated outcome,

short time frames may impede the visibility of the trans-

actions and diminish trust in the process. The complexity

of the issues negotiated in the third project, and the large

number of stakeholders certainly affected the outcome of

this project. Certainly the role of the unions cannot be

underestimated and it would be wise to include their

contribution at the outset. I believe the approach was

suitable, but time constraints meant that power relations

were not approached sensitively. Although we were very

aware of the power relations and the problem of negoti-

ating at an uneven table, there was inadequate time to

prepare, debrief, persuade, and draw stakeholders' atten-

tion to the principles of fourth generation evaluation, the

number of negotiators, their relative experience in nego-

tiation and whom they represent.

Limitations of the approach

The application of FGE to three projects has been

reviewed and some general questions around negotiation

have been raised. The rhetoric of FGE is that it may

empower, liberate and emancipate (Guba & Lincoln 1989).

Certainly FGE's emancipatory appeal has been the driving

force for the three case studies presented. It seems that the

most FGE can give stakeholders is a sense of control over

their lives and/or workplace. The case studies re¯ect

broader structural problems that constrain the actions of

some stakeholder groups, which in a sense are beyond the

parameters of the evaluation, yet which exercise enor-

mous control over the possibilities for change. It should be

noted that preparation time can be considerable, not only

in preparation of stakeholders for whom the process is

novel, but also for those stakeholders whose position in

the organization is less strong. There is an implication and

expectation that consensus will emerge, but this was

clearly not the case in these three examples. The power

aspects of the negotiation cannot be understated and,

indeed, evaluators would bene®t from cautious appreci-

ation and thorough prior analysis of situations. If know-

ledge is power, it should be recognized that the evaluator

has a privileged position. Although the evaluator can

direct negotiation in the process of action planning and

action taking, action is not always the outcome of the

negotiation process. Even if concerns are not addressed,

the fourth generation process has made stakeholder

concerns blatantly transparent and obvious. Whilst

participatory evaluation recognizes the importance of

representing stakeholders' interests, the question is still

one of how to improve fourth generation processes so that

all stakeholders can in¯uence and negotiate those interests.

CONCLUSION

In retrospect, the following criteria may be seen as

prerequisite for a successful negotiated evaluation

outcome:

· consider the environment (e.g. a small organization, a

ward or clinic);

· manage the process by using a maximum number of

stakeholders (40±50 persons);

· ensure commitment of all stakeholders to improve a

situation;

· enrol articulate stakeholders who can express their

concerns;

T. Koch


· negotiate a reasonable time frame; and

· appoint skilled evaluators who are able to manage the

power differentials in the hermeneutic dialectic.

Through applying those criteria, I still believe that FGE

has much to offer health and human service evaluation.

Acknowledgements

Thanks to co-researchers who worked with me in projects

two and three: Patricia (Trish) Mitchell, Maria Zadoroznyj,

Caroline Lewis, Ken Bridge and Sally Hudson. Project two

was funded by a Flinders University of South Australia

Establishment Grant and project three was funded by

RDNS (South Australia) Inc.

References

Appel A.L., Malcolm P. & Nahas V. (1996) Nursing specialization

in NSW, Australia. Clinical Nurse Specialist 10, 76±81.

Bull R. & Hart G. (1995) Clinical nurse specialist: walking the

wire. Contemporary Nurse 4, 25±32.

Davies C. (1997) Gender and the Professional Predicament in

Nursing 1st edn, Volume 1. Open University Press, Buckingham.

Fishman D.B. (1992) Postmodernism comes to programme eval-

uation: a critical review of Guba and Lincoln's fourth genera-

tion evaluation. Evaluation & Programme Planning 15, 263±70.

Guba E.G. & Lincoln Y.S. (1989) Fourth Generation Evaluation.

Sage, Newbury Park, California.

Koch T. (1990) A new clinical career structure for nurses: trial and

evaluation. The South Australian experience. Journal of

Advanced Nursing 15, 860±876.

Koch T. (1993) Toward fourth generation evaluation: listening to

the voices of older patients. A hermeneutic inquiry. Unpub-

lished PhD thesis, University of Manchester, Manchester.

Koch T. (1994) Beyond measurement: fourth generation evalua-

tion in nursing. Journal of Advanced Nursing 20, 1148±1155.

Koch T., Zadoroznyj M., Bridge K., Lewis C. & Hudson S. (1998)

Career structures for South Australian nurses: developing a

research agenda. Collegian 6, 34±37.

Laughlin R. & Broadbent J. (1996) Redesigning fourth generation

evaluation: an evaluation model for the public sector reforms in

the UK? Evaluation 2, 431±451.

Levy J.A. (1982) The staging of negotiations between hospice and

medical institutions. Urban Life 11, 293±311.

Mascord P. (1992) The career structure and lifelong education. In

Issues in Australian Nursing, Volume 1, 3rd edn (Gray G. &

Pratt R. eds), Churchill Livingstone, Melbourne, pp. 281±295.

McCoppin B. & Gardner H. (1994) Tradition and Reality. Chur-

chill Livingstone, Melbourne.

Mitchell P. & Koch T. (1997) An attempt to give nursing home

residents a voice in quality improvement process: the challenge

of frailty. Journal of Clinical Nursing 6, 453±461.

Papineau D. & Kiely M.C. (1996) Participatory evaluation in a

community organization: fostering stakeholder empowerment

and utilization. Evaluation & Programme Planning 19, 79±93.

Rebien C. (1996) Participatory evaluation of development assis-

tance. Evaluation 2, 151±171.

Reed J. & Payton V. (1997) Understanding the dynamics of life in

care homes for older people: implications for de-institutional-

izing practice Health and Social Care in the Community 5, 261±

268.

Strauss A. (1978) Negotiations: Varieties, Contexts, Processes, and

Social Order. Jossey-Bass, San Francisco.

Swenson M.M. (1991) Using fourth-generation evaluation in

nursing. Evaluation and the Health Professions 14, 79±87.

Zadoroznyj M. (1997) Transformation in collective identity of

Australian and US registered nurses. In Nursing Matters

(Keleher H. & McInerney F. eds), Pearson/Churchill Living-

stone, Melbourne.



‘having a say’: negotiation in fourth- generation evaluation

Documents