process evaluations - emcdda home page | … · workbook 4 • process evaluations 3 who/msd/msb...

1Workbook 4 · Process Evaluations

WHO/MSD/MSB 00.2e

ProcessEvaluations

Workbook 4

2 Evaluation of Psychoactive Substance Use Disorder Treatment

WHO/MSD/MSB 00.2e

WHOWorld Health Organization

UNDCPUnited Nations International Drug Control Programme

EMCDDA European Monitoring Center on Drugs and Drug Addiction

World Health Organization, 2000c

This document is not a formal publication of the World Health Organization (WHO) and all rights are reserved bythe Organization. The document may, however, be freely reviewed, abstracted, reproduced and translated, inpart or in whole but not for sale nor for use in conjunction with commercial purposes. The views expressed indocuments by named authors are solely the responsibility of those authors.


WHO/MSD/MSB 00.2e

Acknowledgements

The World Health Organizationgratefully acknowledges the con-tributions of the numerous indi-viduals involved in the preparationof this workbook series, includingthe experts who provided usefulcomments throughout its develop-ment for the Substance Abuse De-partment, directed by Dr. MaryJansen. Financial assistance wasprovided by UNDCP/EMCDDA/Swiss Federal Office of PublicHealth. Alan Ogborne (Canada)wrote the original text for thisworkbook and Brian Rush(Canada) edited the workbook se-ries in earlier stages. JoAnne Epping-

Jordan (Switzerland) wrote furthertext modifications and edited theworkbook series in later stages.Munira Lalji (WHO, SubstanceAbuse Department) and JenniferHillebrand (WHO, SubstanceAbuse Department) also edited theworkbook series in later stages. Thisworkbook’s case examples werewritten by David Cooper (UnitedKingdom), Ambros Uchtenhagen etal. (Switzerland), and edited byGinette Goulet (Canada). MaristelaMonteiro (WHO, SubstanceAbuse Department) provided edi-torial input throughout the devel-opment of this workbook.


WHO/MSD/MSB 00.2e


WHO/MSD/MSB 00.2e

Table of contents

Overview of workbook series 7

What is a process evaluation? 8

Coverage 8

Process 8

Why do a process evaluation? 9

Accountability 9

Programme development and improvement 9

To help others set up similar services or networks 9

How to do a process evaluation 10

1. Questions about coverage at the activity, service, or agency level 10

2. Questions about coverage at the system level 13

3. Questions about treatment process at the activity, service, or agency levels 14

4. Questions about treatment process at the system level 18

It�s your turn 20

Conclusion and a practical recommendation 22

References 23

Appendix 25

Psychoactive substance use 25

Risk behaviour 26

Health and correctional service utilisation 26

Client Motivation 28

Comments about case examples 30


WHO/MSD/MSB 00.2e

Case example of a process evaluation 31Process evaluation of alcohol homedetoxification and assessment 31

References for case example 39

It�s your turn 40

Case example appendix 1: AHDA individual client record 41

Case example appendix 2: AHDA code sheet 42

Case example appendix 3: Confidential client satisfaction scale 45

Case example appendix 4: Confidential supportersatisfaction scale 46

Case example appendix 5: Home detoxication follow-up 47

Case example appendix 6: Home detoxicationgeneral practitioner questionnaire 48

Case example appendix 7: AdvantageSM Software 49

Case example of a process evaluation 51Medical prescription of narcotics background andintermediate results of a Swiss National Project 51

References 59

It�s your turn 61


WHO/MSD/MSB 00.2e

Introductory WorkbookFramework Workbook

Foundation WorkbooksWorkbook 1: Planning EvaluationsWorkbook 2: Implementing Evaluations

Specialised WorkbooksWorkbook 3: Needs Assessment EvaluationsWorkbook 4: Process EvaluationsWorkbook 5: Cost EvaluationsWorkbook 6: Client Satisfaction EvaluationsWorkbook 7: Outcome EvaluationsWorkbook 8: Economic Evaluations

This workbook is part of a series in-tended to educate programme plan-ners, managers, staff and other deci-sion-makers about the evaluation ofservices and systems for the treatmentof psychoactive substance use disor-ders. The objective of this series is toenhance their capacity for carrying outevaluation activities. The broader goalof the workbooks is to enhance treat-ment efficiency and cost-effectivenessusing the information that comes fromthese evaluation activities.

This workbook is about processevaluation. Process evaluation in-

volves assessing the extent to whichyour treatment service or system isserving the people for whom it wasintended, as well as the processesinvolved in programme operationand delivery. The workbook focuseson evaluations that assess:

• who does or doesn’t use yourtreatment service or system

• the way people become involvedand enter treatment

• the intensity, quality and sequenc-ing of treatment activities

Overview ofworkbook series


WHO/MSD/MSB 00.2e

Process evaluations are at aimed at enhanc-ing your current programme by understand-ing it more fully. Process evaluations mea-sure what is done by the programme, andfor whom these services are provided. Ide-ally, process evaluations assist in the identifi-

cation of “active ingredients” of treatment,and assess whether a programme is meetingaccepted standards of care.

In general, process evaluations pose ques-tions in two areas: coverage and process.

Coverage• What proportion of those who might need

the service(s) actually used it/them?

• Has the service, or network of servicesserved the intended clients?

• What were the demographic and clinicalcharacteristics of clients?

• What proportion of clients completedtreatment and what were the characteris-tics of those who dropped out?

What is a processevaluation?

Process• By what route have clients entered

treatment?

• How long was the waiting list and howhas it been managed?

• What actually happened to clients in treat-ment and is this what was intended?

• Were treatment plans consistent with theresults of assessment?

• What was the average length of stay orthe average number of appointments kept?

• How were discharge plans developed?

• Were services within the community treat-ment network well-co-ordinated?

Coverage and process questions can beasked at different levels of treatment (seeFramework workbook): activity, service,agency, or system levels. Procedures for dif-ferent levels will be discussed in detail.

Test your knowledge. Write down whethereach of the following questions is a cover-age question or a process question:

a) Is the programme serving clients it wasintended to serve?

b) How many direct contact hours does eachclient receive?

c) What is the average age of programmeparticipants from each referral source?

d) What percent of clients complete theprogramme?

(Answers: coverage, process, coverage, process)


WHO/MSD/MSB 00.2e

Process evaluations are undertaken for avariety of reasons. They are most useful whenclear objectives have been developed for the

Accountability

evaluation and the intended users are involvedat the planning stage. The main reasons forconducting process evaluations are:

Programme developmentand improvement

Why do a processevaluation?

dence that the funds are being used as ex-pected. Clients, their families, referral agentsand the public at large expect managers oftreatment services and systems to be ac-countable for their use of resources and levelof service provision.

How can the programme be improved?Process evaluations can provide in-depthinformation about the functioning of treat-ment services and systems, and pin-point

areas where improvements might be made.For example, a process study might showduplication in the assessment process.

To help others set up similarservices or networksHow can the programme be expanded toother areas? If a service or treatment systemachieves high rates of success with peoplewith PSU disorders, it will be desirable to

replicate this service or system in otherplaces. Information therefore will be neededabout clients, staffing, and the inter-relation-ship of clinical activities.

Is the programme accomplishing what it isexpected to accomplish? Many groups wantto know the answers to this question. Gov-ernments and social agencies that sponsortreatment for PSU disorders sometimes re-quire recipients of funding to provide evi-


WHO/MSD/MSB 00.2e

When doing a process evaluation, it is im-portant that you follow the general stepsof evaluation planning and implementationas outlined in Workbooks 1 and 2. In ad-dition, it is important to read through the

1. Questions about coverage at theactivity, service, or agency level

sess for purposes of monitoring programmecoverage. The range of information that youcollect will depend somewhat on your levelof resources (Step 2 of evaluation planning -Workbook 1). Regardless, it is important thatyou gather enough information to comparethe characteristics of your clients to the typeof client reflected in your programme objec-tives.

Specific instruments and items that can beused to measure variables related toprogramme coverage are identified in Table1. Many of these instruments are in thepublic domain and are included in a Di-rectory of Outcome Measures publishedby the Addiction Research Foundation(ARF) of Ontario, Canada. The addressfor the ARF and those measures consideredmost relevant for this workbook are includedin Workbook 1, Appendix 2. The measurescan be used to describe clients at the time ofintake. Many can also be used at follow-up.Because most of these questionnaires weredeveloped in Western countries, care shouldbe taken to ensure that the questions areappropriate and understandable in your cul-ture and setting.

Example questions:

• What proportion of those who might needthe activity, service, or agency actuallyused it?

• Has the activity, service, or agency servedthe intended clients?

• What were the demographic and clinicalcharacteristics of clients?

• What proportion of clients completedtreatment and what were the characteris-tics of those who dropped out?

When evaluating programme coverage at theactivity, service, or agency level (see Frame-work manual for definitions), you typicallyinclude information on clients’ age, gender,education and employment status, source ofreferral, place of residence, current PSU andrelated problems, participation in previoustreatment and stability in various life areas.Information on these variables is requiredboth to plan treatment services and provideaccountability information.

There are no established guidelines for se-lecting other characteristics of clients to as-

It is importantthat you gatherenoughinformation tocompare thecharacteristicsof your clientsto the type ofclient reflectedin yourprogrammeobjectives.

information here, to provide you withspecialised details about conducting differ-ent kinds of process evaluations. Four cat-egories of process evaluation questions arepresented below.

How to do aprocess evaluation


WHO/MSD/MSB 00.2e

Table 1: Measures available for basic coverage variables

Variables

AgeGender

Marital Status

EducationEmployment and financial statusSource of referralType of residenceInvolvement in health and justice systemInvolvement of other treatment servicesTreatment Unit Characteristics

Alcohol consumption

Other PSU

Adverse consequences of alcohol use

Adverse consequences of other PSUPSU, harmful use and dependence

HIV-Risk behaviours

General mental health

Physical health

Motivation for treatment

Measures

Recorded in years

Male, female

Typical items can be found in resources such as the ARF Directory ofOutcome Measures (undated), the Resource Manual for the How Good isYour Drug Abuse Treatment Programme (NIDA, 1993), the AlcoholismTreatment Assessment Instruments (Lettieri et al., 1984), and the DrugAbuse Instrument Handbook (Nehemkis et al., 1976)

Use of health and correctional services are captured by a Health and CorrectionsUtilisation Form appropriate for process evaluation and telephone follow-up (Workbook 1, Appendix 2). The Addiction Severity Index (ASI) hasLegal and Employment Subscales (McLellan et al., 1988)

Treatment Unit Form (available from EMCDDA)

Examples of quantity/frequency measures are included in the ARF Directoryof Outcome measures (undated)Lifetime Drinking History (Skinner, 1979)Timeline Followback Method (Sobell and Sobell, 1988)Alcohol subscale of the ASI (McLellan at al., 1988)(see Workbook1, Appendix 2 for a brief format (DHQ) that is useful for bothprocess evaluation and telephone follow-up)

Examples are included in the ARF Directory of Outcome Measures (undated)Drug subscale of the ASI (McLellan et al., 1988)(see Workbook 1, Appendix 2 for a brief format (DHQ) that is useful forboth process evaluation and telephone follow-up)

Examples of some approaches to measuring consequences other thandependence are included in the ARF Directory of Outcome MeasuresShort Alcohol Dependence Data (Raistrick et al., 1983)Alcohol Dependence Scale (Horn et al., 1984)Adverse Consequences of Alcohol Scale (Miller et al., in press)

Drug Abuse Screening Test (Skinner, 1982)

The AUDIT (WHO, 1992)The Composite International Diagnostic Interview (Robins et al., 1989)WHO-CIDI, M -CIDI (Witchen, 1994)Alcohol Dependence Scale (Horn et al., 1984)

(see Workbook 1, Appendix 2 for a brief (DHQ) format that is useful forboth process evaluation and telephone follow-up)

Symptom Checklist 90 (Derogatis, 1977)Profile of Mood States (McNair et al., 1971)Psychiatric Status subscale of the ASI (McLellan et al., 1988)

Health Questionnaire (Brodman et al., 1949)Medical problems subscale of the ASI (McLellan et al., 1988)

McArthur Scale (Gardner et al., 1993)Treatment Entry Questionnaire (Wild, 1996)(see Workbook 1, Appendix 2 for a brief questionnaire)


WHO/MSD/MSB 00.2e

Other client variables relevant to the evalu-ation of programme coverage are: situa-tions associated with PSU, self-efficacyand coping skills, depression and otheraspects of psychological well-being andsocial/marital relationships. Many of these

variables have been shown to influencetreatment compliance and outcomes, andan assessment of these variables is recom-mended if resources permit. Instrumentsthat measure these factors are noted inTable 2.

Example case: researchabout coverage at theactivity, service, or agencylevel

This example programme is the preventioncomponent of a youth PSU services agency.Clients range in age from 16 to 24, and arereferred to the programme by a variety ofsources. The four session preventionprogramme is designed for youth with a lowlevel of PSU and PSU-related problems.

Research questions:1) Has the number of referrals from schools

and probation services increased from theprevious year?

2) What percent of clients are ages 16-19?

Table 2: Measures available for other client characteristics

Variables Measures

PSU situations Inventory of Drinking Situations (Annis et al., 1987)

Inventory of Drug Use Situations (Annis and Graham, 1991)

Self-efficacy and coping skills Drug Taking Confidence Questionnaire (Annis and Martin, 1985)

Situational Confidence Questionnaire (Annis and Graham, 1988)

Coping Behaviours Inventory (Litman et al., 1983)

Depression Beck Depression Inventory (Beck at al., 1961)

CES-D (Radloff, 1977)

Anxiety State-Trait Anxiety Inventory (Spielberger et al., 1970)

Self-esteem/confidence Rosenberg Self-esteem Scale (Rosenberg, 1965)

Family relationships/ functioning Family Assessment Measure (Skinner et al., 1983)

Family Environment Scale (Moos and Moss, 1986)

Family/Social Subscale of the ASI (McLellan et al., 1988)

Marital relationships/ functioning Dyadic Adjustment Scale (Sapinier, 1976)

Perceived support from friend and family Perceived Social Support - Family and Friends(Procidano and Heller, 1983)

3) What is the average age of programmeparticipants from each referral source?

What resources were needed to gatherthis information?Information is collected on an ongoing basis.The programme secretary spent about threehours entering information on 200 clients intothe central database.

How were the data collected?As each client is admitted to the preventionprogramme, counsellors record his/her sex,age, and referral source on a form. At theend of the year, all forms are forwarded tothe secretary for tabulation.

How were the data analysed?Number and percent of clients from eachreferral source was calculated and compared


WHO/MSD/MSB 00.2e

to last year's figures. Average age overall wascalculated, as well as average age from eachreferral source.

What did they find out?The results indicated that the number of re-ferrals from the legal system and from schoolshas increased substantially from last year.Eighty-seven percent of participants were in

the 16-19 age group. The average age, over-all, was 17.8 years. Average age did not varygreatly based on referral source.

What will they do with this information?The programme staff expects continuing in-creased number of referrals. As a result, theyplan to expand their programme in the com-ing year.

2. Questions about coverage at thesystem levelExample questions:

• How many treatment programmes existin the region?

• How many clients are seen by eachprogramme in a year?

• Are there differences in the types ofclients seen at each programme?

Many variables of interest in coverage stud-ies that involve networks of services are thesame as those for studies of individual activi-ties, services, or agencies. The analysis fo-cuses on similarities and differences in clientprofiles across agencies. The profile of allclients served by the network is of consider-able interest.

The scope of information collected to moni-tor coverage across a network of serviceswill likely be smaller than that collected byany one service. This is due largely to thedifficulty in achieving agreement among par-ticipating programmes on how key clientcharacteristics should be measured. Practi-cal issues of managing and analysing the in-formation keep the data collection focusedon critical pieces of information required formonitoring coverage of the overall treatmentsystem.

As with the selection of variables within indi-vidual services, your selection at the system-level should be guided by objectives and in-ter-agency agreements. For example, your

treatment network may determine that cli-ents with significant PSU and psychiatric co-morbidity should be fully assessed and treatedin particular facilities with adequately trainedstaff. Your system evaluation plan should theninclude routine monitoring of the flow of ap-propriate clients to these designated facili-ties. Similarly, a system with agreed upon ad-mission and discharge criteria based onproblem severity would routinely monitor theseverity of clients'problems across allprogrammes in order to ensure that the ap-propriate match to treatment is being madeand that the admission criteria are being fol-lowed system-wide.

Example case: researchabout coverage at thesystem levelA new detoxification programme was devel-oped. The programme has 16 beds, and isallotted to clients from the outlying region.There are several other detoxificationprogrammes within the city boundaries.

Research questions:

1) What percent of clients in the new detoxi-fication programme lives in the outlying area,and what percent comes from within the cityboundaries?

2) Are the clients from the different detoxifi-cation programmes different in terms of age,sex, or PSU?

The analysisfocuses onsimilarities anddifferences inclient profilesacrossagencies.


WHO/MSD/MSB 00.2e

What resources were needed to answerthis question?Managers from the different detoxificationcentres collaborated to devise a one-pagedata collection form. This form was copiedand distributed to clinicians for use with allnew clients.

How were the data collected?Upon admission to each detoxificationprogramme, the admitting clinician com-pletes a one page form, which includes theclient’s age, sex, address, and a series ofquestions about current PSU: type(s) andfrequency. These forms are forwarded toa central site, where they are recorded ina log book and maintained by theprogramme data manager.

How were the data analysed?Clients’ data were divided according to dif-ferent detoxification sites. Within the new

detoxification site, clients’ addresses werecoded for (1) living within the outlying area,or (2) living within the city. The percentageof clients from each geographic area wascalculated. Clients’ ages, sexes, and PSUpatterns were calculated across detoxifica-tion centres.

What did they find out?The majority of clients in the new detoxifi-cation centre lived in the outlying area(87%). Clients from different detoxifica-tion sites differed according to age (23, 35,and 19 years) and percent male (66%,53%, and 78%), but not by PSU type orfrequency.

What did they do with this information?Based on the results, administrators felt com-fortable that they were serving the desiredpopulation with the new detoxification cen-tre. No changes were implemented.

3. Questions about treatmentprocess at the activity, service, oragency levels

Example questions:

• By what route did clients enter treatment?

• What actually happened to clients in treat-ment and is this what was intended?

• What was the average length of stay orthe average number of appointments kept?

Treatment processes in activity, service, oragency level evaluations include those thatinfluence entry into treatment and the admin-istrative and clinical activities to which cli-ents are exposed while in treatment. Vari-ables concerning treatment entry include theextent to which potential clients and referralagents are aware of the service in questionand factors that influence one's decision toseek or recommend treatment. This includes

coercive factors in the justice system. Twoquestionnaires that explore these issues fromthe client's perspective are included in Table1 (McArthur Scale (Gardner et al., 1993);Treatment Entry Questionnaire (Wild,1996)). Other, more qualitative, questionsfor clients may be:

• How did you first hear about (name ofservices)?

• Who first suggested that you come to(name of service)?

• Why did you come to (name of service)as opposed to somewhere else?

• Are you under any pressure to come to(name of service) , for example, fromthe courts, your employer, school orfamily?

Treatmentprocesses inactivity, service,or agency levelevaluationsinclude thosethat influenceentry intotreatment andtheadministrativeand clinicalactivities towhich clientsare exposedwhile intreatment.


WHO/MSD/MSB 00.2e

Qualitative questions for community pro-fessionals about treatment referrals maybe:

• How did you first hear about (name ofservice)?

• Are you aware of the services offered andthe intake process?

• Have you referred clients in the past/doyou intend to refer clients in the future?

• What types of clients do you refer/not referand why?

Treatment process evaluations serve twoimportant functions:

Active ingredientsEvaluations of treatment processes areessential in order to identify the “active in-gredients” of treatment. The clinical pro-cesses in many treatment services may bequite complex and encompass a variety ofdistinctive components. Such components

may include, for example, intake, assessment,group or individual treatment, discharge plan-ning and follow-up.

Standards of careProcess evaluations also help to make judge-ments about the quality of administrative andclinical processes treatment services, againstlocal or more widely used standards aboutwhat is expected in a “quality” treatment fa-cility. Explicit standards for treatment servicesfor PSU disorders have been developed insome regions. One set of standards has beendeveloped by the World Health Organiza-tion (WHO, 1993). These standards arelisted below. Example items may not be ap-plicable to your specific settings, but generalareas can be used as a guide for developingyour own standards.

Basic standards are essential for any treat-ment organisation. Evaluation of adherenceto these standards is an important type ofprocess evaluation.

Basic standardsare essential forany treatmentorganization.

WHO standards of care in PSU treatment

a) Standards on access, availability, andadmission criteria

Examples:• services are available irrespective of

age or gender of all potential patients• services are easily accessible with re-

gard to location, travelling time, andtransportation

b) Standards on assessmentExamples:

• An initial assessment is made in orderto prioritise interventions in a co-ordinated treatment plan

• Methods for determining quantities ofPS(s) ingested are available

c) Standards on treatment content, pro-vision, and organisation

Examples:• A record of patient management,

progress, and onward referral is kept

and updated regularly to ensure conti-nuity of clinical care

• The range of relevant treatment op-tions available is described to the pa-tient

d) Standards on discharge, aftercare,and referral

Examples:• Discharge is based on determination

of patient recovery status• Attention is paid to further treatment

and support which may be required

e) Standards on outreach and early in-tervention

Examples:• There is promotion in settings other than

health facilities (e.g., workplace, schools)of early intervention

• Primary health care, other health care,welfare staff, and police are trained dur-


WHO/MSD/MSB 00.2e

ing their education in the recognition,basic management, and referral of in-dividuals with drug-related disabilities

f) Standards on patients’ rightsExamples:

• Patients are fully informed of the na-ture and content of the treatment aswell as the risks and benefits to be ex-pected

• Physical restraint is not used to detainor restrain patients who are legallycompetent to leave

g) Standards on physical aspects oftreatment settings

Examples:

• The physical environment is designed toprotect the well-being of patients

• Patients being treated on an inpatient/residential basis have access to pri-vacy and recreational facilities

h) Standards on StaffingExamples:

• Appropriately trained staff are eitheravailable on site in treatmentprogrammes, or are available on callat all times when treatment is beingprovided

• Staff have regular supervision by se-nior staff, peer review, and case con-ferences to maintain quality of servicedelivery

Assessment of adherence to basic standardsare not the same as continuous quality im-provement (Rush and Krywonis, 1996), in

which managers and staff seek to constantlyimprove programme performance. Some dif-ferences are listed on this page:

Continuous qualityimprovement

• is driven primarily by the goals of highcustomer satisfaction

• strives to raise the performance of ev-eryone, not just the poor performers

• process empowers staff to become in-volved and effect real change in theirwork environment

• focuses on the whole organisation –both within and across departments

• focuses on prevention but employs aproblem-solving approach using a widerange of tools and analytic methods toidentify causes, understand processes,measure and track

• focuses on the quality and inter-rela-tionship of all services, products andprocesses for clients and other custom-ers (e.g., families, staff). Looks beyondclinical staff to clients, their families,community stakeholders and internalstaff. Everyone is a potential customerperformance and monitor the changesto ensure gains are sustained

Assessment ofbasic standards

• assesses current practice against pre-es-tablished performance criteria or standards

• focuses on departments or individualswhose performance deviates from ac-cepted values or standards

• staff are not closely involved in setting theperformance standards or assessing theirachievement

• occurs within specific departments and as-sesses the functioning of that department

• focuses on programme environment andactivities; tends not to address client out-comes

• focuses on the quality of clinical care de-livered by professionals and received byclients


WHO/MSD/MSB 00.2e

If your organisation uses a continuous qual-ity improvement (CQI) approach, it is im-portant that you consider how the evaluationrelates to this broader process. CQI is notsufficient, in isolation, to make judgementsabout standards of care. Nonetheless, it is auseful option to consider for some agencies.

Creating a CQI Programme

If you are interested in initiating a CQIprogramme, a comprehensive approach isrequired. This includes:

• conducting self-assessments to determineorganisational readiness to implementCQI

• developing a supporting organisationalstructure, culture and leadership

• understanding the programme’s mainfunctions, treatment processes and client;

• developing and training CQI teams andfacilitators

• planning a reward and recognitionprogramme

• designing and adapting CQI approachesand materials to meet the needs of theagency

• reviewing and/or designing informationsystems necessary to support CQI ac-tivities

• infusing a CQI philosophy that guides allthe organisations and staffs actions andactivities

It is important not to be discouraged withthe scope of the task. Agencies must deter-mine the rate of CQI implementation that isfeasible within the context of available re-sources and other internal and external pres-sures. To be successful, implementation of aCQI programme needs to become part ofongoing strategic planning and management.

Example case: researchabout treatment process atthe activity, agency, orservice level

This example programme is a PSU assess-ment/referral service. The mandate of theprogramme is to perform a comprehensiveassessment of every client and to refer cli-ents to appropriate services within the agencyto begin within 2 months of the intake.

Research questions1) What was the average number of days

between the intake and the first sched-uled appointment?

2) What percent of clients did not show fortheir first scheduled appointment?

What resources were needed to answerthis question?In order to keep statistics on appointmentsand attendance, the programme secretarymaintained the database on a weekly basis.

How were the data collected?Attendance data were recorded on dailyappointment logs kept by the assessmentworkers. The intake worker kept a recordof the initial intake and the date of the initiallyscheduled assessment appointment. If achange was made in the appointment, theintake worker reported this to the secretary,who updated this in the database.

How were the data analysed?The information required for the evaluationalready existed in the database. Theprogramme secretary completed the follow-ing descriptive statistics:1) average number of days between intake

and first scheduled appointment2) percent of referrals who attended at least

one orientation session

What did they find out?There were 984 clients who scheduled atleast one assessment appointment during thepast year. The average length of time


WHO/MSD/MSB 00.2e

between the intake and first assessmentappointment was 67 days. Seventy-sixpercent of patients had their first appointmentwithin 60 days of their intake. Thirty-ninepercent of clients were “no-shows” for theirassessment appointments.

What did they do with this information?Based on the finding that 34 percent of pa-tients were not meeting the mandated timeframe of 2 months between assessment and

first appointment, and that about 40 percentwere “no-shows” to their appointments, theagency decided to reorganise so that firstappointments occurred on a specified dayof the week at a specific time. Several staffmembers were on-call during that time frameto see people as they appeared for their ap-pointments. With this new system, more pa-tients could be scheduled earlier, and theinefficiency of staff waiting for “no-shows”was greatly reduced.

4. Questions about treatmentprocess at the system level

Example questions:

• Are different treatment programmes awareof one another?

• Do different treatment programmes referclients to one another?

• What is the relationship between generalmedical services and specialised treat-ment programmes?

The study of PSU treatment systems is a newfield of research and there is no agreementconcerning the types of variables that shouldbe considered in system-level process evalu-ations. One of the first steps in system-levelevaluation is to define the boundaries of thesystem itself. You may choose, for example,to focus only on the specialised alcohol anddrug treatment agencies in a particular juris-diction. Increasingly, however, system-levelplanners and local providers are concernedwith co-ordination among specialised treat-ment programmes and between theseprogrammes and other services that clientsmay need (e.g. mental health services, em-ployment services, services in the justice sys-tem) (Institute of Medicine, 1990).

Co-ordination can be defined as the degreeto which agencies in a given network col-laborate and exchange information and re-sources (staff, funds, material etc.). Co-or-dination is typically assessed using reportsand ratings from directors or managers ofagencies that are expected to work togetherin service planning and delivery. The focusof these reports and ratings has been on:

• mutual awareness - the extent to whichstaff know about each other and their re-spective programmes

• frequency of interaction - how oftenkey staff meet to discuss work-relatedissues

• frequency of cross referrals - how oftenor how many clients are referred to andfrom different services in the network

• information exchange - the extent towhich services exchange information

• staff sharing or exchange - staff of dif-ferent services are permanently or tem-porarily shared or loaned

• other resource exchanges - the extentto which services share funds, meetingrooms, materials or other resources

... areconcerned withco-ordinationamongspecialisedtreatmentprogrammesand betweentheseprogrammesand otherservices thatclients mayneed.


WHO/MSD/MSB 00.2e

• consultations and case conferences -exchanges that concern the treatment ofspecific clients

• overlapping boards - the number ofmembers in common to communityboards of different services

• normalisation of agreements - the ex-tent to which services have developedformal agreements to co-ordinate theiractivities

Example case: researchabout treatment process atthe system level

This example is a network of PSU treatmentcentres that are interested in knowing moreabout assignment to treatment for clients whohave PSU problems. The network has cen-tres in the city centre and in outlying areas.

Research questions:

1) For each centre, from where do referralsoriginate?

2) What is the waiting time for services withineach centre?

What resources were needed to answerthis question?

Case workers recorded referral informationas part of their intake interviews. Consolida-tion of these data and tabulation of resultswere completed by one of the programme'ssecretaries on a monthly basis, taking aboutone hour per month. Each programme man-ager completed an interview and question-naire that assessed current treatment ser-vices.

How were the data collected?

The data collection occurred in two parts.For the first part, assessing the referral source,case workers recorded this information aspart of their intake interviews. Consolidationof these data and tabulation of results werecompleted by a programme secretary on amonthly basis. For the second part, assess-ing treatment services, two interviewers weredispatched to each programme to collect dataon waiting times for current treatment ser-vices.

How were the data analysed?

All information was entered by a programmesecretary into a central records notebook.The secretary calculated the percentage ofreferrals from each referral source for eachprogramme, and categorised waiting timesfor the different types of treatment servicesoffered.

What did they find out?

For the downtown PSU programme, thegreatest number of clients (37 percent) werereferred by the local emergency department.However, the emergency department tendednot to make referrals to outlying clinics, evenfor clients who lived in these regions. Mostreferrals from outlying clinics came from fam-ily members or were self-referrals. The out-lying PSU programmes overlapped consid-erably with the downtown programmes, buttended to have shorter waiting lists.

What did they do with this information?

Researchers concluded that more educationof emergency department staff about theoutlying PSU programmes (including shorterwaiting lists) was warranted. They instituteda brochure campaign and a series of presen-tations at emergency department staff meet-ings to accomplish this goal.


WHO/MSD/MSB 00.2e

It�s your turn

Put the information from this workbook touse for your own setting. Complete theseexercises below.

Remember to use the information fromWorkbooks 1 and 2 to help you complete afull evaluation plan. Review that informationnow, if you have not already done so.

1 Apply your knowledge. Write down tworelevant examples of coverage questionsand two examples of process questionsfor your own setting or treatment network.

Example: What is the average lengthof treatment for cocaine users vs.Opioid users?

1) __________________________________________________________2) __________________________________________________________3) __________________________________________________________4) __________________________________________________________

Decide the most important question(s) topursue in the evaluation (see Workbook1 for guidance on how to do this).

2 Determine whether your setting has aninformation and/or data analysis systemthat routinely generates basic informa-tion on clients, services and dischargecircumstances. This system could becomputer-based or based on paperrecords. These records may be espe-cially useful sources of data for yourprocess evaluation.

3 Using the information provided in thisworkbook, make the following deci-sions:

Decide what method you will use to col-lect the data. Review the information inthis workbook, and Workbook 1, tohelp you decide. For residentialprogrammes, include number of days inresidence. For non-residential services,include the number of sessions at-tended, and the number of appoint-ments missed.


WHO/MSD/MSB 00.2e

4 You will need to prepare an introductoryletter and consent form that explains thepurpose of your study (even for recordreviews). Review Section 1A of Work-book 2, entitled, “Manage Ethical Issues”,for more information about the importanttopic of participants' rights in evaluationresearch.

In general, all participants should be askedpermission ahead of time before beingenrolled in the study. When you do this,you should explain the purpose, nature,and time involved in their participation. Noperson should be forced or coerced toparticipate in the study.

The standard practice is to have eachparticipant sign a consent form, which:

• describes the purpose and methods ofthe study

• explains what they will need to do ifthey participate

• explains that participation is voluntary

In some cases, existing databases may beaccessed without formal permission fromclients. Consult local advisors for guide-lines in your institution and/or region.

Using the information and examples pro-vided in Workbook 2, section 1A, writeyour own introductory letter and consentform.

5 Run a pilot test of your evaluation mea-surement and procedures on 10-15 pa-tients to ensure that everything runssmoothly. Review section 1C of Work-book 2 entitled “Conduct a Pilot Test”for specific information about how to dothis. In general, pilot tests assess thesequestions:

• Do the questions provide useful infor-mation?

• Can the questions be administeredproperly? For example, are they toolong or too complicated to be filled outproperly?

• Can the information be easily managedby people responsible for tallying thedata?

• Does other information need to becollected?


WHO/MSD/MSB 00.2e

In this workbook, we have outlined the ba-sic principles and practices of process evalu-ation of PSU services and systems. With pro-cess evaluations, you are concerned withhow people enter into your treatment ser-vice or system and what happens to themonce there. The specific questions and issuesthat you explore will be dependent on yourunique circumstances and cultural context.These questions and issues must be clearlyidentified through the evaluation planningphase as described in Workbook 1.

In undertaking process evaluation, it is es-sential that you pay close attention to the prin-ciples and practices of data collection andanalysis as outlined in Workbook 2. Trade-offs have to be made as to the rigour withwhich you collect and analyse information toanswer your evaluation questions, and theresources you have available. You must striveto achieve the best possible information withthe time and resources available to you. Youmust carefully document the limitations ofyour findings and conclusions. With theseprinciples in mind, you will be able to under-take practical and useful process evaluationof your treatment service or system.

After completing your treatment evaluation,you want to ensure that your results are putto practical use. One way is to report your

results in written form (described in Work-book 2, Step 4). It is equally important, how-ever, to explore what the results mean foryour programme. Do changes need to hap-pen? If so, what is the best way to accom-plish this?

Return to the expected user(s) of the researchwith specific recommendations based on yourresults. List your recommendations, link themlogically to your results, and suggest a pe-riod for implementation of changes. The ex-amples below illustrate this technique.

Based on the finding that over 60 percent ofclients are waiting more than 2 months for aninitial appointment, we recommend that theprogramme convert to a group assessmentformat that uses paper and pencil question-naires as an initial screening tool.

Remember, process evaluations are a criti-cal step to better understanding the day today functioning of your PSU services. It isimportant to use the information that processevaluations provide to redirect treatment ser-vices. Through careful examination of yourresults, you can develop helpful recommen-dations for your programme. In this way, youcan take important steps to create a “healthyculture for evaluation” within yourorganisation.

Conclusionand a practicalrecommendation


WHO/MSD/MSB 00.2e

References

Addiction Research Foundation. Directoryof Outcome Measures for Addiction Treat-ment Programs. Available for MarketingDepartment, Addiction Research Founda-tion, 33 Russell Street, Toronto, Ontario,Canada, M5S 2S1, undated.

Annis, H.M., & Martin, G. Drug TakingConfidence Questionnaire (DTCQ).Toronto: Addiction Research Foundation,1985.

Annis, H.M., & Graham, J.M. SituationalConfidence Questionnaire (SCQ). User’sguide. Toronto: Addiction Research Foun-dation, 1988.

Annis, H.M., & Graham, J.M. Inventory ofDrug Taking Situations (IDTS). Usersguide. Toronto: Addiction Research Foun-dation, 1991.

Annis, H.M., Graham, J.M., & Davis, C.S.Inventory of Drinking Situations (IDS).User’s guide. Toronto: Addiction ResearchFoundation, 1987.

Beck, A.T., Ward, C.H., Mendelson, M.,Mock, J., & Erbaugh, J. An inventory formeasuring depression. Archives of Gen-eral Psychiatry, 1961, 4:561-571.

Brodman, K., Erdmann, A.J., Lorge, I., &Wolff, H.G. (1949). The Cornell MedicalIndex: An adjunct to medical interview. Jour-nal of the American Medical Association,1949, 140:530-534.

Derogatis, L.R. SCL-90: Administration,scoring and procedures manual-I for therevised version. Baltimore: Clinical Psycho-metrics Research, 1977.

Gardner, W., Hoge, S.K., Bennet, N., et al.Two scales for measuring patients' percep-tions of coercion during mental hospital ad-mission. Behavioral Sciences and the Law,1993, 11, 307-321.

Horn, J.L., Skinner, H.A., Wanberg, K., &Foster, F.M. Alcohol Dependence Scale(ADS) Toronto: Addiction Research Foun-dation, 1984.

Institute of Medicine, Broadening the baseof alcohol treatment. Washington D.C.National Academy of Science Press, 1990.

Lettieri, D.J., Nelson, J.E., & Sayers, M.A.Alcoholism treatment assessment instru-ments. NIAAA treatment handbook series.Rockville, Maryland: NIAAA, 1984.

Litman, G.K., Stapleton, J., Oppenheim,A.N., & Peleg, M. An instrument for mea-suring coping behaviours in hospitalized al-coholics: Implications for relapse preventiontreatment. British Journal of Addiction,1983, 78:269-276.

McLellan, A.T., Laborsky, L., Caccida, J.,et al. Guide to the Addiction Severity In-dex: Background, administration and fieldtesting results. U.S. Department of Healthand Human Services. Rockville, Maryland,1988.

McNair, D.M., Lorr, M., & Droppleman,L.F. Profile of Mood States Manual. SanDiego: Educational and Industrial TestingService, 1971.

Miller, W.R., Tonigan, J.S., & Longabaugh,R. The Drinker Inventory of Conse-quences (Dr.Inc): An instrument for as-sessing adverse consequences of alcoholabuse. Test Manual, NIAAA Project Matchmonograph Series, Vol. 4. NIH Pub. No.95-3911. Washington, DC: US GovernmentPrinting Office, in press.

Moos, R.H., & Moos, B.S. Manual for theFamily Environment Scale (2nd. Ed.). PaloAlto: California, 1986.

National Institute on Drug Abuse. How goodis your drug abuse program? National In-


WHO/MSD/MSB 00.2e

stitute of Health Publication No. 93-3608,Rockville, MD, 1993.

Nehemkis, A., Macari, M.A., & Lettieri, D.J.(Eds.) Drug abuse instrument handbook:Selected items for psychosocial drug re-search. Rockville, MA: National Institute onDrug Abuse, 1976.

Procidano, M.E., & Heller, K. Measures ofperceived social support from friends andfrom family: Three validation studies. Ameri-can Journal of Community Psychology,1983, 11(1):1-24.

Radloff, L.S. The CES-D scale: A self-re-port depression scale for research in the gen-eral population. Applied Psychological Mea-surement, 1977, 1: 385-401.

Raistrick, D., Dunbar, G., & Davidson, R.Development of a questionnaire to measurealcohol dependence. British Journal ofAddiction, 1983, 78: 89-95.

Robins, L.N., Wing, J., Wittchen, H.-U.,Helzer, J.E., Babor, T.F., Burke, J., Farmer,A., Jablensky, A., Pickens, R., Regier, D.A.,Sartorius, N., & Towle, L.H. The Compos-ite International Diagnostic Interview: Anepidemiologic instrument suitable for use inconjunction with different diagnostic systemsand in different cultures. Archives of Gen-eral Psychiatry, 1989, 45:1069-1077.

Rosenberg, M. Society and the AdolescentSelf-Image. Princeton, N.J.: Princeton Uni-versity Press, 1965.

Rush, B.R., & Krywonis, M. Evaluation andcontinuous quality improvement of sub-stance abuse services. Unpublished paperavailable from Addiction Research Foundation,U.W.O. Research Park, 100 Collip Circle,London, Ontario N6C 4X8, 1996.

Sapnier, G.B. Measuring dyadic adjustment.New scales for assessing the quality ofmarriage and similar dyads. Journal of Mar-riage and the Family, 1976, 38:15-28.

Skinner, H.A., Lifetime Drinking History(Manual). Toronto: Addiction ResearchFoundation, 1979.

Skinner, H.A. The Drug Abuse Screening Test.Addictive Behaviors, 1982, 7, 363-371.

Skinner, H.A., Steinhauer, P.O., & Santa-Barbara, J. The family assessment measure.Canadian Journal of Community Mentalhealth, 1983, 2(2), 91-105.

Sobell, L.C., & Sobell, M.B. Time-linefollowback: A technique for assessing self-reported alcohol consumption. In: R. Litten& J. Allen (Eds.). Measuring alcohol con-sumption: Psychosocial and biologicalmethods .New Jersey: Humana Press,1992:41-72.

Spielberger, C.D., Gorsuch, R.L., &Lushene, R.E. .Manual for the State-TraitAnxiety Inventory. Palo Alto, CA: Consult-ing Psychologists Press, 1970.

Weisner, C., Greenfield, T., & Room, R.Trends in the treatment of alcohol problemsin the U.S. general population, 1979-1980.American Journal of Public Health, 1995,85, 55-60.

Wild, C. Treatment Entry Questionnaire.Unpublished Instrument available from Dr.Cam Wild, Addiction Research Foundation,U.W.O. Research Park, 100 Collip Circle,London, Ontario, N6C 4X8, 1996.

Witchen, H.U. Reliability and validity stud-ies of the WHO-Composite InternationalDiagnostic Interview (CIDI): A critical re-view. Psychiatric Research, 1994, 28, 57-84.

World Health Organization. Programme onSubstance Abuse , AUDIT The AlcoholUseDisorders Identification Test: Guidelinesfor Use in Primary Health Care (WHO/PSA/92.4). Geneva: Workd Health Organi-zation, 1992.

World Health Organization Programme onSubstance Abuse, Assessing the standardsof care in substance abuse treatment(WHO/PSA/93.5). Geneva: World HealthOrganization, 1993.


WHO/MSD/MSB 00.2e

Appendix

Psychoactive substance use

SubstanceUsed in past12 months(1= Yes / 2= No)

Number of daysused in past 90(Days)

Average quantityper day of use inpast 90 days*

Use currently aproblem?(1= Yes / 2 = No)

* It may be difficult to quantify the exact amount for certain substances. Indirect estimates can be made from the number of times per day a substance is injected, inhaled, snorted, or smoked.

Alcohol (beer, liquor, wine)

Cocaine/ crack/ coke

Amphetamines/ other stimulants

Cannabis (hash, weed, grass, pot, marijuana)

Benzodiazepines

Barbiturates

Heroin/ opium

Prescription opioids

Over-the-counter codeine preparations

Hallucinogens

Glue/ other inhalants

Tobacco

Other psychoactive substances

Instruments in Appendix 1 are adaptedfrom a data collection protocol for treat-ment process and outcome monitoring be-ing developed by the Addiction ResearchFoundation, Ontario, Canada. Informationabout these instruments can be obtainedfrom: Addiction Research Foundation, 100Collip Circle, Suite 200, London, Ontario,Canada, N6G 4X8.

In addition to considering these instru-ments, a review of the ARF OutcomeMeasures Directory (undated) is highlyrecommended. This Directory containsmany potentially useful instruments forprocess evaluation and discusses reli-ability, validity, and practical issues inadministration.


WHO/MSD/MSB 00.2e

i) During the past 90 days, on how many days did you injectany kind of psychoactive substance?

ii) Have you ever shared a needle, syringe, cooker/spoon orcotton/filter with anyone at any time in your life?

If Yes, during the past 90 days, on how many days did you sharea needle, syringe, cooker/spoon or cotton/filter with anyone?

During the past 90 days, with how many people have youshared?

Risk behaviour

1 Thinking about your use of psychoactive substances, have you:

Never injected Injected prior to one year ago

Injected in the last 12 months Unknown

If ever injected, answer the following questions:

Health and correctional service utilisation

1 Thinking about physical health problems, during the past 90 days, how many:

• times have you had to go to the emergency room times

• nights total did you spend in the hospital nights

• times did you have an outpatient surgical procedure times

• times did you see a doctor in an office or outpatient clinic times

2 a) Thinking about mental health problems, during the past 90 days, how many:

• times have you had to go to the emergency room times

• nights total did you spend in the hospital nights

• times did you see a doctor in an office or outpatient clinic times

days

days

people

2 How often do you use condoms with your sexual partner or partners?

Never Sometimes Always

During the past 90 days, how many times have you hadunprotected sex?

times

3 During the past 90 days, on how many days have you drivena motor vehicle or used a machine at the workplace whileunder the influence of alcohol or other psychoactive sub-stances?

days

Yes No No response


WHO/MSD/MSB 00.2e

b)Are you currently in any type of treatment or counselling for mental or emotionalproblems?

3 Over the last 90 days, how many days have you received alcohol or substance usetreatment at the following places?

• a hospital overnight for withdrawal or related problems days

• an inpatient substance use treatment facility (3 -90 days) days

• a long-term (3 to 12 months) residential program ortherapeutic community for substance use disorder treatment days

• a methadone or other opioid treatment program days

• an assessment or outpatient substance use treatment facility sessions

• a mental health centre or facility as an outpatient sessions

• an employee assistance program sessions

• a family and/or marital counselling service sessions

• an emergency room days

• a private doctor’s office visits

• a prison or jail days

• some other place (please describe __________________ ) days

Yes No No response

4 a) How many self-help meetings, (e.g., AA, NA, ACOA)have you attended for your substance use problem in the past90 days?

b) How many self-help meetings have you attended for is-sues other than substance use problems in the past 90 days?

meetings

meetings

5 a) During the past 90 days, how many days have you been on probation or parole orbeen in jail or custody?

• Probation days

• Parole days

• Jail/prison/closed custody days

• Open custody days

b) During the past 90 days, how many times have you beencharged for breaking the law (please do not count minor trafficviolations)?

times


WHO/MSD/MSB 00.2e

Were you charged with:

• driving while impaired

• drunkenness or other liquor law violation

• possession, distribution, or sale of illegal substances

• sexual assault

• theft (including B&E, theft over and theft under)

• violence against family or others

• major crime

• Other (please describe __________________________ )

Pleasecheck if

Yes

# of chargesin the last

90 days

Client Motivation

Treatment entry questionnaire:Use the following scale to make your ratings

INSTRUCTIONS:Please indicatewhether you agreeor disagree witheach of thefollowingstatements byplacing thenumber that bestreflects your ownpersonal opinionin the blankprovided.Remember, thereare no right orwrong answers,and yourresponses arecompletelyconfidential.

StronglyDisagree

StronglyAgree

1 If I remain in treatment it will probably be because I feel thatit’s the best way to help myself.

2 I plan to go through with a treatment program because I’ll hatemyself if I don’t get my habit under control.

3 I have agreed to follow a treatment program because I wasreferred for treatment by the legal system.

4 I plan to go through with a treatment program because it’s achallenge to learn how to live without misusing psychoactivesubstances.

5 I plan to go through with a treatment program because myfriends and family won't approve of me unless I do.

6 Being in a program is a way for me to avoid getting punishedfor my behaviours.

7 I decided to enter a program because I was interested in get-ting help.

8 I decided to enter a program because I won't like myself verymuch unless my substance use problem is under control.

9 I had no choice about coming into a treatment program.

10 I plan to go through with the treatment program because hav-ing a substance use problem makes it hard for me to do thingsI want to do.

1 2 3 4 5 6 7

1 2 3 4 5 6 7

1 2 3 4 5 6 7

1 2 3 4 5 6 7

1 2 3 4 5 6 7

1 2 3 4 5 6 7

1 2 3 4 5 6 7

1 2 3 4 5 6 7

1 2 3 4 5 6 7

1 2 3 4 5 6 7


WHO/MSD/MSB 00.2e

11 My family made sure that I entered a program.

12 If I remain in treatment it will probably be because others willbe angry with me if I don’t.

13 I decided to enter a program because I really want to makesome changes in my life.

14 I have agreed to follow a program because I want others to seethat I am really trying deal with my habit.

15 I plan to go through with treatment because I’ll be ashamed ofmyself if I don’t.

16 I decided to enter this program because no one other than my-self can change the way I am.

17 The reason I am in treatment is because other people havepressured me into being here.

18 If I remain in treatment it will probably be because I’ll feel likea failure if I don’t.

19 I plan to go through with a treatment program because I’ll getinto trouble with the law if I don’t remain in treatment.

20 I plan to go through with a treatment program because I havefreely chosen to be here.

21 If I remain in treatment it will probably be because people willthink I’m a weak person if I don’t.

22 I decided to enter a program because it feels important for mepersonally to deal with my substance use problem.

23 I have agreed to follow a treatment program because I’ll get in troublewith my friends and family if I don’t follow all the guidelines.

24 I plan to go through with a treatment program because not hav-ing problems due to substances is a choice I really want to make.

25 My friends strongly pressured me to come into a program.

26 If I remain in treatment it will probably be because I’ll feel verybad about myself if I don’t.

27 I have agreed to follow the procedures of the treatment programbecause it’s a personal challenge for me to deal with my problem.

28 I have agreed to follow a treatment program because I waspressured to come.

29 I decided to enter a program because people will like me betterwhen I have dealt with my habit.

30 I was basically forced into a treatment program.

1 2 3 4 5 6 7

1 2 3 4 5 6 7

1 2 3 4 5 6 7

1 2 3 4 5 6 7

1 2 3 4 5 6 7

1 2 3 4 5 6 7

1 2 3 4 5 6 7

1 2 3 4 5 6 7

1 2 3 4 5 6 7

1 2 3 4 5 6 7

1 2 3 4 5 6 7

1 2 3 4 5 6 7

1 2 3 4 5 6 7

1 2 3 4 5 6 7

1 2 3 4 5 6 7

1 2 3 4 5 6 7

1 2 3 4 5 6 7

1 2 3 4 5 6 7

1 2 3 4 5 6 7

1 2 3 4 5 6 7

StronglyDisagree

StronglyAgree


WHO/MSD/MSB 00.2e

Whereas the prior cases in this workbookpresented evaluations related to either cov-erage or process, the following two casesincorporate questions related to both cover-age and process.

The first case example describes the evalua-tion of an alcohol home detoxification ser-vice in the United Kingdom. The evaluatorwanted to know about coverage issues, suchas demographic and clinical characteristicsof clients, and also wanted to know aboutprocess issues, such as number of completeddetoxifications, client workloads for nursepractitioners, and average mileage accruedwhile driving to see clients. Aspects of costevaluation (Workbook 5) and client satisfac-tion evaluation (Workbook 6) also were in-cluded. Of note, the case author/evaluatordescribes procedures for developing a newcomputer-based client tracking system. Withno prior experience with the software, thistask was completed in less than 24 hours.Evaluation efforts were worthwhile, as re-

sults were used to increase awareness of thedetoxification programme and justify furtherfunding.

The second case describes preliminary re-sults from a Swiss evaluation of the medicalprescription of narcotics for heroin-depen-dent people. Evaluators used a variety of datacollection methods, including treatmentprogramme records, patient interviews, andthird party information. They assessed cov-erage issues such as participant characteris-tics, and also process issues such as adher-ence to treatment and safety of the narcoticprescriptions.

It is noteworthy that neither of these casesfocused upon treatment efficacy or outcome.Rather, the main questions centered aroundthe process of treatment itself. Evaluationsof this type are important to establish confi-dence that treatment is serving the intendedclients and being conducted in the desiredmanner.

Comments aboutcase examples


WHO/MSD/MSB 00.2e

The author alone isresponsible for theviews expressed in thiscase example.

Process evaluation of alcohol homedetoxification and assessment

by David B. Cooper, RN, FETCParkholme, Ashreigney, Chulmleigh, Devon EX18 7LYHalls Mill Lane, Bradiford, Barnstaple, Devon EX31 4DPTelephone/Facsmilie +44.0.1769.520 469

Who was asking thequestion(s) and whydid they want theinformation?

Alcohol home detoxification and assessment(AHDA) (or community detoxification) whilstnot new, is a recent, expanding developmentin community service provision. A fundamen-tal concept is that of thorough assessment ofthose individuals referred for treatment dur-ing alcohol withdrawal. An holistic and eclec-tic assessment addresses two key questions(Stockwell 1987):

• Is there a need for any medication to al-leviate withdrawal symptoms?IF YES:

• Are there any reasons for not keeping theindividual within the home environment fordetoxification?

Formally, the role of developing AHDAservices lay with the statutory sector (Na-tional Health Service - NHS). However,over the last seven years, specialist volun-tary sector services have looked towards

employing a qualified nurse(s) to developand evaluate AHDA to:

• compliment the existing counselling ser-vices;

• provide an alternative to inpatient care;

• ensure that those who need inpatient careduring alcohol withdrawal receive theappropriate intervention; those who needclinical supervision but do not require in-tensive inpatient supervision during alco-hol withdrawal can do so within the homeenvironment safely; and of significant im-portance, those who do not need medi-cation or supervision during alcohol with-drawal receive the right level ofintervention from appropriate sourceswithout inappropriate use of limited re-sources.

This case study looks at the processes ofevaluation used by a voluntary sector ser-vice (Suffolk Community Alcohol Service(SCAS)) during a three year AHDA pilotproject (January 1994-December 1996).The case study will also briefly discuss thevalue of the Advantage SM database inAHDA evaluation (see Appendix 7).

Case example of aprocess evaluation


WHO/MSD/MSB 00.2e

About SCAS

SCAS functions as a central source of infor-mation, advice, training and education, coun-selling, organisation and collaborative workin the field of alcohol use throughout thecounty of Suffolk. The predominant part ofthe service facility is in East Suffolk. It be-came apparent in 1992 that during the sixyears of operation, SCAS had seen an in-crease of 296% in requests for help, adviceand intervention. One consequence of thedeveloping referral rate was that agency staffwere increasingly being called upon to pro-vide an alternative to inpatient detoxificationfor problem drinkers requiring short-term,intensive, specialist and clinical intervention(Cooper, 1992). In response to the increasein referrals, and the identified gap in serviceprovision, SCAS proposed to establish athree year project to develop and assess theneed for an alternative to inpatient detoxifi-cation (i.e., a AHDA service).

It was suggested that a Registered Nurse,specialising in home detoxification, could pro-vide a cost effective alternative to in-patientcare, and that the service would go some waytowards meeting the present shortfall in ser-vice provision for problem drinkers in EastSuffolk. A funding proposal was submitted,and approved, for joint social services andhealth authority funding for the AHDA EastSuffolk project. SCAS provided, and man-aged, the AHDA project. The initiative camefrom the director, Mary Jeffries.

The alcohol home detoxification and assess-ment (AHAD) project accepted referral fromany source, including self. The primary crite-ria for referral were that the individual re-quired detoxification. The hypothesis wasthat:

• many individuals requiring detoxificationwere admitted to hospital unnecessarilyfor clinical supervision during the with-drawal period;

• those who were detoxicated at homewere more likely to complete the detoxi-fication;

• those who were detoxicated at homewere satisfied with the service provided;

• many of the referrals received did not re-quire inpatient or home detoxification.

• evaluation of the outcome of each refer-ral and intervention would give some in-dication of the impact on other services,in particular the SCAS counselling ser-vice;

• a period of 6, 12 and 24 month follow-up would be beneficial in terms of servicedevelopment. However, it was agreed thatthis information would not have any sig-nificant bearing on the effectiveness of al-cohol home detoxification. It is generallyacknowledged that other interventionswould have a significant impact on longterm outcome and that clinically super-vised detoxification is a small (rather thanthe whole) part of any treatment package(Cooper, 1994).

What resources wereneeded to collect andinterpret theinformation?

In order to assess the effectiveness of theservice, SCAS needed to collect data ca-pable of providing some answers to the pre-viously mentioned issues and questions. Anurse practitioner (Registered Nurse) wasappointed for the project whose areas ofresponsibility were:

• assessment and provision of clinical careto the client group;

• evaluation of the project; and,

• communication and public relations dur-ing project development.

It was agreed that two forms of data collec-tion were required: (a) the means to collectindividual data from the client, supporter andother professional and services involved inthe care on an individual referral basis; (b)the means to bring together the data for pur-poses of collation and evaluation.


WHO/MSD/MSB 00.2e

It was intended that the Nurse Practitioner(NP) would collect the data on an ongoingbasis. The NP would have the support of apart-time (20 hours per week) Administra-tive / Research Assistant. At the end of theproject, three months had been set aside forthe NP to analyse and report the data. Datawas in fact collected by the NP, but not sys-tematically collated; hence, the subsequenthiring of this author to collage, analyse andreport available data which was then recycledinto the SCAS report quoted.

The work took eight very long working weeks(seven days per week)! It still is believed, aswas recommended in the initial proposal forSCAS funding (prepared by this author), thatthe Nurse Practitioner and one part-time (20hours per week) Administrative / ResearchAssistant and a computer with a compatibleprogramme, could undertake this project.

How were the datacollected?

Individual client record

The Individual Client Record (appendix 1)was a hand-completed from, which was thefirst stage in the tracking system of all refer-rals to the project. Each entry correspondedto the database entry used for the collationand evaluation of the data. As a numericalsystem was felt to be the most appropriate,a separate score sheet was produced as anaid to memory, on which the database wouldbe designed (see appendix 2 for detaileddescription of coding).

Information recorded on the AHDA indi-vidual client record was entered onto theAccess database manually using the AHDAcode sheet to change responses into a nu-merical equivalent. For example, the sourceof referral could be a general practitioner(GP), coded as '1,' or a community psychi-atric nurse (CPN), coded as '4.' A separaterecord of actual client contact was recordedin a diary and hand collated to allow forcross-reference.

Tracking system

The major part of the information requiredto monitor the effectiveness of the projectwas the tracking system. It was decided touse the Access database system, which formspart of the Microsoft Office Professionalsoftware package.

Client awareness

It was agreed that both the client and sup-porter should be fully informed of the pur-pose of the project, and that as part of theassessment process a full explanation of thepurpose of the project would be given. As-surances relating to confidentiality of infor-mation collected for evaluation were made.It was explained that a refusal to allow suchinformation to be passed on would NOTexclude the individual from treatment. Theopportunity to decline or withdraw permis-sion was also given at any time. No one re-fused permission.

Data collection

The nurse practitioner collected data fromvarious sources. These data, together withthe data from the client and supporter satis-faction questionnaire and GP questionnaire,was hand recorded onto the database by theauthor. Using a combination of computeraided collation, and simple hand analysis, acrude picture could be drawn as to the ef-fectiveness of the AHDA project.

Client and supportersatisfaction scale

Recognised client and supporter satisfactionscales are already available and have estab-lished validity (Stockwell et al, 1990 - Ap-pendix 3 & 4). Some minor modifications toreplace the sliding scale with a numerical sys-tem for ease of evaluation were used.

Home detoxification follow-up: The HomeDetoxification Follow-up from consisted offive active parts (appendix 5). The follow-


WHO/MSD/MSB 00.2e

up agreed upon for the 6, 12, 24 months fol-lowing detoxification was not designed toassess the success of the detoxification it-self. It was introduced as an attempt to see ifinterventions following detoxification pro-duced any significant changes in drinkingpattern. It was merely a “look and see”exercise which may or may not assist in fu-ture service planning.

The data collected on the client (appendix3) and supporter (appendix 4) satisfactionscale was first entered on a specially designedAccess database using the numerical codingon the forms. The select button was used todraw out the relevant section of data whichwas hand collated using percentage andsimple numerical comparison. The sectionsallowing for free comment were recorded infull with some minor discussion of content.

In order to balance out consistently high lev-els of satisfaction recorded in different set-tings, the neutral responses were regardedas possibly negative and therefore, positiveratings of below 75% was regarded as causefor concern (Pelletier 1985).

The scale would be applied only to those whocompleted a detoxification in the home. It wasacknowledged that during a three-year pe-riod some individuals might require more thanone detoxification. Therefore, only one ques-tionnaire would be sent to the client and thesupporter. All questionnaires would be sentat the end of the project in three waves.

Wave 1: A pre-coded questionnaire wassent individually by the nurse practitioner toeach client and supporter. For those who hadreceived more than one detoxification, a para-graph in the letter requested that the individualbase the response on the last detoxificationoccasion. The letter also reaffirmed the pur-pose of the project and thanked the individualfor his or her assistance. A stamped ad-dressed envelope was included - addressedto the nurse practitioner. Returned question-naires were logged on the 'Individual Cli-

ent Record' (appendix 1). Those individualsrefusing to complete the from were loggedaccordingly and no further contact was made.

Wave 2: Two weeks after the closing datefor return of Wave 1 questionnaire, a sec-ond questionnaire was sent to non-respon-dents. Again the accompanying letter re-freshed the reader to the purpose of theproject and asked for assistance. This lettercarried a paragraph that stated that as partof the survey process those who had not re-sponded by the deadline date would receivea telephone call from the nurse practitioner.If the client or supporter did not wish to re-ceive such contact, it was requested s/he re-turned the blank form in the reply paid enve-lope and no further contact would be made.The 'Individual Client Record' (appendix1) was updated.

Wave 3: Two weeks following the dead-line, non-respondents received a follow-upcall from the nurse practitioner. This was un-dertaken in a low-key fashion. It was felt im-portant that the individual did not feel pres-sured. The approach was that if there wassomething wrong, it was important to us. The'Individual Client Record' (appendix 1) wasupdated, and completed.

The home detoxificationgeneral practitioner form

The Home Detoxification General Practitio-ner Questionnaire (appendix 6) was origi-nally designed by Kaner & Masterson(1996) and was used with minor adaptationsfor local needs. Using a separate Accessdatabase, and the YES = 1, NO =2 numeri-cal system, it was possible to hand draw in-formation from the computer for numericaland percentage analysis. The free commentswere reproduced in full. The questionnairewas sent to every GP in the East Suffolk areaand consisted primarily of Yes or No typeresponses (later converted to numericalequivalent for ease of collation).


WHO/MSD/MSB 00.2e

How were the dataanalysed?

Operational activity

One area of interest to the service managerswas how many actual clinical working hourswere available to the nurse practitioner. Fromthe gross operational days of the project, thefollowing were deducted to produce a “clini-cal operational time”: annual leave; bankholidays; sickness; training sessions attendedas a student; training sessions given to oth-ers; interim report writing; crown court at-tendance; clinical supervision; professionalpractice and team meetings; managementsupervision.

Most of this was taken from time sheets, thenurse practitioner’s personal profile and di-ary, and the clinical supervisor. Compilationwas by pen and paper. By deduction of thisdata to establish a “clinical operationaltime,” it was possible to ascertain how manyindividuals received assessment, and detoxi-fication supervision within the clinical opera-tional time frame.

Mileage

East Suffolk covers a wide area. As such, itwas felt to be of value if some emphasis wasgiven to travel time as this does have an ef-fect on the number of client contacts. Thiswas completed at the end of the project byusing the official monthly mileage returns.From these it was possible to establish: thetotal mileage; the average mileage per day;the longest day trip; the training mileage andthe average trip.

Client contact

From the individual clinical records it waspossible to establish the number of individualclient contacts (how many times was an in-dividual seen in total) made during the project.From this it was possible to calculate theaverage daily, weekly and monthly contactrate. Unfortunately this information only in-

cluded direct client contact and not indirectcontact with the supporter, the referringagent, by telephone or activities involving mail,etc. This was a shortcoming in the planningprocess of evaluation.

Cost

A primary concern to all funders and serviceproviders is “How much does it cost?” and“Is it cost effective?”. Interestingly, it was notpossible to establish exactly how much itwould cost to keep a patient for a 24-hourperiod on an acute mental health ward, thisbeing the usual area for in-patient detoxifi-cation in the UK from the statutory sector -NHS. So it was agreed to use the bed occu-pancy figure established in the project pro-posal, on which the 3 year funding was based,and compare this with the total funding costas at the project proposal, thus disallowingany built-in inflationary costs (Cooper, 1992).

Using the total completed detoxifications andthe average duration of the home detoxifica-tion, with the average in-patient detoxifica-tion suggested by Cooper (1985, 1988), itwas possible to provide a daily cost, and anindividual client cost, for both in-patient andhome detoxification and compare these.

Tracking system

As mentioned earlier, it was decided to usethe Access database system, which formspart of the Microsoft Office Professionalsoftware package, because the design tech-niques were easy to follow. Individual fieldswere created on the database to correspondwith the entries on the “Individual ClientRecord.”

Whilst it is possible to do some specific datacollection with this package and with the sis-ter program Excel, computer competencewas limited in such techniques. The Accessdatabase allowed for simple numerical dataentry, and by using the “button” filter system,supported by hand calculation, to bring to-gether the essential data, it was possible toeffectively evaluate the collated data. The


WHO/MSD/MSB 00.2e

author, with no previous experience of thesystem, developed the database within 24hours. Using this system, the following datawere analysed:

• how many collective referrals were re-ceived;

• who made the referral;• client gender;• age range of clients;• employment status of the clients;• number of referral occasions the individual

had;• number of referrals received weekly and

any increase or decrease;• number of referrals received monthly and

any increase or decrease;• number of the referrals leading to full as-

sessment;• number of assessments undertaken

weekly;• number of assessments undertaken

monthly;• reasons for no assessment;• number of referrals leading to detoxifica-

tion;• number of completed detoxifications com-

mencing weekly;• number of completed detoxifications com-

mencing monthly;• number of completed detoxifications;• how long did (the completed) detoxifica-

tion last and how much did it cost;• reasons for no-completion of detoxifica-

tion;• due dates of the 6, 12 and 24 month fol-

low-ups and completed or reasons whythey had not been completed;

• completion dates for the client and sup-porter satisfaction scales for Waves, 1, 2and 3, how many were completed, andat what wave did we get the response;

• the outcome of each referral or reasonwhy we did not have a known outcome;

• the impact on other services as a result ofreferrals, assessment or detoxification;

• additional space was provided for anynotes for information considered of value.

Client and supportersatisfaction scales

The client and supporter satisfaction scaledata were compiled along with data from theIndividual Client Record (e.g., age), andthe nurse practitioners clinical notes (e.g.,supporter relationship and sex), using a sepa-rate Access database within Microsoft Of-fice Professional. Using a numerical system,responses to each questionnaire was enteredon the database.


The AHDA project had a total of 650 op-erational days (130 working weeks) of which480 days (74%) were available for clinicaloperation. Forty-nine days (7.5%) werespent on clinical and managerial supervisionand team meetings and 25 days (4%) dividedevenly on education given and received bythe Nurse Practitioner (Ayers, 1997).

Mileage accounted for some 19,334 miles,a mean of 36.8 miles per day, for the 27months of available data. Given the size ofthe area, this figure was considered reason-able (Ayers, 1997).

As noted above, it was not possible to ob-tain an accurate cost for one, 24 hour, inpa-tient stay from the acute mental health unit.However, by using an agreed upon 1993 fig-ure (£173 per day), it was possible to pro-vide some crude cost comparisons. Of the79 completed home detoxification stays av-eraging eight days, the mean cost per clientwas £1,474.93 or £184.36 per day. Theresults suggested that AHDA was slightlymore expensive than the 1993 cost of £173per day for in-patient detoxification. How-ever, in terms of effectiveness, data collectedenabled the project to claim:

• that 90% of those individuals who requiredmedically supervised detoxification did notprogress to in-patient care. The figuresavailable for inpatient bed occupancy


WHO/MSD/MSB 00.2e

during the AHDA project indicated agradual decline in demand;

• that less than 40% of those referred fordetoxification, required a medically super-vised detoxification;

• that home detoxification was more likelyto be completed in comparison to inpa-tient detoxification;

• the average detoxification at home lasted8 days in comparison to 12 days in-pa-tient detoxification (Ayers, 1997).

The true value of any service lies in the satis-faction of the client and supporter. Forty-five(57%) of the 79 individuals who completeddetoxification responded to the client satis-faction scale. Twenty-eight (35%) of sup-porters responded to the supporter satisfac-tion scale. The results suggested that overall,78% of clients and supporters were satisfiedwith the home detoxification. Whilst therewere some minimal comments in terms ofover-or-under sedation, the level of dissat-isfaction with medication was minimal. Ingeneral, it was the supporter who felt lesssupported and it has been agreed that infor-mation given to the supporter needs to bemore overt in any future service development(Ayers, 1997).

The home detoxification follow-up ques-tionnaire had some design faults. For ex-ample, a future questionnaire would needclarification that “supporter” referred toother professional or agency and not thefamily member or friend. The form alsofailed to account for changes in the clienttreatment goal, (e.g., whilst the individualmay have stated total abstinence as thegoal, during the 12 or 24 months, s/he mayhave commenced drinking at a sociallyacceptable level). However, the responseon the questionnaire could not distinguishbetween trouble-free alcohol consumptionand relapse.

These difficulties not withstanding, the datasuggested that those individuals who hadcompleted detoxification at home, and hadfollow-up counselling from a SCAS coun-sellor appeared to maintain their chosen treat-ment goal longer than those with other pro-fessional support, or no intervention.However, this area would need a more sci-entific analysis before any significant claimscould be made.

Seventy-seven percent (n=14) of rural and85% (n=23) urban practices responded tothe GP questionnaire, involving 47% (n=33)rural and 61% (n=75) individual GP’s. Over-all, 58% (n=60) had reported carrying outdetoxifications at home, of which 45 GP’swere in rural practices. Sixty percent of re-spondents had received a visit from the nursepractitioner of which 46% had referred oneor more client to the nurse practitioner incomparison to the 20% from those GP’s whohad not received a nurse practitioner visit(Ayers, 1997).

There were many accompanying letters andcalls offering support for the project, andrequesting personal contact. The question-naire also acted as additional promotionalmaterial for the service project, promptingan increase in referrals from those who hadpreviously not used the service. The dataseemed to suggest that those GP’s were theclient outcome had been successful in termsof completion and reduced GP contact weremore inclined to feel the value of the servicein comparison to those GP’s who had anunsuccessful detoxification referral. Overall,67% of GP’s were satisfied with the service.From the “comment” received, the informa-tion suggested a need for a concerted train-ing programme for GP’s in terms of the na-ture of alcohol related problems, and clientexpectation. Some GP’s quoted a lack ofmotivation from the client, pressure of workand time, cost implications, and lack ofknowledge as reasons for not engaging thisclient group.


WHO/MSD/MSB 00.2e

How were the resultsused?

It is not possible nor practical to detail all theresults from the AHDA project. Much of thedata is confidential forming part of a reportto the management committee. In today’shighly competitive market within the UKNHS, such data is a valuable source of sup-port needed to secure funding for subsequentnext years.

As a result of the service evaluation someareas involving service development havebecome clear, and include:

• An increased level of public relations andtraining activity with the GP. Of the 267referrals received, the Nurse Practitionerassessed 220 individuals, of which 117were first time referral and 38 second. 87were from GP’s and 72 from the SCAScounselling service (Ayers, 1997). Manyof the referrals would have been moreappropriate, had the referring agent hadmore knowledge in the assessment ofclinical intervention during detoxification.It is possible, that the large amount of re-ferrals from the GP included some thatwere a means of side stepping the localhospital procedure.

• Waiting time was felt to be reasonablewith 40% of individuals being seen within48 hours. Some modification to the initialassessment intervention has been madeby SCAS to improve on this figure.

• The AHDA has been funded for a further18 months, whilst a review of all serviceprovision for substance misusers in Suf-folk takes place. The project will continuewith an ongoing evaluation.

• More attention to the supporters’ indi-vidual needs will be given on each visitingoccasion.

• All referrals to the AHDA project will havean initial telephone inquiry to aid filteringof inappropriate referrals. A full assess-ment would only be completed if therewere strong indications that home detoxi-fication was indicated. A trained admin-istrative assistant, to free up the NursePractitioner’s time would complete thefollow-up questionnaire. The Nurse Prac-titioner would increase the role of con-sultant supporting and encouraging pro-fessionals wishing to supervise homedetoxification. All clients would receiveat least one daily visit, with clear writtenjustification should this not be appropri-ate. A client checklist at salient points indetoxification, and care plan, would bemaintained at the client home. More at-tention to the client’s perceived level ofcomfort during detoxification would begiven.

• A GP satisfaction questionnaire to becompleted following each detoxificationmay be introduced.

This case study is believed to be representa-tive of the type of information one could ex-pect from a small, voluntary sector service.The SCAS evaluation aimed to provide suf-ficient information to satisfy the funding agentwith a view to establishing continued finan-cial support. It is sufficient to expect one ortwo members of staff to develop and main-tain this information whilst undertaking clini-cal duties.

Acknowledgement

I am grateful for the co-operation received from the SCAS throughout the development of this case study.I am especially grateful to Ms Mary Jeffries, Director, SCAS, and Mr Colin Ayers, Nurse Practitionerwithout whose support the above would not have been completed. Also to Jo, who acted as proof-reader andsound board. Any errors and or omissions are the sole responsibility of the author.


WHO/MSD/MSB 00.2e

References for case example

Ayers C. Suffolk Community Alcohol Ser-vice (SCAS): Home DetoxificationProject. Suffolk: SCAS, 1997.

British Medical Association and the RoyalPharmaceutical Society. British NationalFormulary March, number 29. London:BMA, 1995.

Committee on Safety of Medicines. Factualinteractions between heminevrin and al-cohol. Current Problems nº 20, Committeeon Safety of Medicines, Department ofHealth, UK, 1987.

Cooper, D. B. The effects of a communityalcohol service, the hidden ‘potentialdrinking problems’ and problem drinking.Report to the District Management Team,Blackburn, Hyndburn and Ribble ValleyHealth Authority (unpublished), 1985.

Cooper, D. B. Ward population survey:alcohol related bed occupancy. Report tothe Assistance Unit General Manager, NorthEast Essex Health Authority, Mental HealthDivision (unpublished), 1988.

Cooper, D. B. A holistic approach to prob-lem drinking in East Suffolk: home detoxi-fication and SCAS. A joint funding proposal.Suffolk Community Alcohol Service (unpub-lished), 1992.

Cooper, D. B. Alcohol home detoxifica-tion and assessment. Oxford: RadcliffeMedical Press, 1994.

Kaner, E. F. S. & Masterson, B. The role ofgeneral practitioners treating alcohol depen-dent patients in the community. Journal ofSubstance Misuse, 1996. 1(3):132-136.

Pelletier M. Client variables to watch out for.Dimensions, 1985, 62: 37-39.

Stockwell, T. The Exeter home detoxifica-tion project. In: Stockwell, T. & Clement, S.Helping the problem drinker: a new ini-tiative in community care. London: CroomHelm, 1987.

Stockwell, T. Bolt, L. Milner, I. et al. Homedetoxification for problem drinkers: accept-ability to clients, relatives, general practitio-ners and the outcome after 60 days. BritishJournal of Addiction, 1990, 85(1):61-70.

Yates, F. E. IT Column - Can your servicelive with a fussy manager? Journal of Sub-stance Misuse, 1996, 1(4): 181-182.


WHO/MSD/MSB 00.2e

It�s your turnWhat are the strengths and the weaknesses of the presented case example? List three posi-tive aspect and three negative aspects:

Strengths of the case study

1

2

3

Weaknesses of the case study

1

2

3


WHO/MSD/MSB 00.2e

Case example appendix 1

AHDA Individual Client RecordNote ALL sections MUST be completed

File Complete? Yes No

File Dead? Yes No

Code

HEADING ENTRY HEADING ENTRY

Source CssOcc1

Sex CssOcc1

Age DateCssOcc1

Employment Status Occ1Rtn

Referral Occ1RtnDate

WkNoRef State CssOcc1

MthNoRef CssOcc2

Assessed DateCssOcc2

AssessDate Occ2Rtn

WkNoAss Occ2RtnDate

MthNoAss State CssOcc2

1 State CssOcc3

Detoxed DateCssOcc3

DetoxDate Occ3Rtn

WkNoDtx Occ3RtnDate

MthNoDtx State CssOcc3

DetoxEnd Outcome

DetoxLength SssOcc1

2 State DateSssOcc1

FUDue6/12 SupOcc1Rtn

FUDone6/12 SupOcc1RtnDate

DateFU6/12 State SssOcc1

State 6/12 SssOcc2

FUDue12/12 DateSssOcc2

FUDone12/12 SupOcc2Rtn

DateFU12/12 SupOcc2RtnDate

State 12/12 State SssOcc2

FUDue24/24 SssOcc3

FUDone24/24 DateSssOcc3

DateFU24/24 SupOcc3Rtn

State 24/24 SupOcc3RtnDate

State SssOcc3


WHO/MSD/MSB 00.2e

HEADING MEANING CODE

Total Total number of referral occasions AutoNumberCode Client Number Enter Client NumberSource Who made the referral 1 GP

2 SCAS Counsellor3 Self4 CPN5 SHO’s6 Relative/Partner7 SSD (SW)8 Probation9 CDT10 Age Concern11 Employer12 Psychiatrist13 Registrar14 Warden15 General Hospital20 Unknown

Episode Is this the 1st, 2nd, 3rd etc., referral Number each referral - 1, 2, 3, etc.,Sex Client Gender 1 Male

2 FemaleAge Clients Age Enter age

11 UnknownEmployment Status Employment Status 1 Employed

2 Unemployed,3 Semi-retired4 Retired,5 Houseperson6 Unknown

Referral Date Referral Received Enter DateWkNoRef In what week number was the referral Enter week number

received, e.g., 1, 20, 52, etc.MthNoRef In what month number was the referral Enter month number

received, e.g., 1, 6, 12, etc.Assessed Was the client assessed? 1 Yes

2 No3 N/A

AssessDate Date of assessment Enter DateWkNoAss In what week number was the assessment Enter week number

received, e.g., 1, 20, 52, etcMthNoAss In what month number was the assessment Enter month number

received, e.g., 1, 6, 12, etc.1 State Why was the referral not assessed? State - use as few words as possible.

This may be coded when we get an idea of typeDetoxed Was the client detoxicated 1 Yes

2 No3 N/A


AHDA Code SheetNote ALL sections MUST be completed


WHO/MSD/MSB 00.2e

DetoxDate Date detoxication commenced Enter DateWkNoDtx In what week number was the detox Enter week number

started, e.g., 1, 20, 52, etcMthNoDtx In what month number was the detox Enter month number

started, e.g., 1, 6, 12, etc.DetoxEnd Date detoxication ended Enter DateDetoxlength For how many days did the detoxication last Enter Number of Days2 State Why did you not detox the client State - use as few words as possible.

This may be coded when we get an idea of typeFUDue6/12 Date the 6 month follow-up due Enter DateFUDone6/12 Has the 6-month follow-up completed? 1 Yes

2 No3 N/A

DateFU6/12 Date the 6 month follow-up completed Enter DateState 6/12 Why the 6 month follow up has not been State - use as few words as possible.

completed This may be coded when we get an idea of typeFUDue12/12 Date the 12 month follow-up due Enter DateFUDone12/12 Has the 12-month follow-up completed? 1 Yes

2 No3 N/A


completed This may be coded when we get an idea of typeFUDue24/24 Date the 24 month follow-up due Enter DateFUDone24/24 Has the 24-month follow-up completed? 1 Yes

2 No3 N/A


completed This may be coded when we get an idea of typeCssOcc1 Was the Client Satisfaction Scale sent 1 Yes

at first posting? 2 No3 N/A

DateCssOcc1 Date the Client Satisfaction Scale was Enter Datesent at first posting?

Occ1Rtn Was the Client Satisfaction Scale 1 Yescompleted at the first posting? 2 No

3 N/AOcc1RtnDate Date the completed Client Satisfaction Enter Date

Scale was returned - first posting?State CssOcc1 Why was the Client Satisfaction Scale State - use as few words as possible.

not completed at first posting? This may be coded when we get an idea of typeCssOcc2 Was the Client Satisfaction Scale sent 1 Yes

at second posting? 2 No3 N/A

DateCssOcc2 Date the Client Satisfaction Scale was Enter Datesent at second posting?

Occ2Rtn Was the Client Satisfaction Scale 1 Yescompleted at the second posting? 2 No

3 N/AOcc2RtnDate Date the completed Client Satisfaction Enter Date

Scale was returned - second posting?State CssOcc2 Why was the Client Satisfaction Scale State - use as few words as possible.

not completed at second posting? This may be coded when we get an idea of typeCssOcc3 Was the client contacted by telephone on 1 Yes

the third occasion following none return on 2 Nosecond occasion of Client Satisfaction Scale? 3 N/A

DateCssOcc3 What date did you make telephone contact? Enter Date



WHO/MSD/MSB 00.2e

Occ3Rtn Did the client return the Client Satisfaction 1 YesScale on the third occasion? 2 No

3 N/AOcc3RtnDate What date was the Client Satisfaction Scale Enter Date

returned on the third occasion?State CssOcc3 Why was the Client Satisfaction Scale State - use as few words as possible.

not completed at third posting? This may be coded when we get an idea of typeOutcome If the client was not detoxicated following 1 Inpatient care East Suffolk

assessment what happened? 2 Inpatient care outside3 Inpatient rehab East Suffolk4 Inpatient rehab outside5 SCAS Counselling6 No further action (NFA) Client request7 Under care of CPN8 Under care of CDT9 Self referral - private care10 Referred on CAT out of area11 Unknown12 Prison

SssOcc1 Was the Supporter Satisfaction Scale 1 Yessent at first posting? 2 No

3 N/ADateSssOcc1 Date the Supporter Satisfaction Scale Enter Date

was sent at first posting?SupOcc1Rtn Was the Supporter Satisfaction Scale 1 Yes

completed at the first posting? 2 No3 N/A

SupOcc1RtnDate Date the completed Supporter Satisfaction Enter DateScale was returned - first posting?

State SssOcc1 Why was the Supporter Satisfaction Scale State - use as few words as possible.not completed at first posting? This may be coded when we get an idea of type

SssOcc2 Was the Supporter Satisfaction Scale 1 Yessent at second posting? 2 No

3 N/ADateSssOcc2 Date the Supporter Satisfaction Scale Enter Date

was sent at second posting?SupOcc2Rtn Was the Supporter Satisfaction Scale 1 Yes

completed at the second posting? 2 No3 N/A

SupOcc2RtnDate Date the completed Supporter Satisfaction Enter DateScale was returned - second posting?

State SssOcc2 Why was the Supporter Satisfaction Scale State - use as few words as possible.not completed at second posting? This may be coded when we get an idea of type

SssOcc3 Was the supporter contacted by telephone 1 Yeson the third occasion following none return 2 Noon second occasion of Supporter 3 N/ASatisfaction Scale?

DateSssOcc3 What date did you make telephone contact? Enter DateSupOcc3Rtn Did the supporter return the Supporter 1 Yes

Satisfaction Scale on the third occasion? 2 No3 N/A

SupOcc3RtnDate What date was the Supporter Satisfaction Enter DateScale returned on the third occasion?

State SssOcc3 Why was the Supporter Satisfaction Scale State - use as few words as possible.not completed at third posting? This may be coded when we get an idea of type

Notes General identifier Additional comment which may assist recall at alater date - use as few words as possible.This may be coded when we get an idea of type



WHO/MSD/MSB 00.2e

1 On the whole, how did you feel during your withdrawal from alcohol?

a) Very comfortable Very uncomfortable

b) Well supported Not well supported

c) Not craving for alcohol Craving for alcohol

d) Very calm Very anxious

e) In control of yourself Controlled by others

f) Very determined Not at all determined

g) Not at all tempted Very tempted to drink

h) Well informed Not at all well informed

2 Was the medication:

a) Too much Too little

b) Too long Too short

3 Were the visits from the nurse practitioner:

Too frequent Not frequent enough

4 During the home detoxification, how helpful were the following:

a) Support from your partner/friend Vital Unhelpful

b) Support from the nurse practitioner Vital Unhelpful

c) Support from other agency Vital Unhelpful

(Please state)

d) Support from your GP Vital Unhelpful

e) Visit from the nurse practitioner Vital Unhelpful

f) Drugs prescribed by your GP Vital Unhelpful

g) The physical check up Vital Unhelpful

h) The breath analyser Vital Unhelpful

5 What did you like most, or find the most helpful about the home detoxification procedure?

____________________________________________________________________________________________________

____________________________________________________________________________________________________

6 What did you like the least, or find the least helpful about the home detoxification procedure?

____________________________________________________________________________________________________

____________________________________________________________________________________________________

Thank you for completing this form. Please check that you have answered ALL the questions


Confidential Client Satisfaction ScaleDate: _______________________________________Code:______________________________________________

These questions are designed to help us evaluate the detoxication procedure and its? effectiveness. Please circle ONE number,per question, that is closest to your feelings.

Please answer ALL the questions

5 4 3 2 1

5 4 3 2 1

5 4 3 2 1

5 4 3 2 1

5 4 3 2 1

5 4 3 2 1

5 4 3 2 1

5 4 3 2 1

5 4 3 2 1

5 4 3 2 1

5 4 3 2 1

5 4 3 2 1

5 4 3 2 1

5 4 3 2 1

5 4 3 2 1

5 4 3 2 1

5 4 3 2 1

5 4 3 2 1

5 4 3 2 1


WHO/MSD/MSB 00.2e


Confidential Supporter Satisfaction ScaleDate: _______________________________________Code:______________________________________________

These questions are designed to help us evaluate the detoxification procedure and its? effectiveness. Please circle ONEnumber, per question, that is closest to your feelings.

Please answer ALL the questions

1 On the whole, how did you feel being involved in this home detoxification?

a) Very confident Not at all confident

b) Well supported Not well supported

c) Well informed Not well informed

d) Very calm Very anxious

e) In control Not in control

2 Were the visits from the nurse practitioner:

Too frequent Not frequent enough

3 During the home detoxification, how helpful were the following:

a) The information sheet for relatives/friends Vital Unhelpful

b) Support from the nurse practitioner Vital Unhelpful

c) Support from other agency Vital Unhelpful

(Please state)

d) Support from the GP Vital Unhelpful

e) Drugs prescribed by the GP Vital Unhelpful

f) The breath analyser checks Vital Unhelpful

g) Having a telephone number for immediate advice Vital Unhelpful

4 What did you find most helpful during this home detoxification?

____________________________________________________________________________________________________

____________________________________________________________________________________________________

5 What did you find the least helpful during this home detoxification?

____________________________________________________________________________________________________

____________________________________________________________________________________________________

Thank you for completing this form. Please check that you have answered ALL the questions

5 4 3 2 1

5 4 3 2 1

5 4 3 2 1

5 4 3 2 1

5 4 3 2 1

5 4 3 2 1

5 4 3 2 1

5 4 3 2 1

5 4 3 2 1

5 4 3 2 1

5 4 3 2 1

5 4 3 2 1

5 4 3 2 1


WHO/MSD/MSB 00.2e

1 On completion of home detoxification did the client (circle)

Attend SCAS for counselling

Attend a rehabilitation centre

Attend A.A. meetings

Receive any other support

If yes to .4. state: __________________________________

2 During the past (circle): 6 months 12 months 24 months - did the client

Have an Antabuse programme

Return to old pattern of drinking

If different drinking pattern, specify (circle):

a) Abstinence

b) Social Drinking

c) Controlled Drinking

d) Relapse

Relapse ++

3 Please indicate original resolve (circle):

a) Social Drinking

b) Controlled Drinking

c) Abstinence

4 Has the client kept to the original resolve (circle):

5 Drinking status on day of follow-up (state): ___________________________

Comments:_______________________________________________________________________________________________________________________________________________________________________________________________

Note: Y = Yes, N = No


Home Detoxication Follow-upCode:_______________________________________________________________________________________________

Date of Detoxification _______________________________________Completed:________________________________

Date of follow-up 6/12 12/12 24/24

N oYes

N oYes

N oYes

N oYes

N oYes

N oYes

N oYes

N oYes

N oYes

N oYes

N oYes

N oYes

N oYes

N oYes

N oYes


WHO/MSD/MSB 00.2e

Have you ever carried out a home alcohol detoxification?

How many times have you been involved in such a procedure during the past year?

Did you carry out the detoxification in conjunction with another worker?

If yes, what profession were they (Nurse, CPN, Health Visitor, other - please specify)

_________________________________________________________________________

Did you engage the SCAS home detoxification nurse?

Did you feel satisfied with the outcome(s) of the detoxification(s)

Could you please say why?

_________________________________________________________________________

_________________________________________________________________________

What do you consider to be the main difficulty for a GP,treating patients at home for their alcohol problem?

_________________________________________________________________________

_________________________________________________________________________

Would it make it much easier for you to undertake more homebased treatments for alcohol problems if:

There was somebody to monitor and visit the patient daily?

There were suggestions for appropriate medication?

There was guidance on the assessment of patients suitable for a home detoxification?

d) There was access to advice and information if uncertain about how to proceed?

There was information about follow-up counselling or support agencies for alcohol users?

f) Anything else?

_________________________________________________________________________

10. Would you be interested to receive a visit from the SCAS nurse practitioner?


Home Detoxication GeneralPractitioner QuestionnaireGeneral Practitioners Name: ______________________________________________________________________________

Practice Address: ______________________________________________________________________________________

______________________________________________________________________________________

Note: Please circle (Y = Yes, N = No) or state where appropriate

________________

N oYes

N oYes

N oYes

N oYes

N oYes

N oYes

N oYes

N oYes

N oYes

N oYes


WHO/MSD/MSB 00.2e


AdvantageSM Software

AdvantageSM

The Alcohol Dependency System (ADS) andthe Drug Dependency System (DDS) werecomprehensive data collection systems pro-duced by Miriam Healthcare, a private com-pany established in 1992 (Yates, 1996).These packages were capable of supplyingall the information outlined in this case study.They also provide the clinical records, clientcontact - direct and indirect - and variousassessment tools and scoring systems, andproduce reports in various formats includingpi-charts and graphs, pre-formatted letters.

Advantagesm is a much-improved version ofthe ADS & DDS system. In comparison tothese software packages, this Windows 3:11and 95 application offers significant improve-ment in terms of a practical, user-friendly,application. Without prior user knowledge ofthe upgraded system, manual, instructions oron-line support, the author quickly found hisway around the programme, and input data(within the limitations of a demonstrationdisc). Anyone with knowledge of Microsoftapplications such as MS Word, Access orExcel would soon find their way around theapplication.

Areas of significant improvement in terms ofease of use and access included:

• Ability to quickly update the picklists.

• Change the system access, and accesslevel without compromising security.

• Screen-printing.

• Report printing, e.g., prescription-cost-ing report.

• Prompts to save changes.

• The use of clear green “4” and red “X”buttons.

• The database report

• Mail and contact addresses.

• Outcome measures.

• Graph with 3D application.

At the time of writing, the author understandsthat the revised version will also include aspecific area to:

Record blood test results, e.g., blood alco-hol concentration (BAC), Gamma GT, AST,ALT, and MCV, etc. Urine test results, anddaily Breathalyser readings. Monitor andrecord observations, e.g., blood pressure,pulse, etc., and progress during detoxifica-tion and any daily modification of drugs usedduring detoxification.

The only area of concern was that includedin the list of drugs used for detoxification wasHeminevrin (chlormethiazole). The BritishNational Formulary (BNF 1995), and theCommittee on Safety of Medicines (1987)advise against the use of this drug for alcoholhome detoxification.

One key problem to overcome with any soft-ware package is to convince the end userthat the package is user friendly. The pack-age was easy to use, and one quickly be-came familiar with the controls and com-mands. The picture, button and list supportwas clear. It was easy to get in and out ofany area, e.g., from the client record file intosubsections of the file, without any real


WHO/MSD/MSB 00.2e

knowledge of the programme. The 'index file'appearance is familiar to most end users, asis the terminology used in each file and sub-file. Just as important is the ability to print offreports and charts quickly and effectively.

A key benefit of the software is that it couldeasily be used on a laptop computer, andtaken into the client's home. This would ob-viously mean that input from the client, in re-lation to the completion of a questionnaire,could be directly applied with minimum su-pervision. Yates (1996) suggests that the cli-ent is more likely to respond truthfully to acomputer questionnaire, and that computersoftware questionnaires be perceived to holdno threat, as long as adequate preparationhas been given on the use of the programme.Thus, the client feels less inclined to give aresponse that will 'please' or influence theoutcome.

Such systems, whilst expensive as an initialoutlay, recoup the value quickly, in time, andman-hours saved, in collation and reportpreparation. However, a word of warning,Yates (1994) correctly suggests that the 'cli-ent [service provider] may hold unrealis-tic expectations and not appreciate thechanges in work practice required to ac-commodate the new software.'

If investment in staff training is not made, thenthe system will hold no value, however, withsufficient training and support such systemsare invaluable to the service provider opera-tional in the proactive field of service provi-sion and justification. The alcohol depen-dency system (ADS), as a package, has anintegrated monitoring and report process foralcohol home detoxification. DISCLAIMER:Whilst the author has evaluated the pack-age, he is not an employee or agent of theorganisation.


WHO/MSD/MSB 00.2e

Who was asking thequestion(s) and whydid they want theInformation?

After a preparatory phase of about threeyears, a national project on medical prescrip-tion of narcotics started in Switzerland nearthe end of 1993. It was implemented by theFederal Office of Public Health, based onthe Federal Government decree of 13 may1992 on the “Advancement of the scientificrelated research for drug prevention and im-provement of living conditions of drug ad-dicts” (Verordnung des Bundesrates 1992).In the context of the official Swiss drugpolicy, the prescription project was only onein a number of initiatives to reduce drug re-lated problems by preventive and therapeu-tic interventions (Bundesratsbeschluss 1991).

Controlled medical prescription has been partof the Swiss treatment system since the lateseventies, but restricted to oral narcotics,mostly methadone. The new project includedprescription of intravenous heroin, morphineand methadone. Especially the inclusion ofheroin raised great interest, as only England

The authors aloneare responsible forthe views expressedin this case example.

had a tradition in prescribing pharmaceuticalheroin to heroin addicts, and only a few coun-tries used it as a potent analgesic. The Swissexperiment reopened the debate on the even-tual usefulness of prescribing heroin to heroinaddicts in the framework of a therapeuticsetting.

A standing National Expert Committee ondrug problems commissioned a study to re-view all international experience with mor-phine and heroin prescription (Mino 1990).The study invited further consideration of adiversified prescription scheme, especially theexperiment with intravenous morphine for ahighly problematic group of heroin users inAmsterdam (Derks 1990, Van Brussel1995). The diversified prescription of nar-cotics in Merseyside, England, also invited areconsideration of prescription practices, al-though the original work of Dole andNyswander demonstrated specific advan-tages of oral methadone in comparison toinjectable heroin. Further, heroin prescrip-tion in British clinics was to a large extentreplaced in the eighties by prescribing oralmethadone (Gossop and Strang 1996). Theexpert committee recommended further ex-perimentation.

Case example of aprocess evaluation

Medical Prescription of NarcoticsBackground and IntermediateResults of a Swiss National ProjectbyA. UchtenhagenA. Dobler-MikolaF. Gutzwiller


WHO/MSD/MSB 00.2e

The legal basis for an eventual experimentwith heroin for heroin addicts was clarifiedby the Federal Department of Justice. Thenarcotic law of 19951 in its article 8 pro-vides that heroin can be used for scientificpurposes only. It was concluded that a sci-entific experiment would be justified.

The main objective was to involve, in a thera-peutic programme, those heroin addicts whocould not or not in a satisfactory way, becontacted by existing treatment programmes.This objective was in line with a policy de-signed to make treatment available for thelargest possible proportion of heroin addicts.The reduction of risk for the spread of HIV-infections was the main motive in addition toa reduction in other types of problems.

Information on scope, size and main condi-tions of the project is summarised as follows:

Substances involved:heroin, morphine, methadone

Application:i.v., p.o., or smoked

Duration of project:3 years (deadline Dec. 31, 1996)

Size of project:original plan = 700 participants(250 receiving heroin)final plan = 1,000 participants(800 receiving heroin)

Entry criteria:• minimum age = 20• heroin dependence = minimum of 2 years• other treatment approaches failed• social/health problems evident• compliance with programme• informed consent

Exclusion criteria:• non-compliance with programme• violence on the premises

Research agenda:approved by National Government and byEthical Committee of Swiss Academy ofMedical Sciences

According to Narcotic Law, the project hadto be a combination of a scientific and atherapeutic programme detailed as follows:

Therapeutic Programme:• Comprehensive medical, psychiatric and

social assessment• Comprehensive medical and psychoso-

cial care, including sheltered living (ifneeded)

• On-site controlled injections (no take-home of injectable substances)

Scientific Programme:• Data collection from therapeutic

programme (medical examinations, labo-ratory findings, daily dosages of pre-scribed substances, other treatment data,observations on behaviour e.a.)

• Data collection from independent inter-viewers (self-report data on social andmedical history and status at regular in-tervals)

• Data collection from third parties (medi-cal records, criminal records)

What resources wereneeded to collectand interpret theinformation?

A first proposal for such an experiment, in-cluding the original research plan, was pre-pared on behalf of the Federal Office of Pub-lic Health, and with the approval of theNational Committee on Narcotics submittedto National Government, to the National Eth-ics Committee of the Swiss Academy ofMedical Sciences and to the Federal DataProtection Officer (Bundesamt fürGesundheitswesen 1993). The objectivesand the detailed working plan, being po-litically, ethically and juridically approvedof, became officially acknowledgedin a Federal Government's decree(Bundesratsbeschluss vom 13. Mai 1992).A research group headed by the authorswas commissioned to conduct the evalua-tion research. The researcher's work was


WHO/MSD/MSB 00.2e

to be supervised and accompanied by theinterdisciplinary National Expert Group ofthe Federal Office of Public Health. Thisoffice also initiated the manufacture andcontrol of the galenic forms (non-syntheticpharmaceutical preparations) required forthe study. The International NarcoticsControl Board, a United Nations bodyresponsible for supervising the applicationof international conventions on narcotics,issued the permits for the importation ofthe required amounts of heroin. Approvalprocedures, operational security and lo-gistics were discussed and organised incollaboration with chief medical officers,chief pharmacists and police authorities incantons where a sub-project was to berealised.

How were the datacollected?

A comprehensive study protocol includeddata to be collected from clinic staff, frominterviews organised by an independent re-search group, as well as data from othersources such as medical records and crimi-nal records (Uchtenhagen et al 1994). Alldata were rigorously protected: the iden-tity of participants being known to the re-spective clinics staff and to controlling au-thorities (Federal Office of Public Health,Cantonal Chief Medical Officer) only. Theresearch team worked with anonymiseddata coded for evaluation purposes. Datacollection was organised in a standardisedform in order to allow an analysis of bothindividual and pooled data. The range ofresearch questions is shown as follows:

Patient-related items:

• Changes in medical/psychiatric status

• Changes in addictive behaviour

• Changes in coping behaviour includingreduction of risk-taking

Substance-related items:

• Pharmacological/toxicological effects ofprescribed substances

• Bioavailability of substances

• Therapeutic applicability of various galenicpreparations

Service-related items:

• Feasibility of project

• Management of safety problems

• Cost-effectiveness of project

Global evaluation:

• Advantages/disadvantages in comparisonto substitution with oral methadone

• Recommendations for therapeutic prac-tice and legislation

Patient-related and service-related itemswere dealt with in a standardised format us-ing questionnaires which to a large extent arecompatible with those used in the evaluationof other methadone programmes and absti-nence programmes. Substance-related datawere partly collected in a standardised formfor all participants, also, partly in specificstudies with a special protocol to be used insubsamples.

As a comparison group, patients enteringregular methadone programs were docu-mented and followed-up on the basis of thesame protocol. Not all sub-projects followedthe same design, varying from double-blindand randomised allocation of substances toindividual indications (Fig. 1).

Double-blind studies are designed in orderto test effects of heroin against those of mor-phine, excluding subjective expectations andconnotations. Rapid metabolisation of heroin


WHO/MSD/MSB 00.2e

to morphine in the human body makes thisstudy especially interesting. The randomisedstudies respect the rules of trials accordingto good clinical practice, whereas individualindication mirrors therapeutic practice andtherefore allows for comparisons especiallywith methadone programmes. One sub-project was designed to serve especiallywomen (including sex-workers); its resultsshould be compared to those for other fe-male participants.

The original research plan was subsequentlymodified. The number of planned morphinetreatment slots was reduced following a num-ber of unfavourable experiences with pre-scribing intravenous morphine (frequent andclearly adverse effects, poor acceptance).For the same reasons, a similar reduction inthe number of slots for intravenous metha-done was decided. On the other hand, theSwiss Council of Minister’s Resolution of 30January 1995 permitted an increase in thenumber of heroine treatment slots to a maxi-

Figure 1: Original Research Plan

Objective Substances Design Controls

A specific effects heroin i.v. double-blind Bheroin-morphine morphine i.v.

B specific effects heroin i.v. randomised Aheroin-morphine morphine i.v.

C substance-specific effects heroin i.v. randomised Dmorphine i.v.methadone i.v.

D substance-specific effects heroin i.v. individual Cmorphine i.v. indication oral methadonemethadone i.v.

E effects of specific heroin i.v. individual women from A-Dprogramme for women morphine i.v. indication

methadone i.v. oral methadone

mum 500. In addition to these changes insample size, the research record was supple-mented by a comprehensive questionnaire oncriminal behaviour (Killias et al 1995).

In May 1995, the Swiss Council of Minis-ters resolved to expand the programme withadditional questions and sub-projects. Thus,the number of places for heroin prescriptionincreased further to 800, and the overall num-ber of participants to 1000. Special researchrecords were used for the supplementaryquestions, so as to avoid having to alter theoriginal research protocol. Additional ques-tions regard the insertion of sub-projects intoexisting methadone clinics, and the treatmentof patients with dual diagnosis.


Intermediate results concern feasibility of theproject and the multi-centre study, the effectsand therapeutical applicability of substances,the characteristics of participants, their re-


WHO/MSD/MSB 00.2e

tention and follow-up data. Two extensivereports on intermediate results have beenpublished (Uchtenhagen et al 1996a,Uchtenhagen et al 1996b).

Feasibility

The results on aspects of feasibility can besummarised as follows:

• 7 out of 9 original sub-projects have beenrealised in 1994 (with differentorganisational structures and financial sup-port)

• non-realisation of 2 sub-projects is dueto low acceptance for morphine injections

• 8 additional sub-projects have beenrealised in 1995

• 4 referenda on sub-projects were heldwith positive results

• no negative impact of prescription clinicson neighbourhoods

• qualified staff recruited, low turnover

• no diversion of narcotic substances intoblack market

• no severe cases of overdose

• 1210 persons were recruited as partici-pants (drop-outs were allowed to be re-placed until January 7, 1996)

Two of the originally authorised sub-projectswere not realised because the side effects ofintravenous morphine injections resulted in alow acceptance for this modality and there-fore two clinics which were foreseen to pre-scribe morphine only could not recruit enoughparticipants. All the other sub-projects wererealised with the support from the respectiveauthorities in the cities of Basel, Bern,Fribourg, Olten, Thun and Zürich. Accord-ing to police information, no negative impacton neighbourhoods was observed, in con-trast to some negative expectations. While

the clinics were inserted into the existing thera-peutic networks without difficulties; an even-tual impact on other treatment approacheswill have to be analysed. Due to the on-sitecontrolled injections, no diversion of narcot-ics substances into the black market tookplace, an also no severe cases of overdose.

Additional sub-projects were realised inGenf, Horgen, Luzern, Solothurn, St. Gallen,Wetzikon, Winterthur and Zug. Referendawere held in Basel and Zug.

The findings for the substances involved aresummarised as follows:

Morphine• frequent histaminic-like reactions when

intravenously injected, especially in fe-males

• correctly identified by participants indouble-blind trials due to side-effects

• low acceptance by participants• long-acting oral morphine well accepted

when oral methadone not applicable

Methadone• untoward local effects when intravenously

injected due to dosage• low acceptance by participants• oral methadone well accepted as basic

substitution medication in combinationwith injectable heroin

Heroin• negligible side effects when intravenously

injected• stable dosages in most participants• heroin cigarettes not satisfactory (low

bioavailability, side effects)• slow-release tablets in experimentation

Conclusion:Heroin more applicable intravenously in com-parison to morphine and methadone; oralapplication to be tested.

Intravenous morphine application was fol-lowed more frequently and more severely byhistaminic-like reactions (local orgeneralised). The side effects are similar to


WHO/MSD/MSB 00.2e

those already know in general medicine, butintensified due to the higher dosages neededin persons with fully developed opiate toler-ance. The side effects are responsible for themostly correct identification of morphine indouble-blind studies, and also for a low ac-ceptance by many participants (58% of thosereceiving morphine left the project orswitched to heroin). MST continues, a long-acting oral morphine preparation, on the otherhand, was well accepted and especially usedwhen oral methadone is not applicable.

Rather surprisingly, intravenous methadonemet a low acceptance as well. Side effectsat the site of injection may explain part of it,probably also due to higher dosages (in com-parison to British practice). However, wecannot exclude a psychological negative ef-fect in those participants who expected toreceive heroin but were given methadone ina randomised fashion. On the other handagain, oral methadone was well accepted byparticipants in combination with injectedheroin, as it allows to reduce daily heroin in-jections from three or four to one, thereforereducing the need for multiple clinic visitseach day and facilitating employment andother activities.

Heroin was also reported to have some sideeffects similar to those observed in intrave-nous morphine, but less severe. After an ini-tial phase of several weeks, a stable dosagecould be reached by most participants, andno unlimited increase of dosage was askedfor. The specially prepared tobacco-freecigarettes, base on woodruff, standardisedwith heroin solutions of 50mg and 100mgproved to be not very satisfactory; up to 90%of heroin was destroyed in the burning pro-cess (tested in laboratory) and bioavailabilityis accordingly low. Other forms of galenicpreparations such as slow-release tablets aretherefore in experimentation.

Description of participants

To what extent participants correspond toentry criteria can be seen as follows:

Anamnestic data and findings at entry, co-horts 1994 and 1995, participants receivingheroin, n=786.

• average age 30.7 years• average duration of heroin dependence

10.4 years• unstable housing condition in 39%• unemployed 82%• illegal activities in 54%• minimal or no contact outside of drug

scene in 39%• court convictions in 84%, ever in prison

51%• daily consumption of cannabis in 31%, of

cocaine in 34%, of alcohol in 19%, ofbenzodiazepines in 25%

• former treatments: 91% methadone main-tenance, 88% detoxification, 50% resi-dential long-term treatment (mostly mul-tiple treatments)

• somatic health impaired in 24%• hepatitis in the past, 74%• HIV seropositivity in 22%• psychiatric hospitalizations in 48%• suicidal attempts in 43%• psychological health impaired in 42%

Conclusion:Entry criteria are respected. Participants areolder, with a longer history of heroin depen-dence, and more marginalized and impairedin comparison to other treatment populations.

These data concern all participants whichentered the project in 1994 and 1995. Thiscohort is, on the average, older than mostother treatment populations from residentialand methadone programmes, with a longerduration of heroin dependence. The numberof former treatments, the amount of socialdeficits including a high amount of delin-quency, and also the amount of health im-pairment demonstrate to what extent the en-try criteria have been respected. The targetgroup for which the project was designedcould effectively be reached, although a mi-nority only was out of any treatment contactfor the last six months before entry. This maybe seen as an effect of a comparatively high


WHO/MSD/MSB 00.2e

threshold due to the comprehensive thera-peutic and scientific programme.

Follow-up results

Follow-up results on participants who en-tered the project in 1994, are summarisedas follows:Retention Rates and Drop-outs (Basis: 1994cohort, participants receiving heroin, n=317.)

Retention rate:• 82% during 6 months• 67% during 15 months

Drop-outs during first 15 months:• 54% of drop-outs changed to another

treatment modality (45% to methadonemaintenance, 9% to abstinence therapy)

• 35% discontinued treatment or were ex-cluded for threat of violence or other se-rious misbehaviour

• 11% were hospitalised, moved or died(4 fatalities, no overdose)

• higher rate of daily cocaine use at entryamong drop-outs

• drop-outs who were excluded were moremarginalized at entry

Conclusion:Higher retention rate in comparison to resi-dential treatment, satisfactory in comparisonto methadone maintenance.

Retention rate during the first 15 months ofparticipation was higher than in most othertreatment programmes for heroin addicts inthe country. Half of those who dropped outduring the first six months went back to an-other treatment modality, mostly to metha-done maintenance. About 25% of drop-outshad to be excluded for intolerable behaviour.Four fatalities were recorded, mainly due tochronic infectious disease.

Entry data of drop-outs indicate that partici-pants with daily cocaine use at entry, thoseengaged in prostitution and those with a his-tory of aggressive acting-out behaviour wereover-represented among drop-outs. For the

following tabulation, participants only whoreceive heroin, stayed for at least one yearand were able to perform the second fol-low-up interview in time are included.

Basis:206 of 366 participants entering trials during1994 and receiving heroin (self-report data,data from urine tests and police data; inbrackets data at entry):

• housing situation stable in 70% (64%)• employment, no contact with drug scene

in 50% (16%)• unemployed, frequent contact to drug

scene in 14% (50%)• unemployed without contact to drug scene

in 36% (35%)• illegal income in 14% (70%) according

to self-report, significant reduction in de-linquency according to police reports

• reduction of daily illegal cocaine use from31% to 7%, but minimal reduction of can-nabis use from 34% to 32% and of ben-zodiazepine use from 20% to 14% (self-report and urine tests)

• prostitution, peddling etc. in 7% (46%)• somatic health improved (significantly re-

duced injection-related syndromes andtroubles of the Autonomous NervousSystem)

• psychological health improved (signifi-cantly reduced number of depressions andparanoid states)

Conclusion:Participants showed significant changes insocial and health status, including risk takingbehaviour.

Changes in social status were observed re-garding housing situation and, more surpris-ingly, employment rate. A reduction of illegalactivities and of contact with other drug us-ers was based on self-report-data and onpolice information (recorded delinquencybefore and after entry into project during 6month periods). Urine controls documenteda reduction of daily cocaine use, not of dailycannabis and benzodiazepine use.


WHO/MSD/MSB 00.2e

Body-mass measures and injection-relatedsyndromes improved significantly (accord-ing to number of abscesses and other der-matological scores), as well as depressive-suicidal syndromes and paranoid and anxietystates. Other somatic problems and also ag-gressive acting-out behaviour showed no sig-nificant changes.

Final comments

By these first intermediate results, the fea-sibility, safety and therapeutic applicabilityof heroin prescription to heroin addictsunder adequate conditions are docu-mented. Beneficial effects during the first12 months of participation were also evi-denced. This conclusion however cannotbe generalised. It concerns a specific andmost marginalized target population of ad-dicts, and a prescription practice embed-ded into a comprehensive assessment andcare programme.

Providing pharmaceutical heroin in thisproject permits to attract addicts who fail inmultiple other treatment approaches and givethem a new opportunity to take courage andto engage in a therapeutic and rehabilitationprogramme, without being forced to abstainfrom their preferred substance. It is not as-sured yet, however, to what extent positivechanges in health and social status will con-tinue over longer periods of time, and to whatextent a drug-free lifestyle can be reachedby participants in a later stage.

This experiment cannot be the basis for anyclaim to support “free heroin”, but it could -it the further evaluation results corroboratethe preliminary positive findings - add asupplementary option for treating long-stand-ing marginalized heroin addicts unable to profitfrom other treatment approaches. This im-plies that other treatment approaches haveto be available in good quality and sufficientnumber, when such an additional option isconsidered.


WHO/MSD/MSB 00.2e

References

Bundesamt für Gesundheitswesen Metha-donbericht, erste Auflage. Bern, 1984.

Bundesamt für Gesundheitswesen Metha-don-bericht, dritte Auflage. Bern, 1996.

Bundesratsbeschluss vom 20. Februar1991 zur Verminderung der Drogen-probleme. Bern, 1991.

Bundesratsbeschluss vom 30. Januar 1995zur Ärztlichen Verschreibung vonBetäubungsmitteln. Bern, 1995.

Bundesratsbeschluss vom Mai 1995 zurÄrztlichen Verschreibung von Betäu-bungsmitteln. Bern, 1995.

Van Brussel G. Das Morphinabgabeprogrammin Amsterdam. Praktische Erfahrungen. In:Ärztliche Verschreibung von Betäubung-smitteln. Wissenschaftliche Erfahrungenund praktische Erfahrungen. Ihs-Midel M.Lotti H, Stamm R, Clerc J. (Hrsg.). HansHuber Verlag, Bern, 1995.

Derks J. Het Amsterdamse Morfineverstrekkingsprogramma. Een longitudinaalonderzoek onder extreem problematischedruggebruikers. Nederlands centrumgeestelijke volksgezondheid. Utrecht, 1990.

Dezentralisierte Drogenhilfe im KantonZürich. Kantonale Gesundheitsdirektion.Zürich 1990.

Gossop M., Strang J. Heroin prescribing inthe British system: historical review. Euro-pean Addiction Research (this issue), 1996.

Killias M, Uchtenhagen A. Méthodologie del’valuation des essais suisses avec prescrip-tion médicale d’opiacés sous l’angle de ladélinquance. Kriminologisches Bulletin,1995, Nr 2/95.

Mino A. Wissenschaftliche Literatur-analyse der kontrollierten Heroin - oderMorphinabgabe. Bundesamt fürGesundheitswesen, Bern, 1990.

Mühle U. Reha 2000. Projektbericht zurSituation und zum Handlungsbedarf in derstationären Drogentherapie. Im Auftragdes Bundesamtes für Gesundheitswesen,1994.

Uchtenhagen A. Suchtrisiko und Suchtbe-handlung in der Schweiz. In: Bäker W, NellesJ. (Hrsg). Drogenpolitik wohin? Paul HauptVerlag Bern, 1991.

Uchtenhagen A., Zimmer-Hofler D., Dobler-Mikola A. Psychosoziale Aspekte der HIV-Infektion bei Heroinabhängigen.Schlussbericht an den SchweizerischenNationalfonds. Sozialpsychiatrischer Dienstder Psychiatrischen Universitätsklinik Zürich,1993.

Uchtenhagen A. Diversifizierte Versch-reihbung von Betäubungsmitteln anHeroinabhängige: Grundlagen, Versuchsplan,Begleitforschung. Schweiz. Rundschau Med(Praxis), 1994, 34: 931-936.

Uchtenhagen A., Gutzwiller F., Dobler-MikolaA. Versuche für eine ärztlicheVerschreibung von Betäubungsmitteln.Studienprotokoll der Begleitevaluation.Psychiatrische Universitätsklinik und Institutfür Sozial -und Präventivmedizin derUniversität Zürich, 1994.

Uchtenhagen A., Gutzwiller F., Dobler-MikolaA., Blätter R. Program for a medical pre-scription of narcotics. Interim report of theresearch representatives. Institut fürSuchtforschung und Institut für Sozial - undPräventivmedizin an der Universität Zürich,1996 a.

Uchtenhagen A., Gutzwiller F., Dobler-MikolaA. Versuche für eine ärztlicheVerschreibung von Betäubungsmitteln.Zweiter Zwischenbericht der Forschungs-beauftragten. Institut für Suchtforschung adInstitut für Sozial - under Präventivmedizinan der Universität Zürich, 1996b.

Verordnung des Bundesrates über dieForderung der wissenschaftlichenBegleitforschung zur Drogenpräventionund Verbesserung der LebensbedingungenDrogenabhängiger. Bern 1992.


WHO/MSD/MSB 00.2e

It�s your turnWhat are the strengths and the weaknesses of the presented case example? List three posi-tive aspect and three negative aspects:

Strengths of the case study

1

2

3

Weaknesses of the case study

1

2

3

process evaluations - emcdda home page | … · workbook 4 • process evaluations 3 who/msd/msb...

Documents