diagnostic practices of evaluators of drunken drivers

5
0740-5472187 $3.00 + .oO Copyright 0 1987 Pergamon Journals Ltd Journal of Subsrance Abuse Treamenr, Vol. 4, pp. 79-83. 1987 Printed in the USA. All rights reserved. ORIGINAL CONTRIBUTION Diagnostic Practices of Evaluators of Drunken Drivers BONNIE SPILLER AND HAROLD ROSENBERG Bradley University, Peoria, IL Abstract- This study was designed to evaluate the impact of driving under the influence (Or/I) arrest history on the diagnostic decisions of DUI evaluators and the reported bases on which al- cohol-related diagnostic decisions are made in DUI cases. Subjects were 70 (out of a potential 140) Illinois certified DUI evaluators who responded by mail to one of four case summaries containing different information about a ‘Went’s” drinking history and arrest history. Results indicated a sig- nificant difference in the frequency with which these DUI evaluators noted whether the “client” had an alcohol problem, with zero, one and two DUI arrests yielding approximately 30%, 15%, and 50% alcohol diagnoses in the absence of DSM-III criteria supporting such a diagnosis. Collaborative reports by significant others and alcohol-related tests (e.g., Michigan Alcoholism Screening Test) were the two most frequently reported bases. The costs of such diagnostic unreliability and the dis- advantages of collaborative reports and screening tests are discussed. Keywords-Driving under influence (DUI), DSM-III, diagnostic practices, assessment. IN AN EFFORT TO REDUCE alcohol-related highway fatalities and traffic injuries, legislatures and policy makers in many states have passed or are considering laws and policies severely penalizing those drivers arrested and convicted of driving under the influence (DUI). For example, Illinois recently began requiring an alcohol evaluation or screening (and treatment if an alcohol disorder is present) for those convicted of DUI. This type of administrative action, combined with judicial and community support, has resulted in an increase in the number of DUI assessment and edu- cation programs. Many advocates hope that the out- come of such programs will include the identification and rehabilitation of problem drinkers and a decrease in recidivism among program participants. However, requiring DUI offenders, as part of their Portions of this article are based on the Masters thesis of the first author submitted to Bradley University, Peoria, IL. Bonnie Spiller is currently a part-time Clinical Associate Psychol- ogist at the University of Illinois College of Medicine at Peoria and a part-time instructor at Bradley University. Harold Rosenberg is Associate Professor and Coordinator, Masters Program in Com- munity-clinical Psychology at Bradley University. Requests for reprints should be sent to either author at the Psy- chology Department, Bradley University, Peoria, IL 61625. court diversion or probation, to undergo an evalua- tion (and possibly treatment) assumes that we can reli- ably and accurately recognize an alcohol problem when one is present and, just as important, that we can recognize when one is not present. Shaffer (1986) has noted that the “assessment and diagnosis of addic- tion is a difficult task, given the conflicting and con- troversial state of affairs in the field” (p. 385) and diagnostic reliability has not been established in the population of evaluators who routinely conduct DUI assessment. Our concern is that a DUI arrest (or arrests) over- shadows other diagnostic evidence and increases the likelihood that an individual will receive an alcohol diagnosis-even though the offender does not have an alcohol problem. This issue is complicated because there are a variety of proposed diagnostic criteria for alcohol use disorders (e.g., NCA, WHO, DSM-III, Jellinek) and some of these systems change over time. Alcohol evaluators use self-styled variations of these changing systems, potentially resulting in inconsistent and erroneous diagnostic decisions. This study was designed to evaluate the influence of DUI arrest his- tory on the diagnostic impressions of DUI evaluators and the reported bases on which these evaluators make alcohol-related diagnostic decisions. 79

Upload: harold

Post on 02-Jan-2017

214 views

Category:

Documents


1 download

TRANSCRIPT

0740-5472187 $3.00 + .oO Copyright 0 1987 Pergamon Journals Ltd

Journal of Subsrance Abuse Treamenr, Vol. 4, pp. 79-83. 1987 Printed in the USA. All rights reserved.

ORIGINAL CONTRIBUTION

Diagnostic Practices of Evaluators of Drunken Drivers

BONNIE SPILLER AND HAROLD ROSENBERG

Bradley University, Peoria, IL

Abstract- This study was designed to evaluate the impact of driving under the influence (Or/I) arrest history on the diagnostic decisions of DUI evaluators and the reported bases on which al- cohol-related diagnostic decisions are made in DUI cases. Subjects were 70 (out of a potential 140) Illinois certified DUI evaluators who responded by mail to one of four case summaries containing different information about a ‘Went’s” drinking history and arrest history. Results indicated a sig- nificant difference in the frequency with which these DUI evaluators noted whether the “client” had an alcohol problem, with zero, one and two DUI arrests yielding approximately 30%, 15%, and 50% alcohol diagnoses in the absence of DSM-III criteria supporting such a diagnosis. Collaborative reports by significant others and alcohol-related tests (e.g., Michigan Alcoholism Screening Test) were the two most frequently reported bases. The costs of such diagnostic unreliability and the dis- advantages of collaborative reports and screening tests are discussed.

Keywords-Driving under influence (DUI), DSM-III, diagnostic practices, assessment.

IN AN EFFORT TO REDUCE alcohol-related highway fatalities and traffic injuries, legislatures and policy makers in many states have passed or are considering laws and policies severely penalizing those drivers arrested and convicted of driving under the influence (DUI). For example, Illinois recently began requiring an alcohol evaluation or screening (and treatment if an alcohol disorder is present) for those convicted of DUI. This type of administrative action, combined with judicial and community support, has resulted in an increase in the number of DUI assessment and edu- cation programs. Many advocates hope that the out- come of such programs will include the identification and rehabilitation of problem drinkers and a decrease in recidivism among program participants.

However, requiring DUI offenders, as part of their

Portions of this article are based on the Masters thesis of the first author submitted to Bradley University, Peoria, IL.

Bonnie Spiller is currently a part-time Clinical Associate Psychol- ogist at the University of Illinois College of Medicine at Peoria and a part-time instructor at Bradley University. Harold Rosenberg is Associate Professor and Coordinator, Masters Program in Com- munity-clinical Psychology at Bradley University.

Requests for reprints should be sent to either author at the Psy- chology Department, Bradley University, Peoria, IL 61625.

court diversion or probation, to undergo an evalua- tion (and possibly treatment) assumes that we can reli- ably and accurately recognize an alcohol problem when one is present and, just as important, that we can recognize when one is not present. Shaffer (1986) has noted that the “assessment and diagnosis of addic- tion is a difficult task, given the conflicting and con- troversial state of affairs in the field” (p. 385) and diagnostic reliability has not been established in the population of evaluators who routinely conduct DUI assessment.

Our concern is that a DUI arrest (or arrests) over- shadows other diagnostic evidence and increases the likelihood that an individual will receive an alcohol diagnosis-even though the offender does not have an alcohol problem. This issue is complicated because there are a variety of proposed diagnostic criteria for alcohol use disorders (e.g., NCA, WHO, DSM-III, Jellinek) and some of these systems change over time. Alcohol evaluators use self-styled variations of these changing systems, potentially resulting in inconsistent and erroneous diagnostic decisions. This study was designed to evaluate the influence of DUI arrest his- tory on the diagnostic impressions of DUI evaluators and the reported bases on which these evaluators make alcohol-related diagnostic decisions.

79

30 B. Spiller and H. Rosenberg

METHOD

i’rocedure

The potential subject pool was 144 individuals and/or agencies approved as of June, 1983, by the Illinois Secretary of State’s office to make DUI-related alco- hol assessments. Four potential subjects were not included in the study (e.g., stimulus materials mailed to them were returned marked “undeliverable”). Each of the 140 remaining potential subjects was sent a packet of materials consisting of (a) a cover letter explaining the study and requesting their participation, (b) one of four case summaries, (c) a Diagnostic Impressions Page (DIP), and (d) a Subject Response Form.

In order to encourage participation in the study, preaddressed, stamped, coded envelopes were included and up to three follow-up letters were mailed to non- responders. The final response rate was 70 out of the potential 140 subjects.

Stimulus Materials

Subjects were randomly assigned to one of four groups that varied with regard to the content of a case summary to be evaluated (summaries are available from authors). Three case summaries were written to exclude information indicating Alcohol Abuse (305.0X) and Alcohol Dependence (303.9X), and all four summaries provided similar information about the “client’s” background except that he had either: (a) no legal involvement, (b) one DUI arrest, (c) two DUI arrests, or (d) two DUI arrests, a Michigan Alco- holism Screening Test (MAST) score above the cutoff (Selzer, 1971), and a report of blackouts.

The DIP contained two questions. The first one asked the assessor to give his or her diagnostic impres- sions of the provided case summary and the second question asked subjects to report the bases upon which they made alcohol-related diagnostic decisions when evaluating an individual referred because of a DUI arrest. The Subject Response Form was used to gather information regarding subjects’ demographic and professional characteristics, but subjects were instructed to nof put their names on any of the pro- vided materials.

Subject Characteristics

Subjects were those 70 individuals who returned their data sheets. Of the 70 respondents, 58.5% were male, 32.9% were female, and 8.6% of the subjects did not indicate their sex. The mean age of the respondents was 40.6 years; the range was 24 to 60 years. The mean number of years of education was 17; the range was 9 to 22 years. The mean number of years that sub-

jects reported they had provided professional mental health services was 9.1; the range was 2 to 30 years. The mean number of years that subjects reported pro- viding alcohol-related mental health services was 6.5; the range was 1 to 20 years.

Subjects were asked to report any Illinois licensure or certification that they held-57% of the respon- dents reported being a Certified Alcoholism Counse- lor (CAC); 13% of the respondents did not indicate whether they were licensed or certified; 12% were reg- istered psychologists; 7% reported that they held no Illinois licenses of any kind; 4% were registered social workers; 3% held medical licensure; 3% held multi- ple licensure; and one subject was a Certified Drug Counselor (CDC).

Subjects were also asked to describe their theoret- ical orientation. Over one-third, 37% of the respon- dents, did not answer this question and 17% reported that they had no particular orientation; 14% reported Reality Therapy as their orientation; 9% reported Dis- ease Concept; 7% reported Systems; 4% reported Behavior Therapy; 4% reported Rogerian; 1% said Dynamic; and 6% indicated an orientation other than those mentioned above.

One-way analyses of variance by group on the edu- cation, age, years of professional experience, and years of alcohol-related professional experience mea- sures were conducted. There were statistically signifi- cant differences among the groups on the years of professional experience (F = 2.88, df = 3,58, p <

.05; M, = 8.5, M2 = 6.8, M3 = 12.5, M4 = 8.8) and years of alcohol-related professional experience (F = 3.97, df = 3,60, p < .Ol; M, = 6.0, M2 = 4.2, M3 = 8.9, M4 = 6.6).

Response Judging

Each of the diagnostic statements provided by the sub- jects was typed on a 5” x 8” notecard to prevent rater bias because of handwriting. The notecards were shuf- fled and randomly divided into two equal groups. Two male, second-year, psychology graduate students, trained in using DSM-III, served as independent judges. Based on written criteria provided to them, the judges’ task was to assign the subjects’ diagnostic impressions to one of two categories: (1) no alcohol diagnosis or (2) alcohol diagnosis.

The judges were instructed to use the following cri- teria to assign subjects’ responses to the categories: “Based on the statements typed on each card, does the subject perceive the person to have a definitive alco- hol problem according to: (a) DSM-III criteria for alcohol abuse or dependence, or (b) Some other for- mal system of criteria, or (c) Their [i.e., subjects’] own system of criteria, or (d) Their [i.e., subjects’] explicit or implicit statement of an alcohol diagnosis or prob-

Diagnostic Practices 81

lem, or (e) Their [i.e., subjects’] explicit or implicit statement of the absence of an alcohol diagnosis or problem.”

Inter-rater reliability was assessed by having each judge rate a randomly selected subset (n = 17) of the other judge’s protocols. Reliability, which was calcu- lated by dividing the total number of the judges’ diag- nostic agreements by the total number of diagnostic agreements plus disagreements, was 85.3%. A conser- vative decision rule was adopted for those cases of dis- agreement between judges or when the subject’s response was ambiguous or he or she declined to give a diagnostic impression (n = 9). Specifically, for the three case summaries excluding information to sup- port a diagnosis, disagreements were coded “no alco- hol problem” (n = 7) ; and for the one case summary including information supporting a diagnosis, dis- agreements were coded “alcohol problem” (n = 2).

RESULTS

First, the hypothesis that DUI arrest history would influence diagnostic impressions in the absence of spe- cific criteria for a DSM-III diagnosis of an alcohol dis- order was examined. The result of a 4 x 2 chi square test (4 case summaries x 2 diagnostic assignments) was statistically significant (x2 = 18.02, df = 3, p < .Ol). As Table 1 indicates, the case summary in which the “client” had two DUIs was rated as having an alcohol problem by approximately 50% of the subjects, al- though even the “client” with one DUI or no DUI was rated as having an alcohol problem by approximately 15% or 30% of the subjects, respectively. The “client” with evidence of both social impairment (two DUIs) and pattern of pathological use (blackouts) and an ele- vated MAST score was rated as having an alcohol problem by almost 90% of the subjects in that case summary condition.

Second, the results from the second question on the DIP were examined to assess the degree to which this sample of DUI evaluators reported using diagnostic bases other than DSM-III when evaluating DUI of- fenders. The most commonly stated bases for evalu- ating a DUI offender included: (a) collateral report (e.g., family member, employer); (b) alcohol-related tests or questionnaires (e.g., MAST, Mortimer-Filkins (Kerlan, Mortimer, Mudge, & Filkins, 1971)); (c) quan- tity and frequency of drinking; (d) medical history, lab tests, and physical appearance; (e) arrest history; (f) family history (primarily of drinking problems); (g) drinking history; (h) “loss of control”; (i) non-alcohol related psychological tests (e.g., MMPI); and (j) Blood Alcohol Content (BAC) at time of arrest for DUI. A complete listing of the reported bases and their fre- quencies is available from the authors.

TABLE 1 Frequencies of Diagnostic Assignments

for the Four Case Summaries

Diagnostic Assignment

Group No Alcohol

Problem Alcohol Problem

1. No DUI 15 (68.2%) 7 (31.8%) 2. One DUI 11 (84.6%) 2 (15.4%) 3. Two DUls 9 (47.4%) 10 (52.6%) 4. Two DlJls, higher MAST 2 (12.5%) 14 (87.5%)

score and blackouts

Most subjects’ responses were lists of specific items of information or a method of obtaining information. Although relatively few of these evaluators responded by listing an established system of diagnostic criteria (DSM-III n = 2, NCA criteria n = 5, Jellinek chart n = 8), specific items they listed often overlapped with DSM-III criteria. For example, 15 subjects listed an item that could be construed as evidence of a pattern of pathologic use (e.g., blackouts, inability to ab- stain), 22 listed items that could .be construed as evi- dence of social or occupational impairment (e.g., job absenteeism), and 17 listed items regarding evidence of dependence (e.g., tolerance, withdrawal). In addition, 12 of those 33 subjects who were rated as having eval- uated the “client” in their case summary as having an alcohol problem used specific DSM-III labels or codes to indicate their diagnostic impression.

Because the majority of subjects did not mention an explicit basis for their diagnostic decision nor did a majority respond to the item requesting their theo- retical orientation, nothing may be said with confi- dence about the relation between the two. For example, the eight subjects who explicitly noted the Jellinek cri- teria as their basis for making diagnostic decisions identified themselves as holding various theoretical or- ientations (“Reality therapy” = 2, “Family systems” = 1; “Structural-strategic” = 1; “Generic” = 1; No an- swer = 3).

DISCUSSION

This study found that there was a significant differ- ence in the frequency with which certified DUI evalu- ators (most of whom were not licensed psychologists, social workers or physicians) noted whether or not a fictitious client in a written case summary had an alco- hol disorder. Specifically, the case summary contain- ing two DUI arrests yielded a higher percentage of alcohol problem impressions than zero and one DUI arrests, although it did not yield as large a percentage

82 B. Spiller and H. Rosenberg

as the case summary that contained two DUI arrests plus clearer evidence of alcohol abuse. Because a DUI offender in Illinois who is designated to have an alco- hol problem is required to complete a treatment pro- gram, approximately one-third of the Group 1 “clients” (no DUI), approximately one-sixth of the Group 2 “clients” (one DUI), and over one half of the Group 3 “clients” (two DUIs) might be referred for treatment even though the provided information did not support an alcohol abuse or dependence disorder according to DSM-III criteria. Additionally, only two of the Group 4 “clients” (two DUIs, blackouts, MAST score > cut- off) were not recognized as meeting the criteria for alcohol abuse according to DSM-III.

In addition to inappropriate treatment referrals, there are other potential “costs” for individuals labeled with an alcohol diagnosis when no alcohol problem is present. For example, the individual may be stigma- tized by the alcohol diagnosis (e.g., restricting his or her employment opportunities, social activities, driver’s license) or the individual may have to take time off work to attend treatment sessions resulting in lost wages/salary. Misdiagnosis, especially over-diagnosis, is a serious potential problem for a proportion of indi- viduals evaluated by certified Illinois DUI assessors.

This study also examined the bases evaluators used to make their alcohol-related diagnostic decisions. Re- ports of significant others (family members, friends, employer) to verify or contradict client report was the most frequently reported basis used when conducting alcohol-related assessments. The use of corroborators raises several issues: (a) Are corroborators more accu- rate than the interviewee? (b) Do not spouses, friends, and employers have reasons to underreport or exag- gerate? (c) If contradictions occur, how does the evaluator decide who to believe? These questions point out some of the disadvantages of relying on cor- roboration as a primary basis for making alcohol- related diagnoses.

Another frequently reported criterion was the use of the objective test results, especially the MAST. However, not all evaluators reported using test results in the same way. For example, some evaluators reported that the MAST was just one of many bases used; other evaluators apparently place more emphasis on test results and use few other criteria; and still other evaluators reportedly use the test qualitatively (e.g., which questions were endorsed and why). Per- haps more importantly, the reliability and validity of tests such as the MAST are questionable (e.g., Jacob- son, 1976), and they are therefore a questionable basis for making an alcohol-related diagnosis.

Although these results present a provocative chal- lenge to the continued evaluation of DUI offenders by such certified evaluators, several limitations of the methodology may reduce their impact and generaliz-

ability. First, the procedure of asking subjects to make a diagnostic decision by reading a case summary instead of conducting a live interview may have af- fected the results. For example, the clinician is pre- vented from using various assessment strategies (e.g., observation of client behavior, eye contact, voice quality, etc.). Hyler, Williams, and Spitzer (1982) found that “the reliability of the major diagnostic classes of DSM-III was higher when diagnoses were made from live interviews than when they were made from case summaries” (p. 1275). However, they went on to note that, “The results obtained may not apply to case vignettes, that is, specially prepared, and gen- erally brief summaries edited to highlight information of differential diagnostic importance and to remove much nondiagnostic information” (p. 1278). In addi- tion, Wachtel (1980) in a DSM-III reliability study using audio, video, and written formats, found that “information format does not contribute to statisti- cally significant differences in inter-rater reliability” (p. 3975-B). To the extent that the case summaries in this study provided the necessary information to be able to differentially diagnose alcohol disorders ac- cording to DSM-III, the results may not be different than what would be obtained using other formats.

Another methodological concern is the way in which the wording of the diagnostic questions may have affected the results. For example, the results may have been different if the subjects had been given a checklist of diagnostic categories and a checklist of items reflecting their bases for making alcohol-related diagnostic decisions. Open-ended questions were used in an effort to not limit subjects’ responses to experi- menter-chosen categories.

Finally, although a response rate of 50% is better than the return rate often reported for mail survey studies, the diagnostic impressions of the other half of the potential respondents is unknown.

Despite these methodological concerns, we believe that these results are meaningful and that the diagnos- tic activity of DUI evaluators is neither reliable nor accurate in a significant number of cases. There seems to be a growing and legitimate concern about the DUI problem and a growing if less legitimate willingness to tolerate false positives in order to process most true positives. However, the clinical, social and legal costs of such diagnostic unreliability lead us to encourage evaluators to be aware of how their stereotypes of DUI offenders may influence their diagnostic and referral decisions. Although stereotypes held by professionals of “alcoholics” have been studied (e.g., Kinney, Ber- gen, & Price, 1982; Palangio, 1986), we suggest that the attitudes and responses of professional, para- professional and lay populations toward the DUI offender be investigated and DUI evaluators be edu- cated to prevent stereotyping and misdiagnosis.

Diagnostic Practices 83

REFERENCES

Hyler, S.E., Williams, J.G., & Spiuer, R.L. (1982). Reliability in

the DSM-Ill field trials: Interview vs. case summary. Archives of General Psychiatry, 39, 1275-1278.

Jacobson, G.R. (1976). The alcoholisms: Detection, diagnosis and assessment. New York: Human Sciences Press.

Kerlan, M.W., Mortimer, R.G., Mudge, B., & Filkins, L.D. (1971).

Court procedures for identilving problem drinkers, Volume I:

Manual (USDOT Publication No. DOT-HS-800-632). Ann

Arbor, Ml: University of Michigan, Highway Safety Research

Institute.

Kinney, J., Bergen, B.J., & Price, T.R. (1982). A perspective on

medical students’ perceptions of alcoholics and alcoholism. Jour- nal of Studies on Alcohol, 43, 488-496.

Palangio, J.S. (1986). Attitudes of psychologists toward alcoholics.

Dissertation Abstracts International, 46, 3228.

Seizer, M.L. (1971). The Michigan Alcoholism Screening Test: The

quest for a new diagnostic instrument. American Journal ofPsy- chiatry, 127, 89-94.

Shaffer, H.J. (1986). Assessment of addictive disorders: The use of

clinical reflection and hypotheses testing. Psychiatric Clinics of North America, 9, 385-398.

Wachtel, J.L. (1980). Psychiatric diagnosis with three clinical for-

mats and DSM-111: A reliability study. Dissertation Abstracts International, 40, 3975.