Journal of Quantitative Criminology, Vol. 14, No. 2, 1998

Defensive Gun Uses: New Evidence from aNational Survey

Philip J. Cook1,3 and Jens Ludwig2

The number of civilian defensive gun uses (DGUs) against criminal attackers isregularly invoked in public policy debates as a benefit of widespread privateownership of firearms. Yet there is considerable uncertainty for the prevalenceof civilian DGUs, with estimates ranging from 108,000 (using the National CrimeVictimization Survey) to 2.5 million (using smaller telephone surveys) per year.In this paper we analyze the results of a new national random-digit-dial telephonesurvey to estimate the prevalence of DGU and then discuss the plausibility ofthe results in light of other well-known facts and possible sources of bias insurvey data for sensitive behaviors. Because DGU is a relatively rare event byany measure, a small proportion of respondents who falsely report a gun use canproduce substantial overestimates of the prevalence of DGU, even if every truedefensive gun user conceals his or her use. We find that estimates from this newsurvey are apparently subject to a large positive bias, which calls into questionthe accuracy of DGU estimates based on data from general-population surveys.Our analysis also suggests that available survey data are not able to determinewhether reported DGU incidents, even if true, add to or detract from publichealth and safety.

1. INTRODUCTION

The widespread private ownership of firearms in the United States mayhave both positive and negative effects on the level (and lethality) of violencethat we experience. Firearms in private hands may reduce injuries by

1Sanford Institute of Public Policy, Duke University, Durham, North Carolina, and NationalConsortium for Violence Research and National Bureau of Economic Research.

2Georgetown Public Policy Institute, Georgetown University, Washington, DC, and Northwest-ern University/University of Chicago Poverty Center, Evanston, Illinois.

3To whom correspondence should be addressed at Sanford Institute for Public Policy Studies,Box 90245, Duke University, Durham, North Carolina 27708. e-mail: COOK@pps.duke.edu.

KEY WORDS: defensive gun use; firearms; survey bias.

l11

providing citizens with the means of protecting themselves against criminalattackers (Kleck, 1988, 1991; Kleck and Gertz, 1995). On the other hand,firearms may increase the lethality of suicide attempts and criminal attacks,as well as causing accidental injury (Zimring, 1968, 1972; Cook, 1991; Kel-lermann et al., 1992, 1993). Understanding the relative magnitudes of thesebenefits and costs is crucial for public policy.

Yet there is currently considerable disagreement concerning the numberof defensive gun uses (DGUs) each year. One data source for estimatingthe incidence of civilian DGU is the National Crime Victimization Survey(NCVS), a nationally representative survey of 59,000 households conductedby the Census Bureau for the U.S. Bureau of Justice Statistics (McDowalland Wiersema, 1994). The NCVS attempts to interview each individual age12 or older in eligible households; housing units are retained in the samplefor 3 years, and the residents are interviewed each 6 months. The most recentpublished estimates from NCVS data suggest 108,000 defensive gun usesannually (Cook et al., 1997).

In sharp contrast are the results of a number of relatively small tele-phone surveys conducted in the last 20 years by private polling organizations.After adjusting for differences in methods and coverage, the estimates fromthese surveys range between 770,000 and 3.6 million people using gunsdefensively against humans each year (Kleck and Gertz, 1995). The mostrecent survey is also the first to be designed expressly for examining DGUand has been used to estimate 2.5 million defensive gun users per year (Kleckand Gertz, 1995). The Kleck and Gertz estimate of 2.5 million DGUs eachyear has received considerable attention in public policy debates and isregularly invoked within the research community as well as by media, politic-ians, and even the Congressional Research Service (Bea, 1994).

In 1994 the Police Foundation, under a grant from the National Insti-tute of Justice, conducted a survey on gun ownership and uses in the UnitedStates. The survey includes a sequence of questions on DGU very similarto the one used by Kleck and Gertz and, thus, provides an opportunity tostudy more closely the nature, reliability, and implications of survey-basedestimates for such uses. Our analysis of these data call into question theaccuracy of DGU estimates based on data from general-population surveys.Our analysis also suggests that available survey data are not able to deter-mine whether reported DGU incidents, even if true, add to or detract frompublic health and safety.

In Section 2 we discuss the possible sources of bias in estimating theprevalence of DGU from survey data. Section 3 contains details about thesurvey data that we analyze. In Section 4 we present the results of ouranalysis, followed by a discussion of the implications of our results for futureresearch and public policy.

112 Cook and Ludwig

2. ESTIMATING THE PREVALENCE OF DEFENSIVE GUNUSE (DGU)

Criminologists frequently make use of survey data to overcome thelimitations of administrative records in estimating parameters of interest. Inthe case of civilian DGU, the concern with using official law enforcementrecords is that many such uses go unreported. Some defensive gun usersmay fail to report their gun uses because they own or carry their firearmsin violation of local, state, or Federal law or are uncertain about the legalityof their actions (Kleck and Gertz, 1995). The hope is that such gun userswill be more willing to report their gun uses to survey interviewers than tolaw enforcement authorities and, in turn, that surveys will produce moreaccurate estimates than those derived from police records.

Yet survey estimates may themselves be subject to bias, because somerespondents who have not used a gun defensively may report that they have(a "false positive"), and others who have used a gun defensively may concealtheir use (a "false negative"). Following Hemenway (1997), we describe thebias in a parameter estimate as the ratio of the estimated prevalence dividedby the true prevalence:4

Bias = [estimated prevalence]/[true prevalence]

''This is a slight modification of Hemenway's (1997) formula.'Recent estimates from the National Institute for Mental Health suggest that 51.3 millionAmerican adults have "one or more mental or addictive disorders" (Bourndon el al., 1994).

Defensive Gun Uses 113

In the formula described in Eq. (1), W represents the number of observationsin the sample, A represents the number of people who experience some eventand report accurately (the number of "true positives"), (N-A) representsthe number of people who do not experience the event, and f+ and f- rep-resent the rates of false positives and false negatives, respectively.

Misreporting may be unintentional or intentional. Unintentional misre-porting may occur when respondents forget about the behaviors or experi-ences in question. The degree to which this problem may plague DGUestimates is unclear, though previous research suggests that forgetting is lesslikely with more consequential events (Skogan, 1990). Alternatively, it ispossible that some respondents are confused during the survey. A nationallyrepresentative survey is likely to include many respondents who will becognitively impaired as a result of substance use, mental illness, or dementiaand, hence, unreliable reporters.5

Respondents may also be unclear about how long ago the events trans-pired (which may lead to the well-known phenomenon of "telescoping").In principle, telescoping can cause estimates to be too high or too low.Respondents either may mistakenly report on events that occurred outsideof the recall period or may fail to report events that happened during therecall period but are mistakenly remembered to have occurred outside ofthe time frame in question. In practice, the former seems to dominate. AsSkogan (1990, p. 262) notes, in criminal victimization surveys telescopingcan increase estimates "by between 40 and 50% depending on the time ofcrime; the inflation rate is greatest for violent crimes and those (often moreserious) that were reported to the police." Surveys such as the NCVS attemptto address this problem by exploiting the fact that sampled households areinterviewed every 6 months; the results of the first interview are used to"bound" the reference period that is asked about on the follow-up survey.Telephone surveys that interview respondents only once are obviously notable to make this correction.

The incentives to intentionally misreport will depend on the behavioror experience in question. Some questions may ask respondents to reportillegal behaviors that they or others have engaged in, in which case respond-ents may be further dissuaded from accurately reporting for fear of possiblelegal action if the survey data is shared with law enforcement. For example,previous research suggests that assaults by family or friends are under-reported in surveys (Skogan, 1981). In the case of defensive gun use, somerespondents who actually experience a DGU may have incentives to misre-port intentionally (report a false negative) because of concerns about thelegality or legitimacy of their gun uses. They may have been carrying thegun illegally at the time or involved in drug dealing or other illicit activity.Or they may simply be unsure of whether they had been within their rightswhen they threatened another person with a gun.

On the other hand, previous research suggests that survey respondentswish to make themselves look good in the eyes of the interviewer, which leadsto "social desirability bias" (Sudman and Bradburn, 1974). For example,respondents are likely to overreport socially desirable behaviors such asvoting and underreport socially undesirable behaviors such as bankruptcy(Bradburn et al, 1979). The false-positive rate may be higher with surveyquestions about DGU than with other potentially illegal behaviors or formsof criminal victimization because DGU reports may be subject to positivesocial desirability bias. Unlike victimization or engaging in some illegal activ-ity, successfully using a gun against a criminal attacker may be viewed as aheroic act. For example, since 1932 the NRA's publication AmericanRifleman has published abbreviated newspaper accounts of DGU. Theseaccounts reflect the accolades that defensive gun users receive when they

114 Cook and Ludwig

prevent a crime, including the Kentucky State Police's highest civilian honorand an award from the Ross County (Ohio) Law Enforcement Officers'Association.6 As one newspaper account reported,

An unidentified NRA member became famous throughout Texas as "TheHunter" when he and his son heard a distress call on their CB radio. Two collegecoeds saw a Waco man shoot Sammy Long, a Texas Department of Public Safetyofficer, and called for help. The hunter arrived on the scene too late to saveLong's life, but killed the thug with a 6 mm rifle. Upton County District AttorneyAubrey Edwards said the coeds and the hunter requested their names not bemade public and said the hunter "deserved a medal" for his action.7

Yet as suggested by Eq. (1), the bias of a parameter estimate dependson both the false-positive and the false-negative rates and the true prevalenceof the behavior in the population (that is, the relative number of respondentswho have an opportunity to provide false positives versus false negatives).Hemenway (1997) notes that if an event is rare—that is, A is small relativeto (N-A)—even a low false-positive rate may lead to overestimates of thepopulation prevalence, regardless of how high the false-negative rate is.

For example, suppose that we have a sample of a size similar to thatused in the Kleck and Gertz (1995) survey, with N= 5000. For the sake ofour example, assume that the true prevalence of DGU is 0.001, twice theprevalence rate that has been estimated from recent years of the NCVS.8 Inthis case, we expect 5 respondents to have experienced a DGU and have thechance to report a false negative, while the remaining 4995 respondents havethe opportunity to report a false positive. Suppose that the false-positiverate is 0.01.9 The resulting estimate from this survey will overstate the preval-ence of DGU by a factor of 10 even if the false-negative rate is as high as100% (that is, none of the defensive gun users report their use). What Kleckand Gertz (1997) refer to as "one-sided speculation" about the importance

"Taken from the National Rifle Association home page on the World Wide Web, 11/29/97.Web address: http://www.nra.org/research/armdctzn.html. The two accounts reported aboveoriginally came from the Corbin, Kentucky, Times-Tribune (October 1980) and the Chillicothe,Ohio, Gazette (March 1967).

7Taken from the National Rifle Association home page on the World Wide Web, 11/29/97.Web address: http://www.nra.org/research/armdctzn.html. Original article taken from TheTimes, San Angelo, Texas, February 1977.

8The most recent NCVS-based estimate for the prevalence of defensive gun use is 0.0005 (Cookel al., 1997). This figure is probably somewhat too low, in part because the NCVS does notask respondents a direct question about DGU (Smith, 1997). Also, the NCVS only asksrespondents who report a criminal victimization to report the defensive measures that weretaken, and some DGUs may occur in response to crimes that are not asked about in thevictimization screens.

9By way of comparison, the false-positive rate for self-reported drug use ranged from 0.009for amphetamines to 0.092 for marijuana compared with urinalysis results (Harrison, 199S,Fig. 1).

Defensive Gun Uses 115

of the false-positive rate is in fact warranted given the algebra associatedwith estimating rare events.

Our example also highlights one explanation for at least part of thediscrepancy between the NCVS and the estimates of Kleck and Gertz (1995).Even if the false-positive and false-negative rates are equal across all of thesesurveys, the scope for upward bias is dramatically reduced in the NCVSsince only those respondents who report a criminal victimization are giventhe opportunity to answer questions about DGU.

In sum, there is no direct evidence on the false-positive rate for DGUbut good reason to believe that it is not trivial. Given even a small rate offalse positives, the infrequency of DGU (even by Kleck and Gertz's measure)suggests that surveys are likely to overestimate the prevalence of this eventregardless of the false-negative rate. The next section describes the surveydata we use to test this hypothesis.

3. DATA

In this section, we review the survey design and questions about defens-ive gun use (DGU). The survey questions about DGU are very similar tothose used by Kleck and Gertz. While the response rate of the survey ana-lyzed in this article is somewhat low, it is no lower than that produced inthe Kleck and Gertz survey.

3.1. Survey Design

The National Study of Private Ownership of Firearms (NSPOF) wasa nationally representative, random-digit-dial telephone survey. Interviewswere conducted during November and December of 1994 by ChiltonResearch Services.

The instrument itself was designed by the Police Foundation. Chiltoninterviewers and supervisors attended a training session that provided back-ground on the purposes of the study and a review of each question in thesurvey instrument; interviewers also participated in practice interviews underthe supervision of senior Chilton staff. The results of 10 pretest interviewsled to a few minor changes to the survey instruments. During the study,approximately 10% of all interviews were randomly selected to be monitoredby senior Chilton staff.

Households were selected by first choosing a central telephone office,characterized by the first six digits of the telephone number, and then ran-domly selecting a household from each central office. Sampling quotas weredefined on the basis of race (Hispanic, African-American, other) and gun

116 Cook and Ludwig

Defensive Gun Uses

ownership. The 6% of households without telephones are not in the sampleframe for the NSPOF.

Each selected telephone number was scheduled to receive as many callsas needed to complete the interview, up to a maximum of six. Using thelast-birthday method, an adult (age 18 or older) was selected to be inter-viewed from each contacted household. Households who initially refusedwere recontacted by Chilton interviewers who had unusual skill in convertingrefusers into participants. Respondents who were not fluent in English, butwere fluent in Spanish, were recontacted by a Spanish-speaking Chiltoninterviewer.

The sample disposition is shown in Table I. Of the 29,917 telephonenumbers that were randomly selected, 9615 were ineligible because the num-bers were not working or not residential, and in 6333 cases Chilton deter-mined that the sample cell quota had been filled on the basis of the initialsurvey questions and terminated the interview. (Data from partially com-pleted surveys were not used in this analysis.) In 4724 cases Chilton wasunable to reach a person, and in 5977 cases Chilton contacted someone butwas unable to obtain information on the screening questions to determineeligibility. We used the Council of American Survey Research Organization'sdefinition for the response rate to calculate a figure of 47% (White, 1983).

While this response rate is somewhat low, there is no reason to believethat the NSPOF is a less representative sample than other telephone surveys

117

Table I. Sample Disposition and Response Rate"

Total telephone numbers

Total ineligible numbersNonworking, nonresidential, language barrier, otherSample quota filled

Total numbers for which eligibility unknownNo answer/busyAnswering machineBreakoffs

Total eligible numbersCompleted surveysRefusals

Response ratep=(No. eligibles)/(No. eligibility determined)R = (No. completed surveys)/[(No. eligibles) + (p) . (No. eligibility unknowns)]

29,917

15,9489,6156,333

10,7013,0391,6855,977

3,2682,568

700

20.5%47.0%

aResponse rate calculation comes from the definition developed by the Council of the AmericanSurvey Research Organization (White, 1983). Breakoffs are cases in which Chilton contacteda person but was unable to ask enough questions to determine the household's eligibilitystatus.

that have been used to study DGU. For example, Kleck and Gertz (1995)report a response rate of 61% for the survey used in their analysis, wherethe response rate is defined as the number of households willing to participatedivided by the number of completed calls to eligible households. If we fol-lowed this procedure, Chilton's response rate would compare favorably withtheirs: of the 3268 numbers in which Chilton contacted a person and deter-mined that the household was eligible, surveys were completed in 79% ofthe cases.

In order to produce nationally representative estimates, Chilton pro-vided population projection weights which adjust for the sampling method.The weights were calculated using control variables for age, race, sex, educa-tion, household income, and the number of adults in each household.Because households with multiple working telephone lines had a higherprobability of selection, we divided the Chilton weights by the number ofphone lines in the home. Since the average number of phone lines persampled household is greater than 1, the Chilton population weights dividedby the number of phone lines per household no longer sum to the entireU.S. population. In order to correct for this, we multiplied all of the phoneline-adjusted weights by 1.0946 so that the sum of the weights equals theU.S. adult population.

All of our results were calculated using the sampling weights. Theweights differ widely across the NSPOF sample, which serves to increase thestandard errors of our point estimates compared to a uniformly weighted(simple) random sample of the same size. Our calculations suggested thatthe appropriate standard errors in our analysis are 1.435 times the standarderrors that would be derived if the NSPOF were a simple random sample(Mendenhall et al., 1990).

Table II presents descriptive statistics of the NSPOF sample and, forcomparison, estimates for the U.S. adult population (ages 18 and over) fromthe U.S. Census (U.S. Bureau of the Census, 1995).

3.2. DGU Questions

The NSPOF survey instrument begins with 53 questions on topics suchas perceptions about local crime trends, attitudes toward a handgun registra-tion law or a handgun ban, household and personal gun ownership, guncarrying, and recent victimization experiences (for the crimes of robbery andburglary).

Each respondent in the NSPOF was then asked the question, "Withinthe past 12 months, have you yourself used a gun, even if it was not fired,to protect yourself or someone else, or for the protection of property athome, work, or elsewhere?" Respondents who said yes were then asked,

118 Cook and Ludwig

Defensive Gun Uses

"How many different times did you use a gun, even if it was not fired, toprotect yourself or your property in the past 12 months?" (emphasis inoriginal). Respondents who did not report a DGU during the past yearwere then asked, "Have you ever used a gun to defend yourself or someone

119

Table II. Characteristics of the NSPOF Samplea

Percentage of respondents who own gun

Male

RaceWhiteBlackHispanicOther

Income$0 to <10K$10 to <15,000$15 to <50,000$50,000 plus

Educational attainmentLess than high schoolHigh school or some collegeCollege or more

R has been arrested for non-traffic offense

Marital statusNever marriedMarriedWidowedDivorced/separated

HHs with children <18 living in HH

Census regionNortheastMidwestSouthWest

1994NSPOF

(N=2568)

24.6

47.5

79.611.54.24.7

11.46.7

48.933.0

21.355.723.0

6.1

19.864.85.89.6

41.7

20.224.234.820.8

Censusestimate

N/A48.8

76.511.58.04.0

8.76.9

N/A37.1

19.158.822.2

N/A

23.360.67.09.2

35.0

19.723.634.821.8

aDescriptive statistics calculated using NSPOF sampling weights divided bynumber of telephone lines in household (to control for higher probabilitiesof selection for multiple-telephone line households) ; because the averagenumber of telephone lines per household in the sample was greater than one,we multiplied each of the weights by the appropriate number (1.0946) toensure projection to the U.S. adult population. Census estimates from theStatistical Abstracts (U.S. Bureau of the Census, 1995). Also note that theincome categories provided in the NSPOF and the Statistical Abstractsbetween $15,000 and $50,000 do not correspond, hence no comparisons arepossible.

else?" Respondents who had reported a DGU either in the past year or everwere asked, "Let's talk about the most recent incident. About how long agowas that?" The responses to this question allowed us to identify those gunuses that reportedly occurred within the past 5 years.

Respondents who reported a DGU experience are then asked 30 addi-tional questions concerning the most recent such experience. Topics coveredinclude whether the use was against an animal or a human, the relationshipbetween the respondent and the perpetrator, the location of the incident,the crime involved, whether the perpetrator was armed, and what therespondent did with the firearm in the incident. Chilton interviewers wereasked to provide their own assessment of whether the respondent was invent-ing the most recent gun use incident.

By way of comparison, in the Kleck and Gertz (1995) survey, respond-ents were asked (p. 161): "Within the past five years, have you yourself oranother member of your household used a gun, even if it was not fired, forself-protection or for the protection of property at home, work, or elsewhere?Please do not include military service, police work, or work as a securityguard" (emphasis in original).

In sum, the NSPOF is a state-of-the-art RDD telephone survey. Com-pared with the survey employed by Kleck and Gertz (1995), the NSPOFDGU questions are very similar and the response rates are comparable.Because the survey conducted by Kleck and Gertz was conducted for theexpress purpose of examining DGU, only those respondents who reporteda DGU plus a randomly selected third of other respondents were asked tocomplete the full survey questionnaire. The result is that 4977 respondentswere asked whether they had experienced a DGU, of whom 1832 completedthe full survey. Because the NSPOF was intended to be a comprehensivesurvey of gun ownership and use, all eligible respondents were asked tocomplete the survey questionnaire. The tradeoff is that a smaller number ofpeople reported on DGUs in the NSPOF relative to the Kleck and Gertzsurvey (2568 versus 4977), though a larger number of respondents in theNSPOF answered broader questions about gun ownership, acquisition, anduse.

4. RESULTS

While the survey responses to the NSPOF imply defensive gun use(DGU) estimates closer to those of Kleck and Gertz than to those producedusing the NCVS, we find evidence to support our earlier hypothesis thatpopulation-based surveys of DGU are subject to upward bias.

120 Cook and Ludwig

Defensive Gun Uses 121

4.1. DGU Prevalence

A total of 54 respondents in the NSPOF reported a DGU during thepast 12 months, of which 50 were against a human. After excluding the 5cases in which Chilton interviewers thought the respondent was inventingthe incident, the 45 remaining respondents who report a gun use against aperson in the past year implied 3.1 million defensive gun users per year, witha 95% confidence interval ranging from 1.8 to 4.5 million (Table III).10

For comparability with the most recent published DGU survey of Kleckand Gertz (1995), we made a number of additional reasonable exclusionsand are left with just 19 respondents. The exclusions included respondentswhose most recent DGU was part of military or protective service work,respondents who did not include a report of a specific crime, respondents

'"Unfortunately, Chilton was unable to provide documentation for the specific instructionsthat were given to interviewers for identifying untruthful DGU reports. As far as we know,there is no research on whether social science interviewers are able reliably to identifyintentional misreporting on social science surveys. We exclude the five cases in which a respond-ent reported a DGU during the past year and the interviewer reported possible misreportingin order to maintain comparability with Kleck and Gertz (1995), who also take the approachof eliminating DGU cases from their survey with "any indication that the incident might notbe genuine" (p. 163). As shown in the Appendix, two of the five cases in which the NSPOFinterviewer suspected misreporting meet at least one of our other criteria for exclusion.

Table III. DGU Estimates for 1- and 5-Year Recall Period: Comparison of NSPOF with KGa

1-year recall

No. respondents who report a DGU

Estimated No. defenders (millions)(95% confidence interval)

No. defenders as percentage of population(95% confidence interval)

Estimated No. DGUs (millions)(95% confidence interval)

NSPOF:all DGUs

againsthumans

45

3.12(1.77-4.54)

1.64(0.92-2.36)

23.0(12.9-33.1)

SelectedNSPOF DGUsagainst humansafter applying

exclusion criteria

19

1.46(0.52-2.44)

0.77(0.27-1.27)

4.7(1.2-7.9)

Kleck & Gertz(1995)

"Type A"DGU

estimate

66

2.55(2.09-3.14)

1.33(1.01-1.65)

2.55(2.09-3.14)

aThe second and third columns both exclude the five defensive gun uses that reportedly occurredwithin the past year but were suspected to be false by the Chilton interviewers. Estimatesbased on exclusion criteria include only those DGUs that are against humans, do not occuras part of protective service work, and in which the respondent saw a perpetrator, includeda specific crime as part of the description, and reported using the gun in some way. Populationprojections are calculated by multilying prevalence estimates from the NSPOF by the adult(18 and over) population in the United States in 1994.

who did not use the firearm (or even mention it to the perpetrator), andrespondents who did not actually see a perpetrator. (The numbers of casesexcluded by each restriction are shown in the Appendix.) As shown in TableIII, these 19 reports implied that 0.8% of American adults (1.5 million) wereinvolved in a DGU in 1994; the 95% confidence interval ranges from 0.3 to1.5%, implying 0.6 to 2.9 million DGUs. The operational definition onwhich this estimate is based is similar to that used by Kleck and Gertz(1995), and in fact the 95% confidence intervals overlap (Table III).

Data from the 45 respondents who report having used a gun defensivelyagainst a human during the year preceding the survey imply 23.0 millionDGUs. We modeled the random variable indicating the number of DGUsreported by a respondent as the product of two random variables: an indi-cator of whether or not the respondent reports any DGUs (governed by thebinomial distribution) and the number of uses reported by DGU reporters(governed by the Poisson distribution). Applying the formula for the vari-ance of the product of two random variables (Becker, 1971, p. 180) suggestsa 95% confidence interval that ranges from 12.9 to 33.1 million. Of the 19defensive gun users who met the Kleck and Gertz-type exclusion criteria, 7reported more than 1 use per year, with an average of 6.7 uses per year; 1respondent reported 20 uses. Taken together, the responses of the 19 usersimplied 4.7 million uses (with a confidence interval of 1.2 to 7.9 million). Incomparison, Kleck and Gertz (1995) find that respondents in their surveywho report a DGU within the past 5 years report an average of about 1.5uses. (They do not ask about frequency of use during the 1-year recallperiod.) If we exclude the NSPOF respondent who reported 20 uses, theaverage number of uses during the past year by defensive gun users in theNSPOF would be about 2.

We also calculated results for the prevalence of DGU involvement dur-ing the 5 years preceding the survey by focusing on those DGUs that werereported to occur 5 or fewer years ago. (Respondents were not asked aboutthe frequency of such involvement.) Our calculations suggested that 4.11%of the U.S. adult population has experienced a DGU during the 5 yearspreceding the survey (with a confidence interval of 2.97 to 5.25%). Afterapplying our exclusion restrictions, we estimated that 1.71% of Americanadults have used a gun defensively (with a 95% confidence interval rangingfrom 0.97 to 2.45%).

4.2. Comparisons with Other Measures

The NSPOF produces estimates for the prevalence of DGU that areconsistent with those produced by Kleck and Gertz (1995), though whetherthese results are unbiased is an open question in light of our earlier discussion

122 Cook and Ludwig

of the possibility and influence of false-positive reports. One way to explorethe credibility of the estimates of DGU from our survey is to compare themwith related estimates from various other sources.

First, based on the NSPOF our best estimate of the number of DGUsin 1994 is 4.7 million, which is 35 times larger than the corresponding esti-mate based on the U.S. Bureau of Justice Statistics' National Crime Victimiz-ation Survey (NCVS). As noted earlier, one difference between the twosurveys is that the NCVS asked only questions concerning self-defense ofrespondents who first report that they have been victimized. In contrast, theNSPOF asked questions about gun self-defense of all respondents, includingthose who said no to the two victimization questions: "During the past 12months—that is, since last November—has anyone robbed, tried to attack,or attacked you?" and "During the past 12 months, did anyone break intoor somehow illegally get into your home?" Of the 19 respondents who wecounted as defensive gun users, 7 said yes to either of these questions. Ofthe 12 respondents who said no to both of these victimization questions butlater reported a self-defense gun use, all but 4 indicated that the crime againstwhich they were defending was robbery, rape, assault, or a break-in.

The number of crimes that are reportedly defended with a gun is difficultto reconcile with the number of crimes estimated from victimization surveys.Respondents were asked to describe the crime or crimes involved in theirmost recent defensive gun use. The DGU reports in the NSPOF impliedpoint estimates for the number of crimes defended with a gun (and 95%confidence intervals) of 322,000 rapes (12,000-632,000), 462,000 aggravatedassaults (101,000-562,000), and 527,000 robberies (130,000-924,000). Byway of comparison, NCVS data estimated that the total number of rapes(including attempts) was 316,000, that of aggravated assaults was 2.48million, and that of robberies was 1.30 million in 1994 (Bureau of JusticeStatistics, 1996). The NSPOF estimate of the number of defensive gun usersthus implies that every rape or rape attempt was thwarted by a DGU, aswas one of five aggravated assaults and two of five robberies. Even if theNCVS-based estimates of criminal victimization rates are off by an order ofmagnitude, the NSPOF-based estimates for DGU are implausible.

We can also make this comparison using victimization estimates basedon the victimization questions (cited above) in the NSPOF. These turn outto be quite close to the 1994 NCVS estimates: 10.4 million robberies,assaults, and attempted assaults (10.8 million if we also include those crimesreported in the DGU sequence but not captured by the victimization ques-tions) versus 10.9 million in the NCVS, and 5.0 million burglaries versus5.5 million in the NCVS. In comparison, the number of self-defense gunuses reported in the NSPOF against an assault (attack or fight), rape, orrobbery in 1994 was 938,000.

Defensive Gun Uses 123

Each defensive gun user is asked to assess the likelihood that someonewould have been killed had a gun not been used defensively. The resultsimplied that 629,000 lives were "likely," "somewhat likely," or "very likely"saved through the DGU (95% confidence of 196,000 to 1,062,000). In com-parison, data gathered by the FBI indicated 21,100 actual homicides in 1994(Federal Bureau of Investigation, 1995). Even if we assume that only 1 ofevery 10 respondents was correct in believing that the gun use saved alife, the NSPOF estimates would imply that around three-quarters of allpotentially fatal attacks are prevented by a gun-wielding citizen. More likely,either respondents are inventing the DGUs that they report or people whouse guns against people are very poor at judging the danger that they orothers face. The latter possibility calls into question the social benefits ofthese gun uses, a point to which we return below.

Finally, Kleck and Gertz have argued that comparisons of survey-basedDGU estimates with the number of crimes implied by victimization surveysare invalid because "a large share [of DGUs] are probably outside the scopeof incidents that realistically are likely to be reported to either the NCVSor police" (Kleck and Gertz, 1995, p. 167).11 In the NSPOF, we find that49% of respondents who report a DGU also indicate that the police foundout about the incident. The NSPOF estimates thus implied that law enforce-ment authorities find out about 750,000 DGUs each year, including 526,000uses that were in the context of a serious violent crime.

In particular, based on the NSPOF we estimated that there are 265,000attempted rapes each year in which the victim used a gun and the policewere notified; we estimated that an additional 141,000 attempted robberieswith gun self-defense were reported to the police. But according to the FBI,the total number of robberies known to the police in 1994 was 619,000,which, together with the NSPOF results, seems to imply that 23% of allrobberies known to the police involved the victim defending himself with agun; and the total number of rapes (including attempts) known to the policein 1994 was 102,000, implying that the number of gun uses against rapistsknown to the police is 2.6 times the number of rapes or attempted rapesknown to the police (Federal Bureau of Investigation, 1995). Another sourcefor estimating the number of robberies and rapes known to the police is theNCVS. According to NCVS estimates from 1994, a total of 719,000 robberiesand 137,000 rapes, attempted rapes, or sexual assaults were reported to thepolice (Bureau of Justice Statistics, 1996). In any case, the comparisons forrobberies are implausible, while the comparisons for rapes are impossible.

4.3. Inconsistencies in Gun Use Reports

Another way to examine the reliability of DGU responses is to examinethe reports for internal consistency. Kleck and Gertz (1997, p. 1449) suggest

"See also Kleck and Gertz (1997, p. 1452) and Kleck (1997, pp. 158-159).

124 Cook and Ludwig

that false positives are unlikely with their own survey of DGU because of thedifficulties respondents face in "provid [ing] as many as nineteen internallyconsistent responses covering the details of the alleged [DGU] incident."While Kleck and Gertz (1995) do not report on the internal consistency ofthe DGU reports in their survey, we find that a large share of the DGUreports with the NSPOF contains what appears to be some form of internalcontradiction (see Table IV).

Each respondent was asked "Which of the following best describes whatwas happening when you used the gun defensively?" The respondent wasthen read the following list, one at a time, to which the respondent providesa yes or no answer: rape, attempted rape, other sexual assault; some otherkind of attack or threat of violence; burglary or theft at home withoutthreat; robbery—someone took or tried to take something with the threator use of force; some other kind of theft or attempted theft; a fight; trespass-ing; and no crime was involved. Respondents were also asked, "Did theperpetrator threaten, attack, or injure you?"

In 8 of the 19 DGUs that met our exclusion criteria, the respondentreported that a serious crime (rape or robbery) was involved. Yet in threeof the eight incidents the respondent also included "no crime involved" aspart of the incident description. Further, in 6 of the 14 cases in which therespondent indicated that the DGU involved a rape, robbery or attack, therespondent also indicated that the perpetrator did not attack or even issuea threat (Table V). Taken together, 7 of the 19 reported gun uses containedat least one of these apparent inconsistencies.

It seems unlikely that these contradictions can be explained by codingerrors because senior Chilton staff monitored interviews and response codingfor a random 10% of calls throughout the study and these apparent inconsist-encies arise in such a large proportion of DGU reports (over a third). It ispossible that some of these respondents are referring to serious crimes thatthey believe to have been preempted by the presence of a gun (that is, a gun

Defensive Gun Uses 125

Table IV. Inconsistencies in DGU Reports

Total reports that contain at least one inconsistencyReport contains Inconsistency 1 : Respondent reports that serious crime (rape

or robbery) was involved, and also reports that "no crime was involved"Report contains Inconsistency 2: Respondent reports that serious crime (rape,

robbery or attack) was involved and also reports that perpetrator neitherattacked nor issued threat

Report contains both Inconsistency 1 and Inconsistency 2

DGU report contains neither inconsistency

Total number of DGU reports

Number ofcases

7

1

42

12

19

Cook and Ludwig

prevented the attack or threat from occurring). This possibility raises thequestion of whether brandishing a gun against someone who has not evenissued a threat should be counted as a social benefit, a subject to which wereturn below. At the very least, these apparent contradictions are consistentwith the idea that a sizable share of DGU reporters are falsifying part oftheir accounts.

5. DISCUSSION

This paper used survey methods similar to those employed in a recentwidely cited study by Kleck and Gertz (1995) and produced comparableresults; yet our comparison of estimates based on NSPOF with othersources, together with puzzling inconsistencies in over a third of the defensivegun use (DGU) reports, lead us to conclude that the estimates are far toohigh. We conclude with a discussion of what information these kinds ofDGU estimates (even if they were unbiased) can provide about the socialbenefits of widespread gun ownership. We then offer some suggestionson how future surveys of DGU might produce more reliable and usefulestimates, as well as some concluding thoughts.

5.1. Interpreting Gun Use Reports

Estimates for the prevalence of DGU have been used in argumentsagainst restrictions on gun ownership and carrying. While we will not evalu-ate such policies here, it is important to note that the available survey data

126

Table V. Reported Type of Crime and Use of Force byPerpetrator in 19 DGUs"

Crime?(most serious) "

RapeRobberyAttackBurglaryTheftTrespass

Threat

113011

Attack?

Attack

012000

Neither

231102

"The columns classify DGUs on the basis of theiranswers to question 75 "Did the perpetrator threaten,attack, or injure you?" The rows classify DGUs on thebasis of their answers to question 72, "Which of thefollowing best describes what was happening when youused the gun defensively?" Q72 permitted multipleanswers; each DGU is categorized by the most seriouscrime mentioned, with the hierarchy of "seriousness"defined by the order in which the crime types are listed.

provide little information about whether DGUs provide a net benefit tosociety or whether they are legally or morally justifiable. Even if the respond-ent is providing an accurate report, it is typically quite difficult to determinewhether he or she acted appropriately.

As shown in Table IV, in 5 of the 19 gun use reports that met ourexclusion criteria, the most serious crime involved was a fight or attack. TheDGU report in these cases is the testimony of one party to an encounter.We are given little context with which to judge these reports and, of course,have no way to learn the other party's view of what happened.

Further, alternative courses of action available to the respondent cannotbe determined from these surveys. Claims about the benefits of DGU assumethat the firearm produces an outcome that is preferable to some (unob-served) alternative sequence of events. Citizens with access to firearms mayuse their guns in situations where other effective responses may be available.

For example, in 6 of the 19 gun uses that met our exclusion criteria,the respondent reported that the encounter occurred "near the respondent'shome" and also indicated that, when he first wanted to use his gun forprotection, the gun was stored somewhere in the home. In other words, inabout one-third of these events the respondent apparently had the optionof staying inside and calling the police.

Finally, DGU estimates cannot provide information about other impor-tant consequences of widespread gun ownership and carrying. Higher ratesof gun ownership may increase the likelihood that criminals arm themselves.On the other hand, widespread gun ownership and carrying may deter crimi-nal activity. Neither of these effects is reflected in DGU estimates.

5.2. Suggestions for Future Research

Below we offer three suggestions for future survey work on DGU thatmay improve our understanding of the prevalence and nature of this event.

First, greater attention should be devoted to screening out false positiveswhen attempting to estimate the prevalence of rare events. As suggested bySection 2 in this paper, the false-positive problem is not limited to studiesof DGU; for example, analysis of the NSPOF data suggested 315,000unintentional gunshot woundings per year, much higher than the 17,000unintentional gunshot wounds treated in hospital emergency rooms eachyear (Sinauer et al., 1996). One possibility for screening out false positivesin the DGU application might be to build more redundancies into thesequence of questions about the details of the encounter. Another possibilitywould be to attempt to match survey reports with official police records inthose cases in which the respondent reports that law enforcement were noti-fied about the gun use. Unfortunately this approach would require survey

Defensive Gun Uses 127

participants to identify themselves and could only be performed for a subsetof reported DGUs. Yet another possibility would be to include questionsabout DGU that are asked of every respondent in a longitudinal survey, togauge the effects of telescoping with one-time surveys (Smith, 1997).

It is worth noting that the NCVS asks DGU questions only of thoserespondents who report a victimization. This method dramatically reducesthe scope of the false-positive problem, though at the cost of excluding thosegun uses associated with crimes omitted from the NCVS crime list (such astrespassing). Of the 19 DGU reporters in the NSPOF, 7 answered yes toquestions about criminal victimization that came earlier in the survey. Thussomething like two-thirds of the difference between the NSPOF- and theNCVS-based DGU estimates may be accounted for by the fact that theNCVS asks only self-defense questions of victims. But even with that adjust-ment, the NSPOF estimate is an order of magnitude greater than the NCVSestimate. It would be helpful to know whether more vigorous attempts toidentify false positives in both surveys would reduce the discrepancy yetfurther.

Second, surveys should include questions about the sequence of eventsleading up to the gun use. This information may help researchers understandboth the respondent's role (if any) in precipitating the incident and alterna-tive courses of action available to the respondent prior to using a gun in theencounter.

Finally, additional information is required about the exact nature ofthese events and how the survey responses differ from what a neutralobserver would report. It may be useful to conduct a study that starts witha sample of gun encounters that are reported to the police and then conductfollow-back surveys of every participant in the encounter. The gun uses thatare reported to the police will not be representative of all gun uses, buta study along these lines would, nevertheless, substantially improve ourunderstanding of the social benefits or costs implied by DGU reports inhousehold surveys.

5.3. Conclusion

Survey estimates for the annual number of DGUs have been offered asa measure of the protective value of private gun ownership and carrying.Because several recent estimates from household survey data suggest thatthere are millions of DGUs each year, some have argued that widespreadgun ownership and carrying are effective in reducing injury from criminalvictimization. It is therefore important for public officials to be aware thatestimates of the prevalence of DGU based on data of the sort analyzed hereappear to suffer from a large positive bias and greatly overstate the preval-ence of DGU. Further, these data do not provide sufficient information to

128 Cook and Ludwig

Defensive Gun Uses 129

distinguish between virtuous and objectionable uses. Hence these estimatescontribute little to evaluating the benefits of widespread gun ownership andcarrying.

APPENDIX

Table AI. Restrictions for DGUs Using 1-Year Recall Period"

Total DGUs against humans in past year reported by NSPOF respondents

Reason for exclusionChilton interviewer thought R inventing incident ("invented")Invented AND R did not report any crime as part of incident ("no crime")Invented AND R did not report using gun in any way as part of incident ("gun

not used") AND no crime AND R used gun as part of law enforcement/protective service work ("police use")

R reports he/she did not see perpetrator ("not see perp")Gun not usedR not see perp AND no crimeGun not used AND no crimeGun not used AND not see perpGun not used AND no crime AND police useGun not used AND no crime AND invented

Total cases excluded

Total gun uses against humans after "restrictive exclusions"

No. cases

50

31

16711551

31

19

"The exclusions reported above (with the exception of "invented") follow the method reportedby Kleck and Gertz (199S). See text for additional details.

ACKNOWLEDGMENTS

This research was sponsored in part by a grant from the NationalInstitute of Justice to the Police Foundation, though the views reflectedherein do not represent those of either funding agency. The data upon whichthis study was based have been archived with the Interuniversity Consortiumfor Political and Social Research (ICPSR) at the University of Michiganand are available to the public according to the usual ICPSR regulations.Thanks go to Brian Komar for valuable research assistance and to EarlHamilton, David Kennedy, Dave Lambert, Michael Maltz, James Mercy,Lois Mock, Wesley Skogan, Eugene Volokh, and three anonymous reviewersfor helpful comments. Any remaining errors are our own.

REFERENCES

Bea, K. (1994). Issue Brief: Gun Control, Congressional Research Service, Washington, DC(Order Code IB94007; revised Sept. 19).

Becker, G. (1971). Economic Theory, Alfred Knopf, New York.Bourndon, K., Rae, D., Narrow, W., Manderscheid, R., and Regier, D. (1994). National

prevalence and treatment of mental and addictive disorders. In Mental Health, UnitedStates, 1994, U.S. Department of Health and Human Services, Center for Mental HealthServices, Washington, DC, pp. 22-35.

Bradburn, N., and Sudman, S. (1979). Improving interview Method and Questionnaire Design,Jossey-Bass, San Francisco.

Bureau of Justice Statistics (1996). National Crime Victimization Survey: Criminal Victimization1994, U.S. Department of Justice, Washington DC.

Cook, P. J. (1985). The case of the missing victims: Gunshot woundings in the National CrimeSurvey. J. Quant. Criminal. 1: 91-102.

Cook, P. J. (1991). The technology of personal violence. In Tonry, M. (ed.), Crime and Justice:An Annual Review of Research, Vol. 14, University of Chicago Press, Chicago, pp. 1-71.

Cook, P. J., Ludwig, J., and Hemenway, D. (1997). The gun debate's new mythical number:How many defensive uses per year? /. Policy Anal. Manage. 16(3): 463-469.

Federal Bureau of Investigation (1995). Uniform Crime Reports for the United States, 1994,U.S. Government Printing Office, Washington, DC.

Harrison, L. D. (1995). The validity of self-reported data on drug use. J. Drug Issues 25(1):91-111.

Hemenway, D. (1997). Survey research and self-defense gun use: An explanation of extremeoverestimates. J. Crim. Law Criminal. 87(4): 1430-1445.

Kellermann, A. L., Rivara, F. P., Somes, G., Reay, D. T., Francisco, J., Banton, J. G., Prodzin-ski, J., Rligner, C, and Hackman, B. B. (1992). Suicide in the home in relation to gunownership. N. Engl. J. Med. 327: 467-472.

Kellermann, A. L., Rivara, F. P., Rushforth, N. B., Banton, J. G., Reay, D. T., Francisco, J.T., Locci, A. B., Prodzinski, J., Hackman, B. B., and Somes, G. (1993). Gun ownershipas a risk factor for homicide in the home. N. Engl. J. Med. 329: 1084-1091.

Kleck, G. (1988). Crime control through the private use of armed force. Sac. Problem 35(1):1-21.

Kleck, G. (1991). Point Blank: Guns and Violence in America, Aldine De Gruyter, New York.Kleck, G. (1997). Targeting Guns: Firearms and Their Control, Aldine de Gruyter, New York.Kleck, G., and Gertz, M. (1995). Armed resistance to crime: The prevalence and nature of

self-defense with a gun. J. Crim. Law Criminal. 86(1): 150-187.Kleck, G., and Gertz, M. (1997). The illegitimacy of one-sided speculation: Getting the defen-

sive gun use estimate down. J. Crim. Law Criminal. 87(4): 1446-1461.McDowall, D., and Wiersema, B. (1994). The incidence of defensive firearm use by U.S. crime

victims, 1987 through 1990. Am. J. Public Health 1982-1984.Mendenhall, W., Wackerly, D., and Scheaffer, R. (1990). Mathematical Statistics with Applica-

tions, 4th ed., PWS-Kent, Boston.Sinauer, N., Annest, J. L., and Mercy, J. A. (1996). Unintentional, nonfatal firearm-related

injuries: A preventable public health burden. JAMA 275(22): 1740-1743.Skogan, W. (1981). Issues in the Measurement of Victimization, U.S. Department of Justice,

Bureau of Justice Statistics, Washington, DC.Skogan, W. (1990). The National Crime Survey Redesign. Public Opin. Q. 54: 256-272.Smith, T. W. (1997). A call for a truce in the DGU war. J. Crim. Law Criminal. 87(4): 1462-

1469.Sparks, R. F. (19xx). Surveys of victimization—An optimistic assessment. In Tonry, M., and

Morris, N. (eds.), Crime and Justice: An Annual Review of Research, Vol. 3, University ofChicago Press, Chicago, pp. 1 -60.

Sudman, S., and Bradburn, N. M. (1974). Response effects. In Surveys: A Review and Synthesis,Aldine, Chicago.

130 Cook and Ludwig

U.S. Bureau of the Census (1995). Statistical Abstract of the United States: 1995, 115th ed.,U.S. Government Printing Office, Washington, DC.

White, A. (1983). Response rate calculation in ROD telephone health surveys: Current prac-tices. In Proceedings of the American Statistical Association, Section on Survey ResearchMethods, American Statistical Association, Washington, DC, pp. 277-282.

Zimring, F. E. (1968). Is gun control likely to reduce violent killings? Univ. Chicago Law Rev.35:721-737.

Zimring, F. E. (1972). The medium is the message: Firearm calibre as a determinant of deathfrom assault. J. Legal Stud. 1: 97-124.

Defensive Gun Uses 131