Forensic Laboratory Independence, Control, and

Exonerations

Patrick L. Warren

May 20, 2015

Abstract

The relationship between forensic laboratories and the other institutions of

law enforcement varies widely over space and time in the United States. Some

jurisdictions have their own local lab within the police or sheriff’s department,

others depend on a statewide lab system either independent or under the state

police, and others still contract with a private lab to process their forensic

evidence. These different organizational forms may shift the incentives lab

technicians and managers have to provide timely and accurate analysis and

testimony. In this paper, I investigate the relationship between one particu-

lar institutional variant, local police/sheriff control of the crime lab, and one

particular outcome in the criminal-justice chain: the conviction of innocent de-

fendants due, in part, to faulty or misleading forensic testimony. Among the

200 largest counties in the U.S., counties with locally controlled labs have lower

rates of exonerations in which faulty forensics were implicated in the original

trial than similarly-situated counties without locally controlled labs that have

similar rates of exonerations where forensic problems were not implicated. This

difference is robust to state fixed-effects, a variety of observable characteristics,

and nearest-neighbor matching. The difference seems to be driven by control,

per se, as matched counties containing state-controlled labs have significantly

higher exoneration rates.

JEL Classification:

Keywords: Forensics, Law Enforcement, Exoneration,

1

1 Introduction

In the past decade, the forensic science community has been buffeted by a number

of scandals. In Massachusetts in 2013, state chemist Sonja Farak pled guilty to tam-

pering with drug evidence, potentially affecting 60,000 samples in 34,000 cases. In

2009, an audit of the Houston Police Department crime lab’s fingerprint unit found

irregularities in over half of the 548 cases reviewed.1 In 2010, lab workers in the North

Carolina state forensic lab were found to have failed to turn over potentially exculpa-

tory evidence, including in death penalty cases. The investigation of the State Bureau

of Investigation blood serology unit yielded a total of 229 cases of misrepresentation

of blood serology. Of the 229 cases, seven persons had been executed, others were on

death row, and some had died in jail.2

In the modern courtroom, forensic evidence plays a very important role. Trials

for defendants facing probative forensic evidence are more likely to end in a con-

viction and result in longer sentences (McEwen 2011). But this impact depends on

reliable testimony, and the anecdotes above suggest the high levels of reliability are

not guaranteed.

Almost simultaneous to the scandals, the National Research Council of the Na-

tional Academy of Science issued a report entitled “Strengthening Forensic Science in

the United States: A Path Forward”, in which a distinguished panel of scientists lays

out a number of recommendations to improve the accuracy and reliability of forensic

science in the United States (Committee on Identifying the Needs of the Forensic

Sciences Community, National Research Council 2009). The first of these recommen-

dations was to foster independent forensic organizations. In their words “[the current

system] leads to significant concerns related to the independence of the laboratory

and its budget. Ideally, public forensic science laboratories should be independent

of or autonomous within law enforcement agencies. In these contexts, the director

would have an equal voice with others in the justice system on matters involving the

laboratory and other agencies.”

The relationship between forensic laboratories and the other institutions of law

enforcement varies widely over space and time in the United States, with wide dif-

ferences in the levels of independence. Some jurisdictions have their own local lab

1“Ex-Chemist Pleads Guilty in Drug Evidence Theft,” Milton J. Valencia, Boston Globe, January7, 2014

2“Scathing SBI Audit Says 230 Cases Tainted by Shoddy Investigations,” Mandy Locke, JosephNeff, and J. Andrew Curliss, Raleigh News and Observer, August 19, 2010.

2

within the police department, sheriff’s department, or the county attorney’s office,

others depend on a statewide lab system either independent of or under the state

police, and others still contract with a private lab to process their forensic evidence.

These differences in organizational form may shift the incentives lab technicians and

managers have to provide timely and accurate analysis and to invest in the resources

necessary to do so. That shift could affect outcomes throughout the criminal-justice

process: crime rates, arrest rates, conviction rates, and exoneration rates.

The theoretical relationship between independence and performance is ambiguous.

Pressures placed on a lab by the police and prosecutor in an adversarial criminal-

justice system can potentially bias the outcomes of their investigations. Concerns

about this effect have dominated the discussion of lab independence in the law and

economics literature (Giannelli 1997, Koppl 2005, Koppl 2010). But there are forces

in the other direction that are, perhaps, under-appreciated in the policy discussion.

When good performance is difficult to specify contractually and/or coordination is

important, direct control can improve outcomes. In fact, many of our theories of

why firms exist, at all, turn on some variant of this idea (Gibbons 2005), and there

is vast empirical support for it (Lafontaine and Slade 2007). In brief, direct control

can provide stronger incentives and improve coordination, but strong incentives to

respond to the principal’s demands can increase both good and bad behavior and

not all coordination is consonant with the pursuit of justice. On net, police control

could either improve or degrade the performance of the crime lab. An empirical

investigation is required, and such an investigation will require variation in the level

of lab independence.

Forensic labs that are entirely independent of law enforcement are rare in the his-

tory of the United States, but even among labs that are related to law enforcement,

there are degrees of independence. Since most police investigations in the U.S. are

handled by city and county agencies, and most prosecutions are handled by city or

county attorneys, a crime lab that is under the control of local authorities might

be the least independent of all. Contrasting their behavior with that of more dis-

tantly connected labs may provide a window into the more general question of the

relationship between independence and performance. In this paper, I investigate the

relationship between one particular institutional variant, local police/sheriff control of

the crime lab, and one particular outcome in the criminal-justice chain: the conviction

of innocent defendants due, in part, to faulty or misleading forensics.

In the main analysis of this paper, I compare the rate of eventual exoneration

3

in counties with a locally-controlled crime lab to that in similarly-situated counties

without a locally controlled lab (although the county may include a lab controlled by

some other entity, such as the state government). In particular, I study the rate of

what I call forensic exonerations, exonerations in which faulty forensic were implicated

as a significant contributor to the initial false conviction, in the 200 largest counties

in the U.S., using the rate of non-forensic problems as a control to proxy for all

non-forensic factors that might relate to exoneration. In this analysis, counties with

locally-controlled labs seem to experience MUCH lower rates of forensic exonerations

than those without. Counties with local labs experience between 0.6 and 1 fewer

exonerations per million residents in 1980, on a mean about about 1.25 exonerations.

The analysis on the extensive margin is similar. About 40 percent of these large

counties have experienced a forensic exoneration recorded in the National Registry

of Exonerations, but counties with locally controlled labs are about 20 percentage

points less likely to have had one. All these results are robust to various county-level

controls and state fixed effects.

These basic results in hand, I explore what is different about the counties with

locally-controlled labs that might be contributing to the difference in exoneration

outcomes. Turning, first, to a more contemporary data set containing judicial out-

comes for felony defendants in large urban counties, I document that the exoneration

results do not seem to arise mechanically from fewer convictions, as might be the

case if locally controlled labs simply provided less evidence or hedged in favor of the

defendant. On the contrary, counties with locally controlled labs have higher felony

convictions rates, fewer dismissals, and more jail sentences than those without. These

results are robust to controlling for case mix and load and spending on police.

Next, from a survey of forensic professionals conducted in 1977 (roughly the be-

ginning of the set of cases that were at risk to appear in the exoneration list) and from

a census of publicly funded crime labs in 2008 (roughly the end of the exoneration

period), I obtain some proxies for the human capital of the people working in the

crime labs: education and salary levels. For all measures, those working in county

crime labs reported higher levels of human capital than those working for state crime

labs. This is evidence that locally-controlled crime labs invest more, as organizations,

in the quality of their staff. Such investments might reduce both type-1 and type-2

errors.

4

2 Empirical Analysis of Error and Forensic Lab

Control

The unit of observation for this paper will be the county. Looking at the 200 most

populous counties in the U.S., over the past three decades, I explore whether the

presence of a locally controlled crime lab is related to the quality of forensic ser-

vice3. The basic idea is to compare rates of exonerations where forensic problems

were implicated in the original trial, while controlling for a proxy for the quality of

convictions, in general. The basic result is that counties with locally controlled crime

labs have lower rates of forensic exonerations than their counterparts who must send

their forensic work to an outside lab (mostly to state-controlled labs).

2.1 Errors in the Criminal Justice Process

People have worried about errors in the criminal justice process for a long time,

including false convictions (Borchard 1932). But with the advent of DNA testing,

errant convictions have been easier to identify and their underlying causes have been

subjected to more careful investigation (Jacoby et al. 2005, Gross 2008, Hampikian,

West and Akselrod 2011, Gross, O’Brien, Hu and Kennedy 2014). Although mis-

leading forensic evidence has appeared in those investigations, its role has not been

pronounced. Historically, more attention has been paid to other sources of error,

such as perjury, witness misidentification, or false confessions. In one striking excep-

tion, Garrett and Neufeld (2009) performed a comprehensive analysis of the forensic

evidence presented in the trials of defendants who were later exonerated by the Inno-

cence Project through DNA evidence, documenting errors is a large fraction (≈ 40%)

of these cases.

Of course, false convictions are only one side of the coin in terms of the errors.

Errors can also lead to false acquittals (or, more generally, to general failures to catch

and punish the perpetrator appropriately). The literature on this side of the error

calculations has focussed on conviction rates, and how variations in budgets, career

incentives, and political incentives affected them (Rasmusen, Raghav and Ramseyer

2009, Gordon and Huber 2009).

Compared to exonerations, conviction rates are a relatively poor measure of errors,

since higher conviction rates (or crime closure rates, more generally) do not necessarily

3All results are robust to limiting to the top-100 counties.

5

mean fewer people getting away with crimes, if many of convictions are in error. For

this reason, I will look first at exonerations as the primary measure of error. Only

with those results in hand can we properly evaluate any effects on conviction rates.

2.2 Control of The Lab

The relationship between crime labs and police has varied considerably over space

and time in the United States. According to a 2008 census by the Bureau of Justice

Statistics, there are 411 publicly funded forensic-science labs in the United States,

217 of which are controlled by state or regional authorities, 90 of which are controlled

by county governments, and 66 of which are controlled by municipal authorities. The

remaining 38 labs are part of the federal government. Getting a handle on the number

of privately-controlled forensic science labs is more difficult, as there is no official

count conducted, but there are 30 private labs that are accredited by the American

Society of Crime Laboratory Directors, the largest and best-known accrediting body

for forensic labs.4

2.3 Data and Econometric Approach

The data on exonerations are drawn from the National Registry of Exonerations,

which is a continually updated list of every known exoneration in the United States

since 1989, where an exoneration is defined as a case in which a person was wrongly

convicted of a crime and later cleared of all the charges based on new evidence of

innocence. It details the year and county of each original trial, the most serious crime

of which the defendant was convicted, and the factors identified as contributing to the

original false convictions: false confession, mistaken witness identification, perjury,

official misconduct, inadequate defense, and false or misleading forensic evidence.

After limiting the sample to violent crimes5 and dropping federal cases, there are 1193

exonerations, of which just over half (605) have murder as the most serious conviction

and about a third are sexual assaults or child sex abuse cases (413). Forensic problems

are identified by the Registry as contributing to the original conviction in 23 percent

of the cases, the second-least common contributor (false confessions are least common

at 13 percent). The most common problems are perjury (57) and official misconduct

4As of July, 2014, there were 184 ASCLD accredited state labs, 135 county or municipal labs,and 31 federal lab.

5Violent crimes include murder, manslaughter, sexual assault, child abuse, arson, assault, at-tempted murder, and robbery.

6

(46), with mistaken witness identification contributing in 40 percent of the cases.

The median year of the initial conviction was 1991, with 95 percent of convictions

occurring after 1978. From these data, I derive the two most important variables in

the empirical analysis. Forensic exonerations, those where faulty forensics is identified

as a key contributor to the original false convictions, will be the measure of forensic

quality, while the number of non-forensic problems in the full set of exonerations

will be the most important control for factors beyond forensic control that might be

driving exonerations.

Many of these exonerations arose from cases pursued by a collection of non-profit

legal clinics known as Innocence Projects. These groups are patterned after the

original Innocence Project, which was founded in 1992 by Barry C. Scheck and Peter

J. Neufeld at the Benjamin N. Cardozo School of Law at Yeshiva University to assist

prisoners who could be proven innocent through DNA testing. Although some of

these organizations accept clients nationally, they tend to be regionally specialized,

and cases that are close to “home” might be prioritized. Since the chance that a given

false conviction ends in exoneration can depends on whether it attracts the attention

of an innocence project, I collected the location of each member organization from

the innocence project and will include whether a county has an innocence project as

an additional control in some specifications.6 Between this control, and the control

for non-forensic problems, above, I hope to control for the potential influence of case

selection.

The only unusual data on county-level institutions included is information about

forensic science labs present in the county. A contemporaneous census of world foren-

sic labs conducted by the Forensic Science Society (Forensic Science Society 1977)

in 1977 provides a comprehensive list of public and private forensic labs. I coded

the presence of forensic labs in each county, as well as who controls the lab, using

historical newspaper and court accounts in cases where control was not clear from

the census. In 1977, 35.5 percent of the top-200 U.S. counties by 1980 population

had some sort of locally-controlled public forensic lab: 15 percent with a county lab

only, 18 percent with a municipal lab, only, and 3.5 percent with both. In the same

period, 31 percent of these counties contained state-controlled labs, with 9 percent of

counties having both locally-controlled and state-controlled labs.

6There are some intermediate-outcome concerns with this specification, if we think that inno-cence projects are more likely to arise in areas with more false-convictions, but I think the bias fromcase-section is the greater worry, so I do include it. Specifications without local Innocence-Projectindicators yield nearly identical estimates

7

More prosaic county-level data on law-enforcement resources, demographics, and

geography are gathered from a variety of common sources. Informational about spend-

ing on police and law enforcement spending comes from the 1982 census of govern-

ments, while population and demographic information come from the 1980 census.

Crime data are drawn from the county-level Department of Justice Uniform Crime

Reports from 1980 through 1983, which are averaged.

Exonerations are rare phenomena and exonerations where forensics were impli-

cated are particularly so. Of the top-200 U.S. counties by 1980 population, 64 percent

have had no exonerations in which forensic errors were identified in the initial con-

victions. So rather than using a panel approach I will instead conduct a cross-section

analysis using the entire time period, examining both the probability of having a

forensic exoneration and the forensic exoneration rate (per million people in 1980).

Thus, I present regressions of the form

FExoni = βLocalLabi + δNonForenProblemsi + ΓXi + εi, (1)

where the outcome variable is either a dummy equal to one if the county i has had

any forensic-related exonerations or the number of forensic exonerations per million

residents. The key variable of interest is an indicator for a locally controlled crime

lab in the county, which I will sometimes divide into two dummies, one representing

the presence of a county-wide local lab, almost always controlled by the sheriff, and

one representing the presence of a public lab in a municipality within the county, but

which does not span the entire geography of the county. If the city and county are

co-terminus, any locally-controlled lab is coded as a county lab.

Every regression will include the key control variable, the number of non-forensic

problems per capita. The purpose of this variable is to try to control for all factors that

contribute to exoneration rates, in general, and which might be correlated in a non-

causal way with the presence of a local crime lab. Thus if, for instance, counties with

local crime labs happen to, say, have higher quality and more professional police force,

and professional police lead to fewer exonerations, then we want to control for that

general lower rate of exonerations. Otherwise, we would improperly attribute that

effect to the locally-controlled crime lab. Any omitted variable that is correlated with

locally-controlled crime labs and forensic exoneration is a risk for biasing estimates,

but if the correlation with forensic exonerations is zero, conditional on the rate of

non-forensic problems, there will be no omitted-variable bias. Essentially, this is the

key identification assumption–conditional on the rate of non-forensic problems (and

8

other control variables, as appropriate), the presence of a locally controlled forensic

lab is uncorrelated with unmodelled factors that causes forensic exonerations (the

error term in regression 1).

The variables included in Xi will vary with specifications, but they include some

subset of: murder and rape rates (crimes per 100k population), an indicator for

a county including the state capital, county-level spending on police (log(spending

M$)), fraction black, fraction hispanic, indicators for census region, and/or state-level

fixed-effects. Finally, these regressions are run on two samples of counties. First, the

full top-200 counties by 1980 population, excluding the counties making up New York

City. New York City is omitted because allocating crime rates and police resources

among the borroughs/counties is impossible. Including them as a single county or

as five identical counties has no substantive effect on the result. Second, a restricted

set of counties containing only those top-200 counties which contain either a locally-

controlled lab or a state-controlled lab. This subset is of interest if we think that the

presence of a convenient lab might have a direct effect, and we want to just contrast

control per se.

Table 1 presents the sample means for the key variables, separated by the presence

of a municipal or county crime lab. From these descriptive statistics, alone, there are a

number of differences between the counties that have local crime labs and those that

do not. First, counties with county labs have lower rates of forensic exonerations,

despite counties with any sort of local labs having much higher rates of non-forensic

exonerations and non-forensic problems. Second, counties with local labs are much

more populous than those without local labs, even in this selected sample of big

counties, and the difference is even bigger for counties with city labs or both. Despite

being twice as populous, counties with only county labs are equally likely to have

experienced a forensic exonerations as those without a lab. But the counties with

municipal labs are so much bigger than those with no local labs that they are much

more likely to have experienced any forensic exonerations. Crime rates are higher

for counties with local labs, and again higher still in those with city labs. They also

spend more on police. It is important to remember that these crime rates come from

the peak of the 80s urban crack epidemic, so they are much higher than current rates.

Counties with local labs have higher fractions black and hispanic. Counties with local

labs are more likely to be located in the midwest and west and less likely to be in the

northeast than counties without local labs. Finally, counties with county labs are less

likely to have a state lab, or to contain the state capital than counties with no local

9

labs, while counties with a city lab but no county lab are more likely to contain the

capital and a state lab.

A consistent story arises just from these means. Large, populous counties in the

midwest and west are likely to have high crime rates, large minority populations,

large police budgets, and their own crime labs. They are also likely to have high rates

of exoneration, in general, but without a proportionately higher rates of forensic

exoneration. The rates for non-forensic problems provides a natural control for these

underlying differences that might be accidentally correlated with the presence of a

local lab. Something like a differences-in-differences estimate is available: do the

differences in forensic exoneration rates between counties with local labs and those

without mirror the differences in non-forensic problems? The answer seems to be no.

If anything, the gap in forensic exonerations rates between those with local labs and

those without seems to go in the opposite direction and is certainly smaller than the

concomitant gap in non-forensic problems, suggesting a positive causal effect of local

control on the quality of forensic analysis and testimony.

Figure 1 illustrates this difference, by calculating the fraction of total exonera-

tions in which faulty forensics were implicated and comparing the mean rate of this

fraction among counties into those including labs controlled by county sheriffs, those

including labs controlled by municipal police, and those including labs of both types.

The blocks indicate the number of counties of each type. They do not add up to 200,

since 53 counties had no exonerations of any type. The dot indicates the mean, while

the whiskers indicate 1 standard error on the mean. The pattern is straightforward.

Counties with more local control of the forensic labs had a forensic problems impli-

cated in a lower fraction of their overall exonerations. Of course, further investigation

is warranted, which the next section pursues.

2.4 Local Control and Exoneration: Regression Estimates

The results in the OLS estimations of variants of equation (1) are presented in Table 2.

The first panel includes only dummies for county- and city-controlled labs, together

with the non-forensic problem rate. The second panel adds a variety of county-

level controls as well as census-region fixed-effects, and the third panel replaces those

census-region dummies with state-specific fixed-effects. Columns (1) and (3) have the

rate of forensic exonerations as the dependant variable, while columns (2) and (4) are

linear probability models with an indicator for the presence of a forensic exoneration

as the dependent variable. Finally, the first two columns are run on the full sample,

10

Figure 1: Forensic Exonerations as Fraction of Total Exonerations

11

while the last two columns restrict the sample to only those counties which have

either a locally-controlled lab or a state-controlled lab (or both).

Throughout all specifications, there is a negative relationship between the presence

of a county-controlled forensic lab in 1977 and the rate of eventual forensic exonera-

tions arising in that county (columns 1 and 3). Counties with county-controlled labs

have between 0.6 and 0.9 fewer forensic exonerations per million residents than we

would we would predict given their rate of non-forensic problems and other observ-

ables. With the mean forensic exoneration rate of about 1.25 per million this is a very

substantial effect. These results are robust to the inclusion of county controls and

state-fixed effects. The relationship for counties containing a city with a lab is also

consistently negative, although rarely statistically significant. Restricting the sample

to counties having a lab, as is done in regression (3), does nothing to upend the result.

In fact, the point estimates are quite consistent, suggesting that the key contrast is

not between counties with and without labs but rather between labs controlled by

the state and those controlled locally.

An alternative way to look at the problem is to ignore the intensive margin and

simply ask whether counties with local crime labs are more or less likely to have

experienced a forensic exoneration, regardless of the number. Since exonerations are

quite rare (128 counties had no forensic exonerations, 42 had exactly one, and only

six counties had five or more), much of the variance in the rate measure is actually

driven by population differences. Since population, and even forensic exonerations,

are likely measured with error there is a chance that the intensive margin is adding

more noise than real information. It is also potentially more sensitive to outliers,

such as Chicago, with 24 forensic exonerations–by far the most.7 Columns (2) and

(4) present the results of a linear-probability model of this type. The results in the

first panel are clearly biased, since there is no control for scale and more populous

counties are clearly more likely to have local crime labs and have experienced at least

one forensic exoneration, if only because they have experience many more crimes and

convictions. The other two panels, however, give quite consistent results. Counties

with locally controlled crime labs are less likely to have experienced a forensic-related

exonoration. Again, county-controlled crime labs have the most robust effect– with

counties having local labs being about 20 percentage points less likely to have had

a forensic exoneration, on a mean of about 40 percent. Counties with cities with

7Chicago is not driving the main results. Dropping it from the analysis moves the point estimatesinsignificantly.

12

labs are also consistently less likely to experience exonerations, although the results

are never statistically significant. Again, the results when restricting the analysis to

counties containing crime labs are quite consistent with the more general analysis,

suggesting that local control is the key factor, rather than simply having a lab close

by.

Although the regressions in Table 2 attempt to control for all factors that might

be related to local control and forensic exonerations, there remain some chance of

bias. The sample means suggested that there were a number of important differences

in average observable characteristics between counties with local control and those

without. Perhaps the linearity assumptions of OLS have failed to capture these differ-

ences adequately and some bias remains. An alternative approach is to concentrate

on one type of local control, say county-controlled labs, and select a quasi-control

group by choosing a subsample of the counties without that type of control which is

“close” to treated group on a variety of other observable characteristics. This kind of

“nearest-neighbor” matching eschews the linearity assumption at the cost of discount-

ing the information in the observations that are “far” from any observations with the

opposite treatment. Table 3 contains the results of this analysis, where the matching

variables in each panel are the exact set used as controls in the OLS analysis, three

matches are chosen for each treated observation, and the dependent variables in the

columns match those in the OLS analysis.

The matching results for the exoneration rates are nearly identical to the OLS

results, both on the lab-containing counties and on the full set. The linear-probability

results are of the same sign as the OLS results, although only statistically significant

in the within-state matches. There are two possible interpretations of this result. In

sum, the results do not seem to be driven by the linear projection assumptions in

OLS.

3 Why the Difference?

Across specifications, samples, and econometric approaches, counties with locally con-

trolled labs have lower rates of exonerations in which faulty forensics were implicated

in the original trial than similarly-situated counties without locally controlled labs.

In this section, I explore what might be different about the criminal justice process in

counties with locally-controlled labs. The available data varies enormously for each

potential difference, maintaining a consistent approach throughout these comparisons

13

is infeasible. Instead, I will present the best data available, even as the time frame

and unit of analysis varies, while being clear, throughout, as to the origin of the data

and my econometric approach. I briefly summarize the results, here, before turning

to each detailed analysis, below.

The first potential difference I explore is whether there are differential conviction

rates in counties with local labs. Perhaps counties with local labs had fewer exon-

erations, controlling for crime rates, because they actually had fewer convictions–

either they were ineffective at catching criminals or in providing good evidence to

prosecutors, so there were, mechanically, fewer false convictions. Or, perhaps, they

are simply trading off in a different way on “close calls” and choosing to have fewer

“close” convictions and, therefore, fewer exonerations. On the contrary, between 2002

and 2006, counties with locally-controlled crime labs had much higher felony convic-

tion rates and much lower dismissal rates than those without, controlling for case mix

and load, population, and policing resources. Thus, neither of these proposed mecha-

nisms seems correct. Counties with locally controlled labs have both more convictions

and fewer exonerations.

A second potential difference is whether the lab personnel were better trained or

more experienced in local labs than in state labs. That seems correct. In two surveys,

one before the period of the bulk of the trials and one after, crime lab employees

at county labs have high levels of human capital than those working in state labs.

They are more likely to have graduate degrees, more experience, and are paid higher

salaries.

3.1 Conviction Rates

The only reliable data on conviction rates at the local level come from the Department

of Justice’s State Court Processing Statistics: Felony Defendants in Large Urban

Counties series, which includes an unbalanced panel of a sample of the 75 largest

urban counties in the U.S. from 1990 to 2006 (United State Department of Justice–

Bureau of Justice Statistics 2002-2006). They track the eventual outcomes of every

defendant ever charged with a felony for up to two years, recording the most serious

crime charged faced, the gender and race of the defendant, and the final outcome of

the charges. The public- use data are aggregated to the county level, so these variables

become data on the mix of initial charges, demographics of defendants, number of

cases, and fraction of cases resulting in various judicial outcomes.

The best contemporaneous measure of the presence of a locally-controlled crime

14

lab comes from the Census of Publicly Funded Forensic Crime Labs (United State

Department of Justice–Bureau of Justice Statistics 2009).8 Since the presence and

control of crime labs did not change much over this decade, I will ignore the panel

nature of the data and instead treat it as a cross-section over the last three sample

years (2002, 2004, 2006), accounting for the correlation in conviction rates if a county

is sampled multiple times by clustering the standard errors at the county level. I

pair this series with data from standard sources on police spending, crime rates, and

population.

The basic differences in outcomes are presented in Figure 2, where the four sub-

graphs present the four potential case outcomes for defendants initially charged with a

felony: felony conviction, misdemeanor conviction, dismissal and acquittal. Each dot

represents the average share of cases resulting in each outcome among counties with

the indicated presence of a locally-controlled crime lab, while the whiskers represent

a standard error from the mean in each direction. The bars represent the number

of observations (right axis). As is immediately apparent from the first sub-graph,

counties with no locally controlled crime labs have felony conviction rates right at

50%, while those with a county or city crime lab, and especially those with both,

have much higher felony conviction rates.

Of course, this difference could be driven by other factors about those counties,

so Table 4 presents the results of a regression that controls for various factors that

might be related to local control of the crime lab and conviction rates, such as the

mix of crimes, case load, and police spending. The regression results reinforce those

from the figure. Defendants in counties with locally-controlled crime labs have higher

felony conviction rates, lower dismissal rates, and a greater likelihood to end up in

jail than those in counties without a locally controlled crime lab.

8Censuses were also conducted in 2002 and 2005, but they did not identify the agency withadministrative control over the labs.

15

Figure 2: Case Outcomes by Control of Lab

(a) Felony Conviction (b) Misdim. Conviction

(c) Dismissal (d) Acquital

16

Table 1: Sample Means by Presence of Local Crime Lab- 200 Largest Counties in1980 (sd)

No Local City Lab County Lab BothForen. Exon. Rate 1.317 1.362 0.969 0.790

(2.716) (1.854) (2.321) (0.986)Forensic Exoneration 0.326 0.500 0.324 0.571

(0.470) (0.509) (0.475) (0.535)Non-F. Problem Rate 6.854 11.51 12.19 9.838

(8.826) (12.17) (22.96) (3.816)Non-F. Exon. Rate 2.960 5.399 5.815 5.213

(4.197) (5.326) (9.239) (3.147)State Lab 0.341 0.367 0.176 0.143

(0.476) (0.490) (0.387) (0.378)Pop. 1980 (M) 0.398 0.902 0.742 2.317

(0.204) (0.961) (0.356) (2.361)Capital County 0.132 0.267 0.0882 0

(0.340) (0.450) (0.288) (0)Inno. Project 0.101 0.500 0.176 0.286

(0.302) (0.509) (0.387) (0.488)Rape per 100k 35.24 52.91 48.21 51.97

(21.87) (16.87) (24.46) (19.72)Murder per 100k 7.199 13.00 10.56 12.25

(6.499) (7.375) (8.228) (8.125)Frac. Black 0.102 0.168 0.124 0.125

(0.111) (0.135) (0.114) (0.118)Frac. Hisp. 0.0519 0.0665 0.0789 0.133

(0.108) (0.103) (0.0902) (0.0961)Census Northeast 0.341 0.200 0.206 0

(0.476) (0.407) (0.410) (0)Census South 0.295 0.267 0.324 0

(0.458) (0.450) (0.475) (0)Census West 0.163 0.133 0.324 0.714

(0.371) (0.346) (0.475) (0.488)Census Midwest 0.194 0.400 0.147 0.286

(0.397) (0.498) (0.359) (0.488)Police Budget (Mil. 1982) 1.418 4.858 3.206 12.37

(1.226) (8.041) (2.190) (15.96)n 130 30 33 7

17

Table 2: Eventual Exonerations by Availability of Local Crime Lab

Full Sample Counties with Labs(1) (2) (3) (4)

F. Exon. Rate Any F. Exon F. Exon. Rate Any F. Exon

Panel 1: Non-Forensic Problems ControlCounty Lab −0.86∗∗∗ −0.05 −0.97∗∗ −0.06

(0.30) (0.08) (0.42) (0.09)City Lab −0.38 0.14 −0.49 0.13

(0.32) (0.09) (0.41) (0.10)Non-F. Problem Rate 0.12∗∗∗ 0.01∗∗∗ 0.11∗∗∗ 0.01∗∗∗

(0.02) (0.00) (0.02) (0.00)Panel 2: County Controls

County Lab −0.69∗∗ −0.21∗∗ −0.63∗ −0.20∗

(0.29) (0.09) (0.35) (0.11)City Lab −0.37 −0.07 −0.24 −0.06

(0.32) (0.10) (0.37) (0.11)Non-F. Problem Rate 0.12∗∗∗ 0.01∗∗∗ 0.12∗∗∗ 0.01∗∗∗

(0.02) (0.00) (0.02) (0.00)Pop. (1980) 1.16∗ 0.39∗∗ 1.32 0.31

(0.65) (0.16) (0.93) (0.23)Capital County 1.66∗∗ 0.20∗ 2.04∗∗ 0.27∗∗

(0.81) (0.10) (0.85) (0.12)Rape per 100k 0.00 −0.00 −0.01 −0.00

(0.01) (0.00) (0.01) (0.00)Murder per 100k 0.03 0.01 0.00 −0.00

(0.05) (0.01) (0.06) (0.01)Frac. Black −6.76∗∗ −0.53 −7.13∗ −0.50

(2.82) (0.54) (3.66) (0.72)Frac. Hisp. −2.27 −0.21 −2.29 0.04

(1.42) (0.35) (2.34) (0.58)Inno Project 0.06 0.04 0.06 0.06

(0.47) (0.11) (0.54) (0.12)log(Police Budget) −0.78 −0.08 −0.75 0.04

(0.49) (0.12) (0.73) (0.17)Panel 3: + State FE

County Lab −0.88∗∗ −0.25∗∗ −0.91∗ −0.29∗∗

(0.36) (0.11) (0.50) (0.14)City Lab −0.64∗ −0.09 −0.71 −0.17

(0.36) (0.13) (0.48) (0.14)Non-F. Problem Rate 0.11∗∗∗ 0.01∗∗∗ 0.11∗∗∗ 0.01∗∗∗

(0.01) (0.00) (0.01) (0.00)Dependant Variable Mean

Sample Mean 1.25 0.36 1.39 0.41n 200 200 123 123

Notes: Regressions on top-200 U.S. counties in 1980 population. Dependent variables are, in (1)and (3), the number of forensic exonerations per million residents and , in (2) and (4), a dummywith 1 indicating the presence of a forensic exoneration. Robust standard errors, with statisticalsignificance of test against zero null indicated by ***: 0.01, **:0.05, and *:0.10.18

Table 3: Eventual Exonerations by Availability of Local Crime Lab (Nearest-NeighborMatch)

F. Exon. Rate Any F. Exon F. Exon. Rate Any F. Exon

Panel 1: Match on Non-Forensic Problem RateCounty Lab −0.77∗∗∗ −0.07 −0.88∗∗ −0.08

(0.27) (0.08) (0.37) (0.09)Panel 2: Also Match on County Controls

County Lab −0.62∗ −0.11 −0.75∗∗ −0.14(0.34) (0.11) (0.30) (0.10)Panel 3: Also Match on State FE

County Lab −0.65∗∗ −0.14∗ −0.88∗∗∗ −0.22∗∗∗

(0.28) (0.08) (0.26) (0.08)n 200 200 123 123

Notes: Nearest-neighbor matching on top-200 U.S. counties in 1980 population. Dependent variablesare, in (1) and (3), the number of forensic exonerations per million residents and , in (2) and (4),a dummy with 1 indicating the presence of a forensic exoneration. Robust standard errors, withstatistical significance of test against zero null indicated by ***: 0.01, **:0.05, and *:0.10.

19

Table 4: Judicial Proceedings Outcomes by Control of Crime lab

Felony Convict Dismissal Prison Sent. Jail Sent.Panel 1: Simple Mean Differences

County Lab 8.43∗ −7.19∗ −1.83 14.81∗∗∗

(4.63) (3.86) (3.63) (4.29)Muni. Lab 11.70∗∗ −7.46∗∗ 5.26 −4.79

(4.43) (3.63) (4.73) (4.26)Panel 2: + Case Mix and Counts, County Pop, and Year FE

County Lab 10.48∗∗ −10.48∗∗∗ −1.01 13.97∗∗∗

(4.60) (3.67) (4.20) (4.99)Muni. Lab 5.42 −4.13 4.63 −6.18

(4.75) (4.26) (4.55) (5.71)% Public −0.10 −0.51∗ −0.12 0.21

(0.41) (0.28) (0.42) (0.40)% Violent −0.87∗∗∗ 0.61∗∗ −0.21 0.32

(0.26) (0.26) (0.26) (0.45)% Property −0.20 0.12 −0.05 0.09

(0.29) (0.24) (0.17) (0.25)Log(Cases) −6.67 13.96∗∗∗ 1.32 −1.98

(4.91) (4.59) (3.05) (4.24)Log(Pop.) 8.36∗∗ −6.96∗∗ −0.54 4.66

(3.80) (3.07) (3.36) (4.45)Police Spend/Cap −0.35∗∗ 0.17∗∗ −0.26 −0.10

(0.14) (0.07) (0.33) (0.30)Sample Mean of Dependent Variable

Sample Mean 56.79 22.61 36.44 34.63n 111 111 111 111

Notes: Dependent Variable is percent of felony defendants with indicated case outcome. Observa-tions from unbalanced panel of a subset of 75 large urban counties from 2002, 2004, and 2006.

3.2 Human Capital of Lab Personnel

Ideal data on the human capital of the laboratory personnel would include information

about the individuals who were working at the lab at the time of the original trial.

Two surveys, one at the beginning of the period and one near the present provide the

best available data on this question. The evidence of human capital in these surveys

differs significantly, but they tell a very consistent story. Federal lab employees have

the highest level of human capital, as evidenced by training, experience, and salary,

while state lab employees have the lowest levels. County and and city employees fall

in between, with county employees outperforming state and city employees in nearly

20

every dimensions and at nearly every level of the hierarchy of responsibility.

A 1976 survey conducted by the Forensic Science Foundation on behalf of the

U.S. Department of Justice investigated the characteristics of every role in the foren-

sic science process: Coroner/Medical Examiners, criminalists, anthropologists, odon-

tologists, psychiatrists, toxicologists, evidence technicians, and questioned-document

examiners (Field et al. 1977). The original data of the survey have been lost, but the

report prepared by the surveyors offers some useful evidence of the level of human

capital in various forensic labs in that era. For criminalists, the forensic scientists

who analyze the bulk of the forensic evidence and provide most trial testimony, the

survey report provides counts of respondents by level of education/experience and

control of the forensic lab. The first two columns of first panel of Table 5 present

means for two of these measures, the fraction with graduate degrees and the fraction

with at least 10 years of experience. For each measure, criminalists working in county

labs outscore those in state labs, while municipal lab employees are less educated

but more experienced than both. No salary information is available for criminalists

broken out by lab control, but the survey does break down the salary of evidence tech-

nicians. Again, county technicians are much more likely to be paid over $15k/year

than state technicians, which are in turn more likely to be highly paid than municipal

technicians.

The same 2009 Department of Justice census of publicly funded forensic crime labs

used to identify local labs for the conviction analysis, above, provides similar evidence

for the end of the sample period. No data on education or experience is provided,

but each lab is asked to report the salary range for employees at various levels of the

organization. In the second panel of Table 5 reports the average of the midpoint of

these salary ranges for each lab control type and occupation. Throughout, employees

at county labs are paid more than their counterparts in state-controlled labs, and

the difference is statistically significant at the 5-percent level. In three of four cases

(all but the director), municipal lab employees are also paid more than state-lab

employees, although the difference is only statistically significant for the analyst and

supervisor positions.

4 Conclusions

The calls for forensic laboratory independence come from the natural suspicion of

excessive coziness between the police and those who provide scientific evidence at

21

Table 5: Education, Experience, and Pay by Control of Crime lab

Panel A: 1976 Forensic Science Foundation SurveyGrad. Degree 10+ Yrs. Exp. Evid. Tech. Sal.> $15k

Federal 42.3 27.0 n/aState 23.0 16.8 28.6County 34.3 26.7 47.4Municipal 14.5 29.6 10.0

Panel B: 2009 Census of Publicly Funded Crime LabsSalary Midpoint

Director Supervisor Senior Analyst AnalystFederal $132,911 $119,569 $94,723 $58,081State $90,878 $72,759 $62,536 $48,594County $100,698 $76,721 $65,871 $53,340Municipal $90,685 $77,148 $64,235 $52,175

trial. The most natural way to induce this separation would be to move the forensic

services from the control of the local officials, who handle much of the investigation

and prosecution of crimes, to a state office. Goldstein (2009), for example, argues that

the state, in particular, has strong incentives to improve forensic science outcomes.

But basic theory of the firm suggests that the decision to separate related activities

comes with both costs and benefits. This paper has provided the first systematic

evidence of the net consequences of combining or separating forensic services from the

local criminal justice organizations and that evidence suggests that calls for increasing

forensic lab independence might be misguided. In particular, I find that counties with

locally controlled crime labs have fewer exonerations and higher conviction rates.

The mechanism is not entirely clear, but there is some evidence that local labs invest

more in their personnel than state-controlled labs do–paying higher salaries and hiring

higher human

Beyond the specifics of crime labs, these results corroborate well with the broader

literature on decentralization and governmental responsiveness (Besley and Coate

2003, Faguet 2004, Rubinchik-Pessach 2005). If most of the gains from good con-

victions and losses from bad convictions are borne locally, the broader evidence on

public good provision squares well with our results–local provision dominates global

provision, even if local government is generally seen as “corrupt, institutionally weak,

or prone to interest-group capture.” Local labs invest more, since they capture more

22

of the benefits, and any pressure to bias results on close calls are offset by fewer close

calls.

References

Besley, Tim and Stephen Coate, “Centralized versus decentralized provision of

local public goods: a political economy approach,” Journal of Public Economics,

2003, pp. 2611–37.

Borchard, Edwin M., Convicting The Innocent, Yale University Press, 1932.

Committee on Identifying the Needs of the Forensic Sciences Community,

National Research Council, “Strengthening Forensic Science in the United

States: A Path Forward,” Technical Report, National Academy of Science 2009.

Faguet, Jean-Paul, “Does Decentralization Increase Government Responsiveness

to Local Needs? Evidence from Bolivia,” Journal of Public Economics, 2004,

pp. 867–93.

Field, Kenneth S. et al., “Assessment of the.Personnel of the Forensic Sciences

Profession,” Technical Report, National Institute of Law Enforcement and Crim-

inal Justice, Law Enforcement Assistance Agency, Department of Justice 1977.

Forensic Science Society, “World List of Forensic Science Laboratories,” Technical

Report 3rd Edition, Harrogate, North Yorkshite, England 1977.

Garrett, Brandon L. and Peter J. Neufeld, “Invalid Forensic Science Testimony

and Wrongfuul Convictions,” Virginia Law Review, 2009.

Giannelli, Paul C, “The Abuse of Evidence in Criminal Cases: The Need for In-

dependent Crime Laboratories,” Virginia Journal of Social Policy and the Law,

1997.

Gibbons, Robert, “Four Formal(izable) Theories of the Firm,” Journal of Economic

Behavior and Organization, 2005.

Goldstein, Ryan M., “Improving Forensic Science through State Oversight,” Texas

Law Review, 2009, pp. 225–58.

23

Gordon, Sanford and Gregory Huber, “The Political Economy of Prosecution,”

Annual Review of Law and Social Science, 2009.

Gross, S., “Convicting the Innocent,” Annual Review of Law and Social Science,

2008, 4, 173–92.

Gross, S, B O’Brien, C. Hu, and E. H. Kennedy, “Rate of false convic-

tion of criminal defendants who are sentenced to death,” Proceedings - National

Academy of Science, 2014, 111 (20), 7230–5.

Hampikian, Greg, Emily West, and Ogla Akselrod, “The Genetics of Inno-

cence: Analysis of 194 U.S. DNA Exonerations,” Annual Review of Genomics

and Human Genetics, 2011, 12, 97–120.

Jacoby, K et al., “Exonerations in the United States 1989 Through 2003.,” Journal

of Criminal Law and Criminology, 2005, 95 (2), 523–53.

Koppl, Roger, “How to Improve Forensic Science,” European Journal of Law and

Economics, 2005.

, “Organization Economics Explains Many Forensic Science Errors,” Journal of

Institutional Economics, 2010.

Lafontaine, Francine and Margaret Slade, “Vertical Integration and Firm

Boundaries: The Evidence,” Journal of Economic Literature, 2007.

McEwen, Tom, “The Role and Impact of Forensic Evidence in the Criminal Justice

System, Final Report,” Technical Report, Institute for Law and Justice, Inc.

2011.

Rasmusen, Eric Bennett, Manu Raghav, and J. Mark Ramseyer, “Convic-

tions versus Conviction Rates, The Prosecutor’s Choice,” American Law and

Economics Review, 2009.

Rubinchik-Pessach, Anna, “Can Decentralization Be Beneficial?,” Journal of Pub-

lic Economics, 2005.

United State Department of Justice–Bureau of Justice Statistics, “State

Court Processing Statistics: Felony Defendants in Large Urban Counties,” 2002-

2006.

24

, “Census of Publicly Funded Forensic Crime Laboratories,” 2009.

25