Reliability & Reliability & Forensic ScienceForensic Science

Paul GiannelliPaul Giannelli

Weatherhead Professor of Weatherhead Professor of LawLaw

Case Western Reserve Case Western Reserve UniversityUniversity

West Virginia: Fred West Virginia: Fred ZainZain

In re Investigation of W.Va. State In re Investigation of W.Va. State Police Police Crime Lab., Serology Crime Lab., Serology Div. Div. 438 S.E. 501 (W. Va. 1993)438 S.E. 501 (W. Va. 1993)

Chief serologistChief serologist fabricated reportsfabricated reports perjured testimonyperjured testimony

Zain’s Acts of Zain’s Acts of misconductmisconduct

(1) overstating the strength of results; (1) overstating the strength of results; (2) overstating the frequency of genetic (2) overstating the frequency of genetic

matches on individual pieces of evidence; matches on individual pieces of evidence; (3) reporting that multiple items of (3) reporting that multiple items of

evidence had been tested, when only a evidence had been tested, when only a single item had been tested;single item had been tested;

(4) reporting inconclusive results as (4) reporting inconclusive results as conclusive;conclusive;

(5) repeatedly altering laboratory (5) repeatedly altering laboratory records; records;

Zain continuedZain continued (6) failing to report conflicting results; (6) failing to report conflicting results; (7) failing to conduct or to report (7) failing to conduct or to report

conducting additional testing to conducting additional testing to resolve conflicting results; resolve conflicting results;

(8) implying a match with a suspect (8) implying a match with a suspect when testing supported only a match when testing supported only a match with the victim; andwith the victim; and

(9) reporting scientifically impossible (9) reporting scientifically impossible or improbable results.or improbable results.

Law ReviewsLaw Reviews

Giannelli, The Abuse of Scientific Evidence in Criminal Cases: The Need for Independent Crime Laboratories, 4 Va. J. Soc. Policy & L. 439 (1997)

Bernstein, Junk Science in the United States and the Commonwealth, 21 Yale J. Int’l L. 123 (1996) (discussing cases in Canada, Australia, New Zealand, and England)

W. Va. Lab: Later W. Va. Lab: Later ProblemsProblems

““The trooper who discovered one of The trooper who discovered one of the more recent scandals at the State the more recent scandals at the State Police crime lab was placed on leave Police crime lab was placed on leave Tuesday pending an investigation of Tuesday pending an investigation of his own work.”his own work.”

Messina, Messina, State Police Lab Review Leaves State Police Lab Review Leaves Agency With Another ShinerAgency With Another Shiner, Charleston , Charleston Gazette, Mar. 13, 2002, at P1A Gazette, Mar. 13, 2002, at P1A

Justice Department Justice Department I.G. I.G. Report on Report on

FBI Lab (1997)FBI Lab (1997) (1) seek accreditation; (1) seek accreditation; (2) require examiners in the (2) require examiners in the

Explosives Unit to have scientific Explosives Unit to have scientific backgrounds in chemistry, backgrounds in chemistry, metallurgy, or engineering; metallurgy, or engineering;

(3) mandate that each examiner (3) mandate that each examiner prepare and sign a separate report prepare and sign a separate report instead of a composite report “without instead of a composite report “without attribution to individual examiners”; attribution to individual examiners”;

I.G. Report continuedI.G. Report continued (4) establish review process for (4) establish review process for

analytical reports by unit chiefs;analytical reports by unit chiefs; (5) prepare adequate case files to (5) prepare adequate case files to

support reports; support reports; (6) monitor court testimony in order (6) monitor court testimony in order

to preclude examiners from to preclude examiners from testifying to matters beyond their testifying to matters beyond their expertise or in ways that are expertise or in ways that are “unprofessional”; and “unprofessional”; and

(7) develop written protocols for (7) develop written protocols for scientific proceduresscientific procedures. .

FBI - DNAFBI - DNA

Jacqueline Blake failed to use Jacqueline Blake failed to use negative controls in DNA testing for negative controls in DNA testing for 2 years in more than a 100 cases.2 years in more than a 100 cases.

Office of Inspector General, U.S. Office of Inspector General, U.S. Department of Justice, The FBI Department of Justice, The FBI Laboratory: A Review of Protocol and Laboratory: A Review of Protocol and Practice Vulnerabilities (May 2004)Practice Vulnerabilities (May 2004)

Bullet Lead Bullet Lead ComparisonComparison

““In a sworn affidavit, [Lundy] admitted In a sworn affidavit, [Lundy] admitted that her trial testimony was untruthful that her trial testimony was untruthful and that the manufacturing batch was and that the manufacturing batch was many times larger than she had many times larger than she had suggested. ... Lundy blamed her conduct suggested. ... Lundy blamed her conduct partly on a sense of crisis in her work, partly on a sense of crisis in her work, fed by ‘new and repeated challenges’ to fed by ‘new and repeated challenges’ to the validity of the science associated the validity of the science associated with bullet lead comparison analysis.”with bullet lead comparison analysis.” Pillar & Mejia, Pillar & Mejia, Science Casts Doubt on FBI’s Science Casts Doubt on FBI’s

Bullet EvidenceBullet Evidence, L.A. Times, Feb. 3, 2003, L.A. Times, Feb. 3, 2003

Oklahoma City: Joyce Oklahoma City: Joyce GilchristGilchrist

““[T]he forensic report was at best [T]he forensic report was at best incomplete, and at worst inaccurate and incomplete, and at worst inaccurate and misleading.”misleading.”

““We find it inconceivable why Ms. We find it inconceivable why Ms. Gilchrist would give such an improper Gilchrist would give such an improper opinion, which she admitted she was not opinion, which she admitted she was not qualified to give.”qualified to give.”

McCarty v. State, 765 P.2d 1215, 1218 McCarty v. State, 765 P.2d 1215, 1218 (Okla. Crim. App. 1988)(Okla. Crim. App. 1988)

Gilchrist continuedGilchrist continued ““Ms. Gilchrist thus provided the jury Ms. Gilchrist thus provided the jury

with evidence implicating Mr. Mitchell with evidence implicating Mr. Mitchell in the sexual assault of the victim which in the sexual assault of the victim which she knew was rendered false and she knew was rendered false and misleading by evidence withheld from misleading by evidence withheld from the defense.”the defense.”

Mitchell v. Gibson, 262 F.3d 1036, 1044 Mitchell v. Gibson, 262 F.3d 1036, 1044 (10th Cir. 2001) (withheld exculpatory (10th Cir. 2001) (withheld exculpatory DNA report)DNA report)

Gilchrist continuedGilchrist continued ““[I]n her missionary zeal to promote [I]n her missionary zeal to promote

the cause of the prosecution she had the cause of the prosecution she had put blinders on her professional put blinders on her professional conscience so that the truth of science conscience so that the truth of science took a back seat to her acting the role took a back seat to her acting the role of an advocate.”of an advocate.” Starrs, Starrs, The Forensic Scientist and the The Forensic Scientist and the

Open MindOpen Mind, 31 J. Forensic Sci. Soc’y 111, , 31 J. Forensic Sci. Soc’y 111, 132-33 (1991)132-33 (1991)

Okla. City P.D. Okla. City P.D. ReportReport

(1) missing evidence in numerous cases, (1) missing evidence in numerous cases, (2) contamination issues due to evidence (2) contamination issues due to evidence

being “stacked all over the chemist’s area,” being “stacked all over the chemist’s area,” (3) systematic destruction of rape evidence (3) systematic destruction of rape evidence

before statute of limitations expired, before statute of limitations expired, (4) lack of peer review in many cases, and (4) lack of peer review in many cases, and (5) lack of proficiency testing although (5) lack of proficiency testing although

such testing had been paid for. such testing had been paid for. Memo from Captain Byron Boshell (Jan. 16, 2001)Memo from Captain Byron Boshell (Jan. 16, 2001)

FBI ReviewFBI Review 8 cases: misidentified hairs in 6 & fibers 8 cases: misidentified hairs in 6 & fibers

in 1 in 1

““The review of the laboratory notes The review of the laboratory notes revealed that they were often incomplete revealed that they were often incomplete or inadequate to support the conclusions or inadequate to support the conclusions reached by the examiner. No reached by the examiner. No documentation existed that would allow documentation existed that would allow the examiner to identity textile fibers the examiner to identity textile fibers associated in one of the cases.”associated in one of the cases.” Special Agent Deedrick, Special Agent Deedrick, Summary of Case Reviews of Forensic Summary of Case Reviews of Forensic

Chemist, Joyce GilchristChemist, Joyce Gilchrist (April 4, 2001) at 1 (April 4, 2001) at 1

Montana: Arnold Montana: Arnold MelnikoffMelnikoff

Erroneous hair evidence in the trial Erroneous hair evidence in the trial of Jimmy Ray Bromgard, who spent of Jimmy Ray Bromgard, who spent 15 years in prison before being 15 years in prison before being exonerated by DNA.exonerated by DNA. Liptak,Liptak, 2 States to Review Lab Work of 2 States to Review Lab Work of

Expert Who Erred on IDExpert Who Erred on ID, N.Y. Times, , N.Y. Times, Dec. 19, 2002, at A24 Dec. 19, 2002, at A24

Melnikoff continuedMelnikoff continued Melnikoff: “[T]he odds were Melnikoff: “[T]he odds were one in one in

one hundredone hundred that two people would that two people would have head hair have head hair oror pubic hair so similar pubic hair so similar that they could not be distinguished by that they could not be distinguished by microscopic comparison and the odds microscopic comparison and the odds of of bothboth head and pubic hair from two head and pubic hair from two people being indistinguishable would people being indistinguishable would be about be about one in ten thousandone in ten thousand.”.” State v. Bromgard, 862 P.2d 1140, 1141 (Mont. State v. Bromgard, 862 P.2d 1140, 1141 (Mont.

1993)1993)

Melnikoff continuedMelnikoff continued ““egregious misstatements” & a egregious misstatements” & a

“fundamental lack of understanding” of “fundamental lack of understanding” of hair comparisons: hair comparisons:

““The witness’s use of probabilities is The witness’s use of probabilities is contrary to the fact that there is not – and contrary to the fact that there is not – and never was – a well established probability never was – a well established probability theory for hair comparison... . If this theory for hair comparison... . If this witness has evaluated hair in over 700 witness has evaluated hair in over 700 cases as he claims in his testimony, then it cases as he claims in his testimony, then it is reasonable to assume that he had made is reasonable to assume that he had made many other misattributions.”many other misattributions.” Innocence Project, Innocence Project, Peer Review ReportPeer Review Report

Houston - DNAHouston - DNA ““[T]he validity of almost any case [T]he validity of almost any case

that has relied upon evidence that has relied upon evidence produced by the lab is produced by the lab is questionable.” questionable.”

Rodney Ellis, Editorial, Rodney Ellis, Editorial, Want Tough on Want Tough on Crime? Start by Fixing HPD Lab.,Crime? Start by Fixing HPD Lab., Houston Chronicle, Sept. 5, 2004 (state Houston Chronicle, Sept. 5, 2004 (state representative).representative).

Proficiency Test: Proficiency Test: Fingerprints Fingerprints

“ “[T]he FBI examiners got very high [T]he FBI examiners got very high proficiency grades, but the tests they proficiency grades, but the tests they took did not. ... [T]he proficiency tests took did not. ... [T]he proficiency tests are less demanding than they should be.”are less demanding than they should be.”

New Scotland Yard examiner: “It’s not New Scotland Yard examiner: “It’s not

testing their ability. … And if I gave my testing their ability. … And if I gave my experts these tests, they’d fall about experts these tests, they’d fall about laughing.”laughing.” U.S. v. Llera Plaza, 188 F. Supp. 2d 549, 558 (E.D. U.S. v. Llera Plaza, 188 F. Supp. 2d 549, 558 (E.D.

Pa. 2002)Pa. 2002)

Proficiency Test: Proficiency Test: HandwritingHandwriting

““Mr. Cawley testified that he achieved a Mr. Cawley testified that he achieved a 100% passage rate on the proficiency tests 100% passage rate on the proficiency tests that he took and that all of his peers that he took and that all of his peers alwaysalways passed their proficiency tests. Mr. passed their proficiency tests. Mr. Cawley said that his peers Cawley said that his peers alwaysalways agreed agreed with each others’ results and with each others’ results and alwaysalways got it got it right. Peer review in such a ‘Lake right. Peer review in such a ‘Lake Woebegone’ environment is not Woebegone’ environment is not meaningful.”meaningful.”

U.S. v. Lewis, 220 F. Supp. 2d 548, 554 (S.D. U.S. v. Lewis, 220 F. Supp. 2d 548, 554 (S.D. W.Va. 2002)W.Va. 2002)

Regulate Crime LabsRegulate Crime Labs Accreditation of labsAccreditation of labs

E.g., New York, Oklahoma, Texas, VirginiaE.g., New York, Oklahoma, Texas, Virginia

Certification of examinersCertification of examiners Proficiency testingProficiency testing

Standardization of proceduresStandardization of procedures Published protocolsPublished protocols

Federal LegislationFederal Legislation

““Justice for All” ActJustice for All” Act Requires states to have an investigative Requires states to have an investigative

entityentity

DNA Identification ActDNA Identification Act Requires accreditation of DNA labs Requires accreditation of DNA labs

within 2 yearswithin 2 years

Defense ExpertsDefense Experts Ake v. Oklahoma, 470 U.S. 68 (1985) Ake v. Oklahoma, 470 U.S. 68 (1985)

(due process right to defense expert for (due process right to defense expert for indigents).indigents). Giannelli, Giannelli, Ake v. Oklahoma: The Right to Expert Ake v. Oklahoma: The Right to Expert

Assistance in a Post-Daubert, Post-DNA WorldAssistance in a Post-Daubert, Post-DNA World, 89 , 89 Cornell L. Rev. 1305 (2004).Cornell L. Rev. 1305 (2004).

Defense expert not appointed in a DNA Defense expert not appointed in a DNA

case “because of a shortage of county case “because of a shortage of county funds.”funds.” Prater v. State, 820 S.W.2d 429, 439 (Ark. 1991).Prater v. State, 820 S.W.2d 429, 439 (Ark. 1991).

State v. HuchtingState v. Huchting ““[W]e disagree with [accused’s] contention [W]e disagree with [accused’s] contention

that the average attorney is ill-equipped to that the average attorney is ill-equipped to defend against [DNA] evidence. To the defend against [DNA] evidence. To the contrary, law libraries – contrary, law libraries – i.e.,i.e., law journals, law journals, practitioners’ guides, annotated law practitioners’ guides, annotated law reports, CLE materials, etc – are teeming reports, CLE materials, etc – are teeming with information and advice for lawyers with information and advice for lawyers preparing to deal with DNA evidence preparing to deal with DNA evidence trial.”trial.”

927 S.W.2d 411, 420 (Mo. Ct. App. 1996)927 S.W.2d 411, 420 (Mo. Ct. App. 1996)

ABA Resolutions ABA Resolutions (2004)(2004)

1. Crime laboratories and medical examiner 1. Crime laboratories and medical examiner offices should be accredited, examiners offices should be accredited, examiners should be certified, and procedures should should be certified, and procedures should be standardized and published to ensure the be standardized and published to ensure the validity, reliability, and timely analysis of validity, reliability, and timely analysis of forensic evidence.forensic evidence.

2. Crime laboratories and medical examiner 2. Crime laboratories and medical examiner offices should be adequately funded.offices should be adequately funded.

3. The appointment of defense experts for 3. The appointment of defense experts for indigent defendants should be required indigent defendants should be required whenever reasonably necessary to the whenever reasonably necessary to the defense. defense.