PROBABILITY AND STATISTICS IN THE LAW

PROBABILITY AND STATISTICS IN THE LAW

Philip Dawid

University College London

STATISTICS = LAWSTATISTICS = LAW

• Interpretation of evidence

• Hypothesis testing

• Decision-making under uncertainty

INGREDIENTSINGREDIENTS

Prosecution Hypothesis G

Defence Hypothesis G

Evidence E

– or posterior odds:

)|( EGP

)|(

)|(

E

E

GP

GP

• BAYESIAN APPROACH• BAYESIAN APPROACH

• FREQUENTIST APPROACH• FREQUENTIST APPROACH

and

)|( GP E

)|( GP E

Find posterior probability of guilt:

Look at & effect on

decision rules

SALLY CLARKSALLY CLARK

• Sally and Stephen Clark’s sons Christopher and Harry died suddenly at ages 11 and 8 weeks, in Sally’s care

• The Clarks claimed that their children had died from natural causes (SIDS??)

• Contested prosecution medical evidence of maltreatment

–SALLY CONVICTED OF MURDER

• A paediatrician testified that, for a family like the Clarks, the probability of one child dying from SIDS is 1 in 8,543

At Trial:At Trial:

• He was asked if the report calculated “the risk of two infants dying in that family by chance.”

• Answer: Yes, you have to multiply 1 in 8,543 times 1 in 8,543 …. [the CESDI study] points out that it’s approximately a chance of

1 in 73 million

WHAT TO THINK?WHAT TO THINK?

• Clear intuitive argument against independence (and thus calculation of “1 in 73 million”)

• BUT probability of 2 natural deaths remains very small

HOW TO CONSIDER?

Prosecutor’s FallacyProsecutor’s Fallacy

)|( EGP

)|( GP E• = 1 in 73 million

• Probability of deaths arising from natural causes is 1 in 73 million

• = 1 in 73 million

• Probability of innocence is 1 in 73 million

Alternatively…Alternatively…

• P(2 babies die of SIDS) = 1/73 million

• P(2 babies die of murder) = 1/2000 million

BOTH figures are equally relevant to the decision between the two possible causes

BAYES:BAYES:

POSTERIOR

ODDS

)(

)(

)(

)(

)|(

)|(

GP

GP

GP

GP

GP

GP

|E

|E

E

E

=LIKELIHOOD

RATIO PRIOR

ODDS

If prior odds = 1/2000 million posterior odds = 0.0365

%5.3)|( EGP

73m ??

IDENTIFICATION EVIDENCEIDENTIFICATION EVIDENCE

:

:

:

S

C

I i

Assume

million10/1)(])[,|(

1])[,|(

xIPxIGP

xIGP

CS

S

E

E

“match probability”

),(: xIxI SC E

Individual i

Criminal

Suspect Evidence:

Match

PROSECUTOR’S ARGUMENTPROSECUTOR’S ARGUMENT

The probability of a match having arisen by innocent means is 1/10 million.

So )|( EGP = 1/10 million

– i.e. )|( EGP is overwhelmingly close to 1

– CONVICT

DEFENCE ARGUMENTDEFENCE ARGUMENT

• Absent other evidence, there are 30 million potential culprits

• 1 is GUILTY (and matches)

• ~3 are INNOCENT and match

• Knowing only that the suspect matches, he could be any one of these 4 individuals

• So 41)|( EGP

–ACQUIT

BAYESBAYES POSTERIOR ODDS = (10 MILLION) “PRIOR” ODDS

)|(

)|(

BGP

BGP

PROSECUTOR’S argument OK if

Only BAYES allows for explicit incorporation of B

2/1)|( BGP

DEFENCE argument OK if million 1/30)|( BGP

MPLR /1

The Island ProblemThe Island Problem

• N+1 on island: N (100) innocent, 1 guilty

• Match, probability = P (0.004)

• Prosecution:

• Defence:

PGP 1)|( E

)1/(1)|( NPGP E

(0.996)

(0.714)

Other ArgumentsOther Arguments

Let number of individuals i having Ii = x be M

)|()|( 1 EE MEGP

– need distribution of M given

Note: Initially

1),|( MMGP E

So

),(: xIxI SC E

):1(Bin~ PNM

Argument 1Argument 1

• Evidence tells us

• So

1M

)1);;1(Bin~|()|( 1 MPNMMEGP E

(0.902)

Argument 2Argument 2

• Evidence tells us 1 (guilty) individual has x

• Our of remaining N innocents, number with x is ; while

• So

):(Bin~ PNM

(0.824)

MM 1

PN

P

PNMMEGPN

)1(

)1(1

));(Bin1~|()|(1

1

E

Argument 3Argument 3

• Evidence E is equivalent to 2 successes on 2 Bernoulli trials with replacement

• So

• So

• Then (0.714

– as for defence)

2

1)|(

N

mmMP E

mNm PPm

NmmMP

12 )1(

1)|( E

)1/(1

)|()|( 1

NP

MEGP

EE

DENIS ADAMSDENIS ADAMS

– Match probability = 1/200 million

1/20 million

1/2 million

Doesn’t fit descriptionVictim: “not him”Unshaken alibiNo other evidence to link to crime

• Sexual assault• DNA match

BAYES’S THEOREMBAYES’S THEOREM

POSTERIOR ODDS on guilt

= LIKELIHOOD RATIO PRIOR ODDS

= 2 million (1 / 200,000)

= 10 (10:1)

Posterior probability of guilt = 10/11

= 91%

Reasonable doubt – ACQUIT

WHAT ABOUT OTHER EVIDENCE?WHAT ABOUT OTHER EVIDENCE?

• Didn’t fit description• Victim: “not him”• Unshaken alibi

LR = 0.1 / 0.9 = 1/9

LR = 0.25 / 0.5 = 1/2

Apply Bayes’s Theorem again:Final odds on guilt = 10 1/9 1/2

}

= 5/9 (5:9) (probability of guilt = 5/14 = 35%)

Dependence on Match Probability

Match probability 1/200m 1/20m 1/2m

Posterior probability of guilt

98% 85% 35%

– number of noughts does matter!

DATABASE SEARCHDATABASE SEARCH

• Crime trace, frequency (match probability) 1 in 1 million

• Search Police DNA database (D) of size 10,000

• Find unique match: “John Smith” (S)

• No other evidence

Defence CaseDefence Case

• Probability of finding a match in database if innocent ~ 10,000 (1/1,000,000) = 1/100

• Match probability of 1/100 is not convincing evidence

• Evidence against John Smith is (significantly) weakened by virtue of database search

– ACQUIT

Prosecution CaseProsecution Case

• We have examined 10,000 individuals

• Of these, 9,999 found not to match

• This has reduced the pool of potential alternative culprits

• Evidence against John Smith is (marginally) strengthened by virtue of database search

– CONVICT

Which likelihood ratio?Which likelihood ratio?• Hypothesis HS: “John Smith did it” is data-

dependent• Replace by hypothesis HD: “Someone in

database D did it”– equivalent after search identifies S (but not before)

• LR = 1/(match probability) is now only 100– weak evidence?

• But HD is a priori 10,000 times more probable than HS

– posterior odds the same! – agrees with prosecution argument

Multiple StainsMultiple Stains

• 2 DNA stains– 1 on sheet, 1 on pillow

– assume 2 perpetrators, 1 stain from each

• John Smith (S) matches pillow stain– associated “match probability” P

• What are appropriate hypotheses, likelihoods, inferences?

HypothesesHypotheses• S left one of 2 stains

• S left pillow stain

• S left pillow stain

• S left neither stain

• S left neither stain

• S didn’t leave pillow stain

2/PLR

PLR

)1(2/)2( PLR

( = prior probability S is guilty)

What to present in Court?

• Hypotheses equivalent (only) after data

• Different prior odds

• Identical posterior odds

Mixed StainsMixed Stains

• Crime trace containing DNA from more than 1 contributor–Rape

–Scuffleetc

O. J. SIMPSONO. J. SIMPSON

Crime

OJS

RG

A

B

C

Marker DQ-Frequency

13%

20%

28%

“MATCH” to OJS

Allele

MATCH PROBABILITY?MATCH PROBABILITY?• PROSECUTION:

Frequency of OJS type

AB: 5%• DEFENCE:

Combined frequency of all matching types

AA, AB, AC, BB, BC, CC: 39%

• LR approach assuming Goldman (AC) in mixture:

AB, BB, BC: 21%• LR approach not assuming Goldman in mixture:

(more complex calculation) ~ 21%

MISSING DNA DATAMISSING DNA DATA

• What if we can not obtain DNA from the suspect ? (or other relevant individual?)

• Sometimes we can obtain indirect information by DNA profiling of relatives

• But analysis is complex and subtle…

HANRATTYHANRATTY

• James Hanratty convicted and executed in 1962

• DNA profile from crime items analysed in 1998

• Population frequency less than 1 in 2.5 million

• DNA profiles from mother and brother – “consistent with” crime DNA being from Hanratty

(“A6” murder and rape, 1961)

PRESS REPORTSPRESS REPORTS

• “There is a 1 in 2.5 million chance that Hanratty was not the A6 killer”

• “The DNA is 2.5 million times more likely to belong to Hanratty than anyone else”

Likelihood Ratio based on profiles of mother and brother (complex calculation):

440

– even though no direct match to Hanratty!

DISPUTED PATERNITYDISPUTED PATERNITY

• MOTHER (m1) of CHILD (c1) claims that PUTATIVE FATHER (pf) is its TRUE FATHER (tf)

But DO have DNA profiles from:

• Two full BROTHERS (b1, b2) of PUTATIVE FATHER

undisputedchild

disputedchild

brothers

• His UNDISPUTED CHILD (c2) and its MOTHER (m2)

• DNA profiles from MOTHER and CHILD No profile from PUTATIVE FATHER

DECISION AIDDECISION AID“PROBABILISTIC EXPERT SYSTEM”

– embodies probabilistic relationships (between inherited genes)

ANALYSISANALYSIS

• Measurements for 12 DNA markers on all 6 individuals

• Enter data, “propagate” through system

• Overall Likelihood Ratio in favour of paternity:

~1300

FURTHER COMPLEX DNA CASES

FURTHER COMPLEX DNA CASES

• Contamination

• Laboratory errors, mix-up, fraud

• Relatives

– …

• Statistics

• Law

• Crime Science

• Psychology

• Economics• Philosophy of

Science

• Geography• Medicine• Ancient History• Computer Science• Education• …

EVIDENCE, INFERENCE AND ENQUIRY

EVIDENCE, INFERENCE AND ENQUIRY

www.evidencescience.org

EVIDENCE SCIENCEEVIDENCE SCIENCE

• Subject- and substance-blind approach• Inference, explanation, causality• Recurrent patterns of evidence• Narrative, argumentation, analysis, synthesis• Cognitive biases• Formal rules• Decision aids• Interdisciplinary studies• …