icfis, leiden 21 august 2014 norman fenton queen mary university of london and agena ltd
Post on 11-Jan-2016
17 Views
Preview:
DESCRIPTION
TRANSCRIPT
ICFIS, Leiden
21 August 2014
Norman Fenton Queen Mary University of London and Agena Ltd
norman@agena.co.uk
Limitations and opportunitiesof the likelihood ratio approach
for evidence evaluation
Fenton, N. E., D. Berger, D. Lagnado, M. Neil and A. Hsu, "When ‘neutral’ evidence still has probative value (with implications from the Barry George Case)", Science and Justice, Volume 54, Issue 4, Pages 274–287, July 2014
Overview
1. Revisiting the Likelihood Ratio – first principles, theoretical benefits and limitations
2. Revisiting the LR in two well-known cases3. Why Bayesian networks are needed 4. Conclusions and way forward
1. REVISITING THE LIKELIHOOD RATIO – FIRST PRINCIPLES
Was Mrs Peacock the murderer?H: “Mrs Peacock guilty”
P(H) = 1/6
E: “The murderer was female
P(E | H) = 1
P(E) = 1/2
By Bayes P(H | E) = 1/3
When does evidence E support a hypothesis H?
When our belief in H increases as a result of observing E, i.e. when P(H | E) > P(H)So, suppose E supports H and that H’ is an alternative mutually exclusive hypothesis. Can we conclude that our belief in H’ must have decreased? NO!! (suppose H’ is “Miss Scarlett is murderer”)Except when H’ = not H
P(H’ | E) = 1 – P(H | E) < 1 – P(H) = P(H’)
A simple formal definition of probative value of evidence
The ratio R: • R > 1 means E supports H• R < 1 means E supports not H• R = 1 means E is neutral for H
Why do we never see this definition used?
Because of obsessive and irrational fear of the explicit
PRIOR P(H)
Instead the Likelihood Ratio is usedIn addition to H we have to consider an alternative
mutually exclusive hypothesis H’
L= By Bayes Theorem providing that H’=not H we can
conclude• L>1 if and only if R>1• L<1 if and only if R<1• L=1 if and only if R=1So L is a valid measure of the probative value of
evidence E for H (when H’ = not H)
Benefits of the LRSimple formula for probative value of evidenceNo need to explicitly consider prior for HForces forensic experts to consider the likelihood of both the prosecution hypothesis and the defence hypothesis
But note:It is meaningless to talk
about the Likelihood Ratio being a measure of the
probative value of evidence without explicit reference to
Bayes Theorem
And especially note…..When
H’ ≠ not H
the notion that the LR is a measure of probative value of evidence is tenuous and potentially misleading
When H’ ≠ not H…• Knowing that LR>1 just tells us that the ratio
of posterior probabilities (of H and H’) is greater than the ratio of prior probabilities
• So all we can conclude is E supports H more than it supports H’. But E may not support H at all because we can still have P(H|E) < P(H)
• Similarly LR=1 only tells us E supports both H and H’ equally. That does NOT mean E is neutral; P(H|E) might be very different to P(H)
Was Mrs Peacock the murderer?H: “Mrs Peacock guilty”
E: “The murderer was female
P(E | H) = 1P(E | not H) = 2/5
LR= 2.5
But if H’: “Miss Scarlet guilty”P(E | H’) = 1
LR=1
Issues and limitations of the LR• ‘probative value’ is not what people think it means
when H’ is different from not H• But it is difficult to work with exhaustive pairs of
hypotheses• Priors can never be truly ignored• Evidence E is rarely ‘simple’ – normally involves E1
and E2 that require separate likelihoods• Can be difficult even to avoid non-mutually exclusive
hypotheses in practice• Even if we get it all right LR of source level hypotheses
tells us NOTHING about LR of offense level hypotheses
ExampleFred and Joe live at the same address. Gun X is registered to that address. Bob is found murdered from a gun shot. Evidence E: “there is a gun in Fred’s house with FDR that matched that from the crime scene.” Fred is charged with the murder of Bob. The offence level hypotheses are: Hp: Fred fired the shot that killed Bob not Hp: Fred did not fire the shot that killed Bob
The source level hypotheses are:Hp1: Fred owned gun that killed Bob not Hp1: Fred did not own gun that killed Bob
Some reasonable assumptions
LR=1 for source level hypotheses…..but E has real probative value on Hp
Essentially irrelevant
Prior state of the BN
Calculating the probability of evidence E under the two values for H1p
P(E | not H1p) = 0.9891 (unchanged from prior)
P(E | H1p) = 0.9891 (unchanged from prior)
Evidence is observed
Probability of Hp jumps from 1% to over 9%
..the evidence is certainly not neutral
2. REVISITING THE LR IN TWO WELL-KNOWN CASES
R v Sally Clark 1999-2003
Convicted and ultimately cleared of murdering her 2 children
Sally Clark Revisited: A new issue in the probability experts’ reasoning
Hd : Sally Clark’s two babies died of SIDSHp : Sally Clark murdered her two babies
“(Prior) probability of Hd over 100 times greater than (prior) probability of Hp”“So assuming LR of 5 posterior of Hd still 20 greater
Hd : Sally Clark’s two babies died of SIDSHp : Sally Clark murdered at least one of her two babies.
(Prior) probability of Hd only 2.5 times greater than the (prior) probability of Hp
R v Barry George, 2001-2007
Jill Dando
R v Barry George (revisiting the Appeal Court judgment)
Hp: Hypothesis “BG was man who shot JD”E: “Single particle of FDR matching that from the gun that killed JD found in BG coat pocketDefence likelihood P(E|not Hp) = 1/100…But Prosecution likelihood P(E| Hp) = 1/100So LR = 1 and evidence ‘has no probative value’But the appeal transcript suggests a problem…
Fenton, N. E., D. Berger, D. Lagnado, M. Neil and A. Hsu, "When ‘neutral’ evidence still has probative value (with implications from the Barry George Case)", Science and Justice, Volume 54, Issue 4, Pages 274–287, July 2014
Confusion from experts about the hypotheses
• Not clear that Hp stated was really the same prosecution hypothesis considered by the experts – H1p: “The particle found in BG’s pocket came from a gun
fired by BG”.– H2p: “The particle found in BG’s pocket came from the gun
that killed JD”.
• Transcript suggests the experts did not adhere to the assumption that defence hypothesis Hd was simply “not Hp”, i.e. “BG was not the man who shot JD”.– H1d: “Integrity of BG coat was corrupted”
LR=1?
P(E | Hp) = P(E | H1d) but the evidence E is not neutral as concluded by expert and accepted by the court. It favours Hp.
3. WHY BAYESIAN NETWORKS ARE NEEDED
More comprehensive BN model needed in BG case
Target is type X
Target is source
Source is type X
Target tested X
Source tested X
Even single piece of forensic match evidence is NOT a 2-node BN
Source is type X
Bayesian nets: what we need to stress
Separate out assumptions from calculationsCan incorporate subjective, expert judgementCan address the standard resistance to using subjective probabilities by using ranges.Easily show results from different assumptions
…but must be seen as the ‘calculator’
The potential of Bayesian Networks
“I assert that we now have a technology that is ready for use, not just by the scholars of evidence, but by trial lawyers.”
Edwards, W. (1991). "Influence Diagrams, Bayesian Imperialism, and the Collins case: an appeal to reason." Cardozo Law Review 13: 1025-107
4. CONCLUSIONS AND WAY FORWARD
Summary• LR and probative value of evidence may not
be what people think it is• In isolation the LR may be highly misleading• Doing things correctly requires fuller models
- BNs• But Bayesian arguments cannot be
presented from first principles
Blatant Plug for Book
CRC Press, ISBN: 9781439809105 , ISBN 10: 1439809100
top related