![Page 1: Information value: the value of evidence Dr David J Marsay, C.Math FIMA A presentation to 19 ISMOR 29.08.2002](https://reader034.vdocuments.site/reader034/viewer/2022050802/56649e9e5503460f94b9f4be/html5/thumbnails/1.jpg)
![Page 2: Information value: the value of evidence Dr David J Marsay, C.Math FIMA A presentation to 19 ISMOR 29.08.2002](https://reader034.vdocuments.site/reader034/viewer/2022050802/56649e9e5503460f94b9f4be/html5/thumbnails/2.jpg)
Information value:the value of evidenceDr David J Marsay, C.Math FIMAA presentation to 19 ISMOR29.08.2002
![Page 3: Information value: the value of evidence Dr David J Marsay, C.Math FIMA A presentation to 19 ISMOR 29.08.2002](https://reader034.vdocuments.site/reader034/viewer/2022050802/56649e9e5503460f94b9f4be/html5/thumbnails/3.jpg)
3
Contents
1 Introduction
2 Examples
3 Theory
4 World-view
5 Implications
6 Conclusions
![Page 4: Information value: the value of evidence Dr David J Marsay, C.Math FIMA A presentation to 19 ISMOR 29.08.2002](https://reader034.vdocuments.site/reader034/viewer/2022050802/56649e9e5503460f94b9f4be/html5/thumbnails/4.jpg)
IntroductionSection 1
![Page 5: Information value: the value of evidence Dr David J Marsay, C.Math FIMA A presentation to 19 ISMOR 29.08.2002](https://reader034.vdocuments.site/reader034/viewer/2022050802/56649e9e5503460f94b9f4be/html5/thumbnails/5.jpg)
5
Introduction
• Information is the life-blood of military C4ISR.
• Any time we prefer one set of information to another we implicitly ‘value’ it.
• We think we could do better:
– lessons identified.
– studies.
• Specifically needed to support UK MOD’s ARP 14 ‘Battlefield picture compilation’.
1
![Page 6: Information value: the value of evidence Dr David J Marsay, C.Math FIMA A presentation to 19 ISMOR 29.08.2002](https://reader034.vdocuments.site/reader034/viewer/2022050802/56649e9e5503460f94b9f4be/html5/thumbnails/6.jpg)
6
Introduction
• We use P(A|B) to denote ‘the probability of A given B’
– P(A) is used to denote the [prior] probability.
• For hypotheses {H} and evidence E:
– Shannon’s ‘entropy’ is calculated from the ‘final’ probabilities, {P(H|E)}.
– Jack Good’s ‘weight of evidence’ is calculated from the likelihoods, {P(E|H)}.
– According to Bayes’ rule, the probabilities can be calculated from the likelihoods and ‘the priors’.
1
![Page 7: Information value: the value of evidence Dr David J Marsay, C.Math FIMA A presentation to 19 ISMOR 29.08.2002](https://reader034.vdocuments.site/reader034/viewer/2022050802/56649e9e5503460f94b9f4be/html5/thumbnails/7.jpg)
ExamplesSection 2
![Page 8: Information value: the value of evidence Dr David J Marsay, C.Math FIMA A presentation to 19 ISMOR 29.08.2002](https://reader034.vdocuments.site/reader034/viewer/2022050802/56649e9e5503460f94b9f4be/html5/thumbnails/8.jpg)
8
Examples
Control: Suppose that a source sends accurate data to a deterministic machine.
• Shannon’s concept does not apply. Nor does the notion of ‘priors’.
• The value of the data can be determined by valuing the function of the machine - no fancy method needed.
• The likelihoods make sense. They are 0 or 1.
2
![Page 9: Information value: the value of evidence Dr David J Marsay, C.Math FIMA A presentation to 19 ISMOR 29.08.2002](https://reader034.vdocuments.site/reader034/viewer/2022050802/56649e9e5503460f94b9f4be/html5/thumbnails/9.jpg)
9
Examples
Command - ‘soft’ aspects:
• For an information artefact (e.g., an INTSUM) to represent the same information implies that all recipients had the same priors. Thus everyone receives everything in the same order.
– Is this realistic?
• Alternatively, one could define some privileged ‘central’ viewpoint for which the information is defined.
– Does this fit doctrine?
– Is it helpful?
2
![Page 10: Information value: the value of evidence Dr David J Marsay, C.Math FIMA A presentation to 19 ISMOR 29.08.2002](https://reader034.vdocuments.site/reader034/viewer/2022050802/56649e9e5503460f94b9f4be/html5/thumbnails/10.jpg)
10
Examples
Command - ‘soft’ aspects:
• The likelihoods {P(E|H)} are a rating of the source of E. They are thus relatively ‘objective’, ‘knowable’ and ‘shareable’.
• Likelihoods relate to current practice (reliability, accuracy).
2
![Page 11: Information value: the value of evidence Dr David J Marsay, C.Math FIMA A presentation to 19 ISMOR 29.08.2002](https://reader034.vdocuments.site/reader034/viewer/2022050802/56649e9e5503460f94b9f4be/html5/thumbnails/11.jpg)
11
Examples
Compilation:
• The work being reported on has looked at the relatively ‘hard’ problem of compilation, particularly ‘Battlefield picture compilation’ under ARP 14.
• Weights of evidence can be used. (See accompanying paper.)
• When is this reliable?
2
![Page 12: Information value: the value of evidence Dr David J Marsay, C.Math FIMA A presentation to 19 ISMOR 29.08.2002](https://reader034.vdocuments.site/reader034/viewer/2022050802/56649e9e5503460f94b9f4be/html5/thumbnails/12.jpg)
Theory
Section 3
![Page 13: Information value: the value of evidence Dr David J Marsay, C.Math FIMA A presentation to 19 ISMOR 29.08.2002](https://reader034.vdocuments.site/reader034/viewer/2022050802/56649e9e5503460f94b9f4be/html5/thumbnails/13.jpg)
13
Theory
Jack Good’s evidence:
• Likelihoods are often straightforward.E.g., P(‘Heads’|‘Fair Coin’) = 0.5 by definition.
• Lab and field testing traditionally establish, in effect, likelihoods.
• Surprise = -log(likelihood).
• Weight of evidence (woe) is surprise, normalised by the prior expected surprise for the same evidence. (So that only ‘relevant detail’ counts.)
3
![Page 14: Information value: the value of evidence Dr David J Marsay, C.Math FIMA A presentation to 19 ISMOR 29.08.2002](https://reader034.vdocuments.site/reader034/viewer/2022050802/56649e9e5503460f94b9f4be/html5/thumbnails/14.jpg)
14
Theory
Evidence is more fundamental than Shannon’s information
• Shannon’s entropy is expected surprise.
• The more useful cross-entropy is likely surprise.
• Woe supports alternative decision methods, such as sequential testing, hypothesis testing.
3
![Page 15: Information value: the value of evidence Dr David J Marsay, C.Math FIMA A presentation to 19 ISMOR 29.08.2002](https://reader034.vdocuments.site/reader034/viewer/2022050802/56649e9e5503460f94b9f4be/html5/thumbnails/15.jpg)
15
Some questionable assumptions
• Shannon assumes that systems of interest are ‘Markov’.
• Shannon noted that ‘state-determined systems’ are ‘Markov’ with probability 1.
• But Smuts (e.g.) noted that evolution drives dynamical systems to adopt synergistic ‘emergent’ structures.
• These had a priori probability 0.
• So for social systems, international relations, military conflict ... we cannot rely on Shannon’s ‘information’.
3
![Page 16: Information value: the value of evidence Dr David J Marsay, C.Math FIMA A presentation to 19 ISMOR 29.08.2002](https://reader034.vdocuments.site/reader034/viewer/2022050802/56649e9e5503460f94b9f4be/html5/thumbnails/16.jpg)
16
Some questionable assumptions
• But can likelihoods be used?
• If we abandon Markov models, how are we to judge if a given algebra of likelihoods is valid?
• We need a ‘world meta-model’ to replace Markov.
3
![Page 17: Information value: the value of evidence Dr David J Marsay, C.Math FIMA A presentation to 19 ISMOR 29.08.2002](https://reader034.vdocuments.site/reader034/viewer/2022050802/56649e9e5503460f94b9f4be/html5/thumbnails/17.jpg)
World-viewSection 4
![Page 18: Information value: the value of evidence Dr David J Marsay, C.Math FIMA A presentation to 19 ISMOR 29.08.2002](https://reader034.vdocuments.site/reader034/viewer/2022050802/56649e9e5503460f94b9f4be/html5/thumbnails/18.jpg)
18
• Allows one to investigate emergence within complex systems.
• Evidence of piece-wise Markov behaviour.
• Developed under the MOD CRP TGs 0,5,10.
SMUTS(synthetic modelling of uncertain temporal systems)
Delayed Double Vipert = 0.0502
BUBs2D5.5
t = 0.2005
4
![Page 19: Information value: the value of evidence Dr David J Marsay, C.Math FIMA A presentation to 19 ISMOR 29.08.2002](https://reader034.vdocuments.site/reader034/viewer/2022050802/56649e9e5503460f94b9f4be/html5/thumbnails/19.jpg)
19
Alternative ideas
• I postulate a model in which systems of interest to the military are Markov in space-time ‘zones’, with more interesting transitions at their boundaries.
• Thus Markov locally, but not globally.
• In essence emergence only happens when an over-adaptation is exploited. (E.g. Ashby, Piaget.)
• Thus, as long as we can learn at least as quickly, we should be able to recognise these situations too.
4
![Page 20: Information value: the value of evidence Dr David J Marsay, C.Math FIMA A presentation to 19 ISMOR 29.08.2002](https://reader034.vdocuments.site/reader034/viewer/2022050802/56649e9e5503460f94b9f4be/html5/thumbnails/20.jpg)
20
Supporting evidence
Applications to, and experiences of:
• warfare
• economics
• international relations.
(My subjective view)
4
![Page 21: Information value: the value of evidence Dr David J Marsay, C.Math FIMA A presentation to 19 ISMOR 29.08.2002](https://reader034.vdocuments.site/reader034/viewer/2022050802/56649e9e5503460f94b9f4be/html5/thumbnails/21.jpg)
21
Reuters data for the Balkans, the 90s4
Evidence of locally Markov behaviour
Balkans April 1989- March 1999KEDS data from www.ukans.edu/~keds
Entropy / Value Aggregated Monthly Phase plot
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
0 10 20 30 40 50 60 70 80 90 100
RSS Value
Tra
nsacti
on
al
En
tro
py,
aft
er
Sh
an
no
n
4/89-4/91
4/91-11/91
11/91-10/95
10/95-4/97
4/97-9/97
9/97-1/99
1/99-3/99
![Page 22: Information value: the value of evidence Dr David J Marsay, C.Math FIMA A presentation to 19 ISMOR 29.08.2002](https://reader034.vdocuments.site/reader034/viewer/2022050802/56649e9e5503460f94b9f4be/html5/thumbnails/22.jpg)
ImplicationsSection 5
![Page 23: Information value: the value of evidence Dr David J Marsay, C.Math FIMA A presentation to 19 ISMOR 29.08.2002](https://reader034.vdocuments.site/reader034/viewer/2022050802/56649e9e5503460f94b9f4be/html5/thumbnails/23.jpg)
23
Implications for ‘information’
Technical differences:
• The difference between the expected weight of evidence (woe) and Shannon’s entropy is not a constant.
• Systems of interest tend to have ‘long range’ sources of uncertainty, in addition to the ‘local’ entropy.
• We need to allow for this and ‘expect the unexpected’ to achieve robustness.
5
![Page 24: Information value: the value of evidence Dr David J Marsay, C.Math FIMA A presentation to 19 ISMOR 29.08.2002](https://reader034.vdocuments.site/reader034/viewer/2022050802/56649e9e5503460f94b9f4be/html5/thumbnails/24.jpg)
24
Implications for ‘information’
Some cases where Shannon might not be appropriate
• Poor ‘local’ information.
• The ‘situation’ cannot necessarily be recognised.
• The ‘target’ is adaptable (particularly if adapting against us).
5
![Page 25: Information value: the value of evidence Dr David J Marsay, C.Math FIMA A presentation to 19 ISMOR 29.08.2002](https://reader034.vdocuments.site/reader034/viewer/2022050802/56649e9e5503460f94b9f4be/html5/thumbnails/25.jpg)
25
Implications for ‘information’
Typical symptoms that Shannon is inadequate:
• Mistakes often reflect a need to validate assumptions.
• Ossification, atrophy and vulnerability (Ashby / Piaget)
5
![Page 26: Information value: the value of evidence Dr David J Marsay, C.Math FIMA A presentation to 19 ISMOR 29.08.2002](https://reader034.vdocuments.site/reader034/viewer/2022050802/56649e9e5503460f94b9f4be/html5/thumbnails/26.jpg)
26
Implications for ‘information’
Notes:
• We can’t expect to have considered all possible hypotheses in advance.
• However, we do know when the truth is ‘something else’ because the weights of evidence are poor for the assumed hypotheses.
• Thus we can detect deception and ‘fixation’ (a form of self-deception).
5
![Page 27: Information value: the value of evidence Dr David J Marsay, C.Math FIMA A presentation to 19 ISMOR 29.08.2002](https://reader034.vdocuments.site/reader034/viewer/2022050802/56649e9e5503460f94b9f4be/html5/thumbnails/27.jpg)
ConclusionsSection 6
![Page 28: Information value: the value of evidence Dr David J Marsay, C.Math FIMA A presentation to 19 ISMOR 29.08.2002](https://reader034.vdocuments.site/reader034/viewer/2022050802/56649e9e5503460f94b9f4be/html5/thumbnails/28.jpg)
28
Conclusions
• The common concepts of information assume that systems are globally ‘simple’.
• Our systems of interest are not simple, but may be piece-wise ‘simple’.
• Jack Good’s ‘weight of evidence’ can be used to ‘bridge’ ‘islands of simplicity’.
• Using ‘weight of evidence’ gives significant ‘added value’ to using just Shannon information.
6
![Page 29: Information value: the value of evidence Dr David J Marsay, C.Math FIMA A presentation to 19 ISMOR 29.08.2002](https://reader034.vdocuments.site/reader034/viewer/2022050802/56649e9e5503460f94b9f4be/html5/thumbnails/29.jpg)