TWO CONCEPTS OF PROBABILITY

So far we have done probability calculations without asking whether “probability” is a well-understood concept.

In fact, probability has been studied systematically for only 300 years or so.

Philosophers, mathematicians, logicians and statisticians disagree about the nature of probability.

Two fundamental probability ideas:

Belief: probability is a measure of how certain your beliefs are or should be.

Frequency: probability is the relative frequency of an outcome on repeated trials of a chance set-up.

Let’s examine the differences between these two.

1

(I) “This die is fair. The probability of rolling a six is 1/6.”

This claim is true or false regardless of what we know.

If it is true, it is because of some features of the die.

It could be explained by reference to the physical properties of the die.

We can test this claim by tossing the die and seeing how often six comes up.

In short: this claim is about the die. It is about something in the world.

When we speak this way, we are using “probability” in the frequency sense.

2

There are different ways to think of the frequency sense of “probability”:

In the long run, the relative frequency of sixes will be 1/6.

The die has a tendency to come up six 1/6 of the time.

The die has a propensity or disposition to come up six 1/6 of the time.

The die has a symmetrical geometric structure.

We will label these ideas:

Frequency-type probability

3

“Unicorns probably never existed.”

Consider some reasons for this claim:

1. There are no fossil records of unicorns.

2. Unicorns appear in fanciful myths.3. No one has ever observed a unicorn.

On this basis one says:

(II) “Considering all the evidence, it is 80% probable that unicorns never existed.”

This is a statement about the proposition:

Unicorns never existed.

It says that it is 80% likely that this proposition is true.

4

What are the differences between (I) and (II)?

1. (II) is not true or false regardless of what we know. It is about evidence we have.

2. It if is true, it is true because of the relationship between evidence and a proposition, not the world.

3. The evidence may be true because of the way the world is (the lack of fossil records, for example) but these facts can’t explain why (II) is true. That depends on inductive logic.

4. It makes no sense to talk about repeated trials to test (II).

5. In short: (II) is about the relation between evidence and a proposition, not the way the world is.

This is the belief sense of “probability”.

5

Two ways to understand the belief sense of “probability”:

I. Interpersonal/Evidential: Any reasonable person who considered

the evidence would agree with the statement.

Probability is objective: the evidence is evidence for everybody.

II. Personal degree of belief: I am personally quite confident that

unicorns never existed. If I had to bet, I’d take 4 to 1 odds that

unicorns never existed.

We call both ideas:

Belief-type probability

6

In sum

The frequency sense of “probability” talks about the number of times a certain kind of event will happen in repeated trials.

The belief sense of “probability” talks about the extent to which evidence supports a proposition.

7

Take a moment to look back at some of the examples in the book. Which ones are best understood as frequency-type examples? Which as belief-type?

Notice that the calculations remain the same either way.

The problem is one of philosophical understanding of the concept “probability”.

8

Important distinction:

Don’t confuse the following:

What a person says (the proposition he/she expresses).

The reasons a person has for believing a proposition.

Of course, you should have reasons for any claim you make.

However, what you say is different from your reasons for saying it.

So it is false that every statement you make is a statement about your own beliefs or reasons.

9

Example: I say: “Unicorns never existed.”

I say this because:1. I read about the fossil records.2. I studied Greek mythology.3. Etc.

These are my reasons.

But the statement I make does not refer to my beliefs or reasons. It is simply the proposition that unicorns never existed.

This is a claim about the world, not me.

Belief-type theories say probability is about the relation between reasons and propositions.

Frequency-type theories say it is about the world.

10

Evidence

It makes no sense to talk about the frequency of a single event. Hence, such claims must be belief-type.

However, we often use frequencies to form belief-type probability statements about single cases.

Example: you’ve tossed a certain coin hundreds of times.

80% of the tosses come up heads.

(Q): What is the probability that this (single) toss will come up heads?

Answer: 80%.

We use the frequency as evidence to support the belief that there is an 80% probability that this toss will come up heads.

11

Frequency Principle

If all you know is the frequency-type probability of a certain kind of event, then the belief-type probability that the event occurs in a single case should equal the frequency-type probability.

[In most cases we are not completely ignorant and have to make a judgment.]

12

Imagine you’ve tossed the biased coin 500 times and seen 399 heads.

You conclude that heads comes up about 80% of the time.

But you’re not 100% confident. You think more trials are needed to be

absolutely sure.

So: you say you are 99% sure that the coin comes up heads about 80% of the time.

Does this make sense? Yes:

Your statement expresses your belief-type probability in a frequency-type probability.

I.e., it expresses your confidence that you’ve identified the right frequency.

13

THEORIES OF PROBABILITY

For most calculations, belief-type and probability-type theorists agree.

We shall take both ideas seriously (chapters 13-15 explore belief-type, 16-19 frequency-type).

But let us take a closer look at some of the theories.

14

Belief-Type Probability

1. Logical Probability

Endorsed by: J. M. Keynes and R. Carnap.

Probability statements express a logical relation between evidence and a proposition.

On this view, probability is always relative to evidence (whether or not this is explicit). There is no categorical probability, only conditional probability, i.e. Pr(H/E).

In other words, no proposition has a probability on its own. It gains probability in relation to evidence.

Let’s take a moment to see how this might be set up.

15

Logic and evidence

Suppose there are three objects, a, b and c. We consider whether they have some property, F.

There are eight logically possible combinations. We’ll call these ‘states’:

1. Fa & Fb & Fc 2. ~Fa & Fb & Fc 3. Fa & ~Fb & Fc 4. Fa & Fb & ~Fc 5. ~Fa & ~Fb & Fc 6. ~Fa & Fb & ~Fc 7. Fa & ~Fb & ~Fc 8. ~Fa & ~Fb & ~Fc

So far so good, but note that there are various ways we can organize this list.

16

Structure descriptions

We can focus on broad structure as follows:

Each object is F: (1) Two objects are F, one is ~F: (2, 3, 4) One object is F, two are ~F: (5, 6, 7) Nothing is F: (8)

Carnap suggests we treat the situation as follows:

1. Give each structure an equal weight (1/4)

2. Partition the states equally within each structure (i.e. 1/4x1/3)

This leads to the following:

17

Carnap’s partition

State Structure Weight p*1. Fa.Fb.Fc I. Each is F 1/4 ¼

2. ~Fa.Fb.Fc 1/12

3. Fa.~Fb.Fc II. Two Fs, one ~F 1/4 1/12

4. Fa.Fb.~Fc 1/12

5. ~Fa.~Fb.Fc 1/12

6. ~Fa.Fb.~Fc III. One F, two ~Fs 1/4 1/12

7. Fa.~Fb.~Fc 1/12

8. ~Fa.~Fb.~Fc

IV. Nothing is F 1/4 ¼

18

Logic and learning from experience

Let P = Fc (i.e. c has property F)Let Q = Fa

Now we can ask, what is Pr(P/Q)?

Pr(P/Q) = Pr(P & Q)/Pr(Q)

Pr(Q) = ¼ + 1/12 + 1/12 + 1/12 = ½ Pr(P & Q) = ¼ + 1/12 = 4/12 = 1/3

So, Pr(P/Q) = (1/3)/½ = 2/3

Since Pr(P) = ½ we have added confirmation to our belief that c is F, so this models learning from experience.

Note: Even Pr(Q) is really conditional (i.e.

given this particular arrangement of individuals and properties).

This is an interpersonal, evidential theory of probability.

19

Some concerns

Why focus on structure? Why not give each state an equal probability of 1/8? Call this partition p.

We can certainly do so. In that case:

Pr(P & Q) = 1/8 + 1/8 = ¼ Pr(Q) = ½

So, Pr(P/Q) = (1/4)/½ = ½.

In other words, the probability doesn’t change given the new evidence, so there is no learning here.

But isn’t this how it should be? There are four states that will give rise to Fa, and of these only two also have Fc.

So how do we assign probabilities?

20

Different distributions

The difference between p and p* corresponds to the difference between Maxwell-Boltzmann statistics and Bose-Einstein statistics used in physics.

Of interest:

No particles obey M-B statistics but photons obey B-E statistics.

We can partition states up yet differently to get Fermi-Dirac statistics, which electrons obey

So, is any particular partition the right one? How can the logical theorist decide?

21

Principle of Insufficient Reason:

Here is an interesting question: what if there is no relevant evidence? In that case, how do we understand the logical theory?

Keynes proposes the following principle:

If there is no reason (evidence) to favour one alternative over any other, they should each be treated as equally probable.

If there are n outcomes and no evidence in favour of either one, the probability of each should be 1/n.

This is also called the Principle of Indifference.

22

This principle makes sense in many cases.

E.g.: you don’t know which side of a coin has come up (it’s underneath someone’s hand). You assign a probability of ½ to each outcome, heads/tails.

However, the principle can lead to problems.

For example: there are two alternatives: either the car behind the garage door is red or it is not red. Would you assign a probability of ½ to each?

No! Since there are so many colours for cars, the chance of one being red is < ½.

23

Bertrand’s Paradox

Here is a more difficult problem associated with the principle of indifference.

Imagine a circle (of radius R) with an inscribed equilateral triangle (XYZ).

Mark the mid-point of the circle ‘O’.

Drop a line from vertex Y, perpendicular to side XZ, until it hits the far side of the circle. It crosses XZ at point W.

It follows that XW = ZW and OWZ = 90

This is the result:

24

The picture

Randomly select a chord of the circle (Reminder: chord = an interior line from

one point on the circumference to another).

What is the probability that the randomly selected chord is longer than the side of the inscribed triangle? I.e. what is Pr(CLSE)

25

The Challenge

Even if we accept the principle of indifference, there seem to be three equally valid ways of answering this question.

26

First solution

AB is the random chord. Let OW hit the circle at C (OC = R):

The chord AB is longer than the side if OW<R/2, shorter if OW>R/2.

27

50-50

Since the chord is selected randomly, there is 50% that it crosses OC on one side of its mid-point and 50% chance that it crosses OC on the other side of its midpoint.

So, we should remain indifferent between the two.

Answer: Pr(CLSE) = ½ = 0.5

28

Second solution

Draw a tangent to the circle at one of the vertices of the inscribed triangle and let be the angle between AB and the tangent:

29

Indifference again

There are three 60 regions in which might be found.

Since we selected the chord at random, there is an equal probability it will be in each.

Therefore: Pr(CLSE) = 1/3

30

Third solution

Inscribe a circle with radius one half that of the main circle:

AB will be longer than the side of the triangle if and only if its centre (W) lies within the inner circle.

31

Indifference yet again

AB is selected at random so we have no reason to suppose it is more likely to occur at one point in the main circle than any other.

So, the probability that it is within the smaller circle is equal to the area of that circle compared to that of the larger circle.

So, Pr(CLSE) = (R2/4)/R2 = ¼.

32

Upshot

The principle of indifference, central to the logical theory, gives three different yet equally persuasive answers to the original question.

But if the principle of indifference is flawed, so is the logical theory.

33

Possible responses

Keynes: none of the alternative possibilities should be ‘divisible’ into anything that has the same form as another possibility.

Consider the car example. We used the alternatives red and not-red.

But not red = blue, green, yellow… These are of the same form as red. So not-red is divisible and should be

ruled out.

Instead, every possibility should be a colour, and now Pr(red) < ½, which seems right.

34

Problem

This solution can’t handle continuum cases.

Example: what is the probability that a randomly selected point is found exactly halfway between 0 and 1 on a number line?

Problem: there are infinitely many indivisible alternatives (it is at 0.1, 0.01, 0.001…).

So, the principle tells us that Pr(0.5) = 1/. What does that mean?

Also : what exactly does indivisible mean?

35

An objection ‘from authority’

Ramsey: there’s no such thing as this logical relation!

36

Upshot

Other solutions to the paradoxes of indifference have been suggested.

E.g.: the principle is a guideline or heuristic.

However, by and large the logical theory has been rejected (a heuristic isn’t a logical principle).

Options:

Those who favour belief-type probability tend to argue for the personal theory (Gillies calls this the subjective theory).

Others prefer frequency approaches.

37

2. Personal Probability

Endorsed by: F. P. Ramsey, B. de Finetti, L. J. Savage. It is the view that:

Probability claims state your personal confidence or degree of belief in a proposition (what you would bet on).

Objection: “subjective”, useless approach.

Response: If you want your beliefs to be consistent then they must obey the basic rules of probability.

Hence, even personal probability should be guided by the calculations, including Bayes’ Theorem.

Objection: what if one doesn’t care about consistency?

Response: This amounts to irrationality (?).

38

Overview of frequency-type theories

3. Limiting Frequency

The view of J. Venn, R. von Mises:

Probability statements give the relative frequency of an outcome “in the long run”.

I.e. Pr(E) is the limit to which the frequency of E-occurrences converges to at infinity.

We reject gambling systems: we consider probability not determined outcomes.

Random sequence = a program that can generate the sequence is as long as the sequence itself.

Objections: Ignores underlying causes of frequency. In reality, no such idealization exists: “in

the long run we’re all dead” (Keynes).

39

4. Propensity

Endorsed/invented by K. Popper:

Emphasizes underlying cause of observed frequency.

Underlying structure of a system exists even if no trials are ever made.

The system still has a tendency or disposition to give a certain frequency in the long run.

This property is called the system’s propensity.

This interests us when we investigate probabilistic systems in nature (Q. M.).

Objection: we must often make probability judgments without knowing the underlying structure (e.g. economics, psychology).

40

Expected Value

Coke machine: Only works 95% of the time. So, Coke charges $0.95 instead of $1.

Frequency-dogmatists: That’s a fair price Most of the time you save a nickel, only

occasionally do you lose 95 cents. So, the expected value is zero.

Belief-dogmatists: Frequency-types get it wrong. Who cares about the long-run frequency

of getting a coke? We care about getting a coke now. Since its 95% probable that I get a coke

now, a price of $0.95 is fair: Expected value = zero. It would be unfair to be forced to use this

machine even if it is “fair” in the long run.

41

Why so many theories?

At first, people thought a lot about dice, lotteries, urns, etc.

In these simple models, the differences don’t matter: all theories give the same results.

Only as models get more sophisticated do differences start to appear.

Therefore, as more and more people started to think about probability, two families of ideas emerged from the simple models: frequency-type and belief-type.

To this day, experts continue to fight for their preferred interpretations.

42

Homework

Do the exercises at the end of chapters 11 and 12.

43