lecture 1: probability models and axioms readings: lecture ... · lecture 1: probability models and...
TRANSCRIPT
LECTURE 1: Probability models and axioms
• Readings: Sections 1.1, 1.2
Lecture outline
• Sample space
• Probability laws
– Axioms
– Some properties
• Examples
– Discrete
– Continuous
• Discussion
– Countable additivity
– Mathematical subtleties
• Interpretations of probabilities
• Sample space
• Probability laws
– Axioms
– Some properties
• Examples
– Discrete
– Continuous
• Discussion
– Countable additivity
– Mathematical subtleties
• Interpretations of probabilities
• Sample space
• Probability laws
– Axioms
– Some properties
• Examples
– Discrete
– Continuous
• Discussion
– Countable additivity
– Mathematical subtleties
• Interpretations of probabilities
• Sample space
• Probability laws
– Axioms
– Some properties
• Examples
– Discrete
– Continuous
• Discussion
– Countable additivity
– Mathematical subtleties
• Interpretations of probabilities
• Sample space
• Probability laws
– Axioms
– Properties that follow from the axioms
• Examples
– Discrete
– Continuous
• Discussion
– Countable additivity
– Mathematical subtleties
• Interpretations of probabilities
Sample space
• List (set) of possible outcomes
• List must be:
– Mutually exclusive
– Collectively exhaustive
– At the “right” granularity
Sample space
• List (set) of possible outcomes, Ω
• List must be:
– Mutually exclusive
– Collectively exhaustive
– At the “right” granularity
• Two steps:
– Describe possible outcomes
– Describe beliefs about likelihood of outcomes
Sample space
• List (set) of possible outcomes
• List must be:
– Mutually exclusive
– Collectively exhaustive
– At the “right” granularity
Sample space
• List (set) of possible outcomes, Ω
• List must be:
– Mutually exclusive
– Collectively exhaustive
– At the “right” granularity
Sample space
• List (set) of possible outcomes, Ω
• List must be:
– Mutually exclusive
– Collectively exhaustive
– At the “right” granularity
Die roll example
X = First roll
1 2 3 4
4
3
2
Y = Second
roll
1
• Let B be the event: min(X,Y ) = 2
• Let M = max(X,Y )
• P(M = 1 | B) =
• P(M = 2 | B) =
Sample space: discrete/finite example
• Two rolls of a tetrahedral die
– Sample space vs. sequential description
X = First roll
1 2 3 4
4
3
2
Y = Second
roll
1
1
2
3
4
1,11,2
1,3
1,4
4,4
• A continuous sample space:(x, y) such that 0 ≤ x, y ≤ 1
x
1
1
y
Sample space: discrete/finite example
• Two rolls of a tetrahedral die
– Sample space vs. sequential description
X = First roll
1 2 3 4
4
3
2
Y = Second
roll
1
1
2
3
4
1,11,2
1,3
1,4
4,4
• A continuous sample space:(x, y) such that 0 ≤ x, y ≤ 1
x
1
1
y
Sample space: discrete/finite example
• Two rolls of a tetrahedral die
– Sample space vs. sequential description
X = First roll
1 2 3 4
4
3
2
Y = Second
roll
1
1
2
3
4
1,11,2
1,3
1,4
4,4
• A continuous sample space:(x, y) such that 0 ≤ x, y ≤ 1
x
1
1
y
Sample space: discrete/finite example
• Two rolls of a tetrahedral die
– Sample space vs. sequential description
X = First roll
1 2 3 4
4
3
2
Y = Second
roll
1
1
2
3
4
1,11,2
1,3
1,4
4,4
• A continuous sample space:(x, y) such that 0 ≤ x, y ≤ 1
x
1
1
y
Sample space: continuous example
• A continuous sample space:
• (x, y) such that 0 ≤ x, y ≤ 1
x
1
1
y
Sample space: continuous example
• A continuous sample space:
• (x, y) such that 0 ≤ x, y ≤ 1
x
1
1
y
Sample space: continuous example
• A continuous sample space:
• (x, y) such that 0 ≤ x, y ≤ 1
x
1
1
ySample space: continuous example
• A continuous sample space:
• (x, y) such that 0 ≤ x, y ≤ 1
x
1
1
y
Sample space: continuous example
• A continuous sample space:
• (x, y) such that 0 ≤ x, y ≤ 1
x
1
1
y
Sample space: continuous example
• A continuous sample space:
• (x, y) such that 0 ≤ x, y ≤ 1
x
1
1
y
Sample space: continuous example
• A continuous sample space:
• (x, y) such that 0 ≤ x, y ≤ 1
x
1
1
y
Probability axioms
• Event: a subset of the sample space
– Probability is assigned to events
Axioms:
1. P(A) ≥ 0
2. P(universe) = 1
3. If A ∩B = Ø,then P(A ∪B) = P(A) +P(B)
• P(s1, s2, . . . , sk) = P(s1) + · · ·+P(sk)
= P(s1) + · · ·+P(sk)
– Axiom 3 needs strengthening
– Do weird sets have probabilities?
Probability axioms
• Event: a subset of the sample space
– Probability is assigned to events
Axioms:
1. P(A) ≥ 0
2. P(universe) = 1
3. If A ∩B = Ø,then P(A ∪B) = P(A) +P(B)
• P(s1, s2, . . . , sk) = P(s1) + · · ·+P(sk)
= P(s1) + · · ·+P(sk)
– Axiom 3 needs strengthening
– Do weird sets have probabilities?
Probability axioms
• Event: a subset of the sample space
– Probability is assigned to events
• Axioms:
1. P(A) ≥ 0
2. P(universe) = 1
3. If A ∩B = Ø,then P(A ∪B) = P(A) +P(B)
• P(s1, s2, . . . , sk) = P(s1) + · · ·+P(sk)
= P(s1) + · · ·+P(sk)
– Axiom 3 needs strengthening
– Do weird sets have probabilities?
Probability axioms
• Event: a subset of the sample space
– Probability is assigned to events
• Axioms:
– Nonnegativity: P(A) ≥ 0
– Normalization: P(Ω) = 1
– (Finite) additivity:If A ∩B = Ø, then P(A ∪B) = P(A) +P(B)
• P(s1, s2, . . . , sk) = P(s1) + · · ·+P(sk)
= P(s1) + · · ·+P(sk)
– Axiom 3 needs strengthening
– Do weird sets have probabilities?
Probability axioms
• Event: a subset of the sample space
– Probability is assigned to events
• Axioms:
– Nonnegativity: P(A) ≥ 0
– Normalization: P(Ω) = 1
– (Finite) additivity:If A ∩B = Ø, then P(A ∪B) = P(A) +P(B)
• P(s1, s2, . . . , sk) = P(s1) + · · ·+P(sk)
= P(s1) + · · ·+P(sk)
– Axiom 3 needs strengthening
– Do weird sets have probabilities?
Probability axioms
• Event: a subset of the sample space
– Probability is assigned to events
• Axioms:
– Nonnegativity: P(A) ≥ 0
– Normalization: P(Ω) = 1
– (Finite) additivity: (to be strengthened later)If A ∩B = Ø, then P(A ∪B) = P(A) +P(B)
• P(s1, s2, . . . , sk) = P(s1) + · · ·+P(sk)
= P(s1) + · · ·+P(sk)
– Axiom 3 needs strengthening
– Do weird sets have probabilities?
Some simple consequences of the axioms
• P(s1, s2, . . . , sk) = P(s1) + · · ·+P(sk)
= P(s1) + · · ·+P(sk)
– Axiom 3 needs strengthening
– Do weird sets have probabilities?
• Axioms
• Consequences
For disjoint sets:
P(A ∪B) = P(A) +P(B)
Some simple consequences of the axioms
• P(A) ≤ 1
• P(Ø) = 0
• A,B,C disjoint: P(A ∪B ∪ C) = P(A) +P(B) +P(C)and similarly for k disjoint events
• P(s1, s2, . . . , sk) = P(s1) + · · ·+P(sk)
= P(s1) + · · ·+P(sk)
– Axiom 3 needs strengthening
– Do weird sets have probabilities?
• Axioms
• Consequences
For disjoint sets:
P(A ∪B) = P(A) +P(B)
Some simple consequences of the axioms
• P(A) ≤ 1
• P(Ø) = 0
• A,B,C disjoint: P(A ∪B ∪ C) = P(A) +P(B) +P(C)and similarly for k disjoint events
• P(s1, s2, . . . , sk) = P(s1) + · · ·+P(sk)
= P(s1) + · · ·+P(sk)
– Axiom 3 needs strengthening
– Do weird sets have probabilities?
• Axioms
• Consequences
For disjoint sets:
P(A ∪B) = P(A) +P(B)
Some simple consequences of the axioms
• P(A) ≤ 1
• P(Ø) = 0
• A,B,C disjoint: P(A ∪B ∪ C) = P(A) +P(B) +P(C)and similarly for k disjoint events
• P(s1, s2, . . . , sk) = P(s1) + · · ·+P(sk)
= P(s1) + · · ·+P(sk)
– Axiom 3 needs strengthening
– Do weird sets have probabilities?
• Axioms
• Consequences
For disjoint sets:
P(A ∪B) = P(A) +P(B)
Some simple consequences of the axioms
• P(A) ≤ 1
• P(Ø) = 0
• A,B,C disjoint: P(A ∪B ∪ C) = P(A) +P(B) +P(C)and similarly for k disjoint events
• P(s1, s2, . . . , sk) = P(s1) + · · ·+P(sk)
= P(s1) + · · ·+P(sk)
– Axiom 3 needs strengthening
– Do weird sets have probabilities?
• Axioms
• Consequences
For disjoint sets:
P(A ∪B) = P(A) +P(B)
Some simple consequences of the axioms
• P(A) ≤ 1
• P(Ø) = 0
• A,B,C disjoint: P(A ∪B ∪ C) = P(A) +P(B) +P(C)and similarly for k disjoint events
• P(s1, s2, . . . , sk) = P(s1) + · · ·+P(sk)
= P(s1) + · · ·+P(sk)
– Axiom 3 needs strengthening
– Do weird sets have probabilities?
• Axioms
• Consequences
For disjoint sets:
P(A ∪B) = P(A) +P(B)
Some simple consequences of the axioms
• P(A) ≤ 1
• P(Ø) = 0
• A,B,C disjoint: P(A ∪B ∪ C) = P(A) +P(B) +P(C)and similarly for k disjoint events
• P(s1, s2, . . . , sk) = P(s1) + · · ·+P(sk)
= P(s1) + · · ·+P(sk)
– Axiom 3 needs strengthening
– Do weird sets have probabilities?
• Axioms
• Consequences
For disjoint sets:
P(A ∪B) = P(A) +P(B)
Some simple consequences of the axioms
• P(A) ≤ 1
• P(Ø) = 0
• P(A) +P(Ac) = 1
• A,B,C disjoint: P(A ∪B ∪ C) = P(A) +P(B) +P(C)and similarly for k disjoint events
• P(s1, s2, . . . , sk) = P(s1) + · · ·+P(sk)
= P(s1) + · · ·+P(sk)
– Axiom 3 needs strengthening
– Do weird sets have probabilities?
Probability axioms
• Event: a subset of the sample space
– Probability is assigned to events
• Axioms:
– Nonnegativity: P(A) ≥ 0
– Normalization: P(Ω) = 1
– (Finite) additivity: (to be strengthened later)If A ∩B = Ø, then P(A ∪B) = P(A) +P(B)
Probability axioms
• Event: a subset of the sample space
– Probability is assigned to events
• Axioms:
– Nonnegativity: P(A) ≥ 0
– Normalization: P(Ω) = 1
– (Finite) additivity: (to be strengthened later)If A ∩B = Ø, then P(A ∪B) = P(A) +P(B)
• Axioms
• Consequences
For disjoint sets:
P(A ∪B) = P(A) +P(B)
Some simple consequences of the axioms
• P(A) ≤ 1
• P(Ø) = 0
• A,B,C disjoint: P(A ∪B ∪ C) = P(A) +P(B) +P(C)and similarly for k disjoint events
• P(s1, s2, . . . , sk) = P(s1) + · · ·+P(sk)
= P(s1) + · · ·+P(sk)
– Axiom 3 needs strengthening
– Do weird sets have probabilities?
• Axioms
• Consequences
For disjoint events:
P(A ∪B) = P(A) +P(B)
Some simple consequences of the axioms
• P(A) ≤ 1
• P(Ø) = 0
• P(A) +P(Ac) = 1
• A,B,C disjoint: P(A ∪B ∪ C) = P(A) +P(B) +P(C)and similarly for k disjoint events
• P(s1, s2, . . . , sk) = P(s1) + · · ·+P(sk)
= P(s1) + · · ·+P(sk)
– Axiom 3 needs strengthening
– Do weird sets have probabilities?
Probability axioms
• Event: a subset of the sample space
– Probability is assigned to events
• Axioms:
– Nonnegativity: P(A) ≥ 0
– Normalization: P(Ω) = 1
– (Finite) additivity: (to be strengthened later)If A ∩B = Ø, then P(A ∪B) = P(A) +P(B)
Some simple consequences of the axioms
• P(s1, s2, . . . , sk) = P(s1) + · · ·+P(sk)
= P(s1) + · · ·+P(sk)
– Axiom 3 needs strengthening
– Do weird sets have probabilities?
Probability axioms
• Event: a subset of the sample space
– Probability is assigned to events
• Axioms:
– Nonnegativity: P(A) ≥ 0
– Normalization: P(Ω) = 1
– (Finite) additivity: (to be strengthened later)If A ∩B = Ø, then P(A ∪B) = P(A) +P(B)
Probability axioms
• Event: a subset of the sample space
– Probability is assigned to events
• Axioms:
– Nonnegativity: P(A) ≥ 0
– Normalization: P(Ω) = 1
– (Finite) additivity: (to be strengthened later)If A ∩B = Ø, then P(A ∪B) = P(A) +P(B)
• Axioms
• Consequences
For disjoint sets:
P(A ∪B) = P(A) +P(B)
Some simple consequences of the axioms
• P(A) ≤ 1
• P(Ø) = 0
• A,B,C disjoint: P(A ∪B ∪ C) = P(A) +P(B) +P(C)and similarly for k disjoint events
• P(s1, s2, . . . , sk) = P(s1) + · · ·+P(sk)
= P(s1) + · · ·+P(sk)
– Axiom 3 needs strengthening
– Do weird sets have probabilities?
• Axioms
• Consequences
For disjoint events:
P(A ∪B) = P(A) +P(B)
Some simple consequences of the axioms
• P(A) ≤ 1
• P(Ø) = 0
• P(A) +P(Ac) = 1
• A,B,C disjoint: P(A ∪B ∪ C) = P(A) +P(B) +P(C)and similarly for k disjoint events
• P(s1, s2, . . . , sk) = P(s1) + · · ·+P(sk)
= P(s1) + · · ·+P(sk)
– Axiom 3 needs strengthening
– Do weird sets have probabilities?
Probability axioms
• Event: a subset of the sample space
– Probability is assigned to events
• Axioms:
– Nonnegativity: P(A) ≥ 0
– Normalization: P(Ω) = 1
– (Finite) additivity: (to be strengthened later)If A ∩B = Ø, then P(A ∪B) = P(A) +P(B)
Some simple consequences of the axioms
• P(s1, s2, . . . , sk) = P(s1) + · · ·+P(sk)
= P(s1) + · · ·+P(sk)
– Axiom 3 needs strengthening
– Do weird sets have probabilities?
• Axioms
• Consequences
For disjoint sets:
P(A ∪B) = P(A) +P(B)
Some simple consequences of the axioms
• P(A) ≤ 1
• P(Ø) = 0
• P(A) +P(Ac) = 1
• A,B,C disjoint: P(A ∪B ∪ C) = P(A) +P(B) +P(C)and similarly for k disjoint events
• P(s1, s2, . . . , sk) = P(s1) + · · ·+P(sk)
= P(s1) + · · ·+P(sk)
– Axiom 3 needs strengthening
– Do weird sets have probabilities?
• Axioms
• Consequences
For disjoint sets:
P(A ∪B) = P(A) +P(B)
Some simple consequences of the axioms
• P(A) ≤ 1
• P(Ø) = 0
• P(A) +P(Ac) = 1
• A,B,C disjoint: P(A ∪B ∪ C) = P(A) +P(B) +P(C)and similarly for k disjoint events
• P(s1, s2, . . . , sk) = P(s1) + · · ·+P(sk)
= P(s1) + · · ·+P(sk)
– Axiom 3 needs strengthening
– Do weird sets have probabilities?
More consequences of the axioms
• If A ⊂ B, then P(A) ≤ P(B)
• P(A ∪B) = P(A) +P(B)−P(A ∩B)
• P(A ∪B) ≤ P(A) +P(B)
• P(A ∪B ∪ C) = P(A) +P(Ac ∩B) +P(Ac ∩Bc ∩ C)
More consequences of the axioms
• If A ⊂ B, then P(A) ≤ P(B)
• P(A ∪B) = P(A) +P(B)−P(A ∩B)
• P(A ∪B) ≤ P(A) +P(B)
• P(A ∪B ∪ C) = P(A) +P(Ac ∩B) +P(Ac ∩Bc ∩ C)More consequences of the axioms
• If A ⊂ B, then P(A) ≤ P(B)
• P(A ∪B) = P(A) +P(B)−P(A ∩B)
• P(A ∪B) ≤ P(A) +P(B)
• P(A ∪B ∪ C) = P(A) +P(Ac ∩B) +P(Ac ∩Bc ∩ C)More consequences of the axioms
• If A ⊂ B, then P(A) ≤ P(B)
• P(A ∪B) = P(A) +P(B)−P(A ∩B)
• P(A ∪B) ≤ P(A) +P(B)
• P(A ∪B ∪ C) = P(A) +P(Ac ∩B) +P(Ac ∩Bc ∩ C)
More consequences of the axioms
• If A ⊂ B, then P(A) ≤ P(B)
• P(A ∪B) = P(A) +P(B)−P(A ∩B)
• P(A ∪B) ≤ P(A) +P(B)
• P(A ∪B ∪ C) = P(A) +P(Ac ∩B) +P(Ac ∩Bc ∩ C)
More consequences of the axioms
• If A ⊂ B, then P(A) ≤ P(B)
• P(A ∪B) = P(A) +P(B)−P(A ∩B)
• P(A ∪B) ≤ P(A) +P(B)
• P(A ∪B ∪ C) = P(A) +P(Ac ∩B) +P(Ac ∩Bc ∩ C)
Die roll example
X = First roll
1 2 3 4
4
3
2
Y = Second
roll
1
• Let B be the event: min(X,Y ) = 2
• Let M = max(X,Y )
• P(M = 1 | B) =
• P(M = 2 | B) =
Sample space: discrete/finite example
• Two rolls of a tetrahedral die
– Sample space vs. sequential description
X = First roll
1 2 3 4
4
3
2
Y = Second
roll
1
1
2
3
4
1,11,2
1,3
1,4
4,4
• A continuous sample space:(x, y) such that 0 ≤ x, y ≤ 1
x
1
1
y
Probability calculation: discrete/finite example
Example
X = First roll
1 2 3 4
4
3
2
Y = Second
roll
1
• Let every possible outcome haveprobability 1/16
• P(X = 1) =
Probability calculation: discrete/finite example
Example
X = First roll
1 2 3 4
4
3
2
Y = Second
roll
1
• Let every possible outcome have probability 1/16
• P(X = 1) =
Probability calculation: discrete/finite example
Example
X = First roll
1 2 3 4
4
3
2
Y = Second
roll
1
• Let every possible outcome have probability 1/16
• P(X = 1) =
Let Z = min(X,Y )
• P(Z = 1) =
• P(Z = 2) =
• P(Z = 3) =
• P(Z = 4) =
Let Z = min(X,Y )
• P(Z = 1) =
• P(Z = 2) =
• P(Z = 3) =
• P(Z = 4) =Let Z = min(X,Y )
• P(Z = 1) =
• P(Z = 2) =
• P(Z = 3) =
• P(Z = 4) =
Discrete uniform law
• Let all sample points be equally likely
• Then,
P(A) =number of elements of A
total number of sample points
• Just count. . .
Continuous uniform law
• Two “random” numbers in [0,1].
x
1
1
y
• Uniform law: Probability = Area
Discrete uniform law
− Assume Ω consists of n equally likely elements− Assume A consists of m elements
Then : P(A) =number of elements of A
number of elements of Ω=
m
n
• Just count. . .
Discrete uniform law
− Assume Ω consists of n equally likely elements− Assume A consists of m elements
Then : P(A) =number of elements of A
number of elements of Ω=
m
n
• Just count. . .
Discrete uniform law
− Assume Ω consists of n equally likely elements− Assume A consists of m elements
Then : P(A) =number of elements of A
number of elements of Ω=
m
n
• Just count. . .
prob =1
n
Discrete uniform law
− Assume Ω consists of n equally likely elements− Assume A consists of k elements
Then : P(A) =number of elements of A
number of elements of Ω=
k
n
• Just count. . .
prob =1
n
Discrete uniform law
• Let all sample points be equally likely
• Then,
P(A) =number of elements of A
total number of sample points
• Just count. . .
Continuous uniform law
• Two “random” numbers in [0,1].
x
1
1
y
• Uniform law: Probability = Area
Probability calculation: continuous example
• Two “random” numbers in [0,1].
x
1
1
y
• Uniform law: Probability = Area
Sample space: continuous example
• A continuous sample space:
• (x, y) such that 0 ≤ x, y ≤ 1
x
1
1
y
Sample space: continuous example
• A continuous sample space:
• (x, y) such that 0 ≤ x, y ≤ 1
x
1
1
ySample space: continuous example
• A continuous sample space:
• (x, y) such that 0 ≤ x, y ≤ 1
x
1
1
y
Sample space: continuous example
• A continuous sample space:
• (x, y) such that 0 ≤ x, y ≤ 1
x
1
1
y
Sample space: continuous example
• A continuous sample space:
• (x, y) such that 0 ≤ x, y ≤ 1
x
1
1
y
Sample space: continuous example
• A continuous sample space:
• (x, y) such that 0 ≤ x, y ≤ 1
x
1
1
y
Probability calculation: continuous example
• Two “random” numbers in [0,1].
x
1
1
y
• Uniform probabilty law: Probability = Area
P(x, y) | x+ y ≤ 1/2
=
P0.5,0.3)
=
Probability calculation: continuous example
• Two “random” numbers in [0,1].
x
1
1
y
• Uniform probability law: Probability = Area
P(x, y) | x+ y ≤ 1/2
=
P0.5,0.3)
=
Probability calculation: continuous example
• Two “random” numbers in [0,1].
x
1
1
y
• Uniform probability law: Probability = Area
P(x, y) | x+ y ≤ 1/2
=
P(0.5,0.3)
=
Probability calculation steps
• Specify the sample space
• Specify a probability law
• Identify an event of interest
• Calculate...
Probability calculation: discrete but infinite sample space
• Sample space: 1,2, . . .
– We are given P(n) =1
2n, n = 1,2, . . .
• P(outcome is even) =
1/2
…..
p
1/4
1/81/16
1 2 3 4
Probability calculation steps
• Specify the sample space
• Specify a probability law
• Identify an event of interest
• Calculate...
Probability calculation: discrete but infinite sample space
• Sample space: 1,2, . . .
– We are given P(n) =1
2n, n = 1,2, . . .
• P(outcome is even) =
1/2
…..
p
1/4
1/81/16
1 2 3 4
Probability calculation: discrete but infinite sample space
• Sample space: 1,2, . . .
– We are given P(n) = 2−n, n = 1,2, . . .
• Find P(outcome is even)
1/2
…..
p
1/4
1/81/16
1 2 3 4
• Solution:
P(2,4,6, . . .) = P(2) +P(4) + · · ·
=1
22+
1
24+
1
26+ · · · =
1
3
• Axiom needed:If A1, A2, . . . are disjoint events, then:
P(A1 ∪A2 ∪ · · · ) = P(A1) +P(A2) + · · ·
Probability calculation: discrete but infinite sample space
• Sample space: 1,2, . . .
– We are given P(n) =1
2n, n = 1,2, . . .
• Find P(outcome is even)
1/2
…..
p
1/4
1/81/16
1 2 3 4
• Solution:
P(2,4,6, . . .) = P(2) +P(4) + · · ·
=1
22+
1
24+
1
26+ · · · =
1
3
• Axiom needed:If A1, A2, . . . are disjoint events, then:
P(A1 ∪A2 ∪ · · · ) = P(A1) +P(A2) + · · ·
Probability calculation: discrete but infinite sample space
• Sample space: 1,2, . . .
– We are given P(n) = 2−n, n = 1,2, . . .
• Find P(outcome is even)
1/2
…..
p
1/4
1/81/16
1 2 3 4
• Solution:
P(2,4,6, . . .) = P(2) +P(4) + · · ·
=1
22+
1
24+
1
26+ · · · =
1
3
• Axiom needed:If A1, A2, . . . are disjoint events, then:
P(A1 ∪A2 ∪ · · · ) = P(A1) +P(A2) + · · ·
Probability calculation: discrete but infinite sample space
• Sample space: 1,2, . . .
– We are given P(n) = 2−n, n = 1,2, . . .
• Find P(outcome is even)
1/2
…..
p
1/4
1/81/16
1 2 3 4
• Solution:
P(2,4,6, . . .) = P(2) +P(4) + · · ·
=1
22+
1
24+
1
26+ · · · =
1
3
• Axiom needed:If A1, A2, . . . are disjoint events, then:
P(A1 ∪A2 ∪ · · · ) = P(A1) +P(A2) + · · ·
Probability calculation: discrete but infinite sample space
• Sample space: 1,2, . . .
– We are given P(n) = 2−n, n = 1,2, . . .
• Find P(outcome is even)
1/2
…..
p
1/4
1/81/16
1 2 3 4
• Solution:
P(2,4,6, . . .) = P(2) +P(4) + · · ·
=1
22+
1
24+
1
26+ · · · =
1
3
• Axiom needed:If A1, A2, . . . are disjoint events, then:
P(A1 ∪A2 ∪ · · · ) = P(A1) +P(A2) + · · ·
Probability calculation: discrete but infinite sample space
• Sample space: 1,2, . . .
– We are given P(n) =1
2n, n = 1,2, . . .
• P(outcome is even) =
1/2
…..
p
1/4
1/81/16
1 2 3 4
• Solution:
P(2,4,6, . . .) = P(2) +P(4) + · · ·
=1
22+
1
24+
1
26+ · · · =
1
3
• Axiom needed:If A1, A2, . . . are disjoint events, then:
P(A1 ∪A2 ∪ · · · ) = P(A1) +P(A2) + · · ·
Countable additivity axiom
• Strengthens the finite additivity axiom
If A1, A2, A3,. . . is an infinite sequence of events,then P(A1 ∪A2 ∪A3 ∪ · · · ) = P(A1) +P(A2) +P(A3) + · · ·
Countable additivity axiom
• Strengthens the finite additivity axiom
If A1, A2, A3,. . . is an infinite sequence of events,then P(A1 ∪A2 ∪A3 ∪ · · · ) = P(A1) +P(A2) +P(A3) + · · ·
Countable additivity axiom
• Strengthens the finite additivity axiom
Countable Additivity Axiom:
If A1, A2, A3,. . . is an infinite sequence of disjoint events,then P(A1 ∪A2 ∪A3 ∪ · · · ) = P(A1) +P(A2) +P(A3) + · · ·
Mathematical subtleties
Mathematical subtleties
• Additivity holds only for “countable” sequences of events
• The unit square (simlarly, the real line, etc.) are not countable
• “Area” is a legitimate probability law on the unit square,as long as we do not try to assign probabilities/areas to “very strange”sets
Mathematical subtleties
• Additivity holds only for “countable” sequences of events
• The unit square (simlarly, the real line, etc.) is not countable
(its elements cannot be arranged in a sequence)
• “Area” is a legitimate probability law on the unit square,as long as we do not try to assign probabilities/areasto “very strange” sets
Mathematical subtleties
• Additivity holds only for “countable” sequences of events
• The unit square (simlarly, the real line, etc.) is not countable
(its elements cannot be arranged in a sequence)
• “Area” is a legitimate probability law on the unit square,as long as we do not try to assign probabilities/areasto “very strange” sets
Countable additivity axiom
• Strengthens the finite additivity axiom
Countable Additivity Axiom:
If A1, A2, A3,. . . is an infinite sequence of disjoint events,then P(A1 ∪A2 ∪A3 ∪ · · · ) = P(A1) +P(A2) +P(A3) + · · ·
Mathematical subtleties
• Additivity holds only for “countable” sequences of events
• The unit square (similarly, the real line, etc.) is not countable
(its elements cannot be arranged in a sequence)
• “Area” is a legitimate probability law on the unit square,as long as we do not try to assign probabilities/areasto “very strange” sets
Interpretations of probability theory
• A narrow view: a branch of math
– Axioms ⇒ theorems
“Thm:” “Frequency” of event A “is” P(A)
• Are probabilities frequencies?
– P(coin toss yields heads) = 1/2
– P(sole shooter of JFK) = 0.92
– P(a piece of equipment aboard the space shuttle fails) = 10−8
• Probability models are a framework fordescribing uncertainty
– Use for consistent reasoning
– Use for predictions, decisions
Interpretations of probability theory
• A narrow view: a branch of math
– Axioms ⇒ theorems
“Thm:” “Frequency” of event A “is” P(A)
• Are probabilities frequencies?
– P(coin toss yields heads) = 1/2
– P(sole shooter of JFK) = 0.92
– P(a piece of equipment aboard the space shuttle fails) = 10−8
• Probability models are a framework fordescribing uncertainty
– Use for consistent reasoning
– Use for predictions, decisions
Interpretations of probability theory
• A narrow view: a branch of math
– Axioms ⇒ theorems
“Thm:” “Frequency” of event A “is” P(A)
• Are probabilities frequencies?
– P(coin toss yields heads) = 1/2
– P(sole shooter of JFK) = 0.92
– P(a piece of equipment aboard the space shuttle fails) = 10−8
• Probability models are a framework fordescribing uncertainty
– Use for consistent reasoning
– Use for predictions, decisions
Interpretations of probability theory
• A narrow view: a branch of math
– Axioms ⇒ theorems
“Thm:” “Frequency” of event A “is” P(A)
• Are probabilities frequencies?
– P(coin toss yields heads) = 1/2
– P(the president of . . . will be reelected) = 0.7
• Probabilities are often intepreted as:
– Betting preferences
– Description of beliefs
• A framework for dealing with experiments that have uncertain outcomes
– Rules for consistent reasoning
– Used for predictions and decisions
– P(sole shooter of JFK) = 0.92
– P(a piece of equipment aboard the space shuttle fails) = 10−8
• Probability models are a framework fordescribing uncertainty
– Use for consistent reasoning
– Use for predictions, decisions
Interpretations of probability theory
• A narrow view: a branch of math
– Axioms ⇒ theorems
“Thm:” “Frequency” of event A “is” P(A)
• Are probabilities frequencies?
– P(coin toss yields heads) = 1/2
– P(the president of . . . will be reelected) = 0.7
• Probabilities are often intepreted as:
– Betting preferences
– Description of beliefs
• A framework for dealing with experiments that have uncertain outcomes
– Rules for consistent reasoning
– Used for predictions and decisions
– P(sole shooter of JFK) = 0.92
– P(a piece of equipment aboard the space shuttle fails) = 10−8
• Probability models are a framework fordescribing uncertainty
– Use for consistent reasoning
– Use for predictions, decisions
Interpretations of probability theory
• A narrow view: a branch of math
– Axioms ⇒ theorems
“Thm:” “Frequency” of event A “is” P(A)
• Are probabilities frequencies?
– P(coin toss yields heads) = 1/2
– P(the president of . . . will be reelected) = 0.7
• Probabilities are often intepreted as:
– Description of beliefs
– Betting preferences
• A framework for analyzing phenomena with uncertain outcomes
– Rules for consistent reasoning
– Used for predictions and decisions
– P(sole shooter of JFK) = 0.92
– P(a piece of equipment aboard the space shuttle fails) = 10−8
• Probability models are a framework fordescribing uncertainty
Interpretations of probability theory
• A narrow view: a branch of math
– Axioms ⇒ theorems
“Thm:” “Frequency” of event A “is” P(A)
• Are probabilities frequencies?
– P(coin toss yields heads) = 1/2
– P(the president of . . . will be reelected) = 0.7
• Probabilities are often intepreted as:
– Betting preferences
– Description of beliefs
• A framework for analyzing phenomena with uncertain outcomes
– Rules for consistent reasoning
– Used for predictions and decisions
– P(sole shooter of JFK) = 0.92
– P(a piece of equipment aboard the space shuttle fails) = 10−8
• Probability models are a framework fordescribing uncertainty
– Use for consistent reasoning
– Use for predictions, decisions
The role of probability theory
Real world
Data
Inference
Analysis
Predictions
Probability theory
Model building
The role of probability theory
Real world
Data
Inference
Analysis
Predictions
Probability theory
Model building
The role of probability theory
Real world
Data
Inference/Statistics
Analysis
Predictions
Probability theory
Model building
The role of probability theory
Real world
Data
Inference/Statistics
Analysis
Predictions
Probability theory
Model building
The role of probability theory
Real world
Data
Inference/Statistics
Analysis
Predictions
Probability theory
Models building
The role of probability theory
Real world
Data
Inference/Statistics
Analysis
Predictions
Probability theory
Model building
The role of probability theory
Real world
Data
Inference/Statistics
(Analysis)
Predictions
Probability theory
Models building
The role of probability theory
Real world
Data
Inference/Statistics
(Analysis)
Predictions
Probability theory
Models building
The role of probability theory
Real world
Data
Inference/Statistics
(Analysis)
Predictions Decisions
Probability theory
Models building