lecture 2

GSN3113: Probability and Stochastic Process – UCP Fall 2015 Dr. Asim BaigGSN3113: Probability and Stochastic Process – UCP Fall 2015 Dr. Asim Baig

GSN3113: Probability and Stochastic Process

Lecture 2SET Theory & Classical

Probability

GSN3113: Probability and Stochastic Process – UCP Fall 2015 Dr. Asim Baig

REVIEW OF PREVIOUS LECTURE

Let look back at the Previous Lecture…


Randomness…• Randomness or Random Phenomena arise

because of– Our partial ignorance of the generating mechanism– The laws governing the phenomena may be

fundamentally random– Our unwillingness to carry our exact analysis

because its not worth the trouble.

• What other factor/reasons may cause randomness in a process?


Back to Probability Theory• Probabilities are useful in situations that involve

randomness.• A probability is a number used to describe how likely

something/some event to occur.• Probability is the study of probabilities• It is “the art of being certain how uncertain you are”

• If an event is certain to happen, it is given a probability of 1.

• If it is certain not to happen, it has probability of 0.• Probabilities can be expressed as factions, as decimal

numbers or as percentages. • If you toss a coin, the probability of getting heads is ½ , which is

same as 0.5, which is same as 50%


Long-run Frequency Interpretation…

• If the probability of an event A in some actual physical experiment is p, then we believe that if the experiment is repeated independently over and over again then the fraction of times the event occurs will be close to P(A).

• This is called the law of large numbers (LLN)

• Let A be one of the events of a random experiment. If we conduct a sequence of n independent trials of this experiment, and if the event A occurs N(A,n) out these n trials then

is called the relative frequency of event A in these n trials


Long run frequency Interpretation…

• Long run frequency interpretation can be stated in terms of relative frequency as

• Which means that given enough trials, the relative frequency will eventually become close to P(A) i.e. you can calculate probability of event A occurring by performing large number of trials.


Coin Tossing: Relative Frequency

• If a fair coin is flipped a large number of times, the proportion of heads will tend to get closer to ½ as the number of tosses increases.


REVIEW OF SET THEORY…Collecting Stuff….


Set Theory… Introduction

• The mathematical basis of probability is the Set Theory.

• We will start with a review of Set Theory so as to refresh the previous knowledge.

• There is a high “Probability” you do not remember most of the set theory.


Introduction to Sets…

• Set is a collection of things. We use capital letters to denote sets.

• The things that together make up the set are elements. We use small letter to denote set elements.

• For Example: • We can have a set A with elements x, y and z. • The symbol denotes set inclusion• Thus means “x is an element of set A”.• Similarly is the opposite of • Thus means “c is not an element of A”


Generation of Sets….

• There are many ways to define a set.• First is to simply list all the elements

• In Engineering we frequently use mathematical rules to define sets….

Or

• Sets can have infinite number of elements

• Note that A = B if and only if


Basic Set Operations…

• Complementation: • Union:

• Also

• Intersection: • Also • can also be written simply as AB.

• Set Difference: • B\A is a set of that do not belong to A


Venn Diagrams

• Venn diagrams are very useful in set theory. They are often used to portray relationships between sets.

• Many identities can be read out simply by examining Venn diagrams

• For Example: • Let • Also let • And let • The Venn Diagram will be….

Ω 4

A B

1 2

3 5 6


Venn Diagram…Ω

A \ B B \ A

A B


An Example… Coffee Venn Diagram


Some Basic Set Identities….• Idempotence: • Commutatively (Symmetry):

• Similarly

• Associativity:

• Distributivity:

• De Morgan Laws:

• Disjoint Sets:• Set A and B are said to be disjoint if and only if • represents empty set• In Probability we say that A and B are independent

To avoid repeating the word “set” multiple times we call a

set of sets a collection/class/family of

sets


Mutually Exclusive…

• A collection of sets is said to be pairwise disjoint or mutually exclusive if an only if

• For Example sets A, B, C are Mutually Exclusive if

•

A B

C


Partition….

• Given a set S, a collection

of subsets of S is said to be a partition of S if

• For all


Some more important Concepts

• Cardinality…• The cardinality or size of a collection or set A, denoted

by |A|, is the number of elements of a collection• Cardinality can be finite or infinite

– A finite set is a set that has a finite number of elements.– A set with infinite number of elements is called an infinite set.– Countable sets:

» Empty set and finite sets are automatically countable.» An infinite set is countable if elements of A can be

enumerated or listed in sequence.– Singleton… A singleton is a set with exactly on element.


Relationship between Set Theory and Probability Theory

Set Theory Probability Theory

Set Event

Universal Set Sample Space / State Space(Ω)

Element Outcome / State (ω)

Event Language

A occurs

A does not occur

Either A or B occurs

Both A and B occur


CLASSICAL PROBABILITY…What are the odds…


Classical Probability….

• Classical Probability is based upon the ratio of the number of outcomes favorable to the occurrence of the event of interest to the total number of possible outcomes.

• It was the first type of probability studied by mathematicians such as Fermat and Pascal.

• They are widely credited with starting the formal study of probability.


Classical Probability• Given a finite sample space “Ω”, the classical probability of

an event A is

• In this form probability is a faction such that • The Denominator represents the number of possible outcomes, and• The Numerator represents the number of outcomes in which the event of

interest occurs.

• We can only calculate probability only if following assumption are true…• Finite Ω -- The number of possible outcomes are finite• Equi-possibility: The outcomes have equal probability of occurrence.


Equi-possibility….• Equi-possibility is also called the following in literature:

• Equi-probable• Equi-possible• Equally Likely• Fair

• The bases for identifying equi-possibility were often:• Physical Symmetry (e.g. a well-balanced dice, made of homogeneous

material in a cubical shape)• A balance of information or knowledge concerning the various possible

outcomes.

• Equi-possibility is meaningful only for finite sample space, therefore, Classical Probability (as a rule) can only work for finite sample space.


Examples…

• Suppose you have 9,999 red balls and 1 black ball. What is the probability of getting a Black Ball.

• Event “A” is getting a Black Ball

• Sample Space “Ω” is a set of all (10000) Balls

• So the probability of event “A” is


Another Example…

• In drawing a card from a deck, there are 52 equally likely outcomes, 13 of which are diamonds. What is the probability of selecting a card with diamonds.

• Event “A” is selecting a diamonds card

• The sample space “Ω” is the deck of cards

• Then the probability of selecting a diamond card is


Some Basic Properties of Classical Probability…

• Here are some basic properties of Classical Probability that will be useful for us in the rest of the course

• this is derived directly from

• Suppose and then

» The probability of an event is equal to the sum of the probabilities of its component outcomes because outcomes are mutually exclusive


Another Example…• Lets play with “dice” now…

• What is the probability of getting a 2 when we roll a dice

• Event “A” is roll a 2• Sample Space “Ω” is the set of all possible values that can

come up for a single throw of a dice

• The probability of Event A occurring will then be

• One dice is too simple …. Lets make things interesting and start working with two dice…


Two Dice… • Assume that the two dice are fair and

independent • Then what is the probability of getting a 5 on a single

throw of two dice?


Two Dice…

• What is the probability of getting a 5 with a single throw of two dice.

• Event “A” is getting a 5

• Sample Space “Ω” is all possible combination of values of the two dice

• Probability of getting a 5 is then


Two Dice… • Not as simple as it seem…• Even a great mathematician like Leibniz was confused by the problem and initially

assumed that it was equally likely to throw as 11 or 12 with two dice….

Interested in working with dice a bit more… Check out these sites with dice simulators…

http://www.dicesimulator.com

And

http://www2.whidbey.net/ohmsmath/webwork/javascript/dice2rol.htm

http://www.dicesimulator.com/




It’s not just two dice only….• Similar mistake was made by another mathematician when he claimed that

• “ If a fair coin is tossed twice, the number of heads that turns up in those two tosses can be 0, 1, or 2. Since there are three outcomes, the chances of each must be 1 in 3”.

• Can you find the mistake in the above statement???

Possible Outcomes Probabilities

H – H ¼

H – T ¼

T – H ¼

T – T ¼

Probabilities


Probability and Gambling…• In the world of gambling, probabilities are often expressed by

odds.• The statement that the odds are n:1 against an event means that

» It is n time more likely that the event will not occur» Mathematically

this implies that

and

• Note that “ODDS” here have nothing to do with even and odd numbers. They represent what you will win in addition to getting you stake back should your guess proves right.

• Example: If I bet $1 on a horse with odds of 7:1• I get $7 + $1 = $8 if the horse wins the race• This means that the only way the bookmaker will break even in the long

run is if the probability of the horse winning is 1/8 (not 1/7)

• Even ODDS mean 1:1 i.e. • You get $1 + $1 if you bet $1• The probability then is ½

lecture 2

Documents

stochastic process ucp

high probability

stochastic processlecture

asim baiglong

asim baiggsn3113

asim baigcoin

asim baigrandomnessrandomness

asim baigback