computer applications for engineers - 4.2 - random... · 2019-11-06 · an interesting number 2...
TRANSCRIPT
Asst. Prof. Dr. Prapun [email protected]
Generating Random Numbers
1
Computer Applications for EngineersET 601
Office Hours: (BKD 3601-7)Wednesday 9:30-11:30Wednesday 16:00-17:00Thursday 14:40-16:00
An interesting number
2
Here is an interesting number:
0.814723686393179
This is the first number produced by the MATLAB random number generator with its default settings.
Start up a fresh MATLAB, set format long, type rand, and it’s the number you get. Verified in MATLAB 2013a
It may seem perverse to use a computer, that most precise and deterministic ofall machines conceived by the human mind, to produce “random” numbers. Morethan perverse, it may seem to be a conceptual impossibility. Any program, after all,will produce output that is entirely predictable, hence not truly “random.”
[Numerical Recipes, Ch 7]
Pseudorandom Number
3
Random numbers were originally either manually or mechanically generated, by using such techniques as spinning wheels, or dice rolling, or card shuffling.
The modern approach is to use a computer to successively generate pseudorandom numbers. Although they are deterministically generated, they approximate
independent uniform (0, 1) random variables. So, “random” numbers in MATLAB are not unpredictable. They are
generated by a deterministic algorithm. The algorithm is designed to be sufficiently complicated so that its output appears to
be random to someone who does not know the algorithm, and can pass various statistical tests of randomness.
Our assumption Assume that we have a good pseudorandom number generators. Example: the rand command in MATLAB.
(Truly) random digits from physical mechanisms
6
Employ physical mechanisms rather than formal algorithms to provide random digits.
Hotbits project: www.fourmilab.ch/hotbits Use radioactive decay. Run by Autodesk founder John Walker If you connect to this site, you can listen to the Geiger counter ticks.
Random.org site: www.random.org Sample atmospheric noise by using a radio tuned between stations. The site also provides a general discussion of random numbers.
Lavarnd site: www.lavarnd.org Take digital photographs of a webcam with its lens cap on. Rely on thermal “noise”
Lavarand site: lavarand.sgi.com Take digital photographs of the patterns
made by colored lava lamps.
rng
7
The sequence of numbers produced by rand is determined by the internal settings of the uniform random number generator that underlies rand, randi,and randn.
You can control that shared random number generator using rng. This can be useful for controlling the repeatability of your
results.
http://www.mathworks.com/support/2013b/matlab/8.2/demos/controlling-random-number-generation.html
( random number generator)
rng default/shuffle
12
Every time you start MATLAB, the generator resets itself to the same state.
You can reset the generator to the startup state at any time in a MATLAB session (without closing and restarting MATLAB) by rng('default') rng default
To avoid repeating the same results when MATLAB restarts: Execute the command rng('shuffle') rng shuffle It reseeds the generator using a different seed
based on the current time.
rng: Save and Restore the Generator Settings
13
Save the current generator settings in s
Restore the saved generator settings
The first call to randichanged the state of the generator, so the second result is different.
References
14
Chapter 3 (Random Numbers) in Sheldon M. Ross. “Simulation.” Academic Press, 2012, 5th Edition
Chapter 9 (p. 245-252) in Peter Olofsson, “Probabilities The Little Numbers That Rule Our Lives”, Wiley, 2006
Chapter 9 (Random Numbers) in Cleve Moler. “Numerical Computing with MATLAB.” SIAM, 2004
Chapter 7 (Random Numbers) (Sections 7.0-7.1.1) in W. H. Press, S. A. Teukolsky, W. T. Vetterling, B. P. Flannery. “Numerical Recipes: The Art of Scientific Computing.” Cambridge, 2007, 3rd Edition.
References
15
Park, S.K., and K.W. Miller. “Random Number Generators: Good Ones Are Hard to Find.” Communications of the ACM, 31(10):1192–1201. 1998.
C. Moler, Random thoughts, “10^435 years is a very long time”, MATLAB News and Notes, Fall, 1995
Asst. Prof. Dr. Prapun [email protected]
Generating Random Variables
16
Computer Applications for EngineersET 601
Office Hours: (BKD 3601-7)Wednesday 9:30-11:30Wednesday 16:00-17:00Thursday 14:40-16:00
Generation of arbitrary RVs
17
There are many built-in families of RV in MATLAB that can be generated by MATLAB’s random command.
Popular families are provided. Binomial, Exponential, Geometric, Normal, Poisson, Uniform Beta, Birnbaum-Saunders, Burr Type XII, Chi-Square, Extreme
Value, F, Gamma, Generalized Extreme Value, Generalized Pareto, Hypergeometric, Inverse Gaussian, Logistic, Loglogistic, Lognormal, Nakagami, Negative Binomial, Noncentral F, Noncentral t, Noncentral Chi-Square, Rayleigh, Rician, Student's t, t Location-Scale, Weibull
doc random How to generate RVs whose families are not built-in?
Bernoulli Trials
18
We want to generalize the old technique.
rand(1,120) < 0.5 randi([0 1],1,120)
1/ 2, 0,1/ 2, 1,0, otherwise.
X
xp x x
To generate a RV X whose pmf is given by
First use rand create a uniform RV U.
Then, set 1, 1/ 2,0, 1/ 2.
UX
U
To generate a RV X whose pmf is given by
First use rand create a uniform RV U.
Then, set
, 1,
1 , 0,0, otherwise.
X
p xp x p x
1, ,0, .
U pX
U p
clear all; close all; S_X = [3 4]; p_X = [1/3 2/3]; n = 1e6;
U = rand(1,n); V = [U < 2/3]; X = S_X(V+1); rf = hist(X,S_X)/n; stem(S_X,rf,'rx') hold on stem(S_X,p_X,'bo') xlim([min(S_X)-1,max(S_X)+1]) legend('Rel. freq. from sim.'... ,'pmf p_X(x)') xlabel('x') grid on
Example
19
To generate a RV X whose pmf is given by
First use rand to create a uniform RV U.
Then, set
Finally, map to .
Of course, we can combine all the steps into just one line:
1/ 3, 3,2 / 3, 4,0, otherwise,
X
xp x x
1, 2 / 3,0, 1/ 3.
UV
U
2 2.5 3 3.5 4 4.5 50
0.1
0.2
0.3
0.4
0.5
0.6
0.7
x
Rel. freq. from sim.pmf pX(x)
X = S_X([rand(1,n) < 2/3]+1);
Example
20
To generate a RV X whose pmf is given by
First use rand to create a uniform RV U.
Then, set
Finally, map to .
1/ 6, 3,1/ 3, 4,1/ 2, 8,0, otherwise.
X
xx
p xx
1, 0 1/ 6,2, 1/ 6 1/ 2,3, 1/ 6 1,
UV U
U
clear all; close all; S_X = [3 4 8]; p_X = [1/6 1/3 1/2]; n = 1e6; F_X = cumsum(p_X); U = rand(1,n); [dum,V] = histc(U,[0 F_X/F_X(end)]); X = S_X(V); rf = hist(X,S_X)/n; stem(S_X,rf,'rx') hold on stem(S_X,p_X,'bo') xlim([min(S_X)-1,max(S_X)+1]) legend('Rel. freq. from sim.'... ,'pmf p_X(x)') xlabel('x') grid on
2 3 4 5 6 7 8 90
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
x
Rel. freq. from sim.pmf pX(x)
histc vs hist
21
N = hist(U,centers) Bins’ centers are defined by the vector centers.
The first bin includes data between -inf and the first center and the last bin includes data between the last bin and inf.
N(k) count the number of entries of vector U whose values falls inside the kth bin.
N = histc(U,edges) Bins’ edges are defined by the vector edges. N(k) count the value U(i) if edges(k) ≤ U(i) < edges(k+1).
The last (additional) bin will count any values of U that match edges(end).
Values outside the values in edges are not counted. May use -inf and inf in edges.
[N,BIN_IND] = histc(U,EDGES) also returns vector BIN_IND indicating the bin index that each entry in U sorts into.
Example: histc
22
>> p_X = [1/6 1/3 1/2];>> F_X = cumsum(p_X)
F_X =
0.1667 0.5000 1.0000
>> U = rand(1,5)
U =
0.2426 0.9179 0.9409 0.1026 0.8897
>> [dum,V] = histc(U,[0 F_X])
dum =
1 1 3 0
V =
2 3 3 1 3
Loops
23
Loops are MATLAB constructs that permit us to execute a sequence of statements more than once.
There are two basic forms of loop constructs: while loops and for loops.
The major difference between these two types of loops is in how the repetition is controlled. The code in a while loop is repeated an indefinite number of
times until some user-specified condition is satisfied. The code in a for loop is repeated a specified number of times,
and the number of repetitions is known before the loops starts.
while loop
24
General form:
The statements in the body are repeatedly executed as long as the expression expr remains true.
while exprbody
end
clear all; close all; S_X = [3 4 8]; p_X = [1/6 1/3 1/2]; n = 1e6; U = rand(1,n); V = zeros(size(U)); %Preallocation for k = 1:n F = p_X(1); m = 1; while (U(k) > F) m = m+1; F = F+p_X(m); end V(k) = m; end X = S_X(V); hist(X) rf = hist(X,S_X)/n; stem(S_X,rf,'rx') hold on stem(S_X,p_X,'bo') xlim([min(S_X)-1,max(S_X)+1]) legend('Rel. freq. from sim.'... ,'pmf p_X(x)') xlabel('x') grid on
Example: Geometric1 RV
25
clear all; close all; p = 1/3; n = 1e6; U = rand(1,n); V = zeros(size(U)); %Preallocation for k = 1:n F = p; pk = p; m = 1; while (U(k) > F) m = m+1; pk = pk*(1-p); F = F + pk; end V(k) = m; end X = V; X_Range = 1:10; p_X = p*(1-p).^(X_Range-1); rf = hist(X,X_Range)/n; stem(X_Range,rf,'rx') hold on stem(X_Range,p_X,'bo') xlim([min(X_Range)-1,max(X_Range)+1]) legend('Rel. freq. from sim.'... ,'pmf p_X(x)') xlabel('x') grid on
1
1
2
1 , 1,2,3,
1
11
1 1
kX
X
kX
kX
p k p p k
p p
p k p pp
p k p p
0 1 2 3 4 5 6 7 8 9 10 110
0.05
0.1
0.15
0.2
0.25
0.3
0.35
x
Rel. freq. from sim.pmf pX(x)
Alternatively, can put the formula for pX(k) here.
User-defined functions in MATLAB
26
Functions are files that can accept input arguments and return output arguments.
We have already seen (and used) MATLAB script files
A script file consists of a sequence of MATLAB statements that you could also type directly into the Command Window.
MATLAB built-in functions The actual contents of the built-in functions are not generally known or seen by the user.
Variables in a function file are local by default. Functions operate on variables within their own workspace, separate from the
workspace you access at the MATLAB command prompt. The names of the file and of the function should be the same (to avoid
confusion). The first line of the function declares the function name, input arguments, and
output arguments; without this line the file would be a script file.
Examples
27
function y = sqr(x) y = x.^2; end sqr.m
This is not necessary in many cases in current versions of MATLAB, but it is considered good style anyway.
function z = multiply(x,y) z = x.*y end
function [m s] = scorestat(x) m = mean(x); s = std(x); end
Anonymous Functions
28
Defined within a single MATLAB statement.
Consist of a single MATLAB expression and any number of input and output arguments.
Can be defined right at the MATLAB command line, or within a function or script.
Give you a quick means of creating simple functions without having to create a file for them each time.
Syntax: f = @(arglist) expression
datasample
29
clear all; close all; S_X = [3 4 8]; p_X = [1/6 1/3 1/2]; n = 1e6; X = datasample(S_X,n,'Weights',p_X); hist(X) rf = hist(X,S_X)/n; stem(S_X,rf,'rx') hold on stem(S_X,p_X,'bo') xlim([min(S_X)-1,max(S_X)+1]) legend('Rel. freq. from sim.'... ,'pmf p_X(x)') xlabel('x') grid on
Inverse Transform Method
30
Let U be a uniform random variable on the interval (0,1).
To generate a continuous random variable X with cumulative distribution function FX, set .
Whether it is feasible to implement depends on whether the inverse function of the integral of pdf is itself feasible to compute, either analytically or numerically.
Inverse function
31
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
2
3
x
g(x) = x3
g-1(x)-3
-2
-1
0
1
2
3
-3
-2
-1
0
1
2
3
x
g(x) = x 3g -1(x)
close all; clear all; x =linspace(-1.5,1.5,100); y = x.^3; plot(x,y,'k','LineWidth',1.5); grid on hold on; xlabel('x') plot(y,x,'LineWidth',1.5) legend('g(x) = x^3', 'g^{-1}(x)') mx = max([max(y) max(x)]); mn = min([min(y) min(x)]); xlim([mn,mx]); ylim([mn,mx]); x = linspace(mn,mx,100); plot(x,x,'--r')
Example: Expo. RV
32
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 50
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
x
F X(x)
Empirical CDF
Theoretical FXEmpirical
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 50
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
x
f X(x)
Theoretical fXEmpirical
close all; clear all; n = 1e6; lambda = 2; U = rand(1,n); X = -log(U)/lambda; minx = 0; maxx = 5; x = linspace(minx,maxx,50); % Verification via the cdf plots F_X = 1-exp(-lambda*x); plot(x,F_X,'LineWidth',1.5) hold on h = cdfplot(X); set(h,'LineStyle','--',... 'Color','r','LineWidth',1.5) xlim([minx,maxx]) ylabel('F_X(x)') legend('Theoretical F_X', 'Empirical') % Verification via the pdf figure f_X = lambda*exp(-lambda*x); plot(x,f_X,'LineWidth',1.5) hold on dx = x(2)-x(1); pdfest = histc(X,x)/(n*dx); plot(x+dx/2,pdfest,'r*') legend('Theoretical f_X', 'Empirical') xlim([minx,maxx]) grid on xlabel('x') ylabel('f_X(x)')
Rejection Method
33
Acceptance-Rejection Technique
Powerful and general
Assume known and computable distribution (pdf or pmf).
Does not require that the cdf and its inverse be readily computable Better than the transformation method discussed earlier.
Assume availability of RV whose support is the same as the
Motivational Example 1
34
It is easy to generate 2-D points uniformly over a finite area.
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
n = 1e3; U = rand(1,n); Y = rand(1,n); plot(Y,U,'.'); figure c = 2; U = rand(1,n); V = c*U; Y = rand(1,n); plot(Y,V,'.');
Motivational Example 1
35
Suppose we want to generate realizations of a continuous random variable X whose pdf is given by
First, we generate 2-D points over the area that cover the pdf above.
Then, we keep only the points that are below .
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
2 , 0 1,0, otherwise.X
x xf x
figure % Define the pdf a = 2; f_X = @(x) a*x.^(a-1); x = linspace(0,1,100); plot(x,f_X(x),'r--') hold on % Keep only the points that % are below the pdf I = [V < f_X(Y)]; plot(Y(I),V(I),'.'); % Use only the x-coordinates X = Y(I);
Motivational Example 1
36
Conclusion: If we had some way of choosing a random point in two dimensions, with uniform probability in the area under the pdf curve, then the x coordinate of that random point would have the desired pdf.
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
10
20
30
40
50
60
70
80
90
100
x
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.5
1
1.5
2
2.5
3
x
f X(x)
Theoretical fXEmpirical
Start with n = 1,000.
Motivational Example 1
37
Conclusion: If we had some way of choosing a random point in two dimensions, with uniform probability in the area under the pdf curve, then the x coordinate of that random point would have the desired pdf.
Start with n = 100,000.
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
1000
2000
3000
4000
5000
6000
7000
8000
9000
10000
x
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
x
f X(x)
Theoretical fXEmpirical
Motivational Example 2
38
Note: It is not possible to directly distribute 2-D points over infinite area.
So, if we want to generate X whose support is infinite, we will need to start with area that is finite.
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 50
0.5
1
1.5
2
2.5
3
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 50
1
2
3
4
5
6
Motivational Example 3
39
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 50
0.5
1
1.5
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 50
0.5
1
1.5
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 50
0.5
1
1.5
0 0.5 1 1.5 2 2.5 3 3.50
50
100
150
x0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
x
f X(x)
Theoretical fXEmpirical
References
40
Chapter 4 (Generating Discrete Random Variables)Chapter 5 (Generating Continuous Random Variables) In Sheldon M. Ross. “Simulation.”
Academic Press, 2012, 5th Edition
Chapter 7 (Random Numbers) (Sections 7.3) in W. H. Press, S. A. Teukolsky, W. T. Vetterling, B. P. Flannery. “Numerical Recipes: The Art of Scientific Computing.” Cambridge, 2007, 3rd Edition.