![Page 1: Even More Random Number Generators Using Genetic Programming](https://reader036.vdocuments.site/reader036/viewer/2022070409/5681443d550346895db0d6b8/html5/thumbnails/1.jpg)
Even More Random Number Generators Using Genetic
Programming
Joe Barker
![Page 2: Even More Random Number Generators Using Genetic Programming](https://reader036.vdocuments.site/reader036/viewer/2022070409/5681443d550346895db0d6b8/html5/thumbnails/2.jpg)
Topics
• Genetic Programming• Random Numbers• Previous Efforts• Design & Implementation• Results• Conclusion• Future• Bibliography
![Page 3: Even More Random Number Generators Using Genetic Programming](https://reader036.vdocuments.site/reader036/viewer/2022070409/5681443d550346895db0d6b8/html5/thumbnails/3.jpg)
Genetic Programming
• Evolve programs for solutions, instead of solutions
• Difficulty of representation
• Higher level than standard EA compounds standard problems
![Page 4: Even More Random Number Generators Using Genetic Programming](https://reader036.vdocuments.site/reader036/viewer/2022070409/5681443d550346895db0d6b8/html5/thumbnails/4.jpg)
Genetic Programming
Gene Expression Programming
• Encodes information in a similar way to genes(operation) to DNA(string)
• Mutation & Crossover obvious string operations
• Care required to avoid gibberish
(2)
![Page 5: Even More Random Number Generators Using Genetic Programming](https://reader036.vdocuments.site/reader036/viewer/2022070409/5681443d550346895db0d6b8/html5/thumbnails/5.jpg)
Genetic Programming
Gene Expression Programming
• Example
(3)
AG-CC-GT-TA-CC2 + 1 * 3
![Page 6: Even More Random Number Generators Using Genetic Programming](https://reader036.vdocuments.site/reader036/viewer/2022070409/5681443d550346895db0d6b8/html5/thumbnails/6.jpg)
Genetic Programming
Expression Trees• Encodes operations in a natural tree
structure– Internal nodes are operations– Leaf-nodes are variables or constants
• Mutation & Crossover follow from the structure
• Layout of the tree avoids non-sensical results
(4)
![Page 7: Even More Random Number Generators Using Genetic Programming](https://reader036.vdocuments.site/reader036/viewer/2022070409/5681443d550346895db0d6b8/html5/thumbnails/7.jpg)
Genetic Programming
Expression Trees• Example
(5)
![Page 8: Even More Random Number Generators Using Genetic Programming](https://reader036.vdocuments.site/reader036/viewer/2022070409/5681443d550346895db0d6b8/html5/thumbnails/8.jpg)
Random Numbers
• Why?– Evolutionary Algorithms– Monte-Carlo Simulations– Software Regression Testing– Game Playing
![Page 9: Even More Random Number Generators Using Genetic Programming](https://reader036.vdocuments.site/reader036/viewer/2022070409/5681443d550346895db0d6b8/html5/thumbnails/9.jpg)
Random Numbers
• What?– “Random” is difficult to define– Even statistical definitions necessarily
describe what we would consider random– Uniform
• 1-2-3-4-5 is Uniformly distributed but not what we would consider random
– Tests exist to try and cover the important aspects of random
(2)
![Page 10: Even More Random Number Generators Using Genetic Programming](https://reader036.vdocuments.site/reader036/viewer/2022070409/5681443d550346895db0d6b8/html5/thumbnails/10.jpg)
Random Numbers
• Tests– Chi-Squared test for closeness of fit
(3)
5.0
)1,(
*
)*(
2
2
nV
pn
pnYV
i i
ii
![Page 11: Even More Random Number Generators Using Genetic Programming](https://reader036.vdocuments.site/reader036/viewer/2022070409/5681443d550346895db0d6b8/html5/thumbnails/11.jpg)
Random Numbers
• Tests– Frequency or Equidistribution test
• Break number space into a small number of blocks• Use the counts for these blocks in Chi-Squared
test for Uniform distribution
– Gap test• Break number space into to classes(Normally
upper and lower parts)• Count length of runs of class 2 between class 1• Use a Chi-Squared test with the following
distribution:
(4)
loneonel ppnp )1(**
![Page 12: Even More Random Number Generators Using Genetic Programming](https://reader036.vdocuments.site/reader036/viewer/2022070409/5681443d550346895db0d6b8/html5/thumbnails/12.jpg)
Random Numbers
• Tests– Entropy
• Arrange the numbers as a bitstring and count occurrences with certain lengths
• 101111110101010110011001001110– 10-11-11-11-01-01-01-01-10-01-10-01-00-11-10– 101-111-110-101-010-110-011-001-001-110
• Use percent occurrences in the following formula:
(5)
nE
ppE
n
i ii
2
1
02
log
1log*
![Page 13: Even More Random Number Generators Using Genetic Programming](https://reader036.vdocuments.site/reader036/viewer/2022070409/5681443d550346895db0d6b8/html5/thumbnails/13.jpg)
Random Numbers
• How?– Computers are deterministic, so we must
approximate– Several classes of pseudo-random number
generators(PRNGs)
(6)
![Page 14: Even More Random Number Generators Using Genetic Programming](https://reader036.vdocuments.site/reader036/viewer/2022070409/5681443d550346895db0d6b8/html5/thumbnails/14.jpg)
Random Numbers
• PRNGs– Linear congruential randomizers
• Some of the earliest known
(7)
cx
cb
ca
cbxax ii
0
1
0
0
0
mod*)*(
• Common choices– Park-Miller: a=7^5 b=0 c=2^31-1– URN08/RANDU: a=65539 b=0 c=2^31
![Page 15: Even More Random Number Generators Using Genetic Programming](https://reader036.vdocuments.site/reader036/viewer/2022070409/5681443d550346895db0d6b8/html5/thumbnails/15.jpg)
Random Numbers
• PRNGs– Shift register randomizers
• SR[a,b,c]
• A common choice is SR[3,28,31]
(8)
12
12
1
c
i
cii
tbtx
xaxt
![Page 16: Even More Random Number Generators Using Genetic Programming](https://reader036.vdocuments.site/reader036/viewer/2022070409/5681443d550346895db0d6b8/html5/thumbnails/16.jpg)
Random Numbers
• PRNGs– Shuffling randomizer
• Uses two other PRNGs• The first PRNG re/fills a list of numbers• The second PRNG selects number from the list
– Inversive– Mersenne Twister
(9)
![Page 17: Even More Random Number Generators Using Genetic Programming](https://reader036.vdocuments.site/reader036/viewer/2022070409/5681443d550346895db0d6b8/html5/thumbnails/17.jpg)
Previous Efforts
• This project is based largely on the work by John R. Koza– Used expression trees as individuals– The tree was executed on numbers 1..16K to
obtain a random sequence– Bit entropy (lengths 1..7)– Non-Terminals=+,-,*,/,%– Terminals=J,0,1,2,3
![Page 18: Even More Random Number Generators Using Genetic Programming](https://reader036.vdocuments.site/reader036/viewer/2022070409/5681443d550346895db0d6b8/html5/thumbnails/18.jpg)
Design & Implementation
• Individuals– Expression Tree– Non-Terminals=+,-,*,/,%
• =XOR• Each non-terminal is equally likely in a random tree
– Terminals=J,0,1,2,3• 2^i = Power of 2 (i=1-31, uniform)• Each terminal is equally likely in a random tree
– Output range is 0..2^32-1– Aged some number of steps before mature
![Page 19: Even More Random Number Generators Using Genetic Programming](https://reader036.vdocuments.site/reader036/viewer/2022070409/5681443d550346895db0d6b8/html5/thumbnails/19.jpg)
Design & Implementation
• Evaluation – Fitness– Bit entropy (lengths 5,6,7,8)– Frequency tests (512 blocks)– Gap test for “runs above the mean” (Up to 10)– The alpha value calculated from the above two tests
was adjusted by the formula:
– All three values were normalized to a maximum of 1 and summed
– These are then averaged over the life of the individual
(2)
25.0
5.01
2p
F
![Page 20: Even More Random Number Generators Using Genetic Programming](https://reader036.vdocuments.site/reader036/viewer/2022070409/5681443d550346895db0d6b8/html5/thumbnails/20.jpg)
Design & Implementation
• Selection– 2 mature parents selected uniformly– There is a small chance of both crossover and
mutation, but most likely only one
• Crossover– Subtrees selected from each parent and
swapped
• Mutation– Replaced subtree with equal or smaller
random tree
(3)
![Page 21: Even More Random Number Generators Using Genetic Programming](https://reader036.vdocuments.site/reader036/viewer/2022070409/5681443d550346895db0d6b8/html5/thumbnails/21.jpg)
Design & Implementation
• Crossover - Mutation
(4)
![Page 22: Even More Random Number Generators Using Genetic Programming](https://reader036.vdocuments.site/reader036/viewer/2022070409/5681443d550346895db0d6b8/html5/thumbnails/22.jpg)
Design & Implementation
• Competition– Replaces the bottom, by fitness, two mature
population members
• Other– No termination, runs indefinitely– HUP signal causes population to be dumped
to file
(5)
![Page 23: Even More Random Number Generators Using Genetic Programming](https://reader036.vdocuments.site/reader036/viewer/2022070409/5681443d550346895db0d6b8/html5/thumbnails/23.jpg)
Results
• Population size=100
• Mature Age=30
• Initial Maximum Tree Depth=10
• Crossover only chance=0.8
• Both chance=0.15
![Page 24: Even More Random Number Generators Using Genetic Programming](https://reader036.vdocuments.site/reader036/viewer/2022070409/5681443d550346895db0d6b8/html5/thumbnails/24.jpg)
Results
• After 10 hours on 5 machines, the best candidate was:
(2)
![Page 25: Even More Random Number Generators Using Genetic Programming](https://reader036.vdocuments.site/reader036/viewer/2022070409/5681443d550346895db0d6b8/html5/thumbnails/25.jpg)
Results
• Performance - Entropy Test
PRNG Avg. Entropy Std. Deviation
Stage 1 25.999974 5.52E-07
Stage 2 25.108252 0.0348352
R250 25.932738 8.43E-05
glibc rand() 25.932582 7.79E-05
Ideal 26.000000 0.000000
(3)
![Page 26: Even More Random Number Generators Using Genetic Programming](https://reader036.vdocuments.site/reader036/viewer/2022070409/5681443d550346895db0d6b8/html5/thumbnails/26.jpg)
Results
• Performance - Frequency Test
PRNG Chi-Sq. Statistic Chi-Sq. Percentile
Stage 1 1.43748 0.0000%
Stage 2 509.50400 47.7160%
R250 537.42400 78.8859%
glibc rand() 488.47300 23.3957%
Ideal 511.33349 50.0000%
(4)
![Page 27: Even More Random Number Generators Using Genetic Programming](https://reader036.vdocuments.site/reader036/viewer/2022070409/5681443d550346895db0d6b8/html5/thumbnails/27.jpg)
Results
• Performance - Gap Test
PRNG Chi-Sq. Statistic Chi-Sq. Percentile
Stage 1 3160280.00000 100.0000%
Stage 2 7.58965 33.1510%
R250 173.47400 100.0000%
glibc rand() 7.23900 29.7294%
Ideal 9.34182 50.0000%
(5)
![Page 28: Even More Random Number Generators Using Genetic Programming](https://reader036.vdocuments.site/reader036/viewer/2022070409/5681443d550346895db0d6b8/html5/thumbnails/28.jpg)
Results
• Performance - Speed Test (random nums/sec)
PRNG Speed
Stage 2 290444
Compiled Stage 2 3.1546E6
R250 2.3256E7
glibc rand() 6.2893E6
(6)
![Page 29: Even More Random Number Generators Using Genetic Programming](https://reader036.vdocuments.site/reader036/viewer/2022070409/5681443d550346895db0d6b8/html5/thumbnails/29.jpg)
Conclusion
• It appears that employing EAs in this manner has promise
• I hesitate to recommend using Stage 2 as a production randomizer as of now, but it does bear more investigation
![Page 30: Even More Random Number Generators Using Genetic Programming](https://reader036.vdocuments.site/reader036/viewer/2022070409/5681443d550346895db0d6b8/html5/thumbnails/30.jpg)
Future Work
• Add prime numbers to the available terminals
• Add more tests, such as periodicity, to the fitness function
• Some type of runtime compilation instead of interpreting the expression trees
![Page 31: Even More Random Number Generators Using Genetic Programming](https://reader036.vdocuments.site/reader036/viewer/2022070409/5681443d550346895db0d6b8/html5/thumbnails/31.jpg)
Bibliography• Koza, John R., Evolving a Computer Program to Generate Random Numbers Using the Genetic
Programming Paradigm, Proceedings of the Fourth International Conference on Genetic Algorithms, Morgan Kaufmann Publishers, Inc., pages 37-44, 1991. http://citeseer.nj.nec.com/john91evolving.html
• Knuth, D. E., The Art of Computer Programming, Volume 2, Second Edition, Addison-Wesley, pages 9-114, Reading, MA, 1981.
• Koza, John R., Genetically Breeding Populations of Computer Programs to Solve Problems in Artificial Intelligence, Proceedings of the Second International Conference on Tools for AI. Washington, November, 1990, IEEE Computer Society Press, Los Alamitos, CA 1990.http://citeseer.nj.nec.com/koza90genetically.html
• Kinnear, Kenneth E. Jr., Evolving a sort: Lessons in genetic programming. Proceedings of the 1993 International Conference on Neural Networks, volume 2. IEEE Press, 1993.http://citeseer.nj.nec.com/kinnear93evolving.html
• Kinnear, Kenneth E. Jr., Generality and Difficulty in Genetic Programming: Evolving a Sort, Proceedings of the Fifth International Conference on Genetic Algorithms, Morgan Kaufmann Publishers, pages 287-294, Inc., 1993.http://citeseer.nj.nec.com/kinnear93generality.html